A full-automatic unmanned aerial vehicle fire inspection and early warning platform

Abstract

The present application relates to the technical field of fire inspection and identification, solves the technical problem of insufficient autonomous operation ability, low automation and intelligence level of the existing unmanned aerial vehicle fire inspection technology in complex environment, and particularly relates to a full-automatic unmanned aerial vehicle fire inspection and early warning platform, which comprises a flight route planning module for generating autonomous flight routes for unmanned aerial vehicles to execute; and an image transmission and identification module for receiving and processing video streams returned by the unmanned aerial vehicle in real time during the execution of the autonomous flight routes by the unmanned aerial vehicle to identify fire information in the video streams. The present application realizes autonomous flight inspection and real-time fire identification of the unmanned aerial vehicle, can generate autonomous flight routes for the unmanned aerial vehicle to execute, so that the unmanned aerial vehicle can perform preset path inspection in high-risk or complex environment without real-time manual command. The present application can make the unmanned aerial vehicle free from excessive dependence on ground operators and significantly improve the automation and intelligence level of the inspection.

Description

Technical Field

[0001] This invention relates to the field of fire inspection and identification technology, and in particular to a fully automated unmanned aerial vehicle (UAV) fire inspection and early warning platform. Background Technology

[0002] With the rapid development of intelligent technologies, drones are increasingly being used in high-risk, complex environments such as firefighting, inspection, and emergency rescue. However, traditional inspection methods, especially in complex environments prone to fire such as buildings and chemical plants, often face numerous challenges. Although significant progress has been made in the field of drone research, most existing research focuses on applications in large-scale outdoor fire scenarios, particularly the development and application of drones for large-scale natural disasters such as forest fires.

[0003] For this type of research, the technological focus typically includes the remote operation capability of drones, long-duration flight endurance, and fire monitoring and disaster assessment. However, research on small inspection drones in built environments remains relatively weak. In these enclosed or semi-enclosed environments, the execution conditions for fire monitoring and inspection tasks are complex and highly dynamic, requiring drones to possess greater autonomy and flexibility, but related technological exploration has not yet received sufficient attention and in-depth research.

[0004] Furthermore, despite the increasing application of drones in firefighting and inspection, existing drone operation modes largely rely on manual operation, lacking sufficient automation and intelligence. Most firefighting drones depend on ground operators for real-time command and flight path adjustments, patrolling and monitoring complex fire scenarios. This process is cumbersome and demands high real-time responsiveness from operators. This not only limits the application potential of drones in high-risk environments but also significantly reduces the efficiency of drone operations and the timeliness of emergency response. Therefore, despite existing drone application examples and technological implementations, improving the autonomous operation capabilities of drones in complex environments, especially enhancing their automation level, remains a key issue in the current development of firefighting drone technology. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a fully automated unmanned aerial vehicle (UAV) fire inspection and early warning platform, which solves the technical problems of insufficient autonomous operation capability and low level of automation and intelligence in existing UAV fire inspection technologies in complex environments.

[0006] To address the aforementioned technical problems, the present invention provides the following technical solution: a fully automated unmanned aerial vehicle (UAV) fire inspection and early warning platform, deployed on a terminal device, the platform comprising: The login module allows users to log in to the platform using an account and password or mobile phone verification. The route planning module is used to generate autonomous flight routes for the drone to execute; and An image transmission and recognition module, linked to the flight path planning module, is used to receive and process the video stream transmitted back by the UAV in real time during the autonomous flight path, in order to identify fire information in the video stream; and The fire source image capturing module, which is linked with the image transmission and recognition module, is used to control the drone to automatically take pictures when the recognition probability of the fire information identified by the image transmission and recognition module exceeds a preset threshold.

[0007] Furthermore, the image transmission and recognition module includes a pre-trained real-time flame detection model for identifying fire information in the video stream. The construction of the real-time flame detection model includes: A multi-scale feature capture module is introduced into the backbone network of the YOLOv8 model to replace the C2f module in the original model. This module is used to guide the real-time flame detection model to capture fire features at different scales and improve the recognition accuracy. An efficient multi-scale attention module is set at the end of the backbone network of the YOLOv8 model. The efficient multi-scale attention module is used to guide the real-time flame detection model to focus on key flame features from a complex background. The YOLOv8 model employs depthwise separable convolutions and spatial channel decoupled downsampling in its backbone and neck networks, respectively replacing the traditional convolutional and downsampling layers in the original model.

[0008] Furthermore, the multi-scale feature capture module includes a parallel convolutional group, which includes multiple deep convolutional layers with different kernel sizes; The multi-scale feature capture module includes a residual connection structure, which is used to add the output after processing by parallel convolution groups, batch normalization, dimensionality increase convolution, activation function and dimensionality decrease convolution to the original input of the multi-scale feature capture module to generate the final feature map.

[0009] Furthermore, the processing procedure of the multi-scale feature capture module includes: Input features First, it passes through a convolutional layer containing multiple convolutional layers of different kernel sizes and depths. Parallel convolutional group processing, convolutional layers at various depths Output The first feature is obtained by directly adding them together to obtain the output of the parallel convolution group. , expressed as: ; in, Indicates the first The output of a deep convolutional layer; For the first feature BatchNorm processing is performed, followed by a 1×1 convolutional layer. The channel dimension is expanded to four times the original size, and the SiLU activation function is applied. The second feature is obtained by performing a nonlinear transformation. The expression is: ; in, The SiLU activation function; Indicates the first feature Perform BatchNorm processing; Then use another 1×1 convolutional layer The fusion features are obtained by restoring the number of channels to their original dimensions. ; Finally, the original input features of the multi-scale feature capture module are connected via residual connections. With fusion features By performing element-wise addition, the final feature map is obtained, expressed as: ; in, This is the feature map output by the multi-scale feature capture module.

[0010] Furthermore, the efficient multi-scale attention module includes a 1×1 branch and a 3×3 branch; The 1×1 branch is used to encode the channel along the spatial height and width directions; The 3×3 branch is used to capture local cross-channel interaction information; The outputs of the two branches are aggregated through a bidirectional matrix dot product operation to generate a spatial attention map.

[0011] Furthermore, the processing procedure of the efficient multi-scale attention module includes: For a given input feature map First, input feature map Divided into channels Group features ,in , For the number of channels, These are the input feature maps. Height and width; Features after grouping Simultaneously, 1×1 and 3×3 branch processing are performed; the 1×1 branch processes the grouped features through two one-dimensional global average pooling operations along the spatial height and width directions, respectively. The encoding is performed, and the encoded results are concatenated and processed through a shared convolutional layer before being decomposed into two vectors; the 3×3 branch captures local cross-channel interaction information through 3×3 convolution; The outputs of the two branches are used to generate a spatial attention map through bidirectional matrix dot product operation. The two are added together and then activated by Sigmoid to obtain the final spatial attention map. Finally, the spatial attention map is multiplied element-wise with the original feature map to obtain the final output.

[0012] Furthermore, the real-time flame detection model uses WIoUv3 as the bounding box regression loss function to replace the CIoU loss function of the YOLOv8 model; The bounding box loss function is used to quantify the difference between the predicted bounding box and the true labeled bounding box.

[0013] Furthermore, the expression for the bounding box loss function is: ; ; ; in, The bounding box loss function; It is used to characterize the degree of anomaly in the predicted bounding box; It is gradient gain; and It is used as a hyperparameter to control the output; and These are the centers of the predicted bounding box and the ground truth bounding box, respectively. and It is the height and width of the smallest closed box that contains both the ground truth bounding box and the predicted box; It is a standard based on overlap. The loss function.

[0014] Furthermore, the route planning module is also used to set the flight parameters of the autonomous flight route, including flight altitude, flight speed, longitude, latitude, and yaw angle.

[0015] By employing the above technical solution, the present invention provides a fully automated unmanned aerial vehicle (UAV) fire inspection and early warning platform, which has at least the following beneficial effects: This invention can easily and conveniently plan flight routes according to different working environments of users, and has multiple adjustable parameters; This invention integrates computer vision technology, enabling drones equipped with ordinary aerial cameras to perform fire inspections without requiring advanced equipment, thus saving costs and lowering the barrier to entry. The entire process, from automatic flight to intelligent recognition, is completed within a single mobile app, independent of specific onboard hardware, making it more flexible and cost-effective. This invention integrates the YOLO model into an Android app, enabling real-time recognition on mobile devices. This differs from many cloud-based or ground-based processor-based solutions, offering faster response and less dependence on network connectivity. Many inspection platforms focus on automatically acquiring images, but the identification and analysis still rely on manual interpretation or post-processing. This invention achieves intelligent identification by integrating the YOLO model, reducing the need for manual intervention. This integrated technology of "flying, shooting, and identifying simultaneously" improves response speed and buys valuable time for early warning. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a network structure diagram of the real-time flame detection model in this invention; Figure 2 This is a network structure diagram of the multi-scale feature capture module in this invention; Figure 3 This is a network structure diagram of DSC2f in this invention. Detailed Implementation

[0017] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. This will allow for a full understanding of how the present application uses technical means to solve technical problems and achieve technical effects, and to facilitate its implementation.

[0018] Traditional unmanned aerial vehicle (UAV) fire detection methods have numerous limitations when facing complex fire-prone environments such as buildings and chemical plants. These environments are typically high-risk and complex, making traditional methods ineffective. For example, in the event of a fire in a chemical plant, traditional UAVs require real-time command and flight path adjustments from ground operators. This is not only cumbersome but also demands high real-time reaction capabilities from the operators, limiting the potential of UAVs in high-risk environments and significantly reducing operational efficiency and the timeliness of emergency response. Without addressing these issues, the efficiency and accuracy of fire detection will be greatly reduced, potentially leading to the spread of the fire and causing even greater losses.

[0019] Based on this, this embodiment proposes a fire detection and early warning platform for DJI drones developed using DJI MSDK. This platform can significantly improve the autonomous operation capability and automation level of drones in complex environments. The platform includes a flight path planning module and an image transmission and recognition module. The flight path planning module generates autonomous flight paths for the drone to execute, while the image transmission and recognition module works in conjunction with the flight path planning module. During the drone's autonomous flight, it receives and processes the video stream transmitted back by the drone in real time to identify fire information within the video stream. In this way, the platform achieves a fully automated process for drones from flight path planning to fire identification, requiring no manual intervention, thereby improving the efficiency of fire detection and the timeliness of emergency response.

[0020] More specifically, the fire detection and early warning platform for DJI drones, developed based on DJI MSDK and provided in this embodiment, consists of a login module, a flight path planning module, an image transmission and recognition module, and a fire source image capture module. Among them: The login module is used by users to log in to the system using their DJI account and password or mobile phone verification. The flight path planning module allows users to define flight routes on a map, set various parameters including altitude and speed, and execute actions (such as starting / stopping recording). Within the map path planning section, users can click on any point on the map—a waypoint—to create a route arranged in waypoint order. In addition, the path planning module provides parameters such as longitude, latitude, altitude, and yaw angle to help users plan more precise waypoint routes, adapting to diverse operational environments and improving mission flexibility and accuracy.

[0021] The image transmission and recognition module is used by users to monitor the real-time transmission of images from drones on the platform, and automatically identify, count, and analyze the probability of fires. The fire recognition part is implemented by pre-training a YOLO model and integrating the trained model into the software.

[0022] The MSFC-YOLO used in this module is a real-time flame detection model. The work includes the following steps: constructing a private flame dataset for urban fires; obtaining publicly available urban flame images from the internet through web crawling; annotating the images with anchor boxes using a labeling tool; and dividing the dataset into training, validation, and test sets in an 8:1:1 ratio; increasing the sample size of the training set through data augmentation; optimizing the original YOLOv8 model to obtain the optimized MSFC-YOLO model; training the MSFC-YOLO model using the augmented training set to obtain the final flame detection weight file; and applying the trained weight file to detect flames on the test set and evaluating the model performance.

[0023] The fire source image capture module is used to automatically photograph fire sources with a probability exceeding 90%. Specifically, the image transmission and recognition module processes the video stream transmitted from the drone in real time. When fire information is identified, it calculates the recognition probability. When the recognition probability exceeds a preset threshold, the image transmission and recognition module sends a trigger signal to the fire source image capture module. Upon receiving the trigger signal, the fire source image capture module immediately controls the drone to take pictures. This ensures that image information of the fire scene is recorded promptly at the first moment of a fire, providing a basis for subsequent fire analysis and handling.

[0024] The image transmission and recognition module includes a pre-trained real-time flame detection model (MSFC-YOLO) for recognizing fire information in video streams. The network structure of MSFC-YOLO is as follows: Figure 1 As shown. More specifically, the construction of the real-time flame detection model includes: In fire detection scenarios, the scale of flames and smoke can vary drastically due to factors such as extreme distance, degree of diffusion, and occlusion. Simultaneously, their shape, color, and texture exhibit significant intra-class differences. Traditional convolutional neural networks, with their fixed receptive fields, have inherent limitations when dealing with such complex, multi-scale, and multi-morphological targets, often leading to missed detections of small targets or incomplete feature extraction of large targets.

[0025] like Figure 2 The diagram shows the network structure of the multi-scale feature capture module. In this embodiment, a multi-scale feature capture module is introduced into the backbone network of the YOLOv8 model to replace the C2f module in the original model. This module guides the real-time flame detection model to capture fire features at different scales, improving recognition accuracy. This design enables the model to simultaneously capture fine local features and macroscopic global features, and effectively integrates this information through residual connections, thereby enhancing the model's ability to identify fire targets. Therefore, the multi-scale feature capture module proposed in this embodiment can effectively address this challenge. Its specific process includes the following steps: Input features First, it is processed by a parallel convolutional group, which contains multiple depthwise convolutional layers with different kernel sizes. ,in These represent depthwise convolution operations with kernel sizes of 3×3, 5×5, and 7×7, respectively. The output of a single depthwise convolutional layer is represented as... ,in Convolutional layers at various depths Output The first feature is obtained by directly adding them together to obtain the output of the parallel convolution group. , expressed as: ; in, Indicates the first The output of a deep convolutional layer; For the first feature BatchNorm processing is performed, followed by a 1×1 convolutional layer. The channel dimension is expanded to four times the original size, and the SiLU activation function is applied. The second feature is obtained by performing a nonlinear transformation. The second feature The expression is: ; in, The SiLU activation function; Indicates the first feature Perform BatchNorm processing; Then use another 1×1 convolutional layer The fusion features are obtained by restoring the number of channels to their original dimensions. ; Finally, the original input features of the multi-scale feature capture module are connected via residual connections. With fusion features By performing element-wise addition, the final feature map is obtained, expressed as: ; in, This is the feature map output by the multi-scale feature capture module.

[0026] In this embodiment, the backbone network is the foundation of the YOLOv8 model, primarily responsible for extracting features at different levels from the input image. The multi-scale feature capture module is a specially designed feature extraction unit whose core function is to simultaneously capture flame features at different scales in the image, thereby enhancing the model's ability to recognize flame targets of varying sizes and shapes. This module replaces the C2f module in the original YOLOv8 model. The C2f module is a component in YOLOv8 used for feature fusion and information transfer. By replacing it with the multi-scale feature capture module, the aim is to improve the model's feature extraction efficiency and robustness in fire detection tasks.

[0027] This embodiment introduces a multi-scale feature capture module into the backbone network of the YOLOv8 model, replacing the original C2f module. This enables the model to more effectively extract and integrate flame features of different scales from the video stream transmitted by the UAV. Specifically, the multi-scale feature capture module can simultaneously process large, medium, and small-sized flame targets in the image, avoiding the missed or false detection problems that may be caused by single-scale feature extraction. For example, this module can provide more comprehensive feature representations for small fire spots or obscured flames at a distance, as well as large-area fires at close range. Therefore, when the UAV executes an autonomous flight path and transmits a video stream, the optimized YOLOv8 model can more accurately and robustly identify fire information in the video stream thanks to its enhanced multi-scale feature capture capabilities. This effectively solves the problems of insufficient recognition accuracy and high false alarm rate of traditional models in complex backgrounds and variable fire scenarios.

[0028] Through the aforementioned technical solution, this application significantly improves the accuracy and robustness of the fully automated UAV fire patrol and early warning platform in identifying fire information. Especially when facing flames of varying sizes, shapes, and against complex backgrounds, the optimized YOLOv8 model can more effectively capture key features, thereby reducing the probability of missed and false alarms. This optimization enables the platform to provide more reliable early warning and location information in actual fire patrol tasks, thus improving the timeliness and effectiveness of fire response and providing stronger technical support for forest fire prevention, urban firefighting, and other fields.

[0029] However, in complex backgrounds, key features of flames are easily obscured by noise, making it difficult for models to accurately identify fires. To address this, this application proposes setting an efficient multi-scale attention module at the end of the backbone network of the YOLOv8 model to guide the model to focus on key flame features from complex backgrounds.

[0030] The efficient multi-scale attention module is placed at the end of the backbone network of the YOLOv8 model, effectively integrating deep features and highlighting key information related to flames. Specifically, this efficient multi-scale attention module includes a 1×1 branch and a 3×3 branch; the 1×1 branch encodes channels along the spatial height and width directions, thereby reducing computational complexity and extracting global contextual information; the 3×3 branch captures local cross-channel interaction information, thereby enhancing the model's ability to perceive local details; the outputs of the two branches are aggregated through bidirectional matrix dot product operations to generate a spatial attention map, which can accurately locate flame-related regions in the image.

[0031] Compared to the original YOLOv8 model, this embodiment can more accurately identify key flame features, thereby reducing false positive and false negative rates. Furthermore, the design of the efficient multi-scale attention module fully considers computational efficiency, ensuring the model's real-time performance in practical applications. The specific process includes the following steps: For a given input feature map First, input feature map Divided into channels Group features ,in , For the number of channels, These are the input feature maps. The height and width. Specifically, this can be represented as: , .

[0032] Features after grouping Simultaneously, 1×1 and 3×3 branch processing are performed; the 1×1 branch processes the grouped features through two one-dimensional global average pooling operations along the spatial height and width directions, respectively. The channels are encoded, and the encoded results are concatenated and processed by a shared convolutional layer before being decomposed into two vectors; the 3×3 branch captures local cross-channel interaction information through 3×3 convolution; The outputs of the two branches are used to generate a spatial attention map through bidirectional matrix dot product operation. The two are added together and then activated by Sigmoid to obtain the final spatial attention map. Finally, the spatial attention map is multiplied element-wise with the original feature map to obtain the final output.

[0033] The 1×1 branch encodes the grouped features along the spatial height and width directions. It obtains directional encoding vectors through one-dimensional global average pooling along these directions, and then generates cross-channel feature representations after shared 1×1 convolution. Specifically, the 1×1 convolution can be viewed as a weighted summation of all channels at each pixel location, forming a new channel feature representation that reflects the importance of the original features in the channel dimension. The 3×3 branch captures local cross-channel interaction information. Using 3×3 convolution, it captures the relationships between features within local regions, leading to a better understanding of the image content. The outputs of the two branches are aggregated through matrix dot product operations to generate a spatial attention map. This matrix dot product allows for bidirectional fusion of the features extracted from the two branches, resulting in a comprehensive attention representation that includes information in both the spatial and channel dimensions. The final spatial attention map reflects the importance of different regions in the image, guiding the model to focus on important areas.

[0034] This embodiment, by setting up 1×1 and 3×3 branches in parallel, can simultaneously extract feature information in both spatial and channel dimensions, and effectively fuse them through matrix dot product operations, thereby generating a more accurate spatial attention map. This spatial attention map can guide the YOLOv8 model to more accurately focus on the key features of the flame, suppress interference from complex backgrounds, and thus improve the accuracy and robustness of fire identification.

[0035] Therefore, the efficient multi-scale attention module can more effectively guide the model to focus on the key features of the flame, thereby improving the accuracy and efficiency of fire identification. It can better balance computational complexity and performance, thus reducing computational resource consumption while ensuring recognition accuracy, making it more suitable for practical application scenarios.

[0036] In existing technologies, the YOLOv8 model is applied to a fully automated UAV fire inspection and early warning platform. While this model performs well in target detection tasks, the traditional convolutional and downsampling layers place a heavy computational burden on the large amounts of video data generated during UAV inspections, affecting the model's real-time performance and efficiency. To address this, this embodiment employs depthwise separable convolutions and spatially channel-decoupled downsampling in the backbone and neck networks of the YOLOv8 model, respectively replacing the traditional convolutional and downsampling layers in the original model. This reduces the number of model parameters and computational complexity while maintaining or improving feature extraction capabilities, all while ensuring detection accuracy.

[0037] Specifically, in the backbone network of the YOLOv8 model, all traditional convolutional layers are replaced with depthwise separable convolutional layers, and in the neck network, all traditional downsampling layers are replaced with spatially channel-decoupled downsampling layers. In implementation, the APIs for depthwise separable convolution and spatially channel-decoupled downsampling provided by existing deep learning frameworks (such as TensorFlow or PyTorch) can be used, and their parameters can be adjusted according to actual needs. For example, the kernel size of depthwise convolution, the number of channels in pointwise convolution, and the downsampling rate of spatially channel-decoupled downsampling can be adjusted to achieve optimal performance. The specific process includes the following steps: Replace the fourth and fifth traditional convolutional layers in the backbone network with depthwise separable convolutions.

[0038] Spatial channel decoupled downsampling is used to replace all traditional convolutional layers in the neck network of the original model.

[0039] By replacing the traditional convolutional layers of the neck C2f module with depthwise separable convolutions, a lighter DSC2f is obtained, such as... Figure 3 As shown.

[0040] This embodiment can significantly reduce the computational complexity and number of parameters of the YOLOv8 model while ensuring the detection accuracy of the model. It improves the operating efficiency and real-time performance of the model on the fully automated UAV fire inspection and early warning platform, enabling the UAV to process video streams faster and detect fire information in a timely manner, thereby improving the efficiency and reliability of fire inspection.

[0041] However, traditional bounding box regression loss functions may suffer from low localization accuracy when processing UAV inspection images, especially when the fire target is small or occluded. To address this, this application proposes using WIoUv3 as the bounding box regression loss function for the YOLOv8 model to optimize its localization performance. Specifically, WIoUv3 calculates the overlap area between the predicted and ground truth bounding boxes and dynamically adjusts the weights based on the overlap area, causing the model to focus more on inaccurately located samples, thereby improving localization accuracy, particularly when the fire target is small or occluded.

[0042] In this embodiment, the optimization of the YOLOv8 model further includes: using WIoUv3 as the bounding box regression loss function for the real-time flame detection model (MSFC-YOLO) to replace the CIoU loss function of the original YOLOv8 model; the bounding box loss function is used to quantify the difference between the predicted bounding box and the ground truth bounding box. WIoUv3 uses a dynamic non-monotonic focusing mechanism to determine the difference based on the anomaly of the predicted bounding box. Dynamically allocate gradient gains to suppress the negative impact of simple and outlier samples, thereby improving the model's generalization ability. The calculation process of the WIoU loss function can be expressed as the following equation: ; ; ; ; ; ; in, , , , These represent the width and height of the ground truth bounding box and the width and height of the predicted bounding box, respectively. This refers to the degree of overlap; It is a standard based on overlap. The loss function; and These are the centers of the predicted bounding box and the ground truth bounding box, respectively. and represent the height and width of the intersecting rectangle of the real bounding box and the predicted bounding box; and are the height and width of the smallest closed bounding box containing the real bounding box and the predicted bounding box; is the union area of the real bounding box and the predicted bounding box; is the bounding box loss function; is used to characterize the abnormality degree of the predicted bounding box; is the gradient gain; and serve as hyperparameters for controlling output.

[0043] This innovative improvement can identify fire information in video streams more accurately, thereby improving the performance of the fully automatic UAV fire inspection and early warning platform. Specifically, adopting the WIoUv3 loss function can improve the detection accuracy of small target fires and reduce the missed detection rate, which makes the fire inspection of UAVs in complex environments more reliable.

[0044] The specific application process of this embodiment is as follows: after the user opens the software on the UAV remote controller, the user logs in to the DJI account through account password or mobile phone verification code in the login module; after pairing the UAV with the remote controller, mission planning is performed, including: waypoint setting, route planning, setting various parameters of waypoints (specific precise longitude and latitude, flight altitude, flight speed), formulating waypoint actions (taking photos, starting recording, stopping recording, gimbal angle adjustment, yaw angle setting); after completing mission planning, clicking start mission, the UAV starts to execute the mission according to preset parameters. In the image transmission and recognition module, the remote controller displays the real-time image input by the UAV, recognizes fire, counts the quantity and recognition probability, and automatically takes photos of fire sources with recognition probability greater than or equal to 90%.

[0045] The present invention realizes autonomous flight inspection and real-time fire recognition by UAV. The core of this platform lies in the linkage between the route planning module and the image recognition module. The route planning module can generate an autonomous flight route for the UAV to execute, so that the UAV can inspect along a preset path in high-risk or complex environments without manual real-time command. The image recognition module receives and processes the video stream returned by the UAV in real time during the UAV executing the autonomous flight route, so as to identify fire information in the video stream. This design enables the UAV to get rid of excessive dependence on ground operators, and significantly improves the automation and intelligence level of inspection.

[0046] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0047] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. Since the above embodiments are substantially similar to the method embodiments, they are described relatively simply; relevant parts can be referred to the descriptions of the method embodiments.

[0048] The above embodiments provide a detailed description of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A fully automated unmanned aerial vehicle (UAV) fire inspection and early warning platform, characterized in that, Deployed on a terminal device, the platform includes: The login module allows users to log in to the platform using an account and password or mobile phone verification. The route planning module is used to generate autonomous flight routes for the drone to execute; and The image transmission and recognition module, which is linked with the flight path planning module, is used to receive and process the video stream transmitted back by the UAV in real time during the process of the UAV executing the autonomous flight path, so as to identify the fire information in the video stream. A fire source image capturing module, which is linked with the image transmission and recognition module, is used to control the drone to automatically take pictures when the recognition probability of the fire information identified by the image transmission and recognition module exceeds a preset threshold. The image transmission and recognition module includes a pre-trained real-time flame detection model for identifying fire information in the video stream. The construction of the real-time flame detection model includes: A multi-scale feature capture module is introduced into the backbone network of the YOLOv8 model to replace the C2f module in the original model. This module is used to guide the real-time flame detection model to capture fire features at different scales and improve the recognition accuracy. The multi-scale feature capture module includes a parallel convolutional group, which includes multiple deep convolutional layers with different kernel sizes. The multiple deep convolutional layers receive the same input feature, and the outputs of each deep convolutional layer are added element by element. The multi-scale feature capture module further includes a residual connection structure, which is used to sequentially perform batch normalization, dimensionality increase convolution, nonlinear activation, and dimensionality decrease convolution on the output of the parallel convolution group to obtain fused features, and then add the fused features to the original input of the multi-scale feature capture module element by element to generate the final feature map.

2. The fully automated unmanned aerial vehicle (UAV) fire inspection and early warning platform according to claim 1, characterized in that, The parallel convolutional group includes three depthwise convolutional layers with kernel sizes of 3×3, 5×5, and 7×7; the dimension-upgrading convolution process expands the channel dimension to four times its original size using a 1×1 convolutional layer; the non-linear activation process applies the SiLU activation function; the processing steps of the multi-scale feature capture module... include: Input features First, it passes through a convolutional layer containing multiple convolutional layers of different kernel sizes and depths. Parallel convolutional group processing, convolutional layers at various depths Output The first feature is obtained by directly adding them together to obtain the output of the parallel convolution group. , expressed as: ; in, Indicates the first The output of a deep convolutional layer; For the first feature BatchNorm processing is performed, followed by a 1×1 convolutional layer. The channel dimension is expanded to four times the original size, and the SiLU activation function is applied. The second feature is obtained by performing a nonlinear transformation. The expression is: ； in, The SiLU activation function; Indicates the first feature Perform BatchNorm processing; Then use another 1×1 convolutional layer The fusion features are obtained by restoring the number of channels to their original dimensions. ; Finally, the original input features of the multi-scale feature capture module are connected via residual connections. With fusion features By performing element-wise addition, the final feature map is obtained, expressed as: ； in, This is the feature map output by the multi-scale feature capture module.

3. The fully automated unmanned aerial vehicle (UAV) fire inspection and early warning platform according to claim 1 or 2, characterized in that, The construction of the real-time flame detection model also includes: An efficient multi-scale attention module is set at the end of the backbone network of the YOLOv8 model. The efficient multi-scale attention module is used to guide the real-time flame detection model to focus on key flame features from a complex background. The YOLOv8 model employs depthwise separable convolutions and spatial channel decoupled downsampling in its backbone and neck networks, respectively replacing the traditional convolutional and downsampling layers in the original model.

4. The fully automated unmanned aerial vehicle (UAV) fire inspection and early warning platform according to claim 3, characterized in that, The efficient multi-scale attention module includes a 1×1 branch and a 3×3 branch; The 1×1 branch is used to encode the channel along the spatial height and width directions; The 3×3 branch is used to capture local cross-channel interaction information; The outputs of the two branches are aggregated through a bidirectional matrix dot product operation to generate a spatial attention map.

5. The fully automated unmanned aerial vehicle (UAV) fire inspection and early warning platform according to claim 3, characterized in that, The processing steps of the efficient multi-scale attention module include: For a given input feature map First, input feature map Divided into channels Group features ,in , For the number of channels, These are the input feature maps. Height and width; Features after grouping Simultaneously, 1×1 and 3×3 branch processing are performed; the 1×1 branch processes the grouped features through two one-dimensional global average pooling operations along the spatial height and width directions, respectively. The encoding is performed, and the encoded results are concatenated and processed through a shared convolutional layer before being decomposed into two vectors; the 3×3 branch captures local cross-channel interaction information through 3×3 convolution; The outputs of the two branches are used to generate a spatial attention map through bidirectional matrix dot product operation. The two are added together and then activated by Sigmoid to obtain the final spatial attention map. Finally, the spatial attention map is multiplied element-wise with the original feature map to obtain the final output.

6. The fully automated unmanned aerial vehicle (UAV) fire inspection and early warning platform according to claim 1, characterized in that, The real-time flame detection model uses WIoUv3 as the bounding box regression loss function to replace the CIoU loss function of the YOLOv8 model. The bounding box loss function is used to quantify the difference between the predicted bounding box and the ground truth bounding box.

7. The fully automated unmanned aerial vehicle (UAV) fire inspection and early warning platform according to claim 6, characterized in that, The expression for the bounding box loss function is: ； ； ； in, The bounding box loss function; It is used to characterize the degree of anomaly in the predicted bounding box; It is gradient gain; and It is used as a hyperparameter to control the output; and These are the centers of the predicted bounding box and the ground truth bounding box, respectively. and It is the height and width of the smallest closed box that contains both the ground truth bounding box and the predicted box; It is a standard based on overlap. The loss function.

8. The fully automated unmanned aerial vehicle (UAV) fire inspection and early warning platform according to claim 1, characterized in that, The route planning module is also used to set the flight parameters of the autonomous flight route, including flight altitude, flight speed, longitude, latitude, and yaw angle.

Images