Flame detection method and system based on image enhancement technology of Laplacian pyramid

Abstract

The invention discloses a flame detection method and system based on an image enhancement technology of a Laplacian pyramid, belongs to the field of unmanned aerial vehicle image processing, and particularly relates to the field of target detection and image enhancement. The invention aims to solve the problems of poor image quality, poor detection effect in a complex smoke and fire environment, low flame detection precision and the like in the existing flame detection system. A flame detection system fusing an image enhancement network LPNet and a YOLOv8 target detection network is provided, an image is decomposed into low-frequency and high-frequency components through a Laplacian pyramid, the low-frequency and high-frequency components are reconstructed after being optimized and enhanced, and flame feature visibility is enhanced. The system is installed on a small unmanned aerial vehicle, is connected with an unmanned aerial vehicle end and a computer end through a 5G communication network, and collects high-definition video images in real time and carries out flame detection. The device is simple in structure, easy to implement, high in target detection precision, suitable for real-time application and capable of being used in the fields of fire detection, fire rescue and the like.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle (UAV) image processing, and in particular to the field of target detection and image enhancement. Background Art

[0002] Object detection is a key issue in computer vision research. Its task is to detect and identify objects in a video sequence given an image. It has important applications in security monitoring, intelligent driving, robotic perception, and other fields. Flame detection is a research hotspot in the field of object detection.

[0003] Computer vision-based flame detection technology can be further divided into two main approaches: those based on traditional computer vision techniques and those based on deep learning. Traditional computer vision-based flame detection primarily employs color-space segmentation techniques and modeling methods based on flame dynamic characteristics. However, due to the variability and non-rigid nature of flame color, these methods are limited in robustness and real-time performance, making them incapable of meeting the high detection accuracy and rapid response requirements of practical applications.

[0004] In recent years, with the rapid development of deep learning technology, it has also been widely used in the field of flame detection. Deep learning-based flame detection methods first require preprocessing the constructed training dataset and annotating the flame and smoke regions in the image. Then, an appropriate deep learning algorithm is selected for training so that the model can accurately identify and detect flames. Compared with traditional methods, deep learning models can automatically learn features from the training data, making them more adaptable to different types of flames and smoke, as well as changes in different environments. Overall, deep learning-based flame detection methods have significant advantages in accuracy, adaptability, real-time performance, resource efficiency, and scalability. Therefore, deep learning-based flame detection methods have become a hot topic of current research, and many scholars are working to further improve the accuracy of flame detection by improving the deep neural network structure and optimizing the detection algorithm.

[0005] The YOLO (You Only Look Once) family of algorithms plays a significant role in flame detection. YOLOv3, with its speed and accuracy, served as a foundation for research. YOLOv4 further improved detection accuracy by introducing advanced feature extractors and an attention mechanism. YOLOv5, with its lightweight design, excels on resource-constrained devices. YOLOv8, with its structural optimizations, has enhanced its detection capabilities for small objects. The YOLO family has now been updated to YOLOv11, which not only offers improved detection performance but also enables tracking, segmentation, and pose estimation. These algorithms not only support real-time flame detection but can also process live video streams from cameras. Furthermore, researchers have utilized deep learning algorithms to automatically extract flame features, improving detection robustness. Continuous optimization of the YOLO algorithm, incorporating flame color, shape, and dynamic characteristics, has significantly improved detection accuracy and response speed, making it a key technology in fire warning systems.

[0006] Image enhancement techniques, ranging from the early Gaussian pyramid to the Laplacian Pyramid Network (LPNet), have played a vital role in image processing and computer vision. The Laplacian Pyramid is a multi-resolution image representation method that extracts high-frequency detail information from a Gaussian pyramid to form a series of progressively smaller images. This is used for image fusion and image pyramid construction.

[0007] In the above methods, the target detection algorithm does not propose an image enhancement detection method for flame detection. Therefore, the present invention proposes a flame detection system that integrates the image enhancement network LPNet and the YOLOv8 target detection network. The system is installed on a small UAV to complete specific target detection tasks. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to provide a flame detection method and system based on Laplacian pyramid image enhancement technology, so as to solve the defects of existing flame detection and tracking systems such as poor image quality, poor detection effect in complex fireworks environments, and low flame detection accuracy.

[0009] The present invention solves its technical problems by adopting the following technical solutions.

[0010] This invention provides a flame detection method and system based on Laplacian pyramid image enhancement technology. The method and system consist of two modules: a drone-based system with a video image acquisition subsystem and a high-definition camera; and a computer-based system with an LPYOLO detector and a visual target detection and display subsystem. The two modules are connected via a 5G communication network.

[0011] The drone's video image acquisition subsystem captures high-definition video images and transmits them to a computer via 5G communication technology. The computer then reads the video images and passes them to a computer-based object detector, where they detect objects. The computer's visual detection and display subsystem displays the detection results. The system updates the detected images in real time at 40-60ms intervals.

[0012] This paper proposes a flame detection method and system based on Laplacian pyramid image enhancement technology. The proposed LPYOLO framework first enhances the input flame image using LPNet, decomposes the image into low-frequency and high-frequency components using the Laplacian pyramid, and then optimizes and enhances each component separately, reconstructing it using the Laplacian pyramid regression. The enhanced image is then fed into the YOLOv8 detection model, which quickly and accurately detects objects in the image in a single stage, ultimately achieving efficient target detection in flame scenes.

[0013] In the above network, LPNet uses the Laplacian pyramid technique to carefully decompose the image into low-frequency and high-frequency components. The low-frequency components capture the global features of the image, while the high-frequency components focus on extracting local details.

[0014] The low-frequency components of the image, obtained through Laplacian pyramid decomposition, are fed into the global enhancement module within LPNet. The image then passes through a two-channel 3×3 convolutional layer. Four parallel convolutions are then applied, using filters of 1×1, 3×3, 5×5, and 7×7, respectively. Each convolutional layer is compressed to a single channel. Feature fusion is then performed through a two-channel 3×3 convolutional layer. This optimizes the image's global contrast and effectively overcomes lighting variations caused by smoke and flames.

[0015] The cross-layer guidance module then extracts key information from the low-frequency components, which are first concatenated. A spatial attention module is then used to locate locations where low-frequency information is highly correlated with high-frequency components. Finally, a 3×3 convolutional layer is used to further refine the feature map. This feature map is then used to guide the local enhancement module to enhance high-frequency details such as flame edges and texture, thereby improving the visibility of flame features.

[0016] The features output by the cross-layer guidance module are upsampled to the local guidance module. A residual block and an affine transformation are used to fuse the guidance information into the extracted high-frequency features. The enhanced image is reconstructed using the Laplacian pyramid. The image is then fed into the YOLOv8 network.

[0017] The YOLOv8 detection network used specifically includes the following steps.

[0018] Step 1: CSPDarkNet (Cross Stage Partial Darknet) is used as its backbone network, which is responsible for extracting deep features from the image.

[0019] Step 2, feature fusion (Neck) uses PAN (Path Aggregation Network) feature fusion technology to integrate feature maps of different scales and provide rich information for the final detection head.

[0020] Step 3: The detection head accurately predicts the flame's bounding box and category, enabling fast and accurate flame detection under smoky conditions.

[0021] In the flame detection method and system based on Laplacian pyramid image enhancement technology proposed in the present invention, the LPNet end-to-end training method allows the network to learn the final result of target detection directly from the input image without the need for additional supervision information. This enables the LPYOLO framework to work stably in various environments, perform well in flame detection tasks, and can promptly identify and respond to fire hazards, which has high practical value.

[0022] Compared with the existing technology, the present invention has the following main advantages.

[0023] (1) The system structure is simple and easy to implement.

[0024] (2) The present invention provides an object detection framework, LPYOLO, that integrates an image enhancement network (LPNet) with the YOLOv8 object detection network. LPNet is introduced to enhance the details of high-frequency components such as flame edges and textures, thereby improving the visibility of flame features and optimizing flame detection scenarios. The algorithm of the present invention has a lightweight design and good generalization capabilities, and can operate stably in various environments. It is suitable for fire detection, fire rescue, and other computer vision applications requiring object detection.

[0025] (3) High target detection accuracy and good performance in complex fireworks.

[0026] (4) While maintaining high detection performance, the system also has a fast processing speed and is suitable for real-time applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is the overall structural block diagram of the system of the present invention.

[0028] Figure 2 It is a block diagram of the single-time workflow of the system of the present invention.

[0029] Figure 3 This is a structural block diagram of the flame detection framework LPYOLO of the present invention.

[0030] Figure 4 This is a structural block diagram of the flame detection method YOLOv8 of the present invention. DETAILED DESCRIPTION

[0031] The technical problem to be solved by the present invention is: a flame detection method and system based on Laplacian pyramid image enhancement technology is needed to solve the defects of existing flame detection and tracking systems such as poor image quality, poor detection effect in complex fireworks environments, and low flame detection accuracy.

[0032] The present invention will be further described below with reference to the embodiments and drawings, but the present invention is not limited thereto.

[0033] The present invention provides a flame detection method and system based on Laplacian pyramid image enhancement technology, such as Figure 1 As shown. It includes two modules, among which the first module is the drone side, which is equipped with a video image acquisition subsystem, and the equipment is the drone and the high-definition camera it carries; the second module is the computer side, which is equipped with an LPYOLO detector and a visual target detection and display subsystem, and the equipment is a computer. The two modules are connected via a 5G communication network. The drone-side video image acquisition subsystem collects high-definition video images and sends them to the computer side via 5G communication technology; the computer side reads the video images and transmits them to the computer-side detector, completes target detection in the detector, and finally displays the detection results in the computer-side visual detection and display subsystem. The system updates the detection image in real time at intervals of 40-60ms.

[0034] The present invention provides a flame detection system based on Laplacian pyramid image enhancement technology, the specific work flow is as follows Figure 2 shown.

[0035] Step 1. Initialize the system.

[0036] Step 2: The drone's high-definition camera starts working, collecting images in real time and transmitting image sequence data to the computer via 5G communication technology.

[0037] Step 3: The computer receives the high-definition image sequence and reads the image sequence.

[0038] Step 4: The image sequence is fed into the LPYOLO detector for visual object detection and display. This involves feeding the image sequence into the Laplacian pyramid of the LPNet component of the LPYOLO detector, which meticulously decomposes the image into low-frequency and high-frequency components. The low-frequency components capture global image features, while the high-frequency components focus on extracting local details. The LPNet-enhanced image is then passed to the YOLOv8 detection network. The detection head accurately predicts the flame's bounding box and category, enabling fast and accurate flame detection even under smoky conditions.

[0039] Step 5: The computer outputs the detected image sequence and displays it on the monitor.

[0040] Step 6. Single system operation is completed.

[0041] The flame detection system of the present invention works cyclically according to the above steps, displays the detection image at intervals of 40-60ms, and updates it in real time.

[0042] The present invention provides a flame detection method based on Laplacian pyramid image enhancement technology, the specific workflow is as follows: Figure 3 As shown:

[0043] Step 1. Read the high-definition image sequence.

[0044] Step 2: Decompose the image into low-frequency and high-frequency components through the Laplacian pyramid.

[0045] Step 3: Optimize and enhance the low-frequency and high-frequency components respectively.

[0046] Step 3-1, the low-frequency components are input into the global enhancement module.

[0047] In Step 3-2, the global enhancement block passes through a 2-channel 3×3 convolutional layer. Then, using four parallel convolutions, the convolutional layers of each path are compressed into a single channel using filters of 1×1, 3×3, 5×5, and 7×7, respectively. This is followed by a 2-channel 3×3 convolutional layer for further feature fusion. This optimizes the global contrast of the image and effectively overcomes lighting variations caused by smoke and flames.

[0048] In step 3-3, the cross-layer guidance module extracts key information from low-frequency components, which are first concatenated. A spatial attention module is then used to locate locations where low-frequency information is highly correlated with high-frequency components. Finally, a 3×3 convolutional layer is used to further refine the feature map. This feature map is then used to guide the local enhancement module to enhance high-frequency details such as flame edges and texture, thereby improving the visibility of flame features.

[0049] Step 3-4: The features output by the cross-layer guidance module are upsampled to the local guidance module, and a residual block and an affine transformation are used to fuse the guidance information into the extracted high-frequency features, thereby obtaining an enhanced image through Laplacian pyramid reconstruction.

[0050] Step 4: The image enhanced by LPNet is passed to the YOLOv8 detection network. Figure 4 shown.

[0051] Step 4-1, CSPDarkNet is used as its backbone network, which is responsible for extracting deep features from the image.

[0052] Step 4-2: The Neck network further processes and fuses the feature maps output by the backbone network through PAN feature fusion technology to enhance the feature expression ability and provide a more robust feature basis for detection.

[0053] Step 4-3: The Head network adopts the current mainstream form of regression and classification decoupling, and predicts the target's bounding box, category, and confidence based on the feature information provided by the Neck network.

[0054] Step 5. Obtain the visual target detection image according to the above steps.

[0055] The present invention provides a flame detection system method and system that integrates the image enhancement network LPNet and the YOLOv8 target detection network, which has the following characteristics.

[0056] (1) This invention provides a flame detection method based on Laplacian pyramid image enhancement technology. The LPNet is introduced to enhance the details of high-frequency components such as flame edges and textures, thereby improving the visibility of flame features and optimizing flame detection scenarios. The algorithm of this invention has a lightweight design and good generalization capabilities. It can operate stably in various environments and is suitable for fire detection, fire rescue, and other computer vision applications requiring target detection.

[0057] (2) The flame detection method based on Laplacian pyramid image enhancement technology is proposed. It has a simple structure, is easy to implement, has high target detection accuracy, and performs well in complex fireworks.

[0058] (3) The flame detection method based on Laplacian pyramid image enhancement technology of the present invention maintains high detection performance while also having a fast processing speed, making it suitable for real-time applications. It performs well in flame detection tasks and can promptly identify and respond to fire hazards, which is of great significance for fire prevention and early response.

[0059] Although the present invention has been particularly shown and described in conjunction with preferred embodiments, it will be understood by those skilled in the art that various changes may be made to the present invention in form and detail without departing from the spirit and scope of the invention as defined in the appended claims, and all such changes are within the scope of protection of the present invention.

Claims

1. A flame detection system based on Laplacian pyramid image enhancement technology, characterized in that: The system consists of two modules: the first, the drone side, which features a video image acquisition subsystem and is equipped with a drone and its high-definition camera; and the second, the computer side, which features an object detector and visual object detection and display subsystem and is equipped with a computer. The two modules are connected via a 5G communication network.

2. The flame detection system based on the Laplacian pyramid image enhancement technology according to claim 1, characterized in that: The drone's video image acquisition subsystem collects high-definition video images and sends them to the computer via 5G communication technology. The computer reads the video images and passes them to the computer's detector, where target detection is completed. Finally, the computer's visual detection and display subsystem displays the detection results.

3. The flame detection system based on the Laplacian pyramid image enhancement technology according to claim 2, characterized in that: The system updates the detection image in real time at an interval of 40-60ms.

4. A flame detection method based on Laplacian pyramid image enhancement technology, characterized in that: The proposed detection framework (LPYOLO) first enhances the input flame image using the Laplacian Pyramid Network (LPNet). This decomposes the image into low-frequency and high-frequency components using the Laplacian pyramid. These components are then optimized and enhanced, and then reconstructed using the Laplacian pyramid regression. The enhanced image is then fed into the YOLOv8 (You Only Look Once, YOLO) detection model, which quickly and accurately detects objects in the image in a single stage, ultimately achieving efficient object detection in flame scenes.

5. The flame detection method based on the Laplacian pyramid image enhancement technology according to claim 4, characterized in that: LPNet uses the Laplacian pyramid technique to carefully decompose the image into low-frequency and high-frequency components. The low-frequency components capture the global features of the image, while the high-frequency components focus on extracting local details.

6. The flame detection method based on the Laplacian pyramid image enhancement technology according to claim 4, characterized in that: The low-frequency components of the image, obtained through Laplacian pyramid decomposition, are fed into the global enhancement module within LPNet. The image then passes through a two-channel 3×3 convolutional layer. Four parallel convolutions are then applied, using filters of 1×1, 3×3, 5×5, and 7×7, respectively. Each convolutional layer is compressed to a single channel. Feature fusion is then performed through a two-channel 3×3 convolutional layer. This optimizes the image's global contrast and effectively overcomes lighting variations caused by smoke and flames.

7. The flame detection method based on the Laplacian pyramid image enhancement technology according to claim 4, characterized in that: The cross-layer guidance module extracts key information from low-frequency components, which are first concatenated. A spatial attention module is then used to locate locations where low-frequency information is highly correlated with high-frequency components. Finally, a 3×3 convolutional layer is used to further refine the feature map. This feature map is then used to guide the local enhancement module to enhance high-frequency details such as flame edges and texture, thereby improving the visibility of flame features.

8. The flame detection method based on the Laplacian pyramid image enhancement technology according to claim 4, characterized in that: The features output by the cross-layer guidance module are upsampled to the local guidance module, and a residual block is used to fuse the guidance information into the extracted high-frequency features through an affine transformation, so that the enhanced image is reconstructed through the Laplacian pyramid.

9. The flame detection method based on the Laplacian pyramid image enhancement technology according to claim 4, characterized in that: The YOLOv8 detection network used specifically includes the following steps: the backbone network (Backbone) is the CSPDarkNet (Cross Stage Partial Darknet) network, which is responsible for extracting deep features from the image; the feature fusion part (Neck) uses PAN (Path Aggregation Network) feature fusion technology to integrate feature maps of different scales, providing rich information for the final detection head; the detection head (Head) accurately predicts the bounding box and category of the flame, achieving fast and accurate flame detection under smoky conditions.

10. The method according to any one of claims 4 to 8, characterized in that LPNet's end-to-end training method allows the network to learn the final result of target detection directly from the input image without the need for additional supervision information. This enables the LPYOLO algorithm to work stably in various environments, perform well in flame detection tasks, and can promptly identify and respond to fire hazards, which has high practical value.

Images