Lightweight improved YOLOv7 flame detection method and system fused with attention mechanism

Abstract

The invention discloses a lightweight class attention mechanism fused improved YOLOv7 flame detection method and system. The method comprises the following steps: acquiring flame and non-flame images, preprocessing the flame and non-flame images, constructing a flame detection data set based on the preprocessed flame and non-flame images, and labeling and dividing the flame detection data set; the original YOLOv7 network model is optimized, and an improved YOLOv7 network model is obtained; training the improved YOLOv7 network model by using the training set, and adjusting model hyper-parameters by using the verification set in the training process to obtain a flame detection model; evaluating the performance of the flame detection model by using the test set; and deploying the flame detection model in a monitoring camera of the power distribution station, and carrying out real-time monitoring and alarming on flames in the power distribution station by using the model. The technical problem that in the prior art, light weight and detection precision of a model are mutually balanced, and actual application requirements are not easy to meet can be solved.

Description

Technical Field

[0001] The present invention relates to flame detection technology in computer vision, and in particular to a lightweight improved YOLOv7 flame detection method, system, device and storage medium integrating an attention mechanism. Background Art

[0002] In recent years, with the expansion of production scope and the rapid development of production technology, the frequency of production accidents has also increased. Among them, fire is one of the most dangerous accidents, and the causes usually involve electrical failures, human negligence, poor management of fireproof materials, natural disasters and lack of fire awareness. By taking improvement measures against these factors, the incidence of fire can be significantly reduced. Therefore, as one of the supporting facilities of the power grid, it is important and necessary to perform flame detection inside the distribution station.

[0003] A large number of electrical equipment are usually integrated in the distribution station room, which may cause fires during operation due to line faults, overloads or other reasons. Once a flame occurs, it is very easy to cause a large-scale electrical failure, or even cause the power grid to be paralyzed, resulting in huge economic losses and safety hazards. Therefore, the implementation of flame detection in the distribution station room can achieve early detection and rapid response to fires, take fire-fighting measures in time, prevent the spread of fire, and effectively ensure the safe operation of the power grid and the life safety of maintenance personnel. This measure not only improves the safety of the distribution station room, but also ensures the continuous and stable power supply.

[0004] At present, the flame detection methods in distribution stations mainly include the following: infrared flame detector, ultraviolet flame detector, multi-sensor fusion flame detection and video image flame detection.

[0005] Among them, both infrared flame detectors and ultraviolet flame detectors use the radiation characteristics of flames for identification. Infrared flame detectors have a larger coverage range but a higher false alarm rate, while ultraviolet flame detectors have a fast response speed but a smaller coverage range and are easily affected by ambient light.

[0006] Compared with single infrared flame detection and ultraviolet detection, multi-sensor fusion flame detection combines the advantages of multiple sensors and improves the overall performance of flame detection. However, it has the disadvantages of high system complexity, high equipment and operation and maintenance costs, and is susceptible to sensor data synchronization and sensor data conflicts.

[0007] Video image flame detection uses a camera to monitor the environment inside the power distribution station and uses a target detection algorithm to identify the flame. This method can accurately locate the flame and provide visual alarm information, which is helpful for remote monitoring and rapid response.

[0008] Existing flame detection methods based on video images are mainly divided into two categories: two-stage target detection algorithms represented by Fast R-CNN and single-stage target detection algorithms represented by the YOLO series. The two-stage target detection algorithm has higher detection accuracy, but the algorithm takes a long time to run and the detection speed is slow, which is not suitable for real-time detection scenarios. The single-stage target detection algorithm has a simple structure, fast detection speed, and good detection accuracy. Among them, the use of a lightweight single-stage target detection algorithm will make the model have fewer parameters, lower calculation complexity, and faster response speed in a real-time detection environment, but at the same time it will reduce the detection accuracy. Summary of the invention

[0009] Purpose of the invention: The purpose of the present invention is to provide a lightweight improved YOLOv7 flame detection method, system, device and storage medium that integrates an attention mechanism, which can solve the technical problem in the prior art that the lightweight of the model and the detection accuracy are mutually balanced and difficult to meet the actual application needs.

[0010] Technical solution: A lightweight improved YOLOv7 flame detection method integrating attention mechanism of the present invention comprises:

[0011] Collect flame images and non-flame images, preprocess the flame images and non-flame images, and build a flame detection data set based on the preprocessed flame images and non-flame images, and annotate and divide the flame detection data set to obtain a training set, a validation set, and a test set;

[0012] The original YOLOv7 network model is optimized to obtain an improved YOLOv7 network model;

[0013] The improved YOLOv7 network model is trained using the training set. During the training process, the hyperparameters of the improved YOLOv7 network model are adjusted using the validation set to obtain a flame detection model. The performance of the flame detection model is evaluated using the test set.

[0014] The flame detection model is deployed in the surveillance camera of the distribution station, and the flame detection model is used to monitor and alarm the flame in the distribution station in real time.

[0015] Furthermore, flame images and non-flame images are collected, the flame images and non-flame images are preprocessed, and a flame detection data set is constructed based on the preprocessed flame images and non-flame images. The flame detection data set is annotated and divided, and the training set, validation set and test set are obtained, including:

[0016] Collect flame images and non-flame images, mix the flame images and non-flame images in proportion, and construct an initial data set;

[0017] Preprocess all images in the initial data set to construct a flame detection data set;

[0018] All images in the flame detection dataset are manually annotated using a preset annotation tool;

[0019] The manually annotated flame detection dataset is divided into training set, validation set and test set in proportion.

[0020] Furthermore, the original YOLOv7 network model is optimized to obtain an improved YOLOv7 network model, including:

[0021] Build the MobileNetv3-small BackBone to replace the original BackBone of the YOLOv7 network model.

[0022] Replace the two CBS modules at the input ends of the two Upsample modules in the Head of the original YOLOv7 network model with GSConv modules respectively;

[0023] Replace the two Upsample modules in the Head of the original YOLOv7 network model with CARAFE modules respectively;

[0024] Add a CBAM module at the output end of the CBS module in the output branch of the image feature map F1 and the image feature map F2 respectively;

[0025] A new CBAM module is added, and the input end of the CBAM module is connected to the output end of the GSConv module at the output end of the SPPCSPC module, and the output end of the CBAM module is connected to the input end of the CARAFE module at the output end of the GSConv module.

[0026] Furthermore, the training set is used to train the improved YOLOv7 network model. During the training process, the validation set is used to adjust the hyperparameters of the improved YOLOv7 network model to obtain a flame detection model. The test set is used to evaluate the performance of the flame detection model, including:

[0027] The CIoU loss function of the original YOLOv7 network model is replaced by the Shape-IoU loss function, which is used to train the improved YOLOv7 network model.

[0028] The images in the training set are input into the improved YOLOv7 network model for iterative optimization. After each training cycle, the indicators of the improved YOLOv7 network model are calculated on the validation set, and the hyperparameters of the improved YOLOv7 network model during the training process are appropriately adjusted. The training process is iterated until the Shape-IoU loss function is fully converged, and the parameters of the improved YOLOv7 network model are saved to obtain the flame detection model. The performance of the flame detection model is evaluated using the test set.

[0029] Furthermore, the flame detection model is deployed in the surveillance camera of the power distribution station, and the flame detection model is used to monitor and alarm the flame in the power distribution station in real time, including:

[0030] The flame detection model is deployed on the surveillance camera of the power distribution station, and the confidence level corresponding to each frame of the video is obtained in real time;

[0031] Determine whether the confidence of the current frame of the video is greater than the confidence threshold. If it is greater than the confidence threshold, it is determined that there is a flame in the current monitoring area;

[0032] Upload the current frame image containing flames, the confidence level of whether it is a flame, the current time, the location name of the monitoring area, and the coordinate values ​​of the four corners of the detection box in the frame image to the server database;

[0033] The server backend sends the above message to the staff, notifying them to take appropriate measures in a timely manner.

[0034] Furthermore, the preprocessing includes rotating and cropping the image, and enhancing image features using denoising and filtering techniques.

[0035] Furthermore, the LabelImg annotation tool was used, set to the YOLO format, to manually annotate all the images in the flame detection dataset.

[0036] Based on the same inventive concept, a lightweight improved YOLOv7 flame detection system integrating attention mechanism of the present invention includes:

[0037] An image acquisition and preprocessing module is used to acquire flame images and non-flame images, preprocess the flame images and non-flame images, construct a flame detection data set based on the preprocessed flame images and non-flame images, and annotate and divide the flame detection data set to obtain a training set, a validation set, and a test set;

[0038] The model optimization module is used to optimize the original YOLOv7 network model to obtain an improved YOLOv7 network model;

[0039] The model training module is used to train the improved YOLOv7 network model using the training set, and to adjust the hyperparameters of the improved YOLOv7 network model using the validation set during the training process to obtain a flame detection model; and to evaluate the performance of the flame detection model using the test set;

[0040] The flame detection module is used to deploy the flame detection model in the monitoring camera of the distribution station, and use the flame detection model to monitor and alarm the flame in the distribution station in real time.

[0041] Based on the same inventive concept, the present invention provides a lightweight improved YOLOv7 flame detection device with fusion attention mechanism, including a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the electronic device implements the steps of the above-mentioned lightweight improved YOLOv7 flame detection method with fusion attention mechanism.

[0042] Based on the same inventive concept, a computer-readable storage medium of the present invention stores a computer program thereon, which, when executed by a processor, implements the steps of the improved YOLOv7 flame detection method with the above-mentioned lightweight fusion attention mechanism.

[0043] Beneficial effects: Compared with the prior art, the significant technical effects of the present invention are as follows:

[0044] The present invention designs a lightweight improved YOLOv7 network model architecture. By replacing the BackBone of the original YOLOv7 network model with the MobileNetv3-Small network, efficient image depth feature extraction is achieved, the overall network parameters are reduced, and the network operation speed is improved.

[0045] The present invention replaces part of the CBS modules in the Head with GSConv modules in the lightweight improved YOLOv7 network model architecture, further realizing the lightweight of the network model and maintaining the detection accuracy.

[0046] The present invention replaces the Upsample module in the Head with the CARAFE module in the lightweight improved YOLOv7 network model architecture, reduces the loss of image features in the upsampling process, and further enhances the feature extraction capability of the feature enhancement network.

[0047] The present invention adds a CBAM module to the Head of the lightweight improved YOLOv7 network model architecture, so that the channel attention mechanism can take into account richer feature information and further optimize the representation ability of the image feature map.

[0048] The present invention uses the Shape-IoU loss function to replace the CIoU loss function of the original YOLOv7 network detection model, which speeds up the training speed of the model and enables the model to more effectively evaluate the accuracy of prediction when encountering targets of different shapes. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a flowchart of a lightweight improved YOLOv7 flame detection method integrating an attention mechanism disclosed in an embodiment of the present invention;

[0050] Figure 2 This is the original YOLOv7 network architecture diagram;

[0051] Figure 3 It is a MobileNetv3-Small network architecture diagram disclosed in an embodiment of the present invention;

[0052] Figure 4 is a structural diagram of a GSConv module disclosed in an embodiment of the present invention;

[0053] Figure 5 is a structural schematic diagram of a CBAM module disclosed in an embodiment of the present invention;

[0054] Figure 6 It is an optimized YOLOv7 network architecture diagram disclosed in an embodiment of the present invention;

[0055] Figure 7 It is a structural schematic diagram of a lightweight improved YOLOv7 flame detection system integrating an attention mechanism disclosed in an embodiment of the present invention;

[0056] Figure 8 It is a structural schematic diagram of a lightweight improved YOLOv7 flame detection device fused with an attention mechanism disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. It will be understood by those skilled in the art that the purposes and advantages that can be achieved with the present invention are not limited to the specific description of the above beneficial effects, and the above and other purposes that can be achieved by the present invention will be more clearly understood based on the following detailed description.

[0058] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed in the present invention can be implemented in hardware, software or a combination of the two. Whether it is implemented in hardware or software depends on the specific application and design of the technical solution. Professional and technical personnel can use different methods to implement the functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0059] Reference to "embodiments" in the present invention means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0060] Example 1

[0061] See also Figure 1 , Figure 1 : is a flowchart of a lightweight improved YOLOv7 flame detection method with fusion attention mechanism disclosed in an embodiment of the present invention. Figure 1 The improved YOLOv7 flame detection method described is applied to a power distribution station, such as flame detection in a power distribution station, etc., and the embodiments of the present invention are not limited thereto. Figure 1 As shown, the improved YOLOv7 flame detection method with lightweight fusion attention mechanism can include the following operations:

[0062] S1. Collect flame images and non-flame images, pre-process the flame images and non-flame images, and construct a flame detection dataset based on the pre-processed flame images and non-flame images, and annotate and divide the flame detection dataset to obtain a training set, a validation set, and a test set. The specific steps are as follows:

[0063] S1.1. Collect flame images and non-flame images, mix the flame images and non-flame images in proportion, and construct an initial data set.

[0064] Surveillance cameras are used to capture images containing flames in indoor and outdoor safe areas. Crawler technology is used to crawl public flame images from image websites, and surveillance videos containing flames are captured from short video platforms and video frames containing flames are intercepted. Public non-flame images are crawled from image websites. Flame images and non-flame images are mixed according to the ratio to construct an initial dataset.

[0065] The flame image data in this embodiment include but are not limited to: flame-smoke images in indoor open scenes and outdoor open scenes, flame-smoke images in closed factory buildings, and flame-smoke images of indoor and outdoor power transmission lines. Non-flame image data include but are not limited to direct sunlight and strong artificial light sources at night. The mixing ratio used in this embodiment is 9:1, and 90% of the data in the data set is flame image data, and 10% of the data is non-flame image data.

[0066] S1.2, preprocess all images in the initial data set to construct a flame detection data set;

[0067] In this embodiment, the preprocessing includes rotating and cropping the image, and enhancing the image features by using denoising and filtering techniques.

[0068] S1.3. Manually annotate all images in the flame detection dataset using a preset annotation tool;

[0069] In this embodiment, the LabelImg annotation tool is used and set to the YOLO format to manually annotate all images in the flame detection dataset.

[0070] S1.4. Divide the manually annotated flame detection dataset into training set, validation set and test set in proportion.

[0071] In this embodiment, the division ratio adopted is 7:2:1, that is, 70% of the data in the data set is divided into a training set for model training, 20% of the data is divided into a validation set for model validation, and 10% of the data is divided into a test set for model testing. The specific division ratio can also be adjusted and optimized according to actual training.

[0072] S2. Optimize the original YOLOv7 network model to obtain an improved YOLOv7 network model, such as Figure 6 shown.

[0073] like Figure 2 As shown in the figure, the original YOLOv7 network model contains three layers: Input (input layer), BackBone (backbone network), and Head (detection head layer). BackBone extracts multi-level features of the image through four CBS modules, four ELAN modules, and three MP-1 modules, and gradually downsamples to increase the receptive field; Head fuses and spatially pools the multi-level features of the image through four CBS modules, four ELAN-W modules, two Upsample modules, two MP-2 modules, one SPPCSPC module, and four Concat modules to enhance the information richness and scale adaptability of the feature map; the fused feature map is further processed through three REP-CBM module branches to generate detection results at multiple scales.

[0074] The specific steps of step S2 are as follows:

[0075] S2.1. Build the MobileNetv3-small BackBone to replace the BackBone of the original YOLOv7 network model.

[0076] The BackBone of the original YOLOv7 network model is the CSPDarknet53 network, which requires high-performance computing resources to meet the accuracy requirements of real-time detection. In order to deploy the flame detection model in edge devices with limited processing units and computing performance, this embodiment makes lightweight improvements to the original YOLOv7 network model, using the MobileNetv3-Small network to replace the BackBone of the original YOLOv7 network model, and retains the Head of the original YOLOv7 network model.

[0077] like Figure 3 As shown, the MobileNetv3-Small network structure of this embodiment includes three parts: the starting part, the middle part, and the last part. The starting part uses a Conv2d module to perform preliminary extraction of image features; the middle part uses eleven Bottleneck modules to perform multi-layer convolution to achieve efficient image depth feature extraction and keep the model lightweight; the last part uses a Conv2d module to integrate the feature map of the image.

[0078] The MobileNetv3 network features the use of deep separable convolution and an inverse residual structure with a linear bottleneck, and a lightweight attention module is added to the inverse residual structure. Replacing the YOLOv7 backbone network with MobileNetv3 can reduce the overall network parameters and improve the network's operating speed.

[0079] like Figure 3 As shown, Figure 3The Conv2d module is represented as Conv2d-a1-b1-c1. In Conv2d-3-16-HS, Conv2d refers to standard convolution, a1=3 refers to the convolution kernel size of 3×3, b1=16 refers to the number of output channels of 16, and c1=HS refers to the use of H-Swish activation function. The meaning of Conv2d-1-1024-HS is similar to this. The Bottleneck module is represented as Bneck-a2-b2-c2-d, Bneck-3-16 In -RE-SE, Bneck refers to the Bottleneck module of MobileNetv3, a2=3 means that the convolution kernel size of the channel-by-channel convolution in the inverse residual structure is 3×3, b2=16 means that the number of output channels is 16, c2=RE means that the ReLU activation function is used (if c2=HS means that the H-Swish activation function is used), and d=SE means that the Bneck uses a lightweight attention module (SE, if d is empty, it means that the Bneck does not use a lightweight attention module). MobileNetv3 outputs an 80×80×24 image feature map F1 at the output of Bneck-3-24-RE, a 40×40×48 image feature map F2 at the output of Bneck-5-48-HS-SE, and a 20×20×1024 image feature map F3 at the output of Conv2d-1-1024-HS.

[0080] S2.2. Replace the CBS modules at the input ends of the two Upsample modules in Head with GSConv modules respectively.

[0081] In this embodiment, the optimization of the feature graph convolution module in the YOLOv7 network model Head is achieved by replacing part of the CBS module with the GSConv module.

[0082] S2.3. Replace the two Upsample modules in Head with CARAFE modules respectively.

[0083] In this embodiment, the optimization of the feature upsampling module in the YOLOv7 network model Head is achieved by replacing the nearest neighbor interpolation upsampling with a lightweight universal upsampling operator CARAFE.

[0084] S2.4. Add CBAM modules to the output ends of the CBS modules in the output branches of F1 and F2.

[0085] S2.5. Add a CBAM module. The input end of the CBAM module is connected to the output end of the GSConv module at the output end of the SPPCSPC module. The output end of the CBAM module is connected to the input end of the CARAFE module at the output end of the GSConv module.

[0086] In this embodiment, the optimization of the feature extraction part in the YOLOv7 network model Head is achieved by adding a CBAM module.

[0087] like Figure 4 As shown, the GSConv module enhances the nonlinear expression ability of the model through the depth-wise separable convolution (DSC) layer and the Shuffle layer, and at the same time, it optimizes the calculation process, reduces redundant calculations, and improves the efficiency and effect of the network. The GSConv module adds the step of feature generation through the depth-wise separable convolution operation. If all CBS modules in the original YOLOv7 network model are replaced with GSConv modules, the network structure of the model will be deepened, making the model too large and complex, and the reasoning time will be longer, affecting the real-time detection performance. In order to maintain a faster reasoning speed and good detection accuracy, this embodiment only replaces the CBS modules at the input ends of the two Upsample modules in the Head with GSConv modules.

[0088] Feature upsampling is a key operation in the target detection task. The feature upsampling part in the YOLOv7 network model uses the nearest neighbor interpolation upsampling. This method performs upsampling operations by copying the nearest pixel values ​​and relies on fixed interpolation for upsampling, which cannot fully utilize the semantic information of the feature map. This embodiment uses a lightweight universal upsampling operator CARAFE to replace the nearest neighbor interpolation upsampling. CARAFE generates an adaptive upsampling kernel by learning the content of each feature point in the feature map, and uses the adaptive upsampling kernel to reorganize the input feature map, so that these feature points gather the semantic information of the context in a wider receptive field, thereby effectively reducing the loss of image features in the upsampling process and further enhancing the feature extraction capability of the feature enhancement network.

[0089] like Figure 5 As shown, the CBAM module is mainly composed of a channel attention part (CA) and a spatial attention part (SA). This structure combines the channel attention mechanism with the spatial attention mechanism, so that CBAM can optimize the effect of feature extraction at multiple levels, so that the network can filter and strengthen key features more effectively. The design of the CBAM module is very lightweight and can be added to multiple positions of the model without significantly increasing the complexity or running time of the model. Therefore, in this embodiment, a CBAM module is added after the convolution module at the output end of the image feature map F1, F2 and F3, respectively, so that the channel attention mechanism can take into account richer feature information and further optimize the representation ability of the image feature map.

[0090] S3. Use the training set to train the improved YOLOv7 network model. During the training process, use the validation set to adjust the hyperparameters of the improved YOLOv7 network model to obtain a flame detection model. Use the test set to evaluate the performance of the flame detection model. The specific steps are as follows:

[0091] S3.1. Use the Shape-IoU loss function to replace the CIoU loss function of the original YOLOv7 network model. The Shape-IoU loss function is used to train the improved YOLOv7 network model.

[0092] S3.2. Input the images in the training set into the improved YOLOv7 network model for iterative optimization. After each training cycle, calculate the indicators of the improved YOLOv7 network model on the validation set, appropriately adjust the hyperparameters of the improved YOLOv7 network model during the training process, iterate the training process until the Shape-IoU loss function fully converges, save the parameters of the improved YOLOv7 network model, and obtain the flame detection model. Use the test set to evaluate the performance of the flame detection model.

[0093] The preprocessed flame detection dataset images are input into the improved YOLOv7 network for iterative optimization. After each training cycle, the model indicators are calculated on the validation set and the hyperparameters in the training process are appropriately adjusted.

[0094] In this embodiment, the model performance is evaluated by three indicators: precision (Precision, P), recall (Recall, R), and mAP.

[0095] The CIoU loss function introduces multiple additional calculation dimensions, which not only increases the computational complexity in the model training process, but also may cause the algorithm to overfit. In the case of limited computing resources, the training speed will be slowed down. In order to solve the above problems, this embodiment uses the Shape-IoU loss function to replace the CIoU loss function. The Shape-IoU loss function improves the shortcomings of the traditional IoU loss function in dealing with complex shape targets by introducing shape similarity metrics, improves the generalization ability of the model in complex scenes, and enables the model to maintain a high detection accuracy when encountering targets of different shapes. When the detected target has a complex shape, the Shape-IoU loss function can more effectively evaluate the accuracy of the prediction.

[0096] S4. Deploy the flame detection model in the surveillance camera of the distribution station, and use the flame detection model to monitor and alarm the flame in the distribution station in real time. The specific steps are as follows:

[0097] S4.1. Deploy the flame detection model on the surveillance camera of the power distribution station and obtain the confidence level corresponding to each frame of the video in real time;

[0098] S4.2, determining whether the confidence of the current frame of the video is greater than the confidence threshold. If it is greater than the confidence threshold, it is determined that there is a flame in the current monitoring area;

[0099] S4.3, upload the current frame image containing flames, the confidence level of whether it is a flame, the current time, the location name of the monitoring area, and the coordinate values ​​of the four corners of the detection box in the frame image to the server database;

[0100] S4.4. The server backend sends the above message to the staff, notifying them to take appropriate measures in a timely manner.

[0101] The lightweight improved YOLOv7 flame detection method with fusion attention mechanism of the present invention has the characteristics of lightweight model, strong real-time performance and high detection accuracy. After the iterative training, it can be deployed in the monitoring cameras of each power distribution station, and a monitoring and early warning system can be established to realize real-time monitoring of flames in the power distribution station.

[0102] Example 2

[0103] See also Figure 7 , Figure 7 The structure diagram of a lightweight improved YOLOv7 flame detection system with fusion attention mechanism disclosed in an embodiment of the present invention can realize flame detection in a power distribution station, specifically including:

[0104] An image acquisition and preprocessing module is used to acquire flame images and non-flame images, preprocess the flame images and non-flame images, construct a flame detection data set based on the preprocessed flame images and non-flame images, and annotate and divide the flame detection data set to obtain a training set, a validation set, and a test set;

[0105] The model optimization module is used to optimize the original YOLOv7 network model to obtain an improved YOLOv7 network model;

[0106] The model training module is used to train the improved YOLOv7 network model using the training set, and to adjust the hyperparameters of the improved YOLOv7 network model using the validation set during the training process to obtain a flame detection model; and to evaluate the performance of the flame detection model using the test set;

[0107] The flame detection module is used to deploy the flame detection model in the monitoring camera of the distribution station, and use the flame detection model to monitor and alarm the flame in the distribution station in real time.

[0108] In an optional embodiment, a lightweight improved YOLOv7 flame detection method with fusion attention mechanism includes: a) collecting flame images and non-flame images, preprocessing the image data, and constructing a flame detection data set; b) optimizing the original YOLOv7 network model to obtain an improved YOLOv7 network model; c) training the improved YOLOv7 network detection model based on the preprocessed flame detection data set to obtain a flame detection model; d) deploying the flame detection model in the monitoring camera of the distribution station to monitor and alarm the flame in the distribution station in real time.

[0109] Example 3

[0110] See also Figure 8 , Figure 8 : is a structural diagram of a lightweight improved YOLOv7 flame detection device with fusion attention mechanism disclosed in an embodiment of the present invention. Figure 8 The described device can be applied to a power distribution station, such as flame detection in a power distribution station, and the embodiments of the present invention are not limited thereto.

[0111] like Figure 8 As shown, the device may include a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the electronic device implements the steps of the method described in the above embodiment and can achieve the technical effect consistent with the above method.

[0112] The memory may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. The device may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the memory may be used to read and write non-removable, non-volatile magnetic media (commonly referred to as a "hard drive"). A program / utility having a set (at least one) of program modules may be stored in, for example, the memory, such program modules including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include the implementation of a network environment. The program modules typically perform the functions and / or methods in the embodiments described herein.

[0113] The processor executes various functional applications and data processing by running the program stored in the memory, such as implementing the method provided in the first embodiment of the present invention.

[0114] Example 4

[0115] Embodiment 4 of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the method described in the above embodiment are implemented and the technical effect consistent with the above method can be achieved.

[0116] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, a device or a device or used in combination with it.

[0117] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0118] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0119] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or combinations thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0120] Of course, the storage medium containing computer executable instructions provided by an embodiment of the present invention, whose computer executable instructions are not limited to the above method operations, can also execute related operations in the method provided by any embodiment of the present invention.

[0121] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A lightweight improved YOLOv7 flame detection method with fusion attention mechanism, characterized in that: include: Collect flame images and non-flame images, preprocess the flame images and non-flame images, and build a flame detection data set based on the preprocessed flame images and non-flame images, and annotate and divide the flame detection data set to obtain a training set, a validation set, and a test set; The original YOLOv7 network model is optimized to obtain an improved YOLOv7 network model; The improved YOLOv7 network model is trained using the training set. During the training process, the hyperparameters of the improved YOLOv7 network model are adjusted using the validation set to obtain a flame detection model. The performance of the flame detection model is evaluated using the test set. The flame detection model is deployed in the surveillance camera of the distribution station, and the flame detection model is used to monitor and alarm the flame in the distribution station in real time.

2. The lightweight improved YOLOv7 flame detection method with fusion attention mechanism according to claim 1, characterized in that: Collect flame images and non-flame images, preprocess the flame images and non-flame images, and build a flame detection dataset based on the preprocessed flame images and non-flame images. Label and divide the flame detection dataset to obtain a training set, a validation set, and a test set including: Collect flame images and non-flame images, mix the flame images and non-flame images in proportion, and construct an initial data set; Preprocess all images in the initial data set to construct a flame detection data set; All images in the flame detection dataset are manually annotated using a preset annotation tool; The manually annotated flame detection dataset is divided into training set, validation set and test set in proportion.

3. The lightweight improved YOLOv7 flame detection method with fusion attention mechanism according to claim 1, characterized in that: The original YOLOv7 network model is optimized to obtain an improved YOLOv7 network model, including: Build the MobileNetv3-small BackBone to replace the original BackBone of the YOLOv7 network model. Replace the two CBS modules at the input ends of the two Upsample modules in the Head of the original YOLOv7 network model with GSConv modules respectively; Replace the two Upsample modules in the Head of the original YOLOv7 network model with CARAFE modules respectively; Add a CBAM module at the output end of the CBS module in the output branch of the image feature map F1 and the image feature map F2 respectively; A new CBAM module is added, and the input end of the CBAM module is connected to the output end of the GSConv module at the output end of the SPPCSPC module, and the output end of the CBAM module is connected to the input end of the CARAFE module at the output end of the GSConv module.

4. The lightweight improved YOLOv7 flame detection method with fusion attention mechanism according to claim 1, characterized in that: The improved YOLOv7 network model is trained using the training set. During the training process, the validation set is used to adjust the hyperparameters of the improved YOLOv7 network model to obtain a flame detection model. The performance of the flame detection model is evaluated using the test set, including: Use the Shape-IoU loss function to replace the CIoU loss function of the original YOLOv7 network model, and use the Shape-IoU loss function to train the improved YOLOv7 network model; The images in the training set are input into the improved YOLOv7 network model for iterative optimization. After each training cycle, the indicators of the improved YOLOv7 network model are calculated on the validation set, and the hyperparameters of the improved YOLOv7 network model are appropriately adjusted during the training process. The training is iterated until the Shape-IoU loss function is fully converged, and the parameters of the improved YOLOv7 network model are saved to obtain the flame detection model. The performance of the flame detection model is evaluated using the test set.

5. The lightweight improved YOLOv7 flame detection method with fusion attention mechanism according to claim 1, characterized in that: The flame detection model is deployed in the surveillance camera of the power distribution station, and the flame detection model is used to monitor and alarm the flame in the power distribution station in real time, including: The flame detection model is deployed on the surveillance camera of the power distribution station, and the confidence level corresponding to each frame of the video is obtained in real time; Determine whether the confidence of the current frame of the video is greater than the confidence threshold. If it is greater than the confidence threshold, it is determined that there is a flame in the current monitoring area; Upload the current frame image containing flames, the confidence level of whether it is a flame, the current time, the location name of the monitoring area, and the coordinate values ​​of the four corners of the detection box in the frame image to the server database; The server backend sends the above message to the staff, notifying them to take appropriate measures in a timely manner.

6. The lightweight improved YOLOv7 flame detection method with fusion attention mechanism according to claim 2, characterized in that: The preprocessing includes rotating and cropping the image, and enhancing the image features by using denoising and filtering techniques.

7. The lightweight improved YOLOv7 flame detection method with fusion attention mechanism according to claim 2, characterized in that: Use the LabelImg annotation tool, set to YOLO format, to manually annotate all images in the flame detection dataset.

8. A lightweight improved YOLOv7 flame detection system with fusion attention mechanism, characterized in that: include: An image acquisition and preprocessing module is used to acquire flame images and non-flame images, preprocess the flame images and non-flame images, construct a flame detection data set based on the preprocessed flame images and non-flame images, and annotate and divide the flame detection data set to obtain a training set, a validation set, and a test set; The model optimization module is used to optimize the original YOLOv7 network model to obtain an improved YOLOv7 network model; The model training module is used to train the improved YOLOv7 network model using the training set, and to adjust the hyperparameters of the improved YOLOv7 network model using the validation set during the training process to obtain a flame detection model; and to evaluate the performance of the flame detection model using the test set; The flame detection module is used to deploy the flame detection model in the monitoring camera of the distribution station, and use the flame detection model to monitor and alarm the flame in the distribution station in real time.

9. A lightweight improved YOLOv7 flame detection device with fusion attention mechanism, characterized in that: The electronic device comprises a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the electronic device implements the steps of the improved YOLOv7 flame detection method with a lightweight fusion attention mechanism as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the improved YOLOv7 flame detection method with a lightweight fusion attention mechanism as described in any one of claims 1 to 7.

Images