Electrical fire identification method based on DB-YOLOv8 and BTFF network

Abstract

The application relates to the technical field of machine vision, and discloses an electrical fire identification method based on a DB-YOLOv8 and a BTFF network. The method comprises the following implementation steps: step 1, acquiring an electrical fire picture, and preprocessing the electrical fire picture to obtain a pretreated image; step 2, taking a YOLOv8 backbone network as the basis, and redesigning the YOLOv8 backbone network into a double-branch backbone network; step 3, introducing three feature layers for FPN feature fusion for each branch in step 2 respectively, and finally inputting six obtained feature layers into a binary tree type feature fusion network; and step 4, training the improved YOLOv8 network model by using a training data set, and acquiring an electrical fire identification model. The method can improve the detection efficiency of high-resolution images and the detection precision of electrical fires, and improves the overall safety of electrical equipment.

Description

Technical Field

[0001] This invention belongs to the field of machine vision technology, specifically relating to an electrical fire identification method based on DB-YOLOv8 and BTFF networks. Background Technology

[0002] Electricity plays a vital role in modern society. While serving humanity as an essential energy source for daily life and production, it is also prone to causing major electrical fires due to various factors, drawing significant public attention. Electrical fires have become a major category of fires affecting fire safety in my country. Machine vision, a branch of artificial intelligence and an important component of it, is used to some extent by most artificial intelligence systems. Simply put, machine vision uses electronic video recording equipment to capture image information and achieve a visual effect similar to that of human and animal eyes.

[0003] In the prior art, a machine vision-based electrical safety hazard identification system (application number CN202011171771.9) includes several independent image acquisition modules, each comprising an image acquisition unit and a lighting system, with the image acquisition module and signal transmission module connected bidirectionally. This invention, by setting up a power supply system, an image acquisition system, a control system, and a signal display system; and a machine vision-based automatic tracking and positioning method and apparatus for identifying fire sources (application number CN202010430901.X), clusters fire source images and extracts fire source features, employing deep learning to construct a fire source identifier, enabling rapid fire source identification, reducing subsequent fire source identification time, and improving the firefighting efficiency of fire-fighting robots.

[0004] In actual image recognition processes, the acquired image data is generally high-resolution, but there are inevitably some images with poor quality or low brightness. In low-brightness or poor-quality images, the brightness, color, and texture features of flames may be masked, making it difficult for detection algorithms to accurately extract fire-related features. For low-brightness images, simple contrast enhancement or brightness adjustment may not be able to recover the details in the image, and may even introduce new distortions. Moreover, fire scenes are often accompanied by smoke or strong light, which further reduces image quality and interferes with the recognition of fire features. Electrical fire scenes may contain complex backgrounds, making it more difficult to distinguish between the background and flames in low-quality images, which can easily lead to misjudgments.

[0005] Therefore, an electrical fire identification method based on DB-YOLOv8 and BTFF networks is provided to address the technical deficiencies mentioned in the background section. Summary of the Invention

[0006] To solve the above technical problems, the application provides an electrical fire identification method based on a DB-YOLOv8 and a BTFF network.

[0007] In a first aspect, the application provides an electrical fire identification method based on a DB-YOLOv8 and a BTFF network, including the following steps:

[0008] Step 1: obtaining an electrical fire picture, pre-processing the electrical fire picture to obtain a pre-processed image, and labeling the pre-processed image to obtain a training data set;

[0009] Step 2: constructing a double-branch backbone network based on a YOLOv8 backbone network, adding a connection layer between the two branches of the double-branch backbone network, and fusing a CBAM module into an SPPF module;

[0010] Step 3: introducing three feature layers from each branch in step 2 for FPN feature fusion, a total of six feature layers, and performing binary tree type feature fusion on the six feature layers to obtain an improved YOLOv8 network model;

[0011] Step 4: training the improved YOLOv8 network model using the training data set, obtaining an electrical fire identification model, inputting an electrical fire picture to be identified into the electrical fire identification model, and obtaining an electrical fire identification result.

[0012] Specifically, in step 1, the pre-processing includes: supplementing the electrical fire picture using a CycleGAN network, determining whether the brightness or quality of the supplemented picture is less than a corresponding threshold, if yes, enhancing the supplemented picture using an SCI algorithm, and obtaining the pre-processed image after traversing all supplemented images.

[0013] Specifically, the CycleGAN network uses two generators and two discriminators.

[0014] Specifically, in step 2, based on the network structure of YOLOv8l, the backbone network is changed into a double-branch structure, the double-branch structure includes an S branch backbone network and an M branch backbone network, the S branch backbone network is used to extract low-level scale features from a high-resolution image, and the M branch backbone network is used to extract high-level scale features from a low-resolution image after downsampling.

[0015] Specifically, the first layer of the S branch backbone network uses a 3x3 convolution kernel with a step of 2, and the remaining layers each use a 3x3 convolution kernel combined with a C2f_1_n module, the 2nd and 3rd layers of the M branch backbone network are ordinary 3x3 convolution kernels, and the structures of the remaining layers are the same as those of the S branch backbone network.

[0016] Specifically, the CBAM module is composed of a channel attention module and a spatial attention module, and the step of fusing the CBAM module into the SPPF module is:

[0017] Step 21, input the feature map, pass through the channel attention module, learn the channel level attention, make the network automatically adjust the weight of each channel when processing the input feature map, and the network pays attention to the feature channel useful for fire detection and ignores redundant or irrelevant channels, generates channel weight and acts on the input feature;

[0018] Step 22, the weighted feature map is transmitted into the spatial attention module again, and the spatial dimension is adjusted by attention, so that the network pays attention to the key fire area in the image and suppresses the background noise, and the spatial attention module helps the model to locate the target more accurately, generates spatial weight, and acts on the spatial dimension of the feature map;

[0019] Step 23, the output of the channel attention module and the spatial attention module is a feature map that enhances important channels and spatial regions.

[0020] Specifically, in step 3, the FPN feature fusion forms a double FPN feature fusion network, the six feature layers are fused in a binary tree type feature fusion network, the feature layers with similar channels are fused two by two, then the three feature layers obtained are fused again in a binary tree type feature fusion network, and finally the two fused feature layers are spliced in channels.

[0021] Specifically, in step 4, the improved YOLOv8 network model uses detection accuracy, recall rate, average accuracy and average precision as evaluation indexes.

[0022] In a second aspect, the present application provides an electrical fire identification method based on the DB-YOLOv8 and BTFF network, which comprises an image data preprocessing module, an image feature extraction module, an image feature fusion module and a fusion layer inspection identification module.

[0023] The image data preprocessing module acquires an electrical fire picture, pre-processes the electrical fire picture to obtain a pre-processed image, and labels the pre-processed image to obtain a training data set.

[0024] The image feature extraction module is based on a YOLOv8 backbone network to construct a double-branch backbone network, adds a connection layer between the two branches of the double-branch backbone network, and fuses a CBAM module into an SPPF module.

[0025] The image feature fusion module respectively leads out three feature layers for FPN feature fusion for each branch in step 2, a total of six feature layers are led out, the six feature layers are subjected to binary tree type feature fusion, and an improved YOLOv8 network model is obtained;

[0026] The fusion layer inspection and identification module trains the improved YOLOv8 network model using the training data set, obtains an electrical fire identification model, inputs an electrical fire picture to be identified into the electrical fire identification model, and obtains an electrical fire identification result.

[0027] Compared with the prior art, the beneficial effects of the technical scheme of the present application are at least as follows:

[0028] 1. The electrical fire identification method based on the DB-YOLOv8 and the BTFF network disclosed in the present application adopts a CycleGAN generative adversarial network for supplementing part of the data set in the data preprocessing stage, supplements part of the data set, solves the problem of insufficient data set, enhances the image with low brightness after part of the supplement by the SCI algorithm, innovatively proposes a double-branch backbone network based on a YOLOv8 backbone network, and fuses a CBAM module into an SPPF module; a double-FPN feature fusion network and a binary tree type feature fusion network are proposed, efficient feature extraction and feature fusion are completed, the details of the picture are reduced, the processed data set is input into the network model for training and electrical fire image test identification, and the identification effect is ideal.

[0029] 2. The present application can improve the detection effect of electrical fire, effectively reduce personnel casualties, and improve the overall safety of electrical equipment. The present application highly combines the digital and intelligent transformation concept of the power grid, conforms to the development trend of informatization, digitization and digitalization of the power industry, and meets the fundamental needs of today's intelligent power grid. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiment or prior art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the provided drawings.

[0031] Figure 1 The flowchart of the electrical fire identification method based on the DB-YOLOv8 and the BTFF network of the present application;

[0032] Figure 2 The block diagram of the electrical fire identification method based on the DB-YOLOv8 and the BTFF network of the present application;

[0033] Figure 3 A CycleGAN network structure diagram of the electrical fire identification method based on the DB-YOLOv8 and the BTFF network of the present application;

[0034] Figure 4 A generator structure diagram of the electrical fire identification method based on the DB-YOLOv8 and the BTFF network of the present application;

[0035] Figure 5 A discriminator structure diagram of the electrical fire identification method based on the DB-YOLOv8 and the BTFF network of the present application;

[0036] Figure 6 An SCI network structure diagram of the electrical fire identification method based on the DB-YOLOv8 and the BTFF network of the present application;

[0037] Figure 7 A double-branch backbone network structure diagram of the electrical fire identification method based on the DB-YOLOv8 and the BTFF network of the present application;

[0038] Figure 8 A C2f_1_n module structure diagram of the electrical fire identification method based on the DB-YOLOv8 and the BTFF network of the present application;

[0039] Figure 9 A Concat connection layer structure diagram of the electrical fire identification method based on the DB-YOLOv8 and the BTFF network of the present application;

[0040] Figure 10 A CBAM module structure diagram of the electrical fire identification method based on the DB-YOLOv8 and the BTFF network of the present application;

[0041] Figure 11 A GSPPF structure diagram of the electrical fire identification method based on the DB-YOLOv8 and the BTFF network of the present application;

[0042] Figure 12 An FPN structure diagram of the electrical fire identification method based on the DB-YOLOv8 and the BTFF network of the present application;

[0043] Figure 13 A binary tree type feature fusion network structure diagram of the electrical fire identification method based on the DB-YOLOv8 and the BTFF network of the present application;

[0044] Figure 14 An improved YOLOv8 network structure diagram of the electrical fire identification method based on the DB-YOLOv8 and the BTFF network of the present application;

[0045] Figure 15 Figure 1 is a graph showing the relationship between the model performance and the confidence of the electrical fire identification method based on the DB-YOLOv8 and BTFF network according to the present application. DETAILED DESCRIPTION

[0046] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. Obviously, the specific embodiments described here are only used to explain the present application and are part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0047] It should be noted that if the present application involves "first", "second" and the like in the embodiments, the "first", "second" and the like are only for description purposes and cannot be understood as indicating or implying the relative importance of the technical features indicated or the number of technical features indicated. Therefore, the features with "first" and "second" can be explicitly or implicitly included at least one of the features. In addition, the technical solutions of each embodiment can be combined with each other, but it must be based on the fact that a person skilled in the art can realize it, and when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist and is not within the scope of protection claimed by the present application.

[0048] Figure 1 Figure 1 is a flowchart of one embodiment of the electrical fire identification method based on the DB-YOLOv8 and BTFF network provided by the present application, which specifically includes the following steps:

[0049] Step 1, obtaining an electrical fire picture, pre-processing the electrical fire picture to obtain a pre-processed image, labeling the pre-processed image to obtain a training data set, wherein the pre-processing includes: using a CycleGAN network to supplement the electrical fire picture, when the brightness or quality of the supplemented picture is less than the corresponding threshold, if yes, using a SCI algorithm to enhance the supplemented picture, after traversing all the supplemented images, obtaining the pre-processed image.

[0050] CycleGAN network is a generative adversarial network (GAN) architecture for unsupervised image-to-image conversion, which uses two generators and two discriminators, so that the model can learn the mapping from one domain to another without paired training data, specifically:

[0051] CycleGAN is a symmetrical structure, in which the upper half is to map the pictures in X (real) source domain to Y (fake) source domain through G (generator), and Dy discriminates whether Y (fake) is a real picture or a generated picture. The idea of cycle consistency is used to remap the generated picture Y (fake) to the picture in X (cycle) source domain through F. In order to prevent the generated picture X (cycle) from losing the content of the picture in X (real) source domain, the cycle consistency loss function is calculated between the generated picture X (cycle) and the picture in X (real) source domain, so as to make the two pictures as similar as possible, that is, , and the lower half structure is to first map Y (real) source domain to X (fake) source domain picture through F, and then map it to Y (cycle) through G (generator), finally realize , and the structure diagram is as shown in Figure 3 .

[0052] The network structure of the generator is composed of an encoder, a feature conversion module and a decoder. The encoder includes three convolutional layers for processing the input image. First, a 7x7 convolutional kernel with a step of 1 is used. Then, two 3x3 convolutional kernels with a step of 2 are used. The three convolutional layers all use ReLU activation function. After the processing of these convolutional layers, the encoder realizes the reduction of the image size and increases the number of channels of the feature map. The feature conversion module is located in the core of the generator and is composed of nine residual modules. The role of these residual modules is to extract the detailed features of the image. Each module contains convolution operation. The structure of the decoder is similar to that of the encoder and is mainly composed of two transposed convolutional layers and a convolutional layer. The transposed convolution first uses interpolation (zero padding) to expand the feature map size, and then uses convolution kernel to perform convolution operation on the interpolated feature map, finally obtains a large feature map. The transposed convolutional layer uses a 3x3 convolutional kernel with a step of 2 and also uses ReLU activation function. The role of the two transposed convolutional layers is to upsample the feature map output by the feature conversion module to restore the image size and reduce the number of channels at the same time. The convolutional kernel size of the last convolutional layer is 7x7 and the activation function is tanh. Such design enables the generator to effectively learn the mapping from the input domain to the target domain, convert the input image to the image in the target domain through the convolutional neural network, and keep the main features and details of the image. The generator learns step by step through training. The feature conversion module between the encoder and the decoder plays the role of connection and conversion, and generates more real and similar images to the target domain distribution. The structure of the generator is as shown in Figure 4 .

[0053] In CycleGAN, the discriminator adopts a PatchGAN model instead of the traditional design of outputting a single probability value. The traditional discriminator usually outputs a probability value representing the authenticity of the input image as a probability in a binary classification task. The difference of PatchGAN is that it divides the input image into multiple local regions and outputs an independent probability value for each region. These probability values form a matrix, usually 16x16, where each matrix element represents the probability of the corresponding region in the image being true. The design philosophy of PatchGAN is to improve the discriminator's perception of local details in the image through this local region evaluation. The size of the receptive field corresponding to each matrix point is set to 70x70 by the parameter, which means that the discriminator can focus on different local information of the image to make more accurate authenticity judgments. Finally, CycleGAN takes the average of all values in this probability matrix to obtain the final judgment result for the entire image. In terms of specific structure, the discriminator of CycleGAN is usually composed of five convolutional layers, each with a kernel size of 4x4 and a stride of 2, and uses LeakyReLU as the activation function. By using the PatchGAN model, the discriminator of CycleGAN can output a matrix form of local region authenticity evaluation, making the model more efficient in capturing local features of the image. The goal of the discriminator is to judge whether the input image is a real image or a generated image through classification. Through training, the model is optimized to enable it to recognize false images generated by the generator and give higher scores to real images. The discriminator and the generator are in an antagonistic training process. The goal of the generator is to deceive the discriminator as much as possible to make the generated image appear realistic enough. Ultimately, through continuous training, the generator can generate increasingly realistic images. The discriminator structure is as shown in Figure 5 .

[0054] The SCI self-calibration illumination network is composed of an illumination estimation module and a self-calibration module. The self-calibration module estimates and adjusts the illumination conditions of the image through a deep learning model, automatically adjusts the insufficient or excessive light parts in the image, improves the exposure stability, and significantly reduces the computational burden. As the input of the next stage of illumination estimation, there is a correlation between the underexposed picture y and the clear picture z, and the most important part is the illumination component x. The SCI network structure is as shown in Figure 6 . The calculation formula of the illumination estimation module is:

[0055]

[0056] where F is the main function of the illumination estimation module. The residual of the t-th stage can improve exposure stability and greatly reduce the calculation burden, and the role is to learn part of the illumination component at each stage in the form of a cascade network, and finally learn the entire illumination component; The illumination component of the t-th stage is the initial illumination component The original input is The low-light image; The parameter introduced by the illumination estimation module The mapping to learn the illumination component , and It is independent of the number of stages, that is, the weight sharing state is maintained at each stage.

[0057] The calculation formula of the self-calibration module is:

[0058]

[0059] Where G is the main function of the self-correction module; The target image output by the t-th stage is Indicates element-wise division; The self-calibration mapping of the t-th stage is The parameterized operator introduced with learnable parameters The parameterized operator introduced with learnable parameters The input after calibration for the next stage, and the number of stages 1≤t≤T, the self-correction module divides the input of each stage by the first stage, and the original low-illumination input is the input of the first stage. Connect the first stage to explore the convergence behavior between stages, introduce a self-calibration mapping To represent the difference between each level of input and the first level of input, the self-correction module can ensure that the outputs of different stages can converge to the same state during the training process. Finally, the conversion at the t-th stage (1≤t≤T) can be represented as:

[0060]

[0061] The loss function of SCI is composed of fidelity loss and smoothing loss, and unsupervised learning is used to expand the network's ability. The fidelity loss is to ensure the pixel-level consistency between the estimated illumination component and the input of each stage, and its formula is:

[0062]

[0063] Where T is the total number of stages.

[0064] The smoothing loss can make the overall light and dark transition of the picture tend to be smooth, and avoid the phenomenon of over-bright or over-dark in a certain area, and its formula is:

[0065]

[0066] where N is the total number of pixels, i represents the i-th pixel, N(i) represents the neighboring pixels of i in the 5x5 sliding window, represents the weight; and are the i-th and j-th illumination components, respectively.

[0067] The formula of the weight is:

[0068]

[0069] where c is the image channel in the YUV color space; = 0.1 is the standard deviation of the Gaussian kernel.

[0070] The formula of the total loss function is:

[0071]

[0072] where, and are the balance coefficients.

[0073] Step 2, a double-branch backbone network is constructed based on the YOLOv8 backbone network, which is a DB-YOLOv8 network. A connection layer is added between the two branches of the double-branch backbone network, and a CBAM module is integrated into the SPPF module. Based on the network structure of YOLOv8l, the backbone network is transformed into a double-branch structure, which reduces the parameter amount while focusing on small targets without affecting the detection of large targets. The double-branch structure includes an S branch backbone network and an M branch backbone network. The S branch backbone network is used to extract low-level scale features from high-resolution images, and the M branch backbone network is used to extract high-level scale features from low-resolution images after downsampling. The connection layer between the two branches enables feature fusion. The structure diagram is shown in Figure 7 .

[0074] When the S branch backbone network is used, the high-resolution image is directly input to the S branch for feature extraction without downsampling, focusing on small target features. The first layer of the S branch backbone network uses a 3x3 convolution kernel with a stride of 2 to reduce the output feature map size by half and increase the channel number to 24. The remaining structure of the S branch backbone network is a 3x3 convolution kernel combined with a C2f_1_n module for feature extraction, which optimizes the network structure and improves the performance of the model. The C2f_1_n module improves the computational efficiency, reduces the computational cost, enhances the feature representation ability, and maintains good accuracy. Through feature separation, the computational load and memory usage can be significantly reduced. The structure of the C2f_1_n module is shown in Figure 8As shown, the input feature map is first subjected to a 1x1 convolution operation and then split into two parts; the separated feature maps are subjected to a convolution operation with two 3x3 convolution kernels, and then the input features and the output features after convolution are spliced through a jump connection to extract more rich features; the feature maps of the two separated paths after processing are spliced, and then a 1x1 convolution operation is performed to output the final feature map.

[0075] The 2nd and 3rd layers of the M branch backbone network are ordinary 3x3 convolution kernels, and the structures of the remaining layers are the same as those of the S branch backbone network. The number of C2f_1_n modules is modified to be twice that of the S branch, and the output channel number is {128, 256, 512, 512}. The M branch backbone network increases the Concat connection layer to fuse features with the S branch backbone network, and the structure diagram of the Concat connection layer is as shown in Figure 9 The outputs of the second, third and fourth layers of the C2f_1_n module of the S branch are spliced with the inputs of the first, second and third layers of the C2f_1_n module of the M branch. After a 1x1 convolution layer, the features are fused, and then a 1x1 convolution layer is used to adjust the channel number. The adjusted channel number is {128, 256, 512} as the input of the first three C2f_1_n modules of the M branch.

[0076] The SPPF module is a spatial pyramid pooling acceleration structure that obtains information of different scales through maximum pooling operation. The CBAM module is introduced into the SPPF module to complete the structural modification of the SPPF module. Through feature fusion of different scales, the feature expression ability is improved, and the calculation cost is reduced.

[0077] The CBAM module is composed of a channel attention module and a spatial attention module, and the structure is as shown in Figure 10 The CBAM module is fused into the SPPF module as shown in

[0078] S1: First, the input feature map is subjected to the channel attention module, and the network automatically adjusts the weight of each channel when processing the input feature map by learning the channel level attention. The network focuses on the feature channels useful for fire detection and ignores redundant or irrelevant channels. The network extracts more meaningful features from the feature map, enhances the detection ability of the model for complex scenes and small target fires, and the calculation formula is:

[0079]

[0080] wherein, represents a sigmoid activation function, is the channel attention weight, is the input feature map, that is, the channel weight can be generated and applied to the input feature;

[0081] S2: Then, the weighted feature map is transmitted into the spatial attention module to adjust the attention of the spatial dimension, so that the network focuses on the key fire area in the image and suppresses the background noise. The spatial attention module helps the model to locate the target more accurately. The calculation formula is:

[0082]

[0083] wherein, represents a sigmoid activation function, is a spatial attention weight, represents a channel splicing operation, generates a spatial weight, and acts on the spatial dimension of the feature map;

[0084] S3: The output of the channel attention module and the spatial attention module is a feature map that enhances important channels and spatial regions.

[0085] The feature map output by the CBAM module is input into the SPPF module. In order to further reduce the loss of details caused by spatial pyramid pooling, a grouping operation is used. The GSPPF structure is as shown in Figure 11 . One group is S group, which only performs feature fusion operation through a 1x1 convolution layer, and the features of small targets are preserved to the greatest extent. The other group is M group, which reduces the channel number by half through a 1x1 convolution layer, and then performs spatial pyramid pooling operation. SPPF introduces spatial pyramid pooling, which can perform pooling operation at different scales and aggregate features of different sizes, effectively extracting multi-scale features. Small target fires usually have low resolution in images and are difficult to be effectively captured by convolutional neural networks. SPPF enhances feature extraction at different scales, so that small target fires can have better feature expression. The feature map output after SPPF is processed by two 1x1 convolution layers, first performing feature fusion and then reducing the channel number by half, and then performing channel splicing with the feature map from S group. Finally, a 1x1 convolution layer is used to output the final fused feature map. While increasing the computational cost, it reduces the loss of small target features to enhance feature expression and improve detection accuracy.

[0086] Step 3, each branch in step 2 is respectively introduced into three feature layers for FPN feature fusion, a total of six feature layers are introduced, and the six feature layers are fused in a binary tree type to obtain an improved YOLOv8 network model. After FPN feature fusion, a double FPN feature fusion network is formed. After the binary tree type feature fusion of the six feature layers, a binary tree type feature fusion network, i.e. BTFF network, is formed. First, the feature layers with similar channels are fused two by two, then the three feature layers obtained are again fused in a binary tree type, and finally the two fused feature layers are spliced in channels.

[0087] Specifically, the FPN feature fusion forms a double FPN feature fusion network, the multi-scale feature layers output by the second, third and fourth layer C2f_1_n modules in the S branch are set as The obtained multi-scale feature layers are processed by FPN, and the processed feature layers are The structure diagram of the FPN is as shown in Figure 12 The processing idea is to fuse the high-level (low resolution) and low-level (high resolution) feature maps to enhance the network's perception ability of different scale targets. The FPN fuses the high-level feature maps of low resolution into the corresponding high-resolution low-level feature maps through up-sampling, and enhances the semantic information of the low-level feature maps through multi-scale feature fusion, so that the network can capture the features of small targets from more detailed information and better detect targets of different scales. The FPN outputs multiple scale feature maps, which are respectively used for target detection of different sizes.

[0088] The multi-scale feature layers output by the second and third layer C2f_1_n modules in the M branch and the feature layers output by the GSPPF module are set as The obtained multi-scale feature layers are processed by FPN, and the processed feature layers are After double FPN processing, six feature layers are obtained These six feature layers will be finally fused by a binary tree type feature fusion to further strengthen the detection effect of targets of different scales.

[0089] In step 3, the six feature layers are fused by a binary tree type feature fusion to form a binary tree type feature fusion network, and the six feature layers obtained after the double branch backbone network is fused by FPN feature fusion are input into the binary tree type feature fusion network, and the structure is as shown in Figure 13 .

[0090] Among them, the three feature layers from the S branch will be halved in size by a 3x3 convolution kernel, and then paired two by two After the Concat layer, the feature layers are obtained , and the feature layers are introduced for detection; then After the size is adjusted by a 3x3 convolution kernel, the feature layers are obtained for detection; finally The feature layers are obtained by Concat splicing , and the feature layers are introduced for detection. This structure continuously fuses low-level features into high-level features, continuously strengthens low-level features in high-level features, and pays more attention to low-level features in multi-scale features.

[0091] Step 4, training the improved YOLOv8 network model using the training data set, obtaining an electrical fire identification model, inputting the electrical fire picture to be identified into the electrical fire identification model, and obtaining an electrical fire identification result.

[0092] In step 4, the deployment environment of the improved YOLOv8 network model is Python 3.8, the CPU is Intel(R) Xeon(R) Gold 5418Y processor, the GPU selects NVIDIA GeForce RTX 4090 24GB graphics card, the operating system is Ubuntu 20.04, the deep learning framework based on PyTorch 1.11.0 is selected, the CUDA version is 11.3, the training round is set to 200, the batch size is set to 64, the learning rate is set to 0.01, the weight decay coefficient is 0.0005, and the momentum is 0.937. The network structure of the improved YOLOv8 of the present application is as shown in Figure 14 The obtained YOLOv8 network model is verified by experiment, and the detection accuracy, recall rate, average accuracy and average precision are used as evaluation indexes. Precision refers to the proportion of actual positive samples in all positive samples predicted by the model, which measures the prediction accuracy of the model for positive samples. Recall refers to the proportion of correctly predicted actual positive samples, which measures the coverage of the model for positive samples. AP is obtained by calculating the area under the precision-recall curve (PR curve), AP integrates the performance of precision and recall at different thresholds, reflects the detection effect of the model at all prediction thresholds, and the larger the AP value, the higher the model detection performance. mAP is the average value of AP, which is used to measure the overall performance of the multi-class target detection model, and is the result of averaging the AP values of all classes. Among them, YOLOv5, YOLOv7 and YOLOv8 are selected as the comparison group for experiment, and the comparison experiment results are as shown in the following table:

[0093]

[0094] The F1 score can also reflect the performance of the target detection model, which is the harmonic mean of precision (Precision) and recall (Recall), and is used to balance the performance of precision and recall. The relationship curve between F1 score and confidence is as shown in Figure 15

[0095] ​The application adopts CycleGAN generative adversarial network in the data preprocessing stage to supplement part of the data set, supplement the part of the data set, solve the problem of insufficient data set, and enhance the SCI algorithm of the image with low brightness after part of the supplement. The double-branch backbone network based on the YOLOv8 main network is innovatively proposed, and the CBAM module is integrated into the SPPF module; the double-FPN feature fusion network and the binary tree type feature fusion network are proposed, which completes efficient feature extraction and feature fusion, and reduces the loss of details of the picture.

[0096] Figure 2 The application provides a DB-YOLOv8 and BTFF network-based electrical fire identification method, and the structure of an embodiment of the method is shown in the figure, which includes an image data preprocessing module, an image feature extraction module, an image feature fusion module, and a fusion layer inspection and identification module.

[0097] The image data preprocessing module adopts CycleGAN generative adversarial network in the data preprocessing stage to supplement part of the data set and enhance the SCI algorithm of the image with low brightness after part of the supplement.

[0098] The image feature extraction module is based on the YOLOv8 main network, which is redesigned as a double-branch backbone network, and a connection layer is added to the double-branch backbone network to enhance the feature fusion and information exchange between the two branches, and the CBAM module is integrated into the SPPF module.

[0099] The image feature fusion module extracts three feature layers from each branch in step 2 for FPN feature fusion, and six feature layers are obtained after the feature fusion of the two branches. The six feature layers are subjected to binary tree type feature fusion, first two feature layers with similar channels are fused, then the three feature layers obtained are subjected to binary tree type feature fusion again, and finally the two fused feature layers are subjected to channel splicing.

[0100] The fusion layer inspection and identification module trains and verifies the network model by using the preprocessed data set, and tests and identifies the electrical fire image by using the obtained weight.

[0101] According to another aspect of the embodiment of the application, a storage medium is provided, which stores program instructions, wherein the program instructions control the device where the storage medium is located to execute the DB-YOLOv8 and BTFF network-based electrical fire identification method.

[0102] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or the entire or part of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0103] The above-mentioned embodiments only express the preferred implementation of the present application, and the description is more specific and detailed, but it cannot be understood as a limitation on the scope of the patent of the present application. It should be noted that, for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of protection of the present application. Therefore, the scope of protection of the patent of the present application should be subject to the appended claims.

Claims

1. An electrical fire identification method based on DB-YOLOv8 and BTFF networks, characterized in that, Includes the following steps: Step 1: Obtain electrical fire images, preprocess the electrical fire images to obtain preprocessed images, and annotate the preprocessed images to obtain training datasets; Step 2: Construct a dual-branch backbone network based on the YOLOv8 backbone network, add a connection layer between the two branches of the dual-branch backbone network, and integrate the CBAM module into the SPPF module; Step 3: Extract three feature layers from each branch in Step 2 and perform FPN feature fusion, resulting in a total of six feature layers. Perform binary tree-type feature fusion on the six feature layers to obtain the improved YOLOv8 network model. Step 4: Train the improved YOLOv8 network model using the training dataset to obtain an electrical fire identification model. Input the electrical fire image to be identified into the electrical fire identification model to obtain the electrical fire identification result. In step 2, based on the YOLOv8l network structure, its backbone network is transformed into a dual-branch structure, namely an S-branch backbone network and an M-branch backbone network. The S-branch backbone network is used to extract low-scale features from high-resolution images, and the M-branch backbone network is used to extract high-scale features from low-resolution images after downsampling. The first layer of the S-branch backbone network uses a 3×3 convolutional kernel with a stride of 2, and the remaining layers each use a 3×3 convolutional kernel paired with a C2f_1_n module. The second and third layers of the M-branch backbone network use ordinary 3×3 convolutional kernels, and the structure of the remaining layers is the same as that of the S-branch backbone network. In step 3, the FPN feature fusion forms a dual FPN feature fusion network. The six feature layers are fused in a binary tree pattern to form a binary tree feature fusion network. First, feature layers with similar channels are fused in pairs. Then, the three feature layers are fused in a binary tree pattern again. Finally, the two fused feature layers are spliced ​​together by channel.

2. The electrical fire identification method based on DB-YOLOv8 and BTFF networks according to claim 1, characterized in that, In step 1, the preprocessing includes: using a CycleGAN network to augment the electrical fire image, determining whether the brightness or quality of the augmented image is less than the corresponding threshold, and if so, using the SCI algorithm to enhance the augmented image. After traversing all augmented images, the preprocessed image is obtained.

3. The electrical fire identification method based on DB-YOLOv8 and BTFF networks according to claim 2, characterized in that, The CycleGAN network uses two generators and two discriminators.

4. The electrical fire identification method based on DB-YOLOv8 and BTFF networks according to claim 1, characterized in that, The CBAM module consists of a channel attention module and a spatial attention module. The steps for fusing the CBAM module into the SPPF module are as follows: Step 21: Input feature map, pass through the channel attention module, learn channel-level attention, so that when the network processes the input feature map, it automatically adjusts the weight of each channel. The network focuses on feature channels that are useful for fire detection, ignores redundant or irrelevant channels, generates channel weights and applies them to the input features. Step 22: The weighted feature map is then fed into the spatial attention module to adjust the spatial dimension, so that the network focuses on the key fire area in the image and suppresses background noise. The spatial attention module helps the model to more accurately locate the target, generate spatial weights, and apply them to the spatial dimension of the feature map. Step 23: The output of the channel attention module and the spatial attention module is a feature map that enhances important channels and spatial regions.

5. The electrical fire identification method based on DB-YOLOv8 and BTFF networks according to claim 1, characterized in that, In step 4, the improved YOLOv8 network model uses detection accuracy, recall, average accuracy, and average precision as evaluation metrics.

6. An electrical fire identification method based on DB-YOLOv8 and BTFF networks, used to implement the electrical fire identification method based on DB-YOLOv8 and BTFF networks as described in any one of claims 1-5, characterized in that, include: Image data preprocessing module, image feature extraction module, image feature fusion module, and fusion layer verification and recognition module; The image data preprocessing module acquires electrical fire images, preprocesses the electrical fire images to obtain preprocessed images, and annotates the preprocessed images to obtain training datasets. The image feature extraction module constructs a dual-branch backbone network based on the YOLOv8 backbone network, adds a connection layer between the two branches of the dual-branch backbone network, and integrates the CBAM module into the SPPF module. The image feature fusion module extracts three feature layers from each branch in step 2 for FPN feature fusion, extracting a total of six feature layers. The six feature layers are then fused in a binary tree structure to obtain an improved YOLOv8 network model. The fusion layer verification and recognition module uses the training dataset to train the improved YOLOv8 network model to obtain an electrical fire recognition model. The electrical fire image to be recognized is then input into the electrical fire recognition model to obtain the electrical fire recognition result.

Images