Electrical equipment fire detection method and system based on improved Faster R-CNN

Abstract

The invention discloses an electrical equipment fire detection method and system based on an improved Faster R-CNN. The method comprises the steps that flame and smoke pictures are acquired and marked; performing training improvement on the Faster R-CNN network, and performing dynamic picture proportion enhancement on the obtained flame and smoke pictures; and performing target detection on the acquired flame and smoke pictures based on the trained and improved Faster R-CNN network. When the improved Faster R-CNN is used for training, dynamic picture proportion enhancement is carried out on an input picture, the training precision of the neural network is improved, through improvement of the Faster R-CNN, the trained model can be used for a complex scene detection task, and the flame and smoke detection precision is better than that of a traditional model.

Description

Technical Field

[0001] The present invention relates to the field of electrical fire detection, and in particular to an electrical equipment fire detection method and system based on an improved Faster R-CNN. Background Art

[0002] At present, my country is in a technological and industrial era. Electrical automation control systems can manage and dispatch daily production in factories, and are an important means for factories to achieve automated management and monitoring. At present, electrical fire prevention mainly relies on smoke sensors, abnormal status of electrical equipment, etc., which have problems such as low efficiency and great damage. Therefore, it is necessary to automatically detect and identify fires in electrical equipment.

[0003] In today's era of industrialized production, the production workshop equipment is highly mechanized and electrified, and mechanical and electrical failures cause circuit short circuits, sparks and other problems. However, the current production fire detection method is still based on traditional smoke, temperature and light sensors. When the sensor detects flames and smoke at a certain distance, it can detect flames and smoke through ionized particles, and then further activate the fire alarm and fire extinguishing system. This method is quite robust, but the detection method relying on sensors has a delay that causes the fire to spread, and the sensor cannot provide information such as the initial location, propagation direction, scale and growth rate of the fire. With the popularization of video surveillance equipment, monitoring equipment has real-time detection and positioning capabilities, so it is necessary to use the fire detection method based on the improved Faster R-CNN for fire detection of electrical equipment. Summary of the invention

[0004] The purpose of the present invention is to provide an electrical equipment fire detection method and system based on improved FasterR-CNN to solve one or more technical problems existing in the prior art and at least provide a beneficial choice or create conditions.

[0005] The present invention adopts the following technical solutions to achieve the above-mentioned invention object:

[0006] The present invention provides an electrical equipment fire detection method based on an improved FasterR-CNN, comprising:

[0007] Get pictures of flames and smoke and label them;

[0008] Improve the training of the FasterR-CNN network and use dynamic image ratio enhancement on the acquired flame and smoke images;

[0009] Based on the trained and improved FasterR-CNN network, target detection is performed on the acquired flame and smoke images.

[0010] Furthermore, the method of obtaining and marking flame and smoke images includes:

[0011] Use OpenCV to convert fire and smoke videos into single-frame images;

[0012] Use the annotation tool to annotate the flame and smoke images, divide them into training and test sets according to the preset ratio, and convert the annotated annotation files into the COCO standard format.

[0013] Furthermore, the FasterR-CNN network includes a Backbone layer, a Neck layer and a Head layer;

[0014] The Backbone layer adopts Swin Transformer-Tiny, the Neck layer includes FPN and RPN, and the Head layer includes a fully connected layer.

[0015] Furthermore, the method for the FasterR-CNN network to perform target detection on the acquired flame and smoke images includes:

[0016] The Backbone layer extracts relevant feature maps from the input flame and smoke images, and the Backbone layer is expanded by FPN to extract the target objects at multiple scales;

[0017] The feature map generated in the Backbone layer is accepted by RPN, and nine anchor boxes with different area sizes and aspect ratios are used to slide on the feature map to generate RoIs;

[0018] RoIpooling is used to extract the features corresponding to each RoI from the feature map, and then these features are sent to the fully connected layer for target detection and frame regression to detect flames and smoke.

[0019] Furthermore, the method for performing target detection on the acquired flame and smoke images by the FasterR-CNN network also includes:

[0020] Before entering the fully connected layer, ROIpooling normalizes and outputs the obtained multiple types of candidate boxes.

[0021] Furthermore, the training loss formula of the FasterR-CNN network is as follows:

[0022] L loss =L RPN +L Multi-Branch

[0023] Where: L loss Including L RPN RPN loss and L Multi-Branch Long position loss;

[0024] L RPN RPN loss and L Multi-Branch The long loss formula is as follows:

[0025]

[0026] Where N represents the number of anchor points or bounding boxes, and the hyperparameter λ balances the training loss of the detection box and mask branches;

[0027] Classification loss L cls and the detection box regression loss L bbox The formula is as follows:

[0028]

[0029] Where, t i Represents the difference between the predicted bounding box and the true label box in the four parameter vectors; represents the difference between the true label box and the correct anchor box; p i Represents the classification probability of anchor box i; Represents the true label probability of anchor box i.

[0030] Furthermore, based on the trained and improved Faster R-CNN network, a target detection method is performed on the acquired flame and smoke images, including:

[0031] Before feeding the annotated COCO standard format annotation file into the neural network, use dynamic image ratio enhancement to adjust the pixel ratio of the training image;

[0032] The adjusted image is input into the Swin Transformer-Tiny network to extract the feature maps of flame and smoke, and generate four layers of feature maps of different sizes;

[0033] A predetermined RoI is set for each point in the flame and smoke feature map, thereby obtaining a plurality of candidate ROIs;

[0034] These candidate ROIs are put into the RPN network for binary classification and detection box regression to screen out some candidate ROIs;

[0035] Perform RoI pooling on the remaining ROIs, correspond the pixels of the original image and the feature map, and correspond the feature map to the fixed features;

[0036] These ROIs are classified by NMS and the detection box regression is used to detect flames and smoke.

[0037] Furthermore, the method for adjusting the pixel ratio of the training image includes:

[0038] Adjust the image before training. First, dynamically adjust the size of the input image so that the shortest side is at least 480 pixels and the longest side is 800 pixels. The dynamic adjustment includes 11 groups, each with a short side of 32 pixels.

[0039] Secondly, the input image size is dynamically adjusted so that the shortest side is at least 400 pixels and at most 600 pixels. Then, by randomly cropping the short side to 384 pixels and the long side to 600 pixels, the dynamic adjustment includes 3 groups of arithmetic progressions, each with a short side of 100 pixels.

[0040] Finally, the input image is dynamically adjusted so that the shortest side is at least 480 pixels and at most 800 pixels, and the longest side is 1333 pixels. The dynamic adjustment includes 11 groups, and each group has an arithmetic progression of 32 pixels in the short side.

[0041] The present invention provides an electrical equipment fire detection system based on an improved Faster R-CNN, comprising:

[0042] An acquisition unit, used to acquire flame and smoke images and mark them;

[0043] The training unit is used to train and improve the Faster R-CNN network and use dynamic image ratio enhancement on the acquired flame and smoke images;

[0044] The target detection unit is used to perform target detection on the acquired flame and smoke images based on the trained and improved Faster R-CNN network.

[0045] The beneficial effects of the present invention are as follows:

[0046] When using the improved Faster R-CNN for training, the input images are dynamically enhanced to improve the training accuracy of the neural network. Through the improvement of Faster R-CNN, the trained model can be used for complex scene detection tasks, and the detection accuracy of flames and smoke is better than that of traditional models. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A flowchart of an electrical equipment fire detection method based on an improved Faster R-CNN according to an embodiment of the present invention;

[0048] Figure 2 A structural diagram based on an improved Faster R-CNN provided according to an embodiment of the present invention;

[0049] Figure 3 A schematic diagram of two connected Swin Transformer blocks provided according to an embodiment of the present invention;

[0050] Figure 4 A structural diagram of the Swin Transformer-Tiny provided according to an embodiment of the present invention;

[0051] Figure 5 This is a schematic diagram of successful flame and smoke detection according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The present invention will be further described below in conjunction with specific examples. The following examples are only used to more clearly illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the present invention.

[0053] Example 1

[0054] Reference Figure 1-Figure 5 The present invention provides an electrical equipment fire detection method based on improved FasterR-CNN, comprising:

[0055] S100, obtains flame and smoke images and marks them;

[0056] S200, improves the training of the FasterR-CNN network and uses dynamic image ratio enhancement;

[0057] S300, based on the trained and improved FasterR-CNN network, performs target detection on the acquired flame and smoke images

[0058] In S100, obtaining the flame and smoke images and marking them specifically includes:

[0059] Use OpenCV to convert the flame and smoke video into single frames, and save one frame every 4 frames.

[0060] The annotation tool for this dataset is Labelme, which is used to annotate 1,100 flame and smoke images. The images are divided into training and test sets at a ratio of 0.9 and 0.1, and are used to train and test the accuracy of the improved algorithm for flames and smoke. The annotation files annotated by Labelme will be converted to the COCO standard format.

[0061] In S300, the FasterR-CNN network includes a Backbone network layer, a Neck layer and a Head detection layer, the Backbone layer uses Swin Trasformer-Tiny, the Neck layer includes FPN (Feature Pyramid Network) and RPN (Region Proposal Network), the Head layer includes a fully connected layer, and the method for FasterR-CNN network to perform target detection on the acquired flame and smoke pictures includes:

[0062] Extracting features (Backbone+FPN+RPN): The Backbone model of target detection has long been dominated by CNN. By designing different weight layers, neural network models of different depths can be established. At present, ResNe(x)t is the main model of deep learning neural networks. FPN is used to expand Backbone in order to better extract targets at multiple scales, which is particularly effective for detecting small targets. The input of the RPN network is the feature map output by Backbone. Nine anchor boxes of different area sizes and aspect ratios are used to slide on the feature map to generate RoIs;

[0063] RoIpooling, fully connected layer: It is necessary to extract the features corresponding to each RoI (Region of Interest) from the feature map, and then send these features to the fully connected layer for object detection and box regression. Before entering the fully connected layer, ROIpooling (region of interest pooling) will normalize the output of the obtained multiple types of candidate boxes; then use the multi-point prediction network, which consists of two prediction branches: a fully connected layer for classification prediction and a regression layer for bounding box coordinate correction;

[0064] Loss function: The training loss of FasterR-CNN consists of two parts: the training loss of RPN and the training loss in the multi-branch prediction network. The total training loss can be calculated by the following formula:

[0065] L loss =L RPN +L Multi-Branch

[0066] In the formula: L loss Including L RPN RPN loss and L Multi-Branch Long loss, below is L RPN and L Multi-Branch The formula is:

[0067]

[0068]

[0069] In the formula, N represents the number of anchor points or bounding boxes. The hyperparameter λ balances the training loss of the detection box and mask branches. The classification loss L cls and the detection box regression loss L bbox It is derived from the following formula:

[0070]

[0071] In the formula, the variable t i Represents the difference between the predicted bounding box and the true label box in the four parameter vectors (the horizontal and vertical coordinate values ​​of the center point of the bounding box, the width and height of the bounding box), represents the difference between the true label box and the correct anchor box; p i represents the classification probability of anchor box i, Represents the true label probability of anchor box i.

[0072] Among them, in S200, the FasterR-CNN network is trained and improved, and dynamic picture ratio enhancement is used, including:

[0073] The Swin Transformer block consists of the following parts: LN (Layer Normalization), MLP (Multi-Layer Perceptron), W-MSA (Window Multi-head Self-Attention), SW-MSA (Shift Win-dow Multi-Head Self-Attention).

[0074] W-MSA uses sub-attention in local windows to improve the efficiency of the model. These local windows are arranged to evenly divide the image in a non-overlapping manner. The global MSA module and the formula based on h×w modules are:

[0075] Ω(MSA)=4hwC 2 +2(hw) 2 C

[0076] Where C is the dimension of the feature. Assume that each window contains M×M patches, substitute it into the formula and multiply it by the number of windows.

[0077] Ω(W-MSA)=4hwC 2 +2M 2 HkDJ

[0078] In the formula: hw is quadratic, and M in the formula is linearly fixed and is set to 7 by default, so W-MSA reduces the computing resources.

[0079] SW-MSA uses the moving window method to connect contextual information to achieve global modeling capabilities, and uses masks to make calculations more efficient.

[0080]

[0081] In the formula: and S represent the output features of the SW-MSA, W-MSA and MLP modules of block L, respectively.

[0082] Swin Transformer structure diagram. Assuming the image size is H×W×3, after 4×4 patch partitioning, the resulting size is Patch merging is to merge adjacent small patches into a large patch, which has the effect of downsampling the feature map, and reduces the dimension of the tensor through a 1×1 convolution. Swin-Transformer has four stages to complete the calculation of self-attention in each small window. The feature map generated by stage I is Phase II is Phase III Stage IV

[0083] Swin-T: C = 96, number of layers = {2,2,6,2}

[0084] Where C refers to the number of channels in the first stage hidden layer.

[0085] Training Improvement Faster R-CNN uses dynamic image ratio enhancement. Dynamic image ratio enhancement adjusts the image before training. First, dynamically adjust the size of the input image so that the shortest side is at least 480 pixels and the maximum is 800 pixels, and the longest side is 1333; secondly, dynamically adjust the size of the input image so that the shortest side is at least 400 pixels and the maximum is 600 pixels, and then randomly crop the short side to 384 pixels and the long side to 600 pixels; finally, dynamically adjust the input image so that the shortest side is at least 480 pixels and the maximum is 800 pixels, and the longest side is 1333. The first and last dynamic adjustments contain 11 groups, each with a short side of 32 pixels, and the second dynamic adjustment contains 3 groups, each with a short side of 100 pixels.

[0086] In S300, a target detection method is performed on the acquired flame and smoke images based on the trained and improved FasterR-CNN network, including:

[0087] First, convert the marked files into COCO format;

[0088] Before the image is put into the neural network, the image ratio is adjusted using the dynamic image ratio enhancement method;

[0089] Input the adjusted image into the Swin Transformer-Tiny network to extract feature maps and generate four layers of feature maps;

[0090] A predetermined RoI is set for each point in these feature maps, thereby obtaining multiple candidate ROIs;

[0091] These candidate ROIs are put into the RPN network for binary classification and detection box regression to screen out some candidate ROIs;

[0092] Perform RoI pooling operation on the remaining ROI (match the pixels of the original image and the feature map, and match the feature map with the fixed features);

[0093] Finally, these ROIs are classified and the detection box is regressed through NMS (Non-Maximum Suppression).

[0094] Example 2

[0095] The present invention provides an electrical equipment fire detection system based on an improved FasterR-CNN, comprising:

[0096] An acquisition unit, used to acquire flame and smoke images and mark them;

[0097] The training unit is used to train and improve the FasterR-CNN network and use dynamic image ratio enhancement on the acquired flame and smoke images;

[0098] The target detection unit is used to perform target detection on the acquired flame and smoke images based on the trained and improved FasterR-CNN network.

[0099] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0100] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0101] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0102] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0103] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An electrical device based on improved FasterR-CNN, characterized in that: include: Get pictures of flames and smoke and label them; Improve the training of the FasterR-CNN network and use dynamic image ratio enhancement on the acquired flame and smoke images; Based on the trained and improved FasterR-CNN network, target detection is performed on the acquired flame and smoke images.

2. The electrical equipment fire detection method based on improved FasterR-CNN according to claim 1, characterized in that: Methods for obtaining and labeling images of fire and smoke include: Use OpenCV to convert fire and smoke videos into single-frame images; Use the annotation tool to annotate the flame and smoke images, divide them into training and test sets according to the preset ratio, and convert the annotated annotation files into the COCO standard format.

3. The electrical equipment fire detection method based on improved FasterR-CNN according to claim 2, characterized in that: The FasterR-CNN network includes a Backbone layer, a Neck layer and a Head layer; The Backbone layer adopts SwinTrasformer-Tiny, the Neck layer includes FPN and RPN, and the Head layer includes a fully connected layer.

4. The electrical equipment fire detection method based on improved FasterR-CNN according to claim 3 is characterized in that: The method for the FasterR-CNN network to perform target detection on the acquired flame and smoke images includes: The Backbone layer extracts relevant feature maps from the input flame and smoke images, and the Backbone layer is expanded by FPN to extract the target objects at multiple scales; The feature map generated in the Backbone layer is accepted by RPN, and nine anchor boxes with different area sizes and aspect ratios are used to slide on the feature map to generate RoIs; The features corresponding to each RoI are extracted from the feature map through RoIpooling, and then the features are sent to the fully connected layer for target detection and frame regression to detect flames and smoke.

5. The electrical equipment fire detection method based on improved FasterR-CNN according to claim 4, characterized in that: The method for performing target detection on the acquired flame and smoke images by the FasterR-CNN network also includes: Before entering the fully connected layer, ROIpooling normalizes and outputs the obtained multiple types of candidate boxes.

6. The electrical equipment fire detection method based on improved FasterR-CNN according to claim 4 or 5, characterized in that: The training loss formula of the FasterR-CNN network is as follows: L loss =L RPN +L Multi-Branch Where: L loss Including L RPN RPN loss and L Multi-Branch Long position loss; L RPN RPN loss and L Multi-Branch The long loss formula is as follows: Where N represents the number of anchor points or bounding boxes, and the hyperparameter λ balances the training loss of the detection box and mask branches; Classification loss L cls and the detection box regression loss L bbox The formula is as follows: Where, t i Represents the difference between the predicted bounding box and the true label box in the four parameter vectors; represents the difference between the true label box and the correct anchor box; p i Represents the classification probability of anchor box i; Represents the true label probability of anchor box i.

7. The electrical equipment fire detection method based on improved Faster R-CNN according to claim 6, characterized in that: The target detection method for the acquired flame and smoke images based on the trained and improved Faster R-CNN network includes: Before feeding the annotated COCO standard format annotation file into the neural network, use dynamic image ratio enhancement to adjust the pixel ratio of the training image; The adjusted image is input into the Swin Transformer-Tiny network to extract the feature maps of flame and smoke, and generate four layers of feature maps of different sizes; A predetermined RoI is set for each point in the flame and smoke feature map, thereby obtaining a plurality of candidate ROIs; These candidate ROIs are put into the RPN network for binary classification and detection box regression to screen out some candidate ROIs; Perform RoIpooling on the remaining ROIs, correspond the pixels of the original image and the feature map, and correspond the feature map to the fixed features; These ROIs are classified by NMS and the detection box regression is used to detect flames and smoke.

8. The electrical equipment fire detection method based on improved Faster R-CNN according to claim 7, characterized in that: The method for adjusting the pixel ratio of the training image includes: Adjust the image before training. First, dynamically adjust the size of the input image so that the shortest side is at least 480 pixels and the longest side is 800 pixels. The dynamic adjustment includes 11 groups, each with a short side of 32 pixels. Secondly, the input image size is dynamically adjusted so that the shortest side is at least 400 pixels and at most 600 pixels. Then, by randomly cropping the short side to 384 pixels and the long side to 600 pixels, the dynamic adjustment includes 3 groups of arithmetic progressions, each with a short side of 100 pixels. Finally, the input image is dynamically adjusted so that the shortest side is at least 480 pixels and at most 800 pixels, and the longest side is 1333 pixels. The dynamic adjustment includes 11 groups, and each group has an arithmetic progression of 32 pixels in the short side.

9. An electrical equipment fire detection system based on improved Faster R-CNN, characterized in that: include: An acquisition unit, used to acquire flame and smoke images and mark them; The training unit is used to train and improve the Faster R-CNN network and use dynamic image ratio enhancement on the acquired flame and smoke images; The target detection unit is used to perform target detection on the acquired flame and smoke images based on the trained and improved Faster R-CNN network.

Images