Fire detection method, system, electronic device and readable storage medium

Abstract

The present application relates to a fire detection method, system, electronic device and readable storage medium, and belongs to the field of computer vision technology. The method includes acquiring an image to be detected; performing feature extraction on the image to be detected to obtain a plurality of deep feature maps of different scales; and determining the category of the image to be detected and the location of the fire accident in the image to be detected according to the plurality of deep feature maps, wherein the category includes the presence of a fire accident or the absence of a fire accident. The present application can efficiently detect whether there is a fire in the image to be detected, as well as the location information of the fire. Compared with manual fire monitoring, the present application has high accuracy, strong timeliness and less labor cost. It uses the deep feature map of the image to be detected for judgment, and can better utilize the information contained in the image to be detected, making the detection process more intelligent.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a fire detection method, system, electronic device and readable storage medium. Background Art

[0002] At present, the forest area in my country is increasing year by year, but with it comes a large number of forest fires. The fire accidents every year not only cause high economic losses, but also cause a large number of casualties. If the early stages of forest fires can be detected in time, safety accidents can be avoided, thereby reducing economic losses and reducing labor costs.

[0003] Common warning methods for forest fires are based on smoke sensors or infrared sensors, both of which are prone to false triggering and have slow responses. With the development of deep learning technology, target detection technology based on deep learning has been widely studied. For example, convolutional neural networks (CNN) can be used to detect targets in images collected by cameras.

[0004] However, existing deep learning-based object detection algorithms usually use complex network structures and large-size images as input to obtain better detection performance. They not only have a large number of parameters and computational complexity, but also require more storage space and frequent calculations during runtime.

[0005] Application Contents

[0006] The present application provides a fire detection method, system, electronic device and readable storage medium, and the technical solution is as follows:

[0007] In a first aspect, a fire detection method is provided, the method comprising:

[0008] Acquire the image to be detected;

[0009] Extracting features from the image to be detected to obtain a plurality of deep feature maps of different scales;

[0010] According to the multiple deep feature maps, the category of the image to be detected and the position of the fire accident in the image to be detected are determined, and the category includes the presence of a fire accident or the absence of a fire accident.

[0011] The beneficial effects of a fire detection method of the present application are as follows:

[0012] Through the fire detection method in the present application, it is possible to efficiently detect whether there is a fire in the image to be detected and the location information of the fire. Compared with manual fire monitoring, the present application has high accuracy, strong timeliness, and less labor cost. It uses the deep feature map of the image to be detected for judgment, and can better utilize the information contained in the image to be detected, making the detection process automated and intelligent, while also improving the accuracy of the detection results. Compared with sensor detection, it is not easily affected by environmental interference and is less likely to make misjudgments.

[0013] Based on the above solution, a fire detection method of the present application can also be improved as follows.

[0014] Further, the feature extraction is performed on the image to be detected to obtain a plurality of deep feature maps of different scales; the category of the image to be detected and the position of the fire accident in the image to be detected are determined according to the plurality of deep feature maps, wherein the category including the presence of a fire accident or the absence of a fire accident is realized by a pre-trained fire detection model, and the fire detection model includes a feature extraction network and a target detection network;

[0015] The feature extraction of the image to be detected is performed to obtain a plurality of deep feature maps of different scales, including:

[0016] By using the feature extraction network, feature extraction is performed on the image to be detected to obtain a plurality of deep feature maps of different scales;

[0017] Determining the category of the image to be detected and the location of the fire accident in the image to be detected according to the plurality of deep feature maps, wherein the category includes the presence of a fire accident or the absence of a fire accident, comprises:

[0018] Through the target detection network, based on the multiple deep feature maps, the category of the image to be detected and the location of the fire accident in the image to be detected are determined, and the category includes the presence of a fire accident or the absence of a fire accident.

[0019] The beneficial effect of adopting the above further scheme is: through the feature extraction network, deep feature maps of multiple different scales of the image to be detected are obtained, and the deep feature map can better characterize the image to be detected, thereby improving the accuracy of target detection, i.e., the category of the image to be detected and the fire location information. Automatic detection of fire accidents is achieved through the target detection network, which can avoid the further development of fire accidents and reduce the loss of manpower and material resources.

[0020] Furthermore, the feature extraction network includes a shallow feature extraction module and a deep feature extraction module. The feature extraction network is used to extract features from the image to be detected to obtain a plurality of deep feature maps of different scales, including:

[0021] Extracting a shallow feature map of the image to be detected using the shallow feature extraction module;

[0022] Extracting a plurality of deep feature maps of different scales corresponding to the shallow feature map using the deep feature extraction module;

[0023] Among them, the shallow feature extraction module includes a convolution layer with a convolution kernel size of 3×3, a stride parameter of 2, and a padding parameter of 1, a batch normalization layer, and an activation function.

[0024] The beneficial effect of adopting the above further scheme is: a shallow feature map is extracted by a shallow feature extraction module, and on this basis, multiple deep feature maps corresponding to the shallow feature map are extracted by a deep feature extraction module, so that more color and detail feature information of the image to be detected can be extracted, and the feature information contained in the deep feature map is further enriched, which can better represent the image to be detected and further improve the accuracy of the category of the image to be detected and the location of the fire.

[0025] Furthermore, the deep feature extraction module includes seven deep feature extraction submodules connected in sequence, and the deep feature extraction module is used to extract multiple deep feature maps of different scales corresponding to the shallow feature map, including:

[0026] Inputting the shallow feature map into the first deep feature extraction submodule to obtain a deep feature map extracted by the first deep feature extraction submodule;

[0027] The second deep feature extraction submodule is used as the current submodule, and the following steps are repeatedly performed until the next submodule of the current submodule is the last deep feature extraction submodule, and the third deep feature extraction submodule, the fifth deep feature extraction submodule and the last deep feature extraction submodule are used to extract deep feature maps of different scales of the image to be detected;

[0028] The following steps include:

[0029] The deep feature map output by the previous deep feature extraction submodule of the current submodule is used as the input of the current submodule to extract the deep feature map output by the current submodule.

[0030] The beneficial effect of adopting the above further scheme is: through the seven deep feature extraction sub-modules connected in sequence, the deep feature map output by the previous deep feature extraction sub-module of the current sub-module is used as the input of the current sub-module, and the deep feature map output by the current sub-module is extracted, which can extract and retain more deep detail information of the image to be detected, further enrich the feature information contained in the deep feature map, can better represent the image to be detected, and further improve the category of the image to be detected and the accuracy of fire location.

[0031] Further, each of the deep feature extraction submodules includes a pixel attention module, a coordinate attention module and a post-processing module, and the deep feature map output by the previous deep feature extraction submodule of the current submodule is used as the input of the current submodule to extract the deep feature map output by the current submodule, including:

[0032] Utilize the pixel attention module to traverse each pixel in the input feature map of the current submodule, extract the color feature information of each pixel, and generate the color feature map of the current submodule according to each color feature information, wherein the input feature map is the deep feature map extracted by the previous deep feature extraction submodule of the current submodule;

[0033] Utilizing the coordinate attention module, extracting spatial position information of each pixel in the color feature map of the current submodule, and generating a spatial position feature map of the current submodule according to the spatial position information of each pixel;

[0034] The post-processing module is used to fuse the spatial position feature map and the input feature map of the current submodule to obtain a deep feature map output by the current submodule.

[0035] The beneficial effect of adopting the above further scheme is: each deep feature extraction sub-module extracts the color information and spatial position information of each pixel in the input feature map of the deep extraction sub-module through the pixel attention module and the coordinate attention module, and fuses the spatial position information of all pixels with the input feature map of the current sub-module through the post-processing module to obtain the deep feature map output by the deep feature extraction sub-module. On this basis, the expression ability of the deep feature map of the deep feature extraction module in the target detection network can be enhanced, thereby further improving the accuracy of target detection.

[0036] Further, each of the second to last deep feature extraction submodules further includes a preprocessing convolution module, which uses the preprocessing convolution module to increase the input channel of the current submodule to supplement the information contained in the deep feature map output by the current submodule;

[0037] The preprocessing convolution module includes a convolution layer with a convolution kernel size of 1×1 and a Stride parameter of 1 and a batch normalization layer.

[0038] The beneficial effect of adopting the above further scheme is: the input channel of the current sub-module is increased by the preprocessing convolution module, and the information contained in the deep feature map output by the current sub-module is further supplemented, which helps the target detection network to accurately identify fire accidents and locate the position of fire accidents in the image to be detected.

[0039] Further, the multiple deep feature maps of different scales of the image to be detected include a deep feature map output by the third deep feature extraction submodule which is an eight-fold downsampled deep feature map, a deep feature map output by the fifth deep feature extraction submodule which is a sixteen-fold downsampled deep feature map, and a deep feature map output by the seventh deep feature extraction submodule which is a thirty-two-fold downsampled deep feature map.

[0040] The fire detection model also includes a multi-scale feature fusion network, which includes a first channel and a second channel built based on a feature pyramid, wherein the first channel and the second channel respectively include sub-channels with eight times downsampling scales, sixteen times downsampling scales, and thirty-two times downsampling scales set in sequence, the connection direction of the first channel is from bottom to top, and the connection direction of the second channel is from top to bottom, and the sub-channels with the same sampling scale in the first channel and the second channel are horizontally connected;

[0041] The method further comprises:

[0042] The eight-fold downsampled deep feature map, the sixteen-fold downsampled deep feature map, and the thirty-two-fold downsampled deep feature map are respectively input into the first channel of the same sampling scale, and for the deep feature map of each sampling scale, the deep feature maps of the same sampling scale in the two channels are fused through the sub-channels after horizontal connection to obtain a fused feature map of the same sampling scale;

[0043] Determining the category of the image to be detected and the location of the fire accident in the image to be detected by the target detection network according to the plurality of deep feature maps includes:

[0044] The target detection network is used to determine the category of the image to be detected based on the fused feature map of the same sampling scale obtained from the deep feature map of each sampling scale.

[0045] The beneficial effect of adopting the above further scheme is: the multiple deep feature maps obtained are fused according to certain rules through the multi-scale feature fusion network to obtain multiple fused feature maps of different sampling scales, and the semantic information and spatial position information of the deep feature maps are further enhanced through the first channel, the second channel and the sub-channels after horizontal connection. The fire area can be focused more accurately in the target detection network, thereby improving the detection performance.

[0046] In a second aspect, a fire detection system is provided, the system comprising an image acquisition unit and a fire detection unit;

[0047] The image acquisition unit is used to acquire the image to be detected;

[0048] The fire detection unit is used to extract features from the image to be detected to obtain multiple deep feature maps of different scales; based on the multiple deep feature maps, determine the category of the image to be detected and the location of the fire accident in the image to be detected, the category including the presence of a fire accident or the absence of a fire accident.

[0049] The beneficial effects of a fire detection system of the present application are as follows:

[0050] Through the fire detection method in the present application, it is possible to efficiently detect whether there is a fire in the image to be detected and the location information of the fire. Compared with manual fire monitoring, the present application has high accuracy, strong timeliness, and less labor cost. It uses the deep feature map of the image to be detected for judgment, and can better utilize the information contained in the image to be detected, making the detection process automated and intelligent, while also improving the accuracy of the detection results. Compared with sensor detection, it is not easily affected by environmental interference and is less likely to make misjudgments.

[0051] According to a third aspect, an electronic device is provided. The electronic device includes a processor and a memory. The memory stores at least one instruction. The at least one instruction is loaded and executed by the processor to implement the method described in the first aspect.

[0052] According to a fourth aspect, a readable storage medium is provided, wherein a computer program is stored in the readable storage medium, and the at least one instruction is loaded and executed by a processor to implement the method described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The present application is further described below in conjunction with the accompanying drawings and embodiments.

[0054] Figure 1 A schematic diagram of the steps of a fire detection method according to an embodiment of the present application;

[0055] Figure 2This is one of the structural schematic diagrams of the MCM submodule of the embodiment of the present application;

[0056] Figure 3 This is the second structural diagram of the MCM submodule of the embodiment of the present application;

[0057] Figure 4 A flow chart of a fire detection method according to an embodiment of the present application;

[0058] Figure 5 A structural diagram of a fire detection system according to an embodiment of the present application

[0059] Figure 6 A schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0060] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0061] Figure 1 Schematic diagram of the steps of a fire detection method provided in an embodiment of the present application. Figure 1 , the embodiment comprises:

[0062] Step S1, obtaining an image to be detected.

[0063] Step S2: extract features from the image to be detected to obtain multiple deep feature maps of different scales.

[0064] Step S3, determining the category of the image to be detected and the position of the fire accident in the image to be detected based on multiple deep feature maps, the category including the presence of a fire accident or the absence of a fire accident.

[0065] Optionally, the method further includes step S10 of normalizing the image to be detected to obtain the normalized image to be detected. Step S2 includes: extracting features from the normalized image to be detected to obtain a plurality of deep feature maps of different scales. Generally, the size of the image to be detected is 416×416.

[0066] Optionally, the deep feature map includes color and texture information.

[0067] Optionally, the position of the fire accident in the image to be detected is located by a visualized rectangular frame.

[0068] Optionally, feature extraction is performed on the image to be detected to obtain multiple deep feature maps of different scales; based on the multiple deep feature maps, the category of the image to be detected and the position of the fire accident in the image to be detected are determined, and the category including the presence or absence of a fire accident is realized by a pre-trained fire detection model, and the fire detection model includes a feature extraction network and a target detection network.

[0069] Feature extraction is performed on the image to be detected to obtain multiple deep feature maps of different scales, including:

[0070] Through the feature extraction network, features of the image to be detected are extracted to obtain multiple deep feature maps of different scales.

[0071] According to multiple deep feature maps, the category of the image to be detected and the location of the fire accident in the image to be detected are determined. The category includes the existence of a fire accident or the absence of a fire accident, including:

[0072] Through the target detection network, based on multiple deep feature maps, the category of the image to be detected and the location of the fire accident in the image to be detected are determined, and the category includes the presence of a fire accident or the absence of a fire accident.

[0073] Optionally, a fire detection method described in the present application is used to detect forest fires.

[0074] Specifically, the existing network resources are used to create a forest fire detection dataset. For example, 2200 images of forest fires retrieved from the Internet can be used, and any 1800 images can be used as training sets, 200 images can be used as validation sets, and 200 images can be used as test sets. For example, the forest fire detection dataset can also be the Fire-Smoke-Dataset in the public dataset DeepQuestAI.

[0075] In the process of training the fire detection model, the images in the training set need to be preprocessed. The preprocessing includes adjusting the brightness, contrast and saturation of the images to be trained in the training set, as well as operations such as shearing transformation, horizontal flipping and normalization in image processing. Through preprocessing, in the process of feature extraction of the images to be trained in the training set, the feature information in the images to be trained is enhanced, and the obtained deep feature map can contain more effective feature information.

[0076] In the process of training the fire detection model, the number of images to be trained in the training set, i.e., the Batch-size parameter, is adaptively set according to the video memory of the graphics card to speed up the speed and efficiency of the training phase. Generally, the Batch-size is 16 or 32.

[0077] Assuming that the batch-size is 1, the process of training the fire detection model is as follows: randomly select an image to be trained from the training set, preprocess the image to be trained, obtain the preprocessed image to be trained, extract features from the preprocessed image to be trained through the feature extraction network, and obtain multiple deep feature maps of different scales of the image to be trained. Through the target detection network, the category of the image to be trained and the location of the fire accident in the image to be trained are determined based on multiple deep feature maps.

[0078] During the implementation process, the feature extraction network can be a network model built based on EfficientNet, and the target detection network can be a network model built based on the ultra-lightweight target detection model NanoDet. Among them, NanoDet is an ultra-fast and lightweight mobile anchor-free target detection model. Based on the above two network models, the network structure of the fire detection model can be made simple, with fewer parameters and less calculation.

[0079] Optionally, the feature extraction network includes a shallow feature extraction module and a deep feature extraction module. Through the feature extraction network, feature extraction is performed on the image to be detected to obtain multiple deep feature maps of different scales, including: using the shallow feature extraction module to extract the shallow feature map of the image to be detected.

[0080] The deep feature extraction module is used to extract multiple deep feature maps of different scales corresponding to the shallow feature map. The shallow feature extraction module includes a convolution layer with a convolution kernel size of 3×3, a stride parameter of 2, and a padding parameter of 1, a batch normalization layer, and an activation function.

[0081] Specifically, the shallow feature extraction module extracts the shallow feature map F of the image to be detected according to the first formula, and the first formula is: F = σ (Bn (conv (X))), X∈R 3×H×W . Where X is the image to be detected, X∈R 3×H×W , W and H are the width and height of the image to be detected, conv is the convolutional layer, B n is the batch normalization layer (BatchNorm2d), and σ is the activation function (ReLU6).

[0082] Optionally, the deep feature extraction module includes seven deep feature extraction sub-modules connected in sequence, and the deep feature extraction module is used to extract multiple deep feature maps of different scales corresponding to the shallow feature map, including: inputting the shallow feature map into the first deep feature extraction sub-module to obtain the deep feature map extracted by the first deep feature extraction sub-module.

[0083] Take the second deep feature extraction submodule as the current submodule, and repeat the following steps until the next submodule of the current submodule is the last deep feature extraction submodule. Use the third deep feature extraction submodule, the fifth deep feature extraction submodule and the last deep feature extraction submodule to extract deep feature maps of different scales of the image to be detected.

[0084] The following steps include: taking the deep feature map output by the previous deep feature extraction submodule of the current submodule as the input of the current submodule, and extracting the deep feature map output by the current submodule.

[0085] Optionally, each deep feature extraction submodule from the second to the last deep feature extraction submodule also includes a preprocessing convolution module, which uses the preprocessing convolution module to increase the input channel of the current submodule to supplement the information contained in the deep feature map output by the current submodule.

[0086] Optionally, each deep feature extraction submodule includes a pixel attention module, a coordinate attention module and a post-processing module, and uses the deep feature map output by the previous deep feature extraction submodule of the current submodule as the input of the current submodule to extract the deep feature map output by the current submodule, including:

[0087] The pixel attention module is used to traverse each pixel in the input feature map of the current submodule, extract the color feature information of each pixel, and generate the color feature map of the current submodule based on the various color feature information. The input feature map is the deep feature map extracted by the previous deep feature extraction submodule of the current submodule.

[0088] The coordinate attention module is used to extract the spatial position information of each pixel in the color feature map of the current submodule, and the spatial position feature map of the current submodule is generated according to the spatial position information of each pixel.

[0089] The post-processing module is used to fuse the spatial position feature map with the input feature map of the current sub-module to obtain the deep feature map output by the current sub-module.

[0090] Optionally, each deep feature extraction submodule also includes a depthwise separable convolution module, and the computational complexity of the fire detection model is reduced by the depthwise separable convolution module.

[0091] Specifically, the deep feature extraction module includes seven deep feature extraction sub-modules connected in sequence, which are represented as G0 module, G1 module, G2 module, G3 module, G4 module, G5 module and G6 module respectively. The pre-processing convolution module 210, the depth-separable convolution module 220, the pixel attention module 230, the coordinate attention module 240 and the post-processing module 250 constitute an MCM (multiple-blocks of color-attention modules) sub-module, as shown in FIG. Figure 2 As shown in the figure. Among them, the G0 module, G1 module, G2 module, G3 module, G4 module, G5 module and G6 module include 1, 2, 2, 3, 3, 4 and 1 MCM sub-modules respectively, but the G0 module is an MCM sub-module without a preprocessing convolution module. Both the pixel attention module and the coordinate attention module are network models built based on the attention mechanism (CoordinateAttention).

[0092] like Figure 3 As shown in the figure, the preprocessing convolution module includes a convolution layer (Conv2d(1×1)) with a convolution kernel size of 1×1 and a stride parameter of 1, a batch normalization layer (BatchNorm), and an activation function (ReLU6). The depthwise separable convolution module (DepthwiseConv2d) includes two types of convolutions with convolution kernel sizes of 3×3 and 5×5, a batch normalization layer (BatchNorm), and an activation function (ReLU6). In each depthwise separable convolution module, convolutions with convolution kernel sizes of 3×3 and 5×5 are used for feature extraction, and then the extracted features are reduced in dimension through a batch normalization layer (BatchNorm) and an activation function (ReLU6) to reduce the amount of calculation and the number of channels.

[0093] Specifically, the output result F of the deep separable convolution module in the jth MCM submodule in the i-th deep feature extraction submodule is calculated by the following second formula: ij , where the second formula is: F ij =σ(B n (conv(F))), where i is 0, 1, 2, ..., 6, j is 1, 2, 3, 4, B n is the batch normalization layer, σ is the activation function, and F is the shallow feature map of the image to be detected.

[0094] The pixel attention module and the coordinate attention module can form a color attention module. The pixel attention module includes a convolution layer (1×1 Conv2d) with a convolution kernel size of 1×1 and a stride parameter of 1 and an activation function (Sigmoid), and then through a skip connection operation (shortcut), fine-grained pixel-level flame feature information can be obtained.

[0095] Specifically, the output result P of the pixel attention module in the jth MCM submodule in the i-th deep feature extraction submodule is calculated by the following third formula: ij , where the third formula is Among them, F ij ∈R C×H×W , C, W and H are the number of channels, the width and height of the image to be detected, respectively, M PA It is the pixel attention module in the jth MCM submodule in the i-th deep feature extraction submodule.

[0096] The coordinate attention module performs adaptive mean pooling operations (XAvgPool) along the horizontal direction and adaptive mean pooling operations (YAvgPool) along the vertical direction. Specifically, two spatial pooling kernels (H, 1) and (1, W) are used to encode each channel in the horizontal and vertical directions respectively to extract the feature information z in the horizontal direction. h and the vertical feature information z w The horizontal feature information and the vertical feature information are concatenated (Concat Conv2d) and fused to further obtain fine-grained feature information. Specifically, the horizontal feature information z is exchanged. h The width and height values ​​of the image are then combined with the vertical feature information z w Perform dimensional stitching.

[0097] For the fusion operation, the convolution layer with a convolution kernel size of 1×1 and a stride parameter of 1, the batch normalization layer (BatchNorm) and the non-linear activation function (Non-linear) are used to fuse the feature information f. Then, the feature information f is separated in the horizontal and vertical directions to obtain two sets of separated feature information: the feature information f after horizontal separation h The feature information after separation from the vertical direction f w According to the feature information of the two groups after separation, a convolution layer (Conv2d) with a convolution kernel size of 1×1 and a stride parameter of 1 and a nonlinear activation function (Non-linear) is used to obtain the feature information f after separation along the horizontal direction. h The corresponding weight g h , feature information after separation along the vertical direction fw The corresponding weight g w .

[0098] The output result C of the coordinate attention module in the jth MCM submodule in the i-th deep feature extraction submodule is calculated by the following fourth formula: ij , where the fourth formula is: C ij =P ij ×g h ×g w , f = δ(conv([Z h ,Z w ])), g h =σ(F h (f h )), g w =σ(F w (f w )), F h is the feature information after horizontal separation f h The corresponding convolutional layer (Conv2d), F w is the feature information after vertical separation f w The corresponding convolutional layer (Conv2d), δ is the non-linear activation function (Non-linear).

[0099] According to the fifth formula, the deep feature map F output by the jth MCM submodule in the i-th deep feature extraction submodule is calculated i-out , where the fifth formula is: F ij is the output result of the depthwise separable convolution of the jth MCM submodule in the i-th deep feature extraction submodule, F post-i It is the output result of the post-processing module of the jth MCM sub-module in the i-th deep feature extraction sub-module.

[0100] Among them, the post-processing module includes a convolution layer with a convolution kernel size of 1×1 and a stride parameter of 1, a batch normalization layer (BatchNorm2d), and an activation function (ReLU6).

[0101] Optionally, the multiple deep feature maps of different scales of the image to be detected include a deep feature map output by the third deep feature extraction submodule, which is an eight-fold downsampled deep feature map F s8 , the deep feature map output by the fifth deep feature extraction submodule is a sixteen-fold downsampled deep feature map F s16 , the deep feature map output by the seventh deep feature extraction submodule is a deep feature map F that is downsampled thirty-two times s32 .

[0102] The fire detection model also includes a multi-scale feature fusion network, which includes a first channel and a second channel built based on a feature pyramid. The first channel and the second channel respectively include sub-channels with eight times downsampling scales, sixteen times downsampling scales, and thirty-two times downsampling scales set in sequence. The connection direction of the first channel is from bottom to top, and the connection direction of the second channel is from top to bottom. The sub-channels with the same sampling scale in the first channel and the second channel are horizontally connected.

[0103] The method also includes: inputting the deep feature map downsampled eight times, the deep feature map downsampled sixteen times, and the deep feature map downsampled thirty-two times into the first channel of the same sampling scale respectively, and for the deep feature map of each sampling scale, fusing the deep feature maps of the same sampling scale in the two channels through the sub-channels after horizontal connection to obtain a fused feature map of the same sampling scale.

[0104] The method comprises: using a target detection network to determine the category of the image to be detected and the location of the fire accident in the image to be detected based on multiple deep feature maps, including: using a target detection network to obtain a fused feature map of the same sampling scale based on the deep feature map of each sampling scale, to determine the category of the image to be detected and the location of the fire accident in the image to be detected.

[0105] Specifically, the deep feature map F is downsampled eight times s8 The corresponding fusion feature map is represented as F s8-16’ , the deep feature map F downsampled by sixteen times s16 The corresponding fusion feature map is represented as F s16” , the deep feature map F downsampled by thirty-two times s32 The corresponding fusion feature map is represented as F s32” .

[0106] Use the convolution layer with a convolution kernel size of 1×1 and a stride parameter of 1 to process F respectively. s8 、F s16 、F s32 , corresponding to the output F s8’ 、F s16’ and F s32’ , F s32’ Upsampling operation is performed to double the F s16’ Perform addition operation and output F s16-32’ Similarly, F s16-32’ Upsampling operation is performed to double the F s8’ Perform addition operation and output F s8-16’ .

[0107] F s8-16’ Downsampling operation is performed to reduce the size by half and F s16-32’ Perform addition operation and output Fs16” . s16” Downsample again to reduce the size by half and F s32’ Perform addition operation and output F s32” , and finally obtain the deep feature map F of the image to be detected s32” 、F s16” and F s8-16’ . Features with rich semantic information and rich spatial location information are obtained through the multi-scale fusion module.

[0108] The target detection network uses the GFL Head in the generalized Focal Loss (Generalized Focal Loss). The target detection network includes two branches, namely the classification branch and the regression branch. The classification branch is used to distinguish the category of the flame to determine whether there is a fire accident in the image to be detected, and the regression branch is used for positioning to determine the location of the fire accident in the image to be detected. Finally, the category of the image to be detected and the fire location information can be determined.

[0109] Specifically, the classification branch and regression branch at each sampling scale are completed by two groups of convolutions, the two groups of convolutions in the classification branch are classification convolution groups, and the two groups of convolutions in the regression branch are regression convolution groups. The classification convolution group and the regression convolution group include two 2D convolutions with a convolution kernel size of 3×3, a stride parameter of 1, and a padding parameter of 1, respectively. The number of input and output channels of each convolution is 96. Then, in the classification branch at each sampling scale, a 2D convolution with a convolution kernel size of 3×3, a stride parameter of 1, and a padding parameter of 1 is set. The number of input channels of the convolution is 96 and the number of output channels is 1. Similarly, in the regression branch at each sampling scale, a 2D convolution with a convolution kernel size of 3×3, a stride parameter of 1, and a padding parameter of 1 is set to build the target detection network. The number of input channels of the convolution is 96 and the number of output channels is 32.

[0110] The deep feature maps of different sampling scales are detected separately through the target detection network, which realizes the hierarchical prediction of the deep feature maps and improves the performance of the fire detection model in detecting fire areas of different scales in the image to be detected.

[0111] Optionally, based on multiple deep feature maps, determining the category of the image to be detected and the location of the fire accident in the image to be detected, including: obtaining a prediction result corresponding to the deep feature map according to the deep feature map of each sampling scale, wherein the prediction result includes the predicted category of the image to be detected and the predicted location of the fire accident in the image to be detected.

[0112] The obtained multiple prediction results are filtered using the non-maximum suppression (NMS) algorithm, and the best prediction result is retained according to predefined rules. The predicted category in the best prediction result is used as the category of the image to be detected, and the predicted position in the best prediction result is used as the position of the fire accident in the image to be detected.

[0113] Among them, the predicted position of the fire accident in the image to be detected is located by the prediction box.

[0114] The specific process of the non-maximum suppression (NMS) algorithm is as follows: obtain multiple prediction results obtained by hierarchical prediction, and the probability corresponding to the prediction category in each prediction result (1) sort all prediction boxes corresponding to the same prediction category according to all probability values, and select the prediction box corresponding to the highest probability value. (2) traverse the remaining prediction boxes in the same prediction category. If the overlapping area (IOU) between any prediction box and the prediction box corresponding to the highest probability value is greater than the set threshold, delete the prediction box (3) in another prediction category, continue to select the prediction box corresponding to the highest probability value, and repeat steps (1) to (2) until the best prediction result is obtained.

[0115] Generally, the threshold is set to 0.6. When the fire area in the image to be detected is small, the prediction result corresponding to the deep feature map downsampled eight times is more accurate. When the fire area in the image to be detected is large, the prediction result corresponding to the deep feature map downsampled thirty-two times is more accurate.

[0116] The loss function of the classification branch in the target detection network can adopt the classification loss guided by feature entropy, which is used in the target detection network to balance the content complexity differences of the samples, that is, the classification loss weight is adaptively allocated according to the content complexity differences of the training samples. The feature entropy value of samples with large content complexity is higher, and a larger weight is given when calculating the classification loss to improve the performance of the training model. Based on GFL, the loss function is defined by the following sixth formula, where the sixth formula is:

[0117] QFL(δ)=-|y-δ| β ((1-y)log(1-δ)+ylog(δ))*∑P(i)logP(i),

[0118] DFL(S i ,S i+1 )=-((y i+1 -y)log(S i )+(yy i )log(S i+1 )),

[0119] Among them, QFL(δ) is the joint representation loss function for classification and prediction box quality, ∑P(i)logP(i) is the entropy value of the feature map output by the second convolution layer in the obtained classification convolution group, δ is the output of the Sigmoid activation function, y is the intersection over union (IOU) between the predicted box and the true labeled box, and the continuous integral is discretized, y i ,y i+1 is the value of two small intervals after discretization, S i , S i+1 y i and i+1 Output after Sigmoid activation function.

[0120] The principle for allocating positive and negative samples is the adaptive training sample selection principle, where positive and negative samples are the division of prediction boxes in the field of target detection. When the intersection-and-union ratio between the prediction box and the true annotation box meets the preset threshold, it is considered a positive sample; otherwise, it is considered a negative sample.

[0121] In the process of training the fire detection model, NVIDIA GPU can be used for training, the stochastic gradient descent optimizer is used, the initial learning rate is set to 0.1, the batch size is adjusted to 16, and the Python language and PyTorch framework are used for training. In the evaluation stage, the CoCo evaluation criteria in the field of target detection are used to evaluate the model. Table 1 shows the experimental results of the comparison between the target detection network in this application and other target detection algorithms.

[0122] Table 1 Comparison results of the target detection network in this application with other target detection algorithms

[0123] Object Detection Algorithms AP AP50 AP75 Yolov5(s) 0.383 0.727 0.357 Faster-RCNN 0.433 0.784 0.432 GridR-CNN 0.434 0.781 0.420 ATSS 0.432 0.800 0.398 Object Detection Network 0.465 0.807 0.482

[0124] As shown in Table 1, other target detection algorithms include Yolov5(s), Faster-RCNN, Grid R-CNN, and ATSS. Yolov5(s) is a single-stage target detection algorithm that uses the PyTorch framework for easy training and use; Faster-RCNN is a classic two-stage target detection algorithm; Grid R-CNN is a two-stage target detection algorithm that first proposed a pixel-level prediction grid point positioning target; ATSS proposed an adaptive method for selecting positive samples.

[0125] In addition, in order to further verify the effectiveness of the color attention module and feature entropy proposed by us in balancing the content complexity of training samples, we also compare with some fire detection algorithms. The experiment uses the DeepQuestAI / Fire-Smoke-Dataset public flame detection dataset, which includes a total of 3,000 datasets. The training set includes three subcategories: "Fire", "NoFire" and "Neurtal", each of which has 900, 900, and 900 images; the test set is also divided into the same three subcategories: "Fire", "NoFire" and "Neurtal", each of which has 100 images for testing. The images of the "NoFire" and "Neurtal" subcategories in the dataset are collectively classified as the "no fire accident" category, and the images of the "Fire" subcategory in the dataset are collectively classified as the "fire accident" category, and then participate in training and evaluation. The accuracy, recall and precision are used as evaluation indicators in the evaluation stage. Table 2 shows the experimental results of the comparison with other fire detection algorithms.

[0126] Table 2 Comparison results of the fire detection model in this application with other fire detection algorithms

[0127] Fire Detection Algorithms Precision Recall Accuracy FireNet 0.8557 0.83 0.8967 ModifiedVGG16 0.8763 0.85 0.91 ModifiedResNet50 0.8857 0.93 0.9367 Fire Detection Model 0.9674 0.89 0.9533

[0128] As shown in Table 2, other fire detection methods include FireNet (FireNet: A Specialized Lightweight Fire & Smoke Detection Model for Real-Time IoT Applications), "Modified VGG16" and "Modified ResNet50". FireNet is a lightweight neural network, and "Modified VGG16" and "Modified ResNet50" refer to the improved "VGG6" and improved "ResNet50" networks (Deep Convolutional Neural Networks for Fire Detection in Images) used for fire detection.

[0129] In order to better illustrate and understand the principle of the method provided by the present application, the scheme of the present application is described below in conjunction with an optional specific embodiment. It should be noted that the specific implementation of each step in the specific embodiment should not be understood as a limitation on the scheme of the present application. On the basis of the principle of the scheme provided by the present application, other implementations that can be thought of by those skilled in the art should also be considered within the scope of protection of the present application.

[0130] See the flowchart of a fire detection method in the embodiment of the present application. Figure 4 As shown, the specific process of a fire detection method provided in this example is as follows:

[0131] The image to be detected 400 is obtained, and normalized to obtain the normalized image to be detected. The pre-processed image to be detected is input into the shallow feature extraction module 410, and the shallow feature extraction module 410 extracts the shallow feature map of the image to be detected.

[0132] The shallow feature map of the image to be detected is passed through the G0 module, G1 module, G2 module, G3 module, G4 module, G5 module and G6 module in the deep feature extraction module 420 in sequence, and the first deep feature map of the image to be detected is output by the G2 module, the second deep feature map of the image to be detected is output by the G4 module, and the third deep feature map of the image to be detected is output by the G6 module.

[0133] A first fused feature map corresponding to the first deep feature map, a second fused feature map corresponding to the second deep feature map, and a third fused feature map corresponding to the third deep feature map are obtained through the multi-scale fusion network 430.

[0134] All fused feature maps are input into the target detection network 440, and prediction result 1 corresponding to the first deep feature map, prediction result 2 corresponding to the second deep feature map, and prediction result 3 corresponding to the third deep feature map are obtained through the target detection network 440. This process can be regarded as hierarchical prediction, and the prediction results include the predicted category of the image to be detected and the position of the fire accident in the image to be detected.

[0135] Generally, many repeated and redundant prediction results will be generated through hierarchical prediction. Suppose that prediction category 1 is the existence of a fire accident, prediction category 2 is the existence of a fire accident, and prediction category 3 is also the existence of a fire accident. Moreover, when the prediction box in prediction category 1 and the real fire accident area in the image to be detected are basically completely overlapped, and the overlap area between the prediction box in prediction result 2, the prediction box in prediction result 3 and the real fire accident area in the image to be detected is small or basically has no overlap area, after processing by the NMS algorithm, prediction category 1 will be retained, that is, the category of the image to be detected is the existence of a fire accident, and the predicted position in prediction result 1 is used as the position of the fire accident in the image to be detected.

[0136] Through the shallow feature extraction module, shallow feature maps can be extracted from the image to be detected, which can pave the way for the extraction of deep feature maps. Through the deep feature extraction module and the color attention module, the feature expression ability of the fire detection network can be enhanced. Through the multi-scale fusion network, feature information of multiple scales can be fused to obtain rich semantic information and spatial location information.

[0137] Based on the same technical concept, the embodiment of the present application also provides a fire detection system 500, such as Figure 5 As shown, the system includes an image acquisition unit 510 and a fire detection unit 520 .

[0138] The image acquisition unit 510 is used to acquire an image to be detected.

[0139] The fire detection unit 520 is used to extract features from the image to be detected to obtain multiple deep feature maps of different scales; determine the category of the image to be detected based on the multiple deep feature maps, the category including the presence of a fire accident or the absence of a fire accident; when the category of the image to be detected is the presence of a fire accident, determine the location of the fire accident in the image to be detected based on the multiple deep feature maps.

[0140] Optionally, the fire detection unit is specifically used to: be implemented by a pre-trained fire detection model, the fire detection model comprising a feature extraction unit and a target detection unit, wherein the feature extraction network in the feature extraction unit and the target detection network in the target detection unit are also pre-trained networks.

[0141] The feature extraction unit is used to extract features from the image to be detected through a feature extraction network to obtain multiple deep feature maps of different scales.

[0142] The target detection unit is used to: determine the category of the image to be detected according to multiple deep feature maps through the target detection network; when the category of the image to be detected is that there is a fire accident, determine the location of the fire accident in the image to be detected according to the multiple deep feature maps.

[0143] Optionally, the feature extraction unit includes a shallow feature extraction unit and a deep feature extraction unit, the feature extraction unit is used to extract a shallow feature map of the image to be detected, and the deep feature extraction unit is used to extract a plurality of deep feature maps of different scales corresponding to the shallow feature map.

[0144] Among them, the shallow feature extraction unit includes a convolution layer with a convolution kernel size of 3×3, a stride parameter of 2, and a padding parameter of 1, a batch normalization layer, and an activation function.

[0145] Optionally, the deep feature extraction unit includes seven deep feature extraction sub-units connected in sequence, and the deep feature extraction unit is specifically used for:

[0146] The shallow feature map is input into the first deep feature extraction subunit to obtain the deep feature map extracted by the first deep feature extraction subunit.

[0147] Take the second deep feature extraction subunit as the current subunit, and repeat the following steps until the next subunit of the current subunit is the last deep feature extraction subunit, and use the third deep feature extraction subunit, the fifth deep feature extraction subunit and the last deep feature extraction subunit to extract deep feature maps of different scales of the image to be detected.

[0148] The following steps above include:

[0149] The deep feature map output by the previous deep feature extraction subunit of the current subunit is used as the input of the current subunit to extract the deep feature map output by the current subunit.

[0150] Optionally, each of the second to last deep feature extraction subunits further includes a preprocessing convolution unit, and the preprocessing convolution unit is used to: increase the input channel of the current subunit to supplement the information contained in the deep feature map output by the current subunit.

[0151] Optionally, each deep feature extraction sub-unit includes a pixel attention unit, a coordinate attention unit and a post-processing unit.

[0152] The pixel attention unit is used to: traverse each pixel in the input feature map of the current sub-unit, extract the color feature information of each pixel, and generate a color feature map of the current sub-unit based on the respective color feature information. The input feature map is the deep feature map extracted by the previous deep feature extraction sub-unit of the current sub-unit.

[0153] The coordinate attention unit is used to extract the spatial position information of each pixel in the color feature map of the current submodule, and generate the spatial position feature map of the current submodule based on the spatial position information of each pixel.

[0154] The post-processing unit is used to fuse the spatial position feature map with the input feature map of the current sub-module to obtain the deep feature map output by the current sub-unit.

[0155] Optionally, each deep feature extraction subunit further includes a depthwise separable convolution unit, and the depthwise separable convolution unit is used to reduce the computational complexity of the fire detection model.

[0156] Optionally, the deep feature maps of multiple different scales of the image to be detected include a deep feature map output by the third deep feature extraction submodule which is an eight-fold downsampled deep feature map, a deep feature map output by the fifth deep feature extraction submodule which is a sixteen-fold downsampled deep feature map, and a deep feature map output by the seventh deep feature extraction submodule which is a thirty-two-fold downsampled deep feature map.

[0157] The fire detection model also includes a multi-scale feature fusion unit, which is used for: a first channel and a second channel built based on a feature pyramid, the first channel and the second channel respectively include sub-channels with eight times downsampling scales, sixteen times downsampling scales and thirty-two times downsampling scales set in sequence, the connection direction of the first channel is from bottom to top, and the connection direction of the second channel is from top to bottom, and the sub-channels with the same sampling scale in the first channel and the second channel are horizontally connected.

[0158] The deep feature map downsampled eightfold, the deep feature map downsampled sixteenfold, and the deep feature map downsampled thirty-twofold are respectively input into the first channel of the same sampling scale. For the deep feature map of each sampling scale, the deep feature maps of the same sampling scale in the two channels are fused through the horizontally connected sub-channels to obtain a fused feature map of the same sampling scale.

[0159] The target detection unit is also used to determine the category of the image to be detected and the position of the fire accident in the image to be detected based on the fused feature map of the same sampling scale obtained from the deep feature map of each sampling scale.

[0160] It should be noted that: the fire detection system and the fire detection method provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0161] In an exemplary embodiment, if Figure 6 As shown, an electronic device 600 is also provided. The electronic device 600 includes a processor 610 and a memory 620. The memory 620 stores at least one instruction 621. The at least one instruction 621 is loaded and executed by the processor 610 to implement the operations performed by the fire detection method in the above embodiment.

[0162] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions, which can be executed by a processor in a terminal to complete the operations performed by the fire detection method in the above embodiment. The computer-readable storage medium can be non-transitory. For example, the computer-readable storage medium can be a ROM (Read-Only Memory), a RAM (Random Access Memory), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0163] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0164] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A fire detection method, It is characterized in that The following steps are involved: Acquire the image to be detected; Extracting features from the image to be detected to obtain a plurality of deep feature maps of different scales; Determining, according to the plurality of deep feature maps, the category of the image to be detected and the location of the fire accident in the image to be detected, wherein the category includes the presence of a fire accident or the absence of a fire accident; The feature extraction is performed on the image to be detected to obtain a plurality of deep feature maps of different scales; the category of the image to be detected and the location of the fire accident in the image to be detected are determined according to the plurality of deep feature maps, wherein the category including the presence of a fire accident or the absence of a fire accident is realized by a pre-trained fire detection model, and the fire detection model includes a feature extraction network and a target detection network; The feature extraction of the image to be detected is performed to obtain a plurality of deep feature maps of different scales, including: By using the feature extraction network, feature extraction is performed on the image to be detected to obtain a plurality of deep feature maps of different scales; Determining the category of the image to be detected and the location of the fire accident in the image to be detected according to the plurality of deep feature maps, wherein the category includes the presence of a fire accident or the absence of a fire accident, comprises: Determining, by the target detection network, the category of the image to be detected and the location of the fire accident in the image to be detected according to the plurality of deep feature maps, wherein the category includes the presence of a fire accident or the absence of a fire accident; The feature extraction network includes a shallow feature extraction module and a deep feature extraction module. The feature extraction network is used to extract features from the image to be detected to obtain a plurality of deep feature maps of different scales, including: Extracting a shallow feature map of the image to be detected using the shallow feature extraction module; Extracting a plurality of deep feature maps of different scales corresponding to the shallow feature map using the deep feature extraction module; The shallow feature extraction module includes a convolution layer with a convolution kernel size of 3×3, a stride parameter of 2, and a padding parameter of 1, a batch normalization layer, and an activation function; The shallow feature extraction module extracts the shallow feature map of the image to be detected according to the first formula; the first formula is: F = σ (Bn (conv (X))), X∈R 3×H×W ; Where F is the shallow feature map, X is the image to be detected, X∈R 3×H×W , W is the width of the image to be detected, H is the height of the image to be detected, conv is the convolutional layer, B n is the batch normalization layer, σ is the activation function; The deep feature extraction module includes seven deep feature extraction submodules connected in sequence, and the deep feature extraction module is used to extract multiple deep feature maps of different scales corresponding to the shallow feature map, including: Inputting the shallow feature map into the first deep feature extraction submodule to obtain a deep feature map extracted by the first deep feature extraction submodule; The second deep feature extraction submodule is used as the current submodule, and the following steps are repeatedly performed until the next submodule of the current submodule is the last deep feature extraction submodule, and the third deep feature extraction submodule, the fifth deep feature extraction submodule and the last deep feature extraction submodule are used to extract deep feature maps of different scales of the image to be detected; The following steps include: Taking the deep feature map output by the previous deep feature extraction submodule of the current submodule as the input of the current submodule, and extracting the deep feature map output by the current submodule; Each of the deep feature extraction submodules includes a pixel attention module, a coordinate attention module and a post-processing module, wherein the deep feature map output by the previous deep feature extraction submodule of the current submodule is used as the input of the current submodule, and the deep feature map output by the current submodule is extracted, including: Utilize the pixel attention module to traverse each pixel in the input feature map of the current submodule, extract the color feature information of each pixel, and generate the color feature map of the current submodule according to each color feature information, wherein the input feature map is the deep feature map extracted by the previous deep feature extraction submodule of the current submodule; Utilizing the coordinate attention module, extracting spatial position information of each pixel in the color feature map of the current submodule, and generating a spatial position feature map of the current submodule according to the spatial position information of each pixel; Using the post-processing module, the spatial position feature map is fused with the input feature map of the current submodule to obtain a deep feature map output by the current submodule; Each of the deep feature extraction submodules further includes: a depthwise separable convolution module; the depthwise separable convolution module includes a 3×3 convolution kernel and a 5×5 convolution kernel, the batch normalization layer and the activation function; in each depthwise separable convolution module, the 3×3 convolution kernel and the 5×5 convolution kernel are used to extract features, and the extracted features are subjected to dimensionality reduction processing through the batch normalization layer and the activation function; The output result of the depth-separable convolution module in the jth MCM submodule in the i-th deep feature extraction submodule is calculated by the second formula; the second formula is: ij =σ(B n (conv(F))); where F ij is the output result of the depthwise separable convolution module, i is 0, 1, 2, ..., 6, j is 1, 2, 3, 4, B n is the batch normalization layer, σ is the activation function, and F is the shallow feature map of the image to be detected; The output result of the pixel attention module in the jth MCM submodule in the i-th deep feature extraction submodule is calculated by the third formula; the third formula is: Among them, P ij is the output result of the pixel attention module, F ij ∈R C×H×W Where C is the number of channels, W is the width of the image to be detected, H is the height of the image to be detected, and M is the width of the image to be detected. PA is the pixel attention module in the jth MCM submodule in the i-th deep feature extraction submodule; The output result of the coordinate attention module in the jth MCM submodule in the i-th deep feature extraction submodule is calculated by the fourth formula, which is: ij =P ij ×g h ×g w , f = δ(conv([Z h ,Z w ])), g h =σ(F h (f h )), g w =σ(F w (f w )), Among them, C ij is the output result of the coordinate attention module, F h is the feature information after horizontal separation f h The corresponding convolutional layer, F w is the feature information after vertical separation f w For the corresponding convolutional layer, δ is a nonlinear activation function; The deep feature map output by the jth MCM submodule in the i-th deep feature extraction submodule is calculated according to the fifth formula; the fifth formula is: Among them, F i-out is the deep feature map, F ij is the output result of the depth-separable convolution module of the jth MCM submodule in the i-th deep feature extraction submodule, F post-i The output result of the post-processing module of the j-th MCM submodule in the i-th deep feature extraction submodule; Each of the second to last deep feature extraction submodules further includes a preprocessing convolution module, which uses the preprocessing convolution module to increase the input channel of the current submodule to supplement the information contained in the deep feature map output by the current submodule; The preprocessing convolution module includes a convolution layer with a convolution kernel size of 1×1 and a Stride parameter of 1 and a batch normalization layer.

2. A fire detection method according to claim 1, It is characterized in that The multiple deep feature maps of different scales of the image to be detected include a deep feature map output by the third deep feature extraction submodule which is an eight-fold downsampled deep feature map, a deep feature map output by the fifth deep feature extraction submodule which is a sixteen-fold downsampled deep feature map, and a deep feature map output by the seventh deep feature extraction submodule which is a thirty-two-fold downsampled deep feature map. The fire detection model also includes a multi-scale feature fusion network, which includes a first channel and a second channel built based on a feature pyramid, wherein the first channel and the second channel respectively include sub-channels with eight times downsampling scales, sixteen times downsampling scales, and thirty-two times downsampling scales set in sequence, the connection direction of the first channel is from bottom to top, and the connection direction of the second channel is from top to bottom, and the sub-channels with the same sampling scale in the first channel and the second channel are horizontally connected; The method further comprises: The eight-fold downsampled deep feature map, the sixteen-fold downsampled deep feature map, and the thirty-two-fold downsampled deep feature map are respectively input into the first channel of the same sampling scale, and for the deep feature map of each sampling scale, the deep feature maps of the same sampling scale in the two channels are fused through the sub-channels after horizontal connection to obtain a fused feature map of the same sampling scale; Determining the category of the image to be detected and the location of the fire accident in the image to be detected by the target detection network according to the plurality of deep feature maps includes: The target detection network is used to determine the category of the image to be detected and the position of the fire accident in the image to be detected based on the fused feature map of the same sampling scale obtained from the deep feature map of each sampling scale.

3. A fire detection system, It is characterized in that including an image acquisition unit and a fire detection unit; The image acquisition unit is used to acquire the image to be detected; The fire detection unit is used to extract features from the image to be detected to obtain a plurality of deep feature maps of different scales; and determine the category of the image to be detected and the location of the fire accident in the image to be detected according to the plurality of deep feature maps, wherein the category includes the presence of a fire accident or the absence of a fire accident; The fire detection unit is specifically used for: The feature extraction is performed on the image to be detected to obtain a plurality of deep feature maps of different scales; the category of the image to be detected and the location of the fire accident in the image to be detected are determined according to the plurality of deep feature maps, wherein the category including the presence of a fire accident or the absence of a fire accident is realized by a pre-trained fire detection model, and the fire detection model includes a feature extraction network and a target detection network; The feature extraction of the image to be detected is performed to obtain a plurality of deep feature maps of different scales, including: By using the feature extraction network, feature extraction is performed on the image to be detected to obtain a plurality of deep feature maps of different scales; Determining the category of the image to be detected and the location of the fire accident in the image to be detected according to the plurality of deep feature maps, wherein the category includes the presence of a fire accident or the absence of a fire accident, comprises: Determining, by the target detection network, the category of the image to be detected and the location of the fire accident in the image to be detected according to the plurality of deep feature maps, wherein the category includes the presence of a fire accident or the absence of a fire accident; The feature extraction network includes a shallow feature extraction module and a deep feature extraction module. The feature extraction network is used to extract features from the image to be detected to obtain a plurality of deep feature maps of different scales, including: Extracting a shallow feature map of the image to be detected using the shallow feature extraction module; Extracting a plurality of deep feature maps of different scales corresponding to the shallow feature map using the deep feature extraction module; The shallow feature extraction module includes a convolution layer with a convolution kernel size of 3×3, a stride parameter of 2, and a padding parameter of 1, a batch normalization layer, and an activation function; The shallow feature extraction module extracts the shallow feature map of the image to be detected according to the first formula; the first formula is: F = σ (Bn (conv (X))), X∈R 3×H×W ; Where F is the shallow feature map, X is the image to be detected, X∈R 3×H×W , W is the width of the image to be detected, H is the height of the image to be detected, conv is the convolutional layer, B n is the batch normalization layer, σ is the activation function; The deep feature extraction module includes seven deep feature extraction submodules connected in sequence, and the deep feature extraction module is used to extract multiple deep feature maps of different scales corresponding to the shallow feature map, including: Inputting the shallow feature map into the first deep feature extraction submodule to obtain a deep feature map extracted by the first deep feature extraction submodule; The second deep feature extraction submodule is used as the current submodule, and the following steps are repeatedly performed until the next submodule of the current submodule is the last deep feature extraction submodule, and the third deep feature extraction submodule, the fifth deep feature extraction submodule and the last deep feature extraction submodule are used to extract deep feature maps of different scales of the image to be detected; The following steps include: Taking the deep feature map output by the previous deep feature extraction submodule of the current submodule as the input of the current submodule, and extracting the deep feature map output by the current submodule; Each of the deep feature extraction submodules includes a pixel attention module, a coordinate attention module and a post-processing module, wherein the deep feature map output by the previous deep feature extraction submodule of the current submodule is used as the input of the current submodule, and the deep feature map output by the current submodule is extracted, including: Utilize the pixel attention module to traverse each pixel in the input feature map of the current submodule, extract the color feature information of each pixel, and generate the color feature map of the current submodule according to each color feature information, wherein the input feature map is the deep feature map extracted by the previous deep feature extraction submodule of the current submodule; Utilizing the coordinate attention module, extracting spatial position information of each pixel in the color feature map of the current submodule, and generating a spatial position feature map of the current submodule according to the spatial position information of each pixel; Using the post-processing module, the spatial position feature map is fused with the input feature map of the current submodule to obtain a deep feature map output by the current submodule; Each of the deep feature extraction submodules further includes: a depthwise separable convolution module; the depthwise separable convolution module includes a 3×3 convolution kernel and a 5×5 convolution kernel, the batch normalization layer and the activation function; in each depthwise separable convolution module, the 3×3 convolution kernel and the 5×5 convolution kernel are used to extract features, and the extracted features are subjected to dimensionality reduction processing through the batch normalization layer and the activation function; The output result of the depth-separable convolution module in the jth MCM submodule in the i-th deep feature extraction submodule is calculated by the second formula; the second formula is: ij =σ(B n (conv(F))); where F ij is the output result of the depthwise separable convolution module, i is 0, 1, 2, ..., 6, j is 1, 2, 3, 4, B n is the batch normalization layer, σ is the activation function, and F is the shallow feature map of the image to be detected; The output result of the pixel attention module in the jth MCM submodule in the i-th deep feature extraction submodule is calculated by the third formula; the third formula is: Among them, P ij is the output result of the pixel attention module, F ij ∈R C×H×W Where C is the number of channels, W is the width of the image to be detected, H is the height of the image to be detected, and M is the width of the image to be detected. PA is the pixel attention module in the jth MCM submodule in the i-th deep feature extraction submodule; The output result of the coordinate attention module in the jth MCM submodule in the i-th deep feature extraction submodule is calculated by the fourth formula, which is: ij =P ij ×g h ×g w , f=δ(conv([Z h ,Z w ])), g h =σ(F h (f h )), g w =σ(F w (f w )), Among them, C ij is the output result of the coordinate attention module, F h is the feature information after horizontal separation f h The corresponding convolutional layer, F w is the feature information after vertical separation f w For the corresponding convolutional layer, δ is a nonlinear activation function; The deep feature map output by the jth MCM submodule in the i-th deep feature extraction submodule is calculated according to the fifth formula; the fifth formula is: Among them, F i-out is the deep feature map, F ij is the output result of the depth-separable convolution module of the jth MCM submodule in the i-th deep feature extraction submodule, F post-i The output result of the post-processing module of the j-th MCM submodule in the i-th deep feature extraction submodule; Each of the second to last deep feature extraction submodules further includes a preprocessing convolution module, which uses the preprocessing convolution module to increase the input channel of the current submodule to supplement the information contained in the deep feature map output by the current submodule; The preprocessing convolution module includes a convolution layer with a convolution kernel size of 1×1 and a Stride parameter of 1 and a batch normalization layer.

4. An electronic device, It is characterized in that The electronic device comprises a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the operation performed by a fire detection method as claimed in any one of claims 1 or 2.

5. A readable storage medium, It is characterized in that The readable storage medium stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the operation performed by the fire detection method according to any one of claims 1 or 2.

Images