A disaster smoke detection method and system based on deep convolutional neural network

Abstract

The present application discloses a disaster smoke detection method and system based on a deep convolutional neural network, which relates to the field of image processing technology. The method includes: performing a first convolution operation on an input image to extract features and generate a primary feature map; performing enhancement processing on the primary feature map to obtain an enhanced feature map; performing multi-scale fusion on the enhanced feature map according to a second convolution operation to obtain multiple feature maps of different scales as high-level feature maps; performing top-down and bottom-up feature fusion on each high-level feature map in scale according to a third convolution operation, correspondingly obtaining multiple top-down fused feature maps and multiple bottom-up fused feature maps; fusing each top-down fused feature map and bottom-up fused feature map to obtain multiple bidirectional cross feature maps; and performing disaster and smoke detection on each bidirectional cross feature map. Multiple convolution operations can more accurately extract the features of the input image, thereby improving the accuracy and robustness of target detection.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a disaster smoke detection method and system based on a deep convolutional neural network. Background Art

[0002] Flame hazard and smoke detection has important application value in the field of fire prevention and safety monitoring. Especially in early fire alarm systems, fast and accurate detection of flames and smoke is crucial. At present, traditional detection methods mainly rely on image processing technology based on low-level features such as color, shape, and motion. Traditional methods can provide relatively reliable detection results in simple backgrounds. However, when faced with complex backgrounds, significant changes in lighting conditions, or other interference, the detection accuracy and robustness of traditional methods often drop significantly. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose a disaster smoke detection method and system based on a deep convolutional neural network to improve the accuracy and robustness of the disaster smoke detection method.

[0004] To achieve the above object, an embodiment of the present application proposes a disaster smoke detection method based on a deep convolutional neural network, the method comprising the following steps:

[0005] Performing a first convolution operation on the input image to extract features and generate a primary feature map;

[0006] Performing enhancement processing on the primary feature map to obtain an enhanced feature map;

[0007] Performing multi-scale fusion on the enhanced feature map according to the second convolution operation to obtain multiple feature maps of different scales as high-level feature maps;

[0008] According to the third convolution operation, each of the high-level feature maps is subjected to top-down and bottom-up feature fusion in scale, and a plurality of top-down fused feature maps and a plurality of bottom-up fused feature maps are correspondingly obtained;

[0009] Fusing each of the top-down fusion feature maps and the bottom-up fusion feature maps to obtain a plurality of bidirectional cross feature maps;

[0010] Disaster and smoke detection is performed on each of the bidirectional cross-feature graphs.

[0011] In some embodiments, performing a first convolution operation on the input image to extract features and generate a primary feature map comprises the following steps:

[0012] Performing a convolution operation on the input image through a spatial and channel reconstruction convolution module to obtain a basic feature map;

[0013] The convolution operation expression of the spatial and channel reconstruction convolution module is:

[0014] ;

[0015] in, is the basic feature map; Represents spatial and channel reconstruction convolution operations; Represents the input image The convolution operation with weight 𝑊, the convolution kernel size is , the stride 𝑠 is automatically filled with 𝑝 to adjust the size of the output feature map; Used to handle space redundancy, Used to handle channel redundancy; represents the batch normalization operation, is the activation function;

[0016] Performing a convolution operation on the basic feature map through a residual block to obtain the primary feature map;

[0017] The convolution operation expression of the residual block is:

[0018] ;

[0019] The residual connection expression of the residual block is:

[0020] ;

[0021] in, and represents two consecutive spatial and channel reconstruction convolution operations, represents the feature map after convolution, is the primary feature map.

[0022] In some embodiments, the step of performing enhancement processing on the primary feature map to obtain an enhanced feature map comprises the following steps:

[0023] Generate a weight matrix corresponding to the primary feature map through a convolution operation of an attention mechanism, and multiply the weight matrix by the primary feature map element by element to obtain a first intermediate feature map;

[0024] The convolution operation expression of the attention mechanism is:

[0025] ;

[0026] in, is the weight matrix, represents the convolution operation for generating the weight matrix, and the Sigmoid function is used to normalize the output; the first intermediate feature map is , ;

[0027] Generating an offset of a deformable convolution layer of the first intermediate feature map through a local convolution operation, and performing a deformable convolution operation according to the offset of the deformable convolution layer and the first intermediate feature map to obtain a second intermediate feature map;

[0028] The expression of the offset of the deformable convolution layer is:

[0029] ;

[0030] The expression of the deformable convolution operation is:

[0031] ;

[0032] in, is the offset of the deformable convolutional layer, Represents the convolution operation of an independent convolutional layer; represents the deformable convolution operation, is the second intermediate feature map;

[0033] Performing Gabor filtering on the second intermediate feature map to obtain a filtered feature map;

[0034] The expression of the Gabor filtering process is:

[0035] ;

[0036] Among them, 𝜎 is used to control the width of the Gaussian function, 𝛾 is used to control the aspect ratio of the elliptical Gaussian distribution, 𝜆 is the wavelength, and 𝜓 is the phase shift; is the pixel coordinate;

[0037] The brightness and contrast of the filtered feature map are adjusted to obtain the enhanced feature map.

[0038] In some embodiments, performing multi-scale fusion on the enhanced feature map according to the second convolution operation to obtain multiple feature maps of different scales as high-level feature maps includes the following steps:

[0039] Performing maximum pooling operations of different scales on the enhanced feature map to obtain multiple maximum pooling feature maps;

[0040] Splicing each of the maximum pooling feature maps, and then inputting the spliced ​​feature maps into the target convolution layer for fusion to obtain a multi-scale feature map;

[0041] The features of the target area in the multi-scale feature map are enhanced by an attention mechanism to obtain the high-level feature map.

[0042] In some embodiments, the performing of top-down and bottom-up feature fusion on each of the high-level feature maps in scale according to the third convolution operation, and correspondingly obtaining a plurality of top-down fused feature maps and a plurality of bottom-up fused feature maps, comprises the following steps:

[0043] The number of channels of each of the high-level feature maps is converted into a consistent dimension through a 1×1 convolution operation to obtain a plurality of converted feature maps;

[0044] Starting from the conversion feature map at the highest scale layer, each conversion feature map is propagated downward layer by layer through interpolation upsampling, and weighted fused with the conversion feature map at the next layer to obtain a first fused feature map;

[0045] The expression of the first fusion feature map is:

[0046] ;

[0047] in, is the first fusion feature map, is the conversion feature map, represents the upsampling operation, is the feature map after depth-wise separable convolution, and is a learnable first fusion weight, wherein the first fusion weight is restricted to be non-negative by a ReLU function;

[0048] Starting from the conversion feature map at the lowest level, each conversion feature map is propagated upward layer by layer through downsampling, and weighted fused with the conversion feature map at the previous level to obtain a second fused feature map;

[0049] The expression of the second fusion feature map is:

[0050] ;

[0051] in, is the second fusion feature map, represents the downsampling operation, is the feature map after depth-wise separable convolution, , and is the learnable second fusion weight;

[0052] Adjusting the first fusion weight and the second fusion weight according to the minimization loss function through a back propagation algorithm, and generating the top-down fusion feature map using the adjusted first fusion weight, and generating the bottom-up fusion feature map using the adjusted second fusion weight;

[0053] The expression of the minimization loss function is:

[0054] ;

[0055] The expression of the top-down fusion feature map is:

[0056] ;

[0057] The expression of the bottom-up fusion feature map is:

[0058] ;

[0059] in, is the minimization loss function, is the top-down fusion feature map, is the bottom-up fusion feature map, A level number representing the conversion feature map;

[0060] The fusing of the top-down fusion feature maps and the bottom-up fusion feature maps to obtain a plurality of bidirectional cross feature maps comprises the following steps:

[0061] Determine an optimized fusion weight according to the adjusted first fusion weight and the adjusted second fusion weight;

[0062] fusing the top-down fusion feature maps and the bottom-up fusion feature maps according to the optimized fusion weights to obtain a plurality of the bidirectional cross feature maps;

[0063] The expression of the bidirectional cross feature map is:

[0064] ;

[0065] in, is the bidirectional cross feature map, is the optimized fusion weight.

[0066] In some embodiments, the performing disaster and smoke detection on each of the bidirectional cross feature graphs comprises the following steps:

[0067] Identify the category of each target in each of the bidirectional cross feature maps through classification convolution operations to obtain a classification result; wherein the targets include the disaster and smoke;

[0068] The expression of the classification result is:

[0069] ;

[0070] in, is the classification result, For the classification convolution operation, represents the batch normalization operation, is the activation function, is the bidirectional cross feature map;

[0071] Determine the probability of each of the targets according to the classification results, and generate the confidence level of each of the probabilities;

[0072] The probability distribution of each of the targets is:

[0073] ;

[0074] in, is the probability distribution, is the probability, is the activation function;

[0075] Outputting the bounding box parameters of each of the targets in each of the bidirectional cross feature maps through a regression convolution operation;

[0076] The expression of the bounding box parameter is:

[0077] ;

[0078] in, are the bounding box parameters, represents the regressive convolution operation;

[0079] Determining the probability of each of the targets, the confidence of each of the probabilities, and the bounding box parameters of each of the targets as the detection results of the disaster and smoke;

[0080] The expression of the detection result is:

[0081] ;

[0082] in, is the test result; is the confidence level.

[0083] In some embodiments, the method further comprises the following steps:

[0084] Optimizing the bounding box parameters using distributed focal loss to obtain optimized bounding box parameters;

[0085] The expression of the optimized bounding box parameters is:

[0086] ;

[0087] in, are the optimized bounding box parameters, represents the distributed focal loss;

[0088] updating the detection result according to the optimized bounding box parameters;

[0089] The updated detection result is expressed as:

[0090] .

[0091] To achieve the above objectives, another aspect of the embodiments of the present application provides a disaster smoke detection system based on a deep convolutional neural network, the system comprising:

[0092] A feature extraction unit, used for performing a first convolution operation on the input image to extract features and generate a primary feature map;

[0093] A feature enhancement unit, used for performing enhancement processing on the primary feature map to obtain an enhanced feature map;

[0094] A first feature fusion unit, configured to perform multi-scale fusion on the enhanced feature map according to the second convolution operation to obtain a plurality of feature maps of different scales as high-level feature maps;

[0095] A second feature fusion unit is used to perform top-down and bottom-up feature fusion on each of the high-level feature maps in scale according to the third convolution operation, and correspondingly obtain a plurality of top-down fused feature maps and a plurality of bottom-up fused feature maps;

[0096] A bidirectional cross fusion unit, used for fusing each of the top-down fusion feature maps and the bottom-up fusion feature maps to obtain a plurality of bidirectional cross feature maps;

[0097] The target detection unit is used to perform disaster and smoke detection on each of the bidirectional cross feature maps.

[0098] To achieve the above objective, another aspect of an embodiment of the present application provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above method when executing the computer program.

[0099] To achieve the above objective, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0100] The embodiments of the present application include at least the following beneficial effects:

[0101] The present application can perform a first convolution operation on the input image to extract features and generate a primary feature map; perform enhancement processing on the primary feature map to obtain an enhanced feature map; perform multi-scale fusion on the enhanced feature map according to the second convolution operation to obtain multiple feature maps of different scales as high-level feature maps; perform top-down and bottom-up feature fusion on each high-level feature map in scale according to the third convolution operation, and obtain multiple top-down fused feature maps and multiple bottom-up fused feature maps accordingly; fuse each top-down fused feature map and bottom-up fused feature map to obtain multiple bidirectional cross feature maps; perform disaster and smoke detection on each bidirectional cross feature map. By implementing deep convolution through multiple convolution operations, the features of the input image can be extracted more accurately, and each top-down fused feature map and bottom-up fused feature map can be fused to realize bidirectional cross fusion features, and further obtain more accurate image features, which overcomes the problems of insufficient feature extraction and insufficient multi-scale feature fusion in related technologies, and then performs disaster and smoke detection based on the accurate and fully fused features of the bidirectional cross feature map, which can improve accuracy and robustness, and overcome the problem of poor adaptability of related technologies in complex environment detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0103] Figure 1 A schematic diagram of a flow chart of a disaster smoke detection method based on a deep convolutional neural network provided in an embodiment of the present application;

[0104] Figure 2 A reference flow chart of a disaster smoke detection method based on a deep convolutional neural network provided in an embodiment of the present application;

[0105] Figure 3 Provided for the embodiments of this application Figure 2 Reference flow chart of S1~S2;

[0106] Figure 4 Provided for the embodiments of this application Figure 2 Reference flow chart of S3 in ;

[0107] Figure 5 A schematic diagram of the structure of a disaster smoke detection system based on a deep convolutional neural network provided in an embodiment of the present application;

[0108] Figure 6 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0109] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of systems and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0110] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0111] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0112] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0113] Before describing the embodiments of the present application in detail, some related technologies involved in the embodiments of the present application are first described as follows:

[0114] Flame hazard and smoke detection has important application value in the field of fire prevention and safety monitoring. Especially in early fire alarm systems, fast and accurate detection of flames and smoke is crucial. At present, traditional detection methods mainly rely on image processing technology based on low-level features such as color, shape, and motion. Traditional methods can provide relatively reliable detection results in simple backgrounds. However, when faced with complex backgrounds, significant changes in lighting conditions, or other interference, the detection accuracy and robustness of traditional methods often drop significantly.

[0115] In recent years, with the rapid development of deep learning technology, especially the widespread application of convolutional neural networks, target detection technology has made significant progress. The detection model based on deep convolutional neural networks has shown high detection accuracy and real-time performance in fire disaster and smoke detection. The deep convolutional neural network model of related technologies can automatically learn the high-level features of flames and smoke, overcome the dependence of traditional methods on manual feature extraction, and make the detection process more intelligent and efficient. However, although the detection method based on deep convolutional neural networks has shown certain advantages in flame and smoke detection, there are still many challenges in feature extraction, feature fusion and adaptability to complex scenes.

[0116] First, flames and smoke have irregular shapes and dynamically changing characteristics. Traditional convolutional neural networks often have limitations when extracting the above complex features, and it is difficult to accurately identify targets in complex backgrounds. Secondly, due to the large range of scale variations of flames and smoke, existing models often cannot fully utilize feature information of different scales when processing multi-scale features, resulting in unstable detection effects. Finally, in low-light and low-contrast scenes, the detection performance of existing models usually decreases significantly, affecting the early detection capabilities of flames and smoke. Therefore, how to design a deep convolutional neural network method that can more effectively extract flame and smoke features and adapt to complex environments is a problem that needs to be solved urgently. This application aims to improve the accuracy and robustness of flame and smoke detection by improving the feature extraction framework to adapt to application scenarios such as fire warning and safety monitoring.

[0117] Therefore, the embodiment of the present application provides a disaster smoke detection method and system based on a deep convolutional neural network. The technical solution of the present application includes: performing a first convolution operation on the input image to extract features and generate a primary feature map; performing enhancement processing on the primary feature map to obtain an enhanced feature map; performing multi-scale fusion on the enhanced feature map according to the second convolution operation to obtain multiple feature maps of different scales as high-level feature maps; performing top-down and bottom-up feature fusion on each high-level feature map in scale according to the third convolution operation, correspondingly obtaining multiple top-down fused feature maps and multiple bottom-up fused feature maps; fusing each top-down fused feature map and bottom-up fused feature map to obtain multiple bidirectional cross feature maps; performing disaster and smoke detection on each bidirectional cross feature map. By implementing deep convolution through multiple convolution operations, the features of the input image can be extracted more accurately, and the top-down fusion feature maps and the bottom-up fusion feature maps can be fused to realize bidirectional cross-fusion features, so as to further obtain more precise image features, thus overcoming the problems of insufficient feature extraction and insufficient multi-scale feature fusion in related technologies. Disaster and smoke detection can be performed based on the accurate and fully fused features of the bidirectional cross-feature map, which can improve accuracy and robustness and overcome the problem of poor adaptability of related technologies in complex environment detection.

[0118] The embodiment of the present application provides a disaster smoke detection method based on a deep convolutional neural network, which relates to the field of image processing technology. The disaster smoke detection method based on a deep convolutional neural network provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements a disaster smoke detection method based on a deep convolutional neural network, etc., but is not limited to the above forms.

[0119] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0120] Reference Figure 1 The embodiment of the present application provides a disaster smoke detection method based on a deep convolutional neural network. The method may include but is not limited to S100 to S150, which are as follows:

[0121] S100: Perform a first convolution operation on the input image to extract features and generate a primary feature map.

[0122] It can be understood that the input image of this embodiment can be any image, and in order to efficiently realize disaster and smoke detection, this embodiment can select images of key areas outdoors or indoors where fires are prone to occur, for example, by acquiring images of the above key areas in real time through monitoring equipment as input images.

[0123] Furthermore, S100 may include S101-S102:

[0124] S101: performing a convolution operation on the input image through a spatial and channel reconstruction convolution module to obtain a basic feature map;

[0125] The convolution operation expression of the spatial and channel reconstruction convolution module is:

[0126] ;

[0127] in, is the basic feature map; Represents spatial and channel reconstruction convolution operations; Represents the input image The convolution operation with weight 𝑊, the convolution kernel size is , the stride 𝑠 is automatically filled with 𝑝 to adjust the size of the output feature map; Used to handle space redundancy, Used to handle channel redundancy; represents the batch normalization operation, is the activation function;

[0128] S102: performing a convolution operation on the basic feature map through a residual block to obtain the primary feature map;

[0129] The convolution operation expression of the residual block is:

[0130] ;

[0131] The residual connection expression of the residual block is:

[0132] ;

[0133] in, and represents two consecutive spatial and channel reconstruction convolution operations, represents the feature map after convolution, is the primary feature map.

[0134] S110: Performing enhancement processing on the primary feature map to obtain an enhanced feature map.

[0135] Specifically, this embodiment can perform enhancement processing on the primary feature map to extract features with higher levels and richer information.

[0136] Furthermore, S110 may include S111 to S114:

[0137] S111: Generate a weight matrix corresponding to the primary feature map through a convolution operation of an attention mechanism, and multiply the weight matrix by the primary feature map element by element to obtain a first intermediate feature map;

[0138] The convolution operation expression of the attention mechanism is:

[0139] ;

[0140] in, is the weight matrix, represents the convolution operation for generating the weight matrix, and the Sigmoid function is used to normalize the output; the first intermediate feature map is , ;

[0141] S112: generating an offset of a deformable convolution layer of the first intermediate feature map through a local convolution operation, and performing a deformable convolution operation according to the offset of the deformable convolution layer and the first intermediate feature map to obtain a second intermediate feature map;

[0142] The expression of the offset of the deformable convolution layer is:

[0143] ;

[0144] The expression of the deformable convolution operation is:

[0145] ;

[0146] in, is the offset of the deformable convolutional layer, Represents the convolution operation of an independent convolutional layer; represents the deformable convolution operation, is the second intermediate feature map;

[0147] S113: performing Gabor filtering on the second intermediate feature map to obtain a filtered feature map;

[0148] The expression of the Gabor filtering process is:

[0149] ;

[0150] Among them, 𝜎 is used to control the width of the Gaussian function, 𝛾 is used to control the aspect ratio of the elliptical Gaussian distribution, 𝜆 is the wavelength, and 𝜓 is the phase shift; is the pixel coordinate;

[0151] S114: Adjust the brightness and contrast of the filtered feature map to obtain the enhanced feature map.

[0152] S120: Perform multi-scale fusion on the enhanced feature map according to the second convolution operation to obtain multiple feature maps of different scales as high-level feature maps.

[0153] Specifically, a convolution operation is performed on the enhanced feature map to generate multiple images of different scales and fuse them to obtain a high-level feature map.

[0154] Furthermore, S120 may include S121 to S123:

[0155] S121: performing maximum pooling operations of different scales on the enhanced feature map to obtain multiple maximum pooling feature maps;

[0156] S122: splicing each of the maximum pooling feature maps, and then inputting the spliced ​​feature maps into the target convolution layer for fusion to obtain a multi-scale feature map;

[0157] S123: Enhance the features of the target area in the multi-scale feature map through an attention mechanism to obtain the high-level feature map.

[0158] S130: performing top-down and bottom-up feature fusion on each of the high-level feature maps in scale according to the third convolution operation, and correspondingly obtaining a plurality of top-down fused feature maps and a plurality of bottom-up fused feature maps.

[0159] It can be understood that high-level feature maps of different scales can be sorted by scale and then subjected to top-down and bottom-up feature fusion respectively.

[0160] Further, S130 may include S131 to S134:

[0161] S131: converting the number of channels of each of the high-level feature maps into a consistent dimension through a 1×1 convolution operation to obtain a plurality of converted feature maps;

[0162] S132: Starting from the conversion feature map at the highest scale layer, propagate each conversion feature map downward layer by layer through interpolation upsampling, and perform weighted fusion with the conversion feature map at the next layer to obtain a first fused feature map;

[0163] The expression of the first fusion feature map is:

[0164] ;

[0165] in, is the first fusion feature map, is the conversion feature map, represents the upsampling operation, is the feature map after depth-wise separable convolution, and is a learnable first fusion weight, wherein the first fusion weight is restricted to be non-negative by a ReLU function;

[0166] S133: Starting from the conversion feature map at the lowest scale layer, propagate each conversion feature map upward layer by layer through downsampling, and perform weighted fusion with the conversion feature map of the previous layer to obtain a second fused feature map;

[0167] The expression of the second fusion feature map is:

[0168] ;

[0169] in, is the second fusion feature map, represents the downsampling operation, is the feature map after depth-wise separable convolution, , and is the learnable second fusion weight;

[0170] S134: adjusting the first fusion weight and the second fusion weight according to the minimization loss function through a back propagation algorithm, and generating the top-down fusion feature map using the adjusted first fusion weight, and generating the bottom-up fusion feature map using the adjusted second fusion weight;

[0171] The expression of the minimization loss function is:

[0172] ;

[0173] The expression of the top-down fusion feature map is:

[0174] ;

[0175] The expression of the bottom-up fusion feature map is:

[0176] ;

[0177] in, is the minimization loss function, is the top-down fusion feature map, is the bottom-up fusion feature map, Indicates the level number of the conversion feature map.

[0178] S140: Fusing each of the top-down fusion feature maps and the bottom-up fusion feature maps to obtain a plurality of bidirectional cross feature maps.

[0179] Specifically, each top-down fusion feature map and the bottom-up fusion feature map are bidirectionally cross-fused to fully fuse the features of the input features.

[0180] Further, S140 may include S141-S142:

[0181] S141: Determine an optimized fusion weight according to the adjusted first fusion weight and the adjusted second fusion weight;

[0182] S142: fusing the top-down fusion feature maps and the bottom-up fusion feature maps according to the optimized fusion weight to obtain a plurality of bidirectional cross feature maps;

[0183] The expression of the bidirectional cross feature map is:

[0184] ;

[0185] in, is the bidirectional cross feature map, is the optimized fusion weight.

[0186] S150: Perform disaster and smoke detection on each of the two-way cross feature graphs.

[0187] Specifically, this embodiment can perform target detection on the bidirectional cross feature map that fully integrates the features of each scale, so as to detect disasters and smoke in the bidirectional cross feature map.

[0188] Further, S150 may include S151-S154:

[0189] S151: Identify the category of each target in each of the bidirectional cross feature maps through a classification convolution operation to obtain a classification result; wherein the targets include the disaster and smoke;

[0190] The expression of the classification result is:

[0191] ;

[0192] in, is the classification result, For the classification convolution operation, represents the batch normalization operation, is the activation function, is the bidirectional cross feature map;

[0193] S152: Determine the probability of each of the targets according to the classification result, and generate the confidence level of each of the probabilities;

[0194] The probability distribution of each of the targets is:

[0195] ;

[0196] in, is the probability distribution, is the probability, is the activation function;

[0197] S153: Outputting the bounding box parameters of each target in each bidirectional cross feature map through a regression convolution operation;

[0198] The expression of the bounding box parameter is:

[0199] ;

[0200] in, are the bounding box parameters, represents the regressive convolution operation;

[0201] S154: Determine the probability of each target, the confidence of each probability, and the bounding box parameter of each target as the detection result of the disaster and smoke;

[0202] The expression of the detection result is:

[0203] ;

[0204] in, is the test result; is the confidence level.

[0205] To further improve the accuracy of the bounding box, the embodiment of the present application may further include S155-S156:

[0206] S155: Optimizing the bounding box parameters using distributed focal loss to obtain optimized bounding box parameters;

[0207] The expression of the optimized bounding box parameters is:

[0208] ;

[0209] in, are the optimized bounding box parameters, represents the distributed focal loss;

[0210] S156: updating the detection result according to the optimized bounding box parameters;

[0211] The updated detection result is expressed as:

[0212] .

[0213] The goal of the distributed focal loss is to generate more accurately located bounding boxes by modeling the distribution of bounding boxes.

[0214] Next, the solution of the embodiment of the present application will be introduced and explained in detail with reference to specific application examples.

[0215] Specifically, refer to Figure 2 , this embodiment may include S1~S4:

[0216] S1. Feature extraction and enhancement: Perform preliminary feature extraction on the input image to generate a basic feature map. Further extract primary features through multiple convolutional layers, residual structures and attention mechanisms, add flame and smoke feature enhancement modules to improve the primary feature map's ability to recognize flames and smoke, and obtain an enhanced feature map to capture the shape and complex patterns of objects.

[0217] S2, Feature Fusion and Advanced Processing: Through deeper convolutional layers and feature fusion modules, enhanced features of different scales are fused at multiple scales, while maintaining the complementarity and enhancement of features at different levels. Finally, by further processing the enhanced feature map, multiple advanced feature maps are output to be passed to the bidirectional cross feature fusion module.

[0218] S3. Bidirectional cross feature fusion: The bidirectional cross feature fusion module receives multi-scale feature maps (i.e., high-level feature maps) from the backbone network, adopts top-down and bottom-up bidirectional cross feature fusion structures, dynamically adjusts weight parameters, and obtains bidirectional cross feature maps to enhance the expressiveness of high-level feature maps and provide more expressive feature representations for subsequent detection tasks.

[0219] S4. Classification and regression output of the detection head: The detection head module processes the multi-scale feature map (i.e., bidirectional cross feature map) transmitted by the bidirectional cross feature fusion module and divides it into two branches: classification and regression. The classification branch is used to determine the target category, and the regression branch is used to predict the position and size of the target's bounding box. Finally, the classification and regression results are integrated to output the target's category, confidence, and bounding box for flame and smoke detection tasks.

[0220] Reference Figure 3 , this embodiment provides a reference flow chart of the above S1~S2.

[0221] In the feature extraction and enhancement stage of S1, the preliminary feature extraction module first extracts low-level features of the input image and generates a basic feature map through convolution operations. The basic feature map mainly captures the basic edge and simple shape information in the image. Based on this, the network can build a preliminary understanding of the image from the most primitive pixel level. In order to further improve the recognition ability of targets such as flames and smoke, the system then introduces multiple convolutional layers and residual structures to gradually extract more complex intermediate features. The residual structure alleviates the gradient vanishing problem caused by the increase in network depth through jump connections, thereby ensuring that the network can learn effective intermediate features. In this process, the attention mechanism is used to dynamically adjust the importance of different areas in the feature map, so that the network can pay more attention to key areas such as flames and smoke. In addition, the feature enhancement module is specifically optimized for the unique features of flames and smoke, further improving the expression ability of high-level features, so that the network can still accurately identify the shape and complex patterns of flames and smoke in complex backgrounds. Specifically, S1 includes the following steps:

[0222] S11. In the flame and smoke feature detection task, the preliminary feature extraction stage is responsible for extracting basic features from the input image. Input image is represented as a 4D tensor , where 𝑁 is the batch size, 𝐻 is the image height, 𝑊 is the image width, and 𝐶 is the number of channels. The input image is first convolved through the spatial and channel reconstruction convolution module, where the spatial and channel reconstruction convolution module contains the spatial redundancy suppression unit (SRU) and the channel redundancy suppression unit (CRU) to simultaneously process spatial and channel redundancy. The expression of the convolution operation of the spatial and channel reconstruction convolution module is:

[0223] ;

[0224] in, Represents the input image The convolution operation with weight 𝑊, the convolution kernel size is , the stride 𝑠 is automatically filled with 𝑝 to determine the size of the output feature map, Used to handle space redundancy, Then process channel redundancy. For group batch normalization operations, is the SiLU activation function. After this step, the generated basic feature map ,in , , which means that after the initial feature extraction, the resolution of the input image is halved while retaining low-level feature information such as edges and simple shapes.

[0225] S12. In the further feature extraction stage, the basic feature map It is further input to the Dark2 module, which consists of multiple spatial and channel reconstruction convolutional layers and residual structures to extract more mid-level features. These features can better represent complex textures, edge combinations, and local patterns.

[0226] The core idea of ​​the residual structure is to add the input directly to the output of the convolutional layer through a shortcut connection, thereby alleviating the common gradient vanishing problem in deep neural networks. The convolution operation of the residual block can be expressed as:

[0227] ;

[0228] The expression of residual connection is as follows:

[0229] ;

[0230] in, and are two consecutive spatial and channel reconstruction convolution operations, Represents the feature map after convolution, which is further optimized by SRU and CRU modules. Through residual connection, the network can maintain the transmission of input information, ensuring that important information in the feature map will not be lost or weakened due to multiple convolution operations.

[0231] In addition, this embodiment also introduces a coordinate attention mechanism. The core of the attention mechanism is to generate a weight matrix 𝐴, which is used to adjust the importance of each feature channel, thereby enhancing the network's attention to key areas. The specific calculation steps of the attention mechanism are as follows:

[0232] ;

[0233] in, is the convolutional layer that generates attention weights, and the Sigmoid function is used to normalize the output to the range of [0, 1]. The obtained weight matrix 𝐴 is consistent with the primary feature map Perform element-by-element multiplication to obtain the enhanced feature map, that is, the first intermediate feature map , through this operation, the characteristic expression of important areas in the image can be better captured.

[0234] S13. In the further feature extraction stage, the flame and smoke feature enhancement module is introduced to further improve the network's sensitivity to flame and smoke features. First, the first intermediate feature map is enhanced by local convolution operation to improve its ability to express local information. Next, deformable convolution is used to capture the geometric changes of the first intermediate image. The offset of the deformable convolution layer It is generated by a separate convolutional layer:

[0235] ;

[0236] Subsequently, the second mid-level feature map is generated through a deformable convolution operation:

[0237] ;

[0238] In addition, in order to better identify the characteristics of flames and smoke, the module also introduces a Gabor filter layer to filter the noise in the second intermediate image to extract texture information and obtain a filtered feature map. The mathematical expression of Gabor filtering is as follows:

[0239] ;

[0240] Among them, the parameter 𝜎 controls the width of the Gaussian function, 𝛾 controls the aspect ratio of the elliptical Gaussian, 𝜆 represents the wavelength, and 𝜓 is the phase shift; is the pixel coordinate. The filter can effectively extract the directional texture features in the image, thereby enhancing the detectability of flames and smoke.

[0241] The brightness and contrast of the filtered feature map are further adjusted through the brightness mask layer and the contrast enhancement layer to make the flame and smoke features more prominent, and finally the enhanced feature map is obtained. It can be better used in subsequent detection tasks.

[0242] In the feature fusion and high-level processing stage of S2, the hierarchy of the feature map is first further explored through deeper convolutional layers. The multi-scale feature fusion module plays a key role in this stage. By effectively integrating feature maps of different scales, it ensures that the network has higher flexibility in dealing with the diversity and complexity of flames and smoke. Specifically, the feature fusion module fuses feature maps of different scales at multiple levels through spatial pooling and convolution operations, so that low-level detail information and high-level semantic information can complement each other. This multi-scale fusion not only retains the feature information at each level, but also further enhances the expressiveness of the feature map and improves the robustness of the overall feature map. After this stage of processing, the output high-level feature map not only contains rich semantic information, but also has higher spatial resolution and recognition ability, laying a solid foundation for the subsequent two-way cross feature fusion module processing. Specifically, S2 includes the following steps:

[0243] S21. The network further extracts high-level features through deeper convolutional layers and feature fusion modules to capture information at more scales. The feature maps are fused at multiple scales through maximum pooling operations at different scales. The specific operations are as follows:

[0244] ;

[0245] The above multi-scale feature map , , After being spliced ​​together, they are input into the final convolutional layer for fusion, and the output multi-scale feature map is This operation enables the effective fusion of features of different scales, providing a more expressive feature representation for the subsequent detection module.

[0246] S22. Introduce the coordinate attention mechanism to enhance feature selectivity. Through the attention mechanism, the network can better identify and amplify the features of target areas such as flames and smoke in the multi-scale feature map. The final output high-level feature map is represented as ,The high-level feature maps will be passed to the subsequent bidirectional cross feature fusion module for further object detection.

[0247] Reference Figure 4 , this embodiment provides a reference flow chart of the above S3.

[0248] In the bidirectional cross-feature fusion stage of S3, a bidirectional cross-feature fusion structure is adopted. The module transitions from the multi-scale fusion of feature maps to the bidirectional fusion processing stage. The bidirectional cross-feature fusion module transmits high-level semantic information layer by layer to the lower-level feature maps through a top-down feature propagation path, so that the feature maps of each scale contain rich semantic information. At the same time, the bottom-up feature convergence path integrates low-level spatial details into high-level feature maps layer by layer, thereby enhancing the spatial detail expression ability of the feature maps. Through the dynamic interaction of these two fusion paths, the bidirectional cross-feature fusion module can effectively adjust the weight distribution of each layer of feature maps in the fusion process, so that the contribution of feature maps of different scales during fusion is optimal. Through this dynamic weight adjustment mechanism, the bidirectional cross-feature fusion module significantly improves the stability and accuracy of the features while ensuring the diversity and expression ability of the feature maps, providing more accurate and multi-dimensional feature representation for subsequent detection tasks. Specifically, S3 includes the following steps:

[0249] S31, in the initial stage of the bidirectional cross feature fusion module, first receive the multi-scale feature map from the backbone network, that is, the high-level feature map ,in , 𝑖 represents the feature maps extracted by different depth network layers. Each high-level feature map covers from low-level edges and textures to high-level complex semantic information. For example, low-level features It captures the basic information in the input image, such as edges and textures, while high-level features It contains more complex semantic information, such as the shape and structure of flames and smoke. In order to unify the feature representations of different scales, the bidirectional cross feature fusion module first uses a 1×1 convolution operation to convert the number of channels of each high-level feature map into a consistent dimension to obtain a converted feature map, thereby reducing the computational complexity and enhancing the fusion effect. This process can be expressed as:

[0250] ;

[0251] in, is the conversion feature map, Indicates the number of unified channels. This step lays the foundation for the subsequent bidirectional cross feature fusion.

[0252] S32, Bidirectional cross feature fusion uses top-down and bottom-up fusion strategies to effectively combine transformed feature maps of different scales. In the top-down path, from the highest level feature map At the beginning, the conversion feature map is propagated down layer by layer through interpolation upsampling and combined with the conversion feature map of the next layer Perform weighted fusion to obtain the first fused feature map. The weighted feature fusion expression is as follows:

[0253] ;

[0254] in, is the first fusion feature map, represents the upsampling operation, is the feature map after depth-wise separable convolution, and It is the first learnable fusion weight, which limits the non-negativity of the weight through the ReLU function to improve the stability of feature fusion.

[0255] In the bottom-up path, the bidirectional cross-feature fusion module transforms the feature map from the lowest layer At the beginning, the conversion feature map is propagated upward layer by layer through downsampling and combined with the conversion feature map of the upper layer After fusion, the second fused feature map is obtained. The weighted feature fusion expression is as follows:

[0256] ;

[0257] in, is the second fusion feature map, represents the downsampling operation, is the feature map after depth-wise separable convolution, , and is the learnable second fusion weight.

[0258] The bidirectional fusion structure in this embodiment enables the information of the transformed feature maps to be complementary and enhanced at multiple scales. In this way, the bidirectional cross feature fusion module can integrate information at different scales, thereby achieving more effective fusion between multi-scale features.

[0259] S33. In the process of feature fusion, in order to better balance the contribution of features at each scale, the bidirectional cross feature fusion module introduces a dynamic weight adjustment mechanism. This mechanism sets a learnable fusion weight and , so that the network can automatically adjust the fusion weights of each feature map according to specific task requirements during training. The goal of adjusting the fusion weights is to optimize the result of feature fusion so that the contribution of each feature map during fusion is optimal, thereby maximizing the expressive power of the feature map. In the fusion of the top-down path, the feature map output after the fusion weights are adjusted is the top-down fusion feature map. ,in:

[0260] ;

[0261] In the fusion of the bottom-up path, the adjusted output feature map is the bottom-up fusion feature map ,in:

[0262] ;

[0263] Each fusion weight can be optimized by the back propagation algorithm during the backbone network training process to minimize the loss function 𝐿 for constraint:

[0264] ;

[0265] During the training process, the fusion weights and It will be adjusted continuously to make the contribution of each fused feature map reach the optimal state during fusion, thereby improving the overall feature expression ability.

[0266] S34, the feature map after bidirectional cross feature fusion and dynamic weight adjustment is integrated into a bidirectional cross feature map in the bidirectional cross feature fusion module, denoted as The bidirectional cross feature map combines the advantages of features at different scales and has stronger expressive power. The bidirectional cross feature map can be expressed as:

[0267] ;

[0268] in, It is the further optimized fusion weight, which is used to fully integrate and enhance multi-scale features. Bidirectional cross feature map It not only combines the advantages of features of different scales, but also has stronger expression ability, and can more effectively support the target detection head for flame and smoke detection tasks. The feature map obtained after processing in this embodiment is particularly suitable for the detection of complex scenes and multi-scale targets, and can significantly improve the accuracy and robustness of detection.

[0269] In the classification and regression output stage of the detection head of S4, the bidirectional cross feature map from the bidirectional cross feature fusion module is received, and the bidirectional cross feature map is sent to two parallel branches of classification and regression respectively. The classification branch first processes the bidirectional cross feature map through a series of convolutional layers to extract information related to the target category, and then outputs the probability distribution of each category through the Softmax activation function to determine the category attribute of the target. At the same time, the regression branch focuses on the spatial positioning of the target, extracts the bounding box information of the target through the convolutional layer, and predicts the position, width and height of the target. In order to improve the effectiveness of the classification and regression tasks, this embodiment can adopt an optimized loss function, especially for the detection scenes of small targets and complex backgrounds. Finally, the detection head module integrates the results of the classification branch and the regression branch, and outputs the category, confidence and bounding box information of each target. Through the above integration process, it is possible to provide accurate and reliable detection results in complex flame and smoke detection tasks, and improve the efficient performance in application scenarios such as fire warning and safety monitoring. Specifically, S4 includes the following steps:

[0270] S41, the detection head module starts with receiving the bidirectional cross feature map from the bidirectional cross feature fusion module , where 𝑖 represents the scale level of the bidirectional cross feature map. Bidirectional cross feature map It contains the feature information of flame and smoke targets extracted at different scales. By designing different scale levels, the detection head can capture flame and smoke target information from small to large, from details to the whole picture. The diversity and richness of the bidirectional cross feature map ensure that the subsequent classification and regression tasks can be performed on high-quality input, thereby achieving high-precision target detection.

[0271] S42. After receiving the bidirectional cross feature map, the detection head module further processes the bidirectional cross feature map through a series of convolution operations and divides it into two parallel branches: a classification branch and a regression branch. The task of the classification branch is to determine the probability of the target category. ,in Represents a bidirectional cross feature map The regression branch focuses on predicting the bounding box parameters of the target , including the center coordinates, width and height of the target. In each branch, the bidirectional cross feature map undergoes multi-layer convolution operations to gradually extract key information for classification and positioning. Specifically, the operation of the classification branch can be formalized as:

[0272] ;

[0273] in, is a classification convolution operation, is a batch normalization layer, is an activation function, such as SiLU or ReLU. The classification result is processed by Softmax to obtain the probability distribution of the category:

[0274] ;

[0275] The regression branch outputs the regression parameters of the bounding box through the convolutional layer :

[0276] ;

[0277] S43. In the classification branch, the detection head calculates each bidirectional cross feature map through multiple convolutional layers. The target category score on . The goal of this process is to output the probability distribution of each position of the bidirectional cross feature map belonging to a certain category 𝐶, which plays a key role in the subsequent non-maximum suppression process. At the same time, in the regression branch, the detection head uses distributed focal loss to further refine the predicted bounding box parameters. The goal of distributed focal loss is to generate more accurate positioning results by modeling the distribution of bounding boxes. Distributed focal loss processes the regression results:

[0278] ;

[0279] in, Represents the bounding box parameters after being processed by the distributed focal loss. The distributed focal loss generates the final prediction result by weighted summing the distribution of the bounding box parameters.

[0280] S44, the detection head module integrates the results of the classification branch and the regression branch. , the final output includes the target category 𝐶, confidence And the bounding box after distributed focal loss The integrated detection results can be expressed as:

[0281] ;

[0282] in, Represented in the bidirectional cross feature map The detection results on the image contain the category, confidence, and bounding box coordinates of each detected target. The detection results play an important role in the final detection task, including the precise positioning and classification of flame and smoke targets. Through post-processing steps such as non-maximum suppression, overlapping detection boxes can be further filtered out to retain the most likely target locations.

[0283] The detection head module can not only capture the diversity of targets through the combination of multi-scale features and deep processing of multi-layer convolution, but also improve the detection accuracy of small targets and targets in complex backgrounds through efficient classification and positioning strategies. This embodiment has high robustness and accuracy in practical applications, especially in scenarios such as fire warning and security monitoring, and can effectively cope with changing environments and complex scenarios.

[0284] The beneficial effects of this embodiment include:

[0285] By introducing deep convolutional neural networks and bidirectional cross feature fusion modules, the accuracy and robustness of fire disaster and smoke detection are effectively improved. By using the improved backbone network structure and the specially designed flame and smoke feature enhancement module, multi-scale features can be extracted more accurately, and effective recognition of flames and smoke in complex scenes can be achieved. Combined with the dynamic weight adjustment mechanism and bidirectional cross feature fusion strategy in the bidirectional cross feature fusion module, the expression ability of the multi-layer feature map is further optimized, thereby significantly improving the ability to detect small targets and handle complex backgrounds. Finally, through the optimized detection head design, excellent detection effects are shown in practical application scenarios such as fire warning and safety monitoring.

[0286] Reference Figure 5 The embodiment of the present application also provides a disaster smoke detection system based on a deep convolutional neural network, which can implement the above-mentioned disaster smoke detection method based on a deep convolutional neural network. The system includes:

[0287] A feature extraction unit, used for performing a first convolution operation on the input image to extract features and generate a primary feature map;

[0288] A feature enhancement unit, used for performing enhancement processing on the primary feature map to obtain an enhanced feature map;

[0289] A first feature fusion unit, configured to perform multi-scale fusion on the enhanced feature map according to the second convolution operation to obtain a plurality of feature maps of different scales as high-level feature maps;

[0290] A second feature fusion unit is used to perform top-down and bottom-up feature fusion on each of the high-level feature maps in scale according to the third convolution operation, and correspondingly obtain a plurality of top-down fused feature maps and a plurality of bottom-up fused feature maps;

[0291] A bidirectional cross fusion unit, used for fusing each of the top-down fusion feature maps and the bottom-up fusion feature maps to obtain a plurality of bidirectional cross feature maps;

[0292] The target detection unit is used to perform disaster and smoke detection on each of the bidirectional cross feature maps.

[0293] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0294] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned disaster smoke detection method based on a deep convolutional neural network when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0295] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0296] See also Figure 6 , Figure 6 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0297] The processor 601 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0298] The memory 602 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 602 can store an operating system and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 602, and the processor 601 calls and executes a disaster smoke detection method based on a deep convolutional neural network in the embodiment of the present application;

[0299] Input / output interface 603, used to implement information input and output;

[0300] Communication interface 604, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0301] A bus 605 that transmits information between various components of the device (e.g., processor 601, memory 602, input / output interface 603, and communication interface 604);

[0302] The processor 601 , the memory 602 , the input / output interface 603 and the communication interface 604 are connected to each other in communication within the device via a bus 605 .

[0303] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned disaster smoke detection method based on a deep convolutional neural network.

[0304] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0305] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0306] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0307] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0308] The system embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0309] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0310] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0311] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0312] In the several embodiments provided in the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0313] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0314] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0315] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0316] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A disaster smoke detection method based on deep convolutional neural network, characterized in that: The method comprises the following steps: Performing a first convolution operation on the input image to extract features and generate a primary feature map; Performing enhancement processing on the primary feature map to obtain an enhanced feature map; Performing multi-scale fusion on the enhanced feature map according to the second convolution operation to obtain multiple feature maps of different scales as high-level feature maps; According to the third convolution operation, each of the high-level feature maps is subjected to top-down and bottom-up feature fusion in scale, and a plurality of top-down fused feature maps and a plurality of bottom-up fused feature maps are correspondingly obtained; Fusing each of the top-down fusion feature maps and the bottom-up fusion feature maps to obtain a plurality of bidirectional cross feature maps; Performing disaster and smoke detection on each of the bidirectional cross-feature graphs; The step of performing a first convolution operation on the input image to extract features and generate a primary feature map comprises the following steps: Performing a convolution operation on the input image through a spatial and channel reconstruction convolution module to obtain a basic feature map; The convolution operation expression of the spatial and channel reconstruction convolution module is: ； in, is the basic feature map; Represents spatial and channel reconstruction convolution operations; Represents the input image With weight The convolution operation with a convolution kernel size of , stride Through AutoFill To adjust the size of the output feature map; Used to handle space redundancy, Used to handle channel redundancy; represents the batch normalization operation, is the SiLU activation function; Performing a convolution operation on the basic feature map through a residual block to obtain the primary feature map; The convolution operation expression of the residual block is: ； The residual connection expression of the residual block is: ； in, and represents two consecutive spatial and channel reconstruction convolution operations, represents the feature map after convolution, is the primary feature map; The step of performing enhancement processing on the primary feature map to obtain an enhanced feature map comprises the following steps: Generate a weight matrix corresponding to the primary feature map through a convolution operation of an attention mechanism, and multiply the weight matrix by the primary feature map element by element to obtain a first intermediate feature map; The convolution operation expression of the attention mechanism is: ； in, is the weight matrix, represents the convolution operation for generating the weight matrix, and the Sigmoid function is used to normalize the output; the first intermediate feature map is , ; Generating an offset of a deformable convolution layer of the first intermediate feature map through a local convolution operation, and performing a deformable convolution operation on the offset of the deformable convolution layer and the first intermediate feature map to obtain a second intermediate feature map; The expression of the offset of the deformable convolution layer is: ； The expression of the deformable convolution operation is: ； in, is the offset of the deformable convolutional layer, Represents the convolution operation of an independent convolutional layer; represents the deformable convolution operation, is the second intermediate feature map; Performing Gabor filtering on the second intermediate feature map to obtain a filtered feature map; The expression of the Gabor filtering process is: ； in, Used to control the width of the Gaussian function, Used to control the aspect ratio of the elliptical Gaussian distribution. is the wavelength, is the phase shift; is the pixel coordinate; The brightness and contrast of the filtered feature map are adjusted to obtain the enhanced feature map.

2. According to claim 1, a disaster smoke detection method based on deep convolutional neural network is characterized in that: The multi-scale fusion of the enhanced feature map according to the second convolution operation to obtain multiple feature maps of different scales as high-level feature maps includes the following steps: Performing maximum pooling operations of different scales on the enhanced feature map to obtain multiple maximum pooling feature maps; Splicing each of the maximum pooling feature maps, and then inputting the spliced ​​feature maps into the target convolution layer for fusion to obtain a multi-scale feature map; The features of the target area in the multi-scale feature map are enhanced by an attention mechanism to obtain the high-level feature map.

3. The disaster smoke detection method based on deep convolutional neural network according to claim 1 is characterized in that: The step of performing top-down and bottom-up feature fusion on each of the high-level feature maps in scale according to the third convolution operation, and correspondingly obtaining a plurality of top-down fused feature maps and a plurality of bottom-up fused feature maps, comprises the following steps: The number of channels of each of the high-level feature maps is converted into a consistent dimension through a 1×1 convolution operation to obtain a plurality of converted feature maps; Starting from the conversion feature map at the highest scale layer, each conversion feature map is propagated downward layer by layer through interpolation upsampling, and weighted fused with the conversion feature map at the next layer to obtain a first fused feature map; The expression of the first fusion feature map is: ； in, is the first fusion feature map, is the conversion feature map, represents the upsampling operation, is the feature map after depth-wise separable convolution, and is a learnable first fusion weight, wherein the first fusion weight is restricted to be non-negative by a ReLU function; Starting from the conversion feature map at the lowest level, each conversion feature map is propagated upward layer by layer through downsampling, and weighted fused with the conversion feature map at the previous level to obtain a second fused feature map; The expression of the second fusion feature map is: ； in, is the second fusion feature map, represents the downsampling operation, is the feature map after depth-wise separable convolution, , and is the learnable second fusion weight; Adjusting the first fusion weight and the second fusion weight according to the minimization loss function through a back propagation algorithm, and generating the top-down fusion feature map using the adjusted first fusion weight, and generating the bottom-up fusion feature map using the adjusted second fusion weight; The expression of the minimization loss function is: ； The expression of the top-down fusion feature map is: ； The expression of the bottom-up fusion feature map is: ； in, is the minimization loss function, is the top-down fusion feature map, is the bottom-up fusion feature map, A level number representing the conversion feature map; The fusing of the top-down fusion feature maps and the bottom-up fusion feature maps to obtain a plurality of bidirectional cross feature maps comprises the following steps: Determine an optimized fusion weight according to the adjusted first fusion weight and the adjusted second fusion weight; fusing the top-down fusion feature maps and the bottom-up fusion feature maps according to the optimized fusion weights to obtain a plurality of the bidirectional cross feature maps; The expression of the bidirectional cross feature map is: ； in, is the bidirectional cross feature map, is the optimized fusion weight.

4. The disaster smoke detection method based on deep convolutional neural network according to claim 1 is characterized in that: The disaster and smoke detection on each of the two-way cross feature graphs comprises the following steps: Identify the category of each target in each of the bidirectional cross feature maps through classification convolution operations to obtain a classification result; wherein the targets include the disaster and smoke; The expression of the classification result is: ； in, is the classification result, For the classification convolution operation, represents the batch normalization operation, is the activation function of the classification convolution, is the bidirectional cross feature map; Determine the probability of each of the targets according to the classification results, and generate the confidence level of each of the probabilities; The probability distribution of each of the targets is: ； in, is the probability distribution, is the probability, is the activation function; Outputting the bounding box parameters of each of the targets in each of the bidirectional cross feature maps through a regression convolution operation; The expression of the bounding box parameter is: ； in, are the bounding box parameters, represents the regressive convolution operation; Determining the probability of each of the targets, the confidence of each of the probabilities, and the bounding box parameters of each of the targets as the detection results of the disaster and smoke; The expression of the detection result is: ； in, is the test result; is the confidence level.

5. The disaster smoke detection method based on deep convolutional neural network according to claim 4 is characterized in that: The method further comprises the following steps: Optimizing the bounding box parameters using distributed focal loss to obtain optimized bounding box parameters; The expression of the optimized bounding box parameters is: ； in, are the optimized bounding box parameters, represents the distributed focal loss; updating the detection result according to the optimized bounding box parameters; The updated detection result is expressed as: 。 6. A disaster smoke detection system based on deep convolutional neural network, characterized in that: The system comprises: A feature extraction unit, used for performing a first convolution operation on the input image to extract features and generate a primary feature map; A feature enhancement unit, used for performing enhancement processing on the primary feature map to obtain an enhanced feature map; A first feature fusion unit, configured to perform multi-scale fusion on the enhanced feature map according to the second convolution operation to obtain a plurality of feature maps of different scales as high-level feature maps; A second feature fusion unit is used to perform top-down and bottom-up feature fusion on each of the high-level feature maps in scale according to the third convolution operation, and correspondingly obtain a plurality of top-down fused feature maps and a plurality of bottom-up fused feature maps; A bidirectional cross fusion unit, used for fusing each of the top-down fusion feature maps and the bottom-up fusion feature maps to obtain a plurality of bidirectional cross feature maps; A target detection unit, used for performing disaster and smoke detection on each of the bidirectional cross feature graphs; The step of performing a first convolution operation on the input image to extract features and generate a primary feature map comprises the following steps: Performing a convolution operation on the input image through a spatial and channel reconstruction convolution module to obtain a basic feature map; The convolution operation expression of the spatial and channel reconstruction convolution module is: ； in, is the basic feature map; Represents spatial and channel reconstruction convolution operations; Represents the input image With weight The convolution operation with a convolution kernel size of , stride Through AutoFill To adjust the size of the output feature map; Used to handle space redundancy, Used to handle channel redundancy; represents the batch normalization operation, is the SiLU activation function; Performing a convolution operation on the basic feature map through a residual block to obtain the primary feature map; The convolution operation expression of the residual block is: ； The residual connection expression of the residual block is: ； in, and represents two consecutive spatial and channel reconstruction convolution operations, represents the feature map after convolution, is the primary feature map; The step of performing enhancement processing on the primary feature map to obtain an enhanced feature map comprises the following steps: Generate a weight matrix corresponding to the primary feature map through a convolution operation of an attention mechanism, and multiply the weight matrix by the primary feature map element by element to obtain a first intermediate feature map; The convolution operation expression of the attention mechanism is: ； in, is the weight matrix, represents the convolution operation for generating the weight matrix, and the Sigmoid function is used to normalize the output; the first intermediate feature map is , ; Generating an offset of a deformable convolution layer of the first intermediate feature map through a local convolution operation, and performing a deformable convolution operation on the offset of the deformable convolution layer and the first intermediate feature map to obtain a second intermediate feature map; The expression of the offset of the deformable convolution layer is: ； The expression of the deformable convolution operation is: ； in, is the offset of the deformable convolutional layer, Represents the convolution operation of an independent convolutional layer; represents the deformable convolution operation, is the second intermediate feature map; Performing Gabor filtering on the second intermediate feature map to obtain a filtered feature map; The expression of the Gabor filtering process is: ； in, Used to control the width of the Gaussian function, Used to control the aspect ratio of the elliptical Gaussian distribution. is the wavelength, is the phase shift; is the pixel coordinate; The brightness and contrast of the filtered feature map are adjusted to obtain the enhanced feature map.

7. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images