A tunnel fire occurrence detection method, device, equipment and storage medium

Abstract

The application discloses a tunnel fire occurrence detection method, device and equipment and a storage medium, and relates to the technical field of computer vision. The method comprises the following steps: after acquiring a live image collected by a tunnel camera, pre-processing the live image to obtain an RGB image; inputting the RGB image into a fire target detection model which has been pre-trained and comprises a feature extraction network, an edge detection module, a convolution module, a pooling module and an output module; outputting a fire target detection result; combining a bounding box feature in the form of a vector feature matrix with an edge feature by using the functions of the feature extraction network, the edge detection module, the convolution module, the pooling module and the output module; and finally, based on the combination result of the two features, carrying out subsequent fire occurrence classification, so as to improve the feature reuse rate and the accuracy of the fire target detection result, and facilitate practical application and promotion.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer vision, and particularly relates to a tunnel fire occurrence detection method, device, equipment and storage medium. BACKGROUND

[0002] Among the safety accidents occurring in traffic tunnels such as highway tunnels and subway tunnels, fire is the most dangerous one. Once a fire occurs in a tunnel, it is very difficult to extinguish, and can cause serious economic losses and casualties. In order to prevent fire accidents in traffic tunnels, people have proposed various early fire occurrence detection technologies. With the large-scale popularization of video monitoring, the fire occurrence automatic detection and early warning scheme based on video image acquisition has very important research significance and use value, and has attracted great attention from people.

[0003] At present, for the traditional and computer vision-based traditional flame detection method, when facing different complex environments and variable flame types, the generalization ability is insufficient, resulting in the problem of inaccurate fire video detection. The existing patent CN202111509268.4 provides a tunnel fire detection scheme based on YOLO neural network, but the YOLO neural network used has the defect of low feature reuse rate in feature extraction, so that the fire occurrence detection result of the whole scheme in the complex tunnel environment still has the problem of limited accuracy, which needs to be further improved. SUMMARY

[0004] The purpose of the present application is to provide a tunnel fire occurrence detection method, device, computer equipment and computer readable storage medium, to solve the problem that the fire occurrence detection result of the existing tunnel fire detection scheme in the complex tunnel environment still has limited accuracy.

[0005] In order to achieve the above purpose, the present application adopts the following technical scheme:

[0006] In a first aspect, a tunnel fire occurrence detection method is provided, comprising:

[0007] acquiring a live image collected by a camera in a tunnel;

[0008] preprocessing the live image to obtain an RGB image with a dimension of , wherein WxW represents the resolution of the RGB image, W represents a positive integer not less than 160, represents the number of channels of the RGB image;

[0009] inputting the RGB image into a pre-trained fire target detection model to output a fire target detection result, wherein the fire target detection model comprises a feature extraction network, an edge detection module, a convolution module, a pooling module, and an output module;

[0010] The feature extraction network is configured to perform feature extraction on the input image to obtain a bounding box feature map with a dimension of w x w x n, wherein w = W / (2 M ), M represents a positive number, and n represents a channel number of the bounding box feature map.

[0011] The edge detection module is configured to perform edge detection on the input image based on a C-V model to obtain an edge feature map.

[0012] The convolution module is configured to perform first convolution processing with a convolution kernel size of 3 x 3 and second convolution processing with a convolution kernel size of 1 x 1 on a combination result of the bounding box feature map and the edge feature map in sequence to obtain a convolution processing result.

[0013] The pooling module is configured to perform pooling processing on the convolution processing result to obtain a pooling processing result in which a high-dimensional vector is flattened into a one-dimensional vector.

[0014] The output module is configured to input each vector in the pooling processing result into an activation function sigmoid to obtain a binary classification result corresponding to each vector and used to represent whether a fire occurs, and then output a binary classification result with the highest confidence as a fire target detection result corresponding to the bounding box feature map.

[0015] Based on the above invention content, a new tunnel fire detection scheme based on a feature extraction network and edge detection technology is provided, that is, after obtaining a live image collected by a tunnel camera, the live image is preprocessed to obtain an RGB image, then the RGB image is input into a pre-trained fire target detection model comprising a feature extraction network, an edge detection module, a convolution module, a pooling module, and an output module, and a fire target detection result is output, and the functions of the feature extraction network, the edge detection module, the convolution module, the pooling module, and the output module are combined to combine the bounding box feature and the edge feature in the form of a vector feature matrix, and finally the subsequent fire classification is carried out based on the combination of the two features, thereby achieving the purposes of improving the feature reuse rate and the accuracy of the fire target detection result, facilitating practical application and promotion.

[0016] In a possible design, the feature extraction network includes an input layer, a dimension increasing layer, and M feature extraction groups arranged in sequence along a processing direction, where the feature extraction group includes a down-sampling layer and a channel-by-channel dense connection module arranged in sequence along the processing direction;

[0017] The dimension increasing layer is configured to perform dimension increasing processing on an image input by the input layer in a channel number dimension to obtain a dimension-increased image with a dimension of WxWxn1, where n1 represents a channel number of the dimension-increased image.

[0018] The down-sampling layer in the mth feature extraction group in the M feature extraction groups and along the processing direction is configured to perform down-sampling processing on a first previous layer output image to obtain an mth down-sampled image with a dimension of , where m represents a positive integer not greater than M, the first previous layer output image refers to an image output by a processing layer located before the down-sampling layer along the processing direction, and n m,1 represents a channel number of the first previous layer output image.

[0019] The channel-by-channel dense connection module of the mth feature extraction group is configured to concatenate, in a channel number dimension, a convolution result and a copy result of the mth down-sampled image to obtain an mth bounding box feature map with a dimension of , where n m represents a channel number of the mth bounding box feature map.

[0020] In a possible design, the dimension increasing layer adopts a convolution layer with a filter number of 32, a convolution kernel size of 3x3, a zero padding number of 1, and a step of 1.

[0021] In a possible design, the down-sampling layer adopts a convolution layer with a filter number of 2 m times a filter number of the dimension increasing layer, a convolution kernel size of 3x3, a zero padding number of 1, and a step of 2.

[0022] In a possible design, the channel-by-channel dense connection module includes at least one channel-by-channel dense connection submodule arranged in sequence along the processing direction, where the channel-by-channel dense connection submodule includes a convolution layer, a copy layer, and a concatenation layer.

[0023] The channel-by-channel convolution layer in the jth channel-by-channel dense connection submodule of the channel-by-channel dense connection module of the mth feature extraction group along the processing direction is configured to perform convolution processing on a second previous layer output image in each channel first, and then concatenate, in a channel number dimension, convolution processing results corresponding to the channels one by one to obtain an image with a dimension of a first feature map of dimension, where j represents a positive integer, the second pre-layer output image refers to an image output by a processing layer located before the channel-wise convolutional layer in the processing direction, n m,j represents a number of channels of the first feature map;

[0024] a copy layer of the jth channel-wise dense connection sub-module, configured to perform copy processing on the second pre-layer output image to obtain a second feature map of dimension , where n m,2 represents a number of channels of the second pre-layer output image;

[0025] a concatenation layer of the jth channel-wise dense connection sub-module, configured to concatenate the first feature map and the second feature map in the channel dimension to obtain a third feature map of dimension .

[0026] In a possible design, the channel-wise convolutional layer is a convolutional layer with a filter number being 2 m times a filter number of the dimension increasing layer, a convolution kernel size being 3x3, a zero padding number being 1, and a step length being 1;

[0027] Alternatively, the channel-wise convolutional layer is a convolutional layer with a filter number being 2 m times a filter number of the dimension increasing layer, a convolution kernel size being 5x5, a zero padding number being 2, and a step length being 1.

[0028] In a possible design, when the value of M is not less than 3, the fire target detection model comprises K convolutional modules, K pooling modules, K output modules, and K-1 down-sampling modules, where K represents a positive integer not less than 2 and less than M;

[0029] a first convolutional module of the K convolutional modules, configured to sequentially perform first convolutional processing with a convolution kernel size of 3x3 and second convolutional processing with a convolution kernel size of 1x1 on a combination result of the bounding box feature map output by the M-K+1th feature extraction group along the processing direction and the edge feature map, to obtain a first convolutional processing result;

[0030] a first pooling module of the K pooling modules, configured to perform pooling processing on the first convolutional processing result to obtain a first pooling processing result in which a high-dimensional vector is flattened into a one-dimensional vector;

[0031] The first output module among the K output modules is used to input each vector in the first pooling result into the activation function sigmoid to obtain a binary classification result that corresponds one-to-one with each vector in the first pooling result and is used to indicate whether a fire has occurred. Then, the binary classification result with the highest confidence is used as the fire target detection result corresponding to the bounding box feature map output by the M-K+1th feature extraction group along the processing direction and output.

[0032] The (k-1)th downsampling module in the K-1 downsampling modules is used to downsample the convolution processing result output by the (k-1)th convolution module in the K convolution modules to obtain the downsampling processing result, where k represents a positive integer not less than 2 and not greater than K;

[0033] The kth convolutional module in the K convolutional modules is used to perform a first convolutional process with a kernel size of 3×3 and a second convolutional process with a kernel size of 1×1 on the combination result of the bounding box feature map output by the M-K+kth feature extraction group along the processing direction and the downsampling processing result output by the (k-1)th downsampling module, so as to obtain the kth convolutional processing result.

[0034] The kth pooling module among the K pooling modules is used to perform pooling processing on the kth convolution processing result to obtain the kth pooling processing result that flattens the high-dimensional vector into a one-dimensional vector.

[0035] The kth output module of the K output modules is used to input each vector in the kth pooling result into the activation function sigmoid to obtain a binary classification result that corresponds one-to-one with each vector in the kth pooling result and is used to indicate whether a fire has occurred. Then, the binary classification result with the highest confidence is used as the fire target detection result corresponding to the bounding box feature map output by the M-K+kth feature extraction group along the processing direction and output.

[0036] Secondly, a fire detection device for tunnels is provided, comprising an image acquisition unit, a preprocessing unit, and a detection unit that are sequentially connected in communication.

[0037] The image acquisition unit is used to acquire on-site images captured by the camera inside the tunnel;

[0038] The preprocessing unit is used to preprocess the scene image to obtain a dimension of The RGB image, where W×W represents the resolution of the RGB image, and W represents a positive integer not less than 160. This indicates the number of channels in the RGB image;

[0039] The detection unit is configured to input the RGB image into a pre-trained fire target detection model to output a fire target detection result, wherein the fire target detection model comprises a feature extraction network, an edge detection module, a convolution module, a pooling module, and an output module.

[0040] The feature extraction network is configured to perform feature extraction on the input image to obtain a bounding box feature map with a dimension of w x w x n, wherein w = W / (2 M ), M represents a positive number, and n represents a channel number of the bounding box feature map.

[0041] The edge detection module is configured to perform edge detection on the input image based on a C-V model to obtain an edge feature map.

[0042] The convolution module is configured to perform first convolution processing with a convolution kernel size of 3 x 3 and second convolution processing with a convolution kernel size of 1 x 1 on a combination result of the bounding box feature map and the edge feature map in sequence to obtain a convolution processing result.

[0043] The pooling module is configured to perform pooling processing on the convolution processing result to obtain a pooling processing result in which a high-dimensional vector is flattened into a one-dimensional vector.

[0044] The output module is configured to input each vector in the pooling processing result into an activation function sigmoid to obtain a binary classification result corresponding to each vector and used to represent whether a fire occurs, and then output a binary classification result with the highest confidence as a fire target detection result corresponding to the bounding box feature map.

[0045] In a third aspect, the present application provides a computer device comprising a memory, a processor, and a transceiver connected in sequence, wherein the memory is configured to store a computer program, the transceiver is configured to receive and send messages, and the processor is configured to read the computer program and execute the tunnel fire detection method according to any possible design of the first aspect.

[0046] In a fourth aspect, the present application provides a computer readable storage medium having instructions stored thereon, wherein the instructions, when executed on a computer, perform the tunnel fire detection method according to any possible design of the first aspect.

[0047] In a fifth aspect, the present application provides a computer program product comprising instructions, wherein the instructions, when executed on a computer, cause the computer to perform the tunnel fire detection method according to any possible design of the first aspect.

[0048] The above-mentioned scheme has the beneficial effects of:

[0049] (1) The application creatively provides a new tunnel fire occurrence detection scheme based on a feature extraction network and an edge detection technology, that is, after obtaining a live image collected by a camera in the tunnel, the live image is preprocessed to obtain an RGB image, then the RGB image is input into a fire target detection model which has been pre-trained and includes a feature extraction network, an edge detection module, a convolution module, a pooling module and an output module, a fire target detection result is output, and the boundary box features and edge features in the form of a vector feature matrix are combined and used through the functional combination of the feature extraction network, the edge detection module, the convolution module, the pooling module and the output module, and finally the subsequent fire occurrence classification is carried out based on the combination result of the two features, so as to achieve the purposes of improving the feature reuse rate and the accuracy of the fire target detection result.

[0050] (2) The feature reuse rate can be improved during feature extraction, the accuracy of the fire target detection result of the whole scheme is further effectively improved, and the calculation cost during feature extraction can be greatly reduced, which is beneficial to quickly obtaining the feature extraction result.

[0051] (3) Small target images can be better processed to obtain small target detection results of different scales, and the fire occurrence detection result is enriched. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only constitute some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0053] Figure 1 The flowchart of the tunnel fire occurrence detection method provided by the embodiment of the application is shown.

[0054] Figure 2 The structure diagram of the fire target detection model provided by the embodiment of the application is shown.

[0055] Figure 3 The layer structure diagram of the feature extraction network provided by the embodiment of the application is shown.

[0056] Figure 4 The layer structure diagram of the channel-by-channel dense connection module provided by the embodiment of the application is shown.

[0057] Figure 5 The working principle diagram of the channel-by-channel convolution layer provided by the embodiment of the application is shown.

[0058] Figure 6 The structural schematic diagram of the tunnel fire occurrence detection device provided by the embodiment of the application is shown.

[0059] Figure 7 The structural schematic diagram of the computer device provided by the embodiment of the application is shown. DETAILED DESCRIPTION

[0060] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the present application will be briefly introduced below in combination with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the drawings is only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings. It should be noted that the description of these embodiments is used to help understand the present application, but does not constitute a limitation on the present application.

[0061] It should be understood that although the terms first and second, etc. may be used herein to describe various objects, these objects should not be limited by these terms. These terms are only used to distinguish one object from another object. For example, a first object can be called a second object, and similarly, a second object can be called a first object, without departing from the scope of the example embodiments of the present application.

[0062] It should be understood that for the term "and / or" which may appear in the present text, it is only a description of the association relationship of the associated objects, which means that there can be three kinds of relationships, for example, A and / or B, which can represent three cases of A alone, B alone, or A and B existing at the same time; for example, A, B and / or C, which means that any one of A, B and C or any combination thereof exists; for the term " / and" which may appear in the present text, it is another description of the relationship of another associated object, which means that there can be two kinds of relationships, for example, A / and B, which can represent two cases of A alone or A and B existing at the same time; in addition, for the character " / " which may appear in the present text, it generally represents an "or" relationship between the associated objects before and after it.

[0063] Embodiment:

[0064] As Figure 1As shown, the tunnel fire occurrence detection method provided by the first aspect of the present embodiment can be, but is not limited to, executed by a computer device with certain computing resources and communicatively connected to the in-tunnel camera, such as a platform server, a personal computer (PC, referring to a multi-purpose computer with size, price and performance suitable for personal use), a smart phone, a personal digital assistant (PDA) or a wearable device, etc. Figure 1 As shown, the tunnel fire occurrence detection method can include, but is not limited to, the following steps S1-S3.

[0065] S1. Obtain a live image collected by an in-tunnel camera.

[0066] In the step S1, the in-tunnel camera is installed inside a traffic tunnel such as a highway tunnel and a subway tunnel, for collecting video images of the live situation inside the tunnel, which can be realized by a conventional monitoring camera. In addition, the specific acquisition method of the live image can be a wired communication method or a wireless communication method.

[0067] S2. Pre-process the live image to obtain an RGB (RGB represents the color of the red, green and blue channels) image with a dimension of , wherein WxW represents the resolution of the RGB image, W represents a positive integer not less than 160, represents the number of channels of the RGB image.

[0068] In the step S2, the specific method of the pre-processing is a conventional method, such as color space conversion processing, size scaling processing and image anti-shake processing, etc. As shown, Figure 3 W can be, for example, 416, i.e. the resolution of the RGB image is 416x416. In addition, since each pixel point in the RGB image has different red channel color value, green channel color value and blue channel color value, the value of is 3.

[0069] S3. Input the RGB image into a pre-trained fire target detection model to output a fire target detection result, wherein the fire target detection model includes, but is not limited to, a feature extraction network, an edge detection module, a convolution module, a pooling module and an output module, etc.

[0070] In the step S3, as shown, Figure 2As shown, specifically, the feature extraction network is configured to perform feature extraction on the input image to obtain a bounding box feature map with a dimension of w x w x n, where w = W / (2 M ), M represents a positive number, and n represents a channel number of the bounding box feature map. The input image can be the RGB image as the sample image to be tested or a sample image in RGB format input in the model training process. The edge detection module is configured to perform edge detection on the input image based on a C-V model to obtain an edge feature map. Since the tunnel is relatively dark, if there is a fire in the tunnel, the fire will appear relatively obvious. Therefore, the edge detection module is added based on this characteristic to outline the contour features of the fire light, so that the fire target detection model can pay more attention to the fire light. The C-V model is an existing region-based active contour model, which takes the pixel gray information of the image as the energy, can skillfully construct the energy function, and finally divides out the contour target by finding the minimum value of the energy function. The convolution module is configured to perform first convolution processing with a convolution kernel size of 3 x 3 and second convolution processing with a convolution kernel size of 1 x 1 on the combined result of the bounding box feature map and the edge feature map in sequence to obtain a convolution processing result. The pooling module is configured to perform pooling processing on the convolution processing result to obtain a pooling processing result in which a high-dimensional vector is flattened into a one-dimensional vector. The output module is configured to input each vector in the pooling processing result into an activation function sigmoid to obtain a binary classification result corresponding to each vector and indicating whether a fire occurs, and then output the binary classification result with the highest confidence as the fire target detection result corresponding to the bounding box feature map. In this way, through the combination of the convolution module, the pooling module, and the output module in sequence, the bounding box feature and the edge feature in the form of a vector feature matrix can be combined and utilized, and finally the subsequent fire occurrence classification is performed based on the combined result of the two features, so as to improve the feature reuse rate and the accuracy of the fire target detection result. In addition, the specific training method of the fire target detection model is a conventional method, the specific method of obtaining the combined result of the bounding box feature map and the edge feature map is a conventional combination method, such as superimposing the two feature maps, and the specific methods of the first convolution processing, the second convolution processing, and the pooling processing are conventional processing methods. The activation function sigmoid is a commonly used binary classification function.

[0071] Therefore, based on the tunnel fire occurrence detection method described in the foregoing steps S1-S3, a new tunnel fire occurrence detection scheme based on feature extraction network and edge detection technology is provided, that is, after obtaining the live image collected by the tunnel camera, the live image is preprocessed to obtain an RGB image, then the RGB image is input into the fire target detection model which has been pre-trained and includes a feature extraction network, an edge detection module, a convolution module, a pooling module and an output module, and a fire target detection result is output, and the boundary box features and edge features in the form of a vector feature matrix are combined and used through the functional combination of the feature extraction network, the edge detection module, the convolution module, the pooling module and the output module, and finally the subsequent fire occurrence classification is carried out based on the combination result of the two features, so as to realize the purposes of improving the feature reuse rate and the accuracy of the fire target detection result, and facilitate actual application and promotion.

[0072] On the basis of the technical scheme of the first aspect, the embodiment further provides a possible design for optimizing feature extraction, that is, as shown in Figure 2 , the feature extraction network includes but is not limited to an input layer, a dimension increasing layer and M feature extraction groups arranged in the processing direction.

[0073] As shown in Figure 2 , the value of M is 5. In the specific structure of the feature extraction network, the dimension increasing layer is used to perform dimension increasing processing on the image input by the input layer in the channel dimension to obtain a dimension-increased image with a dimension of WxWxn1, where n1 represents the channel number of the dimension-increased image; in detail, as shown in Figure 3 , the dimension increasing layer can adopt a convolution layer with a filter number of 32, a convolution kernel size of 3x3, a zero padding number of 1 and a step of 1. The down-sampling layer of the mth feature extraction group in the M feature extraction groups and in the processing direction is used to perform down-sampling processing on the first previous layer output image to obtain an mth down-sampled image with a dimension of , where m represents a positive integer not greater than M, the first previous layer output image refers to an image output by a processing layer located before the down-sampling layer in the processing direction, and n m,1 represents the channel number of the first previous layer output image; the purpose of the down-sampling layer is to enable the feature extraction network to extract features from images with different resolutions so as to obtain high-level feature information of small targets; since the down-sampling layer does not perform dimension increasing on the first previous layer output image in the channel dimension, but performs dimension decreasing on the first previous layer output image in the resolution dimension, in detail, as shown in Figure 3As shown, the down-sampling layer can employ a convolutional layer with a filter number being 2 times of the filter number of the up-sampling layer, a convolution kernel size being 3x3, a zero padding number being 1, and a step being 2. m The channel-by-channel dense connection module of the mth feature extraction group is configured to concatenate the convolution result and the copy result of the mth down-sampled image in the channel number dimension, to obtain an mth bounding box feature map with a dimension of , where n m represents the channel number of the mth bounding box feature map. The channel-by-channel dense connection module is configured to extract features from images with different resolutions and extract information in different receptive field ranges using different sizes of convolution kernels. In addition, the filter number, the convolution kernel size, the zero padding number, and the step are common parameters in existing convolutional layers.

[0074] As shown in Figure 4 , in detail, the channel-by-channel dense connection module includes, but is not limited to, at least one channel-by-channel dense connection submodule arranged in the processing direction in sequence, where the channel-by-channel dense connection submodule includes, but is not limited to, a channel-by-channel convolutional layer, a copy layer, and a concatenation layer, etc. Different channel-by-channel dense connection modules can include the same number or different numbers of channel-by-channel dense connection submodules, as shown in Figure 3 , there are two channel-by-channel dense connection submodules (i.e., "x2") in the channel-by-channel dense connection module of the first feature extraction group, three channel-by-channel dense connection submodules (i.e., "x3") in the channel-by-channel dense connection module of the second feature extraction group, three channel-by-channel dense connection submodules (i.e., "x3") in the channel-by-channel dense connection module of the third feature extraction group, four channel-by-channel dense connection submodules (i.e., "x4") in the channel-by-channel dense connection module of the fourth feature extraction group, and only one channel-by-channel dense connection submodule (i.e., "x1") in the channel-by-channel dense connection module of the fifth feature extraction group. The channel-by-channel convolutional layer of the jth channel-by-channel dense connection submodule in the channel-by-channel dense connection module of the mth feature extraction group is configured to perform convolution processing on the second previous layer output image in each channel first, and then concatenate the convolution processing results corresponding to the channels one by one in the channel number dimension, as shown in Figure 5 , the second previous layer output image has N channels, and the same convolution kernel size is used to perform convolution processing on the images corresponding to each of the N channels to obtain the corresponding convolution processing results), to obtain a first feature map with a dimension of , where j represents a positive integer, the second previous layer output image refers to an image output by a processing layer located before the channel-by-channel convolutional layer in the processing direction, and nm,j represents the number of channels of the first feature map; in order to ensure that the convolution result and the copy result can be spliced subsequently, the channel-by-channel convolution layer cannot reduce or increase the resolution dimension of the second front layer output image, that is, the channel-by-channel convolution layer needs to use a convolution layer with a filter number being 2 m times the filter number of the up-sampling layer, a convolution kernel size of 3*3, a zero padding number of 1 and a step of 1; or, the channel-by-channel convolution layer uses a convolution layer with a filter number being 2 m times the filter number of the up-sampling layer, a convolution kernel size of 5*5, a zero padding number of 2 and a step of 1. The copy layer of the jth channel-by-channel dense connection sub-module is used for copy processing on the second front layer output image to obtain a second feature map with a dimension of , where n m,2 represents the number of channels of the second front layer output image. The splicing layer of the jth channel-by-channel dense connection sub-module is used for splicing the first feature map and the second feature map in the channel number dimension to obtain a third feature map with a dimension of .

[0075] Through the specific design of the foregoing feature extraction network, the following two characteristics of the channel-by-channel dense connection module can be utilized: (1) complete connection of all previous layers and subsequent layers can be established, and (2) feature reuse can be achieved through connection of features in channels, which also improves the feature reuse rate during feature extraction, so that the accuracy of the fire target detection result of the entire scheme is further effectively improved. At the same time, since the channel-by-channel convolution layer is a convolution kernel that operates on each channel once, the calculation cost is 1*N; while in general convolution operation, each channel and each convolution kernel will operate once, that is, assuming that the number of channels is N and the number of convolution kernels is K, the calculation cost is N*K, so the calculation cost during feature extraction can also be greatly reduced, which is beneficial to quickly obtaining the feature extraction result.

[0076] Therefore, based on the foregoing possible design one, the feature reuse rate during feature extraction can also be improved, the accuracy of the fire target detection result of the entire scheme is further effectively improved, and the calculation cost during feature extraction can also be greatly reduced, which is beneficial to quickly obtaining the feature extraction result.

[0077] The embodiment further provides a possible design one for how to obtain different scale small target detection results on the basis of the technical scheme of the foregoing possible design one, that is, as shown in Figure 2 when the value of M is not less than 3, the fire target detection model comprises K convolution modules, K pooling modules, K output modules and K-1 down-sampling modules, where K represents a positive integer not less than 2 and less than M.

[0078] The first convolutional module among the K convolutional modules is used to sequentially perform a first convolutional process with a kernel size of 3×3 and a second convolutional process with a kernel size of 1×1 on the combination result of the bounding box feature map output by the M-K+1th feature extraction group along the processing direction and the edge feature map, to obtain the first convolutional processing result; the first pooling module among the K pooling modules is used to perform pooling processing on the first convolutional processing result to obtain the first pooling processing result that flattens the high-dimensional vector into a one-dimensional vector; the first output module among the K output modules is used to input each vector in the first pooling processing result into the sigmoid activation function to obtain a binary classification result that corresponds one-to-one with each vector in the first pooling processing result and is used to represent whether a fire has occurred, and then the binary classification result with the highest confidence is used as the fire target detection result corresponding to the bounding box feature map output by the M-K+1th feature extraction group along the processing direction and output.

[0079] The (k-1)th downsampling module in the K-1 downsampling modules is used to downsample the convolution result output by the (k-1)th convolution module in the K convolution modules to obtain a downsampled result, where k represents a positive integer not less than 2 and not greater than K. The (k-1)th downsampling module functions similarly to the downsampling layer, but instead of increasing the dimensionality of the convolution result output by the (k-1)th convolution module in the channel number dimension, it decreases the dimensionality of the convolution result output by the (k-1)th convolution module in the resolution dimension. Specifically, the (k-1)th downsampling module can use a filter count that is twice the number of filters in the dimensionality-increasing layer. M-K+k A convolutional layer with a 3×3 kernel size, 1 zero-padding, and a stride of 2.

[0080] The kth convolution module in the K convolution modules is configured to perform first convolution processing with a convolution kernel size of 3*3 and second convolution processing with a convolution kernel size of 1*1 on a combination result of a bounding box feature map output by the M-K+kth feature extraction group along the processing direction and a down-sampling processing result output by the k-1th down-sampling module, to obtain a kth convolution processing result; the kth pooling module in the K pooling modules is configured to perform pooling processing on the kth convolution processing result, to obtain a kth pooling processing result in which a high-dimensional vector is flattened into a one-dimensional vector; and the kth output module in the K output modules is configured to input each vector in the kth pooling processing result into an activation function sigmoid, to obtain a binary classification result corresponding to each vector in the kth pooling processing result and used to represent whether a fire occurs, and then output a binary classification result with the highest confidence as a fire target detection result corresponding to the bounding box feature map output by the M-K+kth feature extraction group along the processing direction.

[0081] As shown in FIG. 1, the value of M is 5, the value of K is 3, and as shown in FIG. 2, it can be predicted that there are three different scales of bounding boxes, i.e., 13*13, 26*26 and 52*52. Figure 2 Figure 3 As shown in FIG. 2, it can be predicted that there are three different scales of bounding boxes, i.e., 13*13, 26*26 and 52*52.

[0082] Therefore, based on the foregoing possible design two, small target images can be better processed, and small target detection results of different scales can be obtained, so as to enrich the fire occurrence detection results.

[0083] As shown in FIG. 1, the value of M is 5, the value of K is 3, and as shown in FIG. 2, it can be predicted that there are three different scales of bounding boxes, i.e., 13*13, 26*26 and 52*52. Figure 6 As shown in FIG. 1, the value of M is 5, the value of K is 3, and as shown in FIG. 2, it can be predicted that there are three different scales of bounding boxes, i.e., 13*13, 26*26 and 52*52.

[0084] The image acquisition unit is configured to acquire a live image collected by a camera in a tunnel.

[0085] The preprocessing unit is configured to perform preprocessing on the live image, to obtain an RGB image with a dimension of W*W*C, wherein W*W*C represents a resolution of the RGB image, W represents a positive integer not less than 160, and C represents a channel number of the RGB image. The detection unit is configured to input the RGB image into a fire target detection model that has been pre-trained, and output a fire target detection result.

[0086] The detection unit is configured to input the RGB image into a fire target detection model that has been pre-trained, and output a fire target detection result.

[0087] ​​The feature extraction network is configured to perform feature extraction on the input image to obtain a bounding box feature map with a dimension of w x w x n, where w = W / (2 M ), M represents a positive number, and n represents a channel number of the bounding box feature map.

[0088] The edge detection module is configured to perform edge detection on the input image based on a C-V model to obtain an edge feature map.

[0089] The convolution module is configured to perform first convolution processing with a convolution kernel size of 3 x 3 and second convolution processing with a convolution kernel size of 1 x 1 on a combination result of the bounding box feature map and the edge feature map in sequence to obtain a convolution processing result.

[0090] The pooling module is configured to perform pooling processing on the convolution processing result to obtain a pooling processing result in which a high-dimensional vector is flattened into a one-dimensional vector.

[0091] The output module is configured to input each vector in the pooling processing result into an activation function sigmoid to obtain a binary classification result corresponding to each vector and used to represent whether a fire occurs, and then output a binary classification result with the highest confidence as a fire target detection result corresponding to the bounding box feature map.

[0092] The working process, working details and technical effects of the foregoing device provided in the second aspect of the embodiment can be referred to the tunnel fire occurrence detection method described in the first aspect, the possible design one or the possible design two, and details are not described herein again.

[0093] As Figure 7As shown, the third aspect of this embodiment provides a computer device for executing the tunnel fire detection method as described in the first aspect, possible design one, or possible design two. The device includes a memory, a processor, and a transceiver connected in sequence. The memory stores a computer program, the transceiver sends and receives messages, and the processor reads the computer program to execute the tunnel fire detection method as described in the first aspect, possible design one, or possible design two. Specifically, the memory may include, but is not limited to, random-access memory (RAM), read-only memory (ROM), flash memory, first-in-first-out (FIFO) memory, and / or first-in-last-out (FILO) memory, etc.; the processor may include, but is not limited to, a microprocessor of the STM32F105 series. Furthermore, the computer device may also include, but is not limited to, a power module, a display screen, and other necessary components.

[0094] The working process, working details and technical effects of the aforementioned computer equipment provided in the third aspect of this embodiment can be found in the tunnel fire detection method described in the first aspect, possible design one or possible design two, and will not be repeated here.

[0095] This fourth aspect of the embodiment provides a computer-readable storage medium storing instructions comprising a method for detecting a fire in a tunnel as described in the first aspect, possible design one, or possible design two. Specifically, the computer-readable storage medium stores instructions that, when executed on a computer, perform the method for detecting a fire in a tunnel as described in the first aspect, possible design one, or possible design two. The computer-readable storage medium refers to a data storage medium, which may include, but is not limited to, floppy disks, optical disks, hard disks, flash memory, USB flash drives, and / or memory sticks. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0096] The working process, working details and technical effects of the aforementioned computer-readable storage medium provided in the fourth aspect of this embodiment can be found in the tunnel fire detection method described in the first aspect, possible design one or possible design two, and will not be repeated here.

[0097] The fifth aspect of the embodiment provides a computer program product comprising instructions which, when executed on a computer, cause the computer to carry out the tunnel fire occurrence detection method according to the first aspect, possibly the design one or possibly the design two. The computer can be a general purpose computer, a special purpose computer, a computer network or other programmable devices.

[0098] Finally, it should be noted that the above only describes the preferred embodiments of the present application and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting fires inside tunnels, characterized in that, include: Acquire on-site images captured by cameras inside the tunnel; The scene image is preprocessed to obtain a dimension of RGB image, where, This indicates the resolution of the RGB image. Represents a positive integer not less than 160. This represents the number of channels in the RGB image; The RGB image is input into a pre-trained fire target detection model, and the fire target detection result is output. The fire target detection model includes a feature extraction network, an edge detection module, a convolution module, a pooling module, and an output module. The feature extraction network is used to perform feature extraction processing on the input image, obtaining a feature with dimension . The bounding box feature map, where, , Represents positive integers. This represents the number of channels in the bounding box feature map; The feature extraction network includes an input layer, an upscaling layer, and a sub-layer arranged sequentially along the processing direction. A feature extraction group, wherein the feature extraction group includes a downsampling layer and a channel-by-channel dense connection module arranged sequentially along the processing direction; the dimensionality-upgrading layer is used to perform dimensionality-upgrading processing on the image input by the input layer in the dimension of channel number, to obtain a dimension of The upgraded image, in which, This represents the number of channels in the upscaling image; in the The first feature extraction group and along the processing direction A downsampling layer with 1 feature extraction group is used to downsample the output image of the first front layer, resulting in a dimension of 1. The 1 downsampled image, of which Indicates not greater than A positive integer, wherein the first front-layer output image refers to the image output by a processing layer located before the downsampling layer in the processing direction. This represents the number of channels in the first front-layer output image; the number of channels in the first front-layer output image... The feature extraction group's channels are densely connected one by one using a module for concatenating the first feature extraction group in the dimension of channel number. The convolution and copy results of the downsampled images yield a dimension of... The There are bounding box feature maps, among which... Indicates the first The number of channels in a bounding box feature map; The edge detection module is used to perform edge detection processing on the input image based on the CV model to obtain an edge feature map; The convolution module is used to sequentially convolve the combined bounding box feature map and the edge feature map, with a kernel size of [missing value]. The first convolution process and the convolution kernel size are The second convolution process yields the convolution result, wherein the combination result refers to the layer superposition result of the bounding box feature map and the edge feature map; The pooling module is used to perform pooling processing on the convolution processing result to obtain a pooling processing result that flattens the high-dimensional vector into a one-dimensional vector. The output module is used to input each vector in the pooling result into the activation function sigmoid to obtain a binary classification result that corresponds one-to-one with each vector and is used to indicate whether a fire has occurred. Then, the binary classification result with the highest confidence is used as the fire target detection result corresponding to the bounding box feature map and output.

2. The method for detecting fires inside tunnels according to claim 1, characterized in that, The dimension-upgrading layer uses 32 filters and a convolutional kernel size of [size missing]. A convolutional layer with zero padding of 1 and a stride of 1.

3. The method for detecting fires inside tunnels according to claim 1, characterized in that, The downsampling layer uses the same number of filters as the upsampling layer. The convolution kernel size is times, A convolutional layer with 1 zero-padding and a stride of 2.

4. The method for detecting fires inside tunnels according to claim 1, characterized in that, The channel-by-channel dense connection module includes at least one channel-by-channel dense connection sub-module arranged sequentially along the processing direction, wherein the channel-by-channel dense connection sub-module includes a convolutional layer, a copy layer, and a splicing layer respectively. In the first In the feature extraction group, the channels of the densely connected module are sequentially connected along the processing direction. The channel-by-channel convolutional layer of the submodule is used to first perform convolution processing on each channel of the output image of the second front layer, and then concatenate the convolution processing results corresponding one-to-one with each channel in the channel-number dimension to obtain a dimension of The first feature map, where... Representing a positive integer, the second preceding layer output image refers to the image output by a processing layer located before the channel-by-channel convolutional layer in the processing direction. This represents the number of channels in the first feature map; The first The copy layer of the submodule, with each channel densely connected, is used to copy the output image of the second front layer, resulting in a dimension of... The second feature map, wherein, This indicates the number of channels in the second front-layer output image; The first The concatenation layer, which densely connects each channel of the sub-module, is used to concatenate the first feature map and the second feature map along the channel number dimension, resulting in a dimension of... The third feature map.

5. The method for detecting fires inside tunnels according to claim 4, characterized in that, The number of filters used in each channel-wise convolutional layer is the same as the number of filters used in the upscaling layer. The convolution kernel size is times, A convolutional layer with 1 zero-padding and a stride of 1; Alternatively, the channel-by-channel convolutional layer may employ a filter number equal to the number of filters in the up-dimensional layer. The convolution kernel size is times, A convolutional layer with 2 zero-padding elements and a stride of 1.

6. The method for detecting fires inside tunnels according to claim 1, characterized in that, when When the value of is not less than 3, the fire target detection model includes: Each convolutional module Individual pooling modules Each output module and One downsampling module, among which... It means not less than 2 and less than Positive integers; The The first convolutional module in the n convolutional modules is used to process the data generated by the first convolutional module along the processing direction. The bounding box feature maps output by each of the feature extraction groups and the edge feature maps are sequentially convolved with kernels of size [size missing]. The first convolution process and the convolution kernel size are The second convolution process yields the result of the first convolution process; The The first pooling module in the pooling module is used to perform pooling processing on the first convolution processing result to obtain the first pooling processing result that flattens the high-dimensional vector into a one-dimensional vector. The The first output module in the first set of output modules is used to input each vector from the first pooling result into the sigmoid activation function to obtain a binary classification result that corresponds one-to-one with each vector from the first pooling result and represents whether a fire has occurred. Then, the binary classification result with the highest confidence is used as the result along the processing direction. The fire target detection results corresponding to the bounding box feature maps output by the feature extraction groups are then output. The The first downsampling module A downsampling module is used to process the data from the sampled sample. The first convolutional module The convolutional results output by each convolutional module are downsampled to obtain the downsampled result, where... It means not less than 2 and not greater than 2. Positive integers; The The first convolutional module A convolutional module is used to process the first convolutional module along the processing direction. The bounding box feature map output by the first feature extraction group and the bounding box feature map output by the first feature extraction group are compared with those of the second feature extraction group. The combined results of the downsampling processing outputs from each downsampling module are then sequentially convolved with kernels of size [size missing]. The first convolution process and the convolution kernel size are The second convolution process yields the... The result of each convolutional process; The The first pooling module A pooling module is used for the first pooling module. The convolution result is then pooled to obtain the first convolution result, which flattens the high-dimensional vector into a one-dimensional vector. Individual pooling processing results; The The first output module The first output module is used to output the first... Each vector from the pooling result is input into the sigmoid activation function to obtain the result that is the same as the first... Each vector in the pooling results corresponds one-to-one with a binary classification result representing whether a fire has occurred. Then, the binary classification result with the highest confidence is used as the result along the processing direction. The fire target detection results corresponding to the bounding box feature maps output by the feature extraction group are then output.

7. A fire detection device for tunnels, characterized in that, It includes an image acquisition unit, a preprocessing unit, and a detection unit that are sequentially connected by communication. The image acquisition unit is used to acquire on-site images captured by the camera inside the tunnel; The preprocessing unit is used to preprocess the scene image to obtain a dimension of RGB image, where, This indicates the resolution of the RGB image. Represents a positive integer not less than 160. This represents the number of channels in the RGB image; The detection unit is used to input the RGB image into a pre-trained fire target detection model and output the fire target detection result. The fire target detection model includes a feature extraction network, an edge detection module, a convolution module, a pooling module, and an output module. The feature extraction network is used to perform feature extraction processing on the input image, obtaining a feature with dimension . The bounding box feature map, where, , Represents positive integers. This represents the number of channels in the bounding box feature map; The feature extraction network includes an input layer, an upscaling layer, and a sub-layer arranged sequentially along the processing direction. A feature extraction group, wherein the feature extraction group includes a downsampling layer and a channel-by-channel dense connection module arranged sequentially along the processing direction; the dimensionality-upgrading layer is used to perform dimensionality-upgrading processing on the image input by the input layer in the dimension of channel number, to obtain a dimension of The upgraded image, in which, This represents the number of channels in the upscaling image; in the The first feature extraction group and along the processing direction A downsampling layer with 1 feature extraction group is used to downsample the output image of the first front layer, resulting in a dimension of 1. The 1 downsampled image, of which Indicates not greater than A positive integer, wherein the first front-layer output image refers to the image output by a processing layer located before the downsampling layer in the processing direction. This represents the number of channels in the first front-layer output image; the number of channels in the first front-layer output image... The feature extraction group's channels are densely connected one by one using a module for concatenating the first feature extraction group in the dimension of channel number. The convolution and copy results of the downsampled images yield a dimension of... The There are bounding box feature maps, among which... Indicates the first The number of channels in a bounding box feature map; The edge detection module is used to perform edge detection processing on the input image based on the CV model to obtain an edge feature map; The convolution module is used to sequentially convolve the combined bounding box feature map and the edge feature map, with a kernel size of [missing value]. The first convolution process and the convolution kernel size are The second convolution process yields the convolution result, wherein the combination result refers to the layer superposition result of the bounding box feature map and the edge feature map; The pooling module is used to perform pooling processing on the convolution processing result to obtain a pooling processing result that flattens the high-dimensional vector into a one-dimensional vector. The output module is used to input each vector in the pooling result into the activation function sigmoid to obtain a binary classification result that corresponds one-to-one with each vector and is used to indicate whether a fire has occurred. Then, the binary classification result with the highest confidence is used as the fire target detection result corresponding to the bounding box feature map and output.

8. A computer device, characterized in that, The device includes a memory, a processor, and a transceiver connected in sequence for communication. The memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the tunnel fire detection method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that... The computer-readable storage medium stores instructions that, when executed on a computer, perform the tunnel fire detection method as described in any one of claims 1 to 6.

Images