Smoke detection method, device, medium and equipment based on deep learning
By using a deep learning-based approach, frame difference method and deep learning model to extract smoke features and calculate confidence, the problem of difficulty in detecting dynamically changing smoke in existing technologies is solved, and accurate tracking and real-time early warning of smoke are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NORTHEASTERN UNIV CHINA
- Filing Date
- 2024-06-27
- Publication Date
- 2026-07-07
AI Technical Summary
Existing smoke detection methods are difficult to accurately detect dynamically changing smoke and have poor tracking performance, especially in complex process production environments where they are difficult to effectively warn of fires.
A deep learning-based approach is adopted to acquire multiple frames of images to be detected in real time, perform grayscale processing, generate a frame difference mask using the frame difference method, calculate the area of the moving object region, and combine a deep learning model to extract smoke features and calculate confidence to determine the smoke detection result.
It can accurately detect dynamically changing smoke, enabling effective smoke tracking and real-time early warning, thus improving the reliability of fire early warning.
Smart Images
Figure CN118887410B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of smoke image recognition technology, and in particular to a smoke detection method, apparatus, medium and device based on deep learning. Background Technology
[0002] In industrial production, some raw materials (such as neodymium iron boron powder) have spontaneous combustion properties in the air. When the concentration reaches a certain level after the ambient temperature is slightly high, they will spontaneously combust. Once spontaneously combusted, they are difficult to extinguish quickly and can easily cause major accidents and significant losses. Therefore, it is crucial to detect early signs of fire and take preventive measures in advance.
[0003] Smoke, temperature, and gases are important early-stage characteristics of fires and can all serve as a basis for fire early warning. Traditional fire early warning systems mainly rely on smoke sensors to detect smoke. This detection method is generally limited by space and smoke concentration, and is particularly difficult to provide effective early warning in complex industrial environments.
[0004] To address the problems of traditional smoke detectors, researchers have proposed using images or videos to identify smoke and thus provide early warnings of fires. Currently, image-based smoke detection methods include local binary pattern recognition, wavelet transform, and optical flow methods. These methods detect smoke by extracting features such as texture and color. However, these methods generally suffer from the following problem: because smoke undergoes a dynamic process of accumulation, rising, wind diffusion, and dissipation, its texture and color also change during this process. Since these smoke detection methods use fixed algorithms and rely on fixed smoke texture and color features for detection, they cannot accurately detect dynamically changing smoke and exhibit poor smoke tracking performance. Summary of the Invention
[0005] In view of this, the present invention provides a smoke detection method, device, medium and equipment based on deep learning, the main purpose of which is to solve the problem that current smoke detection methods cannot accurately detect dynamically changing smoke and have poor smoke tracking performance.
[0006] According to one aspect of this application, a deep learning-based smoke detection method is provided, the method comprising:
[0007] The system acquires multiple frames of images of the target area captured continuously in real time, performs grayscale processing on each frame of the image to be detected to obtain multiple grayscale images, and takes a first preset number of grayscale images as a grayscale image set according to the shooting order of the images to be detected corresponding to the grayscale images to obtain multiple grayscale image sets.
[0008] Each grayscale image set is subjected to frame difference processing to obtain a frame difference mask map corresponding to each grayscale image set. The second preset number of frame difference mask maps are combined into a mask map set according to the order of the grayscale image sets corresponding to the frame difference mask map to obtain multiple mask map sets, and multiple images to be detected corresponding to each mask map set are obtained.
[0009] Calculate the area of the moving object region in each frame difference mask in each mask set, and based on the area of multiple moving object regions in each mask set, determine whether there is a potential smoke region in the multiple images to be detected corresponding to each mask set.
[0010] When a potential smoke region exists in multiple images to be detected corresponding to any mask set, smoke features are extracted from the multiple images to be detected corresponding to the mask set to obtain multiple smoke feature maps.
[0011] Based on the multiple smoke feature maps and the preset deep learning model, the smoke confidence score of each frame of the image to be detected corresponding to any mask map set is calculated, and the smoke detection result is determined according to the smoke confidence score.
[0012] Optionally, the step of performing frame difference processing on each grayscale image set to obtain a frame difference mask map corresponding to each grayscale image set includes:
[0013] Based on the first and last grayscale images in the grayscale image set, calculate the pixel value difference for each pixel.
[0014] A frame difference mask is generated based on the pixel value difference and pixel threshold of each pixel. In the frame difference mask, pixels with a pixel value difference greater than the pixel threshold are set to white, and pixels with a pixel value difference less than or equal to the pixel threshold are set to black.
[0015] Optionally, the step of calculating the area of the moving object region in each frame difference mask in each mask set, and determining whether there is a potential smoke region in the multiple images to be detected corresponding to each mask set based on the areas of multiple moving object regions in each mask set, includes:
[0016] Based on the color of the pixels in the frame difference mask, calculate the area of the moving object region in each frame difference mask in each mask set;
[0017] Based on the area of multiple moving object regions in each mask set, obtain the number of frame difference mask images in each mask set whose moving object regions' areas fall within the area range;
[0018] Based on the number of frame difference mask images, determine whether there are potential smoke regions in the multiple images to be detected corresponding to each mask image set.
[0019] Optionally, determining whether potential smoke regions exist in the multiple images to be detected corresponding to each mask set based on the number of frame difference mask images includes:
[0020] When the number of frame difference mask images is greater than or equal to the number threshold, then the multiple images to be detected corresponding to the mask image set have potential smoke regions.
[0021] When the number of frame difference mask images is less than the number threshold, then the multiple images to be detected corresponding to the mask image set do not have potential smoke regions.
[0022] Optionally, the step of extracting smoke features from multiple images to be detected corresponding to any one of the mask image sets to obtain multiple smoke feature maps includes:
[0023] The image to be detected is convolved based on two-dimensional convolution to obtain a first smoke feature map;
[0024] The first smoke feature map is convolved based on the first dilated convolution to obtain the second smoke feature map. The second smoke feature map is then processed based on the channel attention mechanism to obtain the channel attention feature map.
[0025] The first smoke feature map is convolved based on the second dilated convolution to obtain the third smoke feature map. The third smoke feature map is then processed based on the spatial attention mechanism to obtain the spatial attention feature map. The number of channels in the first dilated convolution and the second dilated convolution are different.
[0026] The channel attention feature map and the spatial attention feature map are fused to obtain the smoke feature map.
[0027] Optionally, determining the smoke detection result based on the smoke confidence level includes:
[0028] When the confidence level of the smoke in the image to be detected is greater than or equal to the confidence threshold, it is determined that there is smoke in the image to be detected.
[0029] Optionally, after determining that smoke exists in the image to be detected, the smoke detection method further includes:
[0030] Obtain the region of interest and frame difference mask corresponding to the image to be detected, and enhance the smoke region in the image to be detected based on the corresponding region of interest and frame difference mask.
[0031] Optionally, the step of performing grayscale processing on each frame of the image to be detected to obtain multiple grayscale images includes:
[0032] Preliminary smoke detection is performed on each frame of the image to be detected to obtain the region of interest corresponding to each frame of the image to be detected.
[0033] Perform grayscale transformation on each region of interest to obtain the grayscale image corresponding to each region of interest.
[0034] According to another aspect of this application, a smoke detection device based on deep learning is provided, comprising:
[0035] The grayscale image set acquisition module is used to acquire multiple frames of images to be detected of the target area captured continuously in real time, perform grayscale processing on each frame of the image to be detected to obtain multiple grayscale images, and take a first preset number of grayscale images as a grayscale image set according to the shooting order of the images to be detected corresponding to the grayscale images to obtain multiple grayscale image sets.
[0036] The mask image set acquisition module is used to perform frame difference processing on each grayscale image set to obtain the frame difference mask image corresponding to each grayscale image set. The second preset number of frame difference mask images are used as a mask image set according to the order of the grayscale image sets corresponding to the frame difference mask images to obtain multiple mask image sets, and multiple images to be detected corresponding to each mask image set are acquired.
[0037] The smoke region determination module is used to calculate the area of the moving object region in each frame difference mask in each mask set, and to determine whether there is a potential smoke region in the multiple images to be detected corresponding to each mask set based on the area of multiple moving object regions in each mask set.
[0038] The smoke feature map acquisition module is used to extract smoke features from the multiple images to be detected corresponding to any mask set when there is a potential smoke region in the multiple images to be detected corresponding to any mask set, and obtain multiple smoke feature maps.
[0039] The detection result determination module is used to calculate the smoke confidence score of each frame of the image to be detected corresponding to any mask image set based on the multiple smoke feature maps and the preset deep learning model, and determine the smoke detection result based on the smoke confidence score.
[0040] Optionally, the mask atlas acquisition module is further configured to:
[0041] Based on the first and last grayscale images in the grayscale image set, calculate the pixel value difference for each pixel.
[0042] A frame difference mask is generated based on the pixel value difference and pixel threshold of each pixel. In the frame difference mask, pixels with a pixel value difference greater than the pixel threshold are set to white, and pixels with a pixel value difference less than or equal to the pixel threshold are set to black.
[0043] Optionally, the smoke area determination module is further configured to:
[0044] Based on the color of the pixels in the frame difference mask, calculate the area of the moving object region in each frame difference mask in each mask set;
[0045] Based on the area of multiple moving object regions in each mask set, obtain the number of frame difference mask images in each mask set whose moving object regions' areas fall within the area range;
[0046] The presence of potential smoke regions in multiple images to be detected corresponding to each mask set is determined based on the number of frame difference mask images.
[0047] Optionally, the smoke area determination module is further configured to:
[0048] When the number of frame difference mask images is greater than or equal to the number threshold, then the multiple images to be detected corresponding to the mask image set have potential smoke regions.
[0049] When the number of frame difference mask images is less than the number threshold, then the multiple images to be detected corresponding to the mask image set do not have potential smoke regions.
[0050] Optionally, the smoke feature map acquisition module is further configured to:
[0051] The image to be detected is convolved based on two-dimensional convolution to obtain a first smoke feature map;
[0052] The first smoke feature map is convolved based on the first dilated convolution to obtain the second smoke feature map. The second smoke feature map is then processed based on the channel attention mechanism to obtain the channel attention feature map.
[0053] The first smoke feature map is convolved based on the second dilated convolution to obtain the third smoke feature map. The third smoke feature map is then processed based on the spatial attention mechanism to obtain the spatial attention feature map. The number of channels in the first dilated convolution and the second dilated convolution are different.
[0054] The channel attention feature map and the spatial attention feature map are fused to obtain the smoke feature map.
[0055] Optionally, the detection result determination module is further configured to:
[0056] When the confidence level of the smoke in the image to be detected is greater than or equal to the confidence threshold, it is determined that there is smoke in the image to be detected.
[0057] Optionally, the detection result determination module is further configured to:
[0058] After determining that smoke exists in the image to be detected, the region of interest and frame difference mask map corresponding to the image to be detected are obtained. Based on the corresponding region of interest and frame difference mask map, the smoke region in the image to be detected is enhanced.
[0059] Optionally, the grayscale image set acquisition module is further configured to:
[0060] Preliminary smoke detection is performed on each frame of the image to be detected to obtain the region of interest corresponding to each frame of the image to be detected.
[0061] Perform grayscale transformation on each region of interest to obtain the grayscale image corresponding to each region of interest.
[0062] According to another aspect of this application, a storage medium is provided that stores at least one executable instruction, which causes a processor to perform the operation corresponding to the deep learning-based smoke detection method described above.
[0063] According to another aspect of this application, a computer device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus;
[0064] The memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the deep learning-based smoke detection method described above.
[0065] By employing the above-described technical solutions, the technical solutions provided by the embodiments of the present invention have at least the following advantages:
[0066] This application provides a smoke detection method, apparatus, device, and medium based on deep learning. It performs grayscale processing on multiple frames of continuously captured images of a target area to obtain multiple grayscale image sets. A frame difference mask is generated for each grayscale image set using the frame difference method. The frame difference mask reflects the moving object region. The multiple frame difference masks are divided into multiple mask sets. The area of the moving object region in each frame difference mask set is calculated. Based on the areas of multiple moving object regions in each mask set, it is determined whether a potential smoke region exists in the image to be detected corresponding to any mask set. When a potential smoke region exists in the image to be detected corresponding to any mask set, the smoke detection method is applied to that mask set. Smoke features are extracted from multiple images to be detected corresponding to a mask image set, resulting in multiple smoke feature maps. Each smoke feature map is input into a preset deep learning model to calculate the smoke confidence score of each frame of the image to be detected. Based on the smoke confidence score, the smoke detection result of each frame of the image to be detected is determined. Since the frame difference mask image can reflect the region of moving objects and can capture and track regions with dynamic features, including smoke, the area of the moving object region in multiple frame difference masks is used to determine the existence of potential smoke regions. By performing smoke feature extraction and smoke detection on the images to be detected containing smoke regions, dynamic smoke can be accurately detected and smoke can be effectively tracked.
[0067] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description
[0068] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:
[0069] Figure 1 A flowchart of a deep learning-based smoke detection method provided in an embodiment of this application is shown;
[0070] Figure 2 A flowchart of another deep learning-based smoke detection method provided in an embodiment of this application is shown;
[0071] Figure 3 This image shows a frame of a smoke detection method based on deep learning, as provided in an embodiment of this application.
[0072] Figure 4This shows another frame of an image to be detected, illustrating a deep learning-based smoke detection method provided in an embodiment of this application.
[0073] Figure 5 The image shows a grayscale image of the region of interest in a deep learning-based smoke detection method provided in an embodiment of this application.
[0074] Figure 6 The image shows a frame difference mask of a smoke detection method based on deep learning provided in an embodiment of this application;
[0075] Figure 7 This illustration shows a smoke feature attention module of a deep learning-based smoke detection method provided in an embodiment of this application.
[0076] Figure 8 The image shown is an enhanced image of a smoke region obtained using a deep learning-based smoke detection method according to an embodiment of this application.
[0077] Figure 9 A flowchart illustrating the model deployment of a deep learning-based smoke detection method provided in an embodiment of this application is shown.
[0078] Figure 10 The test results of a smoke detection method based on deep learning provided in an embodiment of this application are shown.
[0079] Figure 11 This paper illustrates a block diagram of a smoke detection device based on deep learning, as provided in an embodiment of this application.
[0080] Figure 12 A schematic diagram of the structure of a computer device provided in an embodiment of the present invention is shown.
[0081] in,
[0082] Figure 11 In the middle: 1102-Grayscale image set acquisition module; 1104-Mask image set acquisition module; 1106-Smoke region judgment module; 1108-Smoke feature map acquisition module; 1110-Detection result determination module;
[0083] Figure 12 In Chinese: 1202 - Processor; 1204 - Communication interface; 1206 - Memory; 1208 - Communication bus; 1210 - Program. Detailed Implementation
[0084] The present invention will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present invention can be combined with each other.
[0085] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the specific embodiments, structures, features, and effects according to the present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments. In the following description, different "an embodiment" or "an embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.
[0086] To address the shortcomings of current smoke detection methods, such as their inability to accurately detect dynamically changing smoke and poor smoke tracking performance, this application provides a smoke detection method based on deep learning. Figure 1 As shown, the method includes:
[0087] 102: Real-time acquisition of multiple frames of images to be detected of the target area captured continuously, grayscale processing of each frame of the image to be detected to obtain multiple grayscale images, and according to the shooting order of the images to be detected corresponding to the grayscale images, a first preset number of grayscale images are used as a grayscale image set to obtain multiple grayscale image sets.
[0088] 104: Perform frame difference processing on each grayscale image set to obtain the frame difference mask map corresponding to each grayscale image set. Take the second preset number of frame difference mask maps as a mask map set according to the order of the grayscale image sets corresponding to the frame difference mask maps to obtain multiple mask map sets, and obtain multiple images to be detected corresponding to each mask map set.
[0089] 106: Calculate the area of the moving object region in each frame difference mask in each mask set, and based on the area of multiple moving object regions in each mask set, determine whether there is a potential smoke region in the multiple images to be detected corresponding to each mask set.
[0090] 108: When there is a potential smoke region in multiple images to be detected corresponding to any mask set, smoke features are extracted from the multiple images to be detected corresponding to any mask set to obtain multiple smoke feature maps;
[0091] 110: Calculate the smoke confidence score of each frame of the image to be detected corresponding to any mask image set based on multiple smoke feature maps and a preset deep learning model, and determine the smoke detection result based on the smoke confidence score.
[0092] Specifically, the target area is continuously captured in real time to obtain multiple consecutive multi-frame images to be detected. Alternatively, video can be captured of the target area to obtain multiple consecutive multi-frame images to be detected. Each consecutive multi-frame image to be detected is processed into a grayscale image. These grayscale images are grouped together according to the order in which the images to be detected were captured, resulting in multiple grayscale image sets. A frame difference mask is generated for each grayscale image set using the frame difference method. The frame difference mask reflects the moving object region. A second preset number of frame difference masks are grouped together according to the order of the grayscale image sets corresponding to each frame difference mask, resulting in multiple mask sets. The area of the moving object region in each frame difference mask within each mask set is calculated. The area of multiple moving object regions in the mask set is used to determine whether there is a potential smoke region in the image to be detected corresponding to the mask set. When there is a potential smoke region in the image to be detected corresponding to any mask set, smoke features are extracted from multiple images to be detected corresponding to any mask set to obtain multiple smoke feature maps. Each smoke feature map is input into a preset deep learning model to calculate the smoke confidence of the image to be detected corresponding to each smoke feature map. Based on the smoke confidence of the image to be detected corresponding to each smoke feature map, the smoke detection result of the image to be detected corresponding to each smoke feature map is determined.
[0093] This application provides a smoke detection method based on deep learning. Compared with the prior art, since the frame difference mask can reflect the region of moving objects, it can capture and track regions with dynamic features, including smoke. Therefore, the existence of potential smoke regions is determined based on the area of the moving object region of multiple frame difference masks. Smoke feature extraction and smoke detection are performed on the image to be detected containing smoke regions, which can accurately detect dynamically changing smoke and effectively track smoke.
[0094] In one embodiment of the present invention, grayscale processing is performed on each frame of the image to be detected to obtain multiple grayscale images, including:
[0095] Preliminary smoke detection is performed on each frame of the image to be detected to obtain the region of interest corresponding to each frame of the image to be detected.
[0096] Perform grayscale transformation on each region of interest to obtain the grayscale image corresponding to each region of interest.
[0097] In this embodiment, multiple consecutive frames of images to be detected can be obtained by loading video images, or by continuously capturing images. Preliminary smoke recognition is performed on each image, and the region of interest (ROI) corresponding to each frame is determined based on the preliminary smoke recognition results. Subsequent precise smoke detection is then performed based on the ROI, removing interference and improving the smoke recognition effect. For example, existing smoke detection methods can be used to perform preliminary smoke recognition on the images to be detected.
[0098] The region of interest can be represented as:
[0099] I = {I1, I2, ..., I} k ,...} (1)
[0100]
[0101] Where I is a sequence of consecutive image frames to be detected, and k represents the number of frames. k That is, the k-th frame image; I r Indicates from I k The region of interest (ROI) is obtained by cropping, where (x,y) represents the pixel coordinates in the ROI image, and (x′,y′) represents I. r In the corresponding I k The pixel coordinates (x1, y1) and (x2, y2) in the image are respectively I k The coordinates of the top left and bottom right corners of the selected region of interest (usually a rectangle).
[0102] The process of obtaining a grayscale image of the region of interest, converting the region of interest image to grayscale, and storing its adjacent frame images as a grayscale image set can be represented as follows:
[0103] V g ={g r-L+1 ,g r-L+2 ,g r-L+3 ,...,g r} (3)
[0104]
[0105] Where V g A set of grayscale images consisting of grayscale images of regions of interest in L consecutive frames over a time period, where L is the length of the time window; g r isI r grayscale image, These are video sequences I r The B, G, and R channels are represented by α1, α2, and α3, which are the respective channel weights in the BGR color space.
[0106] For example, the grayscale images of the regions of interest corresponding to the images to be detected from the first frame to the Lth frame can be used as one grayscale image set, and the grayscale images of the regions of interest corresponding to the images to be detected from the second frame to the L+1th frame can be used as another grayscale image set. In this way, multiple grayscale image sets can be generated.
[0107] In one embodiment of the present invention, frame difference processing is performed on each grayscale image set to obtain a frame difference mask map corresponding to each grayscale image set, including:
[0108] Calculate the pixel value difference for each pixel based on the first and last grayscale images in the grayscale image set;
[0109] A frame difference mask is generated based on the pixel value difference and pixel threshold of each pixel. In the frame difference mask, pixels with a pixel value difference greater than the pixel threshold are set to white, and pixels with a pixel value difference less than or equal to the pixel threshold are set to black.
[0110] Specifically, the frame difference method detects moving targets by comparing the pixel value differences between adjacent images in a sequence of multiple consecutive images. By subtracting the pixel values of two frames and taking the absolute value of the brightness difference, it is possible to determine whether this value is greater than a threshold to analyze the motion characteristics of the video or image sequence, thereby determining whether there is moving objects in the image sequence.
[0111] In this embodiment, frame difference processing is performed on the first and last grayscale images in the grayscale image set. The pixel value difference of each pixel in these two grayscale images is calculated. Pixels with a pixel value difference greater than a pixel threshold are set to white, and pixels with a pixel value difference less than or equal to the pixel threshold are set to black, thus obtaining a frame difference mask image. In the frame difference mask image, the white pixel areas are the moving object areas, or in other words, the pixel value areas greater than 0 are the moving object areas. The process can be represented as follows:
[0112]
[0113] Where m r This is a frame difference mask image corresponding to a set of grayscale images. It is a binary image, and T represents the pixel threshold. If the difference between corresponding pixels in two images is greater than the pixel threshold, it indicates that there is a moving object in the region.
[0114] In one embodiment, the area of the moving object region in each frame difference mask in each mask set is calculated. Based on the areas of multiple moving object regions in each mask set, it is determined whether there are potential smoke regions in the multiple images to be detected corresponding to each mask set, including:
[0115] Based on the color of the pixels in the frame difference mask, calculate the area of the moving object region in each frame difference mask in each mask set;
[0116] Based on the area of multiple moving object regions in each mask set, obtain the number of frame difference masks in each mask set whose moving object regions' areas fall within the area range;
[0117] The presence of potential smoke regions in multiple images to be detected corresponding to each mask set is determined based on the number of frame difference mask images.
[0118] Specifically, the white areas in the frame difference mask represent the moving object region. Calculating the area of the white areas in the frame difference mask yields the size of the moving object region. Based on the order of the corresponding grayscale image sets, multiple frame difference masks are divided into multiple mask sets. The area of the moving object region in each frame difference mask within each mask set is calculated. This process involves calculating and storing the area of the moving object region in the frame difference masks within multiple consecutive video time windows. This can be represented as:
[0119] V A ={A r-q+1 A r-q+2 A r-q+3 ,...,A r} (6)
[0120] A i =ΣΣm i (x,y),i=r,r+1,...,r+q-1 (7)
[0121] Where V A The area of the moving object region in the frame difference mask over a consecutive time window of length q; the area A of each moving object region. i From frame difference mask image m i The value is obtained by summing the number of regions with pixel values greater than 0.
[0122] Then, the dynamic characteristics of the smoke are analyzed. Specifically, if the area of the moving object region in the frame difference mask fluctuates within a certain range over a period of time, these regions are more likely to be smoke. The process can be represented as follows:
[0123] V S ={S r-q+1 ,S r-q+1 ,S r-q+2 ,...,S r} (8)
[0124]
[0125] Where V S Let A represent the region where smoke may exist within a consecutive time window of length q. Since the dynamic range of smoke changes is usually small, if the area of the moving object is A...i If the area is within a certain area threshold (T0 and T1), then the area of the moving object is considered to be more likely to be smoke.
[0126] In a mask set, the number of frame difference masks whose area of the moving object region falls within the area range is obtained. It is then determined whether the number of frame difference masks is greater than or equal to a threshold. If the number of frame difference masks is greater than or equal to the threshold, then multiple images to be detected corresponding to the mask set contain potential smoke regions. If the number of frame difference masks is less than the threshold, then multiple images to be detected corresponding to the mask set do not contain potential smoke regions.
[0127] For example, in a mask set with 40 frame difference masks, if the number of frame difference masks whose area falls within the area range of the moving object region is greater than or equal to 20, then the target region is considered to have a potential smoke region.
[0128] In one embodiment, smoke features are extracted from multiple images to be detected corresponding to any mask set to obtain multiple smoke feature maps, including:
[0129] The image to be detected is convolved using two-dimensional convolution to obtain the first smoke feature map.
[0130] The first smoke feature map is convolved based on the first dilated convolution to obtain the second smoke feature map. The second smoke feature map is then processed based on the channel attention mechanism to obtain the channel attention feature map.
[0131] The first smoke feature map is convolved based on the second dilated convolution to obtain the third smoke feature map. The third smoke feature map is then processed based on the spatial attention mechanism to obtain the spatial attention feature map. The number of channels in the first dilated convolution and the second dilated convolution are different.
[0132] By fusing the channel attention feature map and the spatial attention feature map, a smoke feature map is obtained.
[0133] In this embodiment, to accurately identify smoke, a Smoke Feature Attention Module (SFAM) is used to extract smoke features from the image to be detected, resulting in a smoke feature map. It should be noted that the Smoke Feature Attention Module can be configured independently or integrated into a pre-trained deep learning model, offering plug-and-play functionality. This module comprises channel attention and spatial attention, with the input image to be detected I... r First, a first feature map F is obtained through two-dimensional convolution. Then, two dilated convolutions with different channel numbers are used to convolve the first feature map F to obtain a second feature map F with a larger receptive field. c and the third feature map F sThen the second feature map F c Channel attention mechanism processing is performed to obtain channel feature map F. c ′, for the third feature map F s进行 Spatial attention mechanism processing yields spatial feature map F. s Finally, the channel feature map F c Spatial Feature Map F s A new smoke feature map F′ is obtained through feature fusion, and this smoke feature map F′ is input into a pre-trained deep learning model. The smoke feature extraction process can be represented as follows:
[0134] F′=F c ′⊕F s ′ (11)
[0135]
[0136] Where σ represents the activation function, δ represents the ReLU function, w1 and w2 represent the weights, and F N (i,j) represents the output feature map F. c or F s The positions on the graph, x and y, represent the convolution kernel k. N The relative position N on the input feature map F is equal to N. c or N s r represents the dilation rate, F(x,y) represents the position on the input feature map F, and i and j represent the positions of the convolution kernel k on the input image I. r The relative positions on the image are: H represents the length of the image, W represents the width of the image, and C represents the number of channels.
[0137] By extracting smoke features through a smoke feature attention module, better smoke tracking results are achieved and interference from environmental factors such as lighting is eliminated.
[0138] In one embodiment of the present invention, determining the smoke detection result based on the smoke confidence level includes:
[0139] When the confidence level of smoke in the image to be detected is greater than or equal to the confidence threshold, it is determined that smoke exists in the image to be detected.
[0140] In one embodiment of the present invention, after determining that smoke exists in the image to be detected, the smoke detection method further includes:
[0141] Obtain the region of interest and frame difference mask corresponding to the image to be detected. Based on the corresponding region of interest and frame difference mask, perform enhancement processing on the smoke region in the image to be detected.
[0142] Specifically, if a certain number of potential smoke regions exist, smoke features are extracted from the current region of interest frame to obtain a smoke feature map. This smoke feature map is then fed into a trained deep learning model to calculate the smoke confidence score. The process can be represented as follows:
[0143]
[0144] Where P(Y) represents the confidence level that smoke exists in the current image; f is the activation function; w and b represent the network weights and biases of the model, respectively; T s As a threshold, if V S There is a value greater than T s If there is a region where smoke may exist, the current frame is fed into the model for inference to obtain the smoke confidence level.
[0145] Determining whether smoke exists in the current frame based on the smoke confidence level can be represented as follows:
[0146]
[0147] Where y is the final result of smoke recognition, 1 indicates the presence of smoke, and 0 indicates the absence of smoke; k is the confidence threshold.
[0148] In one embodiment, if the current frame is determined to contain smoke, the previously obtained frame difference mask map and region of interest map are used to enhance the image to be detected, thereby achieving real-time smoke tracking. This process can be represented as follows:
[0149]
[0150] Among them I k ′ is the enhanced current frame image, which is composed of the enhanced region of interest frame image I. r Let (x, y) represent the pixel coordinates in the enhanced current frame image, and (x′, y′) represent I. r The corresponding I in ′ k The pixel coordinates in ' are (x1, y1) and (x2, y2), which are the top-left and bottom-right corner coordinates of the region of interest (usually a rectangle) selected in step 1, respectively; the enhanced region of interest frame image I r From the original I r The blue channel, green channel, and frame difference mask image m are obtained by channel merging.
[0151] In one embodiment, a real-time smoke alarm processing module is used to improve the real-time response speed of smoke detection. The above algorithm is encapsulated into a class according to its function and processing steps, and then ported to a C++ environment. To improve the model inference speed, TensorRT (a software development kit for high-performance deep learning model inference) is used to deploy the model. The model is converted and then imported into the inference engine and encapsulated into a dynamic link library for external programs to perform inference output.
[0152] In one embodiment, a rare earth recycling and reuse company uses multiple industrial cameras at its raw material site to monitor rare earth waste in different raw material areas in real time. The company obtains real-time video footage from each industrial camera in the raw material area via a local area network. A portion of the industrial video images showing smoke and fire are selected. The video image data dimensions are two spatial dimensions: horizontal and vertical. The resolution is 2560×1440, and each frame is an 8-bit three-channel color image. The video frame rate is 30 frames per second. To achieve real-time response speed for smoke detection, downsampling is performed when processing each frame, with a downsampling rate of 4.
[0153] Smoke detection is performed on the sampled image frames, such as... Figure 2 As shown, the specific process of smoke detection based on deep learning is as follows:
[0154] Step 1: Load the video image and select the region of interest for detection, such as... Figure 3 and Figure 4 To load two frames from a video, where Figure 3 For video frames under normal conditions (i.e., without smoke), Figure 4 For video frames in an abnormal state (i.e., with smoke), the box indicates the selected region of interest;
[0155] Step 2: Obtain grayscale images of adjacent region-of-interest (ROI) frames. Convert the ROI frames to grayscale images and set the video time window length to 30, storing 30 consecutive adjacent ROI frames. One ROI frame's grayscale image is shown below. Figure 5 As shown;
[0156] Step 3: Obtain the frame difference mask. From the 30 frames stored in Step 2, select the first and last frames and perform frame difference analysis. Pixels with a difference greater than 25 are white; otherwise, they are black. This yields the mask image, as shown below. Figure 6 As shown, the white areas in the mask image represent the areas of the moving object;
[0157] Step 4: Calculate the area of the moving object region. Based on Step 3, obtain 40 consecutive frame difference mask images, calculate the area of the moving object region in them, and store them.
[0158] Step 5: Smoke dynamic feature analysis. Determine whether the area of each of the 40 moving object regions stored in Step 4 is within a certain threshold range (35-500). If so, the moving object region is considered a potential smoke region.
[0159] Step 6: Calculate the smoke confidence score. If there are 20 potential smoke regions identified in Step 5, the current region of interest frame is fed into a trained deep learning classification model for smoke recognition. The training set used by the deep learning classification model consists of a public dataset (containing 8022 images of smoke and flames and 900 images without smoke) and video frames collected on-site (containing 4513 images of smoke and 21316 images without smoke). To accurately identify smoke, a plug-and-play smoke feature attention module (SFAM) is added to the trained deep learning model, such as... Figure 7 As shown, this module consists of two parts: channel attention and spatial attention. The input image I r First, a feature map F is obtained through two-dimensional convolution, where the kernel size is 3 and the number of channels is 16. Then, two dilated convolutions with different numbers of channels are used to obtain feature maps F with larger receptive fields. c and F s Here, the void ratio is set to 2, and the number of channels is set to 32 and 48 respectively; then, channel attention and spatial attention are used to obtain the channel and spatial distribution feature maps F with different importance. c ′ and F s Finally, a new feature map F′ is obtained through feature fusion and input into the pre-trained residual network.
[0160] The channel attention first applies the feature map F c Perform global average pooling to generate 1×1×N c (N c The vector is 32), and then passed through two fully connected layers to obtain a weighted feature vector, which is then used to modify the feature map F. c The new feature map is obtained by assigning values, and finally, the new feature map is subjected to dilated convolution to obtain F. c Spatial attention, on the other hand, first processes the feature map F... s Perform a reshape operation to obtain N. s ×HW(N s The feature vector is 48). This feature vector is then transposed and multiplied by the original vector to obtain a feature map of size HW×H×W. Finally, this feature map is used to assign weights to the original feature map F to obtain F. s ′.
[0161] The backbone network of the deep learning model uses ResNet (whose residual structure can obtain deeper semantic information). It is first pre-trained using a public dataset, and then further trained on video frames captured on-site. During training, the model uses the Adam optimization algorithm with a learning rate of 0.0001. The network is trained for 100,000 iterations, and the loss function used is cross-entropy loss, whose loss function L can be expressed as:
[0162]
[0163] Among them, P i (Y) represents the predicted probability value of sample i. The label value of sample i is represented by 1 if there is smoke in the image and 0 if there is no smoke, and N is the number of training samples;
[0164] Step 7: Based on the smoke confidence level in Step 6, if the smoke confidence level identified by the deep learning classification model is greater than 90%, then it is considered that there is smoke in the current frame.
[0165] Step 8: If smoke is present in the current frame, the frame difference mask obtained in Step 3 and the region of interest obtained in Step 1 are used to enhance the current frame in the original video, thereby achieving real-time smoke tracking. Figure 8 As shown, the smoke area is enhanced using a mask;
[0166] Step 9: Considering the impact of deep learning model inference time, to improve the real-time response speed of smoke detection, the above algorithm is first encapsulated into a class according to its function and processing steps, and then ported to a C++ environment. To further improve the model inference speed, TensorRT is used to deploy the model. By converting the model and using the created builder, network, and parser, the converted model is imported into the inference engine. Then, the image to be recognized is imported into the model for inference output. To ensure real-time performance and portability, this invention encapsulates the entire model loading and inference process into a dynamic link library for external programs to call. The model deployment process is as follows: Figure 9 As shown.
[0167] After the above deployment, practical results show that this invention enables real-time dynamic detection of smoke, with a fast detection speed and applicability to actual fire early warning. Test results based on 277.53 hours of video data collected on-site indicate that it can provide a warning at least 10 minutes before flames appear, and the false alarm rate is extremely low, only 0.012%. Figure 10 As shown.
[0168] Practical results show that the present invention has a fast real-time smoke detection speed and can be applied to fire early warning in actual industrial production environments. It can provide a forecast at least 10 minutes before the flame is generated, and the smoke false alarm rate is extremely low, only 0.012%.
[0169] Furthermore, as a response to the above Figure 1 The implementation of the method shown in this invention provides a smoke detection device based on deep learning, such as... Figure 11 As shown, the device includes:
[0170] The grayscale image set acquisition module 1102 is used to acquire multiple frames of images to be detected of the target area captured continuously in real time, perform grayscale processing on each frame of the image to be detected to obtain multiple grayscale images, and take a first preset number of grayscale images as a grayscale image set according to the shooting order of the images to be detected corresponding to the grayscale images to obtain multiple grayscale image sets.
[0171] The mask image set acquisition module 1104 is used to perform frame difference processing on each grayscale image set to obtain the frame difference mask image corresponding to each grayscale image set. The second preset number of frame difference mask images are used as a mask image set according to the order of the grayscale image sets corresponding to the frame difference mask images to obtain multiple mask image sets, and multiple images to be detected corresponding to each mask image set are acquired.
[0172] The smoke region determination module 1106 is used to calculate the area of the moving object region of each frame difference mask in each mask set, and determine whether there is a potential smoke region in the multiple images to be detected corresponding to each mask set based on the area of multiple moving object regions in each mask set.
[0173] The smoke feature map acquisition module 1108 is used to extract smoke features from multiple images to be detected corresponding to any mask set when there is a potential smoke region in multiple images to be detected corresponding to any mask set, and obtain multiple smoke feature maps.
[0174] The detection result determination module 1110 is used to calculate the smoke confidence of each frame of the image to be detected corresponding to any mask image set based on multiple smoke feature maps and a preset deep learning model, and determine the smoke detection result based on the smoke confidence.
[0175] This application provides a smoke detection device based on deep learning. Compared with existing technologies, it performs grayscale processing on multiple frames of continuously captured images of a target area to obtain multiple grayscale image sets. A frame difference mask image is generated for each grayscale image set using the frame difference method. The frame difference mask image can reflect the moving object region. Multiple frame difference masks are divided into multiple mask sets. The area of the moving object region in each frame difference mask image in each mask set is calculated. Based on the areas of multiple moving object regions in each mask set, it is determined whether there is a potential smoke region in the image to be detected corresponding to any mask set. When a potential smoke region exists in the image to be detected corresponding to any mask set, the device detects the smoke region. Smoke features are extracted from multiple images to be detected corresponding to any mask set, resulting in multiple smoke feature maps. Each original object feature map is input into a preset deep learning model to calculate the smoke confidence score of each frame of the image to be detected. Based on the smoke confidence score, the smoke detection result of each frame of the image to be detected is determined. Since the frame difference mask can reflect the region of moving objects, it can capture and track regions with dynamic features, including smoke. Therefore, the existence of potential smoke regions is determined based on the area of the moving object region in multiple frame difference masks. Smoke features are extracted and smoke is detected from the images to be detected containing smoke regions, which can accurately detect dynamically changing smoke and effectively track smoke.
[0176] In one embodiment, the mask atlas acquisition module is further configured to:
[0177] Calculate the pixel value difference for each pixel based on the first and last grayscale images in the grayscale image set;
[0178] A frame difference mask is generated based on the pixel value difference and pixel threshold of each pixel. In the frame difference mask, pixels with a pixel value difference greater than the pixel threshold are set to white, and pixels with a pixel value difference less than or equal to the pixel threshold are set to black.
[0179] In one embodiment, the smoke area determination module is further configured to:
[0180] Based on the color of the pixels in the frame difference mask, calculate the area of the moving object region in each frame difference mask in each mask set;
[0181] Based on the area of multiple moving object regions in each mask set, obtain the number of frame difference masks in each mask set whose moving object regions' areas fall within the area range;
[0182] The presence of potential smoke regions in multiple images to be detected corresponding to each mask set is determined based on the number of frame difference mask images.
[0183] In one embodiment, the smoke area determination module is further configured to:
[0184] When the number of frame difference mask images is greater than or equal to the number threshold, there are potential smoke regions in multiple images to be detected corresponding to the mask image set.
[0185] When the number of frame difference mask images is less than the number threshold, there are no potential smoke regions in the multiple images to be detected corresponding to the mask image set.
[0186] In one embodiment, the smoke feature map acquisition module is further configured to:
[0187] The image to be detected is convolved using two-dimensional convolution to obtain the first smoke feature map.
[0188] The first smoke feature map is convolved based on the first dilated convolution to obtain the second smoke feature map. The second smoke feature map is then processed based on the channel attention mechanism to obtain the channel attention feature map.
[0189] The first smoke feature map is convolved based on the second dilated convolution to obtain the third smoke feature map. The third smoke feature map is then processed based on the spatial attention mechanism to obtain the spatial attention feature map. The number of channels in the first dilated convolution and the second dilated convolution are different.
[0190] By fusing the channel attention feature map and the spatial attention feature map, a smoke feature map is obtained.
[0191] In one embodiment, the detection result determination module is further configured to:
[0192] When the confidence level of smoke in the image to be detected is greater than or equal to the confidence threshold, it is determined that smoke exists in the image to be detected.
[0193] In one embodiment, the detection result determination module is further configured to:
[0194] After determining that smoke exists in the image to be detected, the region of interest and frame difference mask map corresponding to the image to be detected are obtained. Based on the corresponding region of interest and frame difference mask map, the smoke region in the image to be detected is enhanced.
[0195] In one embodiment, the grayscale image set acquisition module is further configured to:
[0196] Preliminary smoke detection is performed on each frame of the image to be detected to obtain the region of interest corresponding to each frame of the image to be detected.
[0197] Perform grayscale transformation on each region of interest to obtain the grayscale image corresponding to each region of interest.
[0198] According to one embodiment of the present invention, a storage medium is provided, the storage medium storing at least one executable instruction, which can execute the deep learning-based smoke detection method in any of the above method embodiments.
[0199] Figure 12 The diagram illustrates a structural schematic of a computer device according to an embodiment of the present invention. The specific embodiments of the present invention do not limit the specific implementation of the computer device.
[0200] like Figure 12 As shown, the computer device may include: a processor 1202, a communications interface 1204, a memory 1206, and a communications bus 1208.
[0201] The processor 1202, communication interface 1204, and memory 1206 communicate with each other via communication bus 1208.
[0202] The communication interface 1204 is used to communicate with other network elements such as clients or other servers.
[0203] The processor 1202 is used to execute program 1210, specifically to perform the relevant steps in the above-described embodiment of the smoke detection method based on deep learning.
[0204] Specifically, program 1210 may include program code that includes computer operation instructions.
[0205] Processor 1202 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The computer device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.
[0206] Memory 1206 is used to store program 1210. Memory 1206 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.
[0207] Specifically, program 1210 can be used to cause processor 1202 to perform the following operations:
[0208] The system acquires multiple frames of images of the target area captured continuously in real time, performs grayscale processing on each frame of the image to be detected to obtain multiple grayscale images, and takes a first preset number of grayscale images as a grayscale image set according to the capture order of the images to be detected corresponding to the grayscale images to obtain multiple grayscale image sets.
[0209] Frame difference processing is performed on each grayscale image set to obtain the frame difference mask map corresponding to each grayscale image set. The second preset number of frame difference mask maps are combined into a mask map set according to the order of the grayscale image sets corresponding to the frame difference mask map, resulting in multiple mask map sets. Multiple images to be detected are obtained corresponding to each mask map set.
[0210] Calculate the area of the moving object region in each frame difference mask in each mask set, and based on the area of multiple moving object regions in each mask set, determine whether there is a potential smoke region in the multiple images to be detected corresponding to each mask set.
[0211] When there is a potential smoke region in multiple images to be detected corresponding to any mask set, smoke features are extracted from the multiple images to be detected corresponding to any mask set to obtain multiple smoke feature maps.
[0212] Based on multiple smoke feature maps and a pre-set deep learning model, the smoke confidence score of each frame of the image to be detected corresponding to any mask map set is calculated, and the smoke detection result is determined according to the smoke confidence score.
[0213] It will be apparent to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. In one embodiment, they can be implemented using device-executable program code, thereby allowing them to be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular hardware and software combination.
[0214] The above embodiments are merely exemplary embodiments of this application and are not intended to limit this application. The scope of protection of this application is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions to this application within its substance and scope of protection, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of this application.
Claims
1. A smoke detection method based on deep learning, characterized in that, include: The system acquires multiple frames of images of the target area captured continuously in real time, performs grayscale processing on each frame of the image to be detected to obtain multiple grayscale images, and takes a first preset number of grayscale images as a grayscale image set according to the shooting order of the images to be detected corresponding to the grayscale images to obtain multiple grayscale image sets. Each grayscale image set is subjected to frame difference processing to obtain a frame difference mask map corresponding to each grayscale image set. The second preset number of frame difference mask maps are combined into a mask map set according to the order of the grayscale image sets corresponding to the frame difference mask map to obtain multiple mask map sets, and multiple images to be detected corresponding to each mask map set are obtained. Calculate the area of the moving object region in each frame difference mask in each mask set, and based on the area of multiple moving object regions in each mask set, determine whether there is a potential smoke region in the multiple images to be detected corresponding to each mask set. When a potential smoke region exists in multiple images to be detected corresponding to any mask set, smoke features are extracted from the multiple images to be detected corresponding to the mask set to obtain multiple smoke feature maps. Based on the multiple smoke feature maps and the preset deep learning model, the smoke confidence score of each frame of the image to be detected corresponding to any mask map set is calculated, and the smoke detection result is determined according to the smoke confidence score.
2. The smoke detection method based on deep learning as described in claim 1, characterized in that, The step of performing frame difference processing on each grayscale image set to obtain the frame difference mask map corresponding to each grayscale image set includes: Based on the first and last grayscale images in the grayscale image set, calculate the pixel value difference for each pixel. A frame difference mask is generated based on the pixel value difference and pixel threshold of each pixel. In the frame difference mask, pixels with a pixel value difference greater than the pixel threshold are set to white, and pixels with a pixel value difference less than or equal to the pixel threshold are set to black.
3. The smoke detection method based on deep learning as described in claim 2, characterized in that, The calculation of the moving object region area of each frame difference mask in each mask set, and the determination of whether there is a potential smoke region in the multiple images to be detected corresponding to each mask set based on the multiple moving object region areas in each mask set, includes: Based on the color of the pixels in the frame difference mask, calculate the area of the moving object region in each frame difference mask in each mask set; Based on the area of multiple moving object regions in each mask set, obtain the number of frame difference mask images in each mask set whose moving object regions' areas fall within the area range; The presence of potential smoke regions in multiple images to be detected corresponding to each mask set is determined based on the number of frame difference mask images.
4. The smoke detection method based on deep learning as described in claim 3, characterized in that, The step of determining whether potential smoke regions exist in multiple images to be detected corresponding to each mask set based on the number of frame difference mask images includes: When the number of frame difference mask images is greater than or equal to the number threshold, then the multiple images to be detected corresponding to the mask image set have potential smoke regions. When the number of frame difference mask images is less than the number threshold, then the multiple images to be detected corresponding to the mask image set do not have potential smoke regions.
5. The smoke detection method based on deep learning as described in claim 1, characterized in that, The step of extracting smoke features from multiple images to be detected corresponding to any one of the mask image sets to obtain multiple smoke feature maps includes: The image to be detected is convolved based on two-dimensional convolution to obtain a first smoke feature map; The first smoke feature map is convolved based on the first dilated convolution to obtain the second smoke feature map. The second smoke feature map is then processed based on the channel attention mechanism to obtain the channel attention feature map. The first smoke feature map is convolved based on the second dilated convolution to obtain the third smoke feature map. The third smoke feature map is then processed based on the spatial attention mechanism to obtain the spatial attention feature map. The number of channels in the first dilated convolution and the second dilated convolution are different. The channel attention feature map and the spatial attention feature map are fused to obtain the smoke feature map.
6. The smoke detection method based on deep learning as described in claim 1, characterized in that, Determining the smoke detection result based on the smoke confidence level includes: When the confidence level of the smoke in the image to be detected is greater than or equal to the confidence threshold, it is determined that there is smoke in the image to be detected. After determining that smoke exists in the image to be detected, the smoke detection method further includes: Obtain the region of interest and frame difference mask corresponding to the image to be detected, and enhance the smoke region in the image to be detected based on the corresponding region of interest and frame difference mask.
7. The smoke detection method based on deep learning as described in claim 1, characterized in that, The process of performing grayscale processing on each frame of the image to be detected to obtain multiple grayscale images includes: Preliminary smoke detection is performed on each frame of the image to be detected to obtain the region of interest corresponding to each frame of the image to be detected. Perform grayscale transformation on each region of interest to obtain the grayscale image corresponding to each region of interest.
8. A smoke detection device based on deep learning, characterized in that, include: The grayscale image set acquisition module is used to acquire multiple frames of images to be detected of the target area captured continuously in real time, perform grayscale processing on each frame of the image to be detected to obtain multiple grayscale images, and take a first preset number of grayscale images as a grayscale image set according to the shooting order of the images to be detected corresponding to the grayscale images to obtain multiple grayscale image sets. The mask image set acquisition module is used to perform frame difference processing on each grayscale image set to obtain the frame difference mask image corresponding to each grayscale image set. The second preset number of frame difference mask images are used as a mask image set according to the order of the grayscale image sets corresponding to the frame difference mask images to obtain multiple mask image sets, and multiple images to be detected corresponding to each mask image set are acquired. The smoke region determination module is used to calculate the area of the moving object region in each frame difference mask in each mask set, and to determine whether there is a potential smoke region in the multiple images to be detected corresponding to each mask set based on the area of multiple moving object regions in each mask set. The smoke feature map acquisition module is used to extract smoke features from the multiple images to be detected corresponding to any mask set when there is a potential smoke region in the multiple images to be detected corresponding to any mask set, and obtain multiple smoke feature maps. The detection result determination module is used to calculate the smoke confidence score of each frame of the image to be detected corresponding to any mask image set based on the multiple smoke feature maps and the preset deep learning model, and determine the smoke detection result based on the smoke confidence score.
9. A storage medium storing at least one executable instruction that causes a processor to perform an operation corresponding to the deep learning-based smoke detection method as described in any one of claims 1-7.
10. A computer device, comprising: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the deep learning-based smoke detection method as described in any one of claims 1-7.