Smoke target detection method and device based on time-series perturbation response graph

Abstract

This invention relates to a method and apparatus for smoke target detection based on a temporal perturbation response map. The method includes: acquiring video data containing smoke targets, and extracting an image sequence from the video data at a fixed frame rate, wherein each frame in the image sequence is an RGB three-channel image. A moving average is performed on multiple consecutive frames of the RGB three-channel images in the image sequence to obtain a temporally smoothed fused image. Based on the fused image, the pixel positions with the highest color frequency within a first time period are determined, and a background model is constructed based on these pixel positions. The difference between each frame of the RGB three-channel image and the background model is calculated to obtain a frame difference map, and a mask map of the smoke target region is estimated based on the frame difference map. The RGB three-channel image and the mask map are concatenated into a four-channel input. A YOLOv8 model is trained based on the four-channel input combined with the bounding boxes of the smoke targets, and the trained YOLOv8 model is then used to detect and output the smoke targets.

Description

Technical Field

[0001] This invention relates to the field of image detection technology, and in particular to a method and apparatus for smoke target detection based on temporal perturbation response maps. Background Technology

[0002] In typical scenarios such as urban surveillance, forest early warning, and industrial plant security, there is often a need for timely and accurate detection and alarming of non-rigid and disturbing targets such as smoke, fire, and leaks. However, traditional target detection models usually rely on single-frame image input, making it difficult to effectively identify targets like smoke, which have characteristics such as transparency, blurred shape, and weak texture in visual appearance. Especially under complex backgrounds or dynamic disturbance conditions, they are prone to missed detections or false positives.

[0003] In existing technologies, one approach transforms smoke detection into a regular object detection task. This involves training general detection models like YOLO using manually labeled images. The main idea is to treat smoke images like ordinary object data, constructing an image-level object detection dataset, and training it using architectures like YOLOv8. The implementation flow is: image acquisition → manual annotation → model training → inference detection. This approach has the advantage of easily reusing existing object detection frameworks and high inference efficiency, but it has a significant drawback: because smoke features are extremely weak and their changes are not obvious in a single frame, the model struggles to learn features that effectively distinguish regular structures (such as fog from background textures), especially in low-light, low-contrast scenes where accuracy drops drastically. Another approach is based on video motion information, extracting moving regions using traditional methods such as frame difference maps and background modeling. For example, dynamic masks can be generated based on inter-frame difference, optical flow, or median filtering to detect moving regions. This approach's flow is: video preprocessing → frame difference map calculation → threshold segmentation → connected component analysis → smoke region identification. This method is highly sensitive to temporal perturbations of smoke and can detect subtle early changes, but it often relies on manual thresholds, has poor robustness to texture perturbations, and is difficult to combine with deep semantic information. It suffers from a high false detection rate, weak generalization ability, and difficulty in adapting to complex background conditions. Summary of the Invention

[0004] Therefore, it is necessary to provide a smoke target detection method and device based on time-series perturbation response maps that can combine video frame difference mask modeling and multi-channel target detection framework to achieve enhanced modeling of dynamic perturbation regions and improve the accuracy and real-time performance of smoke detection.

[0005] This invention provides a smoke target detection method based on a time-series perturbation response map, the method comprising:

[0006] Acquire video data with smoke targets, and extract image sequences from the video data at a fixed frame rate, wherein each frame in the image sequence is an RGB three-channel image;

[0007] A moving average process is performed on multiple consecutive frames of RGB three-channel images in the image sequence to obtain a time-smoothed fused image;

[0008] Based on the fused image, determine the pixel location where the color appears most frequently in the first time period, and construct a background model based on the pixel location;

[0009] Calculate the difference between each frame's RGB three-channel image and the background model to obtain a frame difference map, and estimate the mask map of the smoke target region based on the frame difference map;

[0010] The RGB three-channel image and the mask image are concatenated into a four-channel input in the channel dimension. The YOLOv8 model is trained based on the four-channel input and the bounding boxes of the smoke target region, so as to call the trained YOLOv8 model to detect and output the smoke target.

[0011] In one embodiment, acquiring video data with a smoke target and extracting an image sequence from the video data at a fixed frame rate includes:

[0012] Extract multi-frame RGB three-channel images with time sequence from the video data at a fixed frame rate, and construct the image sequence based on the multi-frame RGB three-channel images with time sequence;

[0013] The image sequence is represented as follows:

[0014]

[0015]

[0016] In the formula, This indicates that the RGB three-channel image of frame t is in channel 1. The pixel value at that location represents the pixel coordinates. For the image spatial domain, The total number of frames in the RGB three-channel images of the image sequence.

[0017] In one embodiment, the step of performing a moving average process on multiple consecutive frames of RGB three-channel images in the image sequence to obtain a time-smoothed fused image includes:

[0018] The expression for performing a moving average processing on multiple consecutive frames of RGB three-channel images in the image sequence is as follows:

[0019]

[0020] In the formula, This indicates that the RGB three-channel image starting from frame t is continuous. Average pixel value of a frame's RGB three-channel image; This indicates the length of the sliding window, used to control the degree of temporal smoothing; Indicates the first Frame RGB three-channel image in channels Pixel value at;

[0021] A one-dimensional average convolution is performed on multiple consecutive RGB three-channel images in the image sequence along one time dimension, expressed as:

[0022]

[0023] In the formula, This represents the mean convolution kernel.

[0024] In one embodiment, determining the pixel location with the highest color frequency within a first time period based on the fused image, and constructing a background model based on the pixel location, includes:

[0025] Statistical analysis is performed on each pixel position in the fused image to obtain the pixel sequence of each pixel position in the fused image at different times;

[0026] Based on the pixel sequence in the fused image at different times for each pixel position, the median in the pixel sequence is determined, and outliers are removed based on the median to obtain the background pixel set.

[0027] In one embodiment, determining the pixel location with the highest color frequency within a first time period based on the fused image, and constructing a background model based on the pixel location, further includes:

[0028] The background pixels at the first moment are determined based on the set of background pixels, and the background pixels at the second moment are calculated by combining the average pixel value of multiple consecutive frames of RGB three-channel images and the preset historical weight.

[0029] Wherein, the first time point and the second time point are adjacent time points, and the first time point is located before the second time point in time sequence.

[0030] In one embodiment, calculating the difference between each frame's RGB three-channel image and the background model to obtain a frame difference map, and estimating the mask map of the smoke target region based on the frame difference map, includes:

[0031] The fused image is filtered by a first-order low-pass filter, and the average value of the difference between the background pixel and the average pixel value of the RGB three-channel image of multiple consecutive frames is calculated on the RGB three-channel to obtain the frame difference map.

[0032] The frame difference map is converted into a binary mask map, and the mask in the binary mask map is segmented and morphologically operated on based on a set threshold to generate a mask map of the smoke target region.

[0033] The present invention also provides a smoke target detection device based on a time-series perturbation response map, the device comprising:

[0034] An image sequence extraction module is used to acquire video data with smoke targets and extract an image sequence from the video data at a fixed frame rate, wherein each frame in the image sequence is an RGB three-channel image.

[0035] The temporal fusion module is used to perform a moving average process on multiple consecutive frames of RGB three-channel images in the image sequence to obtain a temporally smoothed fused image;

[0036] The background model construction module is used to determine the pixel position with the highest color frequency in the first time period based on the fused image, and to construct a background model based on the pixel position.

[0037] The mask image generation module is used to calculate the difference between each frame of RGB three-channel image and the background model to obtain a frame difference map, and to estimate the mask image of the smoke target area based on the frame difference map;

[0038] The smoke target detection module is used to stitch the RGB three-channel image and the mask image into a four-channel input in the channel dimension, and to train the YOLOv8 model based on the four-channel input and the bounding boxes of the smoke target region, so as to call the trained YOLOv8 model to detect and output the smoke target.

[0039] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the smoke target detection method based on the temporal perturbation response map as described above.

[0040] The present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, implements the smoke target detection method based on the temporal perturbation response map as described above.

[0041] The smoke target detection method and apparatus based on the time-series perturbation response map described above have the following advantages compared with the prior art:

[0042] (1) This invention uses a frame difference mask based on sliding fusion and background modeling as an explicit dynamic guiding feature, which is stitched together with the original RGB image to form a four-channel image input YOLOv8 model, realizing multi-channel and multi-modal joint modeling. In addition, by customizing GPU-accelerated temporal sliding convolution, spatial smoothing convolution and exponential background update model, the frame difference mask is generated quickly, and the modeling ability of the target detection model to temporal perturbation changes is supplemented at a low cost, thereby significantly improving the detection ability of weak texture smoke targets.

[0043] (2) The present invention maintains the YOLOv8 backbone network structure in terms of model structure. Only the input channel and the first layer convolution kernel dimension need to be modified. There is no need to change the inference process or loss function. It can be seamlessly integrated into the existing target detection deployment system, and is especially suitable for multi-camera streams, high-resolution surveillance videos, or real-time early warning scenarios at the edge. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0045] Figure 1 This is a flowchart illustrating the smoke target detection method based on time-series perturbation response map provided by the present invention.

[0046] Figure 2 A schematic diagram of the overall process of smoke target detection based on the time-series perturbation response map in a specific embodiment of the present invention;

[0047] Figure 3 A schematic diagram of the sliding window update process of the smoke target detection method based on the time-series perturbation response map in a specific embodiment of the present invention;

[0048] Figure 4 A schematic diagram of the smoke target detection device based on time-series perturbation response diagram provided by the present invention;

[0049] Figure 5 An internal structural diagram of the computer device provided by the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] The following is combined Figures 1-5 The present invention describes a smoke target detection method and apparatus based on a time-series perturbation response map.

[0052] like Figure 1 As shown, in one embodiment, a smoke target detection method based on a time-series perturbation response map includes the following steps:

[0053] Step S110: Acquire video data with smoke targets, and extract image sequences from the video data at a fixed frame rate. Each frame in the image sequence is an RGB three-channel image.

[0054] In some embodiments, the smoke target detection method based on time-series perturbation response map provided by the present invention includes the following steps in step S110:

[0055] Step S111: Extract multi-frame RGB three-channel images with time sequence from the video data at a fixed frame rate, and construct an image sequence based on the multi-frame RGB three-channel images with time sequence.

[0056] The image sequence is represented as follows:

[0057]

[0058]

[0059] In the formula, This indicates that the RGB three-channel image of frame t is in channel 1. The pixel value at that location represents the pixel coordinates. For the image spatial domain, This represents the total number of frames in the RGB three-channel image sequence.

[0060] Combination Figure 2 and Figure 3As shown in the specific embodiment, the smoke target detection method based on temporal perturbation response map provided by this invention addresses the challenge of detecting smoke targets in videos. Smoke, in particular, appears as a dynamic target with significant perturbation, continuity, but irregular shape. The main difficulties in its detection lie in its ability to blend seamlessly with background edges or lighting changes, its variable shape, and its color being close to the background. This example aims to explicitly introduce "temporal perturbation features" into the YOLOv8 target detection algorithm to help the model more accurately identify smoke regions.

[0061] First, acquire video data containing smoke targets, extracting data from the video at a constant frame rate. Extract image sequences (e.g., 25 FPS) Each frame is an RGB three-channel image:

[0062]

[0063]

[0064] In the formula, Indicates the first Frame in channel The pixel value is in the range of ; For pixel coordinates, For the image spatial domain, This represents the total number of frames.

[0065] Step S120: Perform a moving average process on multiple consecutive frames of RGB three-channel images in the image sequence to obtain a time-smoothed fused image.

[0066] In some embodiments, the smoke target detection method based on time-series perturbation response map provided by the present invention includes the following steps in step S120:

[0067] Step S121, the expression for performing moving average processing on multiple consecutive frames of RGB three-channel images in the image sequence is:

[0068]

[0069] In the formula, This indicates that the RGB three-channel image starting from frame t is continuous. Average pixel value of a frame's RGB three-channel image; This indicates the length of the sliding window, used to control the degree of temporal smoothing; Indicates the first Frame RGB three-channel image in channels The pixel value at that location.

[0070] Step S122: Perform a one-dimensional average convolution on multiple consecutive frames of RGB three-channel images in the image sequence along one time dimension. The expression is:

[0071]

[0072] In the formula, This represents the mean convolution kernel.

[0073] Combination Figure 2 and Figure 3 As shown in the specific embodiment, the smoke target detection method based on temporal perturbation response map provided by the present invention, in order to reduce the influence of illumination noise and small perturbations in a single frame, performs continuous perturbation on the smoke target detection method. A moving average is applied to the frame images to obtain a time-smoothed fused image.

[0074]

[0075] In the formula, Indicates from the first Frames begin to be continuous The average pixel value of the frame; The length of the sliding window (e.g.) ), used to control the degree of time-domain smoothing.

[0076] Combination Figure 3 As shown, temporal smoothing is essentially a one-dimensional average convolution in the time dimension, and therefore can also be expressed as:

[0077]

[0078] In the formula, , represents the mean convolution kernel, This indicates rounding down, with a kernel size of . Center-aligned. A one-dimensional convolution kernel on the time axis can be represented as... By using custom convolutions, the continuous process that would otherwise take a long time can be transformed into a more efficient and efficient one. Frame summation can be accelerated using GPUs via PyTorch's interface. To further accelerate subsequent calculations, sliding window updates are used to avoid redundant accumulation, employing the following recursive formula:

[0079]

[0080] Each time a new image frame is entered, a new image frame is removed, thus achieving real-time updates.

[0081] Step S130: Based on the fused image, determine the pixel position where the color appears most frequently in the first time period, and construct a background model based on the pixel position.

[0082] In some embodiments, the smoke target detection method based on time-series perturbation response map provided by the present invention includes the following steps in step S130:

[0083] Step S131: Statistical analysis is performed on each pixel position in the fused image to obtain the pixel sequence of each pixel position in the fused image at different times.

[0084] Step S132: Based on the pixel sequence of the fused image at different times for each pixel position, determine the median in the pixel sequence, and remove outliers based on the median to obtain the background pixel set.

[0085] In some embodiments, the smoke target detection method based on time-series perturbation response map provided by the present invention further includes the following steps in step S130:

[0086] Step S133: Determine the background pixels at the first moment based on the background pixel set, and calculate the background pixels at the second moment by combining the average pixel values ​​of multiple consecutive frames of RGB three-channel images and the preset historical weights.

[0087] The first time point and the second time point are adjacent time points, and the first time point is earlier than the second time point in the time sequence.

[0088] Combination Figure 2 and Figure 3 As shown in the specific embodiment, the smoke target detection method based on temporal perturbation response map provided by the present invention constructs a background image after obtaining a stable temporal fusion image. It represents the color that appears most frequently at a certain pixel location during long-term observation, avoiding the influence of foreground objects (such as people or smoke).

[0089] In this embodiment, by analyzing the position of each pixel... The pixel sequences in the multi-time-time fused image are statistically analyzed:

[0090]

[0091] In the formula, This represents the total number of moments used for modeling (e.g., before the initialization phase). frame), express arrive All A set that consists of.

[0092] Next, define the background pixels as:

[0093]

[0094] In the formula, the median It has the ability to resist outliers and can avoid the impact of foreground occlusion in a few frames. This is because in actual monitoring, the camera may slowly shift and the lighting may slowly drift.

[0095] To accommodate the "slow variable" changes in the background, an exponentially weighted background update is introduced:

[0096]

[0097] In the formula, This represents historical weights; the larger the value, the more stable the system (e.g., ...). ); express The background image from the previous moment; This represents the fused image of the current frame; this formula is equivalent to a first-order low-pass filter, which has the ability to gradually adapt to slow changes.

[0098] Step S140: Calculate the difference between the RGB three-channel image of each frame and the background model to obtain the frame difference map, and estimate the mask map of the smoke target area based on the frame difference map.

[0099] In some embodiments, the smoke target detection method based on time-series perturbation response map provided by the present invention includes the following steps in step S140:

[0100] Step S141: The fused image is filtered by a first-order low-pass filter, and the average value of the difference between the background pixel and the average pixel value of the RGB three-channel image of multiple consecutive frames is calculated on the RGB three-channel to obtain the frame difference map.

[0101] Step S142: Convert the frame difference map into a binary mask map, and perform segmentation and morphological operations on the mask in the binary mask map based on a set threshold to generate a mask map of the smoke target region.

[0102] Combination Figure 2 and Figure 3 As shown in the specific embodiment, the smoke target detection method based on temporal perturbation response map provided by the present invention, after the background model is built, calculates the difference between the current image and the background to obtain a perturbation degree image (frame difference map):

[0103]

[0104] In the formula, This represents the average intensity of the difference between the current image and the background, calculated by averaging the three channels to suppress interference caused by color fluctuations.

[0105] Because of the rapid changes between frames, its corresponding region The values ​​are typically significantly larger than the background area. To extract areas of significant change, a threshold is set. Convert the difference image into a binary mask image:

[0106]

[0107] In the formula, To set a threshold (which can be a fixed value such as 30), this mask represents an area where smoke or other disturbances may exist.

[0108] In this embodiment, since image differences are affected by noise, light spots, etc., the mask after direct segmentation may have small spots and gaps, so morphological operations can be performed on it:

[0109]

[0110]

[0111] In the formula, Indicates corrosion operation. This indicates an expansion operation. For structural elements, such as Cross-shaped core.

[0112] Based on the aforementioned processing, a coherent closed region can be formed, suitable for targets with weak edges in smoke:

[0113]

[0114] In the formula, This represents the screening threshold, which retains larger foreground candidate regions to further exclude non-smoke disturbances.

[0115] Step S150: In the channel dimension, the RGB three-channel image and the mask image are stitched together to form a four-channel input, and the YOLOv8 model is trained based on the four-channel input and the bounding box of the smoke target region, so as to call the trained YOLOv8 model to detect and output the smoke target.

[0116] In some embodiments, the smoke target detection method based on time-series perturbation response map provided by the present invention includes the following steps in step S150:

[0117] Step S151: In the channel dimension, the RGB three-channel image of each frame and the corresponding mask image are concatenated into a four-channel input, and the concatenated image is converted into model input data that meets the input requirements of the YOLOv8 model.

[0118] Step S152: Using the bounding boxes of the smoke target region as output, perform end-to-end target detection training on the YOLOv8 model to obtain the smoke target detection model.

[0119] Step S153: Input the current video frame into the smoke target detection model to generate the frame difference mask map corresponding to the current video frame, so as to obtain the four-channel input corresponding to the current video frame, and generate and output the smoke target detection result based on the four-channel input corresponding to the current video frame.

[0120] Combination Figure 2 and Figure 3 As shown in the specific embodiment, the smoke target detection method based on temporal perturbation response map provided by the present invention first processes the original video frame by frame during the training phase of the YOLOv8 target detection algorithm, that is, processes the original RGB image... Its corresponding mask image By splicing along the channel dimension, a four-channel input is formed:

[0121]

[0122] Next, the stitched image input was modified to support a 4-channel input YOLOv8 model, using the bounding boxes of the smoke region as targets, for end-to-end target detection training. The detection head used the same loss function as the original YOLO (such as CIoU loss, classification loss, objectness loss), and the mask was used as guiding information during training to improve the model's ability to distinguish perturbed targets.

[0123] During the inference phase, for any input video frame, a frame difference mask image corresponding to the current frame is generated in real time through weighted averaging via a sliding window, background estimation, and frame difference calculation. As an auxiliary feature, channels are concatenated to form a four-channel input:

[0124]

[0125] Finally, Input the pre-trained improved four-channel YOLOv8 network, perform forward inference, and obtain the detection results (bounding boxes, confidence, class). The detection boxes in the results can accurately cover the smoke area, and redundant boxes are filtered by NMS (non-maximum suppression) to output the final result of smoke target detection.

[0126] The smoke target detection method based on temporal perturbation response map mentioned above combines the physical meaning of traditional visual frame difference modeling with the generalization ability of deep learning target detection. By explicitly using the temporal perturbation features of smoke as part of the model input, it effectively improves the robustness of smoke target detection.

[0127] The smoke target detection device based on time-series perturbation response map provided by the present invention will be described below. The smoke target detection device based on time-series perturbation response map described below can be referred to in correspondence with the smoke target detection method based on time-series perturbation response map described above.

[0128] like Figure 4 As shown, in one embodiment, a smoke target detection device based on a temporal perturbation response map includes an image sequence extraction module 410, a temporal fusion module 420, a background model construction module 430, a mask map generation module 440, and a smoke target detection module 450.

[0129] The image sequence extraction module 410 is used to acquire video data with smoke targets and extract image sequences from the video data at a fixed frame rate. Each frame in the image sequence is an RGB three-channel image.

[0130] The temporal fusion module 420 is used to perform moving average processing on multiple consecutive frames of RGB three-channel images in the image sequence to obtain a temporally smoothed fused image.

[0131] Background model building module 430 is used to determine the pixel position where the color appears most frequently in the first time period based on the fused image, and to build a background model based on the pixel position.

[0132] The mask generation module 440 is used to calculate the difference between the RGB three-channel image of each frame and the background model to obtain the frame difference map, and to estimate the mask map of the smoke target area based on the frame difference map.

[0133] The smoke target detection module 450 is used to stitch the RGB three-channel image and the mask image into a four-channel input in the channel dimension, and to train the YOLOv8 model based on the four-channel input and the bounding boxes of the smoke target region, so as to call the trained YOLOv8 model to detect and output the smoke target.

[0134] In this embodiment, the smoke target detection device based on time-series perturbation response map provided by the present invention, specifically uses the image sequence extraction module 410 for:

[0135] Extract multi-frame RGB three-channel images with time sequence from video data at a fixed frame rate, and construct an image sequence based on the multi-frame RGB three-channel images with time sequence.

[0136] The image sequence is represented as follows:

[0137]

[0138]

[0139] In the formula, This indicates that the RGB three-channel image of frame t is in channel 1. The pixel value at that location represents the pixel coordinates. For the image spatial domain, This represents the total number of frames in the RGB three-channel image sequence.

[0140] In this embodiment, the time-series fusion module 420 of the smoke target detection device based on time-series perturbation response map provided by the present invention is specifically used for:

[0141] The expression for performing a moving average on multiple consecutive frames of RGB three-channel images in an image sequence is as follows:

[0142]

[0143] In the formula, This indicates that the RGB three-channel image starting from frame t is continuous. Average pixel value of a frame's RGB three-channel image; This indicates the length of the sliding window, used to control the degree of temporal smoothing; Indicates the first Frame RGB three-channel image in channels The pixel value at that location.

[0144] A one-dimensional average convolution is performed on multiple consecutive frames of RGB three-channel images in an image sequence along one time dimension. The expression is:

[0145]

[0146] In the formula, This represents the mean convolution kernel.

[0147] In this embodiment, the background model construction module 430 of the smoke target detection device based on time-series perturbation response map provided by the present invention is specifically used for:

[0148] Statistical analysis is performed on each pixel location in the fused image to obtain the pixel sequence of each pixel location in the fused image at different times.

[0149] Based on the pixel sequence of the fused image at different times for each pixel position, the median of the pixel sequence is determined, and outliers are removed based on the median to obtain the background pixel set.

[0150] In this embodiment, the background model construction module 430 of the smoke target detection device based on time-series perturbation response map provided by the present invention is further used for:

[0151] The background pixels at the first moment are determined based on the set of background pixels. The background pixels at the second moment are calculated by combining the average pixel value of multiple consecutive frames of RGB three-channel images and the preset historical weights.

[0152] The first time point and the second time point are adjacent time points, and the first time point is earlier than the second time point in the time sequence.

[0153] In this embodiment, the smoke target detection device based on time-series perturbation response map provided by the present invention, the mask map generation module 440 is specifically used for:

[0154] The fused image is filtered by a first-order low-pass filter, and the average value of the difference between the background pixel and the average pixel value of the RGB three-channel image of multiple consecutive frames is calculated on the RGB three-channel to obtain a frame difference map.

[0155] The frame difference map is converted into a binary mask map, and the mask in the binary mask map is segmented and morphologically operated on based on a set threshold to generate a mask map of the smoke target region.

[0156] In this embodiment, the smoke target detection module 450 of the smoke target detection device based on the time-series perturbation response map provided by the present invention is specifically used for:

[0157] In terms of channels, each frame's RGB three-channel image and its corresponding mask image are concatenated into a four-channel input, and the concatenated image is converted into model input data that meets the input requirements of the YOLOv8 model.

[0158] Using the bounding boxes of the smoke target region as output, the YOLOv8 model is trained end-to-end for target detection to obtain the smoke target detection model.

[0159] The current video frame is input into the smoke target detection model to generate the frame difference mask map corresponding to the current video frame. The mask map is then stitched together to obtain the four-channel input corresponding to the current video frame. Based on the four-channel input corresponding to the current video frame, the smoke target detection result is generated and output.

[0160] Figure 5 This example illustrates a schematic diagram of the physical structure of an electronic device, which can be a smart terminal. Its internal structure diagram can be as follows: Figure 5 As shown. The electronic device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a smoke target detection method based on a time-series perturbation response map, the method including:

[0161] Acquire video data containing smoke targets, and extract image sequences from the video data at a fixed frame rate. Each frame in the image sequence is an RGB three-channel image.

[0162] A time-smoothed fused image is obtained by performing a moving average process on multiple consecutive frames of RGB three-channel images in an image sequence.

[0163] The pixel locations where the color appears most frequently in the first time period are determined based on the fused image, and a background model is constructed based on the pixel locations.

[0164] Calculate the difference between the RGB three-channel image of each frame and the background model to obtain the frame difference map, and estimate the mask map of the smoke target area based on the frame difference map;

[0165] In terms of channels, the RGB three-channel image and the mask image are concatenated into a four-channel input. The YOLOv8 model is trained based on the four-channel input and the bounding boxes of the smoke target region, so as to call the trained YOLOv8 model to detect and output the smoke target.

[0166] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the electronic device to which the present invention is applied. A specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0167] On the other hand, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, implements a smoke target detection method based on a time-series perturbation response map, the method comprising:

[0168] Acquire video data containing smoke targets, and extract image sequences from the video data at a fixed frame rate. Each frame in the image sequence is an RGB three-channel image.

[0169] A time-smoothed fused image is obtained by performing a moving average process on multiple consecutive frames of RGB three-channel images in an image sequence.

[0170] The pixel locations with the highest color frequency in the first time period are determined based on the fused image, and a background model is constructed based on the pixel locations.

[0171] Calculate the difference between the RGB three-channel image of each frame and the background model to obtain the frame difference map, and estimate the mask map of the smoke target area based on the frame difference map;

[0172] In terms of channels, the RGB three-channel image and the mask image are concatenated into a four-channel input. The YOLOv8 model is trained based on the four-channel input and the bounding boxes of the smoke target region, so as to call the trained YOLOv8 model to detect and output the smoke target.

[0173] In another aspect, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and when the processor executes the computer instructions, it implements a smoke target detection method based on a time-series perturbation response map, the method comprising:

[0174] Acquire video data containing smoke targets, and extract image sequences from the video data at a fixed frame rate. Each frame in the image sequence is an RGB three-channel image.

[0175] A time-smoothed fused image is obtained by performing a moving average process on multiple consecutive frames of RGB three-channel images in an image sequence.

[0176] The pixel locations with the highest color frequency in the first time period are determined based on the fused image, and a background model is constructed based on the pixel locations.

[0177] Calculate the difference between the RGB three-channel image of each frame and the background model to obtain the frame difference map, and estimate the mask map of the smoke target area based on the frame difference map;

[0178] In terms of channels, the RGB three-channel image and the mask image are concatenated into a four-channel input. The YOLOv8 model is trained based on the four-channel input and the bounding boxes of the smoke target region, so as to call the trained YOLOv8 model to detect and output the smoke target.

[0179] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory.

[0180] By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0181] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0182] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A method for smoke target detection based on time-series perturbation response graph, characterized in that, The method includes: Acquire video data with smoke targets, and extract image sequences from the video data at a fixed frame rate, wherein each frame in the image sequence is an RGB three-channel image; A moving average process is performed on multiple consecutive frames of RGB three-channel images in the image sequence to obtain a time-smoothed fused image; Based on the fused image, determine the pixel position where the color appears most frequently in the first time period, and construct a background model based on the pixel position; Calculate the difference between each frame's RGB three-channel image and the background model to obtain a frame difference map, and estimate the mask map of the smoke target region based on the frame difference map; The RGB three-channel image and the mask image are concatenated into a four-channel input in the channel dimension. The YOLOv8 model is trained based on the four-channel input and the bounding boxes of the smoke target region, so as to call the trained YOLOv8 model to detect and output the smoke target.

2. The smoke target detection method based on time-series perturbation response map according to claim 1, characterized in that, The step of acquiring video data with smoke targets and extracting image sequences from the video data at a fixed frame rate includes: Extract multi-frame RGB three-channel images with time sequence from the video data at a fixed frame rate, and construct the image sequence based on the multi-frame RGB three-channel images with time sequence; The image sequence is represented as follows: In the formula, This indicates that the RGB three-channel image of frame t is in channel 1. The pixel value at that location represents the pixel coordinates. For the image spatial domain, The total number of frames in the RGB three-channel images of the image sequence.

3. The smoke target detection method based on time-series perturbation response map according to claim 1, characterized in that, The step of performing a moving average process on multiple consecutive frames of RGB three-channel images in the image sequence to obtain a time-smoothed fused image includes: The expression for performing a moving average processing on multiple consecutive frames of RGB three-channel images in the image sequence is as follows: ， In the formula, This indicates that the RGB three-channel image starting from frame t is continuous. Average pixel value of a frame's RGB three-channel image; This indicates the length of the sliding window, used to control the degree of temporal smoothing; Indicates the first Frame RGB three-channel image in channels Pixel value at; A one-dimensional average convolution is performed on multiple consecutive RGB three-channel images in the image sequence along one time dimension, expressed as: ， In the formula, This represents the mean convolution kernel.

4. The smoke target detection method based on time-series perturbation response map according to claim 3, characterized in that, The step of determining the pixel location with the highest color frequency within a first time period based on the fused image, and constructing a background model based on the pixel location, includes: Statistical analysis is performed on each pixel position in the fused image to obtain the pixel sequence of each pixel position in the fused image at different times; Based on the pixel sequence in the fused image at different times for each pixel position, the median in the pixel sequence is determined, and outliers are removed based on the median to obtain the background pixel set.

5. The smoke target detection method based on time-series perturbation response map according to claim 4, characterized in that, The step of determining the pixel location with the highest color frequency in the first time period based on the fused image, and constructing a background model based on the pixel location, further includes: The background pixels at the first moment are determined based on the set of background pixels, and the background pixels at the second moment are calculated by combining the average pixel value of multiple consecutive frames of RGB three-channel images and the preset historical weight. Wherein, the first time point and the second time point are adjacent time points, and the first time point is located before the second time point in time sequence.

6. The smoke target detection method based on time-series perturbation response map according to claim 5, characterized in that, The step of calculating the difference between each frame's RGB three-channel image and the background model to obtain a frame difference map, and estimating the mask map of the smoke target region based on the frame difference map, includes: The fused image is filtered by a first-order low-pass filter, and the average value of the difference between the background pixel and the average pixel value of the RGB three-channel image of multiple consecutive frames is calculated on the RGB three-channel to obtain the frame difference map. The frame difference map is converted into a binary mask map, and the mask in the binary mask map is segmented and morphologically operated on based on a set threshold to generate a mask map of the smoke target region.

7. The smoke target detection method based on time-series perturbation response map according to claim 6, characterized in that, The process of concatenating the RGB three-channel image and the mask image into a four-channel input along the channel dimension, and training the YOLOv8 model based on the four-channel input combined with the bounding boxes of the smoke target region, so as to call the trained YOLOv8 model to detect and output the smoke target, includes: In terms of channel dimension, each frame's RGB three-channel image and the corresponding mask image are stitched together to form a four-channel input, and the stitched image is converted into model input data that meets the input requirements of the YOLOv8 model; Using the bounding boxes of the smoke target region as output, the YOLOv8 model is trained end-to-end for target detection to obtain a smoke target detection model. The current video frame is input into the smoke target detection model to generate a frame difference mask image corresponding to the current video frame. The mask image is then stitched together to obtain a four-channel input corresponding to the current video frame. Based on the four-channel input corresponding to the current video frame, the smoke target detection result is generated and output.

8. A smoke target detection device based on a time-series perturbation response map, characterized in that, The device includes: An image sequence extraction module is used to acquire video data with smoke targets and extract an image sequence from the video data at a fixed frame rate, wherein each frame in the image sequence is an RGB three-channel image. The temporal fusion module is used to perform a moving average process on multiple consecutive frames of RGB three-channel images in the image sequence to obtain a temporally smoothed fused image; The background model construction module is used to determine the pixel position with the highest color frequency in the first time period based on the fused image, and to construct a background model based on the pixel position. The mask image generation module is used to calculate the difference between each frame of RGB three-channel image and the background model to obtain a frame difference map, and to estimate the mask image of the smoke target area based on the frame difference map; The smoke target detection module is used to stitch the RGB three-channel image and the mask image into a four-channel input in the channel dimension, and to train the YOLOv8 model based on the four-channel input and the bounding boxes of the smoke target region, so as to call the trained YOLOv8 model to detect and output the smoke target.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the smoke target detection method based on the temporal perturbation response map as described in any one of claims 1 to 7.

10. A computer storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the smoke target detection method based on the temporal perturbation response map as described in any one of claims 1 to 7.

Images