A visual-based fog smoke detection method, device, equipment and medium

Abstract

The application belongs to the technical field of smoke detection, and discloses a foggy-day smoke detection method, device, equipment and medium based on vision, which comprises the following steps: performing downsampling on an acquired foggy-day environment image to obtain a to-be-detected image; calculating an HSV color model of the to-be-detected image according to the RGB color feature of the to-be-detected image, and calculating a dark channel mapping of the to-be-detected image according to the to-be-detected image and dark channel priori; inputting the to-be-detected image, the HSV color model and the dark channel mapping into a trained smoke detection network model to obtain a smoke detection result of the to-be-detected image. The application can effectively improve the accuracy and detection precision of smoke detection in a foggy-day environment.

Description

Technical Field

[0001] This application relates to the field of smoke detection technology, and in particular to a vision-based method, apparatus, equipment and medium for detecting smoke in foggy weather. Background Technology

[0002] With the widespread application of surveillance equipment and the development of computer vision technologies, vision-based smoke detection has become possible. Cameras positioned at different locations within a target scene can capture real-time video images of the surrounding environment. These images are then transmitted to edge computing devices for smoke detection. Once smoke is detected, the edge computing device immediately sends an alarm to the control center.

[0003] Vision-based smoke detection relies on visual cues of smoke, such as color and texture information. Most smoke detection methods only consider normal environments; however, in abnormal environments, such as fog, the video image information acquired by the camera will be affected. Since fog and smoke have similar visual characteristics, smoke detection methods trained in normal environments may produce incorrect predictions in foggy conditions. Therefore, existing technologies suffer from inaccurate and low-precision smoke detection in foggy environments. Summary of the Invention

[0004] This application provides a vision-based method, apparatus, equipment, and medium for detecting smoke in foggy weather, which can effectively improve the accuracy and precision of smoke detection in foggy environments.

[0005] In a first aspect, embodiments of this application provide a vision-based method for detecting fog and smoke in foggy weather. This method is applied to a smoke detection device and includes:

[0006] The acquired foggy environment image is downsampled to obtain the image to be detected;

[0007] The HSV color model of the image to be detected is calculated based on the RGB color features of the image to be detected, and the dark channel mapping of the image to be detected is calculated based on the image to be detected and the dark channel prior.

[0008] The image to be detected, the HSV color model, and the dark channel mapping are input into the trained smoke detection network model to obtain the smoke detection result of the image to be detected.

[0009] Furthermore, the trained smoke detection network model includes a fused attention module, a hierarchical feature extractor, and a classification prediction head. The above-mentioned input of the image to be detected, the HSV color model, and the dark channel mapping into the trained smoke detection network model yields the smoke detection results for the image to be detected, including:

[0010] The fusion attention module in the trained smoke detection network model performs feature attention fusion on the image to be detected, the HSV color model, and the dark channel mapping to obtain a fused feature image.

[0011] Global feature information is obtained by extracting features from the fused feature image using a hierarchical feature extractor.

[0012] The smoke detection result is obtained by using a classification prediction head to make predictions based on global feature information.

[0013] Furthermore, the hierarchical feature extractor includes several deep convolutional modules and a self-attention module;

[0014] The above-mentioned hierarchical feature extractor extracts global feature information from the fused feature image, including:

[0015] By changing the size and number of channels of the fused feature image through several deep convolutional modules, feature information of different dimensions is extracted; global feature information is obtained through a self-attention module based on the feature information of different dimensions.

[0016] Furthermore, the classification prediction head includes a global pooling layer and a linear classifier;

[0017] The above-mentioned prediction of global feature information using a classification prediction head yields smoke detection results, including:

[0018] A standard-sized feature image is obtained by performing mean pooling on global feature information through a global pooling layer.

[0019] The smoke detection results are obtained by predicting the feature image of a standard size using a linear classifier.

[0020] Furthermore, the method also includes:

[0021] Construct a pre-trained network model;

[0022] Training steps: Preprocess the foggy images to obtain training image samples; input the training image samples into the pre-trained network model to obtain detection results; the training image samples include RGB color feature samples, HSV color model samples, dark channel mapping samples, and true class information;

[0023] Update steps: Calculate the training loss value based on the detection results and the true class information; calculate the back gradient value based on the training loss value and the gradient descent algorithm; update the weight parameters of the pre-trained network model based on the back gradient value;

[0024] The training and update steps are repeated on the pre-trained network model according to the preset number of iterations to obtain the trained smoke detection network model.

[0025] Furthermore, the above preprocessing of the foggy images yields training image samples, including:

[0026] The foggy image was downsampled using bilinear interpolation to obtain the sampled image;

[0027] The sampled image is cropped according to the principle of translation invariance to obtain training image samples.

[0028] Furthermore, the above calculation of the training loss value based on the detection results and the true category information includes:

[0029] Based on the detection results and the true category information, the training loss value is calculated using the cross-entropy loss function.

[0030] Secondly, embodiments of this application provide a vision-based fog and smoke detection device, which is applied to a smoke detection equipment and includes:

[0031] The sampling module is used to downsample the acquired foggy environment image to obtain the image to be detected;

[0032] The calculation module is used to obtain the HSV color model of the image to be detected based on the RGB color features of the image to be detected, and to obtain the dark channel mapping of the image to be detected based on the image to be detected and the dark channel prior.

[0033] The detection module is used to input the image to be detected, the HSV color model, and the dark channel mapping into the trained smoke detection network model to obtain the smoke detection result of the image to be detected.

[0034] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the steps of a vision-based fog and smoke detection method as described in any of the above embodiments.

[0035] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of a vision-based fog and smoke detection method as described in any of the above embodiments.

[0036] In summary, compared with the prior art, the beneficial effects of the technical solution provided in this application include at least the following:

[0037] This application provides a vision-based fog and smoke detection method. When detecting images obtained by sampling images of foggy environments, it incorporates the RGB color features, HSV (Hue, Saturation, Value) color model, and dark channel prior information of the image. This facilitates the extraction and differentiation of features of smoke and fog by the trained smoke detection network model, thereby accurately detecting whether smoke exists in images of foggy environments and effectively improving the accuracy and precision of smoke detection in foggy environments. Attached Figure Description

[0038] Figure 1 A flowchart of a vision-based fog and smoke detection method provided as an exemplary embodiment of this application.

[0039] Figure 2 This is a structural diagram of a smoke detection network model provided as an exemplary embodiment of this application.

[0040] Figure 3 This is a structural diagram of a depth convolution module provided for an exemplary embodiment of this application.

[0041] Figure 4 A flowchart illustrating the operation of a self-attention module provided in an exemplary embodiment of this application.

[0042] Figure 5 A flowchart illustrating the training steps of a pre-trained network model provided for an exemplary embodiment of this application.

[0043] Figure 6 This is a structural diagram of a vision-based fog and smoke detection device provided as an exemplary embodiment of this application. Detailed Implementation

[0044] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0045] Please see Figure 1 This application provides a vision-based method for detecting fog and smoke in foggy weather. This method is applied to a smoke detection device and may specifically include the following steps:

[0046] Step S1: Downsample the acquired foggy environment image to obtain the image to be detected.

[0047] Specifically, bilinear interpolation can be used to downsample the foggy environment image, and the size of the downsampled image to be detected is 256×256.

[0048] Step S2: Calculate the HSV color model of the image to be detected based on the RGB color features of the image to be detected, and calculate the dark channel mapping of the image to be detected based on the image to be detected and the dark channel prior.

[0049] RGB color characteristics refer to the brightness of an image in the three color channels: red (R), green (G), and blue (B), typically represented by integers. Each RGB channel has 256 levels of brightness, represented numerically from 0, 1, 2... up to 255.

[0050] In the HSV color model, hue (H) is measured in degrees, ranging from 0° to 360°; saturation (S) represents how close a color is to a spectral color, ranging from 0% to 100%, with higher values ​​indicating more saturated colors; and lightness (V) represents the brightness of a color, ranging from 0% (black) to 100% (white).

[0051] Specifically, the HSV color model of the image to be detected can be obtained from the RGB values ​​using the following formula:

[0052]

[0053]

[0054] V = C max

[0055] R′=R / 255

[0056] G′=G / 255

[0057] B′=B / 255

[0058] C max =max(R′, G′, B′)

[0059] C min =min(R′, G′, B′)

[0060] Δ=C max -C min

[0061] The dark channel prior is a statistical feature derived from many outdoor fog-free images. It indicates that in local regions of a fog-free image, there are pixels that have very low values ​​(close to 0) in at least one color channel. Given an image I to be detected, the corresponding dark channel mapping can be obtained by the following formula:

[0062]

[0063] Among them, I y (c) is the c channel of I at pixel y, and (x) is a window centered at pixel x. dark (x) is the dark channel mapping at pixel x. The dark channel mapping obtained from the dark channel prior can make the smoke detection network model pay more attention to the smoke area and reduce background interference from the fog.

[0064] Step S3: Input the image to be detected, the HSV color model, and the dark channel mapping into the trained smoke detection network model to obtain the smoke detection result of the image to be detected.

[0065] Specifically, smoke detection results include four outcomes: Normal, Smoke, Fog, or Smoke with fog.

[0066] The above embodiment provides a vision-based fog and smoke detection method. When detecting the image to be detected obtained by sampling the image of the foggy environment, the RGB color features, HSV color model and dark channel prior information of the image are introduced. This is beneficial for the trained smoke detection network model to extract and distinguish the features of smoke and fog, so as to accurately detect whether there is smoke in the image of the foggy environment, and effectively improve the accuracy and precision of smoke detection in the foggy environment.

[0067] Please see Figure 2 In some embodiments, the trained smoke detection network model may specifically include a fused attention module, a hierarchical feature extractor, and a classification prediction head.

[0068] The above-mentioned input of the image to be detected, the HSV color model, and the dark channel mapping into the trained smoke detection network model yields the smoke detection results for the image to be detected, including:

[0069] The fusion attention module in the trained smoke detection network model performs feature attention fusion on the image to be detected, the HSV color model, and the dark channel mapping to obtain a fused feature image.

[0070] Specifically, let the RGB color features I of the image to be detected RGB HSV color model I HSV Dark Channel Mapping I dark After passing through different convolutional blocks, F is obtained respectively. RGB F HSV and F dark Three 16×112×112 features; then connect F along the channel dimension RGB and F HSV This yields a 32×112×112 feature, to which an SE layer is applied. Then, using F...dark The extracted 1×112×112 spatial attention map yields the final 32×112×112 fused feature image. This can be summarized by the following formula:

[0071] F mid =SE(concat(F) RGB ,F HSV ))

[0072]

[0073]

[0074] Where SE(·) represents the SE layer, F dark (x c ) represents feature F dark At pixel x, in the c-th channel, take the maximum value among the C channels and use it as the value of the spatial attention map A at x. This indicates element-wise multiplication.

[0075] Global feature information is obtained by extracting the fused feature image through a hierarchical feature extractor.

[0076] The smoke detection result is obtained by predicting global feature information using a classification prediction head.

[0077] The above embodiments fuse the RGB color features, HSV color model and dark channel mapping of the image to be detected by the fusion attention module, which is more conducive to the feature extraction of smoke and fog and improves the detection accuracy.

[0078] In some embodiments, the hierarchical feature extractor may include several deep convolutional modules and a self-attention module.

[0079] The above-mentioned hierarchical feature extractor extracts global feature information from the fused feature image, including:

[0080] By changing the size and number of channels of the fused feature image through several deep convolutional modules, feature information of different dimensions is extracted; global feature information is obtained through a self-attention module based on the feature information of different dimensions.

[0081] Please see below. Figure 3 The depthwise convolution module is based on depthwise separable convolution and channel attention, hence it can be named DWBlock. DWBlock contains two 1×1 convolutional layers at the beginning and end, one depthwise separable convolutional layer, and one layer consisting of a global average pooling layer (F...). gp ) and fully connected layer (F fcThe channel attention layer is composed of 1×1 convolutional layers in the header. DWBlock first expands the input fused feature image into a higher-dimensional feature space to increase non-linear representation through a 1×1 convolutional layer in the header, and then inputs the expanded fused feature image into a depthwise separable convolutional layer.

[0082] Compared to conventional convolutions, a kernel in a depthwise separable convolutional layer is responsible for computing only one channel, while the kernel in a conventional convolutional layer needs to perform a dot product operation with the features of all input channels. Therefore, the parameters of a depthwise separable convolutional layer are much smaller than those of a conventional convolutional layer. The output of the depthwise separable convolutional layer enters a channel attention layer for weight parameter adjustment, and finally, at the output stage, it is reduced in dimensionality by a 1×1 convolutional layer at the end. Additionally, there is a skip connection between the input and output to reduce the possibility of overfitting and gradient vanishing.

[0083] In the specific implementation process, this application adopts a four-stage hierarchical feature extractor. The multi-stage feature extractor performs downsampling after each stage, which reduces the size of the feature image. The purpose is to extract feature information in different dimensions: low-stage extracts texture information and high-stage extracts semantic information.

[0084] Each stage contains a stack of several DWBlocks. The first DWBlock in each stage modifies the size and number of channels of the fused feature image to extract feature information in different dimensions.

[0085] Due to the characteristics of convolutional neural networks, they pay more attention to local information in images. For smoke detection in foggy environments, fog and smoke have similar feature information, but generally, fog exists in the entire image, while smoke only exists in certain areas. Therefore, this application introduces a self-attention module on the basis of several deep convolutional modules (DWBlock), and introduces global features through its attention mechanism to enhance the smoke detection network model's ability to perceive the environment.

[0086] Specifically, the self-attention module treats each pixel as a "label" and calculates the correlation between each label. For an input feature information of shape C×H×W, the attention mechanism first flattens it into C×(H×W), denoted as X. Attention(X) is the global feature information of this feature information, and the specific calculation process is as follows:

[0087] Q = X·W Q K = X·W K V = X·W V

[0088]

[0089] Where Q, K, and V are called the query, key, and value, respectively, and W... Q W K W V ∈R C×d Here, is the learnable weight parameter, and d is the dimension of key K. Due to the flattening operation in the attention mechanism, the self-attention module ignores the positional information of its own pixels in the original image. Therefore, positional encoding is needed to extract the positional information. This application uses Locally-Enhanced Positional Encoding (LePE) to extract the positional information of pixels, thereby obtaining global feature information. The overall operation flow of the self-attention module is as follows: Figure 4 As shown.

[0090] The above embodiments differ from traditional convolutional neural networks by focusing more on local information. Furthermore, the self-attention module effectively extracts global features using an attention mechanism, thereby improving the smoke detection network model's ability to distinguish between fog and smoke and enhancing its robustness. In addition, this application is based on an image classification method and adopts a lightweight network design, designing a depthwise separable convolution module (DWBlock) to reduce the number of parameters. This allows for effective deployment on edge computing devices, improving the real-time performance of smoke detection.

[0091] In some embodiments, the classification prediction head may specifically include a global pooling layer and a linear classifier.

[0092] The above-mentioned prediction of global feature information using a classification prediction head yields smoke detection results, including:

[0093] A standard-sized feature image is obtained by performing mean pooling on global feature information through a global pooling layer.

[0094] The smoke detection results are obtained by predicting the feature image of a standard size using a linear classifier.

[0095] In this context, a standard-sized feature image refers to a feature image of size C×1×1.

[0096] Specifically, a linear classifier is used to predict the scores of four image categories (i.e., normal, smoke, fog, or both) on a standard-sized feature image, and the highest score is output as the smoke detection result.

[0097] The above embodiments use the predicted score as the basis for detection and select the image category corresponding to the highest score as the smoke detection result output, which further improves the accuracy of smoke detection results.

[0098] Please see Figure 5 In some embodiments, the method further includes:

[0099] Construct a pre-trained network model.

[0100] Training steps: Preprocess the foggy images to obtain training image samples; input the training image samples into the pre-trained network model to obtain detection results; the training image samples include RGB color feature samples, HSV color model samples, dark channel mapping samples, and true category information.

[0101] Update steps: Calculate the training loss value based on the detection results and the true class information; calculate the back gradient value based on the training loss value and the gradient descent algorithm; update the weight parameters of the pre-trained network model based on the back gradient value.

[0102] The training and update steps are repeated on the pre-trained network model according to the preset number of iterations to obtain the trained smoke detection network model.

[0103] Before the training image samples are input into the pre-trained network model, the device has already determined whether the training image samples contain smoke or fog by reading the real category information in them.

[0104] In the specific implementation process, the stochastic gradient descent optimizer (SGD) is used as the optimizer, with a learning rate of 0.005 and a preset number of iterations of 50. That is, after 50 detection iterations, the network model converges and becomes a trained smoke detection network model that can perform smoke detection.

[0105] The above embodiment calculates the loss between the detection result output by the pre-trained network model and the true category information of the input training image samples, and updates the weight parameters of the pre-trained network model according to the back gradient value obtained from the training loss value. This allows the pre-trained network model to gradually make up for the gap between the detection value and the true value in each iteration, thereby enabling it to accurately detect smoke after training convergence.

[0106] In some embodiments, the preprocessing of the foggy image described above to obtain training image samples includes:

[0107] The foggy image was downsampled using bilinear interpolation to obtain the sampled image.

[0108] The sampled image is cropped according to the principle of translation invariance to obtain training image samples.

[0109] Specifically, the size of the training image samples obtained after cropping is 224×224.

[0110] The above embodiments reduce the amount of data that the pre-trained network model needs to process by downsampling and cropping the original foggy images, thereby improving training efficiency.

[0111] In some embodiments, the calculation of the training loss value based on the detection results and the true category information includes:

[0112] Based on the detection results and the true category information, the training loss value is calculated using the cross-entropy loss function.

[0113] Specifically, the formula for calculating the cross-entropy loss function is as follows:

[0114]

[0115] Where N is the number of samples, K is the number of detection results, and y i,k For the true category information of the i-th sample and the k-th category, p i,k Let be the predicted probability of the k-th detection result for the i-th sample.

[0116] The above embodiments update the weight parameters of the pre-trained network model by calculating the training loss value between the true category information and the detection result, thereby obtaining a smoke detection network model with high detection accuracy.

[0117] Please see Figure 6 Another embodiment of this application provides a vision-based fog and smoke detection device, which is applied to a smoke detection equipment and includes:

[0118] The sampling module 101 is used to downsample the acquired foggy environment image to obtain the image to be detected.

[0119] The calculation module 102 is used to obtain the HSV color model of the image to be detected based on the RGB color features of the image to be detected, and to obtain the dark channel mapping of the image to be detected based on the image to be detected and the dark channel prior.

[0120] The detection module 103 is used to input the image to be detected, the HSV color model, and the dark channel mapping into the trained smoke detection network model to obtain the smoke detection result of the image to be detected.

[0121] The above embodiment provides a vision-based fog and smoke detection device. When detecting the image to be detected obtained by sampling the image of the foggy environment, it introduces the RGB color features, HSV color model and dark channel prior information of the image. This is beneficial for the trained smoke detection network model to extract and distinguish the features of smoke and fog, so as to accurately detect whether there is smoke in the image of the foggy environment, effectively improving the accuracy and precision of smoke detection in the foggy environment.

[0122] The specific limitations of the vision-based fog and smoke detection device provided in this embodiment can be found in the embodiment of the vision-based fog and smoke detection method described above, and will not be repeated here. Each module in the aforementioned vision-based fog and smoke detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0123] This application provides a computer device that may include a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it causes the processor to perform the steps of a vision-based fog and smoke detection method as described in any of the above embodiments.

[0124] The working process, working details, and technical effects of the computer device provided in this embodiment can be found in the embodiment of a vision-based fog and smoke detection method described above, and will not be repeated here.

[0125] This application provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the steps of a vision-based fog and smoke detection method as described in any of the above embodiments. The computer-readable storage medium refers to a data storage medium, which may include, but is not limited to, floppy disks, optical disks, hard disks, flash memory, USB flash drives, and / or Memory Sticks. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0126] The working process, working details, and technical effects of the computer-readable storage medium provided in this embodiment can be found in the embodiment of a vision-based fog and smoke detection method described above, and will not be repeated here.

[0127] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0128] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0129] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A vision-based method for detecting fog and smoke in foggy weather, characterized in that, The method includes: The acquired foggy environment image is downsampled to obtain the image to be detected; The HSV color model of the image to be detected is calculated based on the RGB color features of the image to be detected, and the dark channel mapping of the image to be detected is calculated based on the image to be detected and the dark channel prior. The image to be detected, the HSV color model, and the dark channel mapping are input into the trained smoke detection network model to obtain the smoke detection result of the image to be detected. The trained smoke detection network model includes a fusion attention module, a hierarchical feature extractor, and a classification prediction head. The hierarchical feature extractor includes several deep convolutional modules and a self-attention module. Specifically, the fusion attention module in the trained smoke detection network model performs feature attention fusion on the image to be detected, the HSV color model, and the dark channel mapping to obtain a fused feature image. Specifically, let the RGB color features I of the image to be detected RGB HSV color model I HSV Dark Channel Mapping I dark After passing through different convolutional blocks, respectively, we obtain... , and Three 16×112×112 features; then concatenated along the channel dimension. and This yields a 32×112×112 feature, to which an SE layer is applied; then, using... The extracted 1×112×112 spatial attention map yields the final 32×112×112 fused feature image, specifically as follows: in, Indicates the SE layer. Representation of features At pixel x, in the c-th channel, take the maximum value among the C channels and use it as the value of the spatial attention map A at x. This indicates element-wise multiplication; By modifying the size and number of channels of the fused feature image through several deep convolutional modules, feature information of different dimensions is extracted; global feature information is obtained by the self-attention module based on the feature information of different dimensions; and the smoke detection result is obtained by the classification prediction head based on the global feature information.

2. The method according to claim 1, characterized in that, The classification prediction head includes a global pooling layer and a linear classifier; the process of using the classification prediction head to predict the smoke detection result based on the global feature information includes: The global feature information is averaged by the global pooling layer to obtain a standard-sized feature image. The smoke detection result is obtained by predicting the standard-sized feature image using the linear classifier.

3. The method according to claim 1, characterized in that, The method further includes: Construct a pre-trained network model; Training steps: Preprocess the foggy image to obtain training image samples; input the training image samples into the pre-trained network model to obtain detection results; the training image samples include RGB color feature samples, HSV color model samples, dark channel mapping samples, and true class information; Update steps: Calculate the training loss value based on the detection results and the true category information; calculate the back gradient value based on the training loss value and the gradient descent algorithm; update the weight parameters of the pre-trained network model based on the back gradient value; The training and update steps are repeatedly executed on the pre-trained network model according to a preset number of iterations to obtain the trained smoke detection network model.

4. The method according to claim 3, characterized in that, The preprocessing of foggy images to obtain training image samples includes: The foggy image was downsampled using bilinear interpolation to obtain a sampled image; The sampled image is cropped according to the principle of translation invariance to obtain the training image sample.

5. The method according to claim 3, characterized in that, The step of calculating the training loss value based on the detection result and the true category information includes: Based on the detection results and the true category information, the training loss value is calculated using the cross-entropy loss function.

6. A vision-based fog and smoke detection device, characterized in that, The device includes: The sampling module is used to downsample the acquired foggy environment image to obtain the image to be detected; The calculation module is used to obtain the HSV color model of the image to be detected based on the RGB color features of the image to be detected, and to obtain the dark channel mapping of the image to be detected based on the image to be detected and the dark channel prior. The detection module is used to input the image to be detected, the HSV color model, and the dark channel mapping into a trained smoke detection network model to obtain the smoke detection result of the image to be detected. The trained smoke detection network model includes a fusion attention module, a hierarchical feature extractor, and a classification prediction head. The hierarchical feature extractor includes several deep convolutional modules and a self-attention module. Specifically, the fusion attention module in the trained smoke detection network model performs feature attention fusion on the image to be detected, the HSV color model, and the dark channel mapping to obtain a fused feature image. Specifically, let the RGB color features I of the image to be detected... RGB HSV color model I HSV Dark Channel Mapping I dark After passing through different convolutional blocks, respectively, we obtain... , and Three 16×112×112 features; then concatenated along the channel dimension. and This yields a 32×112×112 feature, to which an SE layer is applied; then, using... The extracted 1×112×112 spatial attention map yields the final 32×112×112 fused feature image, specifically as follows: in, Indicates the SE layer. Representation of features At pixel x, in the c-th channel, take the maximum value among the C channels and use it as the value of the spatial attention map A at x. This indicates element-wise multiplication; By modifying the size and number of channels of the fused feature image through several deep convolutional modules, feature information of different dimensions is extracted; global feature information is obtained by the self-attention module based on the feature information of different dimensions; and the smoke detection result is obtained by the classification prediction head based on the global feature information.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5.

Images