Smoke image detection method based on split Top-K attention mechanism

Abstract

The invention relates to the technical field of fire detection, in particular to a smoke image detection method based on a split Top-K attention mechanism, and the method comprises the steps: training a smoke network model through employing a full supervision method; an extraction module of a third stage and an extraction module of a fourth stage of a feature extraction network of the smoke network model are constructed based on a Transform module of a split Top-K attention mechanism, global features can be effectively extracted from long-distance noise of an image, local and global features are effectively aggregated, and meanwhile noise of an irrelevant long-range dependency relationship is reduced. And the cross-stage fusion module aggregates the output from the four stages, so that the flexibility and fitness of smoke characteristics on different scales can be improved, and the detection precision is improved. And inputting an aggregation result into a linear classifier to predict scores of four preset categories. In conclusion, the accuracy of smoke image category detection under the foggy weather condition is improved.

Description

Technical Field

[0001] The present application relates to the field of fire detection technology, and in particular to a smoke image detection method based on a split Top-K attention mechanism. Background Art

[0002] In industrial environments, fires pose a major threat to human life and cause significant property losses. Therefore, timely detection and control of fires are crucial. In the early stages of a fire, smoke usually appears before the flames, so smoke detection is an important part of the early fire warning system. Smoke detection can be mainly divided into two technical routes. One is based on smoke sensors for detection. Its advantage is high detection accuracy, but the detection range is limited and it is only applicable to specific indoor scenes. In complex industrial workshops and outdoor IoT environments, fixed-position smoke sensors will cause delays and high false alarm rates. The other is based on computer vision technology for detection. The widespread use of monitoring equipment in the IoT environment has given this type of technology a broad space for development. The smoke detection methods based on computer vision can be roughly divided into two categories, one is the traditional image processing method, and the other is the method based on deep learning.

[0003] Early image vision-based smoke detection methods usually use artificially designed prior information based on smoke statistical characteristics, such as color features, texture features, statistical properties, etc. For example, the color of smoke may change significantly when a fire occurs, while the texture characteristics of smoke may reflect its diffusion pattern and speed. However, since the morphology and characteristics of smoke may vary greatly depending on the type of fire, burning materials, and environmental conditions, fixed prior features may not accurately capture smoke in all cases. Moreover, changes in environmental conditions have a particularly significant impact on the method. Statistical properties may also be affected by environmental noise or background changes, increasing the possibility of false alarms. The disadvantage of this method is that it usually requires manual adjustment of filtering parameters, which makes the method applicable only under certain specific conditions. When the detection conditions change, such as different environments and detection distances, there is usually a high false alarm rate. Convolutional neural networks (CNNs) have demonstrated powerful modeling capabilities in the field of image processing. Unlike prior-based methods, these deep learning models do not rely on a single feature, but automatically learn high-level features from a large amount of labeled data, which has been widely used in the field of vision-based smoke recognition and has achieved encouraging results. Deep learning-based methods such as CNNs have greatly improved the accuracy of smoke detection. However, due to the inherent limited receptive field of CNNs and the lack of well-designed architectures, their detection accuracy still needs to be improved. Most studies focus on normal conditions, and limited research has been done on smoke detection under foggy conditions. To address the limitations of CNNs, the Transformer architecture has been introduced to various visual tasks, including smoke detection, where it has shown excellent performance. However, these Transformer-based methods usually have large parameter counts and poor temporal performance, which makes them unsuitable for deployment on edge computing devices. In summary, existing smoke detection methods have the following main disadvantages: prior-based methods lack generalization and accuracy; deep learning-based methods usually have large parameter counts and poor temporal performance, and poor real-time and deployable feasibility on edge devices; most existing methods do not take into account foggy conditions, in which the feature similarity between smoke and fog significantly increases the false detection rate.

[0004] Therefore, how to improve the accuracy of smoke image detection under foggy conditions is an urgent problem to be solved. Summary of the invention

[0005] The present application provides a smoke image detection method based on a split Top-K attention mechanism. By constructing a deep learning network based on the split Top-K attention mechanism, global features can be effectively extracted from the long-distance noise of the image, thereby improving the accuracy of smoke detection under foggy conditions.

[0006] In order to achieve the above object, the present invention adopts the following technical scheme:

[0007] In a first aspect, the present invention provides a smoke image detection method based on a split Top-K attention mechanism, the method comprising:

[0008] Constructing a smoke network model, training the smoke network model using a fully supervised method to obtain a pre-trained smoke network model, wherein the smoke network model includes an adaptive dark channel prior guidance network, a feature extraction network, a cross-stage fusion module, and a linear classifier from the input end to the output end, wherein the feature extraction network includes extraction modules at four different stages, and the extraction modules at the third and fourth stages are constructed based on the Transformer module of the Top-K attention mechanism;

[0009] Collecting a smoke image and inputting it into the adaptive dark channel prior guidance network of the pre-trained smoke network model;

[0010] The adaptive dark channel prior guide network extracts features of the smoke image, and uses the features to distinguish smoke and non-smoke areas to obtain enhanced image features;

[0011] The feature extraction network receives the enhanced image features, and each extraction module sequentially downsamples the enhanced image features to obtain a hierarchical information feature map corresponding to each stage, wherein the extraction modules of the third and fourth stages aggregate local features and global features in the enhanced image features to reduce noise irrelevant to long-range dependencies;

[0012] The stage fusion module aggregates the hierarchical information feature maps of different stages to obtain aggregated features, and inputs the aggregated features into the linear classifier;

[0013] The linear classifier predicts scores of different stages based on the aggregated features, and outputs a category corresponding to the smoke image based on the scores.

[0014] In a preferred example of the present application, it can be further configured that the adaptive dark channel prior guidance network includes a mapping module, an adaptive dark channel extraction module and a prior information fusion module, and the adaptive dark channel prior guidance network extracts features of the smoke image, uses the features to distinguish smoke and non-smoke areas, and obtains enhanced image features, including:

[0015] The mapping module maps the smoke image to obtain mapping features;

[0016] The adaptive dark channel extraction module extracts the prior information of the smoke image;

[0017] The prior information fusion module fuses the mapping features with the prior information to obtain enhanced image features.

[0018] In a preferred example of the present application, it can be further configured that the adaptive dark channel extraction module extracts the prior information of the smoke image, including:

[0019] The smoke image Convert to a tensor Where B and C represent the batch size and the number of channels respectively. represents a set of real numbers, H and W represent the height and width of the image respectively, and the tensor is processed by the minimum operation to obtain the minimum channel tensor;

[0020] Using a maximum pooling layer with a window size of k and applying two negative extractions to the minimum value in the region, the calculation process is as follows:

[0021]

[0022] Among them, T dark is the extracted dark channel tensor, Represents the maximum pooling layer with a window size of k;

[0023] T dark It is input into two convolutional layers and converted into a size of A dark channel prior feature matrix of learnable parameters is provided, and prior information of the smoke image is obtained based on the dark channel prior feature matrix.

[0024] In a preferred example of the present application, it can be further configured that the extraction modules of the third and fourth stages aggregate the local features and global features in the enhanced image features, including:

[0025] The extraction modules of the third and fourth stages aggregate local features and global features in the enhanced image features, including:

[0026] Define the number of attention heads in the i-th stage as N hi , by query and key Generating Attention Maps L = H × W represents the sequence length, where H and W represent the height and width of the image, respectively, and D represents the dimension of the feature. represents the set of real numbers;

[0027] During the self-attention process, the attention map M is divided into two parts using a head-splitting operation, and we get and The split ratio is determined by the head split ratio α;

[0028] The Top-K attention mechanism in the third and fourth stages processes M α , get local features;

[0029] Using dense attention to capture M 1-α The global features are obtained by deriving the global features.

[0030] In a preferred example of the present application, it can be further configured that a Top-K selection operator is introduced into the extraction modules of the third and fourth stages, and the Top-K selection operator is used to identify and retain K attention scores.

[0031] In a preferred example of the present application, it can be further configured that the stage fusion module aggregates the hierarchical information feature graphs of different stages to obtain aggregated features, including:

[0032] The feature alignment module of the stage fusion module normalizes the hierarchical information feature maps of different stages to the same spatial resolution and high number of channels to obtain a standardized feature map;

[0033] The multi-head cross attention module of the stage fusion module receives the standardized feature map, extracts and integrates the information of the standardized feature maps of different stages, and obtains aggregated features.

[0034] In a second aspect, the present application provides a smoke image detection device based on a split Top-K attention mechanism, the device comprising:

[0035] A model building module, used to build a smoke network model, train the smoke network model using a fully supervised method to obtain a pre-trained smoke network model, wherein the smoke network model includes an adaptive dark channel prior guidance network, a feature extraction network, a cross-stage fusion module and a linear classifier from the input end to the output end, wherein the feature extraction network includes extraction modules at four different stages, and the extraction modules at the third and fourth stages are constructed based on the Transformer module of the Top-K attention mechanism;

[0036] A data acquisition module, used for acquiring smoke images and inputting them into the adaptive dark channel prior guidance network of the pre-trained smoke network model;

[0037] A dark channel module, used for controlling the adaptive dark channel prior guidance network to extract features of the smoke image, and using the features to distinguish smoke and non-smoke areas to obtain enhanced image features;

[0038] A feature extraction network module is used to control the feature extraction network to receive the enhanced image features, and each extraction module sequentially downsamples the enhanced image features to obtain a hierarchical information feature map corresponding to each stage, wherein the extraction modules of the third and fourth stages aggregate local features and global features in the enhanced image features to reduce noise irrelevant to long-range dependencies;

[0039] A stage fusion module, used for controlling the stage fusion module to aggregate the hierarchical information feature graphs of different stages to obtain aggregated features, and input the aggregated features into the linear classifier;

[0040] The linear classifier module is used to control the linear classifier to predict scores of different stages based on the aggregated features, and output the category corresponding to the smoke image based on the scores.

[0041] In a third aspect, the present application provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the smoke image detection method based on the split Top-K attention mechanism as described in any one of the above items are implemented.

[0042] In a fourth aspect, the present application provides a computer-readable storage medium having a program stored thereon, wherein when the program is executed by a processor, a smoke image detection method based on a split Top-K attention mechanism as described in any of the above items is implemented.

[0043] In a fifth aspect, the present application provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps of the smoke image detection method based on the split Top-K attention mechanism as described in any one of the above items.

[0044] In summary, compared with the prior art, the technical solution provided in the embodiment of the present application has at least the following beneficial effects:

[0045] The present application provides a smoke image detection method based on a split Top-K attention mechanism. By constructing a smoke network model, the smoke network model is trained using a fully supervised method. Then, a smoke image is collected and input into the pre-trained smoke network model. The model outputs the classification of the smoke image, which can classify the smoke image under foggy conditions more accurately. Among them, the adaptive dark channel prior guidance network in the model can enhance the representation of the smoke image under haze conditions. The feature extraction network obtains the hierarchical information of the image by downsampling. The extraction modules of the third and fourth stages of the feature extraction network are constructed based on the Transformer module of the split Top-K attention mechanism, which can effectively extract global features from the long-range noise of the image, effectively aggregate local and global features, and reduce the noise of irrelevant long-range dependencies. The cross-stage fusion module aggregates the outputs from the four stages, which can improve the flexibility and adaptability of smoke features at different scales, thereby improving the detection accuracy. The aggregation result is input into a linear classifier, and the scores of the four preset categories can be predicted, and the four categories are normal, smoke, fog, and smoke with fog. In summary, this method improves the accuracy of smoke image category detection under foggy conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A flowchart of a smoke image detection method based on a split Top-K attention mechanism is provided for one embodiment of the present application.

[0047] Figure 2 A smoke detection model structure diagram of a smoke image detection method based on a split Top-K attention mechanism provided in an embodiment of the present application.

[0048] Figure 3 A structural diagram of a split Top-K attention mechanism for a smoke image detection method based on a split Top-K attention mechanism provided in one embodiment of the present application.

[0049] Figure 4 A structural diagram of a dual-path attention downsampling method for smoke image detection based on a split Top-K attention mechanism provided in one embodiment of the present application.

[0050] Figure 5 A structural diagram of an adaptive dark channel prior guidance network for a smoke image detection method based on a split Top-K attention mechanism provided in one embodiment of the present application.

[0051] Figure 6 A structural diagram of a prior information fusion module of a smoke image detection method based on a split Top-K attention mechanism provided in one embodiment of the present application.

[0052] Figure 7 A cross-stage fusion module structure diagram of a smoke image detection method based on a split Top-K attention mechanism provided in an embodiment of the present application.

[0053] Figure 8 A structural diagram of a smoke image detection device based on a split Top-K attention mechanism provided for one embodiment of the present application. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0055] In one embodiment of the present application, a smoke image detection method based on a split Top-K attention mechanism is provided. Figure 1 As shown, the method includes:

[0056] S100: Construct a smoke network model, and use a fully supervised method to train the smoke network model to obtain a pre-trained smoke network model. The smoke network model includes an adaptive dark channel prior guidance network, a feature extraction network, a cross-stage fusion module and a linear classifier from the input end to the output end. The feature extraction network includes extraction modules at four different stages. The extraction modules at the third and fourth stages are constructed based on the Transformer module of the Top-K attention mechanism.

[0057] Specifically, the structure of the smoke network model is as follows: Figure 2 As shown in Figure 1, it includes an adaptive dark channel prior guidance network, a feature extraction network consisting of four stages, a cross-stage fusion module and a linear classifier ( Figure 2 The network is an end-to-end fully supervised deep learning network. The network directly learns the mapping from smoke image to smoke image category. The input of the network is a 3-channel RGB image, and the output is the category of the image, which includes normal, smoke, fog, and smoke with fog. The system determines whether to send an alarm to the control center based on the output image category.

[0058] The extraction modules in the third and fourth stages are based on the Transformer module structure of the Top-K attention mechanism. Figure 2 The calculation process is as follows:

[0059] X′ l =X l-1 +STKA(X l-1 );

[0060] X l =X′ l +Le-MLP(X l-1 );

[0061] where X′ l and X l Represents the output of split Top-K attention and local enhancement MLP (Le-MLP).

[0062] In standard self-attention, for a given query Q, key K, and value V, the calculation process is as follows:

[0063] Attention(Q,K,V)=softmax(QK T / √d+pose)V;

[0064] Where d represents the dimension of the input feature sequence and pose represents the position encoding. In the standard global attention, all pixels are closely connected, which weakens the local features and introduces irrelevant long-range dependent noise, affecting the recognition accuracy. This problem is particularly important in the smoke detection task.

[0065] To solve this problem, the split Top-K attention mechanism (STKA) used in this embodiment can filter out irrelevant noise while retaining the most effective attention score and adaptively maintaining dense attention capabilities. The specific structure is as follows: Figure 3 shown.

[0066] Specifically, suppose the number of attention heads in the i-th stage is defined as N hi , by query and key Generating Attention Maps L = H × W represents the sequence length, where H and W represent the height and width of the image, respectively, and D represents the dimension of the feature. represents the set of real numbers, .

[0067] Different from traditional self-attention, we use a head-splitting operation to split the attention map M into two parts, obtaining and The split ratio is determined by the head split ratio α.

[0068] Ma is processed by the Top-K attention mechanism to filter out irrelevant long-term dependencies and retain the most effective attention scores.

[0069] A Top-K selection operator is introduced, which identifies and retains the most informative K attention scores, ensuring that each query only interacts with the most relevant keys and reducing noise and unnecessary interference information.

[0070] The other part, M 1-α Used for regular dense attention to adaptively capture necessary global features.

[0071] Finally, the two parts of the attention map are concatenated and softmaxed to obtain the attention score, which is then multiplied by the value to produce the final output.

[0072] The calculation process of the split Top-K attention mechanism (STKA) is summarized as follows:

[0073]

[0074] STKA(Q,K,V)=softmax(Cat[τ k (M α ), M 1-α ])V;

[0075] Among them, τ k is a learnable Top-K selection operator:

[0076]

[0077] For hierarchical feature extraction backbone networks, effective downsampling strategies can reduce the loss of high-dimensional information and enhance the representation ability of the model. Currently, most visual backbone networks use strided convolution or pooling layers for downsampling. Recently, some studies have begun to explore attention-based downsampling methods, specifically by reducing the number of query tokens in the attention mechanism to achieve downsampling. Dual-path attention downsampling (DP-Attention-DS) combines the advantages of both, such as Figure 4 shown.

[0078] DP-Attention DS consists of two branches: convolutional downsampling and attention downsampling. For attention downsampling, we use pooling as a static local downsampling method and 3×3 deep convolution as a learnable local downsampling method, and combine the results and project them to the query dimension. The results of the attention downsampling module are residually connected with the downsampling results of the strided convolution to form a local and global integration. However, the dual-path attention downsampling also introduces additional computational overhead. In order to maintain real-time reasoning while ensuring accuracy, this embodiment applies DP-Attention-DS between the two stages containing the STKformer block and uses strided convolution for downsampling in the early high-resolution stage.

[0079] The smoke network model is trained using a fully supervised method, and the specific steps include:

[0080] The network is first trained using a labeled dataset containing images of various categories. Each sample in the dataset is labeled with a label indicating the category it belongs to. During the training process, the network learns to identify image features through forward propagation and adjusts weights through back propagation of the loss function to minimize the prediction error. After multiple rounds of iterative training, the network gradually learns to distinguish images into different categories, thereby detecting images containing smoke. The specific training process includes the following steps:

[0081] The training method is a fully supervised method, which is trained by using a labeled dataset containing images of multiple categories. Each sample in the dataset is attached with a label indicating the category it belongs to. During the training process, the network learns to recognize image features through forward propagation and adjusts the weights through back propagation of the loss function to minimize the prediction error.

[0082] The network structure parameters can be set to the number of layers in the following four stages, denoted as [N1, N2, N3, N4], set to [3, 3, 9, 6]. The corresponding channel numbers, denoted as [C1, C2, C3, C4], are set to [32, 48, 120, 224]. In the adaptive dark channel prior guidance network, the window size of the Maxpooling layer used by the adaptive dark channel extraction module is set to k = 5. For the last two stages, the head split ratio α of STKA is set to 0.75 and TopK is set to [100, 25].

[0083] First, we pre-trained the model for 300 epochs on the ImageNet dataset. Then, we fine-tuned the pre-trained network parameters and trained each model 100 times with a basic learning rate of 0.0002 and an initial learning rate of 2×10 -7 5 warm-up periods with a minimum learning rate of 1×10 -6 This example uses a weight decay of 0.05 and applies gradient clipping at 5.0 to stabilize training.

[0084] The AdamW optimizer is used, and epsilon is set to 1×10 -8 , beta is (0.9, 0.999). For the training and verification of the network, the present invention adjusts the image size to 224×224 as input and normalizes it. After experimental verification, the trained network model has a high accuracy rate for smoke detection.

[0085] S200: Collecting a smoke image and inputting the image into the adaptive dark channel prior guidance network of the pre-trained smoke network model;

[0086] S300: The adaptive dark channel prior guidance network extracts features of the smoke image, and uses the features to distinguish smoke and non-smoke areas to obtain enhanced image features;

[0087] Specifically, the structure of the adaptive dark channel prior guidance network is as follows: Figure 5 As shown in the figure, the adaptive dark channel extraction module enhances the adaptability of the model to complex foggy environments, and the prior information fusion module uses a fusion attention strategy to improve the accuracy of smoke detection while retaining image details. The input RGB smoke image has two routes, one is mapped into the feature space through the input mapping module, and the other is input into a learnable prior knowledge extraction module, and then this prior information is fused with the mapping features of the smoke image through a prior knowledge fusion module. These two modules work together to extract and utilize key image features, effectively distinguish between smoke and non-smog areas, and adaptively reduce the impact of foggy conditions.

[0088] S400: the feature extraction network receives the enhanced image features, and each extraction module sequentially downsamples the enhanced image features to obtain a hierarchical information feature map corresponding to each stage, wherein the extraction modules of the third and fourth stages aggregate local features and global features in the enhanced image features to reduce noise irrelevant to long-range dependencies;

[0089] The structure of the feature extraction network is as follows Figure 2 As shown, this embodiment uses dual-path attention downsampling, and the specific implementation includes:

[0090] For hierarchical feature extraction backbone networks, effective downsampling strategies can reduce the loss of high-dimensional information and enhance the representation ability of the model. Currently, most visual backbone networks use strided convolution or pooling layers for downsampling. Recently, some studies have begun to explore attention-based downsampling methods, specifically by reducing the number of query tokens in the attention mechanism to achieve downsampling. Dual-path attention downsampling (DP-Attention-DS) combines the advantages of both, such as Figure 4 As shown in Figure 2. DP-Attention-DS consists of two branches: convolutional downsampling and attention downsampling. For attention downsampling, we use pooling as a static local downsampling method and 3×3 depth convolution as a learnable local downsampling method, and combine the results and project them to the query dimension. The result of the attention downsampling module is residually connected with the downsampling result of the strided convolution to form a local and global integration.

[0091] However, dual-path attention downsampling also introduces additional computational overhead. In order to maintain real-time reasoning while ensuring accuracy, this embodiment applies DP-Attention-DS between the two stages containing the STKformer block and uses stride convolution for downsampling in the early high-resolution stage.

[0092] S500: The stage fusion module aggregates the hierarchical information feature graphs of different stages to obtain aggregated features, and inputs the aggregated features into the linear classifier;

[0093] S600: The linear classifier predicts scores of different stages based on the aggregated features, and outputs a category corresponding to the smoke image based on the scores.

[0094] In this embodiment, by constructing a smoke network model, the smoke network model is trained using a fully supervised method, and then, a smoke image is collected and input into the pre-trained smoke network model. The model outputs the classification of the smoke image, which can more accurately classify the smoke image under foggy conditions. Among them, the adaptive dark channel prior guidance network in the model can enhance the representation of the smoke image under haze conditions. The feature extraction network obtains the hierarchical information of the image by downsampling. The extraction modules of the third and fourth stages of the feature extraction network are constructed based on the Transformer module of the split Top-K attention mechanism, which can effectively extract global features from the long-range noise of the image, effectively aggregate local and global features, and reduce the noise of irrelevant long-range dependencies. The cross-stage fusion module aggregates the outputs from the four stages, which can improve the flexibility and adaptability of smoke features at different scales, thereby improving the detection accuracy. The aggregation result is input into the linear classifier, and the scores of the preset four categories can be predicted, and the four categories are normal, smoke, fog, and smoke with fog. In summary, this method improves the accuracy of smoke image category detection under foggy conditions.

[0095] In some embodiments, the adaptive dark channel prior guidance network includes a mapping module, an adaptive dark channel extraction module and a prior information fusion module, and the adaptive dark channel prior guidance network extracts features of the smoke image, uses the features to distinguish smoke and non-smoke areas, and obtains enhanced image features, including:

[0096] The mapping module maps the smoke image to obtain mapping features;

[0097] The adaptive dark channel extraction module extracts the prior information of the smoke image. Specifically, the implementation steps include:

[0098] The smoke image Convert to a tensor Where B and C represent the batch size and the number of channels respectively. represents a set of real numbers, H and W represent the height and width of the image respectively, and the tensor is processed by the minimum operation to obtain the minimum channel tensor;

[0099] Using a maximum pooling layer with a window size of k and applying two negative extractions to the minimum value in the region, the calculation process is as follows:

[0100]

[0101] Among them, T dark is the extracted dark channel tensor, Represents the maximum pooling layer with a window size of k;

[0102] T dark It is input into two convolutional layers and converted into a size of A dark channel prior feature matrix of learnable parameters is provided, and prior information of the smoke image is obtained based on the dark channel prior feature matrix.

[0103] The prior information fusion module fuses the mapping features with the prior information to obtain enhanced image features.

[0104] In practice, traditional dark channel extraction directly selects the minimum brightness value from the image. Given an input image The corresponding dark channel image I dark It can be obtained by the following formula:

[0105] Among them, I c (y) represents the cth channel of the I channel at pixel y, and Ω(x) represents the local area at x. This method of extracting dark channels effectively identifies fog and mitigates its impact on visual images. However, this method is not suitable for real-time smoke detection systems and cannot be accelerated and trained using deep learning frameworks.

[0106] Furthermore, its fixed parameters limit its generalization ability.

[0107] Therefore, this embodiment uses an adaptive dark channel extraction strategy, which calculates the dark channel in tensor form, improving the speed of the model. This method combines the dark channel prior with the deep learning feature extraction network to improve the adaptability to complex fog environments.

[0108] The process is similar to traditional dark channel extraction. First, the input image Convert to a tensor Where B and C represent the batch size and the number of channels, respectively. The input tensor is first processed by the minimum operation to obtain the minimum channel tensor. Different from the original minimum filter in traditional dark channel extraction, the present invention uses a maximum pooling layer with a window size of k and applies two negations to achieve the minimum value extraction in the region. This design allows the dark channel extraction network to be trained and tested in parallel, and the deep learning framework is used to accelerate the calculation. The calculation process can be summarized as follows:

[0109]

[0110] Among them, T dark is the extracted dark channel tensor. T dark It is further input into two convolutional layers to transform it into a network of size The dark channel prior feature matrix with learnable parameters is constructed. This feature matrix helps the subsequent feature extraction process and enhances the recognition robustness under haze conditions.

[0111] In order to better integrate the adaptive dark channel feature matrix into the subsequent feature extraction process, this embodiment uses a prior information fusion module. This module adopts a structure similar to Transformer, replacing the traditional self-attention layer with prior fusion attention to utilize prior information that helps the feature extraction process, thereby enhancing the representation ability of the model. Its operation process is as follows:

[0112] X′ out =PFA[X p , X I ]+X I ;

[0113] X out =LeMLP(X′ out )+X′ out ;

[0114] Specifically, the features of the original input embedding features are used to generate the Key (K) and Value (V) of PFA through 1×1 convolution and 3×3 deep convolution. At the same time, the prior features extracted by the adaptive dark channel extraction module generate the query (Q prior ) and the additional V value (V prior ). The Key (K) generated from the original output and the Query (Q) derived from the prior features prior ) is used in the transposed attention mechanism to calculate the prior fused attention map (M prior ). Then the obtained M prior With V and V prior Together they are used for attention point multiplication. The final results are concatenated and passed through a 1×1 convolutional layer for channel adjustment to obtain the final output. This process enhances the model's predictive power by integrating prior information and current input data. The overall module is as follows Figure 6 shown.

[0115] In this embodiment, an adaptive dark channel prior guidance mechanism is used, which enhances smoke recognition under foggy conditions and guides network learning more effectively compared to the prior art. It uses a pooling operation instead of a minimum filter, which enables the dark channel prior extraction module to effectively use learnable parameters to adaptively reduce the impact of fog. The extracted prior information guides feature extraction through the prior information fusion module, improving the robustness of the model.

[0116] In some embodiments, the stage fusion module aggregates the hierarchical information feature graphs of different stages to obtain aggregated features, including:

[0117] The feature alignment module of the stage fusion module normalizes the hierarchical information feature maps of different stages to the same spatial resolution and high number of channels to obtain a standardized feature map;

[0118] The multi-head cross attention module of the stage fusion module receives the standardized feature map, extracts and integrates the information of the standardized feature maps of different stages, and obtains aggregated features.

[0119] In practice, the various sizes and shapes of smoke pose a major challenge in smoke recognition tasks. In the hierarchical feature extraction network, information processed at different stages is processed in different dimensions, resulting in differences in their ability to recognize smoke of different sizes. In order to effectively utilize the feature information output by the hierarchical network, this embodiment uses a cross-stage fusion module, such as Figure 7 As shown in the figure, it mainly consists of a feature alignment module and a multi-head cross attention (MHCA). The goal is to enable the model to effectively aggregate feature information from different stages, rather than relying solely on feature information from the final stage for classification. This flexibility enables the model to adapt to the characteristics of smoke of different sizes and improve the accuracy of smoke detection.

[0120] The main function of the Feature Alignment Module (FAM) is to normalize the feature maps (F1, F2, F3, F4) at different stages of the network to the same spatial resolution and high number of channels. This ensures that features from different stages can be considered uniformly and thus effectively fused. Specifically, we first use an AvgPooling layer to reduce the size of the feature maps of the first three stages to Then, the features of all four stages are adjusted to the number of channels through the convolution layer Finally, the features of all four stages are concatenated to form the cross-stage fusion feature

[0121] The cross-stage fusion module adopts a multi-head cross-attention mechanism, which allows the network to learn how to effectively extract and integrate information from features at different stages. This enables the model to focus on finer feature differences, thereby improving detection accuracy. The calculation process of multi-head cross-attention is as follows Figure 7 As shown in the multi-head cross attention in.

[0122] Specifically, in Multi-Head Cross Attention (MHCA), the query comes from the output F4 of the final stage of the model, while the key and value come from the aggregated output F of all stages. fusion . Queries, keys, and values ​​are processed in MHCA to achieve feature interactions, ensuring that features in the early stages are effectively focused on in deeper levels. After completing the MHCA interaction, the data enters a feedforward network similar to that in Transformer to produce the final output. The calculation process can be summarized as follows:

[0123] Qb=Conv 1×1 (F4);

[0124] K h , V h =Conv 1×1 (F fusion );

[0125] F′ out =F4+MHCA(Q b , K h , V h );

[0126] F out =F′ out +LeMLP(F′ out );

[0127] Where Q b represents the query matrix from the underlying stage features, and K h and V h represents the key and value matrices from the hierarchical aggregation features. The input and final output of Le-MLP are denoted as F′ respectively. out and F out .

[0128] In this embodiment, by using a cross-stage fusion method, the problem of feature aggregation in the hierarchical network is overcome, the features of different stages are effectively combined, and flexible adaptation of smoke features at different scales is allowed, thereby improving the accuracy of smoke detection.

[0129] This application also provides a smoke image detection device based on the split Top-K attention mechanism, please refer to Figure 8 As shown, the device comprises:

[0130] A model building module 100 is used to build a smoke network model, and train the smoke network model using a fully supervised method to obtain a pre-trained smoke network model. The smoke network model includes an adaptive dark channel prior guidance network, a feature extraction network, a cross-stage fusion module, and a linear classifier from the input end to the output end. The feature extraction network includes extraction modules at four different stages, and the extraction modules at the third and fourth stages are constructed based on the Transformer module of the Top-K attention mechanism;

[0131] A data acquisition module 200 is used to collect smoke images and input them into the adaptive dark channel prior guidance network of the pre-trained smoke network model;

[0132] A dark channel module 300 is used to control the adaptive dark channel prior guidance network to extract features of the smoke image, and use the features to distinguish smoke and non-smoke areas to obtain enhanced image features;

[0133] A feature extraction network module 400 is used to control the feature extraction network to receive the enhanced image features, and each extraction module sequentially downsamples the enhanced image features to obtain a hierarchical information feature map corresponding to each stage, wherein the extraction modules of the third and fourth stages aggregate local features and global features in the enhanced image features to reduce noise irrelevant to long-range dependencies;

[0134] A stage fusion module 500 is used to control the stage fusion module to aggregate the hierarchical information feature graphs of different stages to obtain aggregated features, and input the aggregated features into the linear classifier;

[0135] The linear classifier module 600 is used to control the linear classifier to predict scores of different stages based on the aggregated features, and output the category corresponding to the smoke image based on the scores.

[0136] The functional implementation of each module in the above-mentioned smoke image detection device based on the split Top-K attention mechanism corresponds to the steps in the above-mentioned smoke image detection method embodiment based on the split Top-K attention mechanism, and its functions and implementation process will not be repeated here one by one.

[0137] The present application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the smoke image detection method based on the split Top-K attention mechanism as described in any of the above embodiments are implemented.

[0138] The present application also provides a computer-readable storage medium, on which a program is stored, wherein the computer-readable storage medium refers to a carrier for storing data, which may include but is not limited to a floppy disk, an optical disk, a hard disk, a flash memory, a USB flash drive and / or a memory stick, etc., and the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The working process, working details and technical effects of the computer-readable storage medium provided in this embodiment can be found in the above embodiment of a smoke image detection method based on a split Top-K attention mechanism, which will not be repeated here.

[0139] The application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the smoke image detection method based on the split Top-K attention mechanism as described in any of the above embodiments.

[0140] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0141] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. The above-mentioned embodiments only express several implementation methods of the present application, and their descriptions are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, without departing from the concept of the present application, several deformations and improvements can be made, which all belong to the protection scope of the present application. Therefore, the protection scope of the patent of this application shall be based on the attached claims.

Claims

1. A smoke image detection method based on split Top-K attention mechanism, It is characterized in that include: Constructing a smoke network model, training the smoke network model using a fully supervised method to obtain a pre-trained smoke network model, wherein the smoke network model includes an adaptive dark channel prior guidance network, a feature extraction network, a cross-stage fusion module, and a linear classifier from the input end to the output end, wherein the feature extraction network includes extraction modules at four different stages, and the extraction modules at the third and fourth stages are constructed based on the Transformer module of the Top-K attention mechanism; Collecting a smoke image and inputting it into the adaptive dark channel prior guidance network of the pre-trained smoke network model; The adaptive dark channel prior guide network extracts features of the smoke image, and uses the features to distinguish smoke and non-smoke areas to obtain enhanced image features; The feature extraction network receives the enhanced image features, and each extraction module sequentially downsamples the enhanced image features to obtain a hierarchical information feature map corresponding to each stage, wherein the extraction modules of the third and fourth stages aggregate local features and global features in the enhanced image features to reduce noise irrelevant to long-range dependencies; The stage fusion module aggregates the hierarchical information feature maps of different stages to obtain aggregated features, and inputs the aggregated features into the linear classifier; The linear classifier predicts scores of different stages based on the aggregated features, and outputs a category corresponding to the smoke image based on the scores.

2. According to claim 1, the smoke image detection method based on split Top-K attention mechanism, It is characterized in that The adaptive dark channel prior guidance network includes a mapping module, an adaptive dark channel extraction module and a prior information fusion module. The adaptive dark channel prior guidance network extracts the features of the smoke image and uses the features to distinguish smoke and non-smoke areas to obtain enhanced image features, including: The mapping module maps the smoke image to obtain mapping features; The adaptive dark channel extraction module extracts the prior information of the smoke image; The prior information fusion module fuses the mapping features with the prior information to obtain enhanced image features.

3. The smoke image detection method based on split Top-K attention mechanism according to claim 2, It is characterized in that The adaptive dark channel extraction module extracts the prior information of the smoke image, including: The smoke image Convert to a tensor Where B and C represent the batch size and the number of channels respectively. represents a set of real numbers, H and W represent the height and width of the image respectively, and the tensor is processed by the minimum operation to obtain the minimum channel tensor; Using a maximum pooling layer with a window size of k and applying two negative extractions to the minimum value in the region, the calculation process is as follows: Among them, T dark is the extracted dark channel tensor, Represents the maximum pooling layer with a window size of k; T dark It is input into two convolutional layers and converted into a size of A dark channel prior feature matrix of learnable parameters is provided, and prior information of the smoke image is obtained based on the dark channel prior feature matrix.

4. The smoke image detection method based on split Top-K attention mechanism according to claim 1, It is characterized in that The extraction modules of the third and fourth stages aggregate local features and global features in the enhanced image features, including: Define the number of attention heads in the i-th stage as N hi , by query and key Generating Attention Maps L = H × W represents the sequence length, where H and W represent the height and width of the image, respectively, and D represents the dimension of the feature. represents the set of real numbers; During the self-attention process, the attention map M is divided into two parts using a head-splitting operation, and we get and The split ratio is determined by the head split ratio α; The Top-K attention mechanism in the third and fourth stages processes M α , get local features; Using dense attention to capture M 1-α The global features are obtained by deriving the global features.

5. According to claim 4, the smoke image detection method based on split Top-K attention mechanism, It is characterized in that include: A Top-K selection operator is introduced into the extraction modules of the third and fourth stages, and the Top-K selection operator is used to identify and retain K attention scores.

6. The smoke image detection method based on split Top-K attention mechanism according to claim 1, It is characterized in that The stage fusion module aggregates the hierarchical information feature maps of different stages to obtain aggregated features, including: The feature alignment module of the stage fusion module normalizes the hierarchical information feature maps of different stages to the same spatial resolution and high number of channels to obtain a standardized feature map; The multi-head cross attention module of the stage fusion module receives the standardized feature map, extracts and integrates the information of the standardized feature maps of different stages, and obtains aggregated features.

7. A smoke image detection device based on split Top-K attention mechanism, It is characterized in that include: A model building module, used to build a smoke network model, train the smoke network model using a fully supervised method to obtain a pre-trained smoke network model, wherein the smoke network model includes an adaptive dark channel prior guidance network, a feature extraction network, a cross-stage fusion module and a linear classifier from the input end to the output end, wherein the feature extraction network includes extraction modules at four different stages, and the extraction modules at the third and fourth stages are constructed based on the Transformer module of the Top-K attention mechanism; A data acquisition module, used for acquiring smoke images and inputting them into the adaptive dark channel prior guidance network of the pre-trained smoke network model; A dark channel module, used for controlling the adaptive dark channel prior guidance network to extract features of the smoke image, and using the features to distinguish smoke and non-smoke areas to obtain enhanced image features; A feature extraction network module is used to control the feature extraction network to receive the enhanced image features, and each extraction module sequentially downsamples the enhanced image features to obtain a hierarchical information feature map corresponding to each stage, wherein the extraction modules of the third and fourth stages aggregate local features and global features in the enhanced image features to reduce noise irrelevant to long-range dependencies; A stage fusion module, used for controlling the stage fusion module to aggregate the hierarchical information feature graphs of different stages to obtain aggregated features, and input the aggregated features into the linear classifier; The linear classifier module is used to control the linear classifier to predict scores of different stages based on the aggregated features, and output the category corresponding to the smoke image based on the scores.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the steps of the smoke image detection method based on the split Top-K attention mechanism as described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a program, wherein when the program is executed by the processor, the smoke image detection method based on the split Top-K attention mechanism as described in any one of claims 1 to 6 is implemented.

10. A computer program product comprising computer instructions, It is characterized in that The computer instructions implement the steps of the method according to claims 1 to 6 when executed by a processor.

Images