Flame detection method and device based on multi-frequency perception and dynamic coding, and server

Abstract

The application discloses a flame detection method and device based on multi-frequency perception and dynamic coding and a server, and belongs to the technical field of target detection. The method comprises the following steps: shallow basic features are extracted from an image through a convolution layer and C2f, and then first-level multi-domain enhanced features and second-level deep enhanced features are obtained through twice convolution and a CSP_HPMB module; the shallow basic features, the first-level multi-domain enhanced features and the second-level deep enhanced features are input into a mixed encoder, convolution, up-sampling, AIFI-DCL and EGConvC3 reconstruction are performed, and shallow fine fusion features, middle cross-scale alignment features and deep global modulation features are generated; and the shallow fine fusion features, the middle cross-scale alignment features and the deep global modulation features are connected and then input into a decoder to generate a flame detection result. Through the network architecture of multi-level feature extraction, multi-domain enhancement and cross-scale fusion decoding, the aerial flame image detection is realized.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a flame detection method, device and server based on multi-frequency sensing and dynamic coding. Background Technology

[0002] Flame detection using drone aerial imagery is a crucial method for wide-area environmental monitoring. Traditional methods rely on operators using drones equipped with visible light or infrared payloads for inspection, followed by manual analysis of massive amounts of imagery to identify and locate flame targets. This approach is severely inadequate when dealing with the massive amounts of data in wide-area monitoring, the sudden and small nature of flame targets in their early stages, the presence of complex backgrounds from the aerial perspective, changes in lighting, and smoke obstruction. Furthermore, manual interpretation is highly dependent on experience, and prolonged work can lead to visual fatigue, resulting in missed detections and misjudgments.

[0003] Therefore, automated detection methods based on deep learning have gradually gained attention. Convolutional neural network target detection models, represented by the YOLO series and Faster R-CNN, have been attempted to be deployed on drone platforms for flame detection. These existing technologies construct a backbone network by stacking convolutional layers to extract features, fuse multi-scale information using feature pyramids, and complete target classification and localization based on preset anchor boxes or region proposals.

[0004] However, the backbone networks of existing detection models are limited by the local receptive field of convolution and lack the utilization of frequency domain information, making it difficult to effectively model the multi-scale global dependencies of flame targets from an aerial perspective. At the same time, the static feedforward network of the encoder cannot adapt to changes in flight conditions and lacks the ability to aggregate multi-scale contexts. Furthermore, the lightweight module design presents a contradiction between training efficiency and inference speed, which severely restricts the performance of flame detection. Summary of the Invention

[0005] This invention provides a flame detection method, device, and server based on multi-frequency sensing and dynamic coding to solve the technical problems of weak global modeling capability, insufficient utilization of frequency domain information, poor network adaptability, and limited accuracy of aerial flame detection in the prior art.

[0006] In a first aspect, embodiments of the present invention provide a flame detection method based on multi-frequency sensing and dynamic coding, comprising: The image is processed through convolutional layers and the C2f module to perform preliminary spatial feature extraction and shallow semantic aggregation, resulting in shallow basic features. These shallow basic features are then processed through convolutional layers and the CSP_HPMB module, and enhanced through three-domain collaborative features to obtain first-level multi-domain enhanced features. The first-level multi-domain enhanced features are processed by convolutional layers and the CSP_HPMB module, and then purified by global feature depth to obtain the second-level deep enhanced features. The CSP_HPMB module is fused with the hybrid perception Mamba sub-module. The hybrid perception Mamba sub-module performs spatial, channel, and frequency domain collaborative feature enhancement through hybrid scale unit, double shuffle attention unit, frequency unit and additive fusion unit. The shallow basic features, first-level multi-domain enhancement features, and second-level deep enhancement features are input into the hybrid encoder. Three-level multi-scale cross-layer feature fusion and reconstruction are performed through convolution, upsampling, AIFI-DCL module, and EGConvC3 module to obtain shallow refined fusion features, mid-level cross-scale alignment features, and deep global modulation features. The AIFI-DCL module performs global multi-scale adaptive feature extraction through multi-head attention and dual-channel dynamic convolution DCL structure. The EGConvC3 module includes convolution branches and multi-level cascaded EGBlock modules, and performs deep feature extraction through the multi-path parallel structure in the EGBlock module. The shallow refined fusion features, the mid-level cross-scale alignment features, and the deep global modulation features are concatenated to generate aggregated features. The aggregated features are then processed by a decoder to generate flame detection results.

[0007] Secondly, embodiments of the present invention also provide a flame detection device based on multi-frequency sensing and dynamic coding, comprising: The basic feature and first-level feature acquisition module is used to process the image through convolutional layers and C2f module to perform preliminary spatial feature extraction and shallow semantic aggregation to obtain shallow basic features. The shallow basic features are then processed through convolutional layers and CSP_HPMB module, and first-level multi-domain enhanced features are obtained through three-domain collaborative feature enhancement. The secondary feature acquisition module is used to process the primary multi-domain enhanced features through convolutional layers and the CSP_HPMB module, and to obtain secondary deep enhanced features through global feature depth purification. The CSP_HPMB module integrates the hybrid perception Mamba sub-module, which performs spatial, channel, and frequency domain collaborative feature enhancement through hybrid scale unit, double shuffle attention unit, frequency unit and additive fusion unit. The feature fusion and reconstruction module is used to input the shallow basic features, first-level multi-domain enhancement features, and second-level deep enhancement features into the hybrid encoder. Through convolution, upsampling, AIFI-DCL module, and EGConvC3 module, three-level multi-scale cross-layer feature fusion and reconstruction is performed to obtain shallow refined fusion features, mid-level cross-scale aligned features, and deep global modulation features. The AIFI-DCL module performs global multi-scale adaptive feature extraction through multi-head attention and dual-channel dynamic convolution DCL structure. The EGConvC3 module includes convolution branches and multi-level cascaded EGBlock modules, and performs deep feature extraction through the multi-path parallel structure in the EGBlock module. The flame detection module is used to connect the shallow refined fusion features, the mid-level cross-scale alignment features, and the deep global modulation features to generate aggregated features. The aggregated features are then processed by the decoder to generate flame detection results.

[0008] Thirdly, embodiments of the present invention also provide a server, comprising: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the flame detection method based on multi-frequency sensing and dynamic coding as provided in the above embodiments.

[0009] The flame detection method, apparatus, and server based on multi-frequency sensing and dynamic coding provided in this invention process images through convolutional layers and a C2f module for preliminary spatial feature extraction and shallow semantic aggregation to obtain shallow basic features. These shallow basic features are then processed through convolutional layers and a CSP_HPMB module, and enhanced with three-domain collaborative features to obtain first-level multi-domain enhanced features. The first-level multi-domain enhanced features are further processed through convolutional layers and a CSP_HPMB module, and refined with global feature depth to obtain second-level deep enhanced features. The CSP_HPMB module integrates a hybrid sensing Mamba sub-module, which performs spatial, channel, and frequency domain collaborative feature enhancement through a hybrid scale unit, a double shuffling attention unit, a frequency unit, and an additive fusion unit. The shallow basic features and first-level multi-domain enhanced features are then processed through convolutional layers and a C2f module to obtain first-level multi-domain enhanced features. Domain enhancement features and secondary deep enhancement features are input into the hybrid encoder. Three-level multi-scale cross-layer feature fusion and reconstruction are performed through convolution, upsampling, the AIFI-DCL module, and the EGConvC3 module to obtain shallow refined fusion features, mid-level cross-scale aligned features, and deep global modulation features. The AIFI-DCL module performs global multi-scale adaptive feature extraction through multi-head attention and a dual-channel dynamic convolutional DCL structure. The EGConvC3 module includes convolutional branches and multi-level cascaded EGBlock modules, performing deep feature extraction through a multi-path parallel structure within the EGBlock module. The shallow refined fusion features, mid-level cross-scale aligned features, and deep global modulation features are concatenated to generate aggregated features. These aggregated features are then processed by the decoder to generate the flame detection result. By fusing hybrid scale units, dual-wash attention units, and frequency units through multi-level CSP_HPMB modules, collaborative feature enhancement is achieved across spatial, channel, and frequency domains. This fully leverages multi-frequency global features that combine spatial details, channel weights, and frequency domain information. The AIFI-DCL module establishes long-distance dependencies between pixels and adaptively captures multi-scale target features. Combined with the multi-path parallel branch structure within the EGConvC3 module, features at each level are deeply refined. This collaboratively completes the fusion and reconstruction of three-level, multi-scale, cross-layer features, effectively reducing the negative impacts of background noise from vegetation, clouds, buildings, smoke obstruction, and imaging brightness distortion. This enables high-precision detection of flame targets in complex aerial photography scenarios. Attached Figure Description

[0010] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0011] Figure 1 This is a flowchart of the flame detection method based on multi-frequency sensing and dynamic coding provided in Embodiment 1 of the present invention; Figure 2This is a schematic diagram of the CSP_HPMB module in the flame detection method based on multi-frequency sensing and dynamic coding provided in Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the hybrid scale unit in the flame detection method based on multi-frequency sensing and dynamic coding provided in Embodiment 1 of the present invention; Figure 4 This is a schematic diagram of the structure of the dual-wash attention unit in the flame detection method based on multi-frequency sensing and dynamic coding provided in Embodiment 1 of the present invention; Figure 5 This is a schematic diagram of the frequency unit structure in the flame detection method based on multi-frequency sensing and dynamic coding provided in Embodiment 1 of the present invention; Figure 6 This is a schematic diagram of the AIFI-DCL module in the flame detection method based on multi-frequency sensing and dynamic coding provided in Embodiment 1 of the present invention; Figure 7 This is a schematic diagram of the EGConvC3 module in the flame detection method based on multi-frequency sensing and dynamic coding provided in Embodiment 1 of the present invention; Figure 8 This is a schematic diagram of the structure of the flame detection device based on multi-frequency sensing and dynamic coding provided in Embodiment 2 of the present invention; Figure 9 This is a schematic diagram of the server structure provided in Embodiment 3 of the present invention. Detailed Implementation

[0012] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0013] Example 1 Figure 1 This is a flowchart of a flame detection method based on multi-frequency sensing and dynamic coding provided in Embodiment 1 of the present invention. This embodiment is applicable to flame monitoring scenarios under wide-area aerial images captured by UAVs, and specifically includes the following steps: Step 110: The image is processed by convolutional layers and the C2f module to perform preliminary spatial feature extraction and shallow semantic aggregation to obtain shallow basic features. The shallow basic features are then processed by convolutional layers and the CSP_HPMB module, and three-domain collaborative feature enhancement is used to obtain first-level multi-domain enhanced features.

[0014] The images are aerial visible light or infrared thermal images collected by UAV patrol operations. The UAVs, equipped with high-resolution visible light cameras or infrared thermal imagers, conduct full-area inspections and searches along a preset route, collecting aerial images in real time. Aerial sampling is repeated across multiple sorties, at different flight altitudes, and under varying lighting conditions to construct a raw image dataset covering multi-scale flame targets and various complex real-world backgrounds. These aerial images generally feature a large field of view and complex background composition, easily becoming cluttered with vegetation, buildings, exposed ground, clouds, smoke, and other interference elements. They also commonly suffer from problems such as large flame target scales, smoke obscuring the target itself, backlight / frontlight distortion, and imaging noise interference. The collected raw images undergo a complete set of preprocessing operations, including data verification, timestamp synchronization, image denoising, and contrast enhancement, to optimize the image signal-to-noise ratio and detail recognition, improving the quality of input images for subsequent detection models. After preprocessing, the images are cached and stored as the raw input data source for subsequent network flame target detection.

[0015] For example, the image is first processed through convolutional layers and C2f. Specifically, the image is processed through two convolutional layers to obtain initial features. These initial features are then input into the C2f module of YOLOv8, where they undergo channel splitting, multi-branch feature transformation, and concatenation fusion to obtain shallow intermediate features. These shallow intermediate features are then processed through convolutional layers and the C2f module, using progressive downsampling and hierarchical feature splitting and aggregation operations to obtain shallow basic features. C2f is a key component in the YOLOv8 network. Its internal process involves the input features first undergoing 1×1 convolution to adjust the channels, then splitting them into two channels. One channel is directly retained as a residual branch, while the other is fed into n cascaded Bottleneck modules for non-linear feature extraction. After concatenation of the two channels, a final 1×1 convolution layer is used to integrate the output channels, widening the gradient path while simultaneously completing shallow feature extraction and fusion.

[0016] Relying solely on shallow basic features only contains basic texture edge information, lacking multi-scale contextual association, effective channel filtering, and frequency domain dimensional information, which is not conducive to distinguishing flames from complex backgrounds. Therefore, further multi-dimensional feature enhancement processing is needed. By using convolutional layers in conjunction with the CSP_HPMB module to simultaneously perform three feature optimization operations, complementary mining of spatial, channel, and frequency domain information is achieved. The CSP_HPMB module performs multi-scale spatial modeling to expand the receptive field, channel attention filtering to suppress invalid background responses, and frequency domain feature enhancement to mine hidden periodic features in parallel. The fusion of these three processes generates a more powerful first-level multi-domain enhanced feature.

[0017] Step 120: The first-level multi-domain enhanced features are processed by convolutional layers and the CSP_HPMB module, and the second-level deep enhanced features are obtained through global feature depth purification. The CSP_HPMB module is fused with the hybrid perception Mamba sub-module. The hybrid perception Mamba sub-module performs spatial, channel, and frequency domain collaborative feature enhancement through hybrid scale unit, double shuffling attention unit, frequency unit and additive fusion unit.

[0018] While the first-level multi-domain enhanced features have achieved initial fusion, the global long-distance correlation mining is insufficient. The features still contain a large amount of redundant aerial background information and smoke interference noise, resulting in insufficient effective feature recognition of the flame target. Therefore, further deep purification and global modeling optimization are needed. For example, the first-level multi-domain enhanced features output in step 110 are first fed into a convolutional layer for channel adaptation and basic compression, and then input into the CSP_HPMB module for deep feature computation. Relying on the multi-branch structure and stacked hybrid perception Mamba sub-modules within CSP_HPMB, the hybrid perception Mamba sub-modules run in parallel with hybrid scale units, double-wash attention units, and frequency units to achieve multi-scale global spatial modeling, channel adaptive filtering to suppress invalid channels, and deep mining of frequency domain features, respectively. Through multi-layer iterative computation, the global dependencies between pixels are further strengthened, and irrelevant background interference such as vegetation, clouds, and buildings is filtered out, highlighting the specific features of the flame target. Finally, a second-level deep enhanced feature with stronger feature representation capabilities is output.

[0019] For example, such as Figure 2As shown, the CSP_HPMB module integrates the Hybrid Sensing Mamba sub-module. It normalizes the input features through layer normalization, then inputs the normalized features into the Hybrid Scale Unit, Double Shuffle Attention Unit, and Frequency Unit, respectively, to obtain the output features of the Hybrid Scale Unit, Double Shuffle Attention Unit, and Frequency Unit. These output features are then element-wise added and fused. Optionally, the CSP_HPMB module's processing flow involves first performing a 1×1 convolution to adjust the channel dimensions, then feeding it into a Split operation to evenly divide the channel into two sets of features with equal channel numbers, forming two branches. The first branch is a direct connection, bypassing all Hybrid Sensing Mamba sub-modules and directly transmitting to the end splicing position. The second branch serves as the main feature flow, leading out a bypass directly to the end splicing layer. The remaining features are sequentially fed into n serially stacked Hybrid Sensing Mamba sub-modules for layer-by-layer computation. Within a single Hybrid Sensing Mamba submodule, the input features are first preprocessed using layer normalization. Then, three parallel branches are branched out: a hybrid scale unit branch, a double-shuffle attention unit branch, and a frequency unit branch, to perform multi-domain feature transformation. The results of the three branches are summed element-wise to obtain the output features of the Hybrid Sensing Mamba submodule. After all n Hybrid Sensing Mamba submodules have been processed, the direct branches, the main branch leading to bypasses, and the output features of each Hybrid Sensing Mamba submodule are concatenated and fused along the channel dimension. Finally, a 1×1 convolution layer is applied to complete channel integration, outputting the final features of the CSP_HPMB module. This process simultaneously performs multi-scale spatial modeling, channel attention filtering, and frequency domain feature enhancement.

[0020] For example, such as Figure 3As shown, the hybrid scale unit obtains gated baseline features by sequentially passing the features after the layer normalization operation through linear layers and activation function mapping. Then, it sequentially performs linear projection, 3×3 depthwise separable convolutional encoding, and activation function processing on the features after the layer normalization operation to obtain the original scale pre-features. The hybrid scale unit further generates original scale global features by performing 2D-SSM operations and normalization on the original scale pre-features. Finally, it sequentially performs downsampling, 3×3 depthwise separable convolution, and activation function mapping on the original scale pre-features to obtain half-scale pre-features. The hybrid scale unit also performs 2D-SSM operations on the half-scale pre-features... The process involves calculation and normalization operations to obtain half-scale global features. These half-scale pre-features are then subjected to downsampling compression, 3×3 depthwise separable convolution, activation functions, 2D-SSM operations, normalization, and upsampling resolution restoration to obtain scale-restored features. The hybrid scale unit further fuses the half-scale global features and scale-restored features element-wise by adding them together and then performing upsampling to obtain dual-scale aggregated features. These dual-scale aggregated features are then summed and aggregated with the original scale global features, gated multiplication with the gated baseline features, and finally, after passing through a linear layer, residuals are added to the features after the layer normalization operation to output the hybrid scale unit's output features. Specifically, the hybrid scale unit's processing flow involves using the layer-normalized features as the overall input. This input feature is split into two parallel operations. The first input sequentially passes through a linear layer and SiLU activation function mapping to generate gated baseline features. The second input first undergoes linear projection, 3×3 depthwise separable convolutional encoding, and SiLU activation function processing to obtain the original scale pre-features. The original-scale pre-feature is further processed into three steps. The first step directly performs 2D-SSM operations combined with layer normalization to generate the original-scale global features. The second step downsamples the original-scale pre-feature, performs 3×3 depthwise separable convolution, and applies activation function mapping to obtain half-scale pre-features. 2D-SSM operations and normalization are then performed on these half-scale pre-features to obtain the half-scale global features. The third step, also based on the half-scale pre-features, performs downsampling compression, 3×3 depthwise separable convolution, activation function, 2D-SSM operations, and normalization again. Then, upsampling restores the resolution to obtain scale-restored features. Finally, the half-scale global features and scale-restored features are element-wise added and fused, and then upsampled to obtain dual-scale aggregated features. The dual-scale aggregated features are summed and aggregated with the original scale global features. The resulting features are then multiplied element-wise with the pre-generated gated baseline features. The feature mapping is then completed through a linear layer. Finally, the residuals of the result are added to the original input features of the hybrid scale unit after layer normalization. The output features of the hybrid scale unit are then output. The three-level multi-scale structure of the hybrid scale unit expands the applicability of 2D selective scanning and captures flame details and global semantics at multiple scales.

[0021] The 2D-SSM operation decomposes the two-dimensional feature map into a one-dimensional sequence along four directions (row forward, row backward, column forward, and column backward) using a four-way serialization unrolling unit, adapting to the temporal recursive operation format. It also uses a parameter adaptive prediction unit to solve for the discretization step size and projection coefficients in real time using a learnable projection layer, dynamically matching the information filtering and mapping intensity during the scanning process. Furthermore, a sequence information recursion unit adaptively transmits contextual information position-by-position within the one-dimensional sequence with linear time complexity, modeling long-distance dependency associations. Finally, a four-dimensional result merging and reshaping unit fuses the results of the four-way operations to reconstruct the two-dimensional feature map, enabling each pixel to gather global contextual information. Optionally, the main process of the 2D-SSM operation is as follows: first, the four-way serialization unrolling unit unrolls the input two-dimensional feature map into a one-dimensional sequence along the four scanning directions (row forward, row backward, column forward, and column backward). Then, the learnable projection layer within the parameter adaptive prediction unit predicts the control parameters required for selective scanning in real time from the input features. These parameters specifically include the discretization step size, which determines the degree of information retention; the projection coefficients, which control the degree of input influence; and the projection coefficients, which control the degree of output mapping. Subsequently, a sequence information recursion unit is used to perform adaptive information recursion operations at each position of each one-dimensional sequence with linear time complexity, establishing long-distance dependencies between pixels. Finally, a four-dimensional result merging and reshaping unit fuses the outputs of the operations in four directions, reconstructing and restoring a complete two-dimensional feature map. This allows each spatial location of the feature to aggregate the contextual information of the entire image, and completes long-distance dependency modeling based on the global receptive field.

[0022] For example, such as Figure 4As shown, the dual-shuffle attention unit obtains the channel features to be segmented by performing 3×3 convolution preprocessing on the features after the layer normalization operation; it also segments the channel features to be segmented along the channel dimension to generate extreme value branch features and mean branch features; it further extracts extreme value statistics by performing global max pooling on the extreme value branch features, and generates extreme value channel attention weights after 1×1 convolution mapping, non-linear activation, 1×1 convolution mapping and activation function; it then uses the extreme value channel attention weights to weight the extreme value branch features element-wise to obtain extreme value enhancement features; and it further performs the same process on the channel features. Mean branch features are extracted using global average pooling, and then processed through 1×1 convolution mapping, non-linear activation, 1×1 convolution mapping, and activation function to generate mean channel attention weights. These mean channel attention weights are then used to weight the mean branch features element-wise to obtain mean-enhanced features. Furthermore, channel shuffling and recombination operations are performed on the extreme value enhancement features and mean enhancement features to achieve cross-channel information interaction and fusion, resulting in channel fusion features. The residuals of the channel fusion features and the features to be divided are then added element-wise to output the output features of the double shuffling attention unit. Optionally, the processing flow of the double shuffling attention unit is as follows: the input features are first transformed by a 3×3 convolution layer, and then divided into two paths along the channel dimension through a channel splitting operation; one path serves as the extreme value branch feature, and the other as the mean branch feature. Extreme value branch features are fed into global max pooling to extract extreme value statistics. These are then processed sequentially through a first 1×1 convolution, ReLU activation, a second 1×1 convolution, and a Sigmoid activation to generate extreme value channel attention weights. These weights are then multiplied element-wise with the original extreme value branch features to obtain the extreme value enhancement features. Mean value branch features are fed into global average pooling to extract mean statistics. These weights are then processed sequentially through a first 1×1 convolution, ReLU activation, a second 1×1 convolution, and a Sigmoid activation to generate mean value channel attention weights. These weights are then multiplied element-wise with the original mean value branch features to obtain the mean enhancement features. The extreme value enhancement features and the mean enhancement features are then fed into a channel shuffling module to complete channel recombination and cross-channel information interaction and fusion, resulting in the channel fusion features. Finally, the channel fusion features and the undivided channel features output by the 3×3 convolution are added element-wise with residuals to output the final features of the double-wash attention unit. The key flame channels are adaptively selected by relying on the double pooling attention mechanism. The cross-channel modeling capability is improved by channel shuffling, which can meet the feature extraction needs of the variable flight conditions in aerial photography scenarios.

[0023] For example, such as Figure 5As shown, the frequency unit performs a Fast Fourier Transform (FFT) on the features after the layer normalization operation to complete the spatial-to-frequency domain mapping, splitting them into phase spectrum features and amplitude spectrum features. The phase spectrum features are sequentially fed into two 3×3 convolutional layers, and activation function operations are performed after each 3×3 convolution to complete the structural information enhancement processing of the phase dimension, generating enhanced phase spectrum features. The amplitude spectrum features are sequentially fed into two 1×1 convolutional layers, and activation function operations are performed after each 1×1 convolution to complete the energy distribution enhancement processing of the amplitude dimension, generating enhanced amplitude spectrum features. The enhanced phase spectrum features and enhanced amplitude spectrum features are subjected to an inverse Fast Fourier Transform (IFFT) operation to restore the frequency domain enhanced features to the spatial domain, generating frequency domain modulation features. The frequency domain modulation features are added element-wise with the features after the layer normalization operation to output the output features of the frequency unit. Optionally, the frequency unit processing flow is as follows: Using the features after layer normalization as input, one path directly retains the original features for subsequent residual connections. The other path first undergoes a Fast Fourier Transform (FFT) operation to convert the spatial domain features to the frequency domain, splitting them into two independent branches: a phase spectrum carrying image structural information and an amplitude spectrum representing frequency energy distribution. The phase spectrum branch sequentially undergoes a first-layer 3×3 convolution, a ReLU activation function, a second-layer 3×3 convolution, and a ReLU activation function to enhance structural information, resulting in enhanced phase spectrum features. The amplitude spectrum branch sequentially undergoes a first-layer 1×1 convolution, a ReLU activation function, a second-layer 1×1 convolution, and a ReLU activation function to enhance energy distribution information, resulting in enhanced amplitude spectrum features. The enhanced phase spectrum features and enhanced amplitude spectrum features are then merged and fed into an Inverse Fast Fourier Transform (IFT) operation to restore the frequency domain features back to the spatial domain, obtaining frequency domain modulation features. Finally, the frequency domain modulation features are added element-wise with the original normalized features of the initial input layer of the frequency unit to output the final features of the frequency unit. This process is used to mine hidden features in the frequency domain dimension of the image, extract the energy and structural details of the flame texture, and improve the network's ability to perceive frequency domain information.

[0024] Step 130: Input the shallow basic features, first-level multi-domain enhancement features, and second-level deep enhancement features into the hybrid encoder. Perform three-level multi-scale cross-layer feature fusion and reconstruction through convolution, upsampling, AIFI-DCL module, and EGConvC3 module to obtain shallow refined fusion features, mid-level cross-scale alignment features, and deep global modulation features. The AIFI-DCL module performs global multi-scale adaptive feature extraction through multi-head attention and dual-channel dynamic convolution DCL structure. The EGConvC3 module includes convolution branches and multi-level cascaded EGBlock modules, and performs deep feature extraction through the multi-path parallel structure in the EGBlock module.

[0025] Shallow basic features, first-level multi-domain enhanced features, and second-level deep enhanced features correspond to three types of differentiated information in images: fine-grained texture, medium-scale semantics, and deep global semantics, respectively. These three types have inconsistent scales and weak feature correlations. Directly feeding them into the decoding and detection layer can lead to scale mismatch, loss of small flame features, and fragmentation of global and local information. Furthermore, aerial images exhibit a large scale span for flames and complex background interference, necessitating multi-scale feature alignment and cross-layer information complementarity and fusion. Therefore, a hybrid encoder is designed to perform layer-by-layer upsampling and adaptive reconstruction of the three-level features from shallow to deep, compensating for information gaps at different levels and enhancing the representation capability of flame targets of varying sizes.

[0026] For example, the hybrid encoder internally employs a three-level cross-layer reconstruction structure with bidirectional fusion. The three types of input features first undergo independent convolution to complete channel dimension regularization preprocessing. Specifically, the second-level depth enhancement features, after convolution, are fed into the AIFI-DCL module, where multi-head attention combined with a dual-channel dynamic convolutional DCL structure completes global multi-scale adaptive feature extraction. The result is first convolved to obtain the first intermediate feature, which is then upsampled and concatenated with the convolutionally preprocessed first-level multi-domain enhancement features. The concatenated feature is then fed into the EGConvC3 module, where its internal multi-level cascaded EGBlock multi-path parallel structure enriches gradient paths and deeply refines fusion information to obtain the second intermediate feature. This second intermediate feature is then convolved to generate the third intermediate feature. After upsampling, the third intermediate feature is concatenated with the convolutionally preprocessed shallow basic features using a concatenated channel concatenation. The concatenated result is fed into the EGConvC3 module for reconstruction optimization, generating shallow, refined fusion features. Simultaneously, the network constructs downlink fusion branches from top to bottom. The shallow, refined fusion features, after convolutional dimensionality reduction, are concatenated with the third intermediate feature and then fed into the EGConvC3 module for deep feature extraction and noise reduction optimization, resulting in mid-level cross-scale aligned features. Further fusion continues downwards, with the mid-level cross-scale aligned features undergoing convolutional operations and concatenated with the first intermediate feature. This concatenation is then performed again by the EGConvC3 module for feature modulation optimization, ultimately yielding deep global modulation features. This completes the three-level, multi-scale, cross-layer feature fusion and reconstruction.

[0027] For example, such as Figure 6As shown, the AIFI-DCL module flattens the secondary deep enhancement features to transform them into a secondary deep enhancement feature sequence, and then superimposes two-dimensional sine-cosine positional encodings onto the secondary deep enhancement feature sequence. The superimposed positional encoding secondary deep enhancement feature sequence is then fed into a multi-head self-attention module to complete global context interaction computation. The computation result is processed by Dropout regularization, and then the residuals are added to the secondary deep enhancement feature sequence. After layer normalization, normalized features are obtained. These normalized features are then reshaped to obtain intermediate features. Layer normalization preprocessing is performed on the intermediate features, followed by channel expansion via 1×1 convolution, and the features are evenly distributed along the channel dimension to obtain fine-scale branch features and wide-scale branches. The AIFI-DCL module employs a 5×5 dynamic depthwise separable convolution to adaptively extract fine-grained flame boundary features, resulting in fine-scale enhanced features. For the wide-scale branch features, a 7×7 dynamic depthwise separable convolution is used to capture surrounding background information over a large area, yielding wide-scale enhanced features. These fine-scale and wide-scale enhanced features are then fed into a channel shuffling unit to achieve multi-scale information interaction and fusion. After compression and restoration of the channel dimensions via 1×1 convolution, the residuals are added to the intermediate features to obtain the output two-dimensional features. The output two-dimensional features are flattened, regularized using Dropout, and then superimposed with the normalized feature residuals. Layer normalization is then performed to obtain the output features of the AIFI-DCL module. Optionally, the AIFI-DCL module's specific process involves using the input secondary depthwise enhanced features as the starting image features. These features are first flattened into a one-dimensional feature sequence, and then superimposed with two-dimensional sine-cosine positional encoding to preserve pixel spatial location information. The data is then fed into a multi-head attention module for global feature interaction computation. After Dropout processing, the result is added to the one-dimensional feature sequence using residuals, followed by layer normalization to obtain normalized features. These normalized features are then reshaped to restore the one-dimensional normalized sequence to two-dimensional features, yielding intermediate features. These intermediate features are then processed within the DCL submodule. First, they undergo a normalization preprocessing layer, then a 1×1 convolution doubles the number of channels. Channel splitting separates them into fine-scale and wide-scale branch features. The fine-scale branch features are fed into a 5×5 dynamic depthwise separable convolution to adaptively extract fine-grained local features such as the flame's fine contour and edge details, resulting in fine-scale enhanced features. The wide-scale branch features are fed into a 7×7 dynamic depthwise separable convolution to capture a wide range of background context information around the target, resulting in wide-scale enhanced features. The two enhanced features are then fed into a channel shuffling module to fully interact across multiple branches and scales. Finally, a 1×1 convolution compresses the channels to restore the original number of channels, and the residuals are added to the intermediate features to obtain the output two-dimensional features.The output 2D features are flattened and Dropout regularization is applied. The residuals are then superimposed with the normalized features. Finally, after layer normalization, the AIFI-DCL operation is completed. This method relies on multi-head attention to establish global long-distance dependencies and uses dual-size dynamic convolution to adapt to aerial photography scenes with varying lighting, altitude, and flame shapes, thereby improving the ability to distinguish flames from interfering backgrounds.

[0028] For example, the EGBlock module performs 3×3 convolution, batch normalization, 1×1 convolution, and batch normalization operations sequentially on the input features through a first convolution branch to achieve deep extraction of spatial detail features; it also performs 3×3 convolution, batch normalization, 1×1 convolution, and batch normalization operations simultaneously on the input features through a second convolution branch to achieve parallel mining of complementary spatial features; it further performs 1×1 convolution, batch normalization, 1×1 convolution, and batch normalization processing on the input features through a third convolution branch to achieve channel dimension feature compression and reorganization; and it performs batch normalization distribution correction on the input features through an identity mapping branch. Optionally, such as... Figure 7 As shown, the processing flow of the EGConvC3 module is as follows: the input features are first divided into two paths. One path is processed by a 1×1 convolution in the convolution branch and retained as a residual pass-through branch. The other path is first processed by a 1×1 convolution and then fed into a multi-level cascaded EGBlock module for deep feature enhancement. In training mode, a single EGBlock adopts a four-path parallel branch structure. The first and second convolution branches are 3×3 convolution chain branches with the same structure, each performing 3×3 convolution, batch normalization (BN), 1×1 convolution, and batch normalization (BN) operations sequentially. The third convolution branch is a 1×1 convolution chain branch, performing 1×1 convolution, batch normalization (BN), 1×1 convolution, and batch normalization (BN) operations sequentially. The fourth branch is an identity mapping branch, configured only with batch normalization (BN) operations. The output features of the four branches are added element-wise to complete the fusion. The fused result is then processed by the SiLU activation function to obtain the output features of a single EGBlock. After being enhanced by n EGBlock serial iterations, the features of this path are concatenated and fused with the features of another path that have undergone convolution in the channel dimension. Finally, the whole is subjected to a 1×1 convolution to complete channel integration and compression, outputting the final features of the EGConvC3 module during the training phase.

[0029] It should be noted that in the test mode, a single EGBlock utilizes structural reparameterization technology to equivalently fuse the four parallel branches during the training phase into a single 3×3 convolutional structure. First, the convolutional layers and BN layers within the two 3×3 convolutional chain branches and one 1×1 convolutional chain branch are fused into independent equivalent convolutional kernels. Then, the 1×1 convolutional kernel is zero-padded and expanded to a 3×3 size. Simultaneously, the BN parameters of the identity mapping branch are converted into equivalent 3×3 convolutional kernels with weights in the center and zeros at the other positions. All equivalent convolutional kernels are added and merged with their corresponding biases to obtain a unique equivalent 3×3 convolution. The input features are directly processed by this fused 3×3 convolution operation, and then the result is output through the SiLU activation function. While ensuring lossless computational accuracy, this eliminates the memory read / write overhead caused by multi-branch parallelism, effectively reducing model inference latency and adapting to the lightweight deployment requirements of UAV onboard terminals.

[0030] Step 140: Connect the shallow refined fusion features, the mid-level cross-scale alignment features, and the deep global modulation features to generate aggregated features. Process the aggregated features through a decoder to generate flame detection results.

[0031] The shallow, refined fusion features, mid-level cross-scale alignment features, and deep global modulation features output by the hybrid encoder are concatenated along the channel dimension to integrate three complementary types of information: small-scale flame details, medium-scale target contours, and deep global semantics, resulting in aggregated features that fuse multi-scale information. These aggregated features are then fed into the detection decoder. Optionally, the decoder employs a Transformer decoder architecture. Based on a set-matching query allocation mechanism (or a pre-defined anchor box matching mechanism), the decoder simultaneously performs target classification and boundary regression calculations on the aggregated features, distinguishing between the flame foreground and complex background, calculating the coordinates of the flame target and its corresponding confidence score, and finally outputting a complete flame detection result carrying the target's location coordinates, target category, and confidence score.

[0032] This embodiment performs preliminary spatial feature extraction and shallow semantic aggregation on the image through convolutional layers and a C2f module to obtain shallow basic features. These shallow basic features are then processed through convolutional layers and a CSP_HPMB module, and enhanced through three-domain collaborative feature enhancement to obtain first-level multi-domain enhanced features. These first-level multi-domain enhanced features are then processed through convolutional layers and a CSP_HPMB module, and refined through global feature depth extraction to obtain second-level deep enhanced features. The CSP_HPMB module integrates a hybrid perception Mamba sub-module, which performs spatial, channel, and frequency domain collaborative feature enhancement through a hybrid scale unit, a double shuffling attention unit, a frequency unit, and an additive fusion unit. The shallow basic features, first-level multi-domain enhanced features, and second-level deep enhanced features are then combined. In the input hybrid encoder, a three-level multi-scale cross-layer feature fusion reconstruction is performed through convolution, upsampling, the AIFI-DCL module, and the EGConvC3 module to obtain shallow refined fusion features, mid-level cross-scale aligned features, and deep global modulation features. The AIFI-DCL module performs global multi-scale adaptive feature extraction through multi-head attention and a dual-channel dynamic convolutional DCL structure. The EGConvC3 module includes convolutional branches and multi-level cascaded EGBlock modules, and performs deep feature extraction through the multi-path parallel structure in the EGBlock module. The shallow refined fusion features, mid-level cross-scale aligned features, and deep global modulation features are concatenated to generate aggregated features. The aggregated features are then processed by the decoder to generate flame detection results. Multi-frequency global features are extracted through a multi-level CSP_HPMB module, and cross-scale feature fusion reconstruction is completed by the AIFI-DCL and EGConvC3 modules, achieving high-precision detection of flame targets in complex aerial photography scenarios.

[0033] Example 2 Figure 8 This is a schematic diagram of the flame detection device based on multi-frequency sensing and dynamic coding provided in Embodiment 2 of the present invention, as shown below. Figure 8 As shown, the device includes: The basic feature and first-level feature acquisition module 210 is used to process the image through a convolutional layer and a C2f module to perform preliminary spatial feature extraction and shallow semantic aggregation to obtain shallow basic features. The shallow basic features are then processed through a convolutional layer and a CSP_HPMB module, and through three-domain collaborative feature enhancement, first-level multi-domain enhanced features are obtained. The secondary feature acquisition module 220 is used to process the primary multi-domain enhanced features through a convolutional layer and the CSP_HPMB module, and to obtain secondary deep enhanced features through global feature depth purification. The CSP_HPMB module integrates the hybrid perception Mamba sub-module. The hybrid perception Mamba sub-module performs spatial, channel, and frequency domain three-domain collaborative feature enhancement through a hybrid scale unit, a double shuffle attention unit, a frequency unit, and an additive fusion unit. The feature fusion and reconstruction module 230 is used to input the shallow basic features, first-level multi-domain enhancement features and second-level deep enhancement features into the hybrid encoder, and perform three-level multi-scale cross-layer feature fusion and reconstruction through convolution, upsampling, AIFI-DCL module and EGConvC3 module to obtain shallow refined fusion features, mid-level cross-scale alignment features and deep global modulation features. The AIFI-DCL module performs global multi-scale adaptive feature extraction through multi-head attention and dual-channel dynamic convolution DCL structure. The EGConvC3 module includes convolution branches and multi-level cascaded EGBlock module, and performs deep feature extraction through the multi-path parallel structure in the EGBlock module. The flame detection module 240 is used to perform a connection operation on the shallow refined fusion features, the mid-level cross-scale alignment features and the deep global modulation features to generate aggregate features, and to process the aggregate features through the decoder to generate flame detection results.

[0034] The flame detection device based on multi-frequency sensing and dynamic coding provided in this embodiment performs preliminary spatial feature extraction and shallow semantic aggregation by processing the image through convolutional layers and a C2f module to obtain shallow basic features. These shallow basic features are then processed through convolutional layers and a CSP_HPMB module, and enhanced through three-domain collaborative feature enhancement to obtain first-level multi-domain enhanced features. The first-level multi-domain enhanced features are then processed through convolutional layers and a CSP_HPMB module, and refined through global feature depth extraction to obtain second-level deep enhanced features. The CSP_HPMB module integrates a hybrid sensing Mamba sub-module, which performs spatial, channel, and frequency domain collaborative feature enhancement through a hybrid scale unit, a double shuffling attention unit, a frequency unit, and an additive fusion unit. The shallow basic features and first-level multi-domain enhanced features are then processed. In the hybrid encoder, features and secondary deep enhancement features are input. Three-level multi-scale cross-layer feature fusion and reconstruction are performed through convolution, upsampling, the AIFI-DCL module, and the EGConvC3 module to obtain shallow refined fusion features, mid-level cross-scale aligned features, and deep global modulation features. The AIFI-DCL module performs global multi-scale adaptive feature extraction through multi-head attention and a dual-channel dynamic convolutional DCL structure. The EGConvC3 module includes convolutional branches and multi-level cascaded EGBlock modules, performing deep feature extraction through a multi-path parallel structure within the EGBlock module. The shallow refined fusion features, mid-level cross-scale aligned features, and deep global modulation features are concatenated to generate aggregated features. These aggregated features are then processed by the decoder to generate flame detection results. Multi-frequency global features are extracted through a multi-level CSP_HPMB module, and cross-scale feature fusion and reconstruction are completed by the AIFI-DCL and EGConvC3 modules, achieving high-precision detection of flame targets in complex aerial photography scenarios.

[0035] Based on the above embodiments, the basic feature and first-level feature acquisition module includes: The initial feature acquisition unit is used to process the image through two convolutional layers to obtain initial features; The shallow intermediate feature acquisition unit is used to input the initial features into the C2f module of YOLOv8, and obtain the shallow intermediate features through channel splitting, multi-branch feature transformation and splicing fusion processing. The shallow basic feature acquisition unit is used to process the shallow intermediate features through convolutional layers and C2f modules, and obtain shallow basic features through progressive downsampling, hierarchical feature splitting and aggregation operations.

[0036] Based on the above embodiments, the secondary feature acquisition module includes: The addition fusion unit is used to normalize the input features through layer normalization operation, and input the features after the layer normalization operation into the mixing scale unit, the double shuffle attention unit and the frequency unit respectively to obtain the output features of the mixing scale unit, the double shuffle attention unit and the frequency unit respectively. Then, the output features of the mixing scale unit, the double shuffle attention unit and the frequency unit are added and fused element by element. The original scale pre-feature acquisition unit is used by the hybrid scale unit to obtain gated baseline features by sequentially passing the features after the layer normalization operation through a linear layer and an activation function mapping, and sequentially performing linear projection, 3×3 depthwise separable convolutional coding and activation function processing on the features after the layer normalization operation to obtain the original scale pre-features. The half-scale pre-feature acquisition unit is used to generate the original scale global features by performing 2D-SSM operation and normalization on the original scale pre-features, and then performing downsampling, 3×3 depth-separable convolution and activation function mapping on the original scale pre-features to obtain the half-scale pre-features. The scale restoration feature acquisition unit is used to obtain the half-scale global features by performing 2D-SSM operation and normalization operation on the half-scale pre-features. The half-scale pre-features are then subjected to downsampling compression, 3×3 depth separable convolution, activation function, 2D-SSM operation, normalization and upsampling resolution restoration operation in sequence to obtain the scale restoration features. The hybrid scale unit's output feature acquisition unit is used to obtain dual-scale aggregated features by adding and fusing half-scale global features and scale restoration features element by element and then performing upsampling. The dual-scale aggregated features are then summed and aggregated with the original scale global features, gated multiplication is performed with the gated reference features, and after passing through a linear layer, the residuals are added with the features after the layer normalization operation to output the hybrid scale unit's output features. The decomposition unit is used to decompose the two-dimensional feature map into a one-dimensional sequence in four directions: forward row, reverse row, forward column, and reverse column, through the four-way serialization expansion unit, to adapt to the temporal recursive operation form. The solving unit is used to solve the discretization step size and projection coefficients in real time through the parameter adaptive prediction unit using a learnable projection layer, and dynamically match the information filtering and mapping intensity of the scanning process. The transmission unit is used to adaptively transmit context information position by position within a one-dimensional sequence in linear time complexity through the sequence information recursion unit, thus modeling long-distance dependency associations. The computation unit is used to fuse the computation results of the four directions through the four-dimensional result merging and reshaping unit to restore and generate a two-dimensional feature map, so as to realize the aggregation of global context information for each pixel. The segmentation unit is used to obtain the channel features to be segmented by performing 3×3 convolution preprocessing on the features after the layer normalization operation; and to segment the channel features to be segmented along the channel dimension to generate extreme value branch features and mean branch features. The extreme value enhancement feature acquisition unit is used to further extract extreme value statistical information by performing global max pooling on the extreme value branch features, and generate extreme value channel attention weights after 1×1 convolution mapping, nonlinear activation, 1×1 convolution mapping and activation function. The extreme value channel attention weights are then used to weight the extreme value branch features element by element to obtain extreme value enhancement features. The mean enhancement feature acquisition unit is used to further extract mean statistical information by performing global average pooling on the mean branch features, and generate mean channel attention weights after passing through 1×1 convolution mapping, non-linear activation, 1×1 convolution mapping and activation function. The mean branch features are then weighted element by element using the mean channel attention weights to obtain the mean enhancement features. The output feature acquisition unit of the double shuffle attention unit is used to further perform channel shuffle recombination operation on the extreme value enhancement feature and the mean enhancement feature to complete cross-channel information interaction fusion and obtain channel fusion feature; and add the residual element-wise of the channel fusion feature and the channel feature to be separated to output the output feature of the double shuffle attention unit. The enhanced phase spectrum feature acquisition unit is used to perform a fast Fourier transform on the features after the layer normalization operation to complete the spatial domain to frequency domain mapping, and split to obtain phase spectrum features and amplitude spectrum features; the phase spectrum features are sequentially fed into two 3×3 convolutional layers, and activation function operations are performed after each 3×3 convolution to complete the structural information enhancement processing of the phase dimension and generate enhanced phase spectrum features. The frequency domain modulation feature acquisition unit is used to sequentially feed the amplitude spectrum features into two 1×1 convolutional layers, and then perform activation function operations after each 1×1 convolution to complete the energy distribution enhancement processing in the amplitude dimension and generate enhanced amplitude spectrum features; and to perform inverse fast Fourier transform operations on the enhanced phase spectrum features and the enhanced amplitude spectrum features to restore the frequency domain enhancement features to the spatial domain and generate frequency domain modulation features. The frequency unit output feature acquisition unit is used to add the frequency domain modulation feature and the feature after the layer normalization operation to the residual element by element, and output the frequency unit output feature.

[0037] Based on the above embodiments, the feature fusion and reconstruction module includes: The flattening unit is used to flatten the secondary depth enhancement features, convert them into a secondary depth enhancement feature sequence, and superimpose two-dimensional sine-cosine position codes onto the secondary depth enhancement feature sequence. The intermediate feature acquisition unit is used to send the secondary deep enhancement feature sequence after superimposed position encoding into the multi-head self-attention module to complete the global context interaction operation. After the operation result is processed by Dropout regularization, it is added to the residual of the secondary deep enhancement feature sequence, and then processed by layer normalization to obtain normalized features. The normalized features are then processed by feature reshaping operation to obtain intermediate features. The branch feature acquisition unit is used to perform layer normalization preprocessing on the intermediate features, then perform channel expansion through 1×1 convolution, and evenly divide along the channel dimension to obtain fine-scale branch features and wide-scale branch features. The wide-scale enhancement feature acquisition unit is used to adaptively extract the fine boundary features of the flame using a 5×5 dynamic depthwise separable convolution to obtain fine-scale enhancement features; and to capture the surrounding background information over a large area using a 7×7 dynamic depthwise separable convolution to obtain wide-scale enhancement features. The output two-dimensional feature acquisition unit is used to send the fine-scale enhancement features and wide-scale enhancement features into the channel shuffling unit to complete the multi-scale information interaction and fusion. After 1×1 convolution compression to restore the channel dimension, the residual is added with the intermediate features to obtain the output two-dimensional features. The output feature acquisition unit of the AIFI-DCL module is used to flatten the output two-dimensional features, perform Dropout regularization, superimpose them with the normalized feature residuals, and then perform layer normalization to obtain the output features of the AIFI-DCL module. The depth extraction unit is used to perform 3×3 convolution, batch normalization, 1×1 convolution and batch normalization operations on the input features in sequence through the first convolution branch to achieve deep extraction of spatial detail features; The parallel mining unit is used to simultaneously perform 3×3 convolution, batch normalization, 1×1 convolution and batch normalization operations on the input features through the second convolution branch, so as to realize the parallel mining of complementary spatial features. The compression and recombination unit is used to perform 1×1 convolution, batch normalization, 1×1 convolution and batch normalization on the input features in sequence through the third convolution branch to achieve channel dimension feature compression and recombination. The normalization distribution correction unit is also used to perform batch normalization distribution correction on the input features via an identity mapping branch.

[0038] The flame detection device based on multi-frequency sensing and dynamic coding provided in the embodiments of the present invention can execute the flame detection method based on multi-frequency sensing and dynamic coding provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method.

[0039] Example 3 Figure 9 This is a schematic diagram of the structure of a server provided in Embodiment 3 of the present invention. Figure 9 A block diagram of an exemplary server 12 suitable for implementing embodiments of the present invention is shown. Figure 9 The server 12 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0040] like Figure 9 As shown, server 12 is presented in the form of a general-purpose computing server. The components of server 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0041] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0042] Server 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by server 12, including volatile and non-volatile media, removable and non-removable media.

[0043] System memory 28 may include computer system readable media in the form of volatile memory, such as RAM 30 and / or cache 32. Server 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 9 Not shown; usually referred to as a "hard drive"). Although Figure 9 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0044] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0045] Server 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing server, display 24, etc.), and with one or more servers that enable users to interact with server 12, and / or with any server (e.g., network card, modem, etc.) that enables server 12 to communicate with one or more other computing servers. This communication can be performed via I / O interface 22. Furthermore, server 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of server 12 via bus 18. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with server 12, including but not limited to: microcode, server drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0046] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the flame detection method based on multi-frequency sensing and dynamic coding provided in the embodiments of the present invention.

[0047] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A flame detection method based on multi-frequency sensing and dynamic coding, characterized in that, include: The image is processed through convolutional layers and the C2f module to perform preliminary spatial feature extraction and shallow semantic aggregation, resulting in shallow basic features. These shallow basic features are then processed through convolutional layers and the CSP_HPMB module, and enhanced through three-domain collaborative features to obtain first-level multi-domain enhanced features. The first-level multi-domain enhanced features are processed by convolutional layers and the CSP_HPMB module, and then purified by global feature depth to obtain the second-level deep enhanced features. The CSP_HPMB module is fused with the hybrid perception Mamba sub-module. The hybrid perception Mamba sub-module performs spatial, channel, and frequency domain collaborative feature enhancement through hybrid scale unit, double shuffle attention unit, frequency unit and additive fusion unit. The shallow basic features, first-level multi-domain enhancement features, and second-level deep enhancement features are input into the hybrid encoder. Three-level multi-scale cross-layer feature fusion and reconstruction are performed through convolution, upsampling, AIFI-DCL module, and EGConvC3 module to obtain shallow refined fusion features, mid-level cross-scale alignment features, and deep global modulation features. The AIFI-DCL module performs global multi-scale adaptive feature extraction through multi-head attention and dual-channel dynamic convolution DCL structure. The EGConvC3 module includes convolution branches and multi-level cascaded EGBlock modules, and performs deep feature extraction through the multi-path parallel structure in the EGBlock module. The shallow refined fusion features, the mid-level cross-scale alignment features, and the deep global modulation features are concatenated to generate aggregated features. The aggregated features are then processed by a decoder to generate flame detection results.

2. The method according to claim 1, characterized in that, The image is processed through convolutional layers and a C2f module to perform preliminary spatial feature extraction and shallow semantic aggregation, resulting in shallow basic features, including: The image is processed through two convolutional layers to obtain initial features; The initial features are input into the C2f module of YOLOv8, and after channel splitting, multi-branch feature transformation and splicing fusion, shallow intermediate features are obtained. The shallow intermediate features are processed through convolutional layers and C2f modules, and shallow basic features are obtained through progressive downsampling, hierarchical feature splitting and aggregation operations.

3. The method according to claim 1, characterized in that, The hybrid sensing Mamba submodule includes: The input features are normalized by layer normalization operation. The features after layer normalization operation are then input into the mixed scale unit, the double shuffle attention unit and the frequency unit respectively to obtain the output features of the mixed scale unit, the double shuffle attention unit and the frequency unit respectively. Then, the output features of the mixed scale unit, the double shuffle attention unit and the frequency unit are added and fused element by element. The hybrid scale unit obtains gated baseline features by sequentially mapping the features after the layer normalization operation through a linear layer and an activation function. The features after the layer normalization operation are then sequentially processed by linear projection, 3×3 depthwise separable convolutional coding, and activation function to obtain the original scale pre-features. The hybrid scale unit also generates original scale global features by performing 2D-SSM operations and normalization on the original scale pre-features, and then performs downsampling, 3×3 depthwise separable convolution and activation function mapping on the original scale pre-features to obtain half-scale pre-features. The hybrid scale unit also obtains half-scale global features by performing 2D-SSM operations and normalization on the half-scale pre-features. The half-scale pre-features are then subjected to downsampling compression, 3×3 depthwise separable convolution, activation function, 2D-SSM operation, normalization, and upsampling resolution restoration operations to obtain scale-restored features. The hybrid scale unit further obtains dual-scale aggregated features by adding and fusing half-scale global features and scale restoration features element by element and then performing an upsampling operation. The dual-scale aggregated features are then summed and aggregated with the original scale global features, and then gated multiplied with the gated reference features. After passing through a linear layer, the residuals are added to the features after the layer normalization operation to output the output features of the hybrid scale unit.

4. The method according to claim 3, characterized in that, The 2D-SSM operation includes: The two-dimensional feature map is decomposed into a one-dimensional sequence in four directions—forward row, reverse row, forward column, and reverse column—through a four-way serialization unpacking unit, which is adapted to the temporal recursive operation form. Furthermore, the parameter adaptive prediction unit utilizes a learnable projection layer to solve the discretization step size and projection coefficients in real time, dynamically matching the information filtering and mapping intensity during the scanning process; Furthermore, it uses a sequence information recursion unit to adaptively pass context information position by position within a one-dimensional sequence in linear time complexity, thus modeling long-distance dependency associations. Furthermore, by fusing the results of operations in four directions through a four-dimensional result merging and reshaping unit, a two-dimensional feature map is generated, enabling each pixel to aggregate global context information.

5. The method according to claim 3, characterized in that, The dual-wash attention unit includes: The features to be segmented are obtained by performing 3×3 convolution preprocessing on the features after the layer normalization operation; the features to be segmented are also segmented along the channel dimension to generate extreme value branch features and mean branch features; Furthermore, extreme value statistics are extracted by performing global max pooling on the extreme value branch features, and extreme value channel attention weights are generated after 1×1 convolution mapping, non-linear activation, 1×1 convolution mapping and activation function. The extreme value channel attention weights are then used to weight the extreme value branch features element by element to obtain extreme value enhancement features. Furthermore, mean statistical information is extracted by global average pooling of the mean branch features, and mean channel attention weights are generated after 1×1 convolution mapping, non-linear activation, 1×1 convolution mapping and activation function. The mean branch features are then weighted element by element using the mean channel attention weights to obtain mean-enhanced features. Furthermore, by performing channel shuffling and recombination operations on the extreme value enhancement features and mean enhancement features, cross-channel information interaction and fusion are completed to obtain channel fusion features; the channel fusion features are then added element-wise with the residuals of the features to be separated to output the output features of the double shuffling attention unit.

6. The method according to claim 3, characterized in that, The frequency unit includes: By performing a Fast Fourier Transform on the features after the layer normalization operation, the spatial domain is mapped to the frequency domain, and the phase spectrum features and amplitude spectrum features are obtained. The phase spectrum features are then fed into two 3×3 convolutional layers, and activation function operations are performed after each 3×3 convolution to complete the structural information enhancement processing of the phase dimension and generate enhanced phase spectrum features. The amplitude spectrum features are sequentially fed into two 1×1 convolutional layers, and activation function operations are performed after each 1×1 convolution to complete the energy distribution enhancement processing in the amplitude dimension and generate enhanced amplitude spectrum features. The enhanced phase spectrum features and enhanced amplitude spectrum features are subjected to inverse fast Fourier transform operations to restore the frequency domain enhanced features to the spatial domain and generate frequency domain modulation features. The frequency domain modulation features are added element-wise with the features after the layer normalization operation to obtain the output features of the frequency unit.

7. The method according to claim 1, characterized in that, The AIFI-DCL module includes: The secondary depth enhancement features are flattened to form a secondary depth enhancement feature sequence, and a two-dimensional sine-cosine position code is superimposed on the secondary depth enhancement feature sequence. The secondary deep enhancement feature sequence after overlay position encoding is fed into the multi-head self-attention module to complete the global context interaction operation. After the operation result is processed by Dropout regularization, it is added to the residual of the secondary deep enhancement feature sequence, and then processed by layer normalization to obtain normalized features. The normalized features are then processed by feature reshaping operation to obtain intermediate features. The intermediate features are preprocessed by layer normalization, and then channel expansion is completed by 1×1 convolution. The fine-scale branch features and wide-scale branch features are obtained by evenly dividing along the channel dimension. The fine-scale branch features are adaptively extracted using a 5×5 dynamic depthwise separable convolution to obtain fine-scale enhanced features; the wide-scale branch features are captured using a 7×7 dynamic depthwise separable convolution to capture surrounding background information over a wide area to obtain wide-scale enhanced features. The fine-scale enhancement features and wide-scale enhancement features are fed into the channel shuffling unit to complete multi-scale information interaction and fusion. After 1×1 convolution compression to restore the channel dimension, the residuals are added with the intermediate features to obtain the output two-dimensional features. The output two-dimensional features are flattened, regularized by Dropout, and then superimposed with the normalized feature residuals. After layer normalization, the output features of the AIFI-DCL module are obtained.

8. The method according to claim 1, characterized in that, The EGBlock module includes: The first convolution branch sequentially performs 3×3 convolution, batch normalization, 1×1 convolution, and batch normalization operations on the input features to achieve deep extraction of spatial detail features; Furthermore, the second convolution branch synchronously performs 3×3 convolution, batch normalization, 1×1 convolution and batch normalization operations on the input features to achieve parallel mining of complementary spatial features; The third convolution branch is used to sequentially perform 1×1 convolution, batch normalization, 1×1 convolution and batch normalization on the input features to achieve channel dimension feature compression and recombination. It also uses an identity mapping branch to perform batch normalization distribution correction on the input features.

9. A flame detection device based on multi-frequency sensing and dynamic coding, characterized in that, include: The basic feature and first-level feature acquisition module is used to process the image through convolutional layers and C2f module to perform preliminary spatial feature extraction and shallow semantic aggregation to obtain shallow basic features. The shallow basic features are then processed through convolutional layers and CSP_HPMB module, and first-level multi-domain enhanced features are obtained through three-domain collaborative feature enhancement. The secondary feature acquisition module is used to process the primary multi-domain enhanced features through convolutional layers and the CSP_HPMB module, and to obtain secondary deep enhanced features through global feature depth purification. The CSP_HPMB module integrates the hybrid perception Mamba sub-module, which performs spatial, channel, and frequency domain collaborative feature enhancement through hybrid scale unit, double shuffle attention unit, frequency unit and additive fusion unit. The feature fusion and reconstruction module is used to input the shallow basic features, first-level multi-domain enhancement features, and second-level deep enhancement features into the hybrid encoder. Through convolution, upsampling, AIFI-DCL module, and EGConvC3 module, three-level multi-scale cross-layer feature fusion and reconstruction is performed to obtain shallow refined fusion features, mid-level cross-scale aligned features, and deep global modulation features. The AIFI-DCL module performs global multi-scale adaptive feature extraction through multi-head attention and dual-channel dynamic convolution DCL structure. The EGConvC3 module includes convolution branches and multi-level cascaded EGBlock modules, and performs deep feature extraction through the multi-path parallel structure in the EGBlock module. The flame detection module is used to connect the shallow refined fusion features, the mid-level cross-scale alignment features, and the deep global modulation features to generate aggregated features. The aggregated features are then processed by the decoder to generate flame detection results.

10. A server, characterized in that, The server includes: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the flame detection method based on multi-frequency sensing and dynamic coding as described in any one of claims 1-8.

Images