A Res-Transformer-based method for identifying the flame state of municipal solid waste incineration based on multi-feature enhancement

Abstract

This invention provides a Res-Transformer-based method for identifying the state of urban solid waste incineration flames based on multi-feature enhancement, including image acquisition and preprocessing, construction of a model for identifying the state of urban solid waste incineration flames, and flame state identification. This invention enriches the training data and improves the model's generalization ability by employing image denoising, dehazing, and geometric transformation preprocessing methods. It achieves efficient and automatic feature extraction and state identification of urban solid waste incineration flame images through deep learning techniques based on multiple feature enhancements. By enhancing the ability to capture details and fusing features at different scales, it improves the accuracy and robustness of identification. This invention realizes an end-to-end automated identification process, simplifies operation steps, improves identification efficiency, and significantly enhances the reliability and stability of urban solid waste incineration systems.

Description

Technical Field

[0001] This invention belongs to the field of urban solid waste incineration flame state recognition technology, specifically involving a Res-Transformer method for urban solid waste incineration flame state recognition based on multi-feature enhancement. Background Technology

[0002] Currently, the annual growth rate of municipal solid waste (MSW) has climbed to the range of 8% to 10%, and this rapid growth poses a serious challenge to waste management. Against this backdrop, municipal solid waste incineration (MSWI) technology has emerged as a widely adopted treatment method. MSWI technology is a highly efficient waste treatment method whose core lies in converting municipal solid waste into ash, flue gas, and recoverable heat energy through a high-temperature combustion process. This technology not only significantly reduces the volume of waste but also effectively controls the generation of secondary pollution and achieves resource reuse through energy recovery.

[0003] However, the application of MSWI (Maintenance, Combustion, and Inertia) technology for municipal solid waste incineration is not without its challenges. Combustion instability is a particularly intractable problem, directly leading to difficulties in meeting pollutant emission standards. It also exacerbates problems such as slagging, ash accumulation, and equipment corrosion within the furnace, and in severe cases, may even cause safety accidents like furnace explosions. Therefore, maintaining combustion stability during the MSWI process is crucial, as it directly affects the efficiency and environmental effectiveness of the entire treatment process. It is worth noting that the operation of many municipal solid waste incineration MSWI facilities still relies on the intuitive judgment of operation experts based on flame images to adjust control strategies. While this method has some practicality, it is easily limited by personal experience, subjective judgment, and insufficient level of intelligent automation, making it difficult to meet the urgent need for optimized operation of the municipal solid waste incineration MSWI process. Summary of the Invention

[0004] Purpose of the invention: The technical problem to be solved by the present invention is to address the shortcomings of the prior art by providing a Res-Transformer-based method for identifying the state of urban solid waste incineration flames based on multi-feature enhancement. This method can overcome the problem that some existing methods cannot fully extract the features of flame images and improve the accuracy of flame combustion state identification.

[0005] To achieve the above objectives, this invention provides a Res-Transformer-based method for identifying the flame state of municipal solid waste incineration based on multi-feature enhancement, comprising the following steps:

[0006] Step S1: Use a high-temperature resistant camera embedded in the furnace wall of the municipal solid waste incinerator to collect images of the flame combustion inside the furnace;

[0007] Step S2: Label the flame combustion images obtained in step S1 (usually manually), and classify the flame combustion state into four categories: normal, off-center burning, smoldering, and smoldering, to obtain the labeled dataset.

[0008] Step S3: Divide the dataset into a training set, a validation set, and a test set (generally in an 8:1:1 ratio) for subsequent training, validation, and testing of the model, respectively.

[0009] Step S4: Preprocess the training set data using image denoising, image dehazing, and geometric transformation preprocessing methods;

[0010] Step S5: Construct the residual network ResNet50 backbone network;

[0011] Step S6: Construct an efficient multi-scale attention module (EMA module) and connect it in series after each bottleneck residual block layer of the ResNet50 backbone network;

[0012] Step S7: Construct a context-aware enhanced attention module to extract context information for each bottleneck residual block layer;

[0013] Step S8: Construct a deformable Transformer module and connect it in series with the Resnet50 residual network backbone network;

[0014] Step S9: The output of the deformable Transformer module is fused with the output of the context-aware enhanced attention module, and the flame state is recognized after passing through a fully connected layer.

[0015] Step S4 includes the following steps:

[0016] Step S41: Image denoising is a fundamental step in image preprocessing, aiming to reduce or eliminate noise in the image and improve image quality. This step uses Gaussian filtering for image denoising, with the following formula:

[0017]

[0018] Where G represents the filtered signal value; It is the standard deviation of the Gaussian function, which determines the width and smoothness of the Gaussian kernel. The larger the Gaussian kernel, the wider it is, resulting in a more pronounced smoothing effect, but it may lead to a loss of image details. The smaller the Gaussian kernel, the narrower it is, resulting in a weaker smoothing effect and an image closer to its original state. x and y are the relative horizontal and vertical coordinates of elements within the convolution kernel, typically calculated with the kernel center as the origin. is the normalization factor, ensuring that the sum of all elements of the Gaussian kernel is 1, so that the brightness of the image does not change after the convolution operation. e is the natural constant.

[0019] Using a discretized Gaussian filter, the weights are calculated from the Gaussian function, and the filtering process is expressed as follows:

[0020]

[0021] Where g(x, y) is the pixel value of the filtered image, f(x, y) is the pixel value at coordinates (x, y) of the original image, G(i, j) is the weight of the Gaussian filter, and i and j represent the relative horizontal and vertical coordinates within the filter kernel, respectively;

[0022] Step S42: Image dehazing aims to restore image details obscured by fog, improving image clarity and contrast. This step uses the dark channel prior algorithm for image dehazing, with the following formula:

[0023]

[0024] Where v represents a pixel, J(v) is the recovered haze-free image, I(v) is the haze-containing image, A is the atmospheric light value, t(v) is the transmittance map, and t0 is a positive constant used to avoid the denominator being zero.

[0025] Step S43: Perform geometric transformations, including image translation and rotation. Geometric transformations are spatial transformations of the image to correct systematic and random errors during image acquisition. The image translation formula is:

[0026]

[0027] Where x' and y' represent the horizontal and vertical coordinates of the translated image, respectively, and dx and dy represent the horizontal and vertical translation amounts, respectively.

[0028] The formula for image rotation is:

[0029]

[0030] Where x'' and y'' represent the x-coordinate and y-coordinate of the rotated image, respectively. It is the rotation angle.

[0031] The image preprocessing steps described above can significantly improve image quality, providing accurate and reliable input for subsequent image recognition algorithms.

[0032] Step S5 includes the following steps:

[0033] Step S51: Construct the input layer and define the input size of the ResNet50 backbone network as 224*224*3 (height*width*color channels). The input layer is responsible for receiving the raw image data.

[0034] Step S52: Construct the initial convolutional layer. The initial part of the ResNet50 backbone network includes a 7×7 standard convolutional layer and a 3×3 max pooling layer to extract low-level features of the image. The formula is as follows:

[0035]

[0036] Among them, F input Represents the input image, Conv 7×7 (·) represents the convolution operation function for a 7×7 convolution kernel;

[0037] Maxpool 3×3 This represents the max pooling function with a sliding window size of 3×3, and F1 is the output after passing through the initial convolutional layer;

[0038] Step S53: Constructing the Bottleneck Residual Block: To further optimize computational efficiency and model performance, a bottleneck design is adopted. First, a 1x1 convolution is used to reduce the number of channels (dimensionality reduction), then a 3x3 convolution is used for feature extraction, and finally a 1x1 convolution is used to restore the number of channels (dimensionality increase). This structure significantly reduces computation and model parameters. Furthermore, skip connections are used, allowing the input to be directly passed to subsequent layers and added to the output of the convolutional layers. This design helps the network retain more information in deeper structures, promotes gradient flow, and effectively trains deeper models. The formula is:

[0039]

[0040] Among them, Conv 1×1 (·) represents the convolution operation function of a 1×1 convolution kernel, relu(·) represents the activation function, and F2 is the output after passing through the bottleneck residual block;

[0041] Step S54: Construct bottleneck residual block layers by stacking bottleneck residual blocks. Dimensionality reduction is performed by adjusting the stride of the last 3×3 convolutional kernel in each layer. A total of four bottleneck residual block layers are constructed, as shown in the formula:

[0042]

[0043] Among them, layer1, layer2, layer3 and layer4 are four bottleneck residual block layers, containing 3, 4, 6 and 3 bottleneck residual blocks respectively; F3 is the output after passing through the four bottleneck residual block layers; the ResNet50 backbone network does not contain the pooling layer and fully connected layer of the original ResNet50 network.

[0044] Step S6 includes the following steps:

[0045] Step S61: For the feature map after the bottleneck residual block, divide the feature map into g sub-feature groups, each group learning different semantics, where g is typically 8. This feature grouping method allows the model to allocate and process more GPU resources. This grouping method not only enhances feature learning in semantic regions but also compresses noise.

[0046] Step S62: For each sub-feature group, three parallel paths are used to extract attention weight descriptors. The first and second paths are 1*1 branches, and the third path is a 3*3 branch. The 1*1 branch uses one-dimensional global average pooling to encode channel information in two spatial directions respectively; the 3*3 branch captures multi-scale feature representations through 3*3 convolution. This not only encodes cross-channel information to adjust the importance of different channels, but also preserves precise spatial structure information in the channels. The output of the 1*1 branch encodes global spatial information through two-dimensional global average pooling, while the output of the 3*3 branch is directly converted to the corresponding dimensional shape. Then, the outputs of the three parallel paths are aggregated through matrix dot product operations to generate the first spatial attention map. Finally, the output feature maps within each group are aggregated through the sigmoid activation function of the two generated spatial attention weight values ​​to capture pixel-level pairing relationships and highlight the global context of all pixels.

[0047] Step S63: Concatenate the processed results of each sub-feature map along the channel dimension.

[0048] Step S7 includes the following steps:

[0049] Step S71: After extracting the feature maps from the shallow layers of the network, perform feature mapping using three identical 1×1 convolution operations to obtain feature maps a, b, and c. For feature maps a and b, the original size is C×H×W, where C is the number of channels, H is the height, and W is the width. After a reshape operation, the size becomes C×(H×W). Then, feature map a is transposed, changing its shape to (H×W)×C, and multiplied with feature map b to obtain the spatial attention value W. spatial Feature map c is also reshaped into C×(H×W). Feature map c and W are then compared... spatialMultiply to obtain the feature map Z1 after filtering in spatial dimensions. Finally, the feature map Z1 is reshaped to become C×H×W.

[0050] Step S72: After extracting the feature maps from the deep layers of the network, perform a reshape operation to obtain three two-dimensional feature maps d, e, and f. Transpose feature map f to make its shape N×C, where N=H×W. Multiply the transposed feature map d and e to obtain the attention value W in the channel dimension. channel , will W channel Multiplying with feature map f yields the filtered feature map Z2 along the channel dimension. Finally, feature map Z2 is reshaped to a shape of C×H×W.

[0051] Step S73: Add the feature maps after high and low layer filtering to obtain the feature result layer, as shown in the following formula:

[0052]

[0053] Wherein, CFFM(·) represents the constructed context-aware enhanced attention module, L1, L2, L3 and L4 are the feature maps after the first, second, third and fourth bottleneck residual block layers in the ResNet50 residual network, respectively, and F5 is the output after passing through the context-aware enhanced attention module.

[0054] Step S8 includes the following steps:

[0055] Step S81: Construct the deformable Transformer module, including the following steps:

[0056] Step S811: Calculate the attention weights, using the following formula:

[0057]

[0058] Among them, Q(F) n ), K(F n ), V(F n ) are the input features F n The query, key, and value are defined by DFConv(·), which represents a deformable convolution operation, and Conv2D(·), which represents a regular two-dimensional convolution operation.

[0059] Step S812: Calculate the spatial attention weights, using the following formula:

[0060]

[0061] Among them, A sp Represents spatial attention weights. This represents the Softmax activation function, and T represents the transpose operation. This represents element-wise multiplication of matrices;

[0062] Step S813: Calculate the deformable attention feature map, using the following formula:

[0063]

[0064] Among them, F df The calculated deformable attention feature map;

[0065] Step S814: To explore spatial context information from the input feature map, a multi-head deformable attention module is constructed using a four-head multi-head attention mechanism, with the following formula:

[0066]

[0067] Among them, F n The input feature map is MD(·), which represents the constructed multi-head deformable attention module.

[0068] Step S82: Construct the deformable Transformer module, as shown in the following formula:

[0069]

[0070] Where LN is the layer normalization function, and F4 is the output after passing through the deformable Transformer module.

[0071] Step S9 includes the following steps:

[0072] Step S91: Fuse the output of the deformable Transformer module with the output of the context-aware enhanced attention module, using the following formula:

[0073]

[0074] Where AP represents global average pooling operation, N represents the total number of image categories, FC(·) represents fully connected layer, and F6 represents the final output;

[0075] Step S92: Construct the loss function L res :

[0076]

[0077] Among them, L res y represents the loss function value of the model. n Let X represent the image label of class n, X represent the model input, and P(·) represent the prediction probability of the model for class n images given input X.

[0078] Step S93: Build and train the model.

[0079] Step S93 includes: using a residual network ResNet50 backbone, an efficient multi-scale attention module, a context-aware enhanced attention module, a deformable Transformer module, and a loss function L. res A model for identifying the flame state of urban solid waste incineration was obtained. The model was trained based on the training set, and its performance was verified based on the validation set. Finally, the model's performance was tested based on the test set.

[0080] The present invention also provides an electronic device, including a processor and a memory, the memory storing program code that, when executed by the processor, causes the processor to perform the steps of the method.

[0081] The present invention also provides a storage medium storing a computer program or instructions that, when the computer program or instructions are run on a computer, execute the steps of the method described.

[0082] This invention offers the following technical advantages: The method employs a ResNet50 residual network and a Transformer structure as the backbone network, leveraging the local feature extraction capabilities of ResNet and the global feature extraction advantages of Transformer to extract local flame combustion features and global information. An efficient multi-scale attention module (EMA) is introduced into the ResNet network, using a multi-scale parallel sub-network to establish long- and short-term dependencies, enhancing the ability to identify flame combustion regions and improving the classification performance of the residual neural network. A deformable multi-head attention module (DMAM) is designed in the Transformer layer, utilizing deformable self-attention to extract long-term feature dependencies, enhancing its global feature extraction capability. A context feature fusion module (CFFM) is designed to efficiently aggregate spatial information from shallow networks and channel information from deep networks, enhancing the cross-layer features extracted by the network. Through these multiple feature enhancement strategies, the accuracy of flame combustion state identification is significantly improved. Attached Figure Description

[0083] Figure 1 This is a schematic diagram of the method flow provided in an embodiment of the present invention.

[0084] Figure 2 This is a schematic diagram of the efficient multi-scale attention module system architecture provided in an embodiment of the present invention.

[0085] Figure 3 This is a schematic diagram of the deformable multi-head attention module system architecture provided in an embodiment of the present invention.

[0086] Figure 4 This is a schematic diagram of the system architecture of the flame combustion state recognition module provided in an embodiment of the present invention.

[0087] Figure 5 This diagram illustrates how the loss changes with the number of iterations during the training of the flame combustion state recognition model provided in this embodiment.

[0088] Figure 6 This diagram illustrates how the accuracy of the flame combustion state recognition model provided in this embodiment changes with the number of iterations during training. Detailed Implementation

[0089] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0090] This invention provides a method for identifying the flame state of municipal solid waste incineration based on multi-feature enhancement using Res-Transformer, such as... Figure 4 As shown, the backbone network is a combination of a ResNet50 residual network and a Transformer network, employing three feature enhancement strategies, such as... Figure 2 , Figure 3 and Figure 4 As shown, these are the efficient multi-scale attention module, the deformable multi-head attention module, and the context feature fusion module; in practical applications, such as... Figure 1 As shown, the present invention specifically performs the following steps:

[0091] Step S1: Use a high-temperature resistant camera embedded in the furnace wall of the municipal solid waste incinerator to collect images of the flame combustion inside the furnace;

[0092] Step S2: Manually label the acquired flame images and classify the flame combustion state into four categories: normal, off-center burning, spreading burning, and smoldering burning, to obtain the labeled dataset.

[0093] Step S3: Divide the dataset into training set, validation set and test set in a ratio of 8:1:1 for subsequent training, validation and testing of the model;

[0094] Step S4 involves preprocessing the training set data using image denoising, image dehazing, and geometric transformation methods; specifically, it includes the following steps:

[0095] Step S41: Image denoising is a fundamental step in image preprocessing, aiming to reduce or eliminate noise in the image and improve image quality. This step uses the Gaussian filtering method, which calculates weights using a Gaussian function to smooth the image and effectively suppress noise. The formula for Gaussian filtering is as follows:

[0096]

[0097] in, This represents the filtered signal value. It is the standard deviation of the Gaussian function, which determines the width and smoothness of the Gaussian kernel. The larger the Gaussian kernel, the wider it is, resulting in a more pronounced smoothing effect, but it may lead to a loss of image details. The smaller the value, the narrower the Gaussian kernel, the weaker the smoothing effect, and the closer the image is to its original state. x and y are the relative coordinates of the elements within the convolution kernel, usually calculated with the center of the convolution kernel as the origin. It is a normalization factor that ensures that the sum of all elements of the Gaussian kernel is 1, so that the brightness of the image will not change after the convolution operation.

[0098] In practical applications, a discretized Gaussian filter is used, and its weights are calculated from a Gaussian function. The filtering process can be represented as:

[0099]

[0100] Where g(x, y) are the filtered image pixel values, f(x, y) are the original image pixel values, G(i, j) are the weights of the Gaussian filter, and i and j are the relative coordinates within the filter kernel.

[0101] Step S42: Image dehazing aims to restore image details obscured by fog, improving image clarity and contrast. This step employs the dark channel prior algorithm, which is based on the prior knowledge that at least one color channel brightness value exists in a local area of ​​the haze-free image. It recovers the haze-free image by calculating transmittance and atmospheric light values. The dehazing formula for the dark channel prior algorithm is:

[0102]

[0103] Where v represents a pixel, J(v) is the recovered haze-free image, I(v) is the haze-containing image, A is the atmospheric light value, t(v) is the transmittance map, and t0 is a positive constant used to avoid the denominator being zero.

[0104] Step S43: Geometric transformation involves spatially transforming the image to correct systematic and random errors during image acquisition. This step selects translation or rotation based on actual needs. The image translation formula is as follows:

[0105]

[0106] Where x' and y' are the translated image coordinates, and dx and dy are the translation amounts in the horizontal and vertical directions, respectively. The image rotation formula is as follows:

[0107]

[0108] Where x'' and y'' represent the x-coordinate and y-coordinate of the rotated image, respectively. It is the rotation angle.

[0109] The image preprocessing steps described above can significantly improve image quality, providing accurate and reliable input for subsequent image recognition algorithms.

[0110] Step S5, construct the ResNet50 backbone network combined with the residual network; specifically including the following steps:

[0111] Step S51: Construct the input layer and define the network's input size as 224*224*3 (height*width*color channels). This layer is responsible for receiving the raw image data.

[0112] Step S52: Construct the initial convolutional layer. The initial part of the ResNet50 residual network consists of a 7×7 standard convolutional layer and a 3×3 max-pooling layer, used to extract low-level features of the image. The formula is as follows:

[0113]

[0114] Among them, F input Represents the input image, Conv 7×7 (·) represents the convolution operation function for a 7×7 convolution kernel, Maxpool. 3×3 This represents the max pooling function with a sliding window size of 3×3, and F1 is the output after passing through the initial convolutional layer.

[0115] Step S53: Constructing the Bottleneck Residual Block: To further optimize computational efficiency and model performance, a bottleneck design is adopted. First, 1x1 convolutions are used to reduce the number of channels (dimensionality reduction), then 3x3 convolutions are used for feature extraction, and finally, 1x1 convolutions are used to restore the number of channels (dimensionality increase). This structure significantly reduces computation and model parameters. Furthermore, skip connections are used, allowing the input to be directly passed to subsequent layers and added to the output of the convolutional layers. This design helps the network retain more information in deeper structures, promotes gradient flow, and effectively trains deeper models. The formula is as follows:

[0116]

[0117] Where F1 is the output of the previous layer, Conv1×1 (·) represents the convolution operation function of a 1×1 convolution kernel, relu(·) represents the activation function, and F2 is the output after passing through the bottleneck residual block.

[0118] Step S54: Construct bottleneck residual block layers. This is done by stacking bottleneck residual blocks, and dimensionality reduction is performed by adjusting the stride of the last 3×3 convolutional kernel in each layer. A total of four bottleneck residual block layers are constructed. The formula is as follows:

[0119]

[0120] Layers 1, 2, 3, and 4 are the four bottleneck residual block layers, containing 3, 4, 6, and 3 bottleneck residual blocks respectively. F1 is the output after passing through the initial convolutional layer, and F3 is the output after passing through the four bottleneck residual block layers. This network does not include the pooling layers and fully connected layers of the original ResNet50 residual network.

[0121] Step S6: Construct an efficient multi-scale attention module (EMA module), which is concatenated after each bottleneck residual block layer of the ResNet50 residual network; specifically, it includes the following steps:

[0122] Step S61: For the feature map after the bottleneck residual block, divide it into g sub-feature groups, where g is typically 8. Each group learns different semantics. This feature grouping method allows the model to allocate and process more GPU resources. This grouping method not only enhances feature learning in semantic regions but also compresses noise.

[0123] Step S62: For each sub-feature group, three parallel paths are used to extract attention weight descriptors. Two paths are 1*1 branches, and the third path is a 3*3 branch. The 1*1 branch uses one-dimensional global average pooling to encode channel information in two spatial directions. The 3*3 branch captures multi-scale feature representations through 3*3 convolution. This not only encodes cross-channel information to adjust the importance of different channels but also preserves precise spatial structure information within the channels. Specifically, the output of the 1*1 branch encodes global spatial information through two-dimensional global average pooling, while the output of the 3*3 branch is directly converted to the corresponding dimensional shape. Then, the outputs of the three branches are aggregated through matrix dot product operations to generate the first spatial attention map. Finally, the output feature maps within each group are aggregated through a sigmoid activation function of the two generated spatial attention weight values ​​to capture pixel-level pairing relationships and highlight the global context of all pixels.

[0124] Step S63: Concatenate the processed results of each sub-feature map along the channel dimension.

[0125] Step S7: Construct a context-aware enhanced attention module to extract contextual information for each bottleneck residual block layer; specifically, this includes the following steps:

[0126] Step S71: After extracting the feature maps from the shallow layers of the network, perform feature mapping using three identical 1×1 convolution operations to obtain feature maps a, b, and c. For feature maps a and b, the original size is C×H×W, where C is the number of channels, H is the height, and W is the width. After a reshape operation, the size becomes C×(H×W). Then, feature map a is transposed, changing its shape to (H×W)×C, and multiplied with feature map b to obtain the spatial attention value W. spatial Feature map c is also reshaped into C×(H×W). Feature map c and W are then compared... spatial Multiply to obtain the feature map Z1 after filtering in spatial dimensions. Finally, the feature map Z1 is reshaped to become C×H×W.

[0127] Step S72: After extracting the feature maps from the deep layers of the network, perform a reshape operation to obtain three two-dimensional feature maps d, e, and f. Transpose feature map f to make its shape N×C, where N=H×W. Multiply the transposed feature map d and e to obtain the attention value W in the channel dimension. channel W channel Multiplying with feature map f yields the filtered feature map Z2 along the channel dimension. Finally, feature map Z2 is reshaped to a shape of C×H×W.

[0128] Step S73: Add the feature maps after high and low layer filtering to obtain the feature result layer, as shown in the following formula:

[0129]

[0130] Wherein, CFFM(·) represents the constructed context-aware enhanced attention module, L1, L2, L3 and L4 are the feature maps after each bottleneck residual block layer in the ResNet50 residual network, and F5 is the output after passing through the context-aware enhanced attention module.

[0131] Step S8: Construct a deformable Transformer module and connect it in series with the residual network Resnet50 network;

[0132] Step S81: Construct a deformable multi-head attention module, including the following steps:

[0133] Step S811: Calculate the attention weights Q(F) n ), K(F n ), V(Fn The formula is as follows:

[0134]

[0135] Among them, Q(F) n ), K(F n ), V(F n ) are the input features F n The query, key, and value are defined by DFConv(·), which represents a deformable convolution operation, and Conv2D(·), which represents a regular two-dimensional convolution operation.

[0136] Step S812: Calculate the spatial attention weights, using the following formula:

[0137]

[0138] Among them, A sp Represents spatial attention weights. This represents the Softmax activation function, and T represents the transpose operation. This represents element-wise multiplication of matrices;

[0139] Step S813: Calculate the deformable attention feature map, using the following formula:

[0140]

[0141] Among them, F df This is the calculated deformable attention feature map.

[0142] Step S814: To explore rich spatial context information from the input feature map, a multi-head deformable attention module is constructed using a four-head multi-head attention mechanism, as shown in the following formula:

[0143]

[0144] Among them, F n The input feature map is MD(·), which represents the constructed multi-head deformable attention module.

[0145] Step S82: Construct the deformable Transformer module, as shown in the following formula:

[0146]

[0147] Here, F3 is the output after passing through the ResNet50 residual network, which is used as the input feature map. LN is the layer normalization function, and F4 is the output after passing through the deformable Transformer module.

[0148] Step S9 involves fusing the output of the deformable Transformer with the output of the context-aware module, and then using a fully connected layer to identify the flame state; this includes the following steps:

[0149] Step S91: Fuse the output of the deformable Transformer with the output of the context-aware module, as shown in the following formula:

[0150]

[0151] Where AP represents global average pooling operation, N represents the total number of image categories, FC(·) represents a fully connected layer, and F6 represents the final output.

[0152] Step S92: Construct the loss function, as shown in the following formula:

[0153]

[0154] Among them, L res y represents the loss function value of the model. n Let X represent the image label of class n, X represent the model input, and P(·) represent the prediction probability of the model for class n images given input X.

[0155] Step S93: Model construction and training, specifically, by constructing a Res-Transformer model for identifying the state of urban solid waste incineration flames based on multi-feature enhancement through the construction of a ResNet50 residual network module, an efficient multi-scale attention module, a deformable Transformer module, a context-aware module, and a loss function, the model is trained based on the flame combustion dataset, the model performance is verified based on the flame combustion dataset, the flame combustion state identification model is obtained, and finally the model performance is tested based on the test set.

[0156] Depend on Figure 5 It can be seen that the model training set loss first decreases steadily and then tends to level off as the number of iterations changes, indicating that the model converges. Figure 6 It can be seen that the accuracy of the model is gradually improving. Therefore, the method for identifying the flame state of urban solid waste incineration based on multi-feature enhancement provided by this invention can effectively overcome the problem of insufficient flame image feature extraction by the model and improve the accuracy of flame combustion state identification.

[0157] This invention provides a Res-Transformer-based method for identifying the flame state of municipal solid waste incineration based on multi-feature enhancement. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A method for identifying the flame state of municipal solid waste incineration based on multi-feature enhancement using Res-Transformer, characterized in that, Includes the following steps: Step S1: Use a high-temperature resistant camera embedded in the furnace wall of the municipal solid waste incinerator to collect images of the flame combustion inside the furnace; Step S2: Label the flame combustion images obtained in Step S1 and classify the flame combustion state into four categories: normal, off-center burning, cross-burning, and smoldering, to obtain the labeled dataset. Step S3: Divide the dataset into a training set, a validation set, and a test set, which will be used for subsequent training, validation, and testing of the model, respectively. Step S4: Preprocess the training set data using image denoising, image dehazing, and geometric transformation preprocessing methods; Step S5: Construct the residual network ResNet50 backbone network; Step S6: Construct an efficient multi-scale attention module, which is connected in series after each bottleneck residual block layer of the ResNet50 backbone network; Step S7: Construct a context-aware enhanced attention module to extract context information for each bottleneck residual block layer; Step S8: Construct a deformable Transformer module and connect it in series with the Resnet50 residual network backbone network; Step S9: The output of the deformable Transformer module is fused with the output of the context-aware enhanced attention module, and the flame state is recognized after passing through a fully connected layer.

2. The method according to claim 1, characterized in that, Step S4 includes the following steps: Step S41: Use Gaussian filtering to denoise the image. The formula is: ， in, This represents the filtered signal value; y is the standard deviation of the Gaussian function; x and y are the relative x and y coordinates of the elements within the convolution kernel. It is the normalization factor; e is the natural constant; Using a discretized Gaussian filter, the filtering process can be represented as follows: ， Where g(x, y) is the pixel value of the filtered image, f(x, y) is the pixel value at coordinates (x, y) of the original image, G(i,j) is the weight of the Gaussian filter, and i and j represent the relative horizontal and vertical coordinates within the filter kernel, respectively; Step S42: Use the dark channel prior algorithm to dehaze the image. The formula is: ， Where v represents a pixel, J(v) is the recovered haze-free image, I(v) is the haze-covered image, A is the atmospheric light value, t(v) is the transmittance map, and t0 is a positive constant. Step S43: Perform geometric transformations, including image translation and rotation. The image translation formula is: ， Where x' and y' represent the horizontal and vertical coordinates of the translated image, respectively, and dx and dy represent the horizontal and vertical translation amounts, respectively. The formula for image rotation is: ， Where x'' and y'' represent the x-coordinate and y-coordinate of the rotated image, respectively. It is the rotation angle.

3. The method according to claim 2, characterized in that, Step S5 includes the following steps: Step S51: Construct the input layer. Define the input size of the ResNet50 backbone network as 224*224*3. The input layer is responsible for receiving the original image data. Step S52: Construct the initial convolutional layer. The initial part of the ResNet50 backbone network includes a 7×7 standard convolutional layer and a 3×3 max pooling layer to extract low-level features of the image. The formula is as follows: ， Among them, F input Represents the input image, Conv 7×7 (·) represents the convolution operation function for a 7×7 convolution kernel; Maxpool 3×3 This represents the max pooling function with a sliding window size of 3×3, and F1 is the output after passing through the initial convolutional layer; Step S53: Constructing the bottleneck residual block: A bottleneck design is adopted. First, the number of channels is reduced through 1*1 convolutions, then features are extracted through 3*3 convolutions, and finally the number of channels is restored through 1*1 convolutions. Skip connections are used to allow the input to be directly passed to the subsequent layers and added to the output of the convolutional layers. The formula is as follows: ， Among them, Conv 1×1 (·) represents the convolution operation function of a 1×1 convolution kernel, relu(·) represents the activation function, and F2 is the output after passing through the bottleneck residual block; Step S54: Construct bottleneck residual block layers by stacking bottleneck residual blocks. Dimensionality reduction is performed by adjusting the stride of the last 3×3 convolutional kernel in each layer. A total of four bottleneck residual block layers are constructed, as shown in the formula: ， Among them, layer1, layer2, layer3 and layer4 are four bottleneck residual block layers, containing 3, 4, 6 and 3 bottleneck residual blocks respectively; F3 is the output after passing through the four bottleneck residual block layers; the ResNet50 backbone network does not contain the pooling layer and fully connected layer of the original ResNet50 network.

4. The method according to claim 3, characterized in that, Step S6 includes the following steps: Step S61: For the feature map after the bottleneck residual block, divide the feature map into g sub-feature groups, and learn different semantics for each group; Step S62: For each sub-feature group, three parallel paths are used to extract attention weight descriptors. The first and second paths are 1*1 branches, and the third path is a 3*3 branch. The 1*1 branch uses one-dimensional global average pooling to encode channel information in two spatial directions respectively; the 3*3 branch captures multi-scale feature representations through 3*3 convolution. The output of the 1*1 branch encodes global spatial information through two-dimensional global average pooling, while the output of the 3*3 branch is directly converted into the corresponding dimensional shape. Then, the outputs of the three parallel paths are aggregated through matrix dot product operations to generate the first spatial attention map. Finally, the output feature maps within each group are aggregated through the sigmoid activation function of the two generated spatial attention weight values ​​to capture pixel-level pairing relationships and highlight the global context of all pixels. Step S63: Concatenate the processed results of each sub-feature map along the channel dimension.

5. The method according to claim 4, characterized in that, Step S7 includes the following steps: Step S71: After extracting the feature maps from the shallow layers of the network, perform feature mapping using three identical 1×1 convolution operations to obtain feature maps a, b, and c. For feature maps a and b, the original size is C×H×W, where C is the number of channels, H is the height, and W is the width. After a reshape operation, the size becomes C×(H×W). Then, feature map a is transposed, changing its shape to (H×W)×C, and multiplied with feature map b to obtain the spatial attention value W. spatial Feature map c is also reshaped into C×(H×W). Feature map c and W are then compared... spatial Multiply to obtain the feature map Z1 after filtering in spatial dimensions. Finally, the feature map Z1 is reshaped to become C×H×W. Step S72: After extracting the feature maps from the deep layers of the network, perform a reshape operation to obtain three two-dimensional feature maps d, e, and f. Transpose feature map f to make its shape N×C, where N=H×W. Multiply the transposed feature map d and e to obtain the attention value W in the channel dimension. channel , will W channel Multiplying with feature map f yields the filtered feature map Z2 along the channel dimension. Finally, feature map Z2 is reshaped to a shape of C×H×W. Step S73: Add the feature maps after high and low layer filtering to obtain the feature result layer, as shown in the following formula: ， Wherein, CFFM(·) represents the constructed context-aware enhanced attention module, L1, L2, L3 and L4 are the feature maps after the first, second, third and fourth bottleneck residual block layers in the ResNet50 residual network, respectively, and F5 is the output after passing through the context-aware enhanced attention module.

6. The method according to claim 5, characterized in that, Step S8 includes the following steps: Step S81: Construct the deformable Transformer module, including the following steps: Step S811: Calculate the attention weights, using the following formula: ， Among them, Q(F) n ), K(F n ), V(F n ) are the input features F n The query, key, and value are defined by DFConv(·), which represents a deformable convolution operation, and Conv2D(·), which represents a two-dimensional convolution operation. Step S812: Calculate the spatial attention weights, using the following formula: ， Among them, A sp Represents spatial attention weights. This represents the Softmax activation function, and T represents the transpose operation. This represents element-wise multiplication of matrices; Step S813: Calculate the deformable attention feature map, using the following formula: ， Among them, F df The calculated deformable attention feature map; Step S814: To explore spatial context information from the input feature map, a multi-head deformable attention module is constructed using a four-head multi-head attention mechanism, with the following formula: ， Among them, F n The input feature map is MD(·), which represents the constructed multi-head deformable attention module. Step S82: Construct the deformable Transformer module, as shown in the following formula: ， Where LN is the layer normalization function, and F4 is the output after passing through the deformable Transformer module.

7. The method according to claim 6, characterized in that, Step S9 includes the following steps: Step S91: Fuse the output of the deformable Transformer module with the output of the context-aware enhanced attention module, using the following formula: ， Where AP represents global average pooling operation, N represents the total number of image categories, FC(·) represents fully connected layer, and F6 represents the final output; Step S92: Construct the loss function L res : ， Among them, L res y represents the loss function value of the model. n Let X represent the image label of class n, X represent the model input, and P(·) represent the prediction probability of the model for class n images given input X. Step S93: Build and train the model.

8. The method according to claim 7, characterized in that, Step S93 includes: using a residual network ResNet50 backbone, an efficient multi-scale attention module, a context-aware enhanced attention module, a deformable Transformer module, and a loss function L. res A model for identifying the flame state of urban solid waste incineration was obtained. The model was trained based on the training set, and its performance was verified based on the validation set. Finally, the model's performance was tested based on the test set.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing program code that, when executed by the processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 8.

10. A storage medium, characterized in that, It stores a computer program or instructions that, when run on a computer, perform the steps of the method as described in any one of claims 1 to 8.

Images