Res-Transform urban solid waste incineration flame state identification method based on multi-feature enhancement

Abstract

The invention provides a Res-Transform urban solid waste incineration flame state recognition method based on multi-feature enhancement. The method comprises the steps of image acquisition and preprocessing, construction of an urban solid waste incineration flame state recognition model and urban solid waste incineration flame state recognition. According to the invention, through adoption of the preprocessing method of image denoising, image defogging and geometric transformation, training data are enriched, and the generalization ability of the model is improved at the same time; efficient automatic feature extraction and state recognition of the urban solid waste incineration flame image are realized through a deep learning technology based on multi-feature enhancement, and the recognition accuracy and robustness are improved by enhancing the detail capture capability and fusing different scale features; according to the method, the end-to-end automatic identification process is realized, the operation steps are simplified, the identification efficiency is improved, and the reliability and the stability of the urban solid waste incinerator system are remarkably improved.

Description

Technical Field

[0001] The invention belongs to the technical field of municipal solid waste incineration flame state recognition, and in particular relates to a municipal solid waste incineration flame state recognition method based on a Res-Transformer with multi-feature enhancement. Background Art

[0002] At present, the annual growth rate of municipal solid waste (MSW) has climbed to the range of 8% to 10%. This rapid growth trend has brought severe challenges to waste treatment. In this context, municipal solid waste incineration (MSWI) technology has emerged as a widely adopted treatment method. MSWI technology is an efficient waste treatment method. Its core is to convert municipal solid waste into ash, flue gas and recoverable heat energy through a high-temperature combustion process. This technology can not only significantly reduce the volume of waste, but also effectively control the generation of secondary pollution and realize the reuse of resources through energy recovery.

[0003] However, the application of MSWI technology for municipal solid waste incineration is not flawless, and it still faces multiple challenges. Among them, the instability of the combustion state is a particularly thorny problem, which directly leads to the difficulty in meeting the pollutant emission standards. At the same time, it also aggravates the problems of slagging, ash accumulation and equipment corrosion in the furnace. In severe cases, it may even cause safety accidents such as furnace explosion. In view of this, it is particularly important to maintain the combustion stability in the MSWI process of municipal solid waste incineration, which is directly related to the efficiency and environmental protection effect of the entire treatment process. It is worth noting that the operation of many MSWI facilities for municipal solid waste incineration still relies on the intuitive judgment of flame images by operating experts to adjust the control strategy. Although this method is practical to a certain extent, it is easily limited by personal experience, subjective judgment and insufficient intelligence level, and it is difficult to meet the current urgent need for optimized operation of the MSWI process of municipal solid waste incineration. Summary of the invention

[0004] Purpose of the invention: The technical problem to be solved by the present invention is to provide a Res-Transformer method for identifying the flame state of urban solid waste incineration based on multi-feature enhancement in view of the shortcomings of the prior art, which can overcome the problem that some existing methods cannot fully extract flame image features and improve the accuracy of flame combustion state recognition.

[0005] To achieve the above object, the present invention provides a Res-Transformer method for identifying the flame state of municipal solid waste incineration based on multi-feature enhancement, comprising the following steps:

[0006] Step S1, using a high temperature resistant camera embedded in the wall of a municipal solid waste incinerator to acquire an image of flame combustion in the furnace;

[0007] Step S2, annotating the flame combustion image obtained in step S1 (generally manually annotated), classifying the flame combustion state into four categories: normal, partial burning, channeling and smoldering, and obtaining an annotated data set;

[0008] Step S3, dividing the data set into a training set, a validation set, and a test set (generally in a ratio of 8:1:1), which are used for subsequent training models, validation models, and testing models, respectively;

[0009] Step S4, preprocessing the training set data using image denoising, image defogging and geometric transformation preprocessing methods;

[0010] Step S5, constructing the residual network Resnet50 backbone network;

[0011] Step S6, construct an efficient multi-scale attention module (EMA module) and connect it in series after each bottleneck residual block layer of the residual network Resnet50 backbone network;

[0012] Step S7, constructing a context-aware enhanced attention module to extract context information of each bottleneck residual block layer;

[0013] Step S8, construct a deformable Transformer module and connect it in series after the residual network Resnet50 backbone network;

[0014] Step S9, the output of the deformable Transformer module is fused with the output of the context-aware enhanced attention module, and the flame state is recognized after passing through the fully connected layer.

[0015] Step S4 includes the following steps:

[0016] Step S41: Image denoising is a basic step in image preprocessing, which aims to reduce or eliminate noise in the image and improve image quality. This step uses the Gaussian filter method to denoise the image, and the formula is:

[0017]

[0018] Where G represents the signal value after filtering; is the standard deviation of the Gaussian function, which determines the width and smoothness of the Gaussian kernel. The larger the value, the wider the Gaussian kernel and the more obvious the smoothing effect, but it may cause loss of image details. The smaller it is, the narrower the Gaussian kernel is, the weaker the smoothing effect is, and the closer the image is to the original state. x and y are the relative horizontal and vertical coordinates of the elements in the convolution kernel, usually calculated with the center of the convolution kernel as the origin; is a normalization factor that ensures that the sum of all elements of the Gaussian kernel is 1, so that the brightness of the image does not change after the convolution operation. e is a natural constant.

[0019] Using a discretized Gaussian filter, the weights are calculated by the Gaussian function, and the filtering process is expressed as:

[0020]

[0021] Where g(x, y) is the pixel value of the filtered image, f(x, y) is the pixel value at the original image coordinate (x, y), G(i, j) is the weight of the Gaussian filter, and i and j represent the relative horizontal and vertical coordinates within the filter kernel, respectively;

[0022] Step S42: Image defogging aims to restore image details obscured by fog and improve image clarity and contrast. This step uses a dark channel prior algorithm to perform image defogging, and the formula is:

[0023]

[0024] Where v represents a pixel, J(v) is the restored haze-free image, I(v) is the haze image, A is the atmospheric light value, t(v) is the transmittance map, and t0 is a positive constant used to avoid the denominator being zero.

[0025] Step S43: Perform geometric transformation, including image translation and rotation. Geometric transformation is a spatial transformation of the image to correct the systematic error and random error in the image acquisition process. The image translation formula is:

[0026]

[0027] Among them, x' and y' represent the horizontal and vertical coordinates of the translated image, respectively, and dx and dy represent the horizontal and vertical translation amounts, respectively;

[0028] The image rotation formula is:

[0029]

[0030] Among them, x'' and y'' respectively represent the horizontal and vertical coordinates of the rotated image. is the rotation angle.

[0031] Through the image preprocessing in the above steps, the quality of the image can be significantly improved, providing accurate and reliable input for the subsequent image recognition algorithm.

[0032] Step S5 includes the following steps:

[0033] Step S51: construct the input layer and define the input size of the residual network Resnet50 backbone network as 224*224*3 (height*width*color channel). The input layer is responsible for receiving the original image data.

[0034] Step S52: Construct the initial convolution layer. The initial part of the residual network Resnet50 backbone network includes a 7×7 standard convolution layer and a 3×3 maximum pooling layer, which are used to extract low-level features of the image. The formula is:

[0035]

[0036] Among them, F input Represents the input image, Conv 7×7 (·) represents the convolution operation function of the 7×7 convolution kernel;

[0037] Maxpool 3×3 represents the maximum pooling function with a sliding window size of 3×3, and F1 is the output after the initial convolution layer;

[0038] Step S53: Construct bottleneck residual block: To further optimize computational efficiency and model performance, a bottleneck design is used. First, the number of channels is reduced (dimensionality reduction) through 1*1 convolution, then feature extraction is performed through 3*3 convolution, and finally the number of channels is restored (dimensionality increase) through 1*1 convolution. This structure significantly reduces the amount of computation and model parameters. In addition, a jump connection is used to allow the input to be directly passed to the subsequent layers and added to the output of the convolution layer. This design helps the network retain more information in the deep structure, promotes gradient flow, and effectively trains deeper models. The formula is:

[0039]

[0040] Among them, Conv 1×1 (·) represents the convolution operation function of the 1×1 convolution kernel, relu(·) represents the activation function, and F2 is the output after the bottleneck residual block;

[0041] Step S54: construct a bottleneck residual block layer by stacking bottleneck residual blocks, and perform dimensionality reduction by adjusting the step size of the last 3×3 convolution kernel of each layer, and construct a total of four bottleneck residual block layers. The formula is:

[0042]

[0043] Among them, layer1, layer2, layer3 and layer4 are four constructed bottleneck residual block layers, which contain 3, 4, 6 and 3 bottleneck residual blocks respectively; F3 is the output after passing through four bottleneck residual block layers; the residual network Resnet50 backbone network does not include the pooling layer and the fully connected layer of the original residual network Resnet50 network.

[0044] Step S6 includes the following steps:

[0045] Step S61: For the feature map after the bottleneck residual block, the feature map is divided into g sub-feature groups, each group learns different semantics, and g is generally 8; the feature grouping method enables the model to be allocated and processed on more GPU resources. This grouping method not only strengthens the feature learning of the semantic area, but also compresses the noise.

[0046] Step S62: For each sub-feature group, three parallel paths are used to extract attention weight descriptors. The first and second paths are 1*1 branches, and the third path is a 3*3 branch. A one-dimensional global average pooling operation is used in the 1*1 branch to encode channel information in two spatial directions respectively; the 3*3 branch captures multi-scale feature representations through 3*3 convolutions; this not only encodes cross-channel information to adjust the importance of different channels, but also retains precise spatial structure information in the channel. The output of the 1*1 branch encodes global spatial information through two-dimensional global average pooling, while the output of the 3*3 branch is directly converted to the corresponding dimensional shape. Then, the outputs of the three parallel paths are aggregated through a matrix dot product operation to generate the first spatial attention map; finally, the output feature map within each group is aggregated through the Sigmoid activation function of the two generated spatial attention weight values ​​to capture the pixel-level pairing relationship and highlight the global context of all pixels;

[0047] Step S63: concatenate the processed results of each sub-feature map in the channel dimension.

[0048] Step S7 includes the following steps:

[0049] Step S71: After extracting the feature map of the shallow layer of the network, perform feature mapping through three identical 1×1 convolution operations to obtain feature maps a, b, and c; for feature maps a and b, the original size is C×H×W, where C is the number of feature map channels, H is the height of the feature map, and W is the width of the feature map. After the reshape operation, the size becomes C×(H×W), and then the feature map a is transposed to a shape of (H×W)×C, and multiplied with the feature map b to obtain the spatial attention value W spatial ; The feature map c is also reshaped into C×(H×W) through the Reshape operation, and the feature map c and W are spatialMultiply to obtain the filtered feature map Z1 in the spatial dimension, and finally reshape the feature map Z1 to C×H×W;

[0050] Step S72: After extracting the feature map from the deep layer of the network, directly reshape it to obtain three two-dimensional feature maps d, e, and f; transpose the feature map f so that the shape becomes N×C, N=H×W, and multiply the transposed feature map d and e to obtain the attention value W of the channel dimension channel ,W channel Multiply it with the feature map f to obtain the filtered feature map Z2 on the channel dimension. Finally, reshape the feature map Z2 to C×H×W.

[0051] Step S73: Add the feature maps after high and low layer screening to obtain the feature result layer, the formula is as follows:

[0052]

[0053] Among them, CFFM(·) represents the constructed context-aware enhanced attention module, L1, L2, L3 and L4 are the feature maps after the first, second, third and fourth bottleneck residual block layers in the residual network Resnet50, respectively, and F5 is the output after the context-aware enhanced attention module.

[0054] Step S8 includes the following steps:

[0055] Step S81: construct a deformable Transformer module, including the following steps:

[0056] Step S811: Calculate the attention weight, the formula is:

[0057]

[0058] Among them, Q(F n )、K(F n )、V(F n ) are the input features F n The query, key, and value of , DFConv(·) represents a deformable convolution operation, and Conv2D(·) represents a normal two-dimensional convolution operation.

[0059] Step S812: Calculate the spatial attention weight, the formula is:

[0060]

[0061] Among them, A sp represents the spatial attention weight, represents the Softmax activation function, T represents the transposition operation, Indicates the multiplication of corresponding elements of the matrix;

[0062] Step S813: Calculate the deformable attention feature map, the formula is:

[0063]

[0064] Among them, F df is the calculated deformable attention feature map;

[0065] Step S814: In order to explore spatial context information from the input feature map, a multi-head attention module containing four heads is used to construct a multi-head deformable attention module, and the formula is:

[0066]

[0067] Among them, F n is the input feature map, MD(·) represents the constructed multi-head deformable attention module;

[0068] Step S82: Construct a deformable Transformer module, the formula is as follows:

[0069]

[0070] Among them, LN is the layer normalization function, and F4 is the output after the deformable Transformer module.

[0071] Step S9 includes the following steps:

[0072] Step S91: Fuse the output of the deformable Transformer module with the output of the context-aware enhanced attention module. The formula is:

[0073]

[0074] Among them, AP represents the global average pooling operation, N represents the total number of image categories, FC(·) represents the fully connected layer, and F6 represents the final output;

[0075] Step S92: Construct loss function L res :

[0076]

[0077] Among them, L res Represents the loss function value of the model, y n represents the image label of the nth class, X represents the model input, and P(·) represents the predicted probability of the model for the nth class image under the input X;

[0078] Step S93: Build a model and train it.

[0079] Step S93 includes: using the residual network Resnet50 backbone network, the efficient multi-scale attention module, the context-aware enhanced attention module, the deformable Transformer module and the loss function L res , a municipal solid waste incineration flame state recognition model is obtained, the model is trained based on the training set, the model performance is verified based on the validation set, and the trained municipal solid waste incineration flame state recognition model is obtained. Finally, the model performance is tested based on the test set.

[0080] The present invention also provides an electronic device, comprising a processor and a memory, wherein the memory stores program code, and when the program code is executed by the processor, the processor executes the steps of the described method.

[0081] The present invention also provides a storage medium storing a computer program or instruction. When the computer program or instruction is run on a computer, the steps of the method described are executed.

[0082] The present invention has the following technical effects: the method of the present invention adopts the residual network ResNet50 and the Transformer structure as the backbone network, and utilizes the local feature extraction capability of the residual network ResNet and the global feature extraction advantage of the Transformer to extract local flame combustion features and global information; an efficient multi-scale attention module (EMA module) is introduced into the residual network ResNet network, and a multi-scale parallel sub-network is used to establish long-short dependencies, and the recognition capability of the flame combustion area is enhanced to improve the classification performance of the residual neural network; a deformable multi-head attention module (DMAM) is designed in the Transformer layer, and deformable self-attention is used to extract long-term feature dependencies and enhance its global feature extraction capability; a context feature fusion module (CFFM) is designed to efficiently aggregate the spatial information of the shallow network and the channel information of the deep network, and enhance the cross-layer features extracted by the network. Through the above multiple feature enhancement strategies, the accuracy of flame combustion state recognition is greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 A schematic diagram of a method flow chart provided by an embodiment of the present invention.

[0084] Figure 2 Schematic diagram of the efficient multi-scale attention module system architecture provided for an embodiment of the present invention.

[0085] Figure 3 Schematic diagram of the deformable multi-head attention module system architecture provided for an embodiment of the present invention.

[0086] Figure 4 A schematic diagram of the flame combustion state recognition module system architecture provided in an embodiment of the present invention.

[0087] Figure 5 This is a schematic diagram of how the loss changes with the number of iterations during the training of the flame combustion state recognition model provided in this embodiment.

[0088] Figure 6 This is a schematic diagram of how the accuracy changes with the number of iterations during the training of the flame combustion state recognition model provided in this embodiment. DETAILED DESCRIPTION

[0089] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0090] The embodiment of the present invention provides a method for identifying the flame state of municipal solid waste incineration based on Res-Transformer with multiple feature enhancement, such as Figure 4 As shown in Figure 2, the backbone network is a residual network Resnet50 combined with a Transformer network, which uses three feature enhancement strategies, such as Figure 2 , Figure 3 and Figure 4 As shown in Figure 1, they are the efficient multi-scale attention module, the deformable multi-head attention module and the context feature fusion module. In practical applications, Figure 1 As shown, the present invention specifically performs the following steps:

[0091] Step S1, using a high temperature resistant camera embedded in the wall of a municipal solid waste incinerator to acquire an image of flame combustion in the furnace;

[0092] Step S2, manually annotating the collected flame images, classifying the flame combustion states into four categories: normal, partial burning, channeling, and smoldering, to obtain an annotated data set;

[0093] Step S3, dividing the data set into a training set, a validation set, and a test set in a ratio of 8:1:1 for subsequent training, validation, and testing of the model;

[0094] Step S4, preprocessing the training set data using image denoising, image defogging and geometric transformation preprocessing methods; specifically includes the following steps:

[0095] Step S41: Image denoising is a basic step in image preprocessing, which aims to reduce or eliminate noise in the image and improve image quality. This step uses the Gaussian filtering method, which calculates weights through the Gaussian function, smoothes the image, and effectively suppresses noise. The formula for Gaussian filtering is as follows:

[0096]

[0097] in, represents the signal value after filtering, is the standard deviation of the Gaussian function, which determines the width and smoothness of the Gaussian kernel. The larger the value, the wider the Gaussian kernel and the more obvious the smoothing effect, but it may cause loss of image details. The smaller it is, the narrower the Gaussian kernel is, the weaker the smoothing effect is, and the closer the image is to the original state. x and y are the relative coordinates of the elements in the convolution kernel, usually calculated with the center of the convolution kernel as the origin. is a normalization factor that ensures that the sum of all elements of the Gaussian kernel is 1, so that the brightness of the image does not change after the convolution operation.

[0098] In practical applications, a discretized Gaussian filter is used, and its weight is calculated by a Gaussian function. The filtering process can be expressed as:

[0099]

[0100] Where g(x, y) is the filtered image pixel value, f(x, y) is the original image pixel value, G(i, j) is the weight of the Gaussian filter, and i and j are the relative coordinates within the filter kernel.

[0101] Step S42: Image defogging aims to restore image details obscured by fog and improve image clarity and contrast. This step uses a dark channel prior algorithm, which is based on the prior knowledge that there is at least one color channel with very low brightness value in a local area of ​​a fog-free image. The algorithm restores the fog-free image by calculating the transmittance and atmospheric light value. The defogging formula of the dark channel prior algorithm is:

[0102]

[0103] Where v represents a pixel, J(v) is the restored haze-free image, I(v) is the haze image, A is the atmospheric light value, t(v) is the transmittance map, and t0 is a positive constant used to avoid the denominator being zero.

[0104] Step S43: Geometric transformation is to transform the image spatially to correct the systematic errors and random errors in the image acquisition process. This step selects translation and rotation according to actual needs. The image translation formula is as follows:

[0105]

[0106] Among them, x' and y' are the coordinates of the image after translation, and dx and dy are the translation amounts in the horizontal and vertical directions respectively. The image rotation formula is as follows:

[0107]

[0108] Among them, x'' and y'' respectively represent the horizontal and vertical coordinates of the rotated image. is the rotation angle.

[0109] Through the image preprocessing in the above steps, the quality of the image can be significantly improved, providing accurate and reliable input for the subsequent image recognition algorithm.

[0110] Step S5, constructing a Resnet50 backbone network combined with a residual network; specifically comprising the following steps:

[0111] Step S51: Construct the input layer and define the input size of the network, which is 224*224*3 (height*width*color channels). This layer is responsible for receiving the original image data.

[0112] Step S52: Construct the initial convolution layer. The initial part of the residual network ResNet50 consists of a 7×7 standard convolution layer and a 3×3 maximum pooling layer, which are used to extract low-level features of the image. The formula is as follows:

[0113]

[0114] Among them, F input Represents the input of the image, Conv 7×7 (·) represents the convolution operation function of the 7×7 convolution kernel, Maxpool 3×3 represents the maximum pooling function with a sliding window size of 3×3, and F1 is the output after the initial convolutional layer.

[0115] Step S53: Construct bottleneck residual block: To further optimize computational efficiency and model performance, a bottleneck design is used. First, the number of channels is reduced (dimensionality reduction) through 1*1 convolution, then feature extraction is performed through 3*3 convolution, and finally the number of channels is restored (dimensionality increase) through 1*1 convolution. This structure significantly reduces the amount of computation and model parameters. In addition, a jump connection is used to allow the input to be directly passed to the subsequent layers and added to the output of the convolutional layer. This design helps the network retain more information in the deep structure, promotes gradient flow, and effectively trains deeper models. The formula is as follows:

[0116]

[0117] Where F1 is the output of the previous layer, Conv1×1 (·) represents the convolution operation function of the 1×1 convolution kernel, relu(·) represents the activation function, and F2 is the output after the bottleneck residual block;.

[0118] Step S54: Construct a bottleneck residual block layer by stacking bottleneck residual blocks, and perform dimensionality reduction by adjusting the step size of the last 3×3 convolution kernel of each layer, so as to construct four bottleneck residual block layers in total. The formula is as follows:

[0119]

[0120] Among them, layer1, layer2, layer3 and layer4 are four bottleneck residual block layers, which contain 3, 4, 6 and 3 bottleneck residual blocks respectively. F1 is the output after the initial convolution layer, and F3 is the output after four bottleneck residual block layers. This network does not contain the pooling layer and fully connected layer of the original residual network Resnet50 network.

[0121] Step S6, constructing an efficient multi-scale attention module (EMA module), which is connected in series after each bottleneck residual block layer of the residual network Resnet50 network; specifically includes the following steps:

[0122] Step S61: For the feature map after the bottleneck residual block, divide it into g sub-feature groups, where the value of g is generally 8, and each group learns different semantics. The feature grouping method enables the model to be allocated and processed on more GPU resources. This grouping method not only strengthens the feature learning of the semantic area, but also compresses the noise.

[0123] Step S62: For each sub-feature group, three parallel paths are used to extract attention weight descriptors. Two paths are 1*1 branches, and the third path is a 3*3 branch. A one-dimensional global average pooling operation is used in the 1*1 branch to encode channel information in two spatial directions respectively. The 3*3 branch captures multi-scale feature representations through 3*3 convolutions. This not only encodes cross-channel information to adjust the importance of different channels, but also retains accurate spatial structure information in the channel. Specifically, the output of the 1*1 branch encodes global spatial information through two-dimensional global average pooling, while the output of the 3*3 branch is directly converted to the corresponding dimensional shape. Then, the outputs of the three branches are aggregated through a matrix dot product operation to generate the first spatial attention map. Finally, the output feature map within each group is aggregated through the Sigmoid activation function of the two generated spatial attention weight values ​​to capture the pixel-level pairing relationship and highlight the global context of all pixels.

[0124] Step S63: concatenate the processed results of each sub-feature map in the channel dimension.

[0125] Step S7, constructing a context-aware enhanced attention module to extract context information of each bottleneck residual block layer; specifically comprising the following steps:

[0126] Step S71: After extracting the feature map of the shallow layer of the network, perform feature mapping through three identical 1×1 convolution operations to obtain feature maps a, b, and c; for feature maps a and b, the original size is C×H×W, where C is the number of feature map channels, H is the height of the feature map, and W is the width of the feature map. After the reshape operation, the size becomes C×(H×W), and then the feature map a is transposed to a shape of (H×W)×C, and multiplied with the feature map b to obtain the spatial attention value W spatial ; The feature map c is also reshaped into C×(H×W) through the Reshape operation, and the feature map c and W are spatial Multiply to obtain the filtered feature map Z1 in the spatial dimension, and finally reshape the feature map Z1 to C×H×W;

[0127] Step S72: After extracting the feature map from the deep layer of the network, directly reshape it to obtain three two-dimensional feature maps d, e, and f; transpose the feature map f so that the shape becomes N×C, N=H×W, and multiply the transposed feature map d and e to obtain the attention value W of the channel dimension channel , W channel Multiply it with the feature map f to obtain the filtered feature map Z2 on the channel dimension. Finally, reshape the feature map Z2 to C×H×W.

[0128] Step S73: Add the feature maps after high and low layer screening to obtain the feature result layer, the formula is as follows:

[0129]

[0130] Among them, CFFM(·) represents the constructed context-aware enhanced attention module, L1, L2, L3 and L4 are the feature maps after each bottleneck residual block layer in the residual network Resnet50, and F5 is the output after the context-aware enhanced attention module.

[0131] Step S8, construct a deformable Transformer module and connect it in series after the residual network Resnet50 network;

[0132] Step S81: Construct a deformable multi-head attention module, including the following steps:

[0133] Step S811: Calculate the attention weight Q(F n )、K(F n )、V(Fn ), the formula is as follows:

[0134]

[0135] Among them, Q(F n )、K(F n )、V(F n ) are the input features F n The query, key, and value of , DFConv(·) represents a deformable convolution operation, and Conv2D(·) represents a normal two-dimensional convolution operation.

[0136] Step S812: Calculate the spatial attention weight, the formula is as follows:

[0137]

[0138] Among them, A sp represents the spatial attention weight, represents the Softmax activation function, T represents the transposition operation, Indicates the multiplication of corresponding elements of the matrix;

[0139] Step S813: Calculate the deformable attention feature map, the formula is as follows:

[0140]

[0141] Among them, F df is the calculated deformable attention feature map.

[0142] Step S814: In order to explore rich spatial context information from the input feature map, a multi-head attention module containing four heads is used to construct a multi-head deformable attention module. The formula is as follows:

[0143]

[0144] Among them, F n is the input feature map, and MD(·) represents the constructed multi-head deformable attention module.

[0145] Step S82: Construct a deformable Transformer module, the formula is as follows:

[0146]

[0147] Among them, F3 is the output after passing through the residual network Resnet50 network, which is used as the input feature map here, LN is the layer normalization function, and F4 is the output after passing through the deformable Transformer module.

[0148] Step S9, fusing the output of the deformable Transformer with the output of the context-aware module, and realizing the recognition of the flame state through the fully connected layer; comprising the following steps:

[0149] Step S91: Fuse the output of the deformable Transformer with the output of the context-aware module. The formula is as follows:

[0150]

[0151] Among them, AP represents the global average pooling operation, N represents the total number of image categories, FC(·) represents the fully connected layer, and F6 represents the final output.

[0152] Step S92: construct a loss function, the formula is as follows:

[0153]

[0154] Among them, L res Represents the loss function value of the model, y n represents the image label of the nth class, X represents the model input, and P(·) represents the predicted probability of the model for the nth class image under the input X;

[0155] Step S93: Model construction and training, specifically, through the construction of the residual network Resnet50 module, the construction of the efficient multi-scale attention module, the construction of the deformable Transformer module, the construction of the context perception module and the construction of the loss function, the construction of the Res-Transformer urban solid waste incineration flame state recognition model based on multi-feature enhancement is carried out, and the model is trained based on the flame combustion data set, the model performance is verified based on the flame combustion data set, the flame combustion state recognition model is obtained, and finally the model performance is tested based on the test set.

[0156] Depend on Figure 5 It can be seen that the loss of the model training set shows a trend of first steadily decreasing and then flattening as the number of iterations changes, indicating that the model converges. Figure 6 It can be seen that the accuracy of the model is gradually improving. Therefore, the method for identifying the flame state of municipal solid waste incineration based on multi-feature enhancement provided by the present invention can fully overcome the problem of insufficient flame image feature extraction by the model and improve the accuracy of flame combustion state identification.

[0157] The present invention provides a method for identifying the flame state of municipal solid waste incineration based on Res-Transformer with multi-feature enhancement. There are many methods and approaches to implement the technical solution. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention. All components not specified in this embodiment can be implemented by existing technologies.

Claims

1. A Res-Transformer method for identifying flame states of municipal solid waste incineration based on multi-feature enhancement, characterized in that: The following steps are involved: Step S1, using a high temperature resistant camera embedded in the wall of a municipal solid waste incinerator to acquire an image of flame combustion in the furnace; Step S2, annotating the flame combustion image obtained in step S1, and classifying the flame combustion state into four categories: normal, partial burning, channeling and smoldering, to obtain an annotated data set; Step S3, dividing the data set into a training set, a validation set and a test set, which are used for subsequent training models, validation models and test models respectively; Step S4, preprocessing the training set data using image denoising, image defogging and geometric transformation preprocessing methods; Step S5, constructing the residual network Resnet50 backbone network; Step S6, construct an efficient multi-scale attention module and connect it in series after each bottleneck residual block layer of the residual network Resnet50 backbone network; Step S7, constructing a context-aware enhanced attention module to extract context information of each bottleneck residual block layer; Step S8, construct a deformable Transformer module and connect it in series after the residual network Resnet50 backbone network; Step S9, the output of the deformable Transformer module is fused with the output of the context-aware enhanced attention module, and the flame state is recognized after passing through the fully connected layer.

2. The method according to claim 1, characterized in that: Step S4 includes the following steps: Step S41: Use Gaussian filtering method to perform image denoising, the formula is: ， in, Represents the signal value after filtering; is the standard deviation of the Gaussian function; x and y are the relative horizontal and vertical coordinates of the elements in the convolution kernel; is the normalization factor; e is a natural constant; Using a discretized Gaussian filter, the filtering process is expressed as: ， Where g(x, y) is the pixel value of the filtered image, f(x, y) is the pixel value at the original image coordinate (x, y), G(i,j) is the weight of the Gaussian filter, and i and j represent the relative horizontal and vertical coordinates within the filter kernel, respectively; Step S42: Use dark channel priori algorithm to perform image defogging, the formula is: ， Where v represents a pixel, J(v) is the restored fog-free image, I(v) is the foggy image, A is the atmospheric light value, t(v) is the transmittance map, and t0 is a positive constant; Step S43: Perform geometric transformation, including image translation and rotation. The image translation formula is: ， Among them, x' and y' represent the horizontal and vertical coordinates of the translated image, respectively, and dx and dy represent the horizontal and vertical translation amounts, respectively; The image rotation formula is: ， Among them, x'' and y'' respectively represent the horizontal and vertical coordinates of the rotated image. is the rotation angle.

3. The method according to claim 2, characterized in that Step S5 includes the following steps: Step S51: construct an input layer, define the input size of the residual network Resnet50 backbone network as 224*224*3, and the input layer is responsible for receiving the original image data; Step S52: Construct the initial convolution layer. The initial part of the residual network Resnet50 backbone network includes a 7×7 standard convolution layer and a 3×3 maximum pooling layer, which are used to extract low-level features of the image. The formula is: ， Among them, F input Represents the input image, Conv 7×7 (·) represents the convolution operation function of the 7×7 convolution kernel; Maxpool 3×3 represents the maximum pooling function with a sliding window size of 3×3, and F1 is the output after the initial convolution layer; Step S53: Construct a bottleneck residual block: adopt a bottleneck design, first reduce the number of channels through 1*1 convolution, then perform feature extraction through 3*3 convolution, and finally restore the number of channels through 1*1 convolution, and use skip connection to allow the input to be directly passed to the subsequent layers and added to the output of the convolution layer. The formula is: ， Among them, Conv 1×1 (·) represents the convolution operation function of the 1×1 convolution kernel, relu(·) represents the activation function, and F2 is the output after the bottleneck residual block; Step S54: construct a bottleneck residual block layer by stacking bottleneck residual blocks, and perform dimensionality reduction by adjusting the step size of the last 3×3 convolution kernel of each layer, and construct a total of four bottleneck residual block layers. The formula is: ， Among them, layer1, layer2, layer3 and layer4 are four constructed bottleneck residual block layers, which contain 3, 4, 6 and 3 bottleneck residual blocks respectively; F3 is the output after passing through four bottleneck residual block layers; the residual network Resnet50 backbone network does not include the pooling layer and the fully connected layer of the original residual network Resnet50 network.

4. The method according to claim 3, characterized in that Step S6 includes the following steps: Step S61: for the feature map after passing through the bottleneck residual block, the feature map is divided into g sub-feature groups, and each group learns different semantics; Step S62: For each sub-feature group, three parallel paths are used to extract attention weight descriptors. The first and second paths are 1*1 branches, and the third path is a 3*3 branch. A one-dimensional global average pooling operation is used in the 1*1 branch to encode channel information in two spatial directions respectively; the 3*3 branch captures multi-scale feature representation through 3*3 convolution; the output of the 1*1 branch encodes global spatial information through two-dimensional global average pooling, and the output of the 3*3 branch is directly converted to the corresponding dimensional shape. Then, the outputs of the three parallel paths are aggregated through a matrix dot product operation to generate the first spatial attention map; finally, the output feature map in each group is aggregated through the Sigmoid activation function of the two generated spatial attention weight values ​​to capture the pixel-level pairing relationship and highlight the global context of all pixels; Step S63: concatenate the processed results of each sub-feature map in the channel dimension.

5. The method according to claim 4, characterized in that Step S7 includes the following steps: Step S71: After extracting the feature map of the shallow layer of the network, perform feature mapping through three identical 1×1 convolution operations to obtain feature maps a, b, and c; for feature maps a and b, the original size is C×H×W, where C is the number of feature map channels, H is the height of the feature map, and W is the width of the feature map. After the reshape operation, the size becomes C×(H×W), and then the feature map a is transposed to a shape of (H×W)×C, and multiplied with the feature map b to obtain the spatial attention value W spatial ; The feature map c is also reshaped into C×(H×W) through the Reshape operation, and the feature map c and W are spatial Multiply to obtain the filtered feature map Z1 in the spatial dimension, and finally reshape the feature map Z1 to C×H×W; Step S72: After extracting the feature map from the deep layer of the network, directly reshape it to obtain three two-dimensional feature maps d, e, and f; transpose the feature map f so that the shape becomes N×C, N=H×W, and multiply the transposed feature map d and e to obtain the attention value W of the channel dimension channel ,W channel Multiply it with the feature map f to obtain the filtered feature map Z2 on the channel dimension. Finally, reshape the feature map Z2 to C×H×W. Step S73: Add the feature maps after high and low layer screening to obtain the feature result layer, the formula is as follows: ， Among them, CFFM(·) represents the constructed context-aware enhanced attention module, L1, L2, L3 and L4 are the feature maps after the first, second, third and fourth bottleneck residual block layers in the residual network Resnet50, respectively, and F5 is the output after the context-aware enhanced attention module.

6. The method according to claim 5, characterized in that Step S8 includes the following steps: Step S81: construct a deformable Transformer module, including the following steps: Step S811: Calculate the attention weight, the formula is: ， Among them, Q(F n )、K(F n )、V(F n ) are the input features F n The query, key, and value of , DFConv(·) represents a deformable convolution operation, and Conv2D(·) represents a two-dimensional convolution operation; Step S812: Calculate the spatial attention weight, the formula is: ， Among them, A sp represents the spatial attention weight, represents the Softmax activation function, T represents the transposition operation, Indicates the multiplication of corresponding elements of the matrix; Step S813: Calculate the deformable attention feature map, the formula is: ， Among them, F df is the calculated deformable attention feature map; Step S814: In order to explore spatial context information from the input feature map, a multi-head attention module containing four heads is used to construct a multi-head deformable attention module, and the formula is: ， Among them, F n is the input feature map, MD(·) represents the constructed multi-head deformable attention module; Step S82: Construct a deformable Transformer module, the formula is as follows: ， Among them, LN is the layer normalization function, and F4 is the output after the deformable Transformer module.

7. The method according to claim 6, characterized in that Step S9 includes the following steps: Step S91: Fuse the output of the deformable Transformer module with the output of the context-aware enhanced attention module. The formula is: ， Among them, AP represents the global average pooling operation, N represents the total number of image categories, FC(·) represents the fully connected layer, and F6 represents the final output; Step S92: Construct loss function L res : ， Among them, L res Represents the loss function value of the model, y n represents the image label of the nth class, X represents the model input, and P(·) represents the predicted probability of the model for the nth class image under the input X; Step S93: Build a model and train it.

8. The method according to claim 7, characterized in that Step S93 includes: using the residual network Resnet50 backbone network, the efficient multi-scale attention module, the context-aware enhanced attention module, the deformable Transformer module and the loss function L res , a municipal solid waste incineration flame state recognition model is obtained, the model is trained based on the training set, the model performance is verified based on the validation set, and the trained municipal solid waste incineration flame state recognition model is obtained. Finally, the model performance is tested based on the test set.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores program codes, and when the program codes are executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 8.

10. A storage medium, characterized in that: A computer program or instruction is stored, and when the computer program or instruction is run on a computer, the steps of the method according to any one of claims 1 to 8 are executed.

Images