Construction and application of rotary kiln combustion state recognition model based on dual mode

Abstract

This scheme provides a method for constructing and applying a rotary kiln combustion state recognition model based on dual modality. The rotary kiln combustion state recognition model uses the rotary kiln combustion image and the corresponding segmentation result as dual modality input, calculates the regional area index and regional brightness index in each sliding window of the multi-head self-attention calculation of the Swin Transformer, and dynamically merges the regional area index and the regional brightness index into regional weights, and uses the regional weights to adjust the current sliding window multi-head self-attention calculation. The regional adaptive attention calculation of this scheme enables the model to more flexibly focus on important feature areas in the combustion state when adaptively allocating attention, so as to improve the rotary kiln combustion state recognition model's recognition of the complex combustion state of the rotary kiln.

Description

Technical Field

[0001] The present application relates to the technical field of kiln image analysis, and in particular to a method for constructing and applying a dual-modality based rotary kiln combustion state recognition model. Background Art

[0002] As an important thermal processing equipment, rotary kiln is widely used in cement, metallurgy, chemical industry and other fields. Its main function is to process various materials through high-temperature calcination to promote physical and chemical transformation of materials. However, the operation process of rotary kiln is complicated and affected by many factors, such as fuel type, calcination temperature, material characteristics, feed speed and air volume. Therefore, how to accurately identify the combustion state of rotary kiln (over-combustion, normal combustion, under-combustion) has become a key problem to be solved. Traditional identification methods mainly rely on the experience of workers, which is difficult to meet the precise control requirements under complex working conditions. Long-term high-intensity monitoring of workers not only increases labor intensity, but also may lead to misjudgment, affecting production efficiency and environmental protection.

[0003] In response to the above problems, the patent with publication number CN117079045A discloses a rotary kiln combustion image classification method based on color correction and convolutional network: this method improves the image quality by introducing color correction technology, and then builds a network model in combination with a convolutional neural network, using the image data set as input to finally generate a three-dimensional prediction result. The system will select the combustion state with the highest prediction score as the combustion state in the current rotary kiln, and output the final recognition result. However, this method relies too much on color correction. If the initial quality of the image is poor or the lighting conditions are complex, the correction effect may not be ideal, resulting in the prediction accuracy of the subsequent convolutional network being affected. In addition, this method is based only on convolutional networks and has limited performance when processing complex or long-term dependent time series information.

[0004] The patent with publication number CN115100578B discloses a method for identifying the flame combustion state of a rotary kiln based on Res2Net and attention mechanism: this method improves the model's ability to predict the flame combustion state by introducing the Res2Net network structure and convolutional attention module (CBAM), and outputs the current combustion state based on the state with the highest score. However, although the introduction of Res2Net and attention mechanism improves the performance of the model, its computational complexity is high, the training time is long, and it is not suitable for real-time applications. In addition, although CBAM can capture global and local information, the attention mechanism may amplify irrelevant information in high-noise data, thereby affecting the accuracy of the recognition results.

[0005] The patent with publication number CN114078225A discloses a rotary kiln state recognition method based on Tucker decomposition and its related equipment: this method constructs process combination data of a time window based on the acquired process parameters, and constructs a spliced ​​image of the firing zone based on the kiln condition video, and then trains the initial full convolution model and the initial residual neural model respectively, and finally uses these two models to identify the combustion state of the rotary kiln. However, this method relies on a large amount of process parameters and video data, and the data acquisition and processing are complex and costly. In addition, although the combined use of Tucker decomposition and the two models theoretically improves the recognition accuracy, the model training and reasoning process has a large amount of computation, poor real-time performance, and high hardware requirements, which is not suitable for industrial sites with limited resources.

[0006] The patent with publication number CN112766410A discloses a method for identifying the firing state of a rotary kiln based on graph neural network feature fusion: this method expands more image features by fusing the image features of undirected graph data generated by the combination of fixed points and random points extracted by the graph neural network, thereby improving the accuracy of identifying the firing state of the rotary kiln. Although the application of graph neural networks enhances the feature extraction capability, it is highly dependent on data, especially accurate fixed point and random point data, otherwise it may lead to inaccurate image feature extraction, thereby affecting the recognition results. In addition, the training and reasoning process of graph neural networks is relatively complex and may require more computing resources, which is not conducive to deployment in low-resource environments. Summary of the invention

[0007] The embodiment of the present application provides a method for constructing and applying a rotary kiln combustion state recognition model based on dual modality, optimizes the design of a rotary kiln combustion state recognition model that can efficiently capture the complex combustion state of the rotary kiln, and improves the accuracy of rotary kiln combustion state recognition by introducing dual modal discrimination and window-level regional feature weights.

[0008] In a first aspect, an embodiment of the present application provides a method for constructing a rotary kiln combustion state recognition model based on dual modes, comprising the following steps:

[0009] Obtain rotary kiln combustion images at different combustion states;

[0010] Inputting the rotary kiln combustion image into the segmentation model and outputting the corresponding segmentation result, wherein the segmentation result marks different combustion areas in the rotary kiln combustion image;

[0011] The rotary kiln combustion image marked with the combustion state and the corresponding segmentation result are input as training data into the rotary kiln combustion state recognition architecture to train and obtain the rotary kiln combustion state recognition model, wherein the rotary kiln combustion state recognition architecture includes an input layer, a dual-channel backbone network, a combination layer and a classification head connected in sequence, wherein the dual-channel backbone network is formed by connecting multiple stages of backbone modules in sequence, and the backbone module of each stage includes parallel original image channels and segmented image channels of the same structure, and the original image channels and the segmented image channels contain a linear layer and at least one region-aware adaptive attention head, and the region-aware adaptive attention head adopts Swin Transformer, and introduces a regional brightness index and a regional area index in the multi-head self-attention calculation of Swin Transformer to perform regional adaptive attention calculation;

[0012] The rotary kiln combustion image is input from the input layer to the original image channel of the dual-channel backbone network, and the segmented image corresponding to the rotary kiln combustion image is input from the input layer to the segmented image channel of the dual-channel backbone network. The original image channel feature map output by the original image channel of the backbone module of each stage and the segmented image channel feature map output by the segmented image channel are superimposed to obtain the stage feature map of the backbone module of the current stage. The stage feature maps of different stages are combined in the combination layer to obtain a feature pyramid. The feature pyramid is input into the classification head and processed in sequence by splicing, normalization and full connection layers, and then the combustion state is output as the classification result.

[0013] In a second aspect, the present application embodiment provides an application method of a rotary kiln combustion state recognition model based on dual mode, comprising the steps of:

[0014] Acquire a rotary kiln combustion image, input the rotary kiln combustion image into a segmentation model, and output a corresponding segmentation result, wherein the segmentation result marks different combustion areas in the rotary kiln combustion image;

[0015] The rotary kiln combustion image and the corresponding segmentation result are input into the rotary kiln combustion state recognition model based on dual modality constructed by the construction method of the rotary kiln combustion state recognition model based on dual modality, and the corresponding combustion state is output.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the method for constructing a rotary kiln combustion state recognition model based on a dual modality or the method for applying a rotary kiln combustion state recognition model based on a dual modality.

[0017] In a fourth aspect, an embodiment of the present application provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute a process, and the process includes the method for constructing a rotary kiln combustion state recognition model based on dual modality or the method for applying a rotary kiln combustion state recognition model based on dual modality.

[0018] The main contributions and innovations of the present invention are as follows:

[0019] This scheme uses the rotary kiln combustion image and the corresponding segmentation result as the dual-modal input to solve the problem of low model recognition accuracy caused by the blurred boundary of the combustion area of ​​the rotary kiln combustion image. The rotary kiln combustion state recognition model designed by this scheme combines the segmentation results and the rotary kiln combustion image to fully capture the complex features of the rotary kiln combustion process; in addition, this scheme uses the backbone network based on Swin Transformer, and introduces the window-level regional feature weights in the multi-head self-attention calculation (W-MSA and SW-MSA) of Swin Transformer, so that the rotary kiln combustion state recognition model based on dual-modality can more flexibly focus on the important feature areas in the combustion state when adaptively allocating attention, thereby improving the recognition accuracy. In addition, this scheme uses a generative adversarial network to generate rotary kiln combustion images to make up for the problem of insufficient sample size, and uses the Ket network as a segmentation model to improve the segmentation accuracy of the segmentation results.

[0020] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0022] Figure 1 It is a structural schematic diagram of a rotary kiln combustion state identification model based on dual mode according to the present application;

[0023] Figure 2 is a schematic diagram of a region-aware adaptive attention head according to an embodiment of the present application;

[0024] Figure 3 is a schematic diagram of the architecture of the Swin Transformer according to an embodiment of the present application;

[0025] Figure 4 It is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0027] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0028] Embodiment 1

[0029] This solution provides a method for constructing a rotary kiln combustion state recognition model based on dual modality. The rotary kiln combustion state recognition model uses the rotary kiln combustion image and the corresponding segmentation result as dual modality input, and introduces window-level regional feature weights in the multi-head self-attention calculation of SwinTransformer to improve the rotary kiln combustion state recognition model's recognition of the rotary kiln's complex combustion state. Specifically, the structural architecture of the rotary kiln combustion state recognition model based on dual modality provided by this solution is as follows: Figure 1 As shown, the method for constructing the rotary kiln combustion state recognition model based on dual mode includes the following steps:

[0030] Obtain rotary kiln combustion images at different combustion states;

[0031] Inputting the rotary kiln combustion image into the segmentation model and outputting the corresponding segmentation result, wherein the segmentation result marks different combustion areas in the rotary kiln combustion image;

[0032] The rotary kiln combustion image marked with the combustion state and the corresponding segmentation result are input as training data into the rotary kiln combustion state recognition architecture to train and obtain the rotary kiln combustion state recognition model, wherein the rotary kiln combustion state recognition architecture includes an input layer, a dual-channel backbone network, a combination layer and a classification head connected in sequence, wherein the dual-channel backbone network is formed by connecting multiple stages of backbone modules in sequence, and the backbone module of each stage includes parallel original image channels and segmented image channels of the same structure, and the original image channels and the segmented image channels contain a linear layer and at least one region-aware adaptive attention head, and the region-aware adaptive attention head adopts Swin Transformer, and introduces a regional brightness index and a regional area index in the multi-head self-attention calculation of Swin Transformer to perform regional adaptive attention calculation;

[0033] The rotary kiln combustion image is input from the input layer to the original image channel of the dual-channel backbone network, and the segmented image corresponding to the rotary kiln combustion image is input from the input layer to the segmented image channel of the dual-channel backbone network. The original image channel feature map output by the original image channel of the backbone module of each stage and the segmented image channel feature map output by the segmented image channel are superimposed to obtain the stage feature map of the backbone module of the current stage. The stage feature maps of different stages are combined in the combination layer to obtain a feature pyramid. The feature pyramid is input into the classification head and processed in sequence by splicing, normalization and full connection layers, and then the combustion state is output as the classification result.

[0034] From the above, it can be seen that the rotary kiln combustion state recognition model constructed by this scheme takes the rotary kiln combustion image and the corresponding segmentation result as bimodal input. The segmentation result can be used to improve the problem of inaccurate classification results caused by blurred regional boundaries in the rotary kiln combustion image. At the same time, the rotary kiln combustion state recognition model adopts the regional perception adaptive attention mechanism to extract features, so that the rotary kiln combustion state recognition model can pay more attention to the characteristics of key areas, further improving the classification accuracy of subsequent combustion state classification results.

[0035] About the rotary kiln combustion state recognition architecture constructed by this solution:

[0036] The input layer of the rotary kiln combustion state recognition architecture is used to input the rotary kiln combustion image and the segmentation image.

[0037] The dual-channel backbone network of the rotary kiln combustion state recognition architecture includes a multi-stage backbone module, wherein the backbone module of each stage includes parallel original image channels and segmented image channels of the same structure, the original image channels and the segmented image channels contain linear layers and at least one region-aware adaptive attention head, and each linear layer includes mapping partitions and linear embedding.

[0038] Specifically, the features on the original image channel input to each trunk head are input to the linear layer, partitioned and positionally encoded through mapping partitioning, and then linearly changed through the linear embedding layer to obtain the corresponding linear change features of the original image. The linear change features of the original image are input to the region-aware adaptive attention mechanism for adaptive attention processing to obtain the original image channel feature map; the features on the segmented image channel input to each trunk head are input to the linear layer, partitioned and positionally encoded through mapping partitioning, and then linearly changed through the linear embedding layer to obtain the corresponding linear change features of the segmented image. The linear change features of the segmented image are input to the region-aware adaptive attention mechanism for adaptive attention processing to obtain the segmented image channel feature map. The original image channel feature map and the segmented image channel feature map are superimposed to obtain the stage feature map of the current stage.

[0039] Exemplarily, the rotary kiln combustion image and the corresponding segmentation results are respectively input into the block coding layer and the linear layer to convert the rotary kiln combustion image and the corresponding segmentation results of (H, W, 3) size into (H / 4, W / 4, C) size, where H represents the height of the rotary kiln combustion image, W represents the width of the rotary kiln combustion image, and C represents the number of channels.

[0040] In a specific embodiment of the present scheme, the dual-channel backbone network includes four stages of backbone modules, wherein the backbone module of the first stage includes a linear layer connected in sequence and two region-aware adaptive attention heads, the backbone module of the second stage includes a linear layer connected in sequence and two region-aware adaptive attention heads, the backbone module of the third stage includes a linear layer connected in sequence and six region-aware adaptive attention heads, and the backbone module of the fourth stage includes a linear layer connected in sequence and two region-aware adaptive attention heads.

[0041] It should be noted that the traditional Swin Transformer attention mechanism only calculates based on pixel-level features when identifying the combustion state of the rotary kiln, which makes it unable to effectively distinguish and emphasize the importance of different areas in the image (such as high temperature area, combustion area, and radiation area). Ultimately, the model does not pay enough attention to key areas when identifying complex combustion states, thus affecting the accuracy of recognition. Therefore, this solution improves the Swin Transformer and introduces the regional brightness index and regional area index in the multi-head self-attention calculation of the Swin Transformer for regional adaptive attention calculation.

[0042] like Figure 2 As shown, Figure 2It is a schematic diagram of the structure of the region-aware adaptive attention head. This scheme introduces the regional brightness index and regional area index into the multi-head self-attention calculation of Swin Transformer for regional adaptive attention calculation. This scheme introduces window-level regional weights in W-MSA and SW-MSA of Swin Transformer.

[0043] Specifically, the regional area index and regional brightness index are calculated in each sliding window of the multi-head self-attention calculation of the Swin Transformer, and the regional area index and regional brightness index are dynamically fused into regional weights, and the regional weights are used to adjust the multi-head self-attention calculation of the current sliding window. The regional adaptive attention calculation of this scheme enables the model to more flexibly focus on important feature areas in the combustion state when adaptively allocating attention.

[0044] Correspondingly, the proportion of pixels in different burning areas of each sliding window of Swin Transformer in the current window is calculated, and the weighted sum of the proportions of different burning areas is taken as the regional area index of the current sliding window; the ratio of the average brightness of all pixels in each sliding window of Swin Transformer to the maximum brightness is calculated as the regional brightness index of the current sliding window, and the regional weight of the current sliding window is obtained by weighting the regional area index and the regional brightness index, and the corresponding relative position encoding in the corresponding attention mechanism is adjusted based on the regional weight.

[0045] In some embodiments, the different combustion zones of the present solution include a high temperature zone, a combustion zone, and a radiation zone.

[0046] For example, the formula for calculating the area index of each sliding window is as follows:

[0047] ;

[0048] ;

[0049] ;

[0050] in , , Respectively represent The proportion of pixels in the high temperature area, combustion area, and radiation area in the sliding window of the current window, , , They represent the pixels in the high temperature area, combustion area, and radiation area in the moving window respectively. Represents the total number of pixels in the current sliding window, The value of is related to the sliding window size selection.

[0051] The weighted sum of the proportions of different burning areas is used as the regional area index of the current sliding window:

[0052] ;

[0053] in , , Respectively represent the weights of high temperature zone, combustion zone and radiation zone, Indicates the current sliding window The regional area index.

[0054] The regional brightness index of the Swin Transformer is calculated as follows:

[0055] ;

[0056] in Indicates the current sliding window The regional brightness index, Indicates the maximum brightness of the current sliding window, represents the brightness of the jth pixel, and n represents the current sliding window The total number of pixels.

[0057] The regional weight of the current sliding window is calculated as follows:

[0058] ;

[0059] in represents the regional weight of the current sliding window, α and β represent the weights of the regional area index and regional brightness index respectively. represents the regional area index, Represents the area brightness index.

[0060] The attention calculation of the corresponding relative position encoding in the attention mechanism based on the regional weight weighted adjustment is:

[0061] ;

[0062] Among them, Attention(Q,K,V) is the attention calculation, softmax() is the activation function, B represents the corresponding relative position encoding, Q represents the query vector, K represents the key vector, and V represents the value vector. Represents the length of the k vector.

[0063] The architecture of Swin Transformer is as follows Figure 3As shown, the corresponding Swin Transformer multi-head self-attention calculation is as follows:

[0064] ;

[0065] in Represents the features input to Swin Transformer, LN() represents normalization, W-MSA() and SW-MSA() both represent multi-head self-attention operations, , , Represent the output features of different stages respectively. represents the output feature map of Swin Transformer, i=1 represents the Swin Transformer corresponding to the original image channel, i=2 represents the Swin Transformer corresponding to the segmented image channel, when i=1, Represents the original image channel feature map. When i=2, Represents the segmented image feature map.

[0066] This scheme superimposes the original image channel feature map output by the original image channel of the backbone module of each stage and the segmented image channel feature map output by the segmented image channel to obtain the stage feature map of the backbone module of the current stage. The formula is as follows:

[0067] ;

[0068] in, Represents the output feature map of Swin Transformer, Represents the stage feature diagram.

[0069] The dual-channel backbone network of the rotary kiln combustion state recognition architecture of this solution uses Swin Transformer to specifically process the combustion areas of the high temperature zone, combustion zone and radiation zone, thereby fully capturing the complex characteristics of the rotary kiln combustion process.

[0070] The combination layer of the rotary kiln combustion state recognition architecture combines the stage feature maps of different stages into the feature pyramid of FPN, and constructs a top-down combined feature map based on the feature pyramid, and performs convolution processing on the combined feature map. The classification head of the rotary kiln combustion state recognition architecture performs channel dimension splicing, normalization, and full connection layer processing on the combined feature map after convolution processing to obtain the classification result.

[0071] The specific FPN feature pyramid construction formula is as follows:

[0072] ;

[0073] in The stage characteristic diagram representing the fourth stage, Represents the combined feature map in the feature pyramid corresponding to the fourth stage, The stage characteristic diagram of the third stage, It represents the combined feature map in the feature pyramid corresponding to the third stage, UpSample() represents upsampling, and scalefactor represents the step size. represents the stage feature map of the second stage, represents the combined feature map in the feature pyramid corresponding to the second stage, It represents the feature map after the convolution of the corresponding combined feature map, Upsample() represents upsampling, conv3*3 represents 3*3 convolution, Indicates the step length.

[0074] The formula for obtaining the classification result by concatenating, normalizing, and fully connecting the combined feature maps after convolution is as follows:

[0075] ;

[0076] Among them, Concat() represents the concatenation of channel dimensions, Norm() represents the normalization operation, and FC() represents the fully connected layer. Represents the output classification result, , , They represent the feature maps after convolution of the combined feature maps in the feature pyramid at different stages, dim represents the feature dimension, Represents the image after channel dimension splicing, Represents a normalized graph.

[0077] In this scheme, according to the set training conditions, the rotary kiln combustion state recognition architecture is trained with training data to obtain a rotary kiln combustion state recognition model. The training conditions may be that the number of iterations is met or the loss value is less than the set value.

[0078] In addition, in order to fully extract the information of each combustion area in the rotary kiln combustion image, this solution additionally inputs the rotary kiln combustion image into the segmentation model to output the corresponding segmentation results, wherein the segmentation results mark different combustion areas in the rotary kiln combustion image.

[0079] In some embodiments, the present solution uses K-Net network training to obtain a segmentation model of a segmentable combustion area. When the K-Net network is used as the framework of the segmentation model, the segmentation model includes a backbone network and an intermediate network, wherein the backbone network and the intermediate network perform feature extraction on the input rotary kiln combustion image to obtain an extracted feature map, and the extracted feature map is combined with the initial convolution kernel of the K-Net network to obtain an initial prediction mask prediction, and the initial prediction mask prediction is sent to the kernel update header to generate a dynamic convolution kernel. , This process is repeated S-2 times, and each updated dynamic convolution kernel is used to generate a new mask prediction. The final dynamic convolution kernel and mask prediction are used to generate the final segmentation result.

[0080] In addition, in some embodiments, since the number of samples of rotary kiln combustion images is too small and the data categories of different combustion states are unbalanced, it is easy to increase the difficulty of model training and limit the generalization ability of the model. Therefore, this scheme adopts a deep convolutional generative network to adversarially generate rotary kiln combustion images.

[0081] At this time, in the step of "obtaining rotary kiln combustion images in different combustion states", this scheme obtains the original rotary kiln combustion image, inputs the original rotary kiln combustion image into the generative adversarial network to generate a virtual rotary kiln combustion image, and integrates the original rotary kiln combustion image and the virtual rotary kiln combustion image as rotary kiln combustion images in different combustion states.

[0082] Regarding the generative adversarial network, it is assumed that the noise distribution used for generation is , the real data distribution is , and the generator and discriminator are G and D respectively, then the goal of the generative adversarial network can be described by the following formula:

[0083] ;

[0085] in It represents the sampled data from the real data distribution. represents the real sample, corresponding to the rotary kiln combustion image, It means the logarithmic probability after inputting the real sample into the discriminator D, E represents the expectation, indicates that z comes from a noise distribution, Represents the samples generated by the noise generator, corresponding to the virtual rotary kiln combustion image, Indicates that the generated samples are input into the discriminator D. represents the logarithmic function, Represents the value function. The object of max is the discriminator, allowing the discriminator to distinguish between real samples and generated samples as accurately as possible. The object of min is the generator, allowing the samples generated by the generator to deceive the discriminator as much as possible.

[0086] Embodiment 2

[0087] Based on the same concept, the embodiment of the present application provides a dual-modal rotary kiln combustion state recognition model, which is constructed according to the construction method of the dual-modal rotary kiln combustion state recognition model described in Example 1.

[0088] Embodiment 3

[0089] Based on the same concept, the embodiment of the present application provides an application method of a rotary kiln combustion state recognition model based on dual mode, comprising the steps of:

[0090] Acquire a rotary kiln combustion image, input the rotary kiln combustion image into a segmentation model, and output a corresponding segmentation result, wherein the segmentation result marks different combustion areas in the rotary kiln combustion image;

[0091] The rotary kiln combustion image and the corresponding segmentation result are input into the rotary kiln combustion state recognition model based on dual modality in the second embodiment to output the corresponding combustion state.

[0092] It should be noted that, in some embodiments, the combustion state is one of over-combustion, normal combustion, and under-combustion.

[0093] In order to verify the effect of the dual-modal rotary kiln combustion state recognition model constructed by this scheme, this scheme selects MobileVIT, ConvNext, and ViT models to classify the same rotary kiln combustion images and obtain the results shown in Table 1;

[0094] Table 1 Test results

[0095] .

[0096] Embodiment 4

[0097] This embodiment also provides an electronic device, referring to Figure 4 , comprises a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any of the above method embodiments.

[0098] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0099] Among them, the memory 404 may include a large capacity memory 404 for data or instructions. For example, but not limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In appropriate cases, the memory 404 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 404 may be inside or outside the data processing device. In a specific embodiment, the memory 404 is a non-volatile memory. In a specific embodiment, the memory 404 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable Programmable Read-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0100] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0101] The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any one of the methods for constructing a rotary kiln combustion state recognition model based on dual modality or the method for applying a rotary kiln combustion state recognition model based on dual modality in the above embodiments.

[0102] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0103] The transmission device 406 can be used to receive or send data via a network. Specific examples of the above-mentioned network may include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0104] The input / output device 408 is used to input or output information. In this embodiment, the input information may be a rotary kiln combustion image, etc., and the output information may be a combustion state, etc.

[0105] Optionally, in this embodiment, the processor 402 may be configured to perform the following steps through a computer program:

[0106] Obtain rotary kiln combustion images at different combustion states;

[0107] Inputting the rotary kiln combustion image into the segmentation model and outputting the corresponding segmentation result, wherein the segmentation result marks different combustion areas in the rotary kiln combustion image;

[0108] The rotary kiln combustion image marked with the combustion state and the corresponding segmentation result are input as training data into the rotary kiln combustion state recognition architecture to train and obtain the rotary kiln combustion state recognition model, wherein the rotary kiln combustion state recognition architecture includes an input layer, a dual-channel backbone network, a combination layer and a classification head connected in sequence, wherein the dual-channel backbone network is formed by connecting multiple stages of backbone modules in sequence, and the backbone module of each stage includes parallel original image channels and segmented image channels of the same structure, and the original image channels and the segmented image channels contain a linear layer and at least one region-aware adaptive attention head, and the region-aware adaptive attention head adopts Swin Transformer, and introduces a regional brightness index and a regional area index in the multi-head self-attention calculation of Swin Transformer to perform regional adaptive attention calculation;

[0109] The rotary kiln combustion image is input from the input layer to the original image channel of the dual-channel backbone network, and the segmented image corresponding to the rotary kiln combustion image is input from the input layer to the segmented image channel of the dual-channel backbone network. The original image channel feature map output by the original image channel of the backbone module of each stage and the segmented image channel feature map output by the segmented image channel are superimposed to obtain the stage feature map of the backbone module of the current stage. The stage feature maps of different stages are combined in the combination layer to obtain a feature pyramid. The feature pyramid is input into the classification head and processed in sequence by splicing, normalization and full connection layers, and then the combustion state is output as the classification result.

[0110] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0111] In general, various embodiments may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0112] Embodiments of the present invention can be implemented by computer software, which is executable by a data processor of a mobile device, such as in a processor entity, or implemented by hardware, or implemented by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros can be stored in any device readable data storage medium, and they include program instructions for performing specific tasks. Computer program products can include one or more computer executable components configured to perform embodiments when the program is running. One or more computer executable components can be at least one software code or a part thereof. In addition, at this point, it should be noted that any box of the logic flow in the figure can represent a program step, or interconnected logic circuits, boxes and functions, or a combination of program steps and logic circuits, boxes and functions. Software can be stored in physical media such as memory chips or storage blocks implemented in processors, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and data variants thereof, CDs. Physical media are non-transient media.

[0113] Those skilled in the art should understand that the technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0114] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for constructing a rotary kiln combustion state recognition model based on dual modes, characterized in that: The following steps are involved: Obtain rotary kiln combustion images at different combustion states; Inputting the rotary kiln combustion image into the segmentation model and outputting the corresponding segmentation result, wherein the segmentation result marks different combustion areas in the rotary kiln combustion image; The rotary kiln combustion image marked with the combustion state and the corresponding segmentation result are input as training data into the rotary kiln combustion state recognition architecture to train and obtain the rotary kiln combustion state recognition model, wherein the rotary kiln combustion state recognition architecture includes an input layer, a dual-channel backbone network, a combination layer and a classification head connected in sequence, wherein the dual-channel backbone network is formed by connecting multiple stages of backbone modules in sequence, and the backbone module of each stage includes parallel original image channels and segmented image channels of the same structure, and the original image channels and the segmented image channels contain linear layers and at least one region-aware adaptive attention head, and the region-aware adaptive attention head adopts Swin Transformer, and introduces regional brightness index and regional area index into the multi-head self-attention calculation of Swin Transformer for regional adaptive attention calculation, wherein the regional area index and regional brightness index are calculated in each sliding window of the multi-head self-attention calculation of Swin Transformer, and the regional area index and regional brightness index are dynamically fused into regional weights, and the multi-head self-attention calculation of the current sliding window is adjusted using the regional weights; wherein the calculation of Swin The proportion of pixels in different burning areas of each sliding window of the Transformer in the current window is taken as the weighted sum of the proportions of different burning areas as the regional area index of the current sliding window; the ratio of the average brightness of all pixels in each sliding window of the Swin Transformer to the maximum brightness is calculated as the regional brightness index of the current sliding window, and the regional weight of the current sliding window is obtained by weighting the regional area index and the regional brightness index, and the corresponding relative position encoding in the corresponding attention mechanism is adjusted based on the regional weight; The rotary kiln combustion image is input from the input layer to the original image channel of the dual-channel backbone network, and the segmented image corresponding to the rotary kiln combustion image is input from the input layer to the segmented image channel of the dual-channel backbone network. The original image channel feature map output by the original image channel of the backbone module of each stage and the segmented image channel feature map output by the segmented image channel are superimposed to obtain the stage feature map of the backbone module of the current stage. The stage feature maps of different stages are combined in the combination layer to obtain a feature pyramid. The feature pyramid is input into the classification head and processed in sequence by splicing, normalization and full connection layers, and then the combustion state is output as the classification result.

2. The method for constructing a dual-mode rotary kiln combustion state recognition model according to claim 1, characterized in that: Each linear layer includes a mapping partition and a linear embedding layer. The features on the original image channel input to each backbone head are input to the linear layer for partitioning and position encoding through mapping partition, and then linearly changed through the linear embedding layer to obtain the corresponding linear change features of the original image. The linear change features of the original image are input to the region-aware adaptive attention mechanism for adaptive attention processing to obtain the original image channel feature map; The features on the segmented image channel input to each backbone head are input into the linear layer for partitioning and position encoding through mapping partitioning, and then linearly changed through the linear embedding layer to obtain the corresponding segmented image linear change features. The segmented image linear change features are input into the region-aware adaptive attention mechanism for adaptive attention processing to obtain the segmented image channel feature map.

3. The method for constructing a dual-mode rotary kiln combustion state recognition model according to claim 1, characterized in that: The combination layer combines the stage feature maps of different stages into the feature pyramid of FPN, and constructs a top-down combined feature map based on the feature pyramid. The combined feature map is convolved. The classification head concatenates, normalizes and processes the combined feature map after convolution in the channel dimension to obtain the classification result.

4. The method for constructing a dual-mode rotary kiln combustion state recognition model according to claim 1, characterized in that: The K-Net network is trained to obtain a segmentation model that can segment the combustion area.

5. The method for constructing a dual-mode rotary kiln combustion state recognition model according to claim 1, characterized in that: The original rotary kiln combustion image is obtained, and the original rotary kiln combustion image is input into the generative adversarial network to generate a virtual rotary kiln combustion image. The original rotary kiln combustion image and the virtual rotary kiln combustion image are integrated as rotary kiln combustion images of different combustion states.

6. An application method of a rotary kiln combustion state recognition model based on dual modes, characterized in that: Includes steps: Acquire a rotary kiln combustion image, input the rotary kiln combustion image into a segmentation model, and output a corresponding segmentation result, wherein the segmentation result marks different combustion areas in the rotary kiln combustion image; The rotary kiln combustion image and the corresponding segmentation result are input into the dual-modal rotary kiln combustion state recognition model constructed by the construction method of the dual-modal rotary kiln combustion state recognition model according to any one of claims 1 to 5 to output the corresponding combustion state.

7. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute a method for constructing a dual-modal rotary kiln combustion state recognition model based on any one of claims 1 to 5 or an application method for a dual-modal rotary kiln combustion state recognition model based on claim 6.

8. A readable storage medium, characterized in that: A computer program is stored in the readable storage medium, and the computer program includes a program code for controlling a process to execute a process, and the process includes a method for constructing a dual-modal rotary kiln combustion state recognition model according to any one of claims 1 to 5 or an application method for a dual-modal rotary kiln combustion state recognition model according to claim 6.

Images