Working condition identification method and device of rotary kiln, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a working condition identification method and device of a rotary kiln, electronic equipment and a storage medium, and belongs to the technical field of industrial automation. The method comprises the following steps: acquiring target data of the rotary kiln in at least two different modes, and inputting the target data into a pre-trained target working condition recognition model; performing feature extraction processing on the target data in different modes through a target working condition recognition model to obtain coding features corresponding to the modes, and performing shallow fusion processing on the coding features in the different modes to obtain shallow fusion features corresponding to the modes; splicing the shallow fusion features in different modes to obtain spliced features, generating a mode weight corresponding to each mode according to the spliced features, and performing deep fusion processing on the shallow fusion features in the corresponding modes based on the mode weights to obtain deep fusion features; and determining a working condition identification result of the rotary kiln according to the deep fusion features. The working condition identification accuracy of the rotary kiln can be improved.

Description

Technical Field

[0001] This application relates to the field of industrial automation technology, and in particular to a method, apparatus, electronic device and storage medium for identifying the operating conditions of a rotary kiln. Background Technology

[0002] In industrial production, a rotary kiln is a continuous thermal process where materials undergo high-temperature physicochemical reactions within a tilted, rotating cylinder. To achieve precise control, related technologies typically deploy two types of monitoring systems: one uses a distributed control system to collect quantitative process parameters such as temperature and pressure in real time, generating time-series information; the other uses industrial vision equipment to acquire key images of the production site, providing intuitive visual information. Subsequently, these technologies integrate the time-series and visual information to comprehensively determine the operating condition identification results, characterizing the current working state or operating mode of the rotary kiln.

[0003] When using fused multimodal features of a rotary kiln to determine operating condition identification results, related technologies often target situations where the semantics of features across multiple modalities are consistent. However, time-series information and image information collected in industrial settings often suffer from low quality, heterogeneity, and semantic inconsistencies. Under these circumstances, the accuracy of operating condition identification results determined using related technologies is poor. Summary of the Invention

[0004] The main objective of this application is to provide a method, apparatus, electronic device, and storage medium for identifying the operating conditions of a rotary kiln, with the aim of improving the accuracy of rotary kiln operating condition identification.

[0005] To achieve the above objectives, a first aspect of this application proposes a method for identifying the operating conditions of a rotary kiln, comprising: Acquire target data of the rotary kiln under at least two different modes, and input the target data into a pre-trained target condition recognition model; By using the target working condition recognition model, feature extraction processing is performed on target data under different modalities to obtain the coded features corresponding to each modality. The coded features under different modalities are then fused to obtain the shallow fused features corresponding to each modality. By splicing shallow fusion features from different modalities, spliced ​​features are obtained. Modal weights corresponding to each modality are generated based on the spliced ​​features. Based on the modal weights, the shallow fusion features of the corresponding modalities are subjected to deep fusion processing to obtain deep fusion features. The operating condition identification results of the rotary kiln are determined based on the deep fusion characteristics.

[0006] In some embodiments, the target data in different modalities includes image data in the visual modality and time-series data in the signal modality, and the target condition recognition model includes an image encoder, a time-series encoder and a first linear layer; Feature extraction is performed on the target data under different modalities to obtain the encoded features corresponding to each modality, including: Image data is processed by an image encoder to obtain image coding features in the visual modality. The image coding features include global and local features of the rotary kiln. The timing data is processed by a timing encoder to extract features and obtain the initial timing coding features under the signal mode. The initial timing coding features are then aligned with the feature dimensions based on the first linear layer to obtain the timing coding features under the signal mode that are aligned with the feature dimensions of the image coding features.

[0007] In some embodiments, the target condition recognition model includes multiple cascaded shallow fusion modules, and the shallow fusion features include shallow fusion image features in the visual modality and shallow fusion temporal features in the signal modality. Shallow fusion processing is performed on the encoded features under different modalities to obtain shallow fused features corresponding to each modality, including: The first and second encoded features are input into the shallow fusion module, and the shallow fusion module performs interactive fusion processing on the first and second encoded features to output the initial fused image features and the initial fused temporal features, respectively. The initial fused image features are used as the first encoding features of the next shallow fusion module, and the initial fused temporal features are used as the second encoding features of the next shallow fusion module. An iterative operation of interactive fusion processing is performed, wherein the initial value of the first encoding feature is the image encoding feature, and the initial value of the second encoding feature is the temporal encoding feature. The initial fused image features output by the last shallow fusion module are used as shallow fused image features, and the initial fused temporal features output by the last shallow fusion module are used as shallow fused temporal features.

[0008] In some embodiments, each shallow fusion module includes a first fusion module and a second fusion module. The first fusion module includes a first linear layer unit and a first multilayer perceptron, and the second fusion module includes a second linear layer unit and a second multilayer perceptron. The shallow fusion module performs interactive fusion processing on the first and second encoded features to output the initial fused image features and the initial fused temporal features, respectively, including: The first encoded feature is input into the first fusion module, and the feature is linearly processed based on the first linear layer unit to obtain the enhanced image encoded feature. The second coding feature is input into the second fusion module, and the feature is linearly processed based on the second linear layer unit to obtain the enhanced temporal coding feature. The first residual connection result is generated based on the first coding feature and the enhanced temporal coding feature, and the first residual connection result is subjected to shallow fusion processing based on the first multilayer perceptron to obtain the initial fused image features; The second residual connection result is generated based on the second coding feature and the enhanced image coding feature, and the second residual connection result is subjected to shallow fusion processing based on the second multilayer perceptron to obtain the initial fusion temporal feature.

[0009] In some embodiments, the target working condition recognition model further includes a second linear layer, and the modal weights include visual modal weights under the visual modality and signal modal weights under the signal modality; Based on modal weights, shallow fusion features under corresponding modalities are subjected to deep fusion processing to obtain deep fusion features, including: Based on visual modality weights and shallow fusion image features, the initial deep fusion image features are determined, and based on signal modality weights and shallow fusion temporal features, the initial deep fusion temporal features are determined. The deep fusion features are obtained by superimposing the initial deep fusion image features and the initial deep fusion temporal features.

[0010] In some embodiments, the target working condition identification model is trained through the following steps, including: Obtain sample data of the rotary kiln under at least two different modes, and input the sample data into the initial working condition identification model; Using the initial working condition identification model, feature extraction processing is performed on sample data under different modalities to obtain sample coding features corresponding to each modality. Then, shallow fusion processing is performed on the sample coding features under different modalities to obtain shallow fusion features corresponding to each modality. The shallow fusion features of samples under different modalities are spliced ​​together to obtain the sample splicing features. The sample modality weights corresponding to each modality are generated based on the sample splicing features. The shallow fusion features of samples under the corresponding modalities are then subjected to deep fusion processing based on the sample modality weights to obtain the deep fusion features of samples. The sample working condition identification result of the rotary kiln is determined based on the deep fusion features of the samples. The total loss value is determined based on the shallow fusion features, deep fusion features and sample working condition identification results under different modalities. The model parameters of the initial working condition identification model are adjusted according to the total loss value to obtain the trained target working condition identification model.

[0011] In some embodiments, the shallow fusion features of samples include shallow fusion image features of samples in the visual modality and shallow fusion temporal features of samples in the signal modality, and the deep fusion features of samples are determined by the deep fusion image features of samples in the visual modality and the deep fusion temporal features of samples in the signal modality. The total loss value is determined based on the shallow fusion features of samples under different modalities, the deep fusion features of samples, and the sample working condition identification results, including: Based on the shallow fusion image features and shallow fusion temporal features of the samples, the initial cross-modal alignment loss value is determined; Based on the deep fusion image features and deep fusion temporal features of the samples, the initial intra-class and inter-class loss values ​​are determined; Based on the results of the sample condition identification, the classification loss value is determined; Based on the initial cross-modal alignment loss value and the initial intra-class and inter-class loss value, the cross-modal alignment coefficient and the intra-class and inter-class coefficient are determined, and the cross-modal alignment loss value is determined based on the initial cross-modal alignment loss value and the cross-modal alignment coefficient, and the intra-class and inter-class loss value is determined based on the initial intra-class and inter-class loss value and the intra-class and inter-class coefficient. The total loss value is obtained by superimposing the cross-modal alignment loss value, the intra-class and inter-class loss value, and the classification loss value.

[0012] In some embodiments, the initial cross-modal alignment loss value is determined based on the shallow fusion image features and the shallow fusion temporal features of the samples, including: Calculate the similarity between the shallow fusion image features and the shallow fusion temporal features of the samples to obtain the initial shallow fusion similarity; The initial shallow fusion similarity is amplified by a preset exponential function to obtain the shallow fusion similarity. The total similarity value is determined based on the shallow fusion similarity, and the similarity ratio is determined based on the ratio of the shallow fusion similarity to the total similarity value. The initial cross-modal alignment loss value is obtained by taking the negative logarithm of the similarity ratio.

[0013] In some embodiments, sample data includes sample image data and sample time series data; Based on the deep fusion image features and temporal features of the samples, the initial intra-class and inter-class loss values ​​are determined, including: Obtain image classification labels for sample image data and time-series classification labels for sample time-series data packets; Calculate the similarity between the deep fusion image features and the deep fusion temporal features of the samples to obtain the initial deep fusion similarity; The similarity between the image classification labels corresponding to the deep fusion image features of the samples and the temporal classification labels corresponding to the deep fusion temporal features of the samples is calculated to obtain the label similarity. The initial intra-class and inter-class loss values ​​are determined based on the difference between the label similarity and the initial deep fusion similarity.

[0014] In some embodiments, determining the classification loss value based on the sample condition identification results includes: Obtain the actual classification label of the rotary kiln; Logarithmic processing is performed on the sample working condition identification results to obtain the predicted classification logarithmic value; The classification loss value is determined by multiplying the true classification label and the predicted classification log value.

[0015] To achieve the above objectives, a second aspect of this application provides a rotary kiln condition identification device, comprising: The acquisition module is used to acquire target data of the rotary kiln under at least two different modes and input the target data into a pre-trained target working condition recognition model. The shallow fusion module is used to extract features from target data under different modalities through the target condition recognition model to obtain the coded features corresponding to each modality, and to perform shallow fusion processing on the coded features under different modalities to obtain the shallow fused features corresponding to each modality. The deep fusion module is used to stitch together shallow fusion features from different modalities to obtain stitched features, generate modal weights for each modality based on the stitched features, and perform deep fusion processing on the shallow fusion features of the corresponding modalities based on the modal weights to obtain deep fusion features. The inference module is used to determine the operating condition identification result of the rotary kiln based on the deep fusion features.

[0016] To achieve the above objectives, a third aspect of the present application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the rotary kiln condition identification method of the first aspect described above.

[0017] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the rotary kiln condition identification method of the first aspect described above.

[0018] The rotary kiln operating condition identification method, apparatus, electronic device, and storage medium proposed in this application acquire target data of the rotary kiln under at least two different modes and input the target data into a pre-trained target operating condition identification model. Through the target operating condition identification model, feature extraction processing is performed on the target data under different modes to obtain the coded features corresponding to each mode. Shallow fusion processing is then performed on the coded features under different modes to obtain shallow fused features corresponding to each mode. The shallow fused features under different modes are then concatenated to obtain concatenated features. Modal weights corresponding to each mode are generated based on the concatenated features. Deep fusion processing is then performed on the shallow fused features under the corresponding modes based on the modal weights to obtain deep fused features. The operating condition identification result of the rotary kiln is determined based on the deep fused features.

[0019] This application's innovative two-stage fusion strategy effectively solves the semantic inconsistency problem caused by low-quality and heterogeneous multimodal data (such as images and time-series information) from rotary kilns in industrial settings. First, through a unique shallow fusion process, this application performs preliminary interaction and alignment of the encoded features of different modalities within a shared space, achieving cross-modal "semantic mutual recognition" and fundamentally reducing information noise introduced by semantic mismatch. Next, in the deep fusion stage, this application adaptively generates and allocates modal weights based on the features after shallow alignment. This allows the operating condition recognition model to intelligently determine the relative importance of image and time-series information under the current operating condition, thereby dynamically enhancing the contribution of key modalities and suppressing noise or interference from low-quality modalities. Finally, this application generates a deep fusion feature that is both information-complementary and robust. The operating condition recognition results determined based on this deep fusion feature significantly improve the accuracy and stability of rotary kiln operating condition recognition in complex industrial environments. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of an optional implementation environment for the rotary kiln condition identification device provided in this application embodiment; Figure 2 This is an optional flowchart of the rotary kiln condition identification method provided in the embodiments of this application; Figure 3 yes Figure 2 Step 102 in the flowchart is an optional implementation. Figure 4 This is a schematic diagram of an optional overall data processing flow for the rotary kiln condition identification method provided in the embodiments of this application; Figure 5 yes Figure 2 Another optional implementation flowchart for step 102 in the diagram; Figure 6 This is a schematic diagram of an optional shallow fusion network data processing flow for the rotary kiln condition identification method provided in this application embodiment; Figure 7 yes Figure 5 Step 102.2.1 is an optional implementation flowchart; Figure 8 yes Figure 2 Step 103 is an optional implementation flowchart; Figure 9 This is an optional flowchart of the rotary kiln condition identification method provided in the embodiments of this application; Figure 10 yes Figure 9 Step 204 in the flowchart is an optional implementation. Figure 11 yes Figure 10Step 204.1.1 is an optional implementation flowchart; Figure 12 yes Figure 10 Step 204.1.2 is an optional implementation flowchart; Figure 13 yes Figure 10 Step 204.1.3 is an optional implementation flowchart; Figure 14 This is a schematic diagram of an optional device module for a rotary kiln operating condition identification device provided in this application embodiment; Figure 15 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0022] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0024] In industrial production, a rotary kiln is a continuous thermal process where materials undergo high-temperature physicochemical reactions within a tilted, rotating cylinder. To achieve precise control, related technologies typically deploy two types of monitoring systems: one uses a distributed control system to collect quantitative process parameters such as temperature and pressure in real time, generating time-series information; the other uses industrial vision equipment to acquire key images of the production site, providing intuitive visual information. Subsequently, these technologies integrate the time-series and visual information to comprehensively determine the operating condition identification results, characterizing the current working state or operating mode of the rotary kiln.

[0025] When using fused multimodal features of a rotary kiln to determine operating condition identification results, related technologies often target situations where the semantics of features across multiple modalities are consistent. However, time-series information and image information collected in industrial settings often suffer from low quality, heterogeneity, and semantic inconsistencies. Under these circumstances, the accuracy of operating condition identification results determined using related technologies is poor.

[0026] Based on this, embodiments of this application provide a method, apparatus, electronic device, and storage medium for rotary kiln condition identification. Through an innovative two-stage fusion strategy, it effectively solves the semantic inconsistency problem caused by low-quality and heterogeneous multimodal data (such as images and time-series information) from industrial rotary kilns. First, this application uses a unique shallow fusion process to perform preliminary interaction and alignment of the encoded features of different modalities within a shared space, achieving cross-modal "semantic mutual recognition" and fundamentally reducing information noise introduced by semantic mismatch. Next, in the deep fusion stage, this application can adaptively generate and allocate modal weights based on the features after shallow alignment. Thus, the condition identification model can intelligently determine the relative importance of image information and time-series information under the current condition, thereby dynamically enhancing the contribution of key modalities and suppressing noise or interference from low-quality modalities. Finally, this application can generate a deep fusion feature that is both information-complementary and robust. The condition identification result determined based on this deep fusion feature significantly improves the accuracy and stability of rotary kiln condition identification in complex industrial environments.

[0027] For example, such as Figure 1 As shown, Figure 1 This is a schematic diagram of an optional implementation environment for the rotary kiln condition identification device provided in this application embodiment. The implementation environment includes a client 11 and a server 12, wherein the rotary kiln condition identification device (which can also be simply referred to as the "condition identification device") is deployed on the server 12. The server 12 receives target data in at least two different modalities sent from at least one client 11. Then, the server 12 inputs the target data into a pre-trained target condition identification model. Next, through the target condition identification model, feature extraction processing is performed on the target data in different modalities to obtain the encoded features corresponding to each modality. Shallow fusion processing is then performed on the encoded features in different modalities to obtain shallow fusion features corresponding to each modality. Afterwards, the shallow fusion features in different modalities are concatenated to obtain concatenated features. Modality weights corresponding to each modality are generated based on the concatenated features. Deep fusion processing is then performed on the shallow fusion features in the corresponding modality based on the modality weights to obtain deep fusion features. Finally, the rotary kiln condition identification result is determined based on the deep fusion features.

[0028] The final rotary kiln operating condition identification results can be used to determine the operating status of the rotary kiln in real time (such as normal, ring formation, under-burning, over-burning and other abnormal conditions), so as to provide key decision support for industrial production in practical applications. For example, operators or automated systems can promptly detect abnormalities in the kiln, optimize combustion control parameters, improve calcination quality and energy utilization efficiency, and at the same time prevent equipment damage and unplanned shutdowns, ensuring the safe, stable and efficient operation of high-temperature industrial processes such as cement and metallurgy.

[0029] The server 12 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Additionally, the server 12 can also be a node server in a blockchain network. The client 11 can be a mobile phone, computer, smart voice interaction device, smart wearable device, smart home appliance, in-vehicle terminal, etc., but is not limited to these. The client 11 and the server 12 can be connected directly or indirectly through wired or wireless communication, which is not limited in this embodiment.

[0030] It should be noted that in this application embodiment, when it involves information related to user characteristics such as basic user information or user identity, the user's permission or consent will be obtained first. Furthermore, the collection, use, and processing of this data will comply with relevant laws, regulations, and standards. In addition, when this application embodiment needs to obtain sensitive personal information of a user, the user's individual permission or consent will be obtained first. Only after obtaining the user's individual permission or consent will the necessary data for the normal operation of this application embodiment be obtained. For example, before obtaining target data under at least two different modalities, the relevant personnel's authorization or consent will be obtained first. Furthermore, other relevant data obtained by the working condition recognition device in this application are all authorized data, and will not be elaborated upon here.

[0031] In this application embodiment, the description will focus on the operating condition identification device of the rotary kiln, such as... Figure 2 As shown, Figure 2 This is an optional flowchart of the rotary kiln condition identification method provided in the embodiments of this application. Figure 2 The method may include, but is not limited to, the following steps 101 to 104. When the rotary kiln condition identification device executes the rotary kiln condition identification method, the specific process is as follows. It should be noted that this embodiment... Figure 2The order of steps 101 to 104 is not specifically limited. The order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0032] Step 101: Obtain target data of the rotary kiln under at least two different modes, and input the target data into the pre-trained target working condition recognition model.

[0033] Step 101 will be described in detail below.

[0034] A rotary kiln is a large, tilted, rotating industrial furnace widely used in industries such as cement, metallurgy, chemicals, and environmental protection for high-temperature calcination or heat treatment of materials. A rotary kiln typically consists of a long, cylindrical steel body lined with refractory material. The body rotates slowly at a certain angle, with material continuously entering from the high end (feed end) and gradually moving towards the low end (discharge end) as the kiln rotates. Simultaneously, under the action of a high-temperature airflow, it completes processes such as drying, preheating, chemical reactions (e.g., carbonate decomposition, clinker calcination), and cooling. In cement production, the rotary kiln is a core piece of equipment used to calcine raw materials into clinker; in the metallurgical industry, it can be used for roasting ores or treating solid waste.

[0035] In the data acquisition phase, the working condition recognition device first acquires target data in at least two different modalities, such as image data captured by industrial vision equipment and time-series data of process parameters collected by a distributed control system. This raw target data serves as the basis for judging the working condition and is uniformly input into a target working condition recognition model that has been pre-trained with a large amount of data and has the ability to recognize working conditions, thereby providing a standardized data entry point for subsequent feature extraction and fusion processing.

[0036] Furthermore, the target data related to the rotary kiln can also be audio / vibration data in the audio mode during the operation of the rotary kiln. Audio / vibration data is collected by installing microphones or vibration sensors on the rotary kiln cylinder or support components to collect acoustic or mechanical vibration signals during equipment operation. These signals can reflect information such as the smoothness of the kiln rotation, the contact state between the support rollers and the rolling ring, and abnormal resonance caused by internal lining material shedding or ring formation. They are important modes for judging mechanical faults and changes in kiln conditions.

[0037] Furthermore, the target data related to the rotary kiln can also be text data in the text modality during the operation of the rotary kiln. Text data can be various unstructured or semi-structured text records generated during the operation of the rotary kiln, such as: inspection logs filled out by operators (e.g., "Kiln vibration increased slightly today"), equipment fault reports (e.g., "Roller temperature alarm, suspected insufficient lubrication"), production scheduling records, shift handover notes, and alarm information or operation instruction text generated by the automation system. This text data usually contains subjective judgments of the equipment status, descriptions of anomalies, or historical handling experience. Although it is not precise numerical data, it contains rich semantic information and can serve as an important supplement to image, sensor time series, and other data. In multimodal operating condition recognition, key events or semantic features can be extracted using natural language processing technology to help the model more comprehensively understand complex operating conditions.

[0038] It should be noted that the target data can also be spectral data in a spectral mode, which can be the visible light or near-infrared spectrum of the combustion flame collected by a spectrometer at the kiln head or flame monitoring point. In other words, the target data of the rotary kiln obtained by the operating condition identification device can also be data in other modes, and can be adaptively adjusted according to the actual situation. This application embodiment does not limit this.

[0039] The target operating condition recognition model is a multimodal deep learning model specifically designed to determine the operating status of rotary kilns. It can receive various types of data (such as images, time-series signals, infrared thermography, and text) collected from different sensors or systems, and perform modal feature extraction, cross-modal fusion, and joint inference on this heterogeneous data. Ultimately, it outputs the current operating condition type of the rotary kiln (such as normal operation, ring formation, under-burning, over-burning, material segregation, or mechanical abnormality). The target operating condition recognition model is pre-trained on a large amount of labeled multimodal historical data, thus possessing comprehensive perception and pattern recognition capabilities for complex industrial processes.

[0040] Step 102: Using the target working condition recognition model, feature extraction processing is performed on the target data under different modalities to obtain the coded features corresponding to each modality. The coded features under different modalities are then subjected to shallow fusion processing to obtain the shallow fused features corresponding to each modality.

[0041] Step 102 is described in detail below.

[0042] In some embodiments, after inputting target data from at least two modalities into the target condition recognition model, the target condition recognition model performs preliminary feature extraction and alignment on the input target data: the target condition recognition model first processes the data (such as images and time series) for different modalities separately, converting them from their original format into high-dimensional numerical vectors, i.e., the encoded features corresponding to each modality. Crucially, in this embodiment, these encoded features are not directly merged; instead, a key shallow fusion process is performed. Furthermore, this shallow fusion process is not a simple feature concatenation, but a cross-modal feature interaction and calibration mechanism. This allows encoded features from different sources, whose semantics may be inconsistent, to be initially aligned within a shared feature space, thereby generating shallow fused features with more consistent semantics and stronger correlation.

[0043] Furthermore, the aligned image coding features and temporal coding features are subjected to shallow fusion processing to form shallow fusion features of multiple modal data under initial interaction. The shallow fusion processing aims to achieve information complementarity and preliminary integration between different modalities while preserving the characteristics and information of the original features of each modality, so that deeper data processing can be performed on the shallow fusion features to obtain deep fusion features.

[0044] To facilitate readers' better understanding of the specific implementation methods of the rotary kiln condition identification method, the target data obtained in this application embodiment includes image data in the visual modality and time-series data in the signal modality for illustrative purposes: In some embodiments, such as Figure 3 As shown, Figure 3 yes Figure 2 Step 102, an optional implementation flowchart, describes the feature extraction process for target data under different modalities to obtain the encoded features corresponding to each modality, including the following steps: 102.1.1 The image data is processed by an image encoder to obtain the image coding features in the visual modality. The image coding features include the global features and local features of the rotary kiln. 102.1.2 The timing encoder is used to extract features from the timing data to obtain the initial timing coding features under the signal mode. Based on the first linear layer, the initial timing coding features are aligned with the feature dimensions to obtain the timing coding features under the signal mode that are aligned with the feature dimensions of the image coding features.

[0045] Steps 102.1.1 to 102.1.2 are described in detail below.

[0046] In some embodiments, such as Figure 4 As shown, Figure 4This is a schematic diagram of an optional overall data processing flow for the rotary kiln operating condition identification method provided in this application embodiment. The target operating condition identification model includes an image encoder, a time-series encoder, and a first linear layer. After acquiring image data in the visual modality and time-series data in the signal modality, feature extraction and encoding processing are first performed on the image data and time-series data respectively. Specifically, in terms of visual modality data processing, the acquired image data is input to the image encoder, which can capture the global and local features of the rotary kiln, thereby generating image-encoded features in the visual modality. Among them, the global features describe the macroscopic understanding of the overall operation of the rotary kiln, which can characterize the overall working state of the rotary kiln, such as the distribution of flue gas concentration and the state of material fluid; while the local features describe the working conditions within a specific location of the rotary kiln, which can characterize the detailed information of specific key areas, such as the flame brightness information of the combustion core area.

[0047] Meanwhile, in terms of temporal modal data processing, the acquired temporal data is input into a temporal encoder. The temporal encoder can capture the dynamic changes and dependencies of sensor signals in the time dimension, thereby generating initial temporal coding features under the signal mode. Specifically, the temporal encoder can mine the cross-variable and cross-time relationships of the temporal data through its internal attention mechanism or recurrent structure. For example, it can model the time series of time data representing temperature, pressure, and rotational speed, and then extract key features reflecting the operating trend, periodic fluctuations, and abnormal patterns of the rotary kiln. These key features can reflect the dynamic behavior of the rotary kiln during operation. Subsequently, the initial temporal coding features are transformed using a first linear layer to make them consistent with the image coding features output by the image encoder in the vector dimension, providing a structurally unified feature representation for subsequent cross-modal fusion.

[0048] The image encoder is a deep neural network model used to convert input image data into a high-dimensional feature representation. The image encoder can be a Residual Network (ResNet), a Vision Transformer (ViT), a Visual Geometry Group Network (VGGNet), etc. The specific type of image encoder can be set according to the actual situation, and this application embodiment does not impose any limitations on this. In this application embodiment, OpenCLIP is used as the image encoder. OpenCLIP is an open-source Contrastive Language-Image Pre-training model. By jointly training the image encoder and text encoder on a large-scale image-text pair dataset, it can map images and corresponding text descriptions to the same semantic space, thereby achieving cross-modal understanding and matching.

[0049] A time-series encoder is a neural network structure specifically designed for processing time-series data. Its goal is to extract time-dependent features from continuous or periodic sensor signals. Time-series encoders can be Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), One-Dimensional Convolutional Neural Network (1D-CNN), Transformer and its variants (such as Temporal Fusion Transformer, Crossformer), etc. The specific type of time-series encoder can be selected based on the actual task's requirements for long-term dependencies, computational efficiency, and multivariate modeling. This application embodiment does not impose any limitations on this. In this application embodiment, Crossformer is used as the time-series encoder. Crossformer is a time-series model based on an attention mechanism. Through a cross-dimensional and cross-temporal attention structure, it can effectively capture complex dependencies in multivariate time series, making it suitable for efficiently encoding multi-channel sensor data such as rotary kiln temperature, pressure, and speed, and extracting key dynamic features reflecting the equipment's operating status.

[0050] The first linear layer is a fully connected linear transformation layer whose main function is to perform dimensionality mapping on the input features. In this embodiment, the first linear layer is used to interface with the initial temporal encoded features output by the temporal encoder, transforming them from their original dimensions to the same feature dimensions as the image encoded features output by the image encoder through matrix transformation. This dimensionality alignment operation ensures that features from different modalities (visual and signal) have the same spatial structure and vector length in the subsequent fusion process, thereby enabling effective cross-modal interaction and fusion. The term "first" here is for distinguishing purposes only and has no special meaning.

[0051] Combination Figure 4 As shown in the embodiments of this application, the image data is obtained through the following formula (1). Image coding features are obtained after passing through the image encoder OpenCLIP. : (1) in, Indicates the image height. Indicates the image width. Indicates the number of image channels. Represents the real number field. This represents the output dimension of the image encoder. This refers to OpenCLIP, a large image encoder based on pre-training.

[0052] Meanwhile, using the following formula (2), the Crossformer time encoder is used to mine the coupling relationships across variables, the data change trend information across time, and the state switching information in the time series data, thereby extracting information from the time series data. Initial temporal coding features were extracted. : (2) in, Indicates the sampling time length. This indicates the number of variables used for timing coding. Represents the real number field. This indicates a timing encoder based on Crossformer.

[0053] Next, the initial temporal coding features are obtained using the following formula (3). After post-processing in the first linear layer, the temporal coding features are obtained. : (3) in, The dimension representing the temporal coding feature. Represents the real number field. This indicates a linear layer.

[0054] In some embodiments, such as Figure 5 As shown, Figure 5 yes Figure 2 Another optional implementation flowchart for step 102 involves performing shallow fusion processing on the encoded features under different modalities to obtain the shallow fused features corresponding to each modality, including steps 102.2.1 to 102.2.3: 102.2.1 The first and second coding features are input into the shallow fusion module, and the shallow fusion module is used to perform interactive fusion processing on the first and second coding features to output the initial fused image features and the initial fused temporal features respectively.

[0055] First, step 102.2.1 will be described in detail.

[0056] In some embodiments, the target working condition identification model also includes a shallow fusion network, such as... Figure 6 As shown, Figure 6 This is a schematic diagram of an optional shallow fusion network data processing flow for the rotary kiln condition identification method provided in this application embodiment. After completing the encoding processing of image data and time-series data, both image encoding features and time-series encoding features are input to... Figure 6The shallow fusion network shown comprises multiple cascaded (N-layer) shallow fusion modules. These modules are connected sequentially, with the output of one module serving as the input to the next, forming a progressively layered fusion structure. Each module is responsible for interacting and fusing the image coding features and temporal coding features of the current stage, generating more comprehensive initial fused image features and initial fused temporal features. These features are then passed to the next shallow fusion module for further processing. Through this iterative cascading mechanism, image coding features in the visual modality and temporal coding features in the signal modality can continuously interact and enhance each other during the shallow fusion network processing stage. This process ultimately refines shallow fused temporal features that simultaneously reflect spatial visual characteristics and temporal dynamic changes, laying the foundation for subsequent deep fusion processing.

[0057] Furthermore, since the multiple cascaded shallow fusion modules in the shallow fusion network involve iterative data processing, the concepts of "first encoded feature" and "second encoded feature" are introduced for ease of understanding. When the acquired target features come from two different modalities, the input of each shallow fusion module in the shallow fusion network will receive two encoded features. The "first encoded feature" refers to the encoded feature under one modality, while the "second encoded feature" refers to the encoded feature under the other modality. In this embodiment, the "first encoded feature" refers to the image-related encoded feature, and the "second encoded feature" refers to the time-related encoded feature. After completing the shallow fusion processing of the input first and second encoded features, each shallow fusion module will output two types of feature data: initial fused image features and initial fused time-series features.

[0058] Next, the paper details how each shallow fusion module interactively fuses the first and second encoded features to output the initial fused image features and the initial fused temporal features, respectively: In some embodiments, such as Figure 7 As shown, Figure 7 yes Figure 5 Step 102.2.1 of the flowchart is an optional implementation that uses a shallow fusion module to perform interactive fusion processing on the first and second encoded features to output the initial fused image features and the initial fused temporal features, respectively, including: A.1 The first encoded feature is input into the first fusion module, and the feature is linearly processed based on the first linear layer unit to obtain the enhanced image encoded feature; A.2 The second coding feature is input into the second fusion module, and the feature is linearly processed based on the second linear layer unit to obtain the enhanced temporal coding feature; A.3 Generate a first residual connection result based on the first coding feature and the enhanced temporal coding feature, and perform shallow fusion processing on the first residual connection result based on the first multilayer perceptron to obtain the initial fused image features; A.4 Generate the second residual connection result based on the second coding feature and the enhanced image coding feature, and perform shallow fusion processing on the second residual connection result based on the second multilayer perceptron to obtain the initial fusion temporal feature.

[0059] Steps A.1 to A.4 are described in detail below.

[0060] In some embodiments, such as Figure 6 As shown, each shallow fusion module includes a first fusion module and a second fusion module. The first fusion module includes a first linear layer unit and a first multilayer perceptron, and the second fusion module includes a second linear layer unit and a second multilayer perceptron. The first fusion module is mainly used to process image-related feature data, while the second fusion module is mainly used to process time-related feature data. That is, the single-layer module structure of the shallow fusion network is as follows: features of different modalities are processed separately using... Projecting each linear layer (first linear layer unit) onto the data, The results obtained from the projection of each linear layer are concatenated and fed into a linear layer while keeping the dimension unchanged. The output is then connected to the feature residuals of other modalities that have not been projected through the linear layer to obtain the residual connection results of all modalities. An independent multilayer perceptron network is customized for the input of the residual connection results of each modality.

[0061] Specifically, the first encoded feature is input to the first fusion module, where it undergoes linear processing based on the first linear layer unit to obtain multiple first linear features. These multiple first linear features are then concatenated, and the concatenated result is input to another linear layer of the first fusion module to output enhanced image encoded features. The first linear layer unit here is a basic neural network layer used to perform linear mapping on the image-related first encoded feature vector. Its purpose is not direct fusion, but rather to project the feature into a feature space more suitable for interacting with temporal information. The first linear layer unit includes multiple connected linear layers, and the specific number of linear layers can be set according to actual conditions; this embodiment does not limit this.

[0062] Furthermore, the target condition recognition model performs the same preprocessing operation on the input time-related second encoded features in a completely symmetrical manner: the second encoded features are input into the second fusion module, and feature linear processing is performed based on the second linear layer unit. Similar to step A.1, the second linear layer unit here also performs a linear transformation on the second encoded features representing time-series information to obtain multiple second linear features. The multiple second linear features are then concatenated, and the concatenated result is input into another linear layer of the second fusion module to output enhanced time-series encoded features.

[0063] Furthermore, the model utilizes the processed temporal information to update and refine the first encoded features of the original input. Specifically, a first residual connection result is generated based on the first encoded features and the enhanced temporal encoded features. The first residual connection result is obtained by performing vector addition on the original image-related encoded features (first encoded features) and the transformed enhanced temporal encoded features. The core advantage of the first residual connection result is that it retains the backbone of the original first encoded features while injecting supplementary information from the temporal modality. Subsequently, the target condition recognition model can perform shallow fusion processing on the first residual connection result based on the first multilayer perceptron to obtain the initial fused image features.

[0064] In addition, the target condition recognition model simultaneously completes the other half of the two-way interaction, namely, using the processed image information to update the second encoded features of the original input. Specifically, a second residual connection result is generated based on the second encoded features and the enhanced temporal encoded features; then, the model performs shallow fusion processing on the second residual connection result based on the second multilayer perceptron, and also uses nonlinear transformation to deepen the fusion effect, finally obtaining the initial fused temporal features.

[0065] In this embodiment, both the first and second multilayer perceptrons are multilayer perceptrons (MLPs). A multilayer perceptron is a feedforward artificial neural network consisting of an input layer, one or more hidden layers, and an output layer. Each layer comprises multiple neurons (or nodes), and adjacent layers are connected via fully connected connections (i.e., each neuron is connected to all neurons in the previous layer). Each connection corresponds to a learnable weight. After linear weighted summation, each layer typically undergoes a non-linear activation function (such as ReLU, Sigmoid, or Tanh) to enable the network to fit complex non-linear relationships. The first and second multilayer perceptrons in this application can learn and extract the complex correlations within the fused features, ultimately obtaining the initial fused image features and the initial fused temporal features.

[0066] It is understood that the embodiments of this application construct a symmetrical, bidirectional information flow mechanism: it does not simply inject information from one modality into another, but through the collaborative work of steps A.3 and A.4, allows image and temporal features to simultaneously absorb each other's features to supplement and improve themselves. The design of generating residual connection results ensures that the original core information of each modality is preserved during the fusion process, effectively avoiding information overlay problems and enhancing the stability of the model. Subsequent processing based on a multilayer perceptron enables the model to capture more complex and deeper nonlinear relationships between modalities. Thus, the initial fused image features and initial fused temporal features output by each shallow fusion module are no longer isolated, but rather feature representations with stronger coupling after semantic calibration and information supplementation.

[0067] After introducing how to obtain the initial fused image features and initial fused temporal features based on any shallow fusion module, the following section will introduce the operations after step 102.2.1: 102.2.2 The initial fused image features are used as the first encoding features of the next shallow fusion module, and the initial fused temporal features are used as the second encoding features of the next shallow fusion module. An iterative operation of interactive fusion processing is performed, wherein the initial value of the first encoding feature is the image encoding feature, and the initial value of the second encoding feature is the temporal encoding feature. 102.2.3 The initial fused image features output by the last shallow fusion module are used as shallow fused image features, and the initial fused temporal features output by the last shallow fusion module are used as shallow fused temporal features.

[0068] Steps 102.2.2 to 102.2.3 are described in detail below.

[0069] In some embodiments, the alignment between modalities is deepened through iteration. Specifically, the initial fused image features generated by the previous shallow fusion module are used as the first coded features of the next shallow fusion module, and the initial fused temporal features are used as the second coded features of the next shallow fusion module, and the aforementioned interaction process is repeated. This iterative operation of performing interactive fusion processing forms a cascaded or stacked structure, making the interaction of features no longer a single, superficial process, but a multi-round, in-depth one. To clarify the starting point of this iterative process, embodiments of this application specify that the initial value of the first coded feature is an image coded feature, and the initial value of the second coded feature is a temporal coded feature, thereby ensuring that the entire iterative alignment process starts from the most original coded features.

[0070] Furthermore, after multiple rounds of iterative interaction, the initial fused image features output by the last shallow fusion module are used as the shallow fused image features, and similarly, the initial fused temporal features output by the last shallow fusion module are used as the shallow fused temporal features. Here, "last" refers to the last of the multiple cascaded shallow fusion modules in sequence, i.e., the endpoint of the iterative operation. The shallow fused image features and shallow fused temporal features have undergone deep interaction and mutual refinement, providing a high-quality feature representation foundation for the next step of deep fusion processing. Thus, this embodiment effectively solves the information noise problem caused by inconsistent semantic expressions of different modalities during the fusion process in traditional rotary kiln condition recognition, thereby achieving semantic mutual recognition of multimodal data.

[0071] In some embodiments, for image coding features and temporal coding features After passing through a shallow fusion network under multimodal processing, the two images yield shallow fused image features that incorporate temporal correlation features. And shallow fusion temporal features that incorporate image-related features. The process can be represented by the following formulas (4) to (9): (4) (5) (6) (7) (8) (9) In the formula, Represents the real number field. and The dimensions representing image coding features and temporal coding features. This indicates the number of heads in the shallow fusion network. This indicates the number of layers in the shallow fusion network (i.e., the number of shallow fusion modules). Indicates a linear layer. Indicates the first A linear layer, This represents a vector concatenation operation. This represents a multilayer perceptron. Indicates the first The initial fused image features output by n (n=1, 2...n) shallow fusion modules. Indicates the first The initial fused temporal features output by each shallow fusion module. The shallow fusion network maps the features of each modality to an independent space and identifies multi-level features under each modality, which helps to express modality-specific information. Cross-modal interaction is achieved through residual connections, effectively solving the problems of spatial heterogeneity and semantic inconsistency between image data and temporal data.

[0072] Understandably, compared to the single, simple feature fusion in traditional methods, this application employs an iterative operation of interactive fusion processing, achieving layer-by-layer exploration and deep alignment of complex, nonlinear relationships between features of different modalities. Each iteration allows image features and temporal features to interact and fuse more fully at a higher level of semantic space, thereby gradually eliminating semantic biases caused by heterogeneity in the original data. Therefore, the final output of shallowly fused image features and shallowly fused temporal features exhibits a much higher intrinsic semantic consistency than the original encoded features, providing highly pure and aligned input for subsequent adaptive deep fusion, and resolving the problem of decreased recognition accuracy caused by modal semantic inconsistency at the initial level.

[0073] Step 103: Concatenate shallow fusion features from different modalities to obtain concatenated features. Generate modal weights for each modality based on the concatenated features. Perform deep fusion processing on the shallow fusion features of the corresponding modality based on the modal weights to obtain deep fusion features.

[0074] Step 103 will be described in detail below.

[0075] In some embodiments, the target condition identification model then performs adaptive deep fusion processing based on the initially aligned features. First, as... Figure 4 As shown, shallow fusion features from different modalities are concatenated to form a concatenated feature containing information from all modalities after initial alignment. Subsequently, the target condition recognition model dynamically generates a corresponding modality weight for each modality based on this concatenated feature. The modality weight is a quantitative indicator representing the relative importance or credibility of the modality information for the final condition judgment under the current input data. Finally, based on this modality weight, deep fusion processing is performed on the shallow fusion features of the corresponding modality, which is represented by a weighted summation, thus obtaining a final deep fusion feature. This deep fusion process ensures that important or high-quality modal information is enhanced, while secondary or noisy modal information is suppressed.

[0076] In some embodiments, the target condition recognition model further includes a second linear layer, and the modal weights include visual modal weights under the visual modality and signal modal weights under the signal modality. For example... Figure 4As shown, after obtaining the stitched features based on the shallow fusion image features and shallow fusion temporal features, the stitched features are processed by the second linear layer, and the result after linear processing is activated (using the Softmax function) to determine the visual modality weights and signal modality weights. This process can be represented by the following formula (10): (10) in, Represents the visual modal weights in adaptive learning. Represents the signal mode weights in adaptive learning. Represents the temporal features of shallow images. This represents the temporal characteristics of shallow fusion. It is an activation function. Indicates a linear layer. This represents a vector concatenation operation. In other words, this embodiment of the application designs an adaptive modality weight learning network based on linear layers to achieve deep fusion of complementary semantic information from image features and temporal features.

[0077] The Softmax function is a commonly used activation function for multi-class classification tasks. It transforms a vector containing arbitrary real numbers (usually the model's original output, i.e., logits) into a probability distribution vector. Specifically, Softmax takes the exponent (e raised to a power) of each element in the vector to ensure it is positive, and then normalizes it by dividing by the sum of all element exponents, so that all components of the output vector are between 0 and 1, and their sum is 1. In this way, each output value can be interpreted as the probability of the corresponding class, facilitating the model's classification decision. For example, in classification tasks, the output of the Softmax layer represents the probability that the input sample belongs to each class, and the class with the highest probability is the prediction result.

[0078] In some embodiments, such as Figure 8 As shown, Figure 8 yes Figure 2 Step 103, an optional implementation flowchart, involves performing deep fusion processing on the shallow fusion features of the corresponding modality based on modal weights to obtain deep fusion features, including the following steps: 103.1.1 Based on visual modality weights and shallow fusion image features, determine the initial deep fusion image features, and based on signal modality weights and shallow fusion temporal features, determine the initial deep fusion temporal features; 103.1.2 The initial deep fusion image features and the initial deep fusion temporal features are superimposed to obtain the deep fusion features.

[0079] Steps 103.1.1 to 103.1.2 are described in detail below.

[0080] In some embodiments, deep fusion features are determined by the following formula (11). : (11) In the formula, Represents the real number field. The dimension of the feature representing the deep fusion feature. Represents the visual modal weights in adaptive learning. Represents the signal mode weights in adaptive learning. This represents shallowly fused image features. This represents the temporal characteristics of shallow fusion. Represents the dot product. Represents deep fused image features. This represents the temporal characteristics of deep fusion.

[0081] It is understood that the target working condition recognition model in this application embodiment has the ability to intelligently determine the relative importance of each modal information based on real-time input data. For example, when the image data quality is poor or the information is blurry, the target working condition recognition model can automatically reduce the visual modal weight, and vice versa, effectively avoiding the "contamination" of the overall result by low-quality data. Subsequently, the overlay operation, based on this dynamic weighting, completes an efficient and lossless feature merging. The "modulation first, merging later" strategy adopted in this application embodiment ensures that the final deep fusion feature can maximize the use of high-quality modal information and suppress noise modal interference, thereby significantly improving the accuracy of working condition recognition and adaptability to complex industrial environments.

[0082] Step 104: Determine the operating condition identification result of the rotary kiln based on the deep fusion characteristics.

[0083] Step 104 will be described in detail below.

[0084] In some embodiments, the target operating condition identification model makes a final operating condition determination based on highly refined fusion features. Specifically, the deep fusion feature generated in step 103 is a robust representation that includes multimodal complementary information and has undergone adaptive weight optimization. After obtaining the deep fusion feature, it is fed into the end classifier of the target operating condition identification model, where it is mapped to one of the predefined operating condition categories. Finally, the category output by the target operating condition identification model is the operating condition identification result of the rotary kiln determined by this method, which provides a decision-making basis for on-site operators or automated control systems.

[0085] The operating condition identification results can specifically include various typical operating state categories during the rotary kiln's operation, such as: normal operation, ring formation, kiln lining collapse, uneven feeding, incomplete combustion, over-burning, under-burning, localized overheating, or abnormal cooling zone. The operating condition identification results reflect the physical state of the equipment, the stability of the thermal regime, and the coordination of process parameters. These results provide operators with real-time operational diagnostics, helping to adjust parameters such as fuel supply, rotational speed, airflow, or feed rate in a timely manner. They can also serve as input for closed-loop regulation or fault warning in automated control systems, thereby ensuring the safe, stable, and efficient operation of the rotary kiln.

[0086] Furthermore, as shown in the figure, this embodiment of the application also designs a fully connected layer network for classification. The deep fusion features are input into the fully connected layer network and passed through the Softmax activation function to obtain the classification result, i.e., the working condition identification result. The process can be represented by the following formula (12): (12) In the formula, It is an activation function. Represents the real number field. Indicates deep fusion characteristics, This indicates a fully connected layer network.

[0087] Understandably, this application addresses the common problem of low-quality, heterogeneous, and semantically inconsistent multimodal data fusion processing in industrial settings using rotary kiln data. The embodiments employ a unique two-stage fusion architecture: First, the shallow fusion processing in step 102 pre-solves the semantic gap between different modalities at the feature level, laying the foundation for subsequent effective fusion. Next, in step 103, by generating modal weights based on the spliced ​​features and performing deep fusion processing, dynamic and adaptive evaluation of the contribution of each modality's information is achieved, effectively amplifying key information and suppressing noise interference. This two-stage feature processing results in higher information quality and robustness in the final deep fusion features, thereby ensuring higher accuracy and reliability of the operating condition identification results determined in step 104.

[0088] The above is an introduction to the inference application of the target working condition recognition model. The following will explain in detail how to train the target working condition recognition model: In some embodiments, such as Figure 9 As shown, Figure 9 This is an optional flowchart of the rotary kiln operating condition identification method provided in this application embodiment. The target operating condition identification model is obtained through the following steps: Step 201: Obtain sample data of the rotary kiln under at least two different modes, and input the sample data into the initial working condition identification model; Step 202: Using the initial working condition recognition model, feature extraction processing is performed on the sample data under different modalities to obtain the sample coding features corresponding to each modality. The sample coding features under different modalities are then subjected to shallow fusion processing to obtain the sample shallow fusion features corresponding to each modality. Step 203: Concatenate shallow fusion features of samples under different modalities to obtain sample concatenation features; generate sample modality weights corresponding to each modality based on the sample concatenation features; perform deep fusion processing on the shallow fusion features of samples under the corresponding modalities based on the sample modality weights to obtain deep fusion features of samples. Step 204: Determine the sample working condition identification result of the rotary kiln based on the deep fusion features of the samples. Determine the total loss value based on the shallow fusion features, deep fusion features, and sample working condition identification results under different modalities. Adjust the model parameters of the initial working condition identification model based on the total loss value to obtain the trained target working condition identification model.

[0089] Steps 201 to 204 are described in detail below.

[0090] In some embodiments, sample data of the rotary kiln under at least two different modes are first obtained. Here, the sample data specifically refers to historical data with real working condition classification labels, which is the basis for supervised learning of the model. Subsequently, these sample data with known answers are input into the initial working condition identification model. Here, the initial working condition identification model refers to a model whose internal model parameters have not yet been fully optimized for the current task.

[0091] In some embodiments, the acquired sample data includes sample image data and sample time-series data. The sample image data can be obtained by first acquiring sample videos and then extracting frames from them. After acquiring the sample image data and sample time-series data, they can be simply aligned in time, and missing values ​​can be removed to obtain preprocessed sample data. Further, worker operation variables from the sample time-series data are used for clustering, and the sample data is labeled. The labels can be divided into three categories: under-burned, properly burned, and over-burned. The labeled sample image data and sample time-series data used for subsequent training and input into the initial working condition recognition model at the same time point are identical.

[0092] In some embodiments, the initial working condition recognition model performs forward propagation computation on the input training samples to complete the initial feature extraction and alignment. Through the initial working condition recognition model, feature extraction processing is performed on sample data under different modalities to obtain sample encoding features corresponding to each modality. These original features are then shallowly fused with the sample encoding features under different modalities, aiming to allow features from different modalities to interact and calibrate at the semantic level, ultimately obtaining shallowly fused sample features corresponding to each modality.

[0093] In some embodiments, the initial working condition recognition model continues to perform forward propagation to complete adaptive deep fusion of features. First, shallow fusion features of samples under different modalities are concatenated to obtain sample concatenated features. Then, sample modality weights corresponding to each modality are generated based on the sample concatenated features. The sample modality weights can dynamically evaluate the importance of information in each modality. Next, the initial working condition recognition model performs deep fusion processing on the shallow fusion features of samples under the corresponding modalities based on the sample modality weights, and obtains deep fusion features of samples through weighted merging.

[0094] In some embodiments, the core steps of the initial working condition recognition model learning are performed: loss calculation and parameter update. First, the initial working condition recognition model determines the sample working condition recognition result of the rotary kiln based on the deep fusion features of the samples, the shallow fusion features of the samples under different modalities, and the total loss value based on the deep fusion features of the samples under different modalities. The total loss value here is a comprehensive evaluation metric; it not only measures the difference between the final predicted result and the true label but also includes the optimization objective for the output features of the intermediate layers of the model (shallow fusion and deep fusion). Finally, the model parameters of the initial working condition recognition model are adjusted based on the total loss value, typically using algorithms such as backpropagation and gradient descent. Through these step-by-step fine-tuning steps, the trained target working condition recognition model is finally obtained, enabling it to accurately identify working conditions.

[0095] The structure and data processing flow of the initial working condition identification model are similar to those of the target working condition identification model. Specific implementation methods can be found in steps 101 to 104, which will be briefly described here and will not be elaborated upon further.

[0096] It is understood that the embodiments of this application design a total loss value that can utilize shallow fusion features, deep fusion features, and sample working condition recognition results to achieve precise "guidance" for different functional modules within the model. This composite loss function can provide a clear semantic alignment target for the shallow fusion module and a basis for learning effective modal weights for the deep fusion module, thereby ensuring that every key component of the model can be adjusted towards optimization. Therefore, the target working condition recognition model obtained through this training method has a more coordinated and efficient internal structure and parameters, and its overall performance and generalization ability far exceed those of traditional rotary kiln single-objective optimization models.

[0097] In some embodiments, such as Figure 10 As shown, Figure 10 yes Figure 9 Step 204, an optional implementation flowchart, determines the total loss value based on the shallow fusion features of samples under different modalities, the deep fusion features of samples, and the sample working condition recognition results, including the following steps: 204.1.1 Based on the shallow fusion image features and shallow fusion temporal features of the samples, the initial cross-modal alignment loss value is determined; 204.1.2 Based on the deep fusion image features and temporal features of the deep fusion samples, determine the initial intra-class and inter-class loss values; 204.1.3 Based on the above sample condition identification results, determine the classification loss value; 204.1.4 Based on the initial cross-modal alignment loss value and the initial intra-class and inter-class loss value, determine the cross-modal alignment coefficient and the intra-class and inter-class coefficient, and determine the cross-modal alignment loss value based on the initial cross-modal alignment loss value and the cross-modal alignment coefficient, and determine the intra-class and inter-class loss value based on the initial intra-class and inter-class loss value and the intra-class and inter-class coefficient; 204.1.5 The cross-modal alignment loss value, intra-class and inter-class loss value, and classification loss value are superimposed to obtain the total loss value.

[0098] Steps 204.1.1 to 204.1.5 are described in detail below.

[0099] In some embodiments, the shallow fusion features of samples include shallow fusion image features of samples in the visual modality and shallow fusion temporal features of samples in the signal modality, and the deep fusion features of samples are determined by the deep fusion image features of samples in the visual modality and the deep fusion temporal features of samples in the signal modality.

[0100] Furthermore, this embodiment of the application performs a specific performance evaluation on the shallow fusion stage of the initial working condition recognition model. Specifically, step 204.1.1 determines the initial cross-modal alignment loss value based on the sample shallow fusion image features and sample shallow fusion temporal features. Here, the initial cross-modal alignment loss value is a quantitative indicator that measures the consistency or alignment degree in the semantic space of two features (sample shallow fusion image features and sample shallow fusion temporal features) originally from different modalities after shallow fusion processing by the shallow fusion network. The calculation of the cross-modal alignment loss value directly affects the output of the shallow fusion network, providing it with a clear optimization target, and prompting the feature representations of different modalities to achieve basic semantic "mutual recognition" before fusion during training.

[0101] Furthermore, embodiments of this application also evaluate the quality of the feature space after deep fusion. Specifically, step 204.1.2 determines the initial intra-class and inter-class loss values ​​based on the sample deep fusion image features and sample deep fusion temporal features. The initial intra-class and inter-class loss values ​​here are a structured loss designed to optimize the feature representation ultimately used for classification. It encourages sample features belonging to the same working condition category to move closer together in space (reducing intra-class distance) while simultaneously causing sample features from different working condition categories to move further apart (increasing inter-class distance), thereby making the feature space more discriminative.

[0102] Furthermore, this embodiment of the application also directly measures the final task performance. Specifically, step 204.1.3 determines the classification loss value based on the sample working condition identification results. The classification loss value here is the most direct supervisory signal; it calculates the difference between the model's predicted output (i.e., the sample working condition identification result) and the true label inherent in the sample data. This loss value is the core driving force for the model to learn the correct classification of working conditions; the smaller its value, the more accurate the initial working condition identification model's prediction.

[0103] Furthermore, this application embodiment also introduces an adaptive weight adjustment mechanism. Specifically, step 204.1.4 determines the cross-modal alignment coefficient and the intra-class and inter-class coefficient based on the initial cross-modal alignment loss value and the initial intra-class and inter-class loss value. This means that the model can dynamically generate coefficients to balance the importance of different loss terms based on the performance of the two sub-tasks of feature alignment and feature space construction in the current training phase (i.e., the magnitude of the initial loss value). Subsequently, the initial condition recognition model can determine the cross-modal alignment loss value based on the initial cross-modal alignment loss value and the cross-modal alignment coefficient, and determine the intra-class and inter-class loss value in the same way based on the initial intra-class and inter-class loss value and the intra-class and inter-class coefficient, thus completing the dynamic weighting of each loss component.

[0104] In some embodiments, the total loss value is determined by the following formula (13): (13) in, This represents the initial cross-modal alignment loss value. Represents the initial intra-class and inter-class loss values, and represents the classification loss value. Represents the cross-modal alignment coefficient. This represents the coefficients within and between classes.

[0105] In some embodiments, the cross-modal alignment coefficient and the intra-class and inter-class coefficients are determined by the following formula (14): (14) in, It is an activation function.

[0106] Furthermore, in this embodiment, all independent optimization objectives are aggregated into a unified guiding signal. Specifically, step 204.1.5 sums the three dynamically weighted loss components—cross-modal alignment loss, intra-class and inter-class loss, and classification loss—to obtain the total loss value. The total loss value comprehensively reflects the model's overall performance in the current training iteration, from feature alignment and feature space quality to final classification accuracy, and it serves as the basis for adjusting the parameters of the initial working model in the next step.

[0107] It is understood that the embodiments of this application construct a multi-objective, adaptive composite loss calculation framework. By aligning loss values ​​across modalities and intra- and inter-class loss values, it applies refined and structured constraints to the shallow and deep fusion processes within the initial working condition recognition model, ensuring that the initial working condition recognition model can learn high-quality intermediate feature representations, thereby achieving higher recognition accuracy and generalization ability overall.

[0108] Next, we will explain in detail how to determine the cross-modal alignment loss, intra-class and inter-class loss, and classification loss: In some embodiments, such as Figure 11 As shown, Figure 11 yes Figure 10 Step 204.1.1, an optional implementation flowchart, determines the initial cross-modal alignment loss value based on the shallow fusion image features and the shallow fusion temporal features of the samples, including the following steps: B.1 Calculate the similarity between the shallow fusion image features and the shallow fusion temporal features of the samples to obtain the initial shallow fusion similarity; B.2 The initial shallow fusion similarity is amplified based on a preset exponential function to obtain the shallow fusion similarity. The total similarity value is determined based on the shallow fusion similarity, and the similarity ratio is determined based on the ratio of the shallow fusion similarity to the total similarity value. B.3 Take the negative logarithm of the similarity ratio to obtain the initial cross-modal alignment loss value.

[0109] Steps B.1 to B.3 are described in detail below.

[0110] In some embodiments, the initial cross-modal alignment loss value is determined by the following formula (15). This guides the shallow fusion network to align multimodal features. (15) in, Represents the logarithmic function. Indicates the number of samples. This represents a predefined exponential function. Representing shallow fused image features and shallow fusion temporal features The similarity (initial shallow fusion similarity). This represents the total similarity score. This represents the similarity ratio.

[0111] Specifically, by calculating the similarity between the shallow fusion image features and the shallow fusion temporal features of the samples, the closeness of these two feature vectors that have undergone initial interaction in the semantic space is measured. The higher the value, the better the features of the two modalities are aligned after shallow fusion.

[0112] The shallow fusion similarity is obtained by amplifying the initial shallow fusion similarity using a preset exponential function. This operation significantly widens the gap between different similarity scores, making highly matched feature pairs stand out more. The total similarity value is calculated by summing the shallow fusion similarities of the current feature pair with all other possible feature pairs within a calculation batch, thus normalizing the denominator. The similarity ratio is essentially a probability value calculated using the Softmax function, representing the confidence that the current feature pair is correctly matched.

[0113] Furthermore, by taking the negative logarithm of the similarity ratio—a standard mathematical transformation—the initial cross-modal alignment loss value is finally obtained. The negative logarithm function here has a key property: when the input similarity ratio approaches 1 (the initial condition recognition model considers the multimodal features to be aligned), its output loss value approaches 0; conversely, when the ratio approaches 0 (multimodal features are not aligned), the initial cross-modal alignment loss value increases sharply. Thus, a smooth and effective penalty mechanism is constructed to guide the parameter adjustment of the initial condition recognition model.

[0114] It is understood that the embodiments of this application guide the initial working condition recognition model to bring the positive sample pairs (i.e., the matched images and temporal features) closer in the feature space, while pushing the negative sample pairs further apart, so as to drive the shallow fusion module to learn how to eliminate the semantic gap between modalities, and ensure that the output features have a high degree of consistency, thereby fundamentally improving the quality of multimodal data fusion.

[0115] In some embodiments, such as Figure 12 As shown, Figure 12 yes Figure 10 Step 204.1.2, an optional implementation flowchart, determines the initial intra-class and inter-class loss values ​​based on the deep fusion image features and temporal features of the samples, including the following steps: C.1 Obtain the image classification labels of the sample image data and the time-series classification labels of the sample time-series data packets; C.2 Calculate the similarity between the deep fusion image features and the deep fusion temporal features of the samples to obtain the initial deep fusion similarity; C.3 Calculate the similarity between the image classification labels corresponding to the deep fusion image features of the samples and the temporal classification labels corresponding to the deep fusion temporal features of the samples to obtain the label similarity; C.4 Determine the initial intra-class and inter-class loss values ​​based on the difference between label similarity and initial deep fusion similarity.

[0116] Steps C.1 to C.4 are described in detail below.

[0117] In some embodiments, the initial intra-class and inter-class loss values ​​are determined by the following formula (16). The spacing between classes: (16) in, Indicates the number of samples. Indicates the first The image classification label and the first The similarity between time-series classification labels (label similarity). Indicates the first The deep fusion image features of the sample and the first The similarity of temporal features of deep fusion of individual samples (initial deep fusion similarity). This represents the difference between the label similarity and the initial deep fusion similarity.

[0118] Image classification labels are category identifiers assigned to each sample image data to indicate the corresponding rotary kiln operating condition, such as "normal operation," "ring formation," and "kiln lining collapse." They are manually or semi-automatically labeled based on visual features reflected in the image (such as flame shape, temperature distribution, and material filling). Image classification labels can guide the initial operating condition recognition model to learn the correspondence between image content and operating condition categories. Time-series classification labels are category identifiers labeled for each segment of sample time-series data. They also indicate the rotary kiln operating condition type within that time period. Time-series classification labels can be labeled based on the changing patterns of time-series data such as temperature, rotation speed, and pressure collected by sensors. They are used to guide the model to identify the evolution patterns of different operating conditions from dynamic time-series signals.

[0119] Understandably, the initial intra-class and inter-class loss values ​​force the initial operating condition identification model to learn a feature space with a better topological structure: that is, the features of samples from the same type of operating condition are highly clustered in space (enhancing intra-class compactness), while the features of samples from different types of operating conditions are far apart (increasing inter-class separability). This provides the final classifier with a high-quality, well-defined feature representation, thereby greatly reducing the difficulty of classification and significantly improving the robustness and accuracy of rotary kiln operating condition identification.

[0120] In some embodiments, such as Figure 13 As shown, Figure 13 yes Figure 10 Step 204.1.3, an optional implementation flowchart, determines the classification loss value based on the sample condition identification results, including the following steps: D.1 Obtain the actual classification label of the rotary kiln; D.2 Logarithmic processing is performed on the sample working condition identification results to obtain the predicted classification logarithmic value; D.3 Determine the classification loss value based on the product of the true classification label and the predicted classification log value.

[0121] Steps D.1 to D.3 are described in detail below.

[0122] In some embodiments, the classification loss value is determined using the cross-entropy loss function via the following formula (17). : (17) in, Indicates the number of categories. This indicates the true classification label of the rotary kiln. This indicates the result of sample working condition identification. Represents a logarithmic function.

[0123] It is understood that the embodiments of this application determine the final classification loss value by performing logarithmic processing on the probability output of the initial working condition recognition model and using the true classification label as a selector for product operation. In this way, a high-confidence false prediction will generate a huge classification loss value, driving all parameters of the entire initial working condition recognition model to optimize in the sole direction of improving the accuracy of the final working condition recognition.

[0124] like Figure 14 As shown, Figure 14 This is a schematic diagram of an optional device module for a rotary kiln operating condition identification device provided in this application embodiment. The rotary kiln operating condition identification device may include the following modules 301 to 304: The acquisition module 301 is used to acquire target data of the rotary kiln under at least two different modes and input the target data into a pre-trained target working condition recognition model; The shallow fusion module 302 is used to perform feature extraction processing on target data under different modalities through the target working condition recognition model to obtain the coding features corresponding to each modality, and to perform shallow fusion processing on the coding features under different modalities to obtain the shallow fusion features corresponding to each modality. The deep fusion module 303 is used to splice shallow fusion features under different modalities to obtain spliced ​​features, generate modal weights corresponding to each modality based on the spliced ​​features, and perform deep fusion processing on the shallow fusion features under the corresponding modalities based on the modal weights to obtain deep fusion features. The inference module 304 is used to determine the operating condition identification result of the rotary kiln based on the deep fusion features.

[0125] The specific implementation of the rotary kiln condition identification device is basically the same as the specific implementation of the rotary kiln condition identification method described above, and will not be repeated here.

[0126] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned rotary kiln condition identification method. This electronic device can be any smart terminal, including tablet computers, vehicle-mounted computers, etc.

[0127] like Figure 15 As shown, Figure 15 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device includes: The processor 401 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 402 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 402 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 402 and is called and executed by the processor 401 to execute the rotary kiln condition identification method of the embodiments of this application. Input / output interface 403 is used to implement information input and output; The communication interface 404 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 405 transmits information between various components of the device (e.g., processor 401, memory 402, input / output interface 403, and communication interface 404); The processor 401, memory 402, input / output interface 403 and communication interface 404 are connected to each other within the device via bus 405.

[0128] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described rotary kiln condition identification method.

[0129] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0130] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0131] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0132] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0133] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0134] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0135] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0136] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0137] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0138] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0139] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0140] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for identifying the operating conditions of a rotary kiln, characterized in that, include: Obtain target data of the rotary kiln under at least two different modes, and input the target data into a pre-trained target working condition recognition model; The target working condition identification model is used to extract features from the target data under different modalities to obtain the coded features corresponding to each modality. The coded features under different modalities are then fused to obtain the shallow fused features corresponding to each modality. By splicing together the shallow fusion features under different modalities, spliced ​​features are obtained. Modal weights corresponding to each modality are generated based on the spliced ​​features. Based on the modal weights, the shallow fusion features under the corresponding modalities are subjected to deep fusion processing to obtain deep fusion features. The operating condition identification result of the rotary kiln is determined based on the deep fusion features.

2. The rotary kiln condition identification method according to claim 1, characterized in that, The target data under different modalities includes image data under the visual modality and time-series data under the signal modality. The target condition recognition model includes an image encoder, a time-series encoder, and a first linear layer. The step of performing feature extraction processing on the target data under different modalities to obtain the encoded features corresponding to each modality includes: The image data is processed by the image encoder to obtain the image coding features under the visual modality, wherein the image coding features include the global features and local features of the rotary kiln; The timing encoder is used to perform feature extraction processing on the timing data to obtain the initial timing coding features under the signal mode. Based on the first linear layer, the initial timing coding features are subjected to feature dimension alignment processing to obtain the timing coding features under the signal mode that are aligned with the feature dimensions of the image coding features.

3. The rotary kiln operating condition identification method according to claim 2, characterized in that, The target working condition recognition model includes multiple cascaded shallow fusion modules, and the shallow fusion features include shallow fusion image features in the visual modality and shallow fusion temporal features in the signal modality. The step of performing shallow fusion processing on the encoded features under different modalities to obtain shallow fused features corresponding to each modality includes: The first and second coding features are input into the shallow fusion module, and the shallow fusion module performs interactive fusion processing on the first and second coding features to output the initial fused image features and the initial fused temporal features, respectively. The initial fused image features are used as the first encoding features of the next shallow fusion module, and the initial fused temporal features are used as the second encoding features of the next shallow fusion module. The iterative operation of the interactive fusion process is then performed, wherein the initial value of the first encoding feature is the image encoding feature, and the initial value of the second encoding feature is the temporal encoding feature. The initial fused image features output by the last shallow fusion module are used as the shallow fused image features, and the initial fused temporal features output by the last shallow fusion module are used as the shallow fused temporal features.

4. The rotary kiln condition identification method according to claim 3, characterized in that, Each of the shallow fusion modules includes a first fusion module and a second fusion module. The first fusion module includes a first linear layer unit and a first multilayer perceptron. The second fusion module includes a second linear layer unit and a second multilayer perceptron. The step of using the shallow fusion module to perform interactive fusion processing on the first coded features and the second coded features to output initial fused image features and initial fused temporal features respectively includes: The first encoded feature is input into the first fusion module, and the feature is linearly processed based on the first linear layer unit to obtain the enhanced image encoded feature. The second coding feature is input into the second fusion module, and the feature is linearly processed based on the second linear layer unit to obtain the enhanced temporal coding feature. A first residual connection result is generated based on the first coding feature and the enhanced temporal coding feature, and a shallow fusion process is performed on the first residual connection result based on the first multilayer perceptron to obtain the initial fused image features; A second residual connection result is generated based on the second coding feature and the enhanced image coding feature, and a shallow fusion process is performed on the second residual connection result based on the second multilayer perceptron to obtain the initial fusion temporal feature.

5. The rotary kiln condition identification method according to claim 3, characterized in that, The target working condition recognition model further includes a second linear layer, and the modal weights include visual modal weights under the visual modality and signal modal weights under the signal modality; The process of performing deep fusion processing on the shallow fusion features under the corresponding modality based on the modality weights to obtain deep fusion features includes: Based on the visual modality weights and the shallow fusion image features, the initial deep fusion image features are determined, and based on the signal modality weights and the shallow fusion temporal features, the initial deep fusion temporal features are determined. The deep fusion features are obtained by superimposing the initial deep fusion image features and the initial deep fusion temporal features.

6. The rotary kiln condition identification method according to claim 1, characterized in that, The target working condition identification model is trained through the following steps: Obtain sample data of the rotary kiln under at least two different modes, and input the sample data into the initial working condition identification model; Using the initial working condition identification model, feature extraction processing is performed on the sample data under different modalities to obtain sample coding features corresponding to each modality. Then, shallow fusion processing is performed on the sample coding features under different modalities to obtain sample shallow fusion features corresponding to each modality. The shallow fusion features of samples under different modalities are spliced ​​together to obtain sample splicing features. Sample modality weights corresponding to each modality are generated based on the sample splicing features. Based on the sample modality weights, the shallow fusion features of samples under the corresponding modalities are subjected to deep fusion processing to obtain deep fusion features of samples. The sample operating condition identification result of the rotary kiln is determined based on the deep fusion features of the sample. The total loss value is determined based on the shallow fusion features of the sample under different modalities, the deep fusion features of the sample, and the sample operating condition identification result. The model parameters of the initial operating condition identification model are adjusted according to the total loss value to obtain the trained target operating condition identification model.

7. The rotary kiln condition identification method according to claim 6, characterized in that, The shallow fusion features of the samples include shallow fusion image features of the samples in the visual modality and shallow fusion temporal features of the samples in the signal modality. The deep fusion features of the samples are determined by the deep fusion image features of the samples in the visual modality and the deep fusion temporal features of the samples in the signal modality. The determination of the total loss value based on the shallow fusion features of the samples under different modalities, the deep fusion features of the samples, and the sample condition identification results includes: Based on the shallow fusion image features and the shallow fusion temporal features of the samples, the initial cross-modal alignment loss value is determined; Based on the deep fusion image features and temporal features of the samples, the initial intra-class and inter-class loss values ​​are determined; Based on the aforementioned sample condition identification results, the classification loss value is determined; Based on the initial cross-modal alignment loss value and the initial intra-class and inter-class loss value, determine the cross-modal alignment coefficient and the intra-class and inter-class coefficient, and determine the cross-modal alignment loss value based on the initial cross-modal alignment loss value and the cross-modal alignment coefficient, and determine the intra-class and inter-class loss value based on the initial intra-class and inter-class loss value and the intra-class and inter-class coefficient; The total loss value is obtained by superimposing the cross-modal alignment loss value, the intra-class and inter-class loss value, and the classification loss value.

8. The rotary kiln condition identification method according to claim 7, characterized in that, The step of determining the initial cross-modal alignment loss value based on the shallow fusion image features and the shallow fusion temporal features of the samples includes: Calculate the similarity between the shallow fusion image features of the samples and the shallow fusion temporal features of the samples to obtain the initial shallow fusion similarity; The initial shallow fusion similarity is amplified by a preset exponential function to obtain a shallow fusion similarity. The total similarity value is determined based on the shallow fusion similarity. The similarity ratio is determined based on the ratio of the shallow fusion similarity to the total similarity value. The initial cross-modal alignment loss value is obtained by taking the negative logarithm of the similarity ratio.

9. The rotary kiln condition identification method according to claim 7, characterized in that, The sample data includes sample image data and sample time series data; The step of determining the initial intra-class and inter-class loss values ​​based on the deep fusion image features and temporal features of the samples includes: Obtain the image classification labels of the sample image data and the time-series classification labels of the sample time-series data packets; Calculate the similarity between the deep fusion image features of the sample and the deep fusion temporal features of the sample to obtain the initial deep fusion similarity; The similarity between the image classification label corresponding to the deep fusion image features of the sample and the temporal classification label corresponding to the deep fusion temporal features of the sample is calculated to obtain the label similarity. The initial intra-class and inter-class loss values ​​are determined based on the difference between the label similarity and the initial deep fusion similarity.

10. The rotary kiln condition identification method according to claim 7, characterized in that, The determination of the classification loss value based on the sample working condition identification result includes: Obtain the actual classification label of the rotary kiln; Logarithmic processing is performed on the sample working condition identification results to obtain the predicted classification logarithmic value; The classification loss value is determined by multiplying the true classification label and the predicted classification logarithm.

11. A rotary kiln operating condition identification device, characterized in that, include: The acquisition module is used to acquire target data of the rotary kiln under at least two different modes and input the target data into a pre-trained target working condition recognition model; The shallow fusion module is used to perform feature extraction processing on the target data under different modalities through the target working condition recognition model to obtain the encoded features corresponding to each modality, and to perform shallow fusion processing on the encoded features under different modalities to obtain the shallow fusion features corresponding to each modality. The deep fusion module is used to splice the shallow fusion features under different modalities to obtain spliced ​​features, generate modal weights corresponding to each modal based on the spliced ​​features, and perform deep fusion processing on the shallow fusion features under the corresponding modal based on the modal weights to obtain deep fusion features; The inference module is used to determine the operating condition identification result of the rotary kiln based on the deep fusion features.

12. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the rotary kiln condition identification method according to any one of claims 1 to 10.

13. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the rotary kiln condition identification method according to any one of claims 1 to 10.

Images