Sintered ore quality multi-modal analysis method

Abstract

The invention relates to a sinter quality multi-modal analysis method. The sinter quality multi-modal analysis method comprises the following steps: acquiring various modal data in a production process; carrying out multi-modal data processing; performing feature representation on the single-mode data; carrying out multi-modal feature fusion; and carrying out multi-modal feature analysis. The method has the advantages that a network used by a deep learning model in multi-modal feature analysis is a network which is trained by collecting a data cluster obtained by processing multi-modal data, performing feature representation and multi-modal feature fusion on single-modal data and using a deep learning framework, and the used deep learning framework can be an RNN (Recurrent Neural Network) model; the network adapts to sintering multi-mode data characteristics, and the method is simple, practical and easy to popularize. Various single-modal feature representations are converted into to-be-fused feature vectors, and a plurality of different to-be-fused feature vectors are subjected to specific mapping to obtain new feature vectors, so that each modal data dimension is effectively balanced, and meanwhile, the framework has combination conditions of various fusion strategies.

Description

Technical Field

[0001] The invention relates to the field of sintered ore quality prediction, and in particular to a sintered ore quality multimodal analysis method. Background Art

[0002] The performance of sintered ore directly affects the stability of the ironmaking system, the quality of molten iron, and energy consumption. Due to the long sintering process, complex structure, and frequent changes in production conditions, there are many factors that affect performance analysis. In addition, although many scholars and experts have established many effective models and system software with the help of machine vision, most of them use single-mode input for analysis, only images or only process parameter values ​​for modeling, which undoubtedly ignores the influence of production factors and process mechanism models. Summary of the invention

[0003] The purpose of the present invention is to provide a multimodal analysis method for sintered ore quality, which integrates multi-domain knowledge of production process data and image data to obtain sintered ore quality index results.

[0004] To achieve the above object, the present invention is implemented through the following technical solutions:

[0005] A multimodal analysis method for sinter ore quality comprehensively analyzes a multimodal data set consisting of production process data and image data to obtain sinter ore quality index results, specifically comprising the following steps:

[0006] S1. Obtain multiple modal data in the production process;

[0007] S2, multimodal data processing;

[0008] S3, feature representation of unimodal data;

[0009] S4, multimodal feature fusion;

[0010] S5. Multimodal feature analysis.

[0011] In S1, multiple modal data in the production process are obtained, including obtaining numerical data and obtaining image data sets. The numerical data are obtained by obtaining production process parameters that are highly correlated with the sintered ore quality indicators. The image data sets are obtained by directly acquiring images from a single camera, or images acquired by multiple cameras and then stitched together, or images that have undergone image preprocessing.

[0012] In S2, multimodal data processing is to synchronize the acquired multiple modal data in time and space to form a data cluster to be analyzed.

[0013] In S3, feature representation of unimodal data is to represent the original data of different modalities as numerical vectors that can be processed by computers or further abstracted into higher-level feature vectors.

[0014] In S4, multimodal feature fusion is to first convert various single-modal features into high-dimensional vectors of the same dimension, then multiply them element by element, and then perform pooling to obtain the feature vector to be fused. m ) After a specific mapping F, a new feature vector is obtained, the formula is as follows:

[0015] Z=F(X1,X2,…,X m ) ①

[0016] In formula ①, F represents a specific mapping, Z represents a new feature vector, and X m Represents the feature vector to be fused.

[0017] In S5, multimodal feature analysis is performed by deep learning on the fused feature data to obtain the sintered ore quality index results.

[0018] The data cluster to be analyzed is the sintered ore on the sintering trolley. Every n meters is a reference segment. The real-time detection data of this segment of sintered ore at different process positions constitute a data cluster to be analyzed. The instant data in the collection period falling within the n-meter segment is encapsulated to form a data cluster to be analyzed. The data contained in each data cluster to be analyzed is the corresponding instant data of a segment of sintered ore running to each detection point.

[0019] The feature representation of numerical data is to transform the numerical data into M*N matrix data consisting of M parameters and N columns of specific values, and form feature vectors through a deep neural network; for unimodal data of image type, the image data set is input into the network to obtain a set of feature vectors, and the feature extraction method is convolutional neural network (CNN) or multi-layer perceptron (MLP).

[0020] Different quality indicators correspond to different process data, or they may correspond to different process data, but the corresponding parameters with lower correlation have lower corresponding coefficient weights during calculation. For parameters with poor correlation in deep learning, they will be automatically weakened in the network calculation after training. To obtain production process parameters with higher correlation with sintered ore quality indicators, correlation analysis can be performed through XGBoost, Pearson, and Spearman correlation methods.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] 1. The network used in the deep learning model in the multimodal feature analysis is a data cluster obtained by collecting multimodal data processing, and then performing feature representation and multimodal feature fusion on the single modal data, and then using the network trained by the deep learning framework. The deep learning framework used can be an RNN model. The network is adapted to the characteristics of the sintering multimodal data of the present invention. The method is simple, practical, and easy to promote.

[0023] 2. Acquire multiple modal data in the production process, including numerical data and image data sets. Acquire numerical data is to obtain production process parameters with high correlation with sinter quality indicators. Acquire image data sets are images directly acquired by a single camera, or images acquired by multiple cameras and stitched together, or images after image preprocessing. The analysis parameters are more comprehensive, including real-time production parameters and machine tail end face image information, temperature field information and other multi-modal data;

[0024] 3. Multimodal data processing is to synchronize the acquired multiple modal data in time and space to form a data cluster to be analyzed, so that the multimodal data of the sintered ore and the production process can correspond more accurately in time and space. In the past, it only corresponded to the production data at a certain moment, rather than the corresponding data of the physical object at different locations and times;

[0025] 4. Feature representation of single-modal data is to represent the original data of different modalities as numerical vectors that can be processed by computers or further abstracted into higher-level feature vectors to facilitate subsequent feature fusion;

[0026] 5. Convert the feature representations of various single modalities into feature vectors to be fused, and obtain new feature vectors by specific mapping of multiple different feature vectors to be fused, effectively balance the dimensions of each modal data, and make the framework have the combination conditions of multiple fusion strategies;

[0027] 6. Multimodal feature analysis is to conduct deep learning on the fused feature data, that is, to apply the fused multimodal features and use the deep learning framework to predict the quality indicators of sintered ore. The parameters of sintering deep learning application are not conventional production process data or a single image, but multimodal data;

[0028] 7. The present invention integrates knowledge from multiple fields, makes full use of various information in the sintering process, expands the amount of available information, and realizes data fusion;

[0029] 8. It integrates multi-domain knowledge of production process data and image data to obtain sintered ore quality index results, such as FeO content, conversion intensity, etc. Different quality indicators may correspond to different process data, or they may be the same but with lower corresponding correlation parameters, and the corresponding coefficient weight is lower during calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flow chart of the multimodal analysis method of sinter quality. DETAILED DESCRIPTION

[0031] The present invention is described in detail below in conjunction with the accompanying drawings, but it should be noted that the implementation of the present invention is not limited to the following embodiments.

[0032] The following examples are implemented on the premise of the technical solution of the present invention, and provide detailed implementation methods and specific operation processes, but the protection scope of the present invention is not limited to the following examples. The methods used in the following examples are conventional methods unless otherwise specified.

[0033] [Example 1]

[0034] See Figure 1 , a multimodal data set consisting of production process data and image data is comprehensively analyzed to predict the quality indicators of sintered ore. The steps are as follows:

[0035] S1. Obtaining multiple modal data in the production process

[0036] Two types of modal data are mainly obtained: one is numerical data, and the other is image data. In the sintering ore production process, the production process parameters with high correlation with the sintering ore quality indicators are mainly obtained, including three types of process parameter types: raw material data, characterization operation parameters, and characterization state parameters. The correlation analysis method adopts the Person correlation coefficient characteristic analysis method to provide a set of available parameter combinations, including ignition temperature, pressure ratio, trolley speed, material layer thickness, main pipe negative pressure, main pipe exhaust gas temperature, No. 18 bellows exhaust gas temperature, No. 19 bellows exhaust gas temperature, No. 20 bellows exhaust gas temperature, No. 21 bellows exhaust gas temperature; to obtain image type data, images directly collected by a single camera, or images collected by multiple cameras and stitched together, or images after image preprocessing can all be used to construct an image data set to form a modal data of the image type. The image data can be a tail section image or a section thermal image.

[0037] S2. Multimodal Data Processing

[0038] Various numerical data and image data are synchronized in time and space to form data clusters to be analyzed; the method for generating data clusters to be analyzed is to use each n-meter segment of the sinter trolley as a reference period, and encapsulate the real-time data of each detection point collection period falling in the n-meter segment to form a data cluster to be analyzed, that is, for each data cluster to be analyzed, the data contained in it is the corresponding real-time data of a section of sinter ore running to each detection point, and each type of production process data may be one or more; when collating data at runtime, the 2-meter segment of the sinter trolley can be used as a reference period, and the real-time data of each detection point collection period falling in the 2-meter segment can be encapsulated to form a three-dimensional data cluster.

[0039] S3. Feature representation of unimodal data

[0040] Represent the raw data of different modalities as computer-processable numerical vectors or further abstract them into higher-level feature vectors;

[0041] The feature representation method for numerical data is to first convert the numerical data into M*N matrix data consisting of M parameters and N columns of specific values, and use a two-layer perceptron (MLP) based on residual connection as the basic module. If the input layer is represented by vector X, the two-layer perceptron can be expressed as:

[0042] f(X)=W2(activation(W1X+b1))+b2 ①

[0043] In formula ①, W1 and b1, W2 and b2 are the weights and offsets to be learned, activation is a nonlinear activation function, usually a Relu function, and its expression is g(x) = max(0,x);

[0044] Connecting the above layer perceptrons using a residual structure, the output layer can be further expressed as:

[0045] Y=f(X)+X=W2(activation(W1X+b1))+b2+X ②

[0046] In formula ②, Y is the output layer and X is the input layer;

[0047] The basic module consisting of two layers of perceptrons connected by a residual structure is stacked, the number of stacks is determined, the network depth is increased, and a nonlinear feature vector is obtained;

[0048] The single-modal data of image type is a data set composed of images, including organic tail cross-sectional images and cross-sectional infrared thermal images. To construct the image data set, a convolutional neural network composed of convolutional layers and pooling layers is generally adopted. In order to increase the network depth without reducing the network's expressive ability, a residual structure is used to connect the network layers.

[0049] S4. Multimodal feature fusion

[0050] This step requires that the feature representations of various single modalities be obtained according to a certain rule to obtain the feature vector to be fused. By integrating different types of features, the performance of the machine learning model is improved to make up for the heterogeneity of different modalities. m ) After a specific mapping F, a new eigenvector Z is obtained, the formula is as follows:

[0051] Z=F(X1,X2,…,X m ) ③

[0052] In formula ③, F represents a specific mapping, Z represents a new feature vector, and X m Represents the feature vector to be fused;

[0053] Taking into account the differences between feature data information of different modalities, in order to effectively utilize the features of each specific modality and realize information complementarity between modalities, the fusion is performed by multiplying tensors by Cartesian products. First, each modality is dimensionally filled with 1, and the Cartesian product of different modalities is calculated to obtain the fused result. This calculation method can not only describe the correlation between features, but also retain the inherent information of specific modalities. The steps can be expressed as follows:

[0054]

[0055] In formula ④, h m is the fused feature vector, h x is the feature vector of the production process data modality to be fused, h y is the feature vector of the tail end face image modality to be fused, h z is the modal eigenvector of the tail temperature field data to be fused.

[0056] S5. Multimodal feature analysis

[0057] By further deep learning the fused feature data, relevant sintered ore quality index results are obtained, wherein the network used by the deep learning model is a network trained by collecting the data clusters obtained in step S2, and then through steps S3 and S4, and then using the RNN deep learning framework.

[0058] In conventional deep learning methods, the input layer is the fused feature data, and the output layer is the sintered ore quality index.

[0059] The network used in the deep learning model in the multimodal feature analysis of the present invention is a data cluster obtained by collecting multimodal data processing, and then performing feature representation and multimodal feature fusion on single-modal data, and then using a deep learning framework to train the network. The deep learning framework used can be an RNN model. The network is adapted to the characteristics of the sintering multimodal data of the present invention. The method is simple, practical and easy to promote. Acquiring multiple modal data in the production process includes acquiring numerical data and acquiring image data sets. Acquiring numerical data is to acquire production process parameters with high correlation with sintered ore quality indicators. Acquiring image data sets is an image directly collected by a single camera, or an image after splicing of images collected by multiple cameras, or an image after image preprocessing. The analysis parameters are more comprehensive, including real-time production parameters and tail end face image information, temperature field information and other multimodal data; multimodal data processing is to synchronize the obtained multiple modal data in time and space to form a data cluster to be analyzed, so that the sintered ore object and the multimodal data of the production process correspond more accurately in time and space. In the past, it only corresponded to the production data at a certain moment, rather than the corresponding data of the object at different positions and at different times; The feature representation of single-modal data is to represent the original data of different modes as numerical vectors that can be processed by computers or further abstracted into higher-level feature vectors, so as to facilitate the subsequent feature fusion; the feature representations of various single modes are converted into feature vectors to be fused, and multiple different feature vectors to be fused are subjected to specific mapping to obtain new feature vectors, effectively balancing the dimensions of each modal data, and at the same time making the framework have the combination conditions of multiple fusion strategies; multimodal feature analysis is to perform deep learning on the fused feature data, that is, to apply the fused multimodal features and use a deep learning framework to realize the prediction of sinter quality indicators. The parameters of sinter deep learning application are not conventional production process data or a single image, but multimodal data; the present invention integrates multi-domain knowledge, makes full use of various information in the sinter process, expands the amount of available information, and realizes data fusion; the multi-domain knowledge of production process data and image data is integrated to obtain the sinter quality indicator results, such as FeO content, conversion intensity, etc. Different quality indicators may correspond to different process data, or they may be the same but with lower corresponding correlation parameters, and the corresponding coefficient weights are lower during calculation.

Claims

1. A multimodal analysis method for sintered ore quality, characterized in that: Comprehensively analyze the multimodal data set consisting of production process data and image data to obtain the sinter quality index results, which specifically includes the following steps: S1. Obtain multiple modal data in the production process; S2, multimodal data processing; S3, feature representation of unimodal data; S4, multimodal feature fusion; S5. Multimodal feature analysis.

2. A multimodal analysis method for sintered ore quality according to claim 1, characterized in that: In S1, the acquisition of multiple modal data in the production process includes acquiring numerical data and acquiring image data sets. Acquiring numerical data is to acquire production process parameters that are highly correlated with sintered ore quality indicators. Acquiring image data sets is to acquire images directly acquired by a single camera, or images acquired by multiple cameras and then stitched together, or images after image preprocessing.

3. A multimodal analysis method for sintered ore quality according to claim 1, characterized in that: In S2, the multimodal data processing is to synchronize the obtained multiple modal data in time and space to form a data cluster to be analyzed.

4. The multimodal analysis method for sintered ore quality according to claim 1, characterized in that: In S3, the feature representation of the single-modal data is to represent the original data of different modalities as numerical vectors that can be processed by computers or further abstract them into higher-level feature vectors.

5. The multimodal analysis method for sintered ore quality according to claim 1, characterized in that: In S4, the multimodal feature fusion is to first convert various single-modal features into high-dimensional vectors of the same dimension, then multiply them element by element, and then perform pooling to obtain the feature vector to be fused, and then combine multiple different feature vectors to be fused (X1, X2, ..., X m ) After a specific mapping F, a new feature vector is obtained, the formula is as follows: Z=F(X1,X2,…,X m ) In formula ①, F represents a specific mapping, Z represents a new feature vector, and X m Represents the feature vector to be fused.

6. A multimodal analysis method for sintered ore quality according to claim 1, characterized in that: In S5, the multimodal feature analysis is performed by deep learning the fused feature data to obtain the sintered ore quality index results.

7. A multimodal analysis method for sintered ore quality according to claim 3, characterized in that: The data cluster to be analyzed is the sintered ore on the sintering trolley. Every n meters is a reference segment. The real-time detection data of this segment of sintered ore at different process positions constitute a data cluster to be analyzed. The instant data in the collection period falling within the n-meter segment is packaged to form a data cluster to be analyzed. The data contained in each data cluster to be analyzed is the corresponding instant data of a segment of sintered ore running to each detection point.

8. A multimodal analysis method for sintered ore quality according to claim 4, characterized in that: The feature representation of numerical data is to transform the numerical data into M*N matrix data consisting of M parameters and N columns of specific numerical values, and form a feature vector through a deep neural network; for unimodal data of image type, the image data set is input into the network to obtain a set of feature vectors.

Images