Autoformer sinter FeO content soft sensing method based on multi-source information fusion

Abstract

The present invention discloses an Autoformer sintered ore FeO content soft measurement method based on multi-source information fusion. First, in view of the complex characteristics of sintering process data such as dynamics, strong nonlinearity and multi-source heterogeneity, a multi-source information fusion strategy is proposed to realize feature-level information fusion and data-level information fusion, which are used for image feature extraction of sintered ore cross-section and multi-source heterogeneous data integration, respectively. Subsequently, more comprehensive sintering process information is input into the downstream Autoformer model in a serial manner. The above-mentioned serial connection not only improves the information capacity, but also provides prior information about the FeO content level of the sintered ore, which is conducive to the Autoformer capturing the complex time series distribution in the sintering process, so that the Autoformer model based on multi-source information fusion has more accurate modeling capabilities in real industrial scenarios.

Description

Technical Field

[0001] The invention belongs to a soft-sensing modeling method for quality indicators of a sintering process, and in particular relates to an Autoformer sintered ore FeO content soft-sensing model based on a multi-source information fusion strategy. Background Art

[0002] The stability of the sintering process state and the quality of sintered ore directly affect the blast furnace condition, product quality and production energy consumption. The FeO content of sintered ore is an important quality indicator for evaluating sintered ore production. In the actual production process, the quality detection of sintered ore is generally completed by sampling and testing. This process takes too long and has significant hysteresis. In view of this situation, experienced operators will estimate the FeO content of sintered ore based on the combustion state of the sintered ore section at the tail of the sintering machine. If the product quality of the sintered ore does not meet the process requirements, the operator will provide corresponding process parameter adjustments based on experience. This simple control mechanism relies too much on experience and has low reliability, which seriously affects the production efficiency and stability of sintered ore quality. Therefore, by combining the easily measurable parameters in the sintering process (such as raw material ratio, operating parameters, equipment parameters and sintering machine state parameters, etc.), a certain mathematical model is established to obtain the functional relationship between easy-to-measure variables and variables that are difficult to obtain in real time (such as sintered ore quality indicators), so as to complete effective and real-time soft measurement of the FeO content of sintered ore, which is of great significance for improving the stability of the sintering process.

[0003] Sintering production is accompanied by a series of complex physical and chemical reactions. The sintering process data has obvious time lag, dynamics and nonlinearity. These data are generally obtained by sensors and exist in numerical form. However, in addition to numerical data, the cross-section of the sintered ore at the tail of the sintering machine directly reflects the combustion state of the sintered ore, and reflects the changes in the quality of the sintered ore in a more intuitive way. In the industrial site, the cross-section information of the sintered ore at the tail of the sintering machine is usually collected by a visible light camera and an infrared thermal imager, and presented in the form of visible light images or temperature distribution. Therefore, the multi-source nature of the data is also a typical feature of the sintering process. How to effectively combine multi-source heterogeneous data to complete the soft measurement of the FeO content of the sintered ore has become a current research hotspot and difficulty. Summary of the invention

[0004] Aiming at the problem that complex multi-source heterogeneous data are difficult to be effectively used for real-time measurement of sinter quality indicators, the present invention proposes an Autoformer sinter FeO content soft measurement model based on a multi-source information fusion strategy. The multi-source information fusion strategy is used to embed the cross-sectional image information of the sinter into the time series to achieve feature-level information fusion and data-level information fusion. The multi-source information fusion provides a comprehensive sintering process description and prior information on the FeO content level for the downstream Autoformer. The overall construction of the model is achieved based on a serial structure, which is conducive to capturing the complex time series distribution in the sintering process and providing accurate soft measurement modeling results.

[0005] The present invention is implemented by the following technical solutions:

[0006] Step 1: Offline modeling;

[0007] Step 1.1: For the same batch of raw materials in the sintering production process, the sampling time of the finished sintered ore is used as the benchmark, and the sintering process data is sampled and traced according to the sintering machine structure, trolley operation status and material conveyor belt, and the sensor data of each part of the entire sintering process and the infrared image data of the sintered ore cross section are collected.

[0008] Step 1.2: Combining expert knowledge and data-driven SE-ResNet classification model, shallow and deep features of the sinter section are extracted respectively to achieve image feature-level information fusion;

[0009] During the sintering process, experts always pay attention to the most basic and important factors that reflect the production quality, so as to estimate the sinter quality indicators in real time and maintain stable production through different control operations. At the tail of the sintering machine, the cross-sectional temperature of the sinter (usually considered by workers as brightness) and the thickness of the sinter red layer are the key features that directly reflect the combustion state. Sintering consistency represents the vertical combustion rate, which is mainly related to the permeability and greatly affects the FeO content of the sinter. At the same time, since the vertical sintering rate determines the end point of combustion, namely the burn-through point (BTP), the overall state of BTP also plays an important role in the evaluation of sinter performance. In general, four shallow characteristics, namely the cross-sectional temperature of the sinter, the thickness of the sinter red layer, the combustion consistency and the overall state of BTP, are extracted according to expert knowledge, which are Temp, Thick, C and S respectively. BTP Taking the infrared thermal imaging image of a sintered ore cross section with a height of h and a width of w as an example, the above characteristics are specifically expressed as follows:

[0010]

[0011]

[0012]

[0013]

[0014] Among them, t ij represents the temperature at the coordinate (i, j) of the infrared thermal imaging image of the sintered ore section, p j is the height of the red layer in the jth column pixel, m j Indicates the height of the highest temperature point in the jth column of pixels;

[0015] Subsequently, the SE-ResNet model is used to classify the FeO content level of the sinter section, and deep feature extraction is performed based on the hierarchical convolutional features learned by the network. ResNet is an efficient deep learning framework that alleviates the difficulty of training deep networks through residual learning modules. Assume that the input of the residual learning module is X, and the expected underlying mapping U(X) is defined as:

[0016] U(X)=F(X)+X

[0017] In the formula, F(X) represents the nonlinear residual mapping layer used for superposition, X is connected to the residual learning module in a quick connection manner, and U(X) is obtained by element-wise addition with F(X). Generally speaking, at least two weight layers are set in F(X) to ensure the effectiveness of the residual module. Taking two weight layers as an example, F(X) can be specifically expressed as:

[0018] F(X)=W2σ(W1X)

[0019] Among them, W1 and W2 are the weight matrices of the first weight layer and the second weight layer respectively, and σ represents the implementation of a nonlinear activation function (such as the ReLU function). Based on the above residual learning module, the two weight layers are replaced by convolutional layers to obtain a convolutional residual module. By superimposing multiple residual convolutional modules, the ResNet model can be built. Since conventional convolution operations are difficult to distinguish the importance of feature channels, the SE module is used here to explore the dependencies between channels and realize the recalibration and selection of feature channels. Assume that the conventional residual convolution module can be expressed as F cov :X→Y, X and Y are the input and output before and after the convolution transformation, respectively. H, W, C and H′, W′, C′ represent the height, width and number of channels of the input and output images, respectively. Y = [y1, y2, ..., y C′ ] represents the specific C′ sub-channel output. The SE module consists of three steps: compression, excitation, and weighting. First, global average pooling is performed on each channel of Y, with the fth channel y f For example, the compression transformation F sq It can be expressed as:

[0020]

[0021] Among them, y f (i, j) represents channel y f The value at the coordinate (i, j) in the middle, sf is the output after the compression transformation of the fth channel. Then the excitation transformation F is performed ex Specifically, the sigmoid activation gating mechanism is introduced to achieve adaptive calibration of channel weights:

[0022] w=F ex (s, W) = δ(g(s, W)) = δ(W FC，2 σ(W Fc，1 s))

[0023] Where w = [w1, w2, ..., w C ′] is the weight set of C′ channels, δ is the sigmoid activation function, σ is the ReLU function, g(s, W) represents a gating structure consisting of two fully connected layers (FC) and a ReLU, s = [s1, s2, ..., s C′ ] indicates F sq C′ outputs, and is the weight matrix of the two FC layers. In this gated structure, the two FC layers reduce (C′→C′ / r) and increase (C′ / r→C′) the channel dimension before and after the ReLU function, respectively, so that the excitation transformation limits the model complexity while capturing the nonlinear dependencies between channels. The parameter r is a hyperparameter that controls the model complexity. Finally, based on the weighted transformation function F we , scale the output Y to get the weighted output For single-channel calculations, F we The calculation can be expressed as:

[0024]

[0025] where w f is the weight of the fth channel, and the symbol · represents the scalar w f With single channel data The product of express Based on the above steps, the SE-ResNet module can be built, and the SE-ResNet model can be built based on the stacking of SE-ResNet modules;

[0026] Based on the infrared thermal imaging image of the cross section of the sintered ore at the tail of the sintering machine, the SE-ResNet network is built to realize the classification of the FeO content of the sintered ore; after the model training is completed, the feature channels with significant weights are selected based on the weights between the layered convolution features learned by the network and the feature channels in each layer. Subsequently, based on the image morphological operation, the noise interference is eliminated, the detail area is highlighted, and the area ratio and temperature of the detail area are calculated respectively to realize the selection of image depth features;

[0027] Based on the above-mentioned image feature extraction, the shallow features and deep features are quantified respectively, and the image feature-level information fusion is completed in the form of variables; then, by calculating the time difference between image acquisition and sensor data sampling, the data-level information fusion of image features and sensor values ​​is realized.

[0028] Step 1.3: Based on step 1.1 and step 1.2, the multi-source fusion information is serially connected with the downstream Autoformer model. The overall implementation process of Autoformer can be composed of N Encoder layers and M Decoder layers; Encoder and Decoder modules are connected end to end within the layer, and the output of Encoder is connected between layers within each Decoder layer. For an input sequence of length L and dimension d The implementation steps of Autoformer are as follows:

[0029] By solving the first seasonal component - Solve for and the second seasonal component _Build the Encoder layer:

[0030]

[0031]

[0032] Among them, AutoCorrelation, FeedForward and SeriesDecomp represent the autocorrelation module, the forward propagation module and the trend decomposition module respectively. represents the output of the l-1th Encoder layer, Represents the initial input of the Encoder layer, and They represent the two seasonal components output by the SeriesDecomp module in the lth Encoder layer, and "_" is the omitted trend component. Repeating the above formula N times can realize the Encoder layer stacking;

[0033] For the Decoder layer, the initialization steps of the module input are as follows:

[0034] Xdes，p：L , X det，p：L = SeriesDecomp(X de，p：L )

[0035] X des =Concat(X des，p：L , X0)

[0036] X det =Concat(X det，p：L , X mean )

[0037] in, represents the overall input of the Decoder layer with a length of L and a dimension of d, X de，p：L Provides historical information from time step p to L, Respectively represent X de，p：L The seasonal and trend components of X0 and X mean Represents 0 and X de The mean of des，p：L and X det，p：L At the end of the decoder layer, the seasonal component input X is constructed des and trend component input X det ; Then, the first, second, and third pairs of seasonal components and trend components of the Decoder layer are calculated in turn:

[0038]

[0039]

[0040]

[0041] In the above formula, represents the output of the l-1th Decoder layer, and is the latent variable output by the Encoder part, Represents the i-th pair of seasonal component and trend component in the l-th Decoder layer; the above trend components are accumulated to obtain the overall trend component of the l-th Decoder layer:

[0042]

[0043] Among them, W l，i , i∈{1, 2, 3} represents the i-th trend component in the l-th Decoder layer The projection matrix of Repeating the above formula M times can achieve Decoder layer stacking; finally, the prediction is completed by the following formula:

[0044]

[0045] Here the Mth Decoder layer seasonal component With the projection matrix W M After multiplication, the dimension is changed and then the trend component of the Mth Decoder layer is Add them together to get the final predicted value;

[0046] The above trend decomposition module uses moving smoothing to decompose the sequence into seasonal components and long-term trend terms. For an input sequence with a length of L and a dimension of d For SeriesDecomp, it can be expressed as:

[0047] X t =AvgPool(Padding(X))

[0048] X s =XX t

[0049] in, represent the seasonal component and the trend component respectively;

[0050] The above autocorrelation module implements the autocorrelation analysis at the subsequence level. The specific execution steps are as follows:

[0051] For an input sequence of length L and dimension d Perform linear mapping to obtain the corresponding matrices Q, K, and V:

[0052] Q=W Q X

[0053] K=W K X

[0054] V=W V X

[0055] Then, the autocorrelation of the subsequences is calculated:

[0056]

[0057] Among them, R Q，K (τ) represents Q t With the sequence K delayed by τ t-τ The correlation between them is calculated by fast Fourier transform.

[0058] Finally, time-delayed aggregation of similar subsequences is achieved:

[0059] τ1,…,τ k =arg Topk(R Q，K (τ)), τ∈{1,…,L}

[0060]

[0061]

[0062] By selecting the k subsequences with the highest correlation, the lengths τ1,…,τ k , and finally calculate the sequence weight Complete weighted aggregation of subsequences;

[0063] After the Autoformer model is built, the FeO content level of the infrared image of the sintered ore section provided by the SE-ResNet network in step 1.2 can be used to generate an additional long-term trend component. This trend component is combined with the original input data of the Decoder part of the Autoformer to provide the current prior quality information for the prediction task. image , the above-mentioned prior quality information is integrated with the Autoformer model as follows:

[0064] y c =SE_ResNet(X image )

[0065] y t，c =AvgPool(Padding(y c )

[0066] X det，c =Concat(X det ,y t，c )

[0067] In the above formula, y c The image level classification result provided by the SE-ResNet model is then smoothed using motion to obtain y c The trend component y t，c , and finally t，c The original trend component input X of the Decoder layer det Combined to obtain the trend component input X with prior quality information det，c .

[0068] Step 2: Online soft measurement;

[0069] Step 2.1: Based on the sampling time of the finished sintered ore, collect the sensor data X of each part of the entire sintering process according to step 1.1 num and infrared image of sinter cross section X image ;

[0070] Step 2.2: Infrared image of the cross section of the sintering machine tail Ximage , extract the shallow features M of the sintered section according to step 1.2 s With deep features M d , achieving feature-level information fusion X f =Concat(M s , M d ) and data-level information fusion X en =Concat(X f , X num );

[0071] Step 2.3: Fusion of multi-source information into data X en Input Autoformer to calculate N Encoder layer latent variables

[0072]

[0073] Then follow step 1.3 to calculate the initial input X of the Decoder layer. des and X det，c , after inputting M decoder layers, soft measurement of FeO content in sintered ore is realized:

[0074]

[0075] The beneficial effects of the present invention are:

[0076] 1. Combining expert knowledge and data-driven SE-ResNet classification model, the sinter section feature extraction is realized to provide more reliable and comprehensive image information.

[0077] 2. By compressing the observed image and fusing the image features with the numerical data according to the sampling time, it is beneficial to establish a time series model of the continuous sintering process and ensure the integrity of the dynamic characteristics of the process data.

[0078] 3. The multi-source information fusion strategy improves the information capacity of downstream tasks, and the classification results of SE-ResNet also provide prior knowledge for sinter quality prediction, thereby improving the accuracy and reliability of Autoformer.

[0079] 4. The Autoformer model of multi-source information fusion builds a serial structure of deep convolutional network and Autoformer, embeds image information into time series, and then captures the global dependency between input and output in the sequence, providing an effective method for soft measurement of quality indicators of complex sintering processes. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1A flow chart is constructed for the Autoformer sinter FeO content soft sensing model based on multi-source information fusion.

[0081] Figure 2 Flow chart of the sintering process.

[0082] Figure 3 This is the key frame selection diagram for the infrared image of the sinter ore section at the tail of the sintering machine.

[0083] Figure 4 Figure 2 shows the SE-ResNet model structure and deep feature extraction diagram.

[0084] Figure 5 Schematic diagram of multi-source information fusion in the sintering process.

[0085] Figure 6 Pearson correlation coefficient heat map between image features and FeO content in sintered ore.

[0086] Figure 7 Figure 2 shows the soft measurement results of FeO content of different models.

[0087] Figure 8 The soft sensor R of FeO content in 16 different models in 20 experiments 2 Indicator box plot. DETAILED DESCRIPTION

[0088] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be pointed out that the embodiments described below are intended to facilitate the understanding of the present invention and do not have any limiting effect on the present invention. Figure 1 The flowchart of the construction of the Autoformer sintered ore FeO content soft sensor model based on multi-source information fusion. The implementation of the following cases is based on Figure 1 Complete the specific steps in.

[0089] (1) Sintering process data collection

[0090] The experimental data for this experiment comes from a 360 square meter sintering machine in a sintering plant of a steel group in South China. As shown in Table 1, there are 19 original variables used to establish the soft sensor model for FeO content in sintered ore, including 17 numerical process variables, 1 infrared image of the sintered ore cross section, and 1 output variable, namely the FeO content test value. Since the entire sintering process generally takes more than 1 hour, the backtracking strategy collects variables at different locations to ensure data quality. Figure 2As shown in the figure, the sampling traceback of the first stage of the sintering process is realized based on the ratio of the length and speed of the feeding belt; in the second and third stages of the sintering process, due to the continuous change of the running speed of the sintering trolley and the ring cooler, the collection of different variables is realized by directly measuring the material transportation time (for example, the sintering reaction time from the n# wind box to the tail of the sintering machine can be measured through the trolley positioning device). Specifically, in Table 1, t is the sampling time of the sintered ore product, t a and t b are the time consumption of the ring cooler operation and the sintering machine tail crushing process, t n is the running time of the sintering trolley from the n# wind box to the tail of the sintering machine, t f It represents the time spent from the feeding bin to the ignition position, l1 and l2 are the lengths of the feeding belts before and after the second mixing drum, v is the speed of the feeding belt, m1 and m2 are the time consumption of the first mixing drum and the second mixing drum, respectively. The variables with "*" are collected from both sides of the sintering machine and then averaged.

[0091] Table 1 Sintering process number variable table

[0092]

[0093] (2) Feature extraction and fusion of infrared images of sintered ore sections

[0094] Based on the infrared image of the sintered ore cross section provided by variable 17 in Table 1, the shallow and deep features of the image are extracted respectively. First, the key frame extraction of the sintered ore cross section is realized for the original video frame extracted by the infrared thermal imager. At the tail of the sintering machine, as the trolley flips, the finished sintered ore continues to fall and break. This process is accompanied by rapid light and dark transitions, which will cause overexposure of the visible light camera. In this case, since the working principle of the online infrared thermal imager is to collect the temperature of each pixel, the data error caused by light changes can be avoided. Figure 3 Parts (1) to (5) provide the transformation between the movement trajectory of the sintered ore at the tail of the sintering machine and the infrared image information of the cross section of the sintered ore. Part (1) is the moment when the sintered ore falls. At this time, the picture is relatively clear. The red layer of the cross section of the sintered ore has just been exposed to the air and the cross section temperature is the highest. Subsequently, the temperature of the red layer of the cross section of the sintered ore drops rapidly after it is fully in contact with the air. As the finished sintered ore falls, dust is raised at the bottom of the sintering machine, which interferes with the thermal imager shooting (for example, in part (2), the dust covers the right side of the cross section of the sintered ore, reducing the image brightness, that is, the measured temperature). As the trolley flips, part (3) provides an image of the sintered ore about to fall. Due to the change in the angle of the trolley, the combustion layer of the cross section of the sintered ore captured at this time becomes smaller. Subsequently, parts (4) and (5) record the generation process of a sintered cross section.

[0095] On the whole, the sintered ore is Figure 3 (1) and (5) of the sintered ore section, the time the sintered ore section is in contact with the cold air is the shortest and is not disturbed by dust. Therefore, this type of image provides more comprehensive temperature information and should be used in the subsequent feature extraction of infrared thermal imager video frames. Figure 3 As shown in part (6) of , in this experiment, the above key frames can be selected based on the global temperature and of each infrared image. It is worth noting that due to the heat treatment nature of the sintering process, the heat shield and the surface of the trolley have higher temperatures at the tail of the sintering machine, which may cause greater interference to the key frame selection of the infrared image. Therefore, the infrared image area used for calculation is reduced from 480×640 to 128×640 to ensure accurate extraction of the key frame. Specifically, the calculation area of ​​the infrared image is used Figure 3 The dashed rectangle in parts (1) to (5) of the figure represents the bottom position of the sintering material layer (i.e., the surface position of the trolley in the horizontal state), and the upper edge of the dashed rectangle is tangent to the heat shield in the figure. Since the online infrared thermal imager used in this experiment is fixed at the tail of the sintering machine, and the physical positions of the sintering trolley and the heat shield remain unchanged, the setting of the above infrared image calculation area is more reasonable.

[0096] After selecting the key frames of the infrared image, the shallow and deep features of the sinter section are extracted based on expert knowledge and the SE-ResNet classification model. First, the shallow features Temp, Thick, C, and S are extracted based on expert knowledge. BTP ; Then, the SE-ResNet model is trained based on the image keyframes for the classification of FeO content in sintered ore, and the feature channels with significant weights in the model are selected layer by layer to extract the deep features of the infrared image. Specifically, the feature channel selection criteria in each convolution layer are as follows:

[0097] w i <u w -2σ w |w i >u w +2σ w

[0098] where w i is the weight of the i-th channel in the current convolutional layer, u w and σ w are the mean and standard deviation of all channel weights in the current convolutional layer, respectively.

[0099] Table 2 provides the structure of the SE-ResNet-18 model and the details of the deep feature extraction of the sinter section. According to Table 2, the channel features provided by the Conv1 layer are not significantly different from the original image, so the feature extraction starts from the Cov2_x layer. For each selected feature channel, the infrared image is first converted into a binary image, and then the image morphological operation is used to eliminate noise interference and highlight the detail area. Finally, the improved binary image (the "A" type deep feature in Table 2) and the infrared image corresponding to the binary image (the "B" type deep feature in Table 2) are obtained. It can be seen from the representative channels shown in Table 2 that the Cov2_x layer focuses on the local high temperature area. In this convolution layer, the features F1 and F2 provide the area proportion and temperature sum of the local high temperature area, respectively. The feature channels of the Cov3_x and Cov4_x layers extract the detail highlights and rough highlights of the image, respectively. Bright spots represent high-temperature spots, and the spacing between bright spots reflects the porosity of the sintering reaction layer to a certain extent; based on this, features F3 and F5 provide the proportion of bright spot area, and features F4 and F6 calculate the temperature and temperature corresponding to the bright spot area to reflect the change in porosity and the combustion state between pores. Due to the superposition of convolutional layers, the feature map of the Cov5_x layer is relatively abstract, so the overall temperature and temperature of the original channel features are used to construct feature F7. It is worth noting that the first six deep image features (from F1 to F6) are highly correlated with the combustion state (local brightness and porosity) of the sintering machine section, and are also reaction features that sintering process experts are very concerned about. This shows that the image features extracted by SE-ResNet-18 are relatively reliable and have high interpretability. From another perspective, some parameters of image morphological operations can also be fine-tuned by experts to improve the quality of deep image features, so that these features are close to the observable but difficult to quantify combustion state of the sintered ore section, so that the feature extraction performance of the model can be further improved under the intervention of experts.

[0100] Table 2 Deep layer feature extraction of sintered ore section

[0101]

[0102] Based on the shallow and deep characteristics of the infrared image of the sintered ore section and the rest of the numerical data of the sintering process, Figure 5 It can realize the fusion of multi-source information in the sintering process.

[0103] (3) Autoformer model training based on multi-source information fusion

[0104] The multi-source fusion information of the sintering process is used as the input of the Autoformer model Encoder, and the sintered ore FeO content grade classification information provided by the SE-ResNet-18 model is used to optimize the input of the Autoformer model Decoder. Finally, the assay value of the FeO content in the sintered ore is used as the target to train the Autoformer model until convergence.

[0105] (4) Autoformer model completion process based on multi-source information fusion

[0106] Step 1: Offline modeling;

[0107] Step 1.1: For the same batch of raw materials in the sintering production process, the sampling time of the finished sintered ore is used as the benchmark, and the sintering process data is sampled and traced according to the sintering machine structure, trolley operation status and material conveyor belt, and the sensor data of each part of the entire sintering process and the infrared image data of the sintered ore cross section are collected.

[0108] Step 1.2: Combining expert knowledge and data-driven SE-ResNet classification model, shallow and deep features of the sinter section are extracted respectively to achieve image feature-level information fusion.

[0109] During the sintering process, experts always pay attention to the most basic and important factors that reflect the production quality, so as to estimate the sinter quality indicators in real time and maintain stable production through different control operations. At the tail of the sintering machine, the cross-sectional temperature of the sinter (usually considered by workers as brightness) and the thickness of the sinter red layer are the key features that directly reflect the combustion state. Sintering consistency represents the vertical combustion rate, which is mainly related to the permeability and greatly affects the FeO content of the sinter. At the same time, since the vertical sintering rate determines the end point of combustion, namely the burn-through point (BTP), the overall state of BTP also plays an important role in the evaluation of sinter performance. In general, four shallow characteristics, namely the cross-sectional temperature of the sinter, the thickness of the sinter red layer, the combustion consistency and the overall state of BTP, are extracted according to expert knowledge, which are Temp, Thick, C and S respectively. BTP Taking the infrared thermal imaging image of a sintered ore cross section with a height of h and a width of w as an example, the above characteristics are specifically expressed as follows:

[0110]

[0111]

[0112]

[0113]

[0114] Among them, t ij represents the temperature at the coordinate (i, j) of the infrared thermal imaging image of the sintered ore section, p j is the height of the red layer in the jth column pixel, m j Indicates the height of the highest temperature point in the j-th column of pixels.

[0115] Subsequently, the SE-ResNet model is used to classify the FeO content level of the sinter section, and deep feature extraction is performed based on the hierarchical convolutional features learned by the network. ResNet is an efficient deep learning framework that alleviates the difficulty of training deep networks through residual learning modules. Assume that the input of the residual learning module is X, and the expected underlying mapping U(X) is defined as:

[0116] U(X)=F(X)+X

[0117] In the formula, F(X) represents the nonlinear residual mapping layer used for superposition, X is connected to the residual learning module in a quick connection manner, and U(X) is obtained by element-wise addition with F(X). Generally speaking, at least two weight layers are set in F(X) to ensure the effectiveness of the residual module. Taking two weight layers as an example, F(X) can be specifically expressed as:

[0118] F(X)=W2σ(W1X)

[0119] Among them, W1 and W2 are the weight matrices of the first weight layer and the second weight layer respectively, and σ represents the implementation of a nonlinear activation function (such as the ReLU function). Based on the above residual learning module, the two weight layers are replaced by convolutional layers to obtain a convolutional residual module. By superimposing multiple residual convolutional modules, the ResNet model can be built. Since conventional convolution operations are difficult to distinguish the importance of feature channels, the SE module is used here to explore the dependencies between channels and realize the recalibration and selection of feature channels. Assume that the conventional residual convolution module can be expressed as F cov :X→Y, X and Y are the input and output before and after the convolution transformation, respectively. H, W, C and H′, W′, C′ represent the height, width and number of channels of the input and output images, respectively. Y = [y1, y2, ..., y C′ ] represents the specific C′ sub-channel output. The SE module consists of three steps: compression, excitation, and weighting. First, global average pooling is performed on each channel of Y, with the fth channel y f For example, the compression transformation F sq It can be expressed as:

[0120]

[0121] Among them, y f (i, j) represents channel y fThe value at the coordinate (i, j), s f is the output of the compression transformation of the fth channel. Then the excitation transformation F is performed ex Specifically, the sigmoid activation gating mechanism is introduced to achieve adaptive calibration of channel weights:

[0122] w=F ex (s, W) = δ(g(s, W)) = δ(W FC，2 σ(W Fc，1 s))

[0123] Where w = [w1, w2, ..., w C′ ] is the weight set of C′ channels, δ is the sigmoid activation function, σ is the ReLU function, g(s, W) represents a gating structure consisting of two fully connected layers (FC) and a ReLU, s = [s1, s2, ..., s C′ ] indicates F sq C′ outputs, and is the weight matrix of the two FC layers. In this gated structure, the two FC layers reduce (C′→C′ / r) and increase (C′ / r→C′) the channel dimension before and after the ReLU function, respectively, so that the excitation transformation limits the model complexity while capturing the nonlinear dependencies between channels. The parameter r is a hyperparameter that controls the model complexity. Finally, based on the weighted transformation function F we , scale the output Y to get the weighted output For single-channel calculations, F we The calculation can be expressed as:

[0124]

[0125] where w f is the weight of the fth channel, and the symbol · represents the scalar w f With single channel data The product of express Based on the above steps, the SE-ResNet module can be built, and the SE-ResNet model can be constructed based on the stacking of SE-ResNet modules.

[0126] Based on the infrared thermal imaging of the sintered ore cross-section at the tail of the sintering machine, the SE-ResNet network was built to realize the grade classification of the FeO content in the sintered ore; after the model training was completed, the 3σ criterion was used to select the feature channels with significant weights according to the hierarchical convolution features learned by the network and the weights between the feature channels in each layer; then, based on the image morphological operation, the noise interference was eliminated and the detail areas were highlighted. The area ratio and temperature sum of the detail areas were calculated respectively to realize the selection of image depth features.

[0127] Based on the above-mentioned image feature extraction, the shallow features and deep features are quantified respectively, and the image feature-level information fusion is completed in the form of variables; then, by calculating the time difference between image acquisition and sensor data sampling, the data-level information fusion of image features and sensor values ​​is realized.

[0128] Step 1.3: Based on Step 1.1 and Step 1.2, the multi-source information is serially connected with the downstream Autoformer model. The overall implementation of Autoformer can be composed of N Encoder layers and M Decoder layers; the Encoder and Decoder modules are connected end to end within the layer, and the output of the Encoder is connected between layers within each Decoder layer. For an input sequence of length L and dimension d The implementation steps of Autoformer are as follows:

[0129] By solving the first seasonal component - Solve for and the second seasonal component _Build the Encoder layer:

[0130]

[0131]

[0132] Among them, AutoCorrelation, FeedForward and SeriesDecomp represent the autocorrelation module, the forward propagation module and the trend decomposition module respectively. represents the output of the l-1th Encoder layer, Represents the initial input of the Encoder layer, and They represent the two seasonal components output by the SeriesDecomp module in the lth Encoder layer, respectively. "_" is the omitted trend component. Repeating the above formula N times can realize Encoder layer stacking.

[0133] For the Decoder layer, the initialization steps of the module input are as follows:

[0134] Xdes，p：L , X det，p：L = SeriesDecomp(X de，p：L )

[0135] X des =Concat(X des，p：L , X0)

[0136] X det =Concat(X det，p：L , X mean )

[0137] in, represents the overall input of the Decoder layer with a length of L and a dimension of d, X de，p：L Provides historical information from time step p to L, Respectively represent X de，p：L The seasonal and trend components of X0 and X mean Represents 0 and X de The mean of des，p：L and X det，p：L At the end of the decoder layer, the seasonal component input X is constructed des and trend component input X det ; Then, the first, second, and third pairs of seasonal components and trend components of the Decoder layer are calculated in turn:

[0138]

[0139]

[0140]

[0141] In the above formula, represents the output of the l-1th Decoder layer, and is the latent variable output by the Encoder part, Represents the i-th pair of seasonal component and trend component in the l-th Decoder layer; the above trend components are accumulated to obtain the overall trend component of the l-th Decoder layer:

[0142]

[0143] Among them, W l，i , i∈{1, 2, 3} represents the i-th trend component in the l-th Decoder layer The projection matrix of Repeating the above formula M times can achieve Decoder layer stacking; finally, the prediction is completed by the following formula:

[0144]

[0145] Here the Mth Decoder layer seasonal component With the projection matrix W M After multiplication, the dimension is changed and then the trend component of the Mth Decoder layer is Add them together to get the final predicted value.

[0146] The above trend decomposition module uses moving smoothing to decompose the sequence into seasonal components and long-term trend terms. For an input sequence with a length of L and a dimension of d For SeriesDecomp, it can be expressed as:

[0147] X t =AvgPool(Padding(X))

[0148] X s =XX t

[0149] in, represent the seasonal component and the trend component respectively.

[0150] The above autocorrelation module implements the autocorrelation analysis at the subsequence level. The specific execution steps are as follows:

[0151] For an input sequence of length L and dimension d Perform linear mapping to obtain the corresponding matrices Q, K, and V:

[0152] Q=W Q X

[0153] K=W K X

[0154] V=W V X

[0155] Then, the autocorrelation of the subsequences is calculated:

[0156]

[0157] Among them, R Q，K (τ) represents Q t With the sequence K delayed by τ t-τ The correlation between them is calculated using fast Fourier transform.

[0158] Finally, time-delayed aggregation of similar subsequences is achieved:

[0159] τ1,…,τ k =arg Topk(R Q，K (τ)), τ∈{1,…,L}

[0160]

[0161]

[0162] By selecting the k subsequences with the highest correlation, the lengths τ1,…,τ k , and finally calculate the sequence weight Complete weighted aggregation of subsequences.

[0163] After the Autoformer model is built, the FeO content level of the infrared image of the sintered ore section provided by the SE-ResNet network in step 1.2 can be used to generate an additional long-term trend component. This trend component is combined with the original input data of the Decoder part of the Autoformer to provide the current prior quality information for the prediction task. image , the above-mentioned prior quality information is integrated with the Autoformer model as follows:

[0164] y c =SE_ResNet(X image )

[0165] y t，c =AvgPool(Padding(y c )

[0166] X det，c =Concat(X det ,y t，c )

[0167] In the above formula, y c The image level classification result provided by the SE-ResNet model is then smoothed using motion to obtain y c The trend component y t，c , and finally t，c The original trend component input X of the Decoder layer det Combined to obtain the trend component input X with prior quality information det，c .

[0168] Step 2: Online soft measurement;

[0169] Step 2.1: Based on the sampling time of the finished sintered ore, collect the sensor data X of each part of the entire sintering process according to step 1.1 num and infrared image of sinter cross section X image ;

[0170] Step 2.2: Infrared image of the cross section of the sintering machine tail Ximage , extract the shallow features M of the sintered section according to step 1.2 s With deep features M d , achieving feature-level information fusion X f =Concat(M s , M d ) and data-level information fusion X en =Concat(X f , X num );

[0171] Step 2.3: Fusion of multi-source information into data X en Input Autoformer to calculate N Encoder layer latent variables

[0172]

[0173] Then follow step 1.3 to calculate the initial input X of the Decoder layer. des and X det，c , after inputting M decoder layers, soft measurement of FeO content in sintered ore is realized:

[0174]

[0175] (5) Soft sensor performance test

[0176] In order to verify the application effect of the Autoformer sintered ore FeO content soft sensor model based on multi-source information fusion, a total of 2379 samples were collected since December 2021, and the time between adjacent key frames of infrared thermal imaging videos was taken as the sampling interval, of which the first 1665 samples were used for model training and the last 714 samples were used for model testing. The sequence length of the time series samples was set to 10. In order to preliminarily verify the effectiveness of the extracted image features, Figure 6 A heat map of the Pearson correlation coefficients between the image features and the FeO content of the sinter is provided. Figure 6 It can be seen that the correlation between the cross-sectional temperature of the sintered ore and the assay value of the FeO content is the lowest, which means that it is difficult to estimate the change in the quality of the sintered ore simply by the sum of the cross-sectional temperatures of the sintered ore; SE-ResNet is based on a data-driven training model, focusing on the temperature characteristics of different areas of the image layer by layer, which makes up for the above defects and reflects the necessity of deep feature extraction. Overall, the Pearson correlation coefficients of the remaining 10 image features (including 3 shallow image features and 7 deep image features) and the FeO content of the finished sintered ore are all higher than 0.2, which greatly enriches the data information of the sintering process. Finally, the soft measurement results of the Autoformer (MIF-Autoformer) model based on multi-source information fusion are shown in the figure. Figure 7As shown in the figure, soft measurement models such as gated recurrent neural network (GRU), partial least squares based on gated recurrent neural network (GRU-PLS), Transformer, LogTrans and Autoformer are provided for performance comparison; as can be seen from the figure, the soft measurement results provided by the models built based on recurrent neural network (GRU and GRU-PLS) have large errors, while the soft measurement accuracy of the other four models is significantly improved, among which the MIF-Autoformer model has the best fitting effect; it is worth noting that although the Autoformer model also provides a good fitting effect, the soft measurement value fluctuates around the FeO test value, which intuitively reflects the performance improvement effect of the multi-source information fusion strategy on the Autoformer model. The specific soft measurement results of the model can be found in Table 3. The soft measurement effect is measured by the root mean square error (RMSE) and the determination coefficient (R 2 ) to measure together.

[0177] Table 3 Comparison of soft measurement results of FeO content in sintered ore of different models

[0178]

[0179] In order to more clearly demonstrate the improvement effect provided by the multi-source information fusion strategy for the model, Figure 8 The following ablation experiments are additionally provided. Box plots are drawn using the prediction results of the above five basic models and their improved models. Figure 8 In the figure, "SF" represents only shallow features of the image provided to the downstream model, and "HF" represents mixed features that provide both shallow and deep features to the downstream model. Obviously, "SF" and "HF" can improve the prediction accuracy of different models in most cases, and the latter has a more significant improvement effect. This is mainly because the mixed features provide more comprehensive image information, which also makes "HF" show significant versatility in different models. "MIF" not only introduces image mixed features to the downstream Autoformer model, but also provides prior information on the FeO content of sintered ore, thereby further improving the accuracy and stability of the model.

[0180] Overall, the Autoformer model based on multi-source information fusion has the best soft measurement effect. Inspired by multi-source heterogeneous data in the sintering process, the model combines expert knowledge and data-driven SE-ResNet classification model to realize sinter cross-sectional feature extraction and provide more reliable and comprehensive image information; then, the image features are fused with numerical data according to the sampling time, and the continuity of the data is retained when realizing multi-source information fusion, which provides the necessary foundation for the construction of downstream time series models; finally, the Autoformer model is built on the basis of multi-source information fusion, and the sub-sequence level correlation analysis and data aggregation are used to capture the complex time series distribution of sintering process data, providing accurate and stable soft measurement results for the FeO content of sintered ore.

[0181] The above-mentioned embodiments only express several implementation modes of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

Claims

1. An Autoformer sintered ore FeO content soft measurement method based on multi-source information fusion, characterized in that: Expert knowledge and data-driven SE-ResNet classification model are used to extract shallow and deep features of sinter section respectively, realize feature-level information fusion, and realize data-level information fusion of image features and other numerical data in the sintering process according to sampling time points; multi-source information fusion data is used in the downstream Autoformer model to capture the global dependency between input data and output data with more comprehensive sintering process information, thereby obtaining accurate and stable soft measurement results of FeO content in sinter; the multi-source information comes from: For the same batch of raw materials in the sintering production process, the sampling time of the finished sintered ore is used as the benchmark, and the sintering process data is sampled and traced according to the sintering machine structure, the trolley operation status and the material conveyor belt, and the sensor data of each part of the entire sintering process and the infrared image data of the sintered ore section are collected; The method of multi-source information fusion is: Combining expert knowledge and data-driven SE-ResNet classification model, shallow and deep features of sintered ore sections are extracted respectively, realizing image feature-level information fusion; During the sintering process, four shallow characteristics, including sinter cross-section temperature, sintering red layer thickness, combustion consistency and BTP overall state, were extracted based on expert knowledge. BTP Indicates the infrared thermal image of the sinter cross section with height h and width w respectively. The above features are expressed as: Among them, t ij represents the temperature at the coordinate (i, j) of the infrared thermal imaging image of the sintered ore section, p j is the height of the red layer in the jth column pixel, m j Indicates the height of the highest temperature point in the jth column of pixels; Subsequently, the SE-ResNet model is used to classify the FeO content level of the sinter section, and deep feature extraction is performed based on the hierarchical convolutional features learned by the network; assuming that the input of the residual learning module is X, the expected underlying mapping U(X) is defined as: U(X)=F(X)+X In the formula, F(X) represents the nonlinear residual mapping layer used for superposition, X is connected to the residual learning module in a quick connection manner, and U(X) is obtained by element-wise addition with F(X). More than two weight layers are set in F(X) to ensure the effectiveness of the residual module; based on the above residual learning module, the weight layer in F(X) is replaced by a convolutional layer to obtain a convolutional residual module, and the ResNet model is built by superimposing multiple residual convolution modules; the SE module is used to explore the dependency between channels to realize the recalibration and selection of feature channels; the conventional residual convolution module is expressed as X and Y are the input and output before and after the convolution transformation, respectively. H, W, C and H′, W′, C′ represent the height, width and number of channels of the input and output images, respectively. Y = [y1, y2…, y C′ ] represents the specific C′ sub-channel output; the SE module consists of three steps: compression, excitation, and weighting; first, global average pooling is performed on each channel of Y, and the fth channel y f , compression transformation F sq It is expressed as: Among them, y f (i, j) represents channel y f The value at the coordinate (i, j), s f is the output after compression transformation of the fth channel; then the excitation transformation F is performed ex , introduce the sigmoid activation gating mechanism to achieve adaptive calibration of channel weights: w=F ex (s,W)=δ(g(s,W))=δ(W FC,2 σ(W FC,1 s)) Where w = [w1, w2, ..., w C′ ] is the weight set of C′ channels, δ is the sigmoid activation function, σ is the ReLU function, g(s, W) represents a gating structure consisting of two fully connected layers (FC) and a ReLU, s = [s1, s2, ..., s C′ ] indicates F sq C′ outputs, and is the weight matrix of the two FC layers, and the parameter r is a hyperparameter that controls the complexity of the model; finally, based on the weighted transformation function F we , scale the output Y to get the weighted output For single-channel calculations, F we The calculation is expressed as: where w f is the weight of the fth channel, and the symbol · represents the scalar w f With single channel data The product of express The f-th weighted channel of ; Based on the above steps, the SE-ResNet module is built, and the SE-ResNet model is built based on the stacking of SE-ResNet modules; Based on the infrared thermal imaging image of the cross section of the sintered ore at the tail of the sintering machine, the SE-ResNet network is built to realize the classification of the FeO content of the sintered ore; after the model training is completed, the feature channels with significant weights are selected based on the weights between the layered convolution features learned by the network and the feature channels in each layer. Subsequently, based on the image morphological operation, the noise interference is eliminated, the detail area is highlighted, and the area ratio and temperature of the detail area are calculated respectively to realize the selection of image depth features; Based on the above image feature extraction, the shallow features and deep features are quantified respectively, and the image feature level information fusion is completed in the form of variables; then, by calculating the time difference between image acquisition and sensor data sampling, the data level information fusion of image features and sensor values ​​is realized; The Autoformer is constructed as follows: The multi-source fusion information is integrated with the downstream Autoformer model in a serial connection mode; the overall implementation process of the Autoformer consists of N Encoder layers and M Decoder layers; the Encoder and Decoder are connected end to end in the layer, and the output of the Encoder is connected between layers within each Decoder layer; for an input sequence of length L and dimension d The implementation steps of Autoformer are as follows: By solving the first seasonal component - Solve for and the second seasonal component -Build the Encoder layer: Among them, AutoCorrelation, FeedForward and SeriesDecomp represent the autocorrelation module, the forward propagation module and the trend decomposition module respectively. represents the output of the l-1th Encoder layer, Represents the initial input of the Encoder layer, and They represent the two seasonal components output by the SeriesDecomp module in the lth Encoder layer, and “-” is the omitted trend component. Repeat formula (1) and formula (2) N times to achieve Encoder layer stacking. For the Decoder layer, the initialization steps of the module input are as follows: X des,p：L ,X det,p：L =SeriesDecomp(X de,p：L ) (3) X des =Concat(X des,p：L ,X0) (4) X det =Concat(X det,p：L ,X mean ) (5) in, represents the overall input of the Decoder layer with a length of L and a dimension of d, X de,p：L Provides historical information from time step p to L, X des,p：L , Respectively represent X de,p：L The seasonal and trend components of X0 and X mean Represents 0 and X de The mean of des,p：L and X det,p：L At the end of the decoder layer, the seasonal component input X is constructed des and trend component input X det ; Then, the first, second, and third pairs of seasonal components and trend components of the Decoder layer are calculated in turn: in, represents the output of the l-1th Decoder layer, and is the latent variable output by the Encoder part, Represents the i-th pair of seasonal component and trend component in the l-th Decoder layer; the trend component is accumulated to obtain the overall trend component of the l-th Decoder layer: Here, W l，i , i∈{1, 2, 3} represents the i-th trend component in the l-th Decoder layer The projection matrix of Repeating formula (6)-formula (9) M times can realize the decoder layer stacking; finally, the prediction is completed by the following formula: Here the Mth Decoder layer seasonal component After multiplying with the projection matrix WM, the dimension is changed and then multiplied with the trend component of the Mth Decoder layer Add them together to get the final predicted value; The trend decomposition module SeriesDecomp(·) uses moving smoothing to decompose the series into seasonal components and long-term trend terms. For an input series with a length of L and a dimension of d, In terms of , this module is represented as: X t =AvgPool(Padding(X))(11) X s =X-X t (12) Among them, X s , represent the seasonal component and the trend component respectively; The AutoCorrelation module AutoCorrelation(·) implements the autocorrelation analysis at the subsequence level. The execution steps are as follows: For an input sequence of length L and dimension d Perform linear mapping to obtain the corresponding matrices Q, K, and V: Q=W Q X (13) K=W K X (14) V=W V X (15) Then, the autocorrelation of the subsequences is calculated: Among them, R Q,K (τ) represents Q t and the sequence K delayed by τ t-τ The correlation between them is calculated by fast Fourier transform; Finally, time-delayed aggregation of similar subsequences is achieved: τ1,…,τ k =arg Topk(R Q,K (τ)),τ∈{1,…,L} (17) By selecting the k subsequences with the highest correlation, the lengths τ1,…,τ k , and finally calculate the sequence weight Complete weighted aggregation of subsequences; After the Autoformer model is built, the FeO content level of the infrared image of the sintered ore section provided by the SE-ResNet network is used to generate an additional long-term trend component, which is then merged with the original input data of the Decoder part of the Autoformer to provide the current prior quality information for the prediction task. image , the above-mentioned prior quality information is integrated with the Autoformer model as follows: and c =SE_ResNet(X image ) (20) y t，c =AvgPool(Padding(y c ) (21) X det,c =Concat(X det ,y t,c ) (22) In formula (20)-formula (22), yc is the image level classification result provided by the SE-ResNet model, and then moving smoothing can be used to obtain y c The trend component y t，c , and finally t，c The original trend component input X of the Decoder layer det Combined to obtain the trend component input X with prior quality information det,c .

2. The Autoformer sintered ore FeO content soft measurement method based on multi-source information fusion according to claim 1 is characterized in that: The application process of the Autoformer sintered ore FeO content soft sensing model is as follows: Step 4.1: Based on the sampling time of the finished sintered ore, collect sensor data X at each part of the entire sintering process num and infrared image of sinter cross section X image ; Step 4.2: Infrared image of the cross section of the sintering machine tail X image , extract the shallow characteristics of the sintering section M s With deep features M d , realizing feature-level information fusion X f =Concat(M s , M d ) and data-level information fusion X en =Concat(X f , X num ); Step 4.3: Fusion of multi-source information into data X en Input Autoformer to calculate N Encoder layer latent variables Then calculate the initial input X of the Decoder layer des and X det,c , after inputting M decoder layers, soft measurement of FeO content in sintered ore is realized:

Images