A sintering quality intelligent optimization method fusing multi-modal perception and knowledge reasoning

Abstract

The application discloses a sintering quality intelligent optimization method fusing multi-modal perception and knowledge reasoning, and belongs to the technical field of intelligent manufacturing of steel smelting. The method comprises the following steps: synchronously collecting time sequence process parameters and tail section video stream data of a sintering process; obtaining a quality index prediction value; when the prediction value exceeds a threshold value, generating a structured abnormal data packet; then, taking the data packet as a parameter, searching a cause-effect chain from a sintering knowledge graph; then, inputting the cause-effect chain and the data packet into a large model adapted in two stages, generating a structured output containing a diagnosis conclusion, a confidence and an operation suggestion under forced cause-effect constraints; generating and executing a quality abnormality diagnosis and closed-loop control scheme. The application deeply couples a prediction model, a knowledge graph, a large model and an industrial control system, solves the chain fracture problem that a prediction result cannot be automatically converted into an executable decision, and realizes credible diagnosis and safe closed-loop control of sintering quality.

Description

Technical Field

[0001] This invention belongs to the field of intelligent manufacturing technology in iron and steel smelting, and in particular relates to an intelligent optimization method for sintering quality that integrates multimodal perception and knowledge reasoning. Background Technology

[0002] Sinter, as a crucial raw material in blast furnace ironmaking, directly impacts blast furnace operation, fuel ratio, and molten iron quality through its chemical composition, strength properties, metallurgical performance, and sintering thermal state. Existing methods for predicting sinter quality largely rely on a single data source: one type uses time-series process parameters such as raw material information, bellows parameters, and trolley speed to establish a predictive model, reflecting process variable fluctuations but failing to directly depict the true sintering endpoint state at the tail section; another type relies on visible light or infrared images of the tail section for empirical analysis of the location and morphology of the red-hot layer. While intuitive, this approach is susceptible to influences from lighting, moisture, shading, and operating condition fluctuations, making stable, quantitative, and high-precision predictions difficult.

[0003] Meanwhile, existing technologies often have gaps in the "prediction-diagnosis-optimization" chain. Even if a quality anomaly is predicted, operators still need to manually investigate the cause by combining process experience. There is a lack of an automated diagnostic mechanism that can integrate process parameters, visual status, historical cases and expert knowledge, making it difficult to form a closed-loop control. If a general large model is further introduced for diagnosis, four major technical bottlenecks will be faced. (1) Passive questioning leads to delayed response. It is necessary to manually input the prediction results into the large model to ask questions, which cannot achieve seamless automation from the occurrence of an anomaly to the initiation of diagnosis. (2) Causal illusion leads to unreliable diagnosis. The general large model is prone to generating "illusion" diagnoses that are unrelated to the actual process causal chain, and the conclusions are difficult to trace and verify. (3) Free format leads to failure to execute automatically. The natural language suggestions output by the large model cannot be directly parsed and executed by the industrial control system, and still need to be manually converted. (4) Unknown risks lead to non-closed-loop execution. There is a lack of risk assessment and graded execution mechanism for operation suggestions, which makes it impossible to safely and reliably control the basic automation system.

[0004] Therefore, there is an urgent need for an intelligent sintering quality control system that integrates high-precision prediction, reliable causal diagnosis, and safe closed-loop control. Summary of the Invention

[0005] In view of this, the present invention discloses an intelligent optimization method for sintering quality that integrates multimodal perception and knowledge reasoning.

[0006] The present invention adopts the following technical solution: A method for intelligent optimization of sintering quality that integrates multimodal perception and knowledge reasoning includes the following steps: H1. Acquire timing process parameters and tail section video stream data during the sintering process; H2. Preprocess the timing process parameters to obtain processed timing process parameters, and obtain a timing feature vector based on the processed timing process parameters; H3. Select the optimal tail section image corresponding to the unloading cycle from the tail section video stream data; H4. Preprocess the optimal tail section image to obtain a standardized section image; H5. Input the standardized cross-sectional image into the red fire layer semantic segmentation model to obtain the red fire layer binary mask; H6. Extract visual feature vectors based on the binary mask of the red flame layer; H7. The temporal feature vector and the visual feature vector are fused to form a fused feature vector; H8. The fused feature vectors from multiple consecutive sampling times are combined to form a fused feature sequence. The fused feature sequence is then input into an LSTM quality prediction model that is automatically optimized by an optimization algorithm to obtain the sinter quality index prediction results. H9. When the prediction result exceeds the preset threshold, the "prediction-retrieval-reasoning-execution" chain architecture is triggered to generate and execute a quality anomaly diagnosis and closed-loop control scheme.

[0007] Furthermore, H9 specifically includes: H9-1. Automatic Triggering of Diagnosis and Generation of Structured Abnormal Data Packets: When the prediction result exceeds a preset threshold, a structured abnormal data packet is automatically generated; H9-2. Causal knowledge retrieval: Using the information in the structured abnormal data packet as query parameters, extract the causal chain between abnormal state features and abnormal quality indicators in the pre-constructed sintering knowledge graph through a fixed-length path retrieval algorithm; H9-3. Reasoning under mandatory causal constraints: The structured abnormal data packet and the causal chain are input into a large model reasoning module. The large model reasoning module generates structured output under mandatory causal constraints. The structured output includes diagnostic conclusions, evidence chains, confidence levels, and operational suggestions. The large model reasoning module is adapted using a two-stage method of "instructional fine-tuning + constraint deployment". H9-4. Instruction Conversion and Hierarchical Execution: The operation suggestions are converted into structured instruction objects, and hierarchical control logic is executed according to the confidence level and the preset operation risk level.

[0008] Furthermore, the time-series process parameters mentioned in H1 include information on the composition of the mixed material, the instantaneous feed rate of the raw material, the water distribution ratio, the ignition temperature, the thickness of the material layer, the trolley speed, the negative pressure of the air box, the air box temperature, the composition of the exhaust gas, and the corresponding sinter quality test data; the visual feature vector mentioned in H6 includes at least the average thickness of the red-fire layer, the area ratio of the red-fire layer, the average brightness of the red-fire layer, and the continuity index of the red-fire layer.

[0009] Furthermore, the screening described in H3 includes: setting a fixed region of interest in the tail section video frame, calculating the average brightness of each frame in the fixed region of interest; constructing a brightness change rate sequence; determining that the red fire layer event has started when the brightness change rate first exceeds a preset threshold; and selecting the frame whose brightness reaches a local maximum and whose absolute value of the brightness change rate is less than a stable threshold as the best tail section image for the unloading cycle within the preset time window corresponding to the triggering event.

[0010] Furthermore, the semantic segmentation model of the red fire layer described in H5 is an improved U-Net model. The improved U-Net model uses a pre-trained EfficientNet-B4 as the encoder, an attention-gated decoder, and is trained using Dice loss jointly supervised by the main output and auxiliary output.

[0011] Furthermore, the fusion described in H7 is an attention-weighted fusion, which includes: concatenating the temporal feature vector with the visual feature vector to form a joint feature vector; calculating the importance score of each dimension of the joint feature vector; obtaining the attention weight of each dimension through Softmax normalization; and weighting the joint feature vector according to the attention weight to obtain the fused feature vector.

[0012] Furthermore, the optimization algorithm described in H8 is the whale optimization algorithm. The LSTM hyperparameters optimized by the optimization algorithm include the number of hidden layer neurons, learning rate, dropout rate, number of training rounds, and batch size. The optimization objective is to minimize the mean squared error on the validation set.

[0013] Furthermore, the structured anomaly data packet described in H9-1 includes at least: anomaly indicator name, predicted value, threshold, over-limit direction, visual feature vector corresponding to the current time, and time-series feature statistics.

[0014] Furthermore, the method for constructing the sintering knowledge graph described in H9-2 includes: defining concept types and relation types in the sintering domain based on ontology methods; extracting entities and relations from structured databases and unstructured texts and storing them in a graph database; and limiting the path length to 1 to 3 hops using the fixed-length path retrieval algorithm described in H9-2.

[0015] Furthermore, the two-stage approach of "instructional fine-tuning + restrictive deployment" described in H9-3 is adapted to include: During the instructional fine-tuning phase, a triplet dataset containing instructions, inputs, and outputs was used for fine-tuning. The inputs included anomaly predictions, current state features, and causal knowledge chains, while the outputs were natural language paragraphs with titles. The aim was to enable the model to learn evidence-based causal reasoning abilities. During the constrained deployment phase, by designing specialized system prompts, the model is required to output structured objects that strictly follow the preset JSON Schema during inference, which include diagnostic conclusions, confidence levels, chains of evidence, and lists of executable operation instructions; The reasoning under the mandatory causal constraint described in H9-3 also includes: After the model outputs, its diagnostic conclusions and evidence chain fields are parsed and semantic consistency is verified with the input causal chain. If the consistency score is lower than the preset threshold, the output will be rejected and the system will revert to the human expert intervention mode. The operation commands output by the model are subject to domain restrictions, requiring that the parameter names in the commands must belong to the preset list of controllable parameters; otherwise, a manual confirmation process is triggered. The hierarchical control logic described in H9-4 also includes: Obtain the confidence score (output by the large model); Establish an operation-risk level mapping table to classify executable operations into three levels: low risk, medium risk, and high risk; Decision-making logic: If the confidence level is greater than or equal to the first threshold and the operational risk level is low, the instruction will be automatically sent to the PLC system via the API interface; if the confidence level is between the first and second thresholds or the operational risk level is medium, an instruction to be confirmed will be generated and pushed to the operator interface; if the confidence level is less than or equal to the second threshold or the operational risk level is high, no automatic instruction will be generated, and only a diagnostic report will be output for manual reference.

[0016] A sinter quality prediction and closed-loop control device, the device being used to implement the above-mentioned method, specifically comprising: The acquisition module is used to synchronously acquire the timing process parameters and tail section video stream data of the sintering process. The processing module is used to preprocess the timing process parameters; The filtering module is used to filter the best tail section image from the tail section video stream; The standardization module is used to standardize the best tail section image; The segmentation module is used to obtain the binary mask of the red fire layer using the red fire layer semantic segmentation model; The module is used to extract visual feature vectors based on the binary mask of the red flame layer; The fusion module is used to fuse temporal feature vectors and visual feature vectors; The prediction module is used to obtain the predicted results of sinter quality indicators by using an LSTM quality prediction model that is automatically optimized using an optimization algorithm. The chain-based diagnostic control module is used to execute the "prediction-retrieval-reasoning-execution" chain architecture when the prediction result is abnormal, and to generate and execute a closed-loop control scheme.

[0017] A computer storage medium storing a computer program, the computer program being executed by a processor using the method described above.

[0018] An electronic device includes a memory and a processor, the processor executing executable instructions in the memory to implement the method described above.

[0019] The present invention has the following advantages over the prior art: This invention constructs a closed-loop "prediction-retrieval-reasoning-execution" chain, linking quality prediction, knowledge retrieval, large-scale model reasoning, and industrial control, achieving seamless integration from state perception to automatic regulation. Through a forced causal constraint generation mechanism, the output range of the large model is limited to the causal chain provided by the knowledge graph, effectively suppressing the "illusion" problem common in industrial diagnostics and making diagnostic conclusions traceable and verifiable. A two-stage adaptation strategy of "teaching fine-tuning + constraint deployment" allows the large model to not only master causal reasoning capabilities but also output in a structured format. Combined with an instruction conversion module, natural language suggestions can be automatically parsed and executed by the industrial system. The introduction of structured abnormal data packets to automatically trigger diagnosis eliminates manual intervention, significantly shortening the response time from anomaly occurrence to diagnostic reporting. Finally, hierarchical execution control based on confidence and risk levels leverages the automation advantages of the large model while retaining redundancy for manual intervention, providing a feasible path for the safe implementation of generative AI in industrial control.

[0020] This invention deeply couples prediction models, knowledge graphs, large models, and industrial control systems, solving the problem of chain breakage where prediction results cannot be automatically converted into executable decisions, and realizing reliable diagnosis and safe closed-loop control of sintering quality. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the intelligent optimization method of the present invention. Detailed Implementation

[0023] To better understand the technical solution of the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0024] It should be understood that the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0025] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, provides a method for predicting and controlling sintered ore quality based on a chain architecture of "prediction-retrieval-reasoning-execution." It should be noted that, to avoid obscuring the invention with unnecessary detail, the drawings only show structures and / or processing steps closely related to the present invention, while omitting other details less relevant to the invention. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0026] Example 1

[0027] A method for intelligent optimization of sintering quality that integrates multimodal perception and knowledge reasoning includes the following steps: H1. Acquire timing process parameters and tail section video stream data during the sintering process; H2. Preprocess the timing process parameters to obtain processed timing process parameters, and obtain a timing feature vector based on the processed timing process parameters; H3. Select the optimal tail section image corresponding to the unloading cycle from the tail section video stream data; H4. Preprocess the optimal tail section image to obtain a standardized section image; H5. Input the standardized cross-sectional image into the red fire layer semantic segmentation model to obtain the red fire layer binary mask; H6. Extract visual feature vectors based on the binary mask of the red flame layer; H7. The temporal feature vector and the visual feature vector are fused to form a fused feature vector; H8. The fused feature vectors from multiple consecutive sampling times are combined to form a fused feature sequence. The fused feature sequence is then input into an LSTM quality prediction model that is automatically optimized by an optimization algorithm to obtain the sinter quality index prediction results. H9. When the prediction result exceeds the preset threshold, the "prediction-retrieval-reasoning-execution" chain architecture is triggered to generate and execute a quality anomaly diagnosis and closed-loop control scheme.

[0028] Furthermore, H9 specifically includes: H9-1. Automatic Triggering of Diagnosis and Generation of Structured Abnormal Data Packets: When the prediction result exceeds a preset threshold, a structured abnormal data packet is automatically generated; H9-2. Causal knowledge retrieval: Using the information in the structured abnormal data packet as query parameters, extract the causal chain between abnormal state features and abnormal quality indicators in the pre-constructed sintering knowledge graph through a fixed-length path retrieval algorithm; H9-3. Reasoning under mandatory causal constraints: The structured abnormal data packet and the causal chain are input into a large model reasoning module. The large model reasoning module generates structured output under mandatory causal constraints. The structured output includes diagnostic conclusions, evidence chains, confidence levels, and operational suggestions. The large model reasoning module is adapted using a two-stage method of "instructional fine-tuning + constraint deployment". H9-4. Instruction Conversion and Hierarchical Execution: The operation suggestions are converted into structured instruction objects, and hierarchical control logic is executed according to the confidence level and the preset operation risk level.

[0029] Furthermore, the time-series process parameters mentioned in H1 include information on the composition of the mixed material, the instantaneous feed rate of the raw material, the water distribution ratio, the ignition temperature, the thickness of the material layer, the trolley speed, the negative pressure of the air box, the air box temperature, the composition of the exhaust gas, and the corresponding sinter quality test data; the visual feature vector mentioned in H6 includes at least the average thickness of the red-fire layer, the area ratio of the red-fire layer, the average brightness of the red-fire layer, and the continuity index of the red-fire layer.

[0030] Furthermore, the screening described in H3 includes: setting a fixed region of interest in the tail section video frame, calculating the average brightness of each frame in the fixed region of interest; constructing a brightness change rate sequence; determining that the red fire layer event has started when the brightness change rate first exceeds a preset threshold; and selecting the frame whose brightness reaches a local maximum and whose absolute value of the brightness change rate is less than a stable threshold as the best tail section image for the unloading cycle within the preset time window corresponding to the triggering event.

[0031] Furthermore, the semantic segmentation model of the red fire layer described in H5 is an improved U-Net model. The improved U-Net model uses a pre-trained EfficientNet-B4 as the encoder, an attention-gated decoder, and is trained using Dice loss jointly supervised by the main output and auxiliary output.

[0032] Furthermore, the fusion described in H7 is an attention-weighted fusion, which includes: concatenating the temporal feature vector with the visual feature vector to form a joint feature vector; calculating the importance score of each dimension of the joint feature vector; obtaining the attention weight of each dimension through Softmax normalization; and weighting the joint feature vector according to the attention weight to obtain the fused feature vector.

[0033] Furthermore, the optimization algorithm described in H8 is the whale optimization algorithm. The LSTM hyperparameters optimized by the optimization algorithm include the number of hidden layer neurons, learning rate, dropout rate, number of training rounds, and batch size. The optimization objective is to minimize the mean squared error on the validation set.

[0034] Furthermore, the structured anomaly data packet described in H9-1 includes at least: anomaly indicator name, predicted value, threshold, over-limit direction, visual feature vector corresponding to the current time, and time-series feature statistics.

[0035] Furthermore, the method for constructing the sintering knowledge graph described in H9-2 includes: defining concept types and relation types in the sintering domain based on ontology methods; extracting entities and relations from structured databases and unstructured texts and storing them in a graph database; and limiting the path length to 1 to 3 hops using the fixed-length path retrieval algorithm described in H9-2.

[0036] Furthermore, the two-stage approach of "instructional fine-tuning + restrictive deployment" described in H9-3 is adapted to include: During the instructional fine-tuning phase, a triplet dataset containing instructions, inputs, and outputs was used for fine-tuning. The inputs included anomaly predictions, current state features, and causal knowledge chains, while the outputs were natural language paragraphs with titles. The aim was to enable the model to learn evidence-based causal reasoning abilities. During the constrained deployment phase, by designing specialized system prompts, the model is required to output structured objects that strictly follow the preset JSON Schema during inference, which include diagnostic conclusions, confidence levels, chains of evidence, and lists of executable operation instructions; The reasoning under the mandatory causal constraint described in H9-3 also includes: After the model outputs, its diagnostic conclusions and evidence chain fields are parsed and semantic consistency is verified with the input causal chain. If the consistency score is lower than the preset threshold, the output will be rejected and the system will revert to the human expert intervention mode. The operation commands output by the model are subject to domain restrictions, requiring that the parameter names in the commands must belong to the preset list of controllable parameters; otherwise, a manual confirmation process is triggered. The hierarchical control logic described in H9-4 also includes: Obtain the confidence score (output by the large model); Establish an operation-risk level mapping table to classify executable operations into three levels: low risk, medium risk, and high risk; Decision-making logic: If the confidence level is greater than or equal to the first threshold and the operational risk level is low, the instruction will be automatically sent to the PLC system via the API interface; if the confidence level is between the first and second thresholds or the operational risk level is medium, an instruction to be confirmed will be generated and pushed to the operator interface; if the confidence level is less than or equal to the second threshold or the operational risk level is high, no automatic instruction will be generated, and only a diagnostic report will be output for manual reference.

[0037] A sinter quality prediction and closed-loop control device based on a chain architecture, the device being used to implement the above-mentioned method, specifically comprising: The acquisition module is used to synchronously acquire the timing process parameters and tail section video stream data of the sintering process. The processing module is used to preprocess the timing process parameters; The filtering module is used to filter the best tail section image from the tail section video stream; The standardization module is used to standardize the best tail section image; The segmentation module is used to obtain the binary mask of the red fire layer using the red fire layer semantic segmentation model; The module is used to extract visual feature vectors based on the binary mask of the red flame layer; The fusion module is used to fuse temporal feature vectors and visual feature vectors; The prediction module is used to obtain the predicted results of sinter quality indicators by using an LSTM quality prediction model that is automatically optimized using an optimization algorithm. The chain-based diagnostic control module is used to execute the "prediction-retrieval-reasoning-execution" chain architecture when the prediction result is abnormal, and to generate and execute a closed-loop control scheme.

[0038] A computer storage medium storing a computer program, the computer program being executed by a processor using the method described above.

[0039] An electronic device includes a memory and a processor, the processor executing executable instructions in the memory to implement the method described above.

[0040] Example 2

[0041] like Figure 1 As shown, the intelligent optimization method for sintering quality integrating multimodal perception and knowledge reasoning of the present invention includes the following steps: H1. Synchronously acquire the timing process parameters and tail section video stream data of the sintering process.

[0042] H2. Perform outlier filtering, missing value imputation, and timestamp alignment on the timing process parameters to obtain the processed timing process parameters.

[0043] H3. Select the optimal tail section image corresponding to the unloading cycle of each vehicle from the tail section video stream data.

[0044] H4. Denoise, enhance contrast, and correct perspective on the best tail section image to obtain a standardized section image.

[0045] H5. Input the standardized cross-sectional image into the improved U-Net red fire layer semantic segmentation model to obtain the red fire layer binary mask.

[0046] H6. Extract the average thickness, area ratio, average brightness, and continuity index of the red flame layer from the binary mask of the red flame layer to form a visual feature vector.

[0047] H7. Extract the processed time-series process parameters within a preset time window centered on the visual feature timestamp, calculate statistical features to obtain a time-series feature vector, and perform attention-weighted fusion of the time-series feature vector and the visual feature vector to form a fused feature vector.

[0048] H8. The fused feature vectors from multiple consecutive sampling times are combined into a fused feature sequence, which is then input into the LSTM quality prediction model that is automatically optimized by the whale optimization algorithm to obtain the prediction results of key quality indicators of sinter.

[0049] H9. When the prediction result exceeds a preset threshold, a chain architecture of "prediction-retrieval-reasoning-execution" is executed, specifically including the following sub-steps: H9-1. Automatically generate a structured anomaly data package containing the name of the anomaly indicator, the predicted value, the threshold, the direction of exceeding the limit, and the statistical values ​​of the current visual features and time series features.

[0050] H9-2. Using the structured abnormal data packet as the query parameter, extract causal chains with a length of 1 to 3 hops from the pre-constructed sintering knowledge graph using a fixed-length path retrieval algorithm.

[0051] H9-3. Input the abnormal data packet and the causal chain together into the large model inference module. Under the mandatory causal constraint mechanism (including inference restrictions based on causal chains, output semantic consistency verification, and instruction parameter field restrictions), generate a structured output containing diagnostic conclusions, confidence levels, evidence chains, and operational suggestions.

[0052] H9-4. Convert operation suggestions into structured JSON instruction objects and implement hierarchical control based on model confidence and preset operation risk level: high confidence and low risk instructions are automatically sent to the PLC system; medium confidence or medium risk instructions are pushed to the operator interface for confirmation; low confidence or high risk instructions only output reports for manual reference.

[0053] S1. Clearly define inputs and outputs The quality prediction model of this invention is the WOA LSTM sinter quality prediction model. Its input includes two types of data: The first type is the time-series process parameters from the field system, including raw material information (instantaneous feed rate, chemical composition, water distribution ratio, etc.), sintering process parameters (ignition temperature, material layer thickness, trolley speed, air box negative pressure, air box temperature, etc.) and corresponding quality test data.

[0054] The second category is video stream data from industrial cameras at the tail of the sintering machine, which is then processed in subsequent steps and converted into visual feature input.

[0055] The purpose of using time-series process parameters is to characterize the batching state, thermal regime, ventilation state and sintering progress during the sintering process; the purpose of using the tail section video stream is to directly characterize the hot state and cross-sectional morphology of the red-fired layer at the sintering endpoint. The two together support high-precision quality prediction.

[0056] The direct output of this invention is the predicted value of key quality indicators for sintered ore, including the chemical composition of sintered ore (FeO, CaO, SiO2, MgO, Al2O3, K2O, Na2O, etc.), strength properties, metallurgical properties, technical and economic indicators, and sintering thermal state information (bed permeability, BTP, BRP, sintering endpoint position, etc.). Furthermore, the deeper output of this invention is a quality anomaly diagnostic report and operational adjustment suggestions generated through a chain-like architecture when quality predictions are abnormal.

[0057] S2. Time-series data acquisition (corresponding to H1-H2) The data collection frequency is defined as once per hour, with 24 sets of data collected daily. Historical and real-time data are read through the production line PLC system, process database, and testing database, and timestamps are uniformly assigned. The collected data includes information on the composition of different batches of mixed materials, instantaneous feed rate, water distribution ratio, ignition temperature, material layer thickness, trolley speed, air box negative pressure, air box temperature, and corresponding sinter quality test results and thermal state information.

[0058] S3. Time series data preprocessing (corresponding to H2) Box plots were used for outlier detection. Let the 25th percentile and 75th percentile be... and Then the interquartile range is: The upper and lower bounds for outlier detection are as follows: in, The outlier detection coefficient is 1.5, preferably selected. Values ​​falling within the interval... Data outside of these categories is recorded as outliers and either removed or corrected.

[0059] For missing values, linear interpolation is used. If adjacent known data points are respectively... and Then at the missing time Parameter value at for: After outlier filtering and missing value completion, time series data from different sources are aligned using a unified timestamp.

[0060] S4. Acquisition and screening of tail section images (corresponding to H3-H4) A high-definition industrial camera (visible light or infrared) is deployed at the unloading end of the sintering machine, facing the unloading section of the trolley. The camera's optical axis is approximately perpendicular to the exposed tail section after unloading. The camera continuously captures video streams of the tail section formed during the trolley unloading process. The images include the red-hot layer area, the unburnt area, and the section boundary information. The video acquisition industrial control computer and the time-series database are synchronized via Network Time Protocol (NTP) to ensure that each frame has a precise timestamp.

[0061] Optimal tail section image selection method: Set a fixed region of interest in the video frame. Calculate the first The average brightness of the frame in this area : in For the first Frame in pixel coordinates The brightness value at that location, For the region The total number of pixels within the frame. A brightness curve is constructed from the average brightness of each frame. Define the rate of change in brightness between adjacent frames. : in , The first Frame and the The frame's timestamp. When First time exceeding the threshold At that time, the event of the red-hot layer appearing is determined to have begun. Then, within the preset time window... The internal search brightness reaches a local maximum and Less than the stability threshold The frame, where Tw is the length of the time window, is 1, meaning that in t n The system searches for the best frame within one second before and after the current trolley unloading cycle and uses that frame as the optimal tail section image. After completing one cycle, it continues searching for the next trigger point from the end of the window, repeating the above steps to obtain the optimal tail section image sequence for different trolley cycles.

[0062] S5. Image preprocessing (corresponding to H4) The best tail section image is then subjected to noise reduction, contrast enhancement, and perspective correction in sequence.

[0063] Noise reduction: A non-local means algorithm is used. For the pixel location to be processed... The denoising result is expressed as: in For the search window, The weights are calculated based on pixel neighborhood similarity and satisfy the following conditions: .

[0064] Contrast-Limited Adaptive Histogram Equalization (CLAHE) is employed to divide the image into several local sub-blocks, calculate the gray-level histograms for each sub-block, and apply cropping constraints. Gray-level remapping is then performed based on the cumulative distribution function after cropping, and finally, bilinear interpolation is applied to the results of adjacent sub-blocks.

[0065] Let the homogeneous coordinates of any point in the image be... The corrected homogeneous coordinates are Then we have: in To be obtained in advance through a calibration plate or fixed reference point Perspective transformation matrix. The resulting image is a standardized cross-sectional image.

[0066] S6. Red Fire Layer Semantic Segmentation (corresponding to H5) Experts in the field of sintering manually annotated historical tail section images: pixels that are continuously high temperature and bright and correspond to the combustion area at the end of sintering are marked as the target area of ​​the red fire layer, and the rest are marked as the background, generating a binary mask image of the red fire layer.

[0067] This invention employs an improved UNet model. The input is processed and uniformly scaled through steps S4 and S5. RGB cross-sectional image, with the real label as The binary mask image of the red fire layer. The model encoder uses an EfficientNetB4 network pre-trained on ImageNet (with the top pooling and classification layers removed); the decoder consists of four cascaded upsampling blocks, and an attention gating module is introduced to weight the skip connection features; at the same time, auxiliary output branches are set in several intermediate decoding layers to enhance intermediate layer supervision.

[0068] Let the set of predicted pixels for the main output of the model be... The actual mask pixel set is Then L dice The loss is: in Smoothing coefficient (preferred) The main output and each auxiliary output are compared with the true labels to calculate the Dice loss, and the weighted sum is used as the total loss. End-to-end training is performed using the Adam optimizer and backpropagation algorithm. After training, the online-acquired and preprocessed tail section image is input into the model, and the main segmentation prediction map (i.e., the red-fire layer binary mask) is output.

[0069] S7. Quantitative Feature Extraction of the Red Flame Layer (corresponding to H6) Let the binary mask image of the red flame layer be... The target area (red fire layer) has a pixel value of 1, while the background has a pixel value of 0. The image height and width are respectively... and .

[0070] Average thickness : Along the image width direction ( ), Count the projected thickness of the red fire layer in the vertical direction column by column. Let the number of valid columns (i.e., the number of columns with at least one red fire layer pixel) be . The average thickness is then: Area percentage Let the total number of target pixels in the red fire layer be The total number of pixels in the image is The area ratio is: Average brightness set up In order to use with the mask The preprocessed cross-sectional image (grayscale values) with strict alignment. The average brightness of the red-hot layer region is: Continuity Index First, find the width of the minimum bounding rectangle of the red-hot layer region. Then, the mask is projected vertically, and the number of columns with projection values ​​greater than 0 is counted. The continuity index is defined as: Based on the above four characteristic quantities , , , Together they constitute the visual feature vector .

[0071] S8. Multimodal feature fusion (corresponding to H7) Let visual feature vectors be defined. Source: timestamp The best cross-sectional image. Extracted from the process database. Centered on, with radius time window The time-series process parameters within the window are used to calculate statistical measures such as the mean, standard deviation, maximum, and minimum values ​​of the data within that window, resulting in a fixed-length time-series feature vector. (dimension is) ).Will and By concatenating the features, a joint feature vector is obtained. : in, The dimension is ,but Dimensions .

[0072] To adaptively evaluate the importance of features in each dimension, let... The The elements are ( Its unnormalized attention score The calculation is as follows: in: A learnable weight matrix (dimensions) In practice, it is applied to each scalar. (This may be expanded or processed element-by-element; this is a simplified explanation). For bias terms (dimensions) ); For the projection vector (dimension) ); It is the hyperbolic tangent activation function.

[0073] After normalization using the Softmax function, the th... Attention weights of dimensional features : Finally, the feature vectors are fused. The weighted sum of features across all dimensions: Output vector The time-series process information and visual morphological information corresponding to the current cross-sectional state are integrated and used as a unified input for the subsequent quality prediction model.

[0074] S9. WOALSTM Quality Prediction Model (corresponding to H8) Let the sequence of fused feature vectors obtained in chronological order be . The LSTM input samples are formed by a sliding time window of length L, and the output is the predicted value of the sinter quality index at the corresponding time point. LSTM models are used to learn the nonlinear temporal mapping between multi-time-step process state evolution and quality indicators.

[0075] The Whale Optimization (WOA) algorithm is used to automatically optimize the key hyperparameters of the LSTM: number of hidden layer neurons, learning rate, dropout rate, number of training epochs, and batch size. The position vector of a whale is denoted as: = [Neurons, LearningRate, DropoutRate, Epochs, BatchSize](17) The mean squared error (MSE) on the validation set is used as the optimization objective (a smaller MSE indicates better performance). During algorithm initialization, N_w whales are randomly generated within a predefined search space. For each whale, an LSTM is trained according to the hyperparameter configuration it represents, and the MSE is calculated on the validation set. During iteration, the position vector is updated using three mechanisms: prey encirclement, spiral update, and random search. When the maximum number of iterations is reached or the validation set MSE converges, the optimal hyperparameter combination X_best is output, and the final LSTM model is retrained using this combination to output the prediction results of key quality indicators for sintered ore.

[0076] S10. Construction of Sintering Knowledge Graph (corresponding to H9-2) A structured knowledge foundation for anomaly diagnosis is constructed. First, core concept classes in the sintering domain are defined based on ontology methods (including at least process parameter classes, equipment classes, state characteristic classes, quality indicator classes, fault cause classes, and control measure classes) and relationship types (including at least "impact," "cause," "association," "constraint," and "adjustment suggestion"). Then, entities and relationships are extracted from two types of data sources: structured data sources (production process database, quality testing database, equipment operation database) are generated through field mapping and ETL; unstructured text (process specifications, equipment manuals, fault handling records, technical documents, expert experience) is supplemented through entity recognition, relationship extraction, and rule validation. After construction, the knowledge is stored in a graph database, supporting rapid graph retrieval and relational path reasoning.

[0077] S11. Two-stage adaptation of large models and forced causal constraint reasoning (corresponding to H9-3) ChatGLM was selected as the base model for natural language understanding, knowledge reasoning, and diagnostic report generation. To ensure the model possesses expertise in the sintering field, a two-stage approach of "instructional fine-tuning + constrained deployment" was adopted.

[0078] In the instructional fine-tuning phase: Process principle documents, operating procedures, fault case libraries, technical literature, and expert experience texts are collected to construct a "command input / output" triple dataset. The input includes anomaly predictions, current state features, and causal knowledge chains; the output is a natural language paragraph with a title (e.g., "[Diagnostic Conclusion]", "[Evidence Chain]", "[Operational Recommendation]"). During training, low-rank adaptive (LoRA) parameters are introduced into the Query, Key, Value, and Output projection matrices in the Transformer layer of the model. The original parameters of the base model are frozen, and only the LoRA adapter is updated, injecting domain knowledge at a lower cost.

[0079] Constraint Deployment Phase: Through specially designed system prompts, the model output is required to strictly follow the preset JSONSchema structured object, including fields such as diagnosis, confidence, evidence chain, and actions.

[0080] Forced causal constraint generation mechanism: When constructing the input for the large model, the structured causal chain output by the causal retrieval module is injected as an independent field into the prompt words, and it is explicitly required that "diagnostic conclusions and operational suggestions must be strictly based on the given causal chain and must not introduce external irrelevant causes."

[0081] After the model outputs, its diagnostic conclusions and evidence chain fields are parsed and compared with the input causal chain for keyword matching and semantic consistency verification. If the consistency score is lower than a preset threshold (e.g., 0.6), the output is rejected and the system reverts to the human expert intervention mode.

[0082] The operation commands output by the model are subject to domain restrictions, requiring that the parameter names in the commands must belong to a preset list of controllable parameters (such as bellows negative pressure, trolley speed, etc.). Parameters that do not belong to the list will automatically trigger a manual confirmation process.

[0083] S12. Tiered Implementation Based on Confidence Level and Risk (corresponding to H9-4) To achieve safe and reliable command issuance, a hierarchical execution control method is proposed: Obtain the confidence score (0-1) of the large model output.

[0084] Establish an operational risk level mapping table to classify executable operations into: low risk L1 (such as adjusting the negative pressure of the bellows and the speed of the trolley), medium risk L2 (such as adjusting the thickness of the material layer and the water distribution ratio), and high risk L3 (such as changing the type of ore and stopping or starting the machine).

[0085] Decision-making logic: • If the confidence level is ≥ 0.9 and the operational risk level is L1, the instruction will be automatically sent to the PLC system via the API interface; • If 0.7 ≤ confidence level < 0.9 or the operational risk level is L2, a confirmation instruction will be generated, pushed to the operator interface, and highlighted. • If the confidence level is < 0.7 or the operational risk level is L3, no automatic instructions will be generated; only a diagnostic report will be output for manual reference.

[0086] S13. Implementation of closed-loop control The above steps together constitute a complete closed loop of "prediction, retrieval, reasoning, and execution." The prediction result receiving module obtains the quality prediction value output from step S9; the anomaly triggering module determines whether the threshold is exceeded; when an anomaly occurs, the diagnostic reasoning module sends the anomaly indicators, relevant time-series process parameters, red-fire layer visual features, and causal chains to the sintering expert brain (fine-tuned large model + knowledge graph retrieval module + prompt word construction module); the expert brain outputs a structured diagnostic report and operational suggestions; the hierarchical execution module issues instructions or pushes pending confirmation based on confidence level and risk level; the adjusted process state is collected, fused, and predicted again, forming a continuously optimized closed-loop control.

[0087] Devices, media and equipment This invention also provides a sinter quality prediction and closed-loop control device for implementing the above method, specifically including: Acquisition module: Used to synchronously acquire timing process parameters and tail section video stream data of the sintering process; Processing module: used to perform outlier filtering, missing value imputation, and timestamp alignment on the timing process parameters; Filtering module: Used to filter the best tail section image from the tail section video stream; Standardization module: used for denoising, contrast enhancement, and perspective correction of the best tail section image; Segmentation module: Used to input standardized cross-sectional images into the improved U-Net model to obtain a binary mask for the red-fire layer; Fusion module: used to perform attention-weighted fusion of temporal feature vectors and visual feature vectors; Prediction module: Used to obtain sinter quality prediction results using the WOA LSTM model; Chain-based diagnostic control module: When the prediction result is abnormal, it executes the "prediction, retrieval, reasoning, and execution" chain architecture to generate and execute a closed-loop control scheme.

[0088] The present invention also provides a computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any of the preceding claims.

[0089] The present invention also provides an electronic device, including a memory and a processor, wherein the processor executes executable instructions in the memory to implement the method described in any of the preceding claims. The embodiments of the present invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for intelligent optimization of sintering quality integrating multimodal perception and knowledge reasoning, characterized in that, Includes the following steps: H1. Acquire timing process parameters and tail section video stream data during the sintering process; H2. Preprocess the timing process parameters to obtain processed timing process parameters, and obtain a timing feature vector based on the processed timing process parameters; H3. Select the optimal tail section image corresponding to the unloading cycle from the tail section video stream data; H4. Preprocess the optimal tail section image to obtain a standardized section image; H5. Input the standardized cross-sectional image into the red fire layer semantic segmentation model to obtain the red fire layer binary mask; H6. Extract visual feature vectors based on the binary mask of the red flame layer; H7. The temporal feature vector and the visual feature vector are fused to form a fused feature vector; H8. The fused feature vectors from multiple consecutive sampling times are combined to form a fused feature sequence. The fused feature sequence is then input into an LSTM quality prediction model that is automatically optimized by an optimization algorithm to obtain the sinter quality index prediction results. H9. When the prediction result exceeds the preset threshold, the "prediction-retrieval-reasoning-execution" chain architecture is triggered to generate and execute a quality anomaly diagnosis and closed-loop control scheme.

2. The method according to claim 1, characterized in that, H9 specifically includes: H9-1. Automatic Triggering of Diagnosis and Generation of Structured Abnormal Data Packets: When the prediction result exceeds a preset threshold, a structured abnormal data packet is automatically generated; H9-2. Causal knowledge retrieval: Using the information in the structured abnormal data packet as query parameters, extract the causal chain between abnormal state features and abnormal quality indicators in the pre-constructed sintering knowledge graph through a fixed-length path retrieval algorithm; H9-3. Reasoning under a mandatory causal constraint mechanism: The structured abnormal data packet and the causal chain are input into a large model reasoning module. The large model reasoning module generates structured output under mandatory causal constraints. The structured output includes diagnostic conclusions, evidence chains, confidence levels, and operational suggestions. The large model reasoning module is adapted using a two-stage method of "instructional fine-tuning + constraint deployment". H9-4. Instruction Conversion and Hierarchical Execution: The operation suggestions are converted into structured instruction objects, and hierarchical control logic is executed according to the confidence level and the preset operation risk level.

3. The method according to claim 2, characterized in that, The time-series process parameters mentioned in H1 include information on the composition of the mixed material, the instantaneous feed rate of the raw material, the water distribution ratio, the ignition temperature, the thickness of the material layer, the speed of the trolley, the negative pressure of the air box, the air box temperature, the composition of the exhaust gas, and the corresponding sinter quality test data; the visual feature vector mentioned in H6 includes at least the average thickness of the red-fire layer, the area ratio of the red-fire layer, the average brightness of the red-fire layer, and the continuity index of the red-fire layer.

4. The method according to claim 3, characterized in that, The filtering described in H3 includes: setting a fixed region of interest in the tail section video frame, calculating the average brightness of each frame in the fixed region of interest; constructing a brightness change rate sequence; determining that the red fire layer event has started when the brightness change rate first exceeds a preset threshold; and selecting the frame whose brightness reaches a local maximum and whose absolute value of the brightness change rate is less than a stable threshold as the best tail section image for the unloading cycle within the preset time window corresponding to the triggering event.

5. The method according to claim 4, characterized in that, The semantic segmentation model of the red fire layer described in H5 is an improved U-Net model. The improved U-Net model uses a pre-trained EfficientNet-B4 as the encoder, an attention-gated decoder, and is trained using Dice loss jointly supervised by the main output and auxiliary output.

6. The method according to claim 5, characterized in that, The fusion described in H7 is an attention-weighted fusion, which includes: concatenating the temporal feature vector with the visual feature vector to form a joint feature vector; calculating the importance score of each dimension of the joint feature vector; obtaining the attention weight of each dimension through Softmax normalization; and weighting the joint feature vector according to the attention weight to obtain the fused feature vector.

7. The method according to claim 6, characterized in that, The structured anomaly data packet described in H9-1 includes at least: anomaly indicator name, predicted value, threshold, over-limit direction, visual feature vector corresponding to the current time, and time-series feature statistics.

8. The method according to claim 7, characterized in that, The method for constructing the sintering knowledge graph described in H9-2 includes: defining concept types and relation types in the sintering domain based on ontology methods; extracting entities and relations from structured databases and unstructured texts and storing them in a graph database; and limiting the path length of the fixed-length path retrieval algorithm described in H9-2 to 1 to 3 hops.

9. A sinter quality prediction and closed-loop control device, characterized in that, The apparatus is used to implement the method according to any one of claims 1-8, specifically including: The acquisition module is used to synchronously acquire the timing process parameters and tail section video stream data of the sintering process. The processing module is used to preprocess the timing process parameters; The filtering module is used to filter the best tail section image from the tail section video stream; The standardization module is used to standardize the best tail section image; The segmentation module is used to obtain the binary mask of the red fire layer using the red fire layer semantic segmentation model; The module is used to extract visual feature vectors based on the binary mask of the red flame layer; The fusion module is used to fuse temporal feature vectors and visual feature vectors; The prediction module is used to obtain the predicted results of sinter quality indicators by using an LSTM quality prediction model that is automatically optimized using an optimization algorithm. The chain-based diagnostic control module is used to execute the "prediction-retrieval-reasoning-execution" chain architecture when the prediction result is abnormal, and to generate and execute a closed-loop control scheme.

10. A computer storage medium, characterized in that, The medium stores a computer program, which is executed by a processor to implement the method described in any one of claims 1-8.

Images