A method and device for evaluating ironmaking product quality based on multi-source data

Abstract

This application discloses a method and device for evaluating the quality of ironmaking products based on multi-source data, which relates to the field of image processing, including: inputting a feature map of the ironmaking product to be evaluated into an encoder to obtain multiple intermediate coding features and target coding features, jump-connecting the multiple intermediate coding features into a multi-scale feature fusion module to obtain multiple fused coding features, and inputting the target coding features into a parallel information focusing module to obtain parallel information focusing features, jump-connecting the multiple fused coding features and then inputting them into a decoder with the parallel information focusing features to obtain multiple candidate product features, analyzing the multiple candidate product features to obtain an output image of the ironmaking product to be evaluated, obtaining the wrinkle-smoothed area ratio in the output image, determining the smooth quality of the ironmaking product to be evaluated, and performing defect evaluation statistics on multiple continuous smooth qualities collected by video to obtain a quality defect evaluation of the ironmaking product. This application improves segmentation performance.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and device for evaluating the quality of ironmaking products based on multi-source data. Background Art

[0002] With the increasing speed of iron production lines and the continuous improvement of quality requirements, manual visual inspection methods can no longer meet quality assurance requirements. On the other hand, with the rapid development of information technology in recent years, online inspection technology based on machine vision has reached or even exceeded manual inspection in terms of detection accuracy. It also has advantages over manual inspection in areas such as fatigue resistance, and can also realize the informatization of quality inspection and quality management.

[0003] While existing technologies have made some progress in improving the segmentation performance of ironwork products, they still face the following shortcomings: Traditional ironwork product segmentation methods rely on manual image annotation by professionals, a time-consuming and labor-intensive process. Annotation results are influenced by experience and judgment, especially in low-contrast areas and complex ironwork product structures. Segmentation results can suffer from small product sensitivity, multi-scale adaptability, and interference from diseased areas. Existing methods still need to improve their ability to extract fine-grained and complex features of ironwork products. There is a lack of effective dynamic adaptive processing mechanisms for the complex morphology and defective areas of ironwork products. Existing methods are still insufficient in balancing global contextual information with local detail features. Summary of the Invention

[0004] In view of the above problems, this application provides a method and device for ironmaking product quality evaluation based on multi-source data to achieve the effective fusion of global and local features, balance the relationship between global context information and local detail features, enhance the segmentation robustness in complex product morphology and lesion areas, and capture the fine-grained and complex features of the product to improve segmentation performance. The specific solution is as follows:

[0005] In a first aspect, the present application provides a method for evaluating the quality of ironmaking products based on multi-source data, comprising:

[0006] Collecting a feature image of the ironmaking product to be evaluated, inputting the feature image into an encoder to obtain a plurality of intermediate coding features and a target coding feature;

[0007] Inputting the plurality of intermediate coding features into a multi-scale feature fusion module through jump connections to obtain a plurality of corresponding fused coding features, and inputting the target coding features into a parallel information focusing module to obtain a parallel information focusing feature;

[0008] Jump-connecting the multiple fused coding features and inputting them together with the parallel information focused features into a decoder to obtain multiple candidate product features;

[0009] Obtaining an output image of the ironmaking product to be evaluated based on feature analysis of the plurality of candidate products;

[0010] Obtaining a wrinkle smoothing area ratio in the output image to determine the smoothing quality of the ironmaking product to be evaluated;

[0011] Defect evaluation statistics are performed on the smooth quality of multiple consecutive ironmaking products to be evaluated captured by video to obtain a quality defect evaluation of the ironmaking products.

[0012] In a possible implementation, the encoder includes:

[0013] Adopting adaptive gated residual blocks, combined with residual connections and adaptive gating mechanisms, and introducing spatial random dropout modules to alleviate overfitting;

[0014] In the encoder, the adaptive gated residual block contains two convolutional layers and their corresponding batch normalization and activation layers.

[0015] In a possible implementation, inputting the feature image into an encoder to obtain a plurality of intermediate coding features and a target coding feature further includes:

[0016] The first feature of the feature map is processed by the first convolutional layer and subjected to batch normalization and the ReLU activation function of the activation layer to obtain a first intermediate coding feature;

[0017] Passing the first intermediate coding feature through the second convolutional layer and the batch normalization process again to obtain a second intermediate coding feature;

[0018] The spatial random dropout module processes the second intermediate coding feature to reduce noise in the feature map and enhance regularization effect;

[0019] The adaptive gating mechanism of the adaptive gating residual block extracts global context information through global average pooling, and generates gating coefficients through two fully connected layers and Sigmoid activation function to adjust the weight of the residual term;

[0020] The residual connection part performs channel matching on the first feature to ensure that the number of channels is consistent with the main branch feature, and then performs weighted addition with the second intermediate coding feature, and obtains the corresponding target coding feature through the ReLU activation function.

[0021] In a possible implementation, inputting the target coding feature into a parallel information focusing module to obtain a parallel information focusing feature includes:

[0022] A channel attention module is used to aggregate the global features of each channel, generate channel importance weights, and adjust the channel expression ability of the input feature map;

[0023] Input the target encoding features, extract the global descriptor of each channel through the global average pooling operation, perform feature compression and expansion through a two-layer fully connected network, and generate channel weights;

[0024] The generated channel weight is multiplied element-by-element by the input target encoding feature to complete the enhancement in the channel dimension;

[0025] The spatial attention module captures global spatial information and generates spatial attention weights by performing maximum pooling and average pooling on the target encoding features in the spatial dimension.

[0026] Performing maximum pooling and average pooling operations on the target coding features in the spatial dimension to generate two spatial feature maps;

[0027] The two spatial feature maps are concatenated in the spatial dimension, and the result is passed through a 7×7 convolutional layer to generate spatial attention weights;

[0028] The spatial weight is multiplied element-by-element by the input target encoding feature to enhance the spatial dimension;

[0029] After completing the channel and spatial attention optimization, the final parallel information focusing features are generated through a fusion operation.

[0030] In a possible implementation, the skip-connecting and inputting the multiple intermediate coding features into a multi-scale feature fusion module to obtain corresponding multiple fused coding features includes:

[0031] Preset the skip connection features of the encoder, perform 3×3 convolution, batch normalization and ReLU activation operations on each intermediate encoding feature to obtain a unified processed feature;

[0032] In each decoding layer, the intermediate encoded features are bilinearly interpolated to adjust the target resolution;

[0033] The Transformer’s attention mechanism is used to capture the dependencies between multi-scale features, stack all adjusted features along the new dimension, and reshape them into sequence representations;

[0034] Reshape the features into a tensor of the target shape and further permute the dimensions to meet the input requirements of the Transformer encoder;

[0035] After passing through the Transformer encoder, the global context information is integrated and multiple fused coding features are output.

[0036] In a possible implementation, the jump-connecting of the multiple fused coding features and inputting the parallel information focused features into a decoder together to obtain multiple candidate product features includes:

[0037] Adaptive gated residual upsampling block is used to include upsampling and skip connection operations;

[0038] The input parallel information focusing features are upsampled to half the size of the feature map through bilinear interpolation, and then spliced with the feature map of the corresponding layer in the multi-scale feature fusion module to integrate the semantic information of the encoding stage;

[0039] The spliced feature maps are then processed by an adaptive gated residual block to enhance the feature fusion effect and restore image details to obtain multiple candidate product features.

[0040] In a possible implementation, obtaining the wrinkle smoothness area ratio in the output image and determining the smoothness quality of the ironmaking product to be evaluated includes:

[0041] Obtaining the number N1 of rough feature points and N2 of smooth feature points in the output image;

[0042] The calculated ratio of the rough feature points to the total feature points and the ratio of the hole area on the surface of the ironmaking product are expressed as the surface wrinkle smooth area ratio K, and the calculation formula is as follows:

[0043]

[0044] Where: K-surface wrinkle smooth area ratio (%);

[0045] N1-number of rough feature points;

[0046] N2-number of smoothing feature points;

[0047] Determining the smoothness quality of the ironmaking product to be evaluated based on the surface wrinkle smoothness area ratio;

[0048] The smooth quality includes excellent ironmaking products, good ironmaking products, qualified ironmaking products, and unqualified ironmaking products.

[0049] A second aspect of the present application provides an ironmaking product quality evaluation device based on multi-source data, comprising:

[0050] one or more processors;

[0051] A memory having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for evaluating the quality of ironmaking products based on multi-source data provided in the first aspect of the present application and any possible implementation of the first aspect.

[0052] By means of the above technical solution, the present application provides a method and device for evaluating the quality of ironmaking products based on multi-source data, which adopts a strong feature dynamic fusion gated U-Net. By introducing the centrality idea in graph theory, the core module is designed as the central node, and the key information of the multi-layer encoder is dynamically aggregated, thereby realizing the effective fusion of global and local features. The feature fusion hub module balances the relationship between global context information and local detail features by utilizing the long-range dependency of multi-layer features. The adaptive gated residual block uses a dynamic gating mechanism to screen key features and suppress redundant information, thereby enhancing the segmentation robustness in complex product morphologies and pathological areas. The parallel information focusing module combines channel and spatial attention mechanisms, effectively models global position information and long-range dependencies, captures fine-grained and complex features of the product, and further improves the segmentation performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0054] Figure 1 A flowchart of a method for evaluating the quality of ironmaking products based on multi-source data as claimed in an embodiment of the present application;

[0055] Figure 2 This is a diagram showing the structure of an adaptive gated residual block for an ironmaking product quality evaluation method based on multi-source data as claimed in an embodiment of the present application;

[0056] Figure 3 A structural diagram of a parallel information focusing module of an ironmaking product quality evaluation method based on multi-source data as claimed in an embodiment of the present application;

[0057] Figure 4 A feature navigation hub structure diagram of a method for evaluating the quality of ironmaking products based on multi-source data as claimed in an embodiment of the present application;

[0058] Figure 5 A Transformer structure diagram of a method for evaluating the quality of ironmaking products based on multi-source data as claimed in an embodiment of the present application;

[0059] Figure 6 This is a structural diagram of an adaptive gated residual upsampling block of an ironmaking product quality evaluation method based on multi-source data as claimed in an embodiment of the present application. DETAILED DESCRIPTION

[0060] The following describes the embodiments of the present application in conjunction with the accompanying drawings. The terms used in the implementation methods of the present application are only used to explain the specific embodiments of the present application and are not intended to limit the present application.

[0061] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0062] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0063] According to the first embodiment of the present invention, the present invention claims a method for evaluating the quality of ironmaking products based on multi-source data, referring to Figure 1 ,include:

[0064] Collecting a feature map of the ironmaking product to be evaluated, inputting the feature map into an encoder to obtain multiple intermediate coding features and a target coding feature;

[0065] Inputting the plurality of intermediate coding features into a multi-scale feature fusion module through jump connections to obtain a plurality of corresponding fused coding features, and inputting the target coding features into a parallel information focusing module to obtain a parallel information focusing feature;

[0066] Jump-connecting the multiple fused coding features and inputting them together with the parallel information focused features into a decoder to obtain multiple candidate product features;

[0067] Obtaining an output image of the ironmaking product to be evaluated based on feature analysis of the plurality of candidate products;

[0068] Obtaining a wrinkle smoothing area ratio in the output image to determine the smoothing quality of the ironmaking product to be evaluated;

[0069] Defect evaluation statistics are performed on the smooth quality of multiple consecutive ironmaking products to be evaluated captured by video to obtain a quality defect evaluation of the ironmaking products.

[0070] The main goal of this invention is to overcome the shortcomings of existing technologies in ironmaking product segmentation through multiple innovative technologies, particularly addressing the issues of sensitivity to small products, multi-scale adaptability, and interference from diseased areas. Through three core modules—an adaptive gated residual block, a parallel information focusing module, and a feature navigation hub—the model comprehensively optimizes and improves data feature extraction, application of attention mechanisms, and multi-scale feature fusion, significantly improving the accuracy and robustness of ironmaking product segmentation.

[0071] In response to the problems of interference from lesion areas and poor adaptability of multi-scale products in the segmentation of ironmaking products, the present invention proposes an adaptive gated residual block (AGRB). This module combines residual connections and adaptive gating mechanisms, significantly improving the model's performance in multi-scale product feature extraction and enhancement, thereby enhancing its sensitivity to products of different scales. In order to further avoid overfitting, AGRB also introduces a spatial random inactivation (Spatial Dropout) module to enhance the generalization ability and robustness of the model. In the encoder part, AGRB is responsible for extracting and compressing multi-scale features layer by layer, while in the decoder, the adjusted AGRB is used to restore image resolution, ensuring accurate segmentation of products of different scales and fine restoration of details.

[0072] In terms of feature fusion and global modeling, this paper dynamically integrates multi-scale features from the encoder through a feature navigation hub (FN-Hub) combined with a Transformer-based global attention mechanism. This module effectively captures long-range dependencies between multi-layer features, promotes efficient fusion of global and local features, and significantly improves the model's ability to process global contextual information for both coarse and fine products.

[0073] To address the problem of extracting fine-grained and complex product features, this paper designs a Parallel Information Focusing Module (PFAM). By fusing channel-attention and spatial-attention mechanisms, this module enhances useful features in both the channel and spatial dimensions, thereby improving feature expression and fine-grained feature capture. Through this parallel optimization, PFAM effectively improves the ability to extract fine-grained and complex product features, further enhancing segmentation accuracy and robustness.

[0074] The synergistic effect of these modules significantly improves the model's performance in complex product morphologies and low-contrast scenes, enabling EFDG-UNet to excel in small product sensitivity, robustness to lesion areas, and overall segmentation performance, especially in complex product areas, demonstrating efficient and robust segmentation capabilities.

[0075] Furthermore, the encoder comprises:

[0076] Adopting adaptive gated residual blocks, combined with residual connections and adaptive gating mechanisms, and introducing spatial random dropout modules to alleviate overfitting;

[0077] In the encoder, the adaptive gated residual block contains two convolutional layers and their corresponding batch normalization and activation layers.

[0078] Furthermore, the step of inputting the feature map into an encoder to obtain a plurality of intermediate encoding features and a target encoding feature further includes:

[0079] The first feature of the feature map is processed by the first convolutional layer and subjected to batch normalization and the ReLU activation function of the activation layer to obtain a first intermediate coding feature;

[0080] Passing the first intermediate coding feature through the second convolutional layer and the batch normalization process again to obtain a second intermediate coding feature;

[0081] The spatial random dropout module processes the second intermediate coding feature to reduce noise in the feature map and enhance regularization effect;

[0082] The adaptive gating mechanism of the adaptive gating residual block extracts global context information through global average pooling, and generates gating coefficients through two fully connected layers and Sigmoid activation function to adjust the weight of the residual term;

[0083] The residual connection part performs channel matching on the first feature to ensure that the number of channels is consistent with the main branch feature, and then performs weighted addition with the second intermediate coding feature, and obtains the corresponding target coding feature through the ReLU activation function.

[0084] Among them, reference Figure 2 In this embodiment, in the encoder, the core structure of AGRB consists of two convolutional layers and their corresponding batch normalization and activation layers. First, the input feature map x is processed by the first convolutional layer and then undergoes batch normalization and ReLU activation function to obtain the intermediate feature x1, as shown in Formula 1.

[0085] x1=ReLU(BN1(Conv1(x))) (1);

[0086] Next, the intermediate feature x1 is processed again through the second convolutional layer and batch normalization to obtain the feature x2. The process is shown in Formula 2.

[0087] x2=BN2(Conv2(x1)) (2);

[0088] The spatial dropout module processes x2 to reduce noise in the feature map and enhance regularization. AGRB's adaptive gating mechanism extracts global context information through global average pooling and generates the gating coefficient g through two fully connected layers and a sigmoid activation function, as shown in Equation 3.

[0089] g=σ(FC2(ReLU(FC1(AdaptiveAvgPool(x2))))) (3);

[0090] The value of g ranges from (0 to 1) and is used to adjust the weight of the residual term. The residual connection performs channel matching on x to ensure that its number of channels is consistent with the main branch features. It then performs a weighted addition with x2 and activates it through ReLU to obtain the output y, as shown in Formula 4.

[0091] y=ReLU(x2+g·x shortcut ) (4).

[0092] Furthermore, the step of inputting the target coding feature into a parallel information focusing module to obtain a parallel information focusing feature further includes:

[0093] A channel attention module is used to aggregate the global features of each channel, generate channel importance weights, and adjust the channel expression ability of the input feature map;

[0094] Input the target encoding features, extract the global descriptor of each channel through the global average pooling operation, perform feature compression and expansion through a two-layer fully connected network, and generate channel weights;

[0095] The generated channel weight is multiplied element-by-element by the input target encoding feature to complete the enhancement in the channel dimension;

[0096] The spatial attention module captures global spatial information and generates spatial attention weights by performing maximum pooling and average pooling on the target encoding features in the spatial dimension.

[0097] Performing maximum pooling and average pooling operations on the target coding features in the spatial dimension to generate two spatial feature maps;

[0098] The two spatial feature maps are concatenated in the spatial dimension, and the result is passed through a 7×7 convolutional layer to generate spatial attention weights;

[0099] The spatial weight is multiplied element-by-element by the input target encoding feature to enhance the spatial dimension;

[0100] After completing the channel and spatial attention optimization, the final parallel information focusing features are generated through a fusion operation.

[0101] Among them, reference Figure 3 In this embodiment, first, input the target encoding feature X∈R B×C×H×W The global descriptor of each channel is extracted through the global average pooling operation, denoted as Y avg ∈R B×C Subsequently, these descriptors are compressed and expanded through two layers of fully connected networks to generate channel weights W CA , as shown in Formula 5.

[0102] W CA =σ(W2·σ(W1·Y avg )) (5);

[0103] Where W1∈R C×C / r and W2∈R C / r×C is the weight matrix of the fully connected layer, r is the channel compression ratio, δ represents the ReLU activation function, and σ is the Sigmoid activation function. The generated channel weights are element-wise multiplied by the input target encoding features to complete the channel dimension enhancement, as shown in Equation 6.

[0104] X CA =X·W CA (6);

[0105] The spatial attention module captures global spatial information by performing maximum pooling and average pooling on the target encoding features in the spatial dimension, thereby generating spatial attention weights. Figure 6 As shown in , firstly, the input feature map x is subjected to maximum pooling and average pooling operations in the spatial dimension to generate two spatial feature maps, as shown in Formula 7.

[0106] Y avg =AvgPool(X),Y max =MaxPool(X) (7);

[0107] Among them, Y avg , Y max ∈R B×1×H×W Then, the two are concatenated in the spatial dimension, and the result is passed through a 7×7 convolutional layer to generate the spatial attention weight, as shown in Formula 8.

[0108] w SA =σ(f 7×7 ([Y avg ; Y max ])) (8);

[0109] Among them, [;] represents the splicing in the channel dimension, f 7×7 is a 7×7 convolution operation, and σ is the Sigmoid activation function. Finally, the spatial weights are element-wise multiplied with the input target encoding features to complete the enhancement of the spatial dimension, as shown in Formula 9.

[0110] X SA =X·W SA (9);

[0111] After completing the channel and spatial attention optimization, the module generates the final output feature map through a fusion operation. The results of channel attention and spatial attention are first concatenated in the channel dimension, as shown in Equation 10.

[0112] X fusion =[X CA ;X SA ] (10);

[0113] where X fusion ∈R B×2C×H×W Next, the fused features are channel-compressed through a 1×1 convolutional layer and further optimized through BatchNormalization and ReLU activation functions, as shown in Formula 11.

[0114] X out =δ(BN(f 1×1 (X fusion ))) (11);

[0115] Where δ represents the ReLU activation function and BN represents BatchNormalization. The final feature map X out Used for subsequent task processing.

[0116] The PFAM module captures the channel global dependency and spatial context information of the feature map through independent optimization of channel attention and spatial attention. The fusion operation further integrates the advantages of these two attention mechanisms to provide high-quality feature representation for subsequent tasks.

[0117] Furthermore, the skip connection of the multiple intermediate coding features is input into the multi-scale feature fusion module to obtain the corresponding multiple fused coding features, including:

[0118] Preset the skip connection features of the encoder, perform 3×3 convolution, batch normalization and ReLU activation operations on each intermediate encoding feature to obtain a unified processed feature;

[0119] In each decoding layer, the intermediate encoded features are bilinearly interpolated to adjust the target resolution;

[0120] The Transformer’s attention mechanism is used to capture the dependencies between multi-scale features, stack all adjusted features along the new dimension, and reshape them into sequence representations;

[0121] Reshape the features into a tensor of the target shape and further permute the dimensions to meet the input requirements of the Transformer encoder;

[0122] After passing through the Transformer encoder, the global context information is integrated and multiple fused coding features are output.

[0123] Among them, reference Figure 4 In this embodiment, first, it is assumed that the jump connection feature of the encoder is recorded as S i , where i = 1, 2, ..., n represents the number of coding layers, and In order to unify the number of channels, we i Perform 3×3 convolution, batch normalization, and ReLU activation operations to obtain unified processed features As shown in Formula 12.

[0124]

[0125] in, is the processed feature, C h is the number of channels after unification, Conv 3×3 , BN, and ReLU represent 3×3 convolution, batch normalization, and ReLU activation function, respectively.

[0126] At each layer of decoding D j In order to match the spatial size of the decoder, we transform the feature map Adjust to target resolution (H j , W j ). This process is completed through bilinear interpolation, and its mathematical expression is shown in Formula 13.

[0127]

[0128] in, is the pixel value at position (x, y) at the target resolution, is the value of the four feature points closest to (x, y) in the original feature map, w mn is the interpolation weight, as shown in Formula 14.

[0129] w mn =(1-|xx m |)(1-|yy n |) (14);

[0130] Next, we use the Transformer attention mechanism to capture these multi-scale features First, all resized features are stacked along the new dimension and reshaped into a sequence representation, as shown in Formula 15.

[0131]

[0132] Among them, F j is the stacked multi-scale feature, n is the number of feature layers, C h is the number of channels after unification, (H j , W j ) is the target resolution.

[0133] We reshape the features into a shape of (B, n×C h , H j ×W j ) and further permuted to (H j ×W j , B, n×C h ) to meet the input requirements of the Transformer encoder.

[0134] After passing through the Transformer encoder, the features are able to integrate global context information, and the output representation is shown in Formula 16.

[0135]

[0136] in, It is the encoded output feature, which contains multi-scale global context information.

[0137] We restore the output features to the original spatial dimensions (B, n×C h , H j ×W j ), and perform an average operation on the coding layer dimension to obtain the fused features, as shown in Formula 17.

[0138]

[0139] in, is the fused feature, F j，i Represents the fusion result of the i-th layer encoding features.

[0140] The fused features F are processed by 1×1 convolution, batch normalization and ReLU activation to match the number of channels C required by the decoder. j , generate the final feature map o j , as shown in Formula 18.

[0141]

[0142] in, is the generated decoder feature, C j is the target channel number.

[0143] In the Transformer module, we use a multi-head self-attention mechanism to model the complex dependencies between features. The structure diagram is as follows: Figure 5 As shown. Let the input feature be represented as F, where F∈R L×B×d , L=H j ×W j The core calculation formula of multi-head self-attention is shown in Formula 19.

[0144]

[0145] Where Q = FW Q , K=FW K 、V=FW V , W Q 、W K 、W V ∈R d×d are the linear transformation weight matrices of query, key, and value, respectively, and d is the dimension of the key vector. Through this process, the decoder can more accurately utilize multi-scale information to generate segmentation results with rich details and clear semantics.

[0146] Furthermore, the jump connection of the multiple fusion coding features and the parallel information focus features are input into the decoder together to obtain multiple candidate product features, including:

[0147] Adaptive gated residual upsampling block is used to include upsampling and skip connection operations;

[0148] The input parallel information focusing features are upsampled to half the size of the feature map through bilinear interpolation, and then spliced with the feature map of the corresponding layer in the multi-scale feature fusion module to integrate the semantic information of the encoding stage;

[0149] The spliced feature maps are then processed by an adaptive gated residual block to enhance the feature fusion effect and restore image details to obtain multiple candidate product features.

[0150] Among them, reference Figure 6 In this embodiment, the input feature map is first upsampled to half its size using bilinear interpolation and then concatenated with the feature map of the corresponding layer in FN-Hub to integrate semantic information from the encoding stage. The concatenated feature map is then processed by the AGRB module to enhance feature fusion and restore image details. The adaptive gating mechanism in UAGRB dynamically adjusts the weights of the residual connections, thereby optimizing the importance of different features during the upsampling process and enabling the network to better preserve subtle structures.

[0151] For embodiments of the present invention, an evaluation of the extraction algorithm is performed after the ironmaking products are extracted;

[0152] In the image segmentation task, in order to more comprehensively evaluate the segmentation effect, the following performance indicators are used: intersection over union (IoU), accuracy (ACC), sensitivity (Se), specificity (Sp), F1 score (F1), and area under the curve (AUC). The specific definitions are shown in formulas 20-23.

[0153]

[0154] Among them, TP, TN, FP and FN represent true positive, true negative, false positive and false negative, respectively.

[0155] The meanings of the various metrics are as follows: ACC represents the proportion of pixels correctly classified. Se (sensitivity) reflects the model's ability to identify positive pixels, while Sp (specificity) indicates the model's correct identification rate for negative pixels. The F1 score combines precision and sensitivity to evaluate the model's overall classification performance. AUC measures the model's overall performance at different discrimination thresholds.

[0156] In the experiment, the first and second data source datasets were used for training and testing. These two datasets are public standard datasets used for ironmaking product segmentation.

[0157] The first data source dataset contains 40 RGB color product images, each with a resolution of 565x584. We split the data into training and validation sets in an 8:2 ratio from the official training set, while the remaining 20 images remain unchanged for the test set. The second data source dataset contains 28 RGB color product images, each with a resolution of 999x960. We split the data into training, validation, and test sets in a 7:2:1 ratio, with the first 19 images used as training, 6 as validation, and 3 randomly selected images used as the test set.

[0158] In medical image segmentation tasks, proper preprocessing can significantly improve the performance of the model. The dataset was subjected to green channel extraction, contrast-limited adaptive histogram equalization (CLAHE), and gamma correction to optimize the quality of the color ironmaking images.

[0159] To minimize color variations caused by different acquisition devices or lighting conditions, we first extracted the green channel from the RGB image. The green channel typically has higher contrast, which is particularly beneficial for extracting iron products. We then converted the RGB image to grayscale using the formula:

[0160] I Gray =0.3×I Red +0.59×I Green +0.11×I Blue (twenty four)

[0161] Among them, I Gray Represents the pixel intensity of the grayscale image, I Red , I Green and l Blue Represents the pixel intensity of the red, green, and blue channels, respectively.

[0162] Next, to enhance the contrast between the product and the background in the image, we applied the CLAHE operation. CLAHE enhances contrast while suppressing noise. This method sets a threshold and evenly distributes pixel values exceeding the threshold to other grayscale levels, forming a new histogram and performing adaptive equalization on it. We set the threshold to 2 and selected an 8×8 grid size, segmenting the image into non-overlapping regions and processing them separately to ensure stable and consistent results.

[0163] To further highlight the difference in brightness between the product and surrounding tissue, we use gamma correction technology. Gamma correction adjusts the image's gamma curve to change the brightness distribution, thereby enhancing the contrast between low-brightness and high-brightness areas. The mathematical expression for gamma correction is:

[0164]

[0165] Among them, I out and I in where γ represents the output and input pixel intensities, respectively, and γ represents the gamma value. In this study, we set the gamma value to 0.8 to enhance contrast in low-brightness areas and highlight the details of the ironworks products. After gamma correction, the brightness difference between the product area and the background is significantly increased, and the structure of small products is particularly clear.

[0166] After a series of preprocessing operations, the ironworks images, originally low in contrast and blurred in color, have been significantly improved. The processed images not only highlight product details but also significantly enhance the contrast between the products and surrounding non-product areas. To increase the diversity of the training data, the preprocessed images are cropped into 48×48 non-overlapping tiles. This slicing method increases the number of training samples, alleviates overfitting in deep learning models, and provides the model with more diverse feature samples, thereby improving the performance of the ironworks product segmentation task.

[0167] In terms of accuracy (Acc), EFDG-UNet achieved 0.9736, which is competitive with current high-performance models such as UNet3+ and SDDC-Net, and significantly higher than traditional U-Net (0.9691) and LadderNet (0.9561), demonstrating its precision in overall segmentation tasks. Its specificity (Sp) of 0.9856 is also at the forefront, second only to a few models such as ResUNet++ (0.9837), reflecting its efficient recognition of background areas.

[0168] In terms of sensitivity (Se), EFDG-UNet (0.8438) significantly outperforms classic methods (such as U-Net's 0.7948) and some modern models (such as ResUNet++'s 0.7856), demonstrating its superior ability to capture small products. Although slightly lower in sensitivity than SDDC-Net (0.8603) and DCA-CNN (0.8745), its high balance between specificity and accuracy gives it a more balanced overall performance.

[0169] EFDG-UNet achieved top-tier results in both AUC (0.9886) and F1 score (0.8412). The AUC reflects the robustness of the segmentation model at different thresholds, while the F1 score reflects the balance between precision and recall. Compared to other recently proposed models (such as UNet3+, which achieved an F1 score of 0.8254), EFDG-UNet's segmentation performance demonstrates superior versatility.

[0170] EFDG-UNet excels across multiple key metrics, particularly in accuracy (Acc) and overall segmentation performance. Its high sensitivity (Se) and specificity (Sp) values demonstrate its ability to strike a good balance between segmenting small products and background regions. Its excellent AUC and F1 performance further validate the model's robustness and reliability, making it a leading method for ironmaking product segmentation.

[0171] EFDG-UNet demonstrates significant advantages in segmenting complex product areas. In the segmentation of coarse products, EFDG-UNet demonstrates greater coherence and completeness, accurately preserving the overall shape of the product compared to other models. In the segmentation of tiny products, EFDG-UNet not only accurately identifies product branches and intersections, but also significantly reduces misidentifications, avoiding the incorrect segmentation of background areas as products. Overall, EFDG-UNet demonstrates greater robustness and higher segmentation accuracy in fine-grained product segmentation tasks.

[0172] Furthermore, the step of obtaining the wrinkle smoothness area ratio in the output image and determining the smoothness quality of the ironmaking product to be evaluated includes:

[0173] Obtaining the number N1 of rough feature points and N2 of smooth feature points in the output image;

[0174] The calculated ratio of the rough feature points to the total feature points and the ratio of the hole area on the surface of the ironmaking product are expressed as the surface wrinkle smooth area ratio K, and the calculation formula is as follows:

[0175]

[0176] Where: K-surface wrinkle smooth area ratio (%);

[0177] N1-number of rough feature points;

[0178] N2-number of smoothing feature points;

[0179] Determining the smoothness quality of the ironmaking product to be evaluated based on the surface wrinkle smoothness area ratio;

[0180] The smooth quality includes excellent ironmaking products, good ironmaking products, qualified ironmaking products, and unqualified ironmaking products.

[0181] In this embodiment, the smoothness quality is determined by referring to Table 1;

[0182] Table 1 Smooth quality decision table

[0183]

[0184] According to a second embodiment of the present invention, the present invention claims protection for an ironmaking product quality evaluation device based on multi-source data, comprising:

[0185] one or more processors;

[0186] A memory having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the ironmaking product quality evaluation method based on multi-source data.

[0187] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0188] In addition, the functional units in the various embodiments of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units. The above is only an implementation method of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

[0189] The above detailed description of the specific embodiments of the invention is intended only as an example, and the present application is not limited to the specific embodiments described above. For those skilled in the art, any equivalent modifications or substitutions of the invention are also within the scope of the present application. Therefore, equivalent changes, modifications, and improvements made without departing from the spirit and scope of the present application should be included within the scope of the present application.

Claims

1. A method for evaluating the quality of ironmaking products based on multi-source data, characterized in that: include: Collecting a characteristic map of the ironmaking product to be evaluated, inputting the characteristic map into an encoder to obtain a plurality of intermediate coding features and a target coding feature; Inputting the plurality of intermediate coding features into a multi-scale feature fusion module through jump connections to obtain a plurality of corresponding fused coding features, and inputting the target coding features into a parallel information focusing module to obtain a parallel information focusing feature; Jump-connecting the multiple fused coding features and inputting them together with the parallel information focused features into a decoder to obtain multiple candidate product features; Obtaining an output image of the ironmaking product to be evaluated based on feature analysis of the plurality of candidate products; Obtaining a wrinkle smoothing area ratio in the output image to determine the smoothing quality of the ironmaking product to be evaluated; performing defect evaluation statistics on the smoothing quality of multiple consecutive ironmaking products to be evaluated captured in the video to obtain a quality defect evaluation of the ironmaking product; Wherein, the encoder comprises: Adopting adaptive gated residual blocks, combined with residual connections and adaptive gating mechanisms, and introducing spatial random dropout modules to alleviate overfitting; In the encoder, the adaptive gated residual block includes two convolutional layers and their corresponding batch normalization and activation layers; the inputting the feature map into the encoder to obtain multiple intermediate coding features and target coding features includes: processing the first feature of the feature map through the first convolutional layer, and passing it through the batch normalization and ReLU activation function of the activation layer to obtain the first intermediate coding feature; Passing the first intermediate coding feature through the second convolutional layer and the batch normalization process again to obtain a second intermediate coding feature; The spatial random dropout module processes the second intermediate coding feature to reduce noise in the feature map and enhance regularization effect; The adaptive gating mechanism of the adaptive gating residual block extracts global context information through global average pooling, and generates gating coefficients through two fully connected layers and Sigmoid activation function to adjust the weight of the residual term; The residual connection part performs channel matching on the first feature to ensure that the number of channels is consistent with the main branch feature, and then performs weighted addition on the feature obtained after the second intermediate coding feature is processed by the spatial random inactivation module, and obtains the corresponding target coding feature through the ReLU activation function; the target coding feature is input into the parallel information focusing module to obtain the parallel information focusing feature, including: using the channel attention module to aggregate the global features of each channel, generate channel weights, and adjust the channel expression ability of the input feature map; Input the target encoding features, extract the global descriptor of each channel through the global average pooling operation, perform feature compression and expansion through a two-layer fully connected network, and generate channel weights; The generated channel weight is multiplied element-by-element with the input target encoding feature to complete the enhancement in the channel dimension; the spatial attention module captures global spatial information and generates spatial weights by performing maximum pooling and average pooling on the target encoding feature in the spatial dimension; Performing maximum pooling and average pooling operations on the target coding features in the spatial dimension to generate two spatial feature maps; The two spatial feature maps are concatenated in the spatial dimension, and the result is passed through a 7×7 convolutional layer to generate spatial weights. The spatial weights are multiplied element-wise with the input target encoding features to complete the spatial dimension enhancement. After completing the channel and spatial attention optimization, the final parallel information focusing features are generated through a fusion operation.

2. The ironmaking product quality evaluation method based on multi-source data according to claim 1, characterized in that: The skip connection of the multiple intermediate coding features into the multi-scale feature fusion module to obtain the corresponding multiple fused coding features includes: Preset the skip connection features of the encoder, perform 3×3 convolution, batch normalization and ReLU activation operations on each intermediate encoding feature to obtain a unified processed feature; In each layer of the decoder, the intermediate encoded features are bilinearly interpolated to adjust the target resolution; The Transformer’s attention mechanism is used to capture the dependencies between multi-scale features, stack all adjusted features along the new dimension, and reshape them into sequence representations; Reshape the features into a tensor of the target shape and further permute the dimensions to meet the input requirements of the Transformer encoder; After passing through the Transformer encoder, the global context information is integrated and multiple fused coding features are output.

3. The ironmaking product quality evaluation method based on multi-source data according to claim 2, characterized in that: The jump connection of the multiple fusion coding features and the parallel information focus features are input into the decoder to obtain multiple candidate product features, including: Adaptive gated residual upsampling block is used for upsampling and skip connection operations; The input parallel information focusing features are upsampled to half the size of the feature map through bilinear interpolation, and then spliced with the feature map of the corresponding layer in the multi-scale feature fusion module to integrate the semantic information of the encoding stage; the spliced feature map is then processed by an adaptive gated residual block to enhance the feature fusion effect and restore image details to obtain multiple candidate product features.

4. The ironmaking product quality evaluation method based on multi-source data according to claim 3, characterized in that: The step of obtaining the wrinkle smoothing area ratio in the output image and determining the smoothing quality of the ironmaking product to be evaluated includes: Obtaining the number N1 of rough feature points and the number N2 of smooth feature points in the output image; The ratio of the rough feature points to the total feature points and the ratio of the hole area on the surface of the ironmaking product are calculated and expressed as the surface wrinkle smooth area ratio K. The calculation formula is as follows: Where: K—surface wrinkle smooth area ratio (%); N1—Number of rough feature points; N2—number of smoothing feature points; Determining the smoothness quality of the ironmaking product to be evaluated based on the surface wrinkle smoothness area ratio; The smooth quality includes excellent ironmaking products, good ironmaking products, qualified ironmaking products, and unqualified ironmaking products.

5. An ironmaking product quality evaluation device based on multi-source data, characterized in that: include: one or more processors; A memory having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement a method for evaluating the quality of ironmaking products based on multi-source data according to any one of claims 1 to 4.

Images