Ironmaking product quality evaluation method and device based on multi-source data

Abstract

The invention discloses an ironmaking product quality evaluation method and device based on multi-source data, and relates to the field of image processing, and the method comprises the steps: inputting a collected feature map of a to-be-evaluated ironmaking product into an encoder, obtaining a plurality of intermediate coding features and target coding features, inputting the plurality of intermediate coding features into a multi-scale feature fusion module in a jumping connection manner to obtain a plurality of fusion coding features, and inputting the target coding features into a parallel information focusing module to obtain parallel information focusing features, connecting the plurality of fused coding features in a jumping manner, inputting the fused coding features and parallel information focusing features into a decoder to obtain a plurality of candidate product features, analyzing according to the plurality of candidate product features to obtain an output image of the ironmaking product to be evaluated, obtaining a wrinkle smooth area ratio in the output image, and deciding the smooth quality of the ironmaking product to be evaluated. And performing flaw evaluation statistics on a plurality of continuous smooth qualities acquired by the video to obtain the quality flaw evaluation of the ironmaking product. The segmentation performance is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and device for evaluating the quality of ironmaking products based on multi-source data. Background Art

[0002] With the high speed of iron production lines and the continuous improvement of quality requirements, the method of manual visual inspection can no longer meet the requirements of quality assurance. On the other hand, with the rapid development of information technology in recent years, online inspection technology based on machine vision has reached or exceeded manual inspection in terms of inspection accuracy, and has advantages that manual inspection cannot match in terms of fatigue resistance, etc., and can also realize the informatization of quality inspection and quality management.

[0003] Although the existing technology has made some progress in improving the segmentation performance of ironmaking products, it still has the following shortcomings: the traditional ironmaking product segmentation method relies on professionals to manually annotate images, which is time-consuming and laborious. The annotation results are affected by experience and judgment. Especially in low-contrast areas and complex ironmaking product structures, the segmentation results may have small product sensitivity, multi-scale adaptability and interference from lesion areas. The existing methods still need to be improved in extracting fine-grained and complex features of ironmaking products; there is a lack of effective dynamic adaptive processing mechanism for the characteristics of complex ironmaking product morphology and defective areas; in terms of balancing global context information and local detail features, the ability of existing methods is still insufficient. Summary of the invention

[0004] In view of the above problems, this application provides a method and device for evaluating the quality of ironmaking products based on multi-source data, so as to achieve the effective fusion of global and local features, balance the relationship between global context information and local detail features, enhance the segmentation robustness in complex product morphology and lesion areas, and capture the fine-grained and complex features of the product to improve the segmentation performance. The specific scheme is as follows:

[0005] The first aspect of the present application provides a method for evaluating the quality of ironmaking products based on multi-source data, comprising:

[0006] Collecting a feature image of the ironmaking product to be evaluated, inputting the feature image into an encoder, and obtaining a plurality of intermediate coding features and a target coding feature;

[0007] Inputting the plurality of intermediate coding features into a multi-scale feature fusion module through jump connection to obtain a plurality of corresponding fused coding features, and inputting the target coding features into a parallel information focusing module to obtain a parallel information focusing feature;

[0008] The plurality of fused coding features are jump-connected and input into a decoder together with the parallel information focusing feature to obtain a plurality of candidate product features;

[0009] Obtaining an output image of the ironmaking product to be evaluated based on the characteristic analysis of the plurality of candidate products;

[0010] Obtaining a wrinkle smoothing area ratio in the output image to determine the smoothing quality of the ironmaking product to be evaluated;

[0011] The smooth quality of a plurality of continuous ironmaking products to be evaluated captured by video is statistically evaluated for defects to obtain a quality defect evaluation of the ironmaking product.

[0012] In a possible implementation, the encoder includes:

[0013] Adopting adaptive gated residual blocks, combining residual connections and adaptive gating mechanisms, and introducing spatial random dropout modules to reduce overfitting;

[0014] In the encoder, the adaptive gated residual block contains two convolutional layers and their corresponding batch normalization and activation layers.

[0015] In a possible implementation, inputting the feature image into an encoder to obtain a plurality of intermediate coding features and a target coding feature further includes:

[0016] The first feature of the feature map is processed by the first convolutional layer, and is subjected to batch normalization and the ReLU activation function of the activation layer to obtain a first intermediate coding feature;

[0017] Passing the first intermediate coding feature through the second convolution layer and the batch normalization process again to obtain a second intermediate coding feature;

[0018] The spatial random dropout module processes the second intermediate coding feature to reduce noise in the feature map and enhance the regularization effect;

[0019] The adaptive gating mechanism of the adaptive gating residual block extracts global context information through global average pooling, and generates gating coefficients through two fully connected layers and Sigmoid activation function to adjust the weight of the residual term;

[0020] The residual connection part performs channel matching on the first feature to ensure that the number of channels is consistent with the main branch feature, and then performs weighted addition with the second intermediate coding feature, and obtains the corresponding target coding feature through the ReLU activation function.

[0021] In a possible implementation, inputting the target coding feature into a parallel information focusing module to obtain a parallel information focusing feature includes:

[0022] A channel attention module is used to aggregate the global features of each channel, generate channel importance weights, and adjust the channel expression ability of the input feature map;

[0023] Input the target encoding features, extract the global descriptor of each channel through the global average pooling operation, realize feature compression and expansion through two layers of fully connected networks, and generate channel weights;

[0024] The generated channel weight is multiplied element by element with the input feature map to complete the enhancement in the channel dimension;

[0025] The spatial attention module captures global spatial information and generates spatial attention weights by performing maximum pooling and average pooling on the feature map in the spatial dimension.

[0026] Performing maximum pooling and average pooling operations on the first feature in the channel dimension to generate two spatial feature maps;

[0027] The two spatial feature maps are concatenated in the channel dimension, and the result is passed through a 7×7 convolutional layer to generate spatial attention weights;

[0028] The spatial weight is multiplied element by element with the input feature map to enhance the spatial dimension;

[0029] After completing the channel and space attention optimization, the final parallel information focusing feature is generated through a fusion operation.

[0030] In a possible implementation, the skip-connecting and inputting the multiple intermediate coding features into a multi-scale feature fusion module to obtain the corresponding multiple fused coding features includes:

[0031] Presetting the skip connection features of the encoder, performing 3×3 convolution, batch normalization, and ReLU activation operations on each intermediate encoding feature to obtain a uniformly processed feature;

[0032] In each decoding layer, the intermediate encoded features are bilinearly interpolated to adjust the target resolution;

[0033] The Transformer attention mechanism is used to capture the dependencies between multi-scale features, stack all adjusted features along the new dimension, and reshape them into sequence representations;

[0034] Reshape the features into a tensor of the target shape and further permute the dimensions to meet the input requirements of the Transformer encoder;

[0035] After passing through the Transformer encoder, the global context information is integrated and multiple fused coding features are output.

[0036] In a possible implementation, the jump connection of the multiple fusion coding features and the parallel information focusing features are input into a decoder to obtain multiple candidate product features, including:

[0037] Adopting adaptive gated residual upsampling block including upsampling and skip connection operations;

[0038] The input parallel information focusing features are upsampled to half the size of the feature map through bilinear interpolation, and then concatenated with the feature map of the corresponding layer in the multi-scale feature fusion module to integrate the semantic information of the encoding stage;

[0039] The spliced ​​feature maps are then processed by an adaptive gated residual block to enhance the feature fusion effect and restore image details to obtain multiple candidate product features.

[0040] In a possible implementation, obtaining the wrinkle smoothing area ratio in the output image and determining the smoothing quality of the ironmaking product to be evaluated includes:

[0041] Obtaining the number N1 of rough feature points and N2 of smooth feature points in the output image;

[0042] The calculated ratio of the rough feature points to the total feature points, the ratio of the hole area on the surface of the ironmaking product, and the surface wrinkle smoothing area ratio The calculation formula is as follows:

[0043] ;

[0044] Where: —Surface wrinkle smooth area ratio;

[0045] —Number of rough feature points;

[0046] —Number of smoothing feature points;

[0047] Determining the smoothness quality of the ironmaking product to be evaluated according to the surface wrinkle smoothness area ratio;

[0048] The smooth quality includes excellent ironmaking products, good ironmaking products, qualified ironmaking products, and unqualified ironmaking products.

[0049] A second aspect of the present application provides an ironmaking product quality evaluation device based on multi-source data, comprising:

[0050] one or more processors;

[0051] A memory having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for evaluating the quality of ironmaking products based on multi-source data provided in the first aspect of the present application and any possible implementation of the first aspect.

[0052] By means of the above technical solution, the present application provides a method and device for evaluating the quality of ironmaking products based on multi-source data, which adopts a strong feature dynamic fusion gated U-Net. By introducing the centrality idea in graph theory, the core module is designed as a central node, and the key information of the multi-layer encoder is dynamically aggregated, thereby realizing the effective fusion of global and local features. The feature fusion hub module balances the relationship between global context information and local detail features by utilizing the long-range dependencies of multi-layer features. The adaptive gated residual block uses a dynamic gating mechanism to screen key features and suppress redundant information, thereby enhancing the segmentation robustness in complex product morphology and pathological areas. The parallel information focusing module combines channel and spatial attention mechanisms, effectively models global position information and long-range dependencies, captures fine-grained and complex features of the product, and further improves the segmentation performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0054] Figure 1 A workflow diagram of a method for evaluating the quality of ironmaking products based on multi-source data as claimed in an embodiment of the present application;

[0055] Figure 2 An adaptive gated residual block structure diagram of an ironmaking product quality evaluation method based on multi-source data as claimed in an embodiment of the present application;

[0056] Figure 3 A parallel information focusing module structure diagram of an ironmaking product quality evaluation method based on multi-source data claimed in an embodiment of the present application;

[0057] Figure 4 A feature navigation hub structure diagram of a method for evaluating the quality of ironmaking products based on multi-source data as claimed in an embodiment of the present application;

[0058] Figure 5 A Transformer structure diagram of an ironmaking product quality evaluation method based on multi-source data as claimed in an embodiment of the present application;

[0059] Figure 6 This is a structural diagram of an adaptive gated residual upsampling block of an ironmaking product quality evaluation method based on multi-source data as claimed in an embodiment of the present application. DETAILED DESCRIPTION

[0060] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0061] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0062] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0063] According to the first embodiment of the present invention, the present invention claims a method for evaluating the quality of ironmaking products based on multi-source data, referring to Figure 1 ,include:

[0064] Collecting a characteristic graph of the ironmaking product to be evaluated, inputting the characteristic graph into an encoder, and obtaining a plurality of intermediate coding features and a target coding feature;

[0065] Inputting the plurality of intermediate coding features into a multi-scale feature fusion module through jump connection to obtain a plurality of corresponding fused coding features, and inputting the target coding features into a parallel information focusing module to obtain a parallel information focusing feature;

[0066] The plurality of fused coding features are jump-connected and input into a decoder together with the parallel information focusing feature to obtain a plurality of candidate product features;

[0067] Obtaining an output image of the ironmaking product to be evaluated based on the characteristic analysis of the plurality of candidate products;

[0068] Obtaining a wrinkle smoothing area ratio in the output image to determine the smoothing quality of the ironmaking product to be evaluated;

[0069] The smooth quality of a plurality of continuous ironmaking products to be evaluated captured by video is statistically evaluated for defects to obtain a quality defect evaluation of the ironmaking product.

[0070] The main goal of this invention is to overcome the shortcomings of the existing technology in the segmentation of ironmaking products through a number of innovative technologies, especially for the sensitivity of small products, multi-scale adaptability and interference of lesion areas. Through the three core modules of adaptive gated residual block, parallel information focusing module and feature navigation hub, the model is comprehensively optimized and improved in data feature extraction, application of attention mechanism and multi-scale feature fusion, thus significantly improving the accuracy and robustness of ironmaking product segmentation.

[0071] In order to solve the problems of interference from lesion areas and poor adaptability of multi-scale products in the segmentation of ironmaking products, the present invention proposes an adaptive gated residual block (AGRB). This module combines residual connections and adaptive gating mechanisms, significantly improving the performance of the model in multi-scale product feature extraction and enhancement, thereby enhancing the sensitivity to products of different scales. In order to further avoid overfitting, AGRB also introduces a spatial dropout module to enhance the generalization and robustness of the model. In the encoder part, AGRB is responsible for extracting and compressing multi-scale features layer by layer, while in the decoder, the adjusted AGRB is used to restore image resolution, ensuring accurate segmentation of products of different scales and fine restoration of details.

[0072] In terms of feature fusion and global modeling, the present invention dynamically integrates multi-layer features of different scales in the encoder through the feature navigation hub (FN-Hub) combined with the Transformer-based global attention mechanism. This module can effectively capture the long-range dependencies between multi-layer features, promote the efficient fusion of global and local features, and greatly improve the model's ability to process global context information of coarse and fine products.

[0073] In order to solve the problem of extracting fine-grained and complex features of products, the present invention designs a parallel information focusing module (PFAM). By fusing channel attention and spatial attention mechanisms, this module can strengthen useful features in both channel and spatial dimensions, thereby improving feature expression capabilities and fine-grained feature capture capabilities. Through this parallel optimization, PFAM effectively improves the ability to extract fine-grained and complex features of products, thereby further improving the accuracy and robustness of segmentation.

[0074] The synergistic effect of these modules significantly improves the performance of the model in complex product morphology and low-contrast scenes, making EFDG-UNet excel in small product sensitivity, lesion area robustness and overall segmentation performance, especially in complex product areas, showing efficient and robust segmentation capabilities.

[0075] Furthermore, the encoder comprises:

[0076] Adopting adaptive gated residual blocks, combining residual connections and adaptive gating mechanisms, and introducing spatial random dropout modules to reduce overfitting;

[0077] In the encoder, the adaptive gated residual block contains two convolutional layers and their corresponding batch normalization and activation layers.

[0078] Furthermore, the step of inputting the feature map into an encoder to obtain a plurality of intermediate encoding features and a target encoding feature further includes:

[0079] The first feature of the feature map is processed by the first convolutional layer, and is subjected to batch normalization and the ReLU activation function of the activation layer to obtain a first intermediate coding feature;

[0080] Passing the first intermediate coding feature through the second convolution layer and the batch normalization process again to obtain a second intermediate coding feature;

[0081] The spatial random dropout module processes the second intermediate coding feature to reduce noise in the feature map and enhance the regularization effect;

[0082] The adaptive gating mechanism of the adaptive gating residual block extracts global context information through global average pooling, and generates gating coefficients through two fully connected layers and Sigmoid activation function to adjust the weight of the residual term;

[0083] The residual connection part performs channel matching on the first feature to ensure that the number of channels is consistent with the main branch feature, and then performs weighted addition with the second intermediate coding feature, and obtains the corresponding target coding feature through the ReLU activation function.

[0084] Among them, refer to Figure 2 In this embodiment, in the encoder, the core structure of AGRB contains two convolutional layers and their corresponding batch normalization and activation layers. First, the input feature map Processed by the first convolution layer, and then batch normalized and ReLU activated to obtain intermediate features , the process is shown in formula 1.

[0085] (1);

[0086] Next, the intermediate features Once again through the second convolution layer and batch normalization processing, the feature is obtained , the process is shown in formula 2.

[0087] (2);

[0088] Spatial random dropout module pair Processing is performed to reduce the noise in the feature map and enhance the regularization effect. AGRB's adaptive gating mechanism extracts global context information through global average pooling, and generates the gating coefficient g through two fully connected layers and Sigmoid activation function, as shown in Formula 3.

[0089] (3);

[0090] Among them, the value range of g is between (0,1) and is used to adjust the weight of the residual term. Perform channel matching to ensure that the number of channels is consistent with the main branch characteristics, and then Perform weighted addition and get the output through ReLU activation , as shown in Formula 4.

[0091] (4).

[0092] Furthermore, the step of inputting the target coding feature into a parallel information focusing module to obtain a parallel information focusing feature further includes:

[0093] A channel attention module is used to aggregate the global features of each channel, generate channel importance weights, and adjust the channel expression ability of the input feature map;

[0094] Input the target encoding features, extract the global descriptor of each channel through the global average pooling operation, realize feature compression and expansion through two layers of fully connected networks, and generate channel weights;

[0095] The generated channel weight is multiplied element by element with the input feature map to complete the enhancement in the channel dimension;

[0096] The spatial attention module captures global spatial information and generates spatial attention weights by performing maximum pooling and average pooling on the feature map in the spatial dimension.

[0097] Performing maximum pooling and average pooling operations on the first feature in the channel dimension to generate two spatial feature maps;

[0098] The two spatial feature maps are concatenated in the channel dimension, and the result is passed through a 7×7 convolutional layer to generate spatial attention weights;

[0099] The spatial weight is multiplied element by element with the input feature map to enhance the spatial dimension;

[0100] After completing the channel and space attention optimization, the final parallel information focusing feature is generated through a fusion operation.

[0101] Among them, refer to Figure 3 In this embodiment, first, the feature map is input The global descriptor of each channel is extracted through the global average pooling operation, which is expressed as Subsequently, these descriptors are passed through a two-layer fully connected network to achieve feature compression and expansion, generating channel weights , as shown in Formula 5.

[0102] (5);

[0103] in, and is the weight matrix of the fully connected layer, is the channel compression ratio, represents the ReLU activation function, is the Sigmoid activation function. The generated channel weights are multiplied element-by-element with the input feature map to complete the enhancement in the channel dimension, as shown in Formula 6.

[0104] (6);

[0105] The spatial attention module captures global spatial information by performing maximum pooling and average pooling on the feature map in the spatial dimension, thereby generating spatial attention weights. Figure 6 As shown, first, the input feature map Perform maximum pooling and average pooling operations on the channel dimension to generate two spatial feature maps, as shown in Formula 7.

[0106] (7);

[0107] in, Next, the two are concatenated in the channel dimension, and the result is passed through a 7×7 convolutional layer to generate the spatial attention weight, as shown in Formula 8.

[0108] (8);

[0109] Among them, [;] represents concatenation in the channel dimension, is a 7×7 convolution operation, is the Sigmoid activation function. Finally, the spatial weight is multiplied element-by-element with the input feature map to complete the enhancement of the spatial dimension, as shown in Formula 9.

[0110] (9);

[0111] After completing the channel and spatial attention optimization, the module generates the final output feature map through a fusion operation. The results of channel attention and spatial attention are first concatenated in the channel dimension, as shown in Formula 10.

[0112] (10);

[0113] in Next, the fused features are channel compressed through a 1×1 convolutional layer and further optimized through BatchNormalization and ReLU activation functions, as shown in Formula 11.

[0114] (11);

[0115] in represents the ReLU activation function, Represents BatchNormalization. The final feature map Used for subsequent task processing.

[0116] The PFAM module captures the channel global dependency and spatial context information of the feature map through independent optimization of channel attention and spatial attention. The fusion operation further integrates the advantages of these two attention mechanisms to provide high-quality feature representation for subsequent tasks.

[0117] Furthermore, the skip connection of the multiple intermediate coding features is input into the multi-scale feature fusion module to obtain the corresponding multiple fused coding features, including:

[0118] Presetting the skip connection features of the encoder, performing 3×3 convolution, batch normalization, and ReLU activation operations on each intermediate encoding feature to obtain a uniformly processed feature;

[0119] In each decoding layer, the intermediate encoded features are bilinearly interpolated to adjust the target resolution;

[0120] The Transformer attention mechanism is used to capture the dependencies between multi-scale features, stack all adjusted features along the new dimension, and reshape them into sequence representations;

[0121] Reshape the features into a tensor of the target shape and further permute the dimensions to meet the input requirements of the Transformer encoder;

[0122] After passing through the Transformer encoder, the global context information is integrated and multiple fused coding features are output.

[0123] Among them, refer to Figure 4In this embodiment, first, it is assumed that the jump connection feature of the encoder is recorded as ,in represents the number of coding layers, and In order to unify the number of channels, we Perform 3×3 convolution, batch normalization, and ReLU activation operations to obtain unified processed features , as shown in Formula 12.

[0124] (12);

[0125] in, is the processed feature, is the number of channels after unification, , BN, and ReLU represent 3×3 convolution, batch normalization, and ReLU activation function, respectively.

[0126] At each layer of decoding In order to match the spatial size of the decoder, we transform the feature map Resize to target resolution This process is completed through bilinear interpolation, and its mathematical expression is shown in Formula 13.

[0127] (13);

[0128] in, is the position at the target resolution The pixel value of is the original feature map with The values ​​of the four nearest feature points, is the interpolation weight, as shown in Formula 14.

[0129] (14);

[0130] Next, we use the Transformer attention mechanism to capture these multi-scale features First, all adjusted features are stacked along the new dimension and reshaped into sequence representation, as shown in Formula 15.

[0131] (15);

[0132] in, is the stacked multi-scale feature, n is the number of feature layers, C h is the number of channels after unification, is the target resolution.

[0133] We reshape the features into the shape Tensor of , and further permuted dimensions into , to meet the input requirements of the Transformer encoder.

[0134] After passing through the Transformer encoder, the features are able to integrate global context information, and the output representation is shown in Formula 16.

[0135] (16);

[0136] in, It is the encoded output feature, which contains multi-scale global context information.

[0137] We restore the output features to the original spatial dimensions , and perform an average operation on the encoding layer dimension to obtain the fused features, as shown in Formula 17.

[0138] (17);

[0139] in, is the fused feature, Represents the fusion result of the i-th layer encoding features.

[0140] The fused features Processed through 1×1 convolution, batch normalization, and ReLU activation to match the number of channels required by the decoder , generate the final feature map , as shown in Formula 18.

[0141] (18);

[0142] in, is the generated decoder feature, is the target channel number.

[0143] In the Transformer module, we use a multi-head self-attention mechanism to model the complex dependencies between features. The structure diagram is as follows: Figure 5 As shown. Let the input feature be expressed as ,in , The core calculation formula of multi-head self-attention is shown in Formula 19.

[0144] (19);

[0145] in, , , , , , are the linear transformation weight matrices for query, key, and value, respectively, is the dimension of the key vector. Through this process, the decoder can more accurately utilize multi-scale information and generate segmentation results with rich details and clear semantics.

[0146] Furthermore, the jump connection of the multiple fusion coding features and the parallel information focusing features are input into a decoder together to obtain multiple candidate product features, including:

[0147] Adopting adaptive gated residual upsampling block including upsampling and skip connection operations;

[0148] The input parallel information focusing features are upsampled to half the size of the feature map through bilinear interpolation, and then concatenated with the feature map of the corresponding layer in the multi-scale feature fusion module to integrate the semantic information of the encoding stage;

[0149] The spliced ​​feature maps are then processed by an adaptive gated residual block to enhance the feature fusion effect and restore image details to obtain multiple candidate product features.

[0150] Among them, refer to Figure 6 In this embodiment, first, the input feature map is upsampled to half the size of the feature map through bilinear interpolation, and then concatenated with the feature map of the corresponding layer in FN-Hub to integrate the semantic information of the encoding stage. The concatenated feature map is then processed by the AGRB module to enhance the feature fusion effect and restore image details. The adaptive gating mechanism in UAGRB dynamically adjusts the weights of the residual connection, thereby optimizing the importance of different features during the upsampling process, allowing the network to better preserve subtle structures.

[0151] For embodiments of the present invention, an evaluation of the extraction algorithm is performed after the ironmaking product is extracted;

[0152] In the image segmentation task, in order to more comprehensively evaluate the segmentation effect, the following performance indicators are used: intersection over union (IoU), accuracy (ACC), sensitivity (Se), specificity (Sp), F1 score (F1) and area under the curve (AUC). The specific definitions are shown in formulas 20-23.

[0153] (20)

[0154] (twenty one)

[0155] (twenty two)

[0156] (twenty three)

[0157] Among them, TP, TN, FP and FN represent true positive, true negative, false positive and false negative, respectively.

[0158] The meanings of each indicator are as follows: ACC represents the proportion of pixels that are correctly classified. Se (sensitivity) reflects the model's ability to identify positive pixels, while Sp (specificity) represents the model's correct recognition rate of negative pixels. The F1 score combines precision and sensitivity to evaluate the comprehensive classification performance of the model. AUC measures the overall performance of the model at different discrimination thresholds.

[0159] In the experiment, the first data source and the second data source dataset were used for training and testing. These two datasets are public standard datasets used for ironmaking product segmentation.

[0160] The first data source dataset contains 40 RGB color product images, each with a resolution of 565x584. We divide the official training set into training and validation sets in an 8:2 ratio, and the remaining 20 test sets remain unchanged. The second data source dataset contains 28 RGB color product images, each with a resolution of 999x960. We divide the data into training, validation, and test sets in a 7:2:1 ratio, with the first 19 images used as training sets and 6 as validation sets, and 3 randomly selected as test sets.

[0161] In the task of medical image segmentation, proper preprocessing can significantly improve the performance of the model. The dataset was subjected to green channel extraction, contrast-limited adaptive histogram equalization (CLAHE), and gamma correction in order to optimize the quality of the color ironmaking images.

[0162] In order to reduce the color differences caused by different acquisition equipment or lighting conditions, we first extracted the green channel from the RGB image. The green channel usually has a higher contrast, which is particularly beneficial for the extraction of ironmaking products. Then, we converted the RGB image into a grayscale image using the formula to further reduce the interference of color information on segmentation:

[0163] (twenty four)

[0164] in, represents the pixel intensity of the grayscale image, , and Represents the pixel intensity of the red, green, and blue channels, respectively.

[0165] Next, to enhance the contrast between the product and the background in the image, we applied the CLAHE operation. CLAHE can suppress noise while enhancing contrast. This method sets a threshold, evenly distributes pixel values ​​exceeding the threshold to other gray levels, forms a new histogram, and performs adaptive equalization on it. We set the threshold to 2 and select an 8×8 grid size to divide the image into non-overlapping areas and process them separately to ensure the stability and consistency of the results.

[0166] In order to further highlight the difference in brightness between the product and the surrounding tissue, we use gamma correction technology. Gamma correction adjusts the gamma curve of the image and changes the brightness distribution, thereby enhancing the contrast between low-brightness and high-brightness areas. The mathematical expression of gamma correction is:

[0167] (25)

[0168] in, and denote the output and input pixel intensities, respectively, is the gamma value. In this study, we set the gamma value to 0.8 to enhance the contrast of low-brightness areas and highlight the details of ironmaking products. After gamma correction, the brightness difference between the product area and the background is significantly increased, especially the structure of small products is more clearly visible.

[0169] After a series of preprocessing operations, the ironmaking images with low contrast and blurred colors have been significantly improved. The processed images not only highlight the product details, but also significantly enhance the contrast between the products and the surrounding non-product areas. In order to increase the diversity of training data, we cropped the preprocessed images into 48×48 non-overlapping small blocks. This slicing method can increase the number of training samples, alleviate the overfitting problem of deep learning models, and provide the model with more diverse feature samples, thereby improving the performance of ironmaking product segmentation tasks.

[0170] From the perspective of accuracy (Acc), EFDG-UNet reaches 0.9736, which is competitive with current high-performance models (such as UNet3+ and SDDC-Net), and significantly higher than traditional U-Net (0.9691) and LadderNet (0.9561), indicating its accuracy in the overall segmentation task. The specificity (Sp) is 0.9856, which is also in the leading position, second only to a few models such as ResUNet++ (0.9837), reflecting its efficient recognition ability of background areas.

[0171] In terms of sensitivity (Se), EFDG-UNet's performance (0.8438) is significantly better than classic methods (such as U-Net's 0.7948) and some modern models (such as ResUNet++'s 0.7856), indicating that it has an advantage in capturing small products. Although it is slightly lower than SDDC-Net (0.8603) and DCA-CNN (0.8745) in sensitivity, its high balance between specificity and accuracy makes it more balanced in overall performance.

[0172] EFDG-UNet's AUC (0.9886) and F1 value (0.8412) are both among the top. Among them, AUC reflects the robustness of the segmentation model under different thresholds, while the F1 value reflects the coordination ability between precision and recall. Compared with other recently proposed models (such as UNet3+ with an F1 value of 0.8254), EFDG-UNet's segmentation performance shows better comprehensiveness.

[0173] EFDG-UNet performs well in multiple key indicators, especially in accuracy (Acc) and comprehensive segmentation performance. Its high sensitivity (Se) and specificity (Sp) values ​​indicate that EFDG-UNet can well balance the segmentation capabilities of small products and background areas. Its excellent performance in AUC and F1 indicators further verifies the robustness and reliability of the model, making it one of the leading methods for ironmaking product segmentation tasks.

[0174] EFDG-UNet shows significant advantages in segmenting complex product areas. In the segmentation of coarse products, EFDG-UNet shows higher coherence and completeness. Compared with other models, EFDG-UNet can accurately maintain the overall shape of the product. In the segmentation of tiny products, EFDG-UNet can not only accurately identify product branches and intersections, but also significantly reduce misidentification and avoid incorrectly segmenting background areas as products. Overall, EFDG-UNet has stronger robustness and higher segmentation accuracy in fine-grained product segmentation tasks.

[0175] Furthermore, the step of obtaining the wrinkle smoothing area ratio in the output image and determining the smoothing quality of the ironmaking product to be evaluated includes:

[0176] Obtaining the number N1 of rough feature points and N2 of smooth feature points in the output image;

[0177] The calculated ratio of the rough feature points to the total feature points, the ratio of the hole area on the surface of the ironmaking product, and the surface wrinkle smoothing area ratio The calculation formula is as follows:

[0178] ;

[0179] Where: —Surface wrinkle smooth area ratio;

[0180] —Number of rough feature points;

[0181] —Number of smoothing feature points;

[0182] Determining the smoothness quality of the ironmaking product to be evaluated according to the surface wrinkle smoothness area ratio;

[0183] The smooth quality includes excellent ironmaking products, good ironmaking products, qualified ironmaking products, and unqualified ironmaking products.

[0184] In this embodiment, the smoothness quality is determined by referring to Table 1;

[0185] Table 1 Smooth quality decision table

[0186] Surface wrinkle smooth area ratio K (%) Smooth quality grade Remark K≤1% Excellent ironmaking products The surface is smooth and clean, without any quality defects; 1%＜K≤5% Good ironmaking products The surface is smooth with a few holes; 5%＜K≤10% Qualified ironmaking products The surface is relatively flat, with some local wrinkles and defects, and requires certain treatment. K＞10% Substandard ironmaking products The surface is uneven and has many wrinkles and defects, so additional color difference measurement is required. If the color difference meets the standard, it needs to be smoothed to reach the level of a good ironmaking product.

[0187] According to a second embodiment of the present invention, the present invention claims protection for an ironmaking product quality evaluation device based on multi-source data, comprising:

[0188] one or more processors;

[0189] A memory having one or more programs stored thereon, when the one or more programs are executed by the one or more processors, the one or more processors implement the ironmaking product quality evaluation method based on multi-source data.

[0190] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0191] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. The above is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of this application, or directly or indirectly used in other related technical fields, is also included in the patent protection scope of the present application.

[0192] The specific implementation methods of the invention are described in detail above, but they are only examples, and the present application is not limited to the specific implementation methods described above. For those skilled in the art, any equivalent modification or substitution of the invention is also within the scope of the present application, and therefore, the equalization, modification, and improvement made without departing from the spirit and principle of the present application should be included in the scope of the present application.

Claims

1. A method for evaluating the quality of ironmaking products based on multi-source data, characterized in that: include: Collecting a characteristic graph of the ironmaking product to be evaluated, inputting the characteristic graph into an encoder, and obtaining a plurality of intermediate coding features and a target coding feature; Inputting the plurality of intermediate coding features into a multi-scale feature fusion module through jump connection to obtain a plurality of corresponding fused coding features, and inputting the target coding features into a parallel information focusing module to obtain a parallel information focusing feature; The plurality of fused coding features are jump-connected and input into a decoder together with the parallel information focusing feature to obtain a plurality of candidate product features; Obtaining an output image of the ironmaking product to be evaluated based on the characteristic analysis of the plurality of candidate products; Obtaining a wrinkle smoothing area ratio in the output image to determine the smoothing quality of the ironmaking product to be evaluated; The smooth quality of a plurality of continuous ironmaking products to be evaluated captured by video is statistically evaluated for defects to obtain a quality defect evaluation of the ironmaking product.

2. The ironmaking product quality evaluation method based on multi-source data according to claim 1 is characterized in that: Encoder, including: Adopting adaptive gated residual blocks, combining residual connections and adaptive gating mechanisms, and introducing spatial random dropout modules to reduce overfitting; In the encoder, the adaptive gated residual block contains two convolutional layers and their corresponding batch normalization and activation layers.

3. The ironmaking product quality evaluation method based on multi-source data according to claim 2 is characterized in that: The step of inputting the feature map into an encoder to obtain a plurality of intermediate encoding features and a target encoding feature comprises: The first feature of the feature map is processed by the first convolutional layer, and is subjected to batch normalization and the ReLU activation function of the activation layer to obtain a first intermediate coding feature; Passing the first intermediate coding feature through the second convolution layer and the batch normalization process again to obtain a second intermediate coding feature; The spatial random dropout module processes the second intermediate coding feature to reduce noise in the feature map and enhance the regularization effect; The adaptive gating mechanism of the adaptive gating residual block extracts global context information through global average pooling, and generates gating coefficients through two fully connected layers and Sigmoid activation function to adjust the weight of the residual term; The residual connection part performs channel matching on the first feature to ensure that the number of channels is consistent with the main branch feature, and then performs weighted addition with the second intermediate coding feature, and obtains the corresponding target coding feature through the ReLU activation function.

4. The method for evaluating the quality of ironmaking products based on multi-source data according to claim 3, characterized in that: The step of inputting the target coding feature into a parallel information focusing module to obtain a parallel information focusing feature comprises: A channel attention module is used to aggregate the global features of each channel, generate channel importance weights, and adjust the channel expression ability of the input feature map; Input the target encoding features, extract the global descriptor of each channel through the global average pooling operation, realize feature compression and expansion through two layers of fully connected networks, and generate channel weights; The generated channel weight is multiplied element by element with the input feature map to complete the enhancement in the channel dimension; The spatial attention module captures global spatial information and generates spatial attention weights by performing maximum pooling and average pooling on the feature map in the spatial dimension. Performing maximum pooling and average pooling operations on the first feature in the channel dimension to generate two spatial feature maps; The two spatial feature maps are concatenated in the channel dimension, and the result is passed through a 7×7 convolutional layer to generate spatial attention weights; The spatial weight is multiplied element by element with the input feature map to enhance the spatial dimension; After completing the channel and space attention optimization, the final parallel information focusing feature is generated through a fusion operation.

5. The method for evaluating the quality of ironmaking products based on multi-source data according to claim 4, characterized in that: The skip connection of the multiple intermediate coding features is input into the multi-scale feature fusion module to obtain the corresponding multiple fused coding features, including: Presetting the skip connection features of the encoder, performing 3×3 convolution, batch normalization, and ReLU activation operations on each intermediate encoding feature to obtain a uniformly processed feature; In each decoding layer, the intermediate encoded features are bilinearly interpolated to adjust the target resolution; The Transformer attention mechanism is used to capture the dependencies between multi-scale features, stack all adjusted features along the new dimension, and reshape them into sequence representations; Reshape the features into a tensor of the target shape and further permute the dimensions to meet the input requirements of the Transformer encoder; After passing through the Transformer encoder, the global context information is integrated and multiple fused coding features are output.

6. The method for evaluating the quality of ironmaking products based on multi-source data according to claim 5, characterized in that: The jump connection of the multiple fusion coding features and the parallel information focusing features are input into a decoder to obtain multiple candidate product features, including: Adopting adaptive gated residual upsampling block including upsampling and skip connection operations; The input parallel information focusing features are upsampled to half the size of the feature map through bilinear interpolation, and then concatenated with the feature map of the corresponding layer in the multi-scale feature fusion module to integrate the semantic information of the encoding stage; The spliced ​​feature maps are then processed by an adaptive gated residual block to enhance the feature fusion effect and restore image details to obtain multiple candidate product features.

7. The method for evaluating the quality of ironmaking products based on multi-source data according to claim 5, characterized in that: The step of obtaining the wrinkle smoothing area ratio in the output image and determining the smoothing quality of the ironmaking product to be evaluated comprises: Obtaining the number N1 of rough feature points and N2 of smooth feature points in the output image; The calculated ratio of the rough feature points to the total feature points, the ratio of the hole area on the surface of the ironmaking product, and the surface wrinkle smoothing area ratio The calculation formula is as follows: ； Where: —Surface wrinkle smooth area ratio; —Number of rough feature points; —Number of smoothing feature points; Determining the smoothness quality of the ironmaking product to be evaluated according to the surface wrinkle smoothness area ratio; The smooth quality includes excellent ironmaking products, good ironmaking products, qualified ironmaking products, and unqualified ironmaking products.

8. An ironmaking product quality evaluation device based on multi-source data, characterized in that: include: one or more processors; A memory having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement a method for evaluating the quality of ironmaking products based on multi-source data according to any one of claims 1 to 7.

Images