A steel non-metallic inclusion segmentation method, system, computer device, computer readable storage medium and computer program product

Abstract

The application discloses a steel non-metallic inclusion segmentation method, system, computer equipment, computer readable storage medium and computer program product, and the method comprises the following steps: step 1: obtaining the metallographic micrograph of various non-metallic inclusions as an original sample; step 2: inputting the processed image data into a backbone network, and extracting multi-scale global dependence features of inclusions through a multi-stage self-attention mechanism; step 3: inputting the multi-scale global dependence features into a decoder, and obtaining high-dimensional feature descriptions by using a pyramid pooling and top-down feature fusion mechanism; step 4: performing scale unification and feature aggregation on the high-dimensional feature descriptions to obtain prediction probability distributions of each pixel point belonging to different non-metallic inclusion categories; and step 5: calculating a hybrid loss function based on the prediction probability distribution of each pixel point and its real mask to optimize the model.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and deep learning technology, and in particular to a method, system, computer device, computer-readable storage medium, and computer program product for segmenting non-metallic inclusions in steel. Background Technology

[0002] With the increasing demands on the service life of components in aerospace and marine engineering, accurate quantitative characterization of non-metallic inclusions in steel has become a key aspect of improving steel quality and quality control. Traditional inclusion detection mainly relies on manual microscopic observation and analysis or simple image processing based on grayscale thresholds. This detection method is not only inefficient and highly subjective, requiring extremely high skill from the inspectors, but also struggles to achieve accurate segmentation and classification when faced with inclusions of complex shapes.

[0003] In recent years, deep learning methods based on convolutional neural networks (CNNs) have been increasingly applied to the automated detection of inclusions, improving detection efficiency to some extent. However, due to the inherent local receptive field of convolutional operations, existing CNN models face significant bottlenecks in handling subtle feature relationships and global semantic understanding against a large-scale metallographic background. Furthermore, when dealing with non-metallic inclusions of varying shapes and sizes, existing CNN models often struggle to simultaneously achieve accurate local feature localization and capture of long-range global dependencies, resulting in limited segmentation accuracy. Therefore, there is an urgent need in this field for a new method that can overcome the limitations of local receptive fields and possess strong global semantic understanding and multi-scale feature fusion capabilities to achieve high-precision, intelligent quantitative analysis of non-metallic inclusions in steel.

[0004] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing a method, system, computer equipment, computer-readable storage medium, and computer program product for segmenting non-metallic inclusions in steel. Based on visual Transformer and multi-scale feature fusion, it is suitable for high-precision automated detection and rating of non-metallic inclusions in metallographic microscopic images of steel. It solves the problems of limited multi-scale feature extraction and lack of long-distance feature dependence in CNN-based segmentation methods under complex metallographic scenes, which leads to the easy omission of small point inclusions and the easy breakage and interference of large-scale inclusions by complex backgrounds.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A method for separating non-metallic inclusions in steel includes the following steps: Step 1: Obtain metallographic microscopic images of various non-metallic inclusions as raw samples, and perform data restoration and dynamic cropping preprocessing; Step 2: Input the processed image data into the backbone network and extract the multi-scale global dependency features of the inclusions through a multi-stage self-attention mechanism; Step 3: Input the multi-scale global dependency features into the decoder, and use pyramid pooling and top-down feature fusion mechanisms to obtain high-dimensional feature descriptions; Step 4: Perform scale unification and feature aggregation on the high-dimensional feature descriptions to obtain the predicted probability distribution of each pixel belonging to different non-metallic inclusion categories; Step 5: Calculate the hybrid loss function based on the predicted probability distribution of each pixel and its true mask, optimize the model, and use the optimized model to output the final pixel-level semantic segmentation mask.

[0007] Furthermore, in step 1, the dynamic cropping process includes the following steps: For the original metallographic micrograph and the corresponding ground truth mask of inclusion defects, the mask set of independent defects is extracted using the connected component analysis algorithm. Calculate its zeroth-order spatial moment and first-order spatial moment to obtain the geometric centroid of the defect. The formula is as follows: ; ; in, For the mask image in coordinates Pixel indicator value at; Cut to the preset target size Adaptive dynamic trimming is performed based on the defect scale: When the maximum side length of the defect At that time, it was determined to be a small-scale target; based on the geometric centroid Construct a single bounding box, calculate the initial x and y coordinates (x1 and y1), and obtain the clipping boundary after translation correction. The formulas for calculating x1 and y1 are as follows: ; ; in, The width is the original image width. This is the floor operator; When the maximum side length of the defect At that time, it was determined to be a large-scale target; the outer margin was set. Calculate the eccentricity parameter of the defect. Construct a dynamic sliding step decay function for shape awareness. Based on geometric centroid and defect physical boundary Generate a set of spatial slice anchor points The formula is as follows: ; ; ; in, The minimum side length of the defect; As the reference sliding step size, This is the attenuation control coefficient. The aspect ratio threshold; and These are the coordinates of the left and top boundaries of the minimum bounding rectangle of the defect, respectively. P is the sliding step index, D is the outer margin, W is the dynamic sliding step size, H is the original image width, and H is the original image height. The final output is a uniform resolution single-target metallographic image slice containing topological continuity and its corresponding ground truth mask. For the cropped uniform resolution ground truth mask, construct RLE encoding and point it to its corresponding metallographic image, compress the data, and divide it into training and test sets.

[0008] Furthermore, in step 2, extracting multi-scale features of inclusions through a multi-stage self-attention mechanism includes the following steps: Preprocessed metallographic image The input metallographic image is fed into a ResT-type Transformer backbone network. It first undergoes preliminary spatial downsampling and feature channel augmentation via the Stem module, generating a dimension of [missing information]. Initial feature embedding; Subsequently, the feature sequence flows through four concatenated feature extraction stages in sequence, and at the beginning of each stage, spatial downsampling is performed using a block embedding layer; Subsequently, the feature tensor is normalized by the layer and then enters the efficient multi-head self-attention module. The efficient multi-head self-attention module compresses the spatial dimension of the keys and values ​​in the attention mechanism through strided depthwise convolution operations, and extracts long-distance global spatial dependencies. The specific process is as follows: for an input length of... Dimensions Feature sequences Highly efficient multi-head self-attention module for query matrix Maintain full-resolution projection, key matrix Sum matrix Introducing spatial dimensionality reduction operators Query matrix Key matrix Value matrix The calculation formula is as follows: ; ; ; in, , , Query matrix Key matrix Value matrix The corresponding learnable linear mapping weight matrix, This indicates the use of step size. Depthwise separable convolution operations reduce the length of the key-value pair feature space sequence from... Compress to ; Subsequently, the reconstructed global attention response matrix Z is calculated, as shown in the following formula: ; in, This is a relative position offset term. This is the scaling factor; After four stages of extraction, the network outputs multi-layer feature sets corresponding to different resolutions and number of channels. .

[0009] Furthermore, in step 3, a high-dimensional feature description is obtained using pyramid pooling and a top-down feature fusion mechanism, which includes the following steps: Targeting the deepest global semantic features in the feature set It is then input into the pyramid pooling module, which applies it in parallel. Adaptive average pooling operators with four different grid sizes Extracting multi-scale context and applying it via bilinear interpolation operator After restoring the scale, compared with the original Cascade along the channels to output deep aggregated features The formula is as follows: ; in, For multi-scale context aggregation functions, This is a matrix concatenation operation along the channel dimension of the feature map. It is a bilinear interpolation operator. To pass different grid sizes The adaptive average pooling operator, This is a two-dimensional convolution operator. This is the dimensionality-reduced convolution kernel weight matrix; Subsequently, a top-down feature pyramid cascaded feedback network is constructed to establish fused features. The stepwise feedback recursive equation is as follows: ; ; in, , for Smooth convolution operator, This is a convolution mapping function used to generate spatial heterogeneity gating masks. It is the Sigmoid activation function. For spatial heterogeneity gated mask matrix, For element-wise multiplication of the characteristic tensor, The element-wise summation of the feature tensors For double spatial upsampling, This is a deep aggregation feature. These are shallow features of the same level of backbone. Map weights to channel alignment for channel dimension alignment; Finally, a fused feature set containing full-scale information is obtained. .

[0010] Furthermore, in step 4, the steps for obtaining the predicted probability distribution of each pixel belonging to different inclusion categories are as follows: A fusion feature set containing full-scale information Features at each scale are upsampled to the target baseline resolution and stitched along the channels to construct a high-dimensional feature description of full-scale information. ; Subsequently, the high-dimensional feature description After being projected through the convolutional layer to the channel space corresponding to the target defect category, the corresponding... The output tensor of each classification category, where... This indicates the total number of classification categories for non-metallic inclusions; Finally, the tensor is restored to the spatial scale of the original input image through bilinear interpolation, and then converted into a probability distribution through an activation function to obtain the predicted probability of each pixel in the original image belonging to a specific non-metallic inclusion category.

[0011] Furthermore, in step 5, the steps for optimizing the model based on the predicted probability distribution of each pixel and its actual mask to calculate the hybrid loss function are as follows: A hybrid loss function combining binary cross-entropy and Dice coefficients is constructed for joint optimization of the model; the binary cross-entropy loss term... The formulas for calculating the Dice coefficient loss term are as follows: ; ; in, The total number of pixels in the image. The total number of categories. For pixels Does it belong to a category? The true label, Predict pixels for the model Category The probability, It is a smoothing constant; Constructing as training rounds increase Dynamically evolving adaptive weight adjustment function This makes it change with the number of training rounds. Dynamic evolution, The formula is as follows: ; ; in, For the first The percentage of pixel frequency of inclusions in the current batch. , , To control the hyperparameters, this function ensures that the model focuses on global search in the early stages of training and accurately focuses on hard-to-distinguish samples with very small areas in the later stages. By combining the predicted probability distribution with the true mask, the joint optimization formula for the total loss function is constructed as follows: ; The total loss is minimized by using the backpropagation algorithm, and the model parameters are iteratively updated using the optimization algorithm. The model is iteratively optimized until it converges, so as to output a high-precision pixel-level semantic segmentation mask during the inference stage.

[0012] To achieve the above objectives, the present invention also employs the following technical solution: A system for separating non-metallic inclusions in steel, used to implement any of the methods provided by the present invention, the system comprising the following modules: The first module is the metallographic microscopic image dynamic cropping and preprocessing module; The second module is the backbone network multi-scale global feature extraction module. The third module is the pyramid pooling and multi-level feature fusion module. The fourth module is the feature aggregation and probability distribution prediction module. The fifth module is the hybrid loss joint optimization and mask output module.

[0013] To achieve the above objectives, the present invention also employs the following technical solution: A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the methods provided by the present invention.

[0014] To achieve the above objectives, the present invention adopts the following technical solution: A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of any of the methods provided in this invention.

[0015] To achieve the above objectives, the present invention adopts the following technical solution: A computer program product includes a computer program that, when executed by a processor, implements the steps of any of the methods described in this invention.

[0016] Compared with the prior art, the beneficial effects of this invention are as follows: 1. To address the limitations of CNN-based segmentation methods in complex metallographic scenarios, such as limited multi-scale feature extraction and lack of long-range feature dependencies, leading to the easy omission of small point inclusions and the fragility and susceptibility to complex background interference in large-scale inclusions, a comparative experiment was conducted using the same metallographic microscopic image data of non-metallic inclusions in steel as the method of this invention. The same data processing flow and evaluation metrics were used to evaluate the performance of different segmentation methods. Specifically, the method of this invention uses ResT-Small as the feature extraction backbone network, combined with the aforementioned adaptive dynamic cropping, multi-scale feature fusion, and dynamic weighted loss function for non-metallic inclusion segmentation; the comparative method uses the Deeplabv3+ network with EfficientNet-B4 as its backbone network. The segmentation results of the two methods on the same test data were compared to evaluate the pixel-level segmentation capability of the method of this invention for non-metallic inclusion regions in steel.

[0017] For evaluation metrics, the mean Dice coefficient (mDice) and mean Intersection over Union (mIoU) were used as the segmentation accuracy metrics. mDice measures the degree of overlap between the predicted segmented region and the ground truth labeled region, while mIoU measures the intersection-over-union ratio between the predicted and ground truth regions. Higher values ​​for both metrics indicate that the segmentation results of the non-metallic inclusions obtained by the model are closer to the ground truth labeled results, and the better the segmentation performance.

[0018] Table 1: Performance metrics of different models on the test set

[0019] As shown in Table 1, under the same test data and evaluation metrics, the Deeplabv3+ model has an mDice of 90.346% and an mIoU of 88.849%; after adopting the method of this invention, the mDice reaches 95.613% and the mIoU reaches 93.440%. Compared with the Deeplabv3+ method, the method of this invention improves the mDice by 5.267 percentage points and the mIoU by 4.591 percentage points.

[0020] The comparison results above demonstrate that the method of this invention can effectively improve the segmentation accuracy of non-metallic inclusions in steel. Specifically, this invention improves the effective pixel ratio of inclusion targets at different scales through adaptive dynamic cropping, establishes global feature associations of inclusion regions through the efficient self-attention mechanism in ResT-Small, enhances the feature representation of targets and boundary regions at different scales through multi-scale spatial context information extraction and multi-level feature fusion, and improves the model optimization effect under class imbalance conditions through a dynamically weighted BCE-Dice joint loss function. The combined effect of these technical measures enables the method of this invention to outperform the Deeplabv3+ comparison method in both mDice and mIoU evaluation metrics.

[0021] 2. A reconstructed adaptive dynamic cropping preprocessing mechanism effectively solves the problems of extreme scale differences and background interference: This invention addresses the characteristics of metallographic images with extremely high resolution and minimal defects by designing a dynamic cropping and anti-cross-boundary offset strategy based on connected component analysis. This strategy, while unifying the model input resolution, adaptively adjusts the cropping window according to the target size and eliminates irrelevant defect interference, preserving the local contextual texture of inclusions to the greatest extent. Furthermore, this processing method avoids feature distortion caused by conventional global scaling, closely aligning with the actual distribution of high-throughput, unstructured metallographic microscopic images from industrial production lines, providing high-quality, clean samples for efficient model learning.

[0022] 3. A hybrid loss function specifically for metallographic defects was constructed to alleviate the problem of imbalanced extreme samples: This invention analyzes the problem of background pixels dominating in metallographic segmentation. After extensive and rigorous experimental verification and testing, a hybrid loss function (BCEDiceLoss) combining binary cross-entropy (BCE) and Dice coefficient weighting was used as the optimal optimization objective. By reasonably adjusting the loss weights, this function can guide the model to focus on the overall morphology of difficult samples and small inclusions during backpropagation, significantly reducing the occurrence of mask breaks and holes, and improving the robustness of model training.

[0023] 4. A significant leap in detection accuracy under complex working conditions is achieved by relying on an advanced visual Transformer architecture: Under the synergistic effect of the aforementioned preprocessing strategies and dedicated optimization objectives, this invention facilitates the development of a visual Transformer architecture for capturing long-distance dependencies and fusing features across all scales. To verify the effectiveness of the proposed non-metallic inclusion segmentation method in steel, the same experimental data as the proposed method were used to compare the proposed method with the Deeplabv3+ segmentation method. The proposed method uses ResT-Small as the backbone network, combined with adaptive dynamic pruning, multi-scale feature fusion, and a dynamic weighted loss function; the comparison method uses the Deeplabv3+ model with EfficientNet-B4 as the backbone network. Under the same test data and evaluation metrics, the non-metallic inclusion segmentation results of the two methods were quantitatively compared. The evaluation metrics included the average Dice coefficient mDice and the average intersection-union ratio mIoU. In particular, it improved the perception and localization capabilities of tiny point-like inclusions, providing reliable visual technology support for high-precision and intelligent rating of steel quality. Attached Figure Description

[0024] Figure 1 A flowchart of a method for separating non-metallic inclusions in steel; Figure 2 A schematic diagram of the multi-stage visual Transformer backbone network (ResT-S) architecture; Figure 3 This is a schematic diagram of the pyramid pooling and multi-level feature fusion decoder (UperHead) structure. Detailed Implementation

[0025] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0026] Example 1: A method for separating non-metallic inclusions in steel, such as Figure 1-3 As shown, it includes the following steps: Step 1: Obtain metallographic microscopic images of various non-metallic inclusions as original samples, and perform data restoration and dynamic cropping preprocessing.

[0027] In this embodiment, in step 1, the original unstructured dataset file is parsed, the RGB metallographic microscopic image is reconstructed using a decoding algorithm, and the corresponding binary mask image is generated based on the labeled coordinates.

[0028] Extracting the mask set of independent defects using connected component analysis algorithms Calculate its zeroth-order spatial moment and first-order spatial moment to obtain the geometric centroid of the defect. The formula is as follows: ; ; in, For the mask image in coordinates The pixel indicator value at the location (1 for the target area, 0 for the background).

[0029] Cut to the preset target size Adaptive dynamic trimming is performed based on the defect scale.

[0030] When the maximum side length of the defect At that time, it was determined to be a small-scale target; based on the geometric centroid To prevent clipping from exceeding the bounds, a translation correction is introduced. The initial x and y coordinates (x1 and y1) are calculated to obtain the clipping boundary after translation correction. The formulas for calculating x1 and y1 are as follows: ; ; in, The width is the original image width. This is the floor operator.

[0031] When the maximum side length of the defect When the target is identified as a large-scale object, to avoid the breakage of the edge topological features of slender defects due to a fixed step size, this embodiment proposes an adaptive step size dynamic optimization model based on the defect morphology distribution; and sets the outer expansion margin. Calculate the eccentricity parameter of the defect. Construct a dynamic sliding step decay function for shape awareness. Based on geometric centroid and defect physical boundary Generate a set of spatial slice anchor points The formula is as follows: ; ; ; in, The minimum side length of the defect; As the reference sliding step size, This is the attenuation control coefficient. The aspect ratio threshold; and These are the coordinates of the left and top boundaries of the minimum bounding rectangle of the defect, respectively. Here, P is the sliding step index, P is the outer margin, and D is the dynamic sliding step size. To determine the eccentricity of the defect The adaptively calculated dynamic sliding step size, where W is the width of the original image and H is the height of the original image; the final output is a uniform resolution single-target metallographic image slice containing topological continuity and its corresponding ground truth mask.

[0032] As the inclusions become thinner and longer, the sliding step size automatically decreases exponentially, thereby enabling denser overlapping sampling in the high-frequency region of the defect. Mathematically, this ensures the strict morphological continuity of the thin, long non-metallic inclusions across slices.

[0033] For the cropped uniform resolution ground truth mask, construct RLE encoding and point it to its corresponding metallographic image, compress the data, and divide it into training and test sets.

[0034] In this embodiment, all cropped defects have a uniform resolution. For the very few cases where the width or height is still smaller after cropping... The image is expanded at the edges to a uniform resolution while ensuring that the image texture size and stretching remain unchanged.

[0035] The cropping process applies to both the metallographic image and its corresponding mask image.

[0036] After cropping, for the generated uniform resolution single-target defect metallographic image slices, the Run-Length Encoding (RLE) algorithm is used to extract the transition positions of pixel values ​​in the corresponding mask image, and an encoding sequence is constructed. ,in and The first The starting index and length of each consecutive target pixel segment are determined. Finally, the serialized data is divided into a model training set and a test set according to a preset ratio.

[0037] Step 2: Input the processed image data into the backbone network and extract the multi-scale global dependency features of the inclusions through a multi-stage self-attention mechanism.

[0038] In this embodiment, in step 2, the preprocessed metallographic image is... The input metallographic image is fed into a ResT-type Transformer backbone network. It first undergoes preliminary spatial downsampling and feature channel augmentation via the Stem module, generating a dimension of [missing information]. The initial feature embedding.

[0039] Subsequently, the feature sequence flows through four concatenated feature extraction stages in sequence, with spatial downsampling performed at the beginning of each stage using a block embedding layer.

[0040] Subsequently, the feature tensor is normalized by the layer and then enters the efficient multi-head self-attention module (EMSA). The efficient multi-head self-attention module compresses the spatial dimension of the keys and values ​​in the attention mechanism through strided depth convolution operations, and extracts long-distance global spatial dependencies.

[0041] The specific process is as follows: for an input length of... Dimensions Feature sequences Highly efficient multi-head self-attention module for query matrix Maintain full-resolution projection, key matrix Sum matrix Introducing spatial dimensionality reduction operators to query matrices Key matrix Value matrix The calculation formula is as follows: ; ; ; in, , , Query matrix Key matrix Value matrix The corresponding learnable linear mapping weight matrix, This indicates the use of step size. Depthwise separable convolution operations reduce the length of the key-value pair feature space sequence from... Compress to .

[0042] Subsequently, the reconstructed global attention response matrix Z is calculated, as shown in the following formula: ; in, This is a relative position offset term used to explicitly encode the two-dimensional spatial relative geometric distance between pixel blocks in a metallographic image slice. This is the scaling factor.

[0043] This mechanism preserves long-distance dependency features while reducing computational complexity from Constraint to .

[0044] After four stages of extraction, the network outputs multi-layer feature sets corresponding to different resolutions and number of channels. This set macroscopically represents the hierarchical features from shallow local textures in metallography to deep global semantics.

[0045] Step 3: Input the multi-scale global dependency features into the decoder and use pyramid pooling and top-down feature fusion mechanisms to obtain high-dimensional feature descriptions.

[0046] In this embodiment, in step 3, the deepest global semantic features in the feature set are targeted. It is then input into the pyramid pooling module (PPM), which applies the data in parallel. Adaptive average pooling operators with four different grid sizes Extracting multi-scale context and applying it via bilinear interpolation operator After restoring the scale, compared with the original Cascade along the channels to output deep aggregated features The formula is as follows: ; in, For multi-scale context aggregation functions, This is a matrix concatenation operation along the channel dimension of the feature map. It is a bilinear interpolation operator. To pass different grid sizes The adaptive average pooling operator, This is a two-dimensional convolution operator. This is the dimensionality-reducing convolution kernel weight matrix, used to compress the number of channels after concatenation and expansion to the standard representation dimension.

[0047] Subsequently, a top-down feature pyramid cascaded feedback network is constructed to establish fused features. The stepwise feedback recursive equation is as follows: ; ; in, , for A smooth convolution operator is used to eliminate the aliasing effect caused by upsampling. This is a convolution mapping function used to generate spatial heterogeneity gating masks. It is the Sigmoid activation function. For spatial heterogeneity gated mask matrix, For element-wise multiplication of the characteristic tensor, The element-wise summation of the feature tensors For double spatial upsampling, This is a deep aggregation feature. These are shallow features of the same level of backbone. Channel alignment mapping weights are used for channel dimension alignment; this process is recursively applied level by level. This recursive equation adaptively evaluates the information confidence of deep semantics and shallow texture at each pixel location, achieving high-precision pixel-level alignment between metallographic deep semantics and shallow texture, and finally obtaining a fused feature set containing full-scale information. .

[0048] In this embodiment, in step 3, considering that metallographic scratches and grain boundary noise are extremely strong in shallow features, this embodiment abandons the traditional direct element-by-element addition and fusion, and proposes a heterogeneous feature alignment mechanism based on context-aware gating.

[0049] Step 4: Perform scale unification and feature aggregation on the high-dimensional feature descriptions to obtain the predicted probability distribution of each pixel belonging to different non-metallic inclusion categories.

[0050] In this embodiment, in step 4, the fused feature set containing full-scale information is... Features at each scale in the data are upsampled to the target reference resolution. (and splice along the channels to construct a high-dimensional feature description of full-scale information) .

[0051] Subsequently, the high-dimensional feature description After being projected through the convolutional layer to the channel space corresponding to the target defect category, we obtain... The output tensor of each classification category ( (This represents the number of inclusion categories, C=5).

[0052] Finally, bilinear interpolation is used to restore the tensor to the spatial scale of the original input image. The algorithm then transforms the data into a probability distribution using an activation function, thereby obtaining the predicted probability that each pixel in the original image belongs to a specific non-metallic inclusion category.

[0053] Step 5: Calculate the hybrid loss function based on the predicted probability distribution of each pixel and its true mask, optimize the model, and use the optimized model to output the final pixel-level semantic segmentation mask.

[0054] In this embodiment, in step 5, to address the gradient annihilation problem caused by the low pixel ratio of inclusions and the imbalance between positive and negative samples in metallographic micrographs, a hybrid loss function combining scale-sensitive binary cross-entropy and the Dice coefficient (which measures region overlap to enhance shape coherence) is constructed to jointly optimize the model; the binary cross-entropy loss term... The formulas for calculating the Dice coefficient loss term are as follows: ; ; in, The total number of pixels in the image. The total number of categories. For pixels Does it belong to a category? The true label, Predict pixels for the model Category The probability, This is the smoothing constant.

[0055] To overcome extreme imbalance, a system is constructed that adapts to the number of training epochs. Dynamically evolving adaptive weight adjustment function This makes it change with the number of training rounds. Dynamic evolution, The formula is as follows: ; ; in, For the first The percentage of pixel frequency of inclusions in the current batch. , , To control hyperparameters, this function ensures that the model focuses on global search in the early stages of training and accurately focuses on hard-to-distinguish samples with very small areas in the later stages.

[0056] The joint optimization formula for the total loss function is as follows: .

[0057] The total loss is minimized by using the backpropagation algorithm, and the model parameters are iteratively updated using the optimization algorithm. The model is iteratively optimized until it converges, so as to output a high-precision pixel-level semantic segmentation mask during the inference stage.

[0058] The method provided in this embodiment is suitable for automated inspection and intelligent rating of steel quality on modern industrial production lines. Addressing the challenges of strong background noise interference, extremely wide defect scale ranges, and extreme sample imbalance in actual metallographic microscopic images, this embodiment achieves high-precision pixel-level semantic segmentation of various non-metallic inclusions. This solution effectively overcomes the inherent shortcomings of traditional manual microscopic inspection, such as low efficiency and high subjectivity, providing a highly reliable and high-throughput intelligent visual solution for the purity assessment and quality control of steel materials in the high-end equipment manufacturing field.

[0059] Example 2: A non-metallic inclusion separation system in steel, used to implement the method of Embodiment 1, includes: The first module, the metallographic microscopic image dynamic cropping preprocessing module, is used to analyze the original metallographic image and extract the defect boundary; for different defect scales, it adaptively performs anti-outbound cropping or multi-overlapping sliding window segmentation to output single-target slices with uniform resolution; then, it uses the Run-Length Encoding (RLE) algorithm to complete data serialization compression and sample set partitioning.

[0060] The second module, the multi-scale global feature extraction module, is based on a hierarchical visual Transformer backbone network. It receives preprocessed image slices and captures global long-distance dependencies through multi-stage block embedding and multi-head self-attention mechanisms. It extracts and outputs a multi-level set of original feature maps containing different spatial resolutions and channel dimensions.

[0061] The third module, the pyramid pooling and multi-level feature fusion module, receives the original feature map set, aggregates the global context information of the deepest features using pyramid pooling, and then constructs a top-down feature pyramid path, upsamples the deep semantics and fuses them with the shallow details of the same level step by step, and outputs a feature pyramid set that integrates multi-scale information.

[0062] The fourth module, Feature Aggregation and Probability Distribution Prediction Module, uniformly upsamples the feature pyramid set to the reference resolution and stitches them along the channels to construct a high-dimensional feature descriptor; it restores the original input space scale through linear projection and activation function, and outputs the probability distribution tensor of each pixel belonging to a specific non-metallic inclusion category.

[0063] The fifth module, Hybrid Loss Joint Optimization and Mask Output Module: During the training phase, based on the predicted probability distribution and the true mask, a hybrid loss function combining the weighted combination of binary cross-entropy and Dice coefficients is calculated, and the network weights are iteratively updated to alleviate extreme class imbalance; during the inference phase, a high-precision pixel-level semantic segmentation mask is directly output using the converged model.

[0064] Example 3: A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method of Embodiment 1.

[0065] Example 4: A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method of Embodiment 1.

[0066] Example 5: A computer program product includes a computer program that, when executed by a processor, implements the steps of the method of Embodiment 1.

[0067] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for separating non-metallic inclusions in steel, characterized in that, Includes the following steps: Step 1: Obtain metallographic microscopic images of various non-metallic inclusions as raw samples, and perform data restoration and dynamic cropping preprocessing; Step 2: Input the processed image data into the backbone network and extract the multi-scale global dependency features of the inclusions through a multi-stage self-attention mechanism; Step 3: Input the multi-scale global dependency features into the decoder, and use pyramid pooling and top-down feature fusion mechanisms to obtain high-dimensional feature descriptions; Step 4: Perform scale unification and feature aggregation on the high-dimensional feature descriptions to obtain the predicted probability distribution of each pixel belonging to different non-metallic inclusion categories; Step 5: Calculate the hybrid loss function based on the predicted probability distribution of each pixel and its true mask, optimize the model, and use the optimized model to output the final pixel-level semantic segmentation mask.

2. The method for separating non-metallic inclusions in steel according to claim 1, characterized in that, Step 1, the dynamic cropping process includes the following steps: For the original metallographic micrograph and the corresponding ground truth mask of inclusion defects, the mask set of independent defects is extracted using the connected component analysis algorithm. Calculate its zeroth-order spatial moment and first-order spatial moment to obtain the geometric centroid of the defect. The formula is as follows: ； ； in, For the mask image in coordinates Pixel indicator value at; Cut to the preset target size Adaptive dynamic trimming is performed based on the defect scale: When the maximum side length of the defect At that time, it was determined to be a small-scale target; based on the geometric centroid Construct a single bounding box, calculate the initial x and y coordinates (x1 and y1), and obtain the clipping boundary after translation correction. The formulas for calculating x1 and y1 are as follows: ； ； in, The width is the original image width. This is the floor operator; When the maximum side length of the defect At that time, it was determined to be a large-scale target; the outer margin was set. Calculate the eccentricity parameter of the defect. Construct a dynamic sliding step decay function for shape awareness. Based on geometric centroid and defect physical boundary Generate a set of spatial slice anchor points The formula is as follows: ； ； ； in, The minimum side length of the defect; As the reference sliding step size, This is the attenuation control coefficient. The aspect ratio threshold; and These are the coordinates of the left and top boundaries of the minimum bounding rectangle of the defect, respectively. P is the sliding step index, D is the outer margin, W is the dynamic sliding step size, H is the original image width, and H is the original image height. The final output is a uniform resolution single-target metallographic image slice containing topological continuity and its corresponding ground truth mask. For the cropped uniform resolution ground truth mask, construct RLE encoding and point it to its corresponding metallographic image, compress the data, and divide it into training and test sets.

3. The method for separating non-metallic inclusions in steel according to claim 1, characterized in that, Step 2, extracting multi-scale features of inclusions through a multi-stage self-attention mechanism, includes the following steps: Preprocessed metallographic image The input metallographic image is fed into a ResT-type Transformer backbone network. It first undergoes preliminary spatial downsampling and feature channel augmentation via the Stem module, generating a dimension of [missing information]. Initial feature embedding; Subsequently, the feature sequence flows through four concatenated feature extraction stages in sequence, and at the beginning of each stage, spatial downsampling is performed using a block embedding layer; Subsequently, the feature tensor is normalized by the layer and then enters the efficient multi-head self-attention module. The efficient multi-head self-attention module compresses the spatial dimension of the keys and values ​​in the attention mechanism through strided depthwise convolution operations, and extracts long-distance global spatial dependencies. The specific process is as follows: for an input length of... Dimensions Feature sequences The efficient multi-head self-attention module for query matrix Maintain full-resolution projection, key matrix Sum matrix Introducing spatial dimensionality reduction operators Query matrix Key matrix Value matrix The calculation formula is as follows: ； ； ； in, , , Query matrix Key matrix Value matrix The corresponding learnable linear mapping weight matrix, This indicates the use of step size. Depthwise separable convolution operations reduce the length of the key-value pair feature space sequence from... Compress to ; Subsequently, the reconstructed global attention response matrix Z is calculated, as shown in the following formula: ； in, This is a relative position offset term. This is the scaling factor; After four stages of extraction, the network outputs multi-layer feature sets corresponding to different resolutions and number of channels. .

4. The method for separating non-metallic inclusions in steel according to claim 3, characterized in that, Step 3 utilizes pyramid pooling and a top-down feature fusion mechanism to obtain a high-dimensional feature description, which includes the following steps: Targeting the deepest global semantic features in the feature set It is then input into the pyramid pooling module, which applies it in parallel. Adaptive average pooling operators with four different grid sizes Extracting multi-scale context and applying it via bilinear interpolation operator After restoring the scale, compared with the original Cascade along the channels to output deep aggregated features The formula is as follows: ； in, For multi-scale context aggregation functions, This is a matrix concatenation operation along the channel dimension of the feature map. It is a bilinear interpolation operator. To pass different grid sizes The adaptive average pooling operator, This is a two-dimensional convolution operator. This is the dimensionality-reduced convolution kernel weight matrix; Subsequently, a top-down feature pyramid cascaded feedback network is constructed to establish fused features. The stepwise feedback recursive equation is as follows: ； ； in, , for Smooth convolution operator, This is a convolution mapping function used to generate spatial heterogeneity gating masks. It is the Sigmoid activation function. For spatial heterogeneity gated mask matrix, For element-wise multiplication of the characteristic tensor, The element-wise summation of the feature tensors. For double spatial upsampling, This is a deep aggregation feature. These are shallow features of the same level of backbone. Map weights to channel alignment for channel dimension alignment; Finally, a fused feature set containing full-scale information is obtained. .

5. The method for separating non-metallic inclusions in steel according to claim 4, characterized in that, In step 4, the steps to obtain the predicted probability distribution of each pixel belonging to different inclusion categories are as follows: A fusion feature set containing full-scale information Features at each scale are upsampled to the target baseline resolution and stitched along the channels to construct a high-dimensional feature description of full-scale information. ; Subsequently, the high-dimensional feature description After being projected through the convolutional layer to the channel space corresponding to the target defect category, the corresponding... The output tensor of each classification category, where... This indicates the total number of classification categories for non-metallic inclusions; Finally, the tensor is restored to the spatial scale of the original input image through bilinear interpolation, and then converted into a probability distribution through an activation function to obtain the predicted probability of each pixel in the original image belonging to a specific non-metallic inclusion category.

6. The method for separating non-metallic inclusions in steel according to claim 1, characterized in that, In step 5, the following steps are taken to optimize the model based on the calculated hybrid loss function of the predicted probability distribution of each pixel and its actual mask: A hybrid loss function combining binary cross-entropy and Dice coefficients is constructed for joint optimization of the model; the binary cross-entropy loss term... The formulas for calculating the Dice coefficient loss term are as follows: ； ； in, The total number of pixels in the image. The total number of categories. For pixels Does it belong to a category? The true label, Predict pixels for the model Category The probability, It is a smoothing constant; Constructing as training rounds increase Dynamically evolving adaptive weight adjustment function This makes it change with the number of training rounds. Dynamic evolution, The formula is as follows: ； ； in, For the first The percentage of pixel frequency of inclusions in the current batch. , , To control the hyperparameters, this function ensures that the model focuses on global search in the early stages of training and accurately focuses on hard-to-distinguish samples with very small areas in the later stages. By combining the predicted probability distribution with the true mask, the joint optimization formula for the total loss function is constructed as follows: ； The total loss is minimized by using the backpropagation algorithm, and the model parameters are iteratively updated using the optimization algorithm. The model is iteratively optimized until it converges, so as to output a high-precision pixel-level semantic segmentation mask during the inference stage.

7. A system for separating non-metallic inclusions in steel, characterized in that, For implementing the method according to any one of claims 1-6, the system comprises the following modules: The first module is the metallographic microscopic image dynamic cropping and preprocessing module; The second module is the backbone network multi-scale global feature extraction module. The third module is the pyramid pooling and multi-level feature fusion module. The fourth module is the feature aggregation and probability distribution prediction module. The fifth module is the hybrid loss joint optimization and mask output module.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1-6.

Images