A method and system for automatic rating of non-metallic inclusions in steel

Abstract

The application discloses a kind of steel non-metallic inclusion single sheet and full field of view automatic rating method and system.The method obtains metallographic image, through the multiscale result fusion of whole image recognition and local area recognition, obtains non-metallic inclusion candidate area;Candidate area is extracted to connected domain feature, based on geometry, color, gray, texture and spatial feature to class A sulfide, B class alumina, C class silicate and D class spherical oxide are classified, and according to the weighted adjustment of particle distribution characteristics to classification probability, the spatial relationship of B class candidate area is merged and secondary classification;According to the morphological optimization and rating of morphological characteristics of different categories of inclusions.For full field of view image, by splicing adjacent field of view and scanning candidate local area, the worst field of view of each category and system is determined, and the rating result is output.The application can improve the automation degree and accuracy of non-metallic inclusion classification, integration and rating.

Description

Technical Field

[0001] This invention relates to the field of microstructure analysis technology of metallic materials, specifically to a single-sheet and full-field automatic rating method and system for non-metallic inclusions in steel. Background Technology

[0002] Non-metallic inclusions in steel refer to various non-metallic substances formed during steel smelting, solidification, and hot and cold working, ultimately remaining within the steel matrix. Their origins are complex and diverse. These inclusions can be divided into endogenous inclusions and exogenous inclusions. Endogenous inclusions mainly refer to oxides, sulfides, nitrides, and other compounds produced during steel smelting due to the precipitation of oxygen, sulfur, and nitrogen, as well as the use of deoxidizers. Exogenous inclusions originate from the flaking and mixing of slag, refractory materials, or other external substances. The presence of non-metallic inclusions has a significant impact on steel properties; their inhomogeneity directly affects product performance and lifespan. Therefore, strictly controlling their formation and introduction during steelmaking is of great significance for improving steel performance and quality, and thus promoting the development of the manufacturing industry.

[0003] To accurately control the content of non-metallic inclusions in steel, the industry generally uses GB / T 10561 or ASTM E45 standards for determination and rating. However, traditional methods for rating non-metallic inclusions in steel mainly rely on manual microscopic observation and comparison, which has many drawbacks. Specifically, manual rating is inefficient, especially leading to high labor costs in batch testing; manual rating is easily affected by factors such as operator visual fatigue, leading to misjudgments; in addition, manual rating is highly subjective, and different inspectors may have different rating results for the same image, making it difficult to ensure the consistency and repeatability of rating results.

[0004] Existing automated detection technologies mainly fall into two categories: The first category is detection schemes based on deep learning. For example, the existing patent CN114694144A discloses a method for identifying inclusions that combines target detection and semantic segmentation. This scheme only completes the localization, segmentation, and basic size calculation of inclusions, without designing specific post-processing strategies for the different morphologies of the four types of inclusions. It lacks sufficient ability to repair inclusions that are broken, adhered, or clustered, which can easily lead to rating distortion. Furthermore, it only supports single-image field-of-view analysis and cannot achieve the full-field-of-view batch detection required by standards. The second category is schemes based on traditional image processing. For example, the existing patent CN117893472A uses filtering, contour extraction, and geometric feature analysis to achieve inclusion classification and rating. This scheme uses a uniform contour merging and judgment rule for all types of inclusions, which cannot adapt to the morphological differences of different categories of inclusions. It suffers from problems such as confusion between categories A and C, failure to merge clustered particles of category B, and missing classification of spherical inclusions of category D. It has weak generalization ability and also lacks full-field-of-view traversal and worst-case field-of-view localization functions. Summary of the Invention

[0005] The purpose of this invention is to provide a single-sheet and full-field automatic rating method and system for non-metallic inclusions in steel, so as to solve the problems mentioned in the background art.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a method for automatic classification and rating of non-metallic inclusions in steel from a single image, characterized in that it includes: Obtain the metallographic images to be graded; Multi-scale recognition of metallographic images is performed using whole-image recognition and local region recognition. The whole-image recognition results are filtered based on the local region recognition results, and the filtered results are fused with the local region recognition results to obtain candidate regions for non-metallic inclusions. Connected component labeling is performed on candidate regions of non-metallic inclusions, image features of each connected component are extracted, and the image features are input into a classifier to obtain the classification probabilities of type A sulfides, type B alumina, type C silicates and type D spherical oxides. The category weight coefficients are determined according to the particle distribution characteristics of the connected components, and the classification probabilities are weighted and adjusted. The first round of classification results are determined according to the weighted and adjusted classification probabilities. For candidate regions classified as B in the first round, they are merged according to the spatial relationship between adjacent particles. Image features are then re-extracted from the merged B-class candidate regions and a second round of classification is performed. Morphological optimization was performed on inclusions of types A, B, C, and D, and a grade index was calculated based on the length, width, grain size, or number of the optimized inclusions to obtain the rating result of the metallographic image.

[0007] Preferably, it also includes an image preprocessing step: The scale region in the metallographic image is located, the scale region is converted to the HSV color space, and the scale detection region is generated based on the red color range. The effective scale bar is determined based on the length, shape, and area of ​​the red connected region, and the pixel size is determined based on the actual length of the scale bar. Morphological connections are made to the scale bars that are obscured by the scale characters, and the actual span of the scale bars is determined based on the correspondence between the connected areas and the original red areas. The length value and unit information are extracted from the scale area using optical character recognition, and the scale bars and scale text in the metallographic image are removed based on the scale information. When the scale information is not recognized, the preset pixel size or the manually set pixel size is used.

[0008] Preferably, the whole image recognition uses the semantic segmentation network UNet to infer the whole metallographic image, and the local region recognition uses the slice fine-tuning model of UNet to infer the image. UNet sets attention gating at the jump connection between the encoder and the decoder, and sets a void space pyramid pooling bottleneck layer at the end of the encoder.

[0009] Preferably, the whole image recognition adopts a full-image model, and is combined with a boundary dilation algorithm to generate the basic recognition result of the whole image; The local region recognition adopts a slice fine-tuning model, and the recognition results of multiple local regions are superimposed by window weighting to reduce splicing artifacts between local regions; Based on the support level and confidence level of the local identification results, the local candidate regions are filtered according to rules, and the filtered local candidate regions are fused with the basic identification results to obtain the non-metallic inclusion candidate regions.

[0010] Preferably, the normalized contrast is calculated for each connected component, and when the connected component belongs to a low-contrast bright area, the connected component is removed from the candidate region of non-metallic inclusions. Image features of connected components include geometric features, color features, grayscale features, texture features, and spatial features. Among them, geometric features include the bounding rectangle, area, perimeter, Freret diameter, Hu invariant moments, and PCA tip features; color features include BGR mean, standard deviation, extrema, channel ratio, and color richness; grayscale features include mean, variance, Michelson contrast, RMS contrast, and grayscale entropy; texture features include LBP histogram, edge density, and gradient; and spatial features include particle group distribution, centroid spacing, principal axis orientation, and collinearity.

[0011] Preferably, determining the category weight coefficients and performing a second round of classification includes: The foreground region corresponding to the connected component in the metallographic image is segmented twice to obtain the distribution of independent particles within the connected component. Based on the number of independent particles, the ratio of particle area to connected component area, the aspect ratio of the connected component, and the arrangement relationship of particles, the category weight coefficient corresponding to each category is determined, and the classification probability of the corresponding category is multiplied by the category weight coefficient. The first-round classification results are determined based on the weighted adjusted classification probabilities, and connected regions that satisfy the distribution characteristics of Class B particle groups are identified as Class B candidate regions. Adjacent particles with spatial proximity and directional consistency in the B-class candidate regions are merged. Image features are then re-extracted after merging, and a second round of classification is performed on the merged B-class candidate regions. Based on the particle size characteristics of region B and the spatial arrangement relationship between region D and region B in the second round of classification results, the classification results are corrected.

[0012] Preferred morphological optimization and rating include: Calculate the minimum bounding rectangle for the connected regions of type A sulfides and type C silicates, determine the actual maximum width along the main axis of the minimum bounding rectangle, and extend and merge adjacent or intersecting bounding rectangles. Directional expansion is performed on Class B alumina inclusions. Particles in the same group are determined based on the directional and distance relationships between adjacent particles, and particles in the same group are merged. Contrast filtering and size filtering were performed on D-type spherical oxide inclusions, and they were classified into D-type or Ds-type according to the diameter of the filtered particles. The grade index is calculated based on the category, length, width, particle size, or quantity of various non-metallic inclusions, using the rating rules corresponding to the category.

[0013] This invention also proposes a full-field automatic rating method for non-metallic inclusions in steel, comprising: Acquire multiple metallographic images of adjacent fields of view, and stitch them together according to a preset grid layout to form a complete field of view; From the full field of view, candidate local regions are identified for worst-case field of view search, and window scanning is performed on the candidate local regions; Multi-scale identification of non-metallic inclusions within each window is performed, and the identification results of the entire window are fused with the identification results of local areas to obtain candidate regions of non-metallic inclusions within the window. Connectivity features are extracted and classified for candidate regions of non-metallic inclusions within the window. The classification probability is adjusted according to the particle distribution characteristics of the connected regions. Spatial relationship merging and second-round classification are performed on candidate regions classified as class B in the first round. Morphological optimization was performed based on the morphological characteristics of inclusions of types A, B, C, and D, and the category rating results of the window were calculated based on the length, width, particle size, or number of the optimized inclusions. The rating results for each window are compared according to the category and system of non-metallic inclusions to determine the worst field of view representative for each type of non-metallic inclusion; and the location, category and rating result of the worst field of view representative are output.

[0014] The present invention also proposes an automatic rating system for non-metallic inclusions in steel, comprising: an image input device for inputting a metallographic image to be rated or multiple metallographic images of adjacent fields of view; An image recognition device is used to perform whole-image recognition and local region recognition on metallographic images, and to fuse the whole-image recognition results and local region recognition results to obtain candidate regions for non-metallic inclusions. A classification device is used to classify connected regions into Class A sulfides, Class B alumina, Class C silicates, and Class D spherical oxides based on image features of connected regions in candidate regions of non-metallic inclusions, and to adjust the classification probability by weighting according to the particle distribution characteristics of the connected regions. A morphological optimization and rating device is used to perform morphological optimization on connected components according to classification results, and to calculate a level index based on the length, width, particle size or number of the optimized connected components. The result output device is used to output the classification and rating results of non-metallic inclusions. The image input device, image recognition device, classification device, morphological optimization and rating device, and result output device are connected to the processor, and the processor is connected to the memory.

[0015] Compared with the prior art, the beneficial effects of the present invention are: This invention enables automatic segmentation, classification, and rating of non-metallic inclusions in steel, reducing the workload of manual inspection. Through multi-scale recognition, classification probability weighting, and secondary classification of Class B regions, it improves the identification and integration of small and adjacent inclusions. Furthermore, by scanning the entire field of view window, it accurately locates the worst field of view for various inclusions, thereby improving the consistency and reliability of the rating results. Attached Figure Description

[0016] Figure 1 A schematic diagram showing the classification results of non-metallic inclusions in categories A, C, and D; Figure 2 A schematic diagram showing the classification results of non-metallic inclusions in categories B and D; Figure 3 This is a preview diagram of the full field of view rating. Figure 4 This is a schematic diagram of the full-field-of-view jigsaw puzzle. Figure 5 A preview diagram showing the classification of the full-view puzzle; Figure 6 This is a schematic diagram of the worst field of view for a Class A microsystem. Figure 7 This is a schematic diagram of the worst field of view for a Class A coarse series. Figure 8 This is a schematic diagram of the worst field of view for a Class D fine system; Figure 9 This is a schematic diagram of the worst field of view for a Class D coarse-grained system. Figure 10 These are test results for various grades of steel. Detailed Implementation

[0017] The present invention will be further described below with reference to the accompanying drawings. It should be noted that the following embodiments are for illustrative purposes only and are not intended to limit the invention. For those skilled in the art, equivalent substitutions or conventional variations made without departing from the concept of the present invention should be included within the scope of protection of the present invention.

[0018] Non-metallic inclusions in steel are a significant factor affecting steel performance, mainly classified into four categories: A (sulfides), B (alumina), C (silicates), and D (spherical oxides). Accurate classification and rating are crucial for steel quality control. This invention utilizes a two-level architecture based on UNet segmentation and XGBoost classification to achieve automatic classification and rating of non-metallic inclusions in steel. It incorporates deep learning semantic segmentation, machine learning classification, and traditional image post-processing techniques, and can be applied to steel quality inspection and automatic inclusion rating in steel enterprises. This method has two main functions: single-image evaluation and full-field evaluation, which are illustrated below through Examples 1 and 2.

[0019] Example 1: Single Image Evaluation. This example provides a method for automatic classification and rating of non-metallic inclusions in a single metallographic image, specifically including the following steps.

[0020] Step 1) Metallographic Image. Input the metallographic image (pic) to be rated into the system for model input to identify inclusions. The metallographic images are acquired according to GB / T 10561-2023, including four types of non-metallic inclusions: A (sulfides), B (alumina), C (silicates), and D (spherical oxides). The images are acquired using a 100× metallographic microscope with a Zeiss CMOS camera, with a typical size of 1564×1564 pixels to ensure that the segmentation model can adapt to different types of metal materials. The non-metallic inclusion regions in the acquired metallographic images are manually labeled using LabelMe software, marking the rectangular outlines of the four types of inclusions. Then, the labeled masks are extracted and converted into binary images. The training images and the binary labeled images are matched one-to-one to form a dataset for training the deep learning model. A scale bar removal step is added during the dataset preprocessing stage: by combining the red mask of the HSV color space with connected component filtering, and using a two-round morphological dilation and image inpainting algorithm, the red scale bar and scale text in the lower right corner of the metallographic image are automatically removed, eliminating the interference of the scale bar on the subsequent segmentation model.

[0021] Step 2) Pixel Size Detection and Scale Bar Removal. The system supports automatic OCR detection of the scale bar information in the lower right corner of the image to obtain the actual pixel size. The specific method is as follows: First, locate the lower right corner (starting from 55%) of the metallographic image, convert the scale bar area to the HSV color space, create a mask using the red HSV range, and detect the red scale bar; select the longest horizontal red connected component (length greater than 50 pixels, aspect ratio greater than 3) as the effective scale bar, with the largest area used as the fallback mechanism; use horizontal large-scale dilation (25×3 convolution kernel, 2 iterations) to connect the color bar segments obscured by scale characters (such as "100μm"); use the horizontal span of the original red mask in the dilated area as the true bar length, and combine it with the 100μm standard length to calculate the pixel size (μm / px). OCR recognition uses the EasyOCR algorithm to perform text recognition in the scale bar area, extracting numbers and unit information, using a mixed Chinese and English recognition mode, and performing regular expression matching after recognizing numbers to extract length values. If OCR recognition fails, the pixel size or microscope preset value manually set by the user in the UI will be used. After pixel size detection is completed, the system automatically performs red scale bar removal: using HSV color space red mask combined with connected component filtering, and through two rounds of morphological dilation and image inpainting algorithms, the color bars and scale text in the lower right corner of the scale bar are accurately removed to prevent the scale bar area from being misidentified as impurities by subsequent segmentation models.

[0022] Step 3) UNet Semantic Segmentation and Three-Model Fusion. The semantic segmentation network UNet, combined with a three-model fusion strategy, is used to identify the foreground region of non-metallic inclusions in the entire metallographic image. UNet enhances the standard UNet architecture in two ways: an attention gate mechanism is embedded at the encoder-decoder skip connection, enabling the decoder to automatically suppress background responses and enhance the weights of inclusion regions when stitching together shallow features from the encoder; a dilated spatial pyramid pooling (ASPP) bottleneck layer is introduced at the end of the encoder, with dilation rates of 6, 12, and 18, respectively. This captures multi-scale contextual information through parallel dilated convolutions, effectively addressing the segmentation challenges of inclusions with large size ranges. The VGG16 backbone network is selected as the encoder for UNet, loaded with ImageNet pre-trained weights. Its network structure is relatively simple, computationally efficient, and easy to deploy and apply in industrial scenarios.

[0023] The model training employs a two-stage training strategy. The first stage freezes the VGG16 encoder parameters and only trains the decoder parameters for 50 iterations, fully warming up the decoder's decoding capability for inclusion features. The second stage unfreezes all network parameters and performs 200 rounds of global joint fine-tuning, enabling the encoder and decoder to collaboratively adapt to the metallographic image feature distribution. Training uses a joint optimization of cross-entropy loss and Dice loss, with an initial learning rate of 1×10⁻⁶. -4 The cosine decays to 1×10-6 The batch size is 2. Leveraging the general feature extraction capabilities of the pre-trained VGG16, the model convergence speed is accelerated and overfitting is avoided in industrial scenarios with limited labeled samples. Based on the two-stage training, a slice fine-tuning training stage is added: sliding sampling of the training set images is performed with a 1024×1024 resolution window and a 512-pixel stride, executing 100 rounds of supplementary training to specifically improve the model's segmentation recall rate for small inclusions (particle size less than 5 micrometers).

[0024] In the inference stage, a three-model fusion strategy is employed to further improve segmentation accuracy. The first layer: a full-image model (512×512 resolution) infers the entire image, using the NiBlack boundary expansion algorithm to generate a basic mask. The second layer: a slice-fine-tuning model (1024×1024 resolution window, 512-pixel stride) uses Hann window weighted stacking to eliminate stitching artifacts, directly adopting high-confidence regions (support ratio ≥ 0.5) as foreground. The third layer: rule-based filtering is applied to the block inference results, supplementing candidate regions that meet confidence conditions (95th percentile probability ≥ 0.98, number of supported blocks ≥ 1, local contrast ≥ 1.0) as foreground. The union of the three layers yields the final segmentation mask. The metallographic image undergoes four downsampling operations during the encoder process, and multi-scale features are extracted using the VGG16 backbone network. The decoder feature map is upsampled four times to restore the original spatial resolution size of the image, and then fused with the corresponding dimension feature map from the encoder. Finally, a binary segmentation mask is output through a 1×1 convolution.

[0025] Step 4) XGBoost Classification. For the foreground region obtained from the fused segmentation, global contrast pre-filtering is first performed: normalized contrast is calculated for each connected component. When C < 0.05 and the mean grayscale value of the three channels of BGR in the connected component is > 200, the connected component is removed from the mask, filtering out low-contrast bright areas. After filtering, connected components are labeled, and the contour and attributes of each connected component are extracted. A 96-dimensional feature vector covering ten categories is extracted for each connected component, including: geometric features (circumscribed rectangle, area, perimeter, Freette diameter, Hu invariant moments, PCA tip features, etc.), color features (BGR mean / standard deviation / maximum / channel ratio / color richness), grayscale features (mean / variance / Michelson contrast / RMS contrast / grayscale entropy), texture features (LBP histogram, edge density, gradient), and spatial features (particle group distribution, centroid spacing, principal axis direction, collinearity, etc.). The feature vectors are then input into the XGBoost classifier. The classification model training process employs an early stopping strategy. Training is stopped when the accuracy on the validation set does not improve for 10 consecutive rounds, the optimal model weights are saved, and the classification model and normalizer are finally obtained.

[0026] The classification reasoning process employs a reward and penalty mechanism and a two-round progressive classification strategy. The first round of classification involves inputting 96-dimensional features into XGBoost to obtain four class probabilities (PA, PB, PC, PD). Otsu adaptive thresholding is applied to the foreground region corresponding to the connected components in the metallographic image to identify the distribution of independent particles within the region. Regions with ≥3 actual particles are classified as Class B candidate regions, receiving a Class B reward coefficient of PB×2.0; regions without this condition are penalized with a Class B penalty coefficient of PB×0.012. When the number of connected components is 1, the particle area is ≥5 / 9 of the connected component area, and the aspect ratio is <3, a Class D reward of PD×20 is obtained. Class D is forcibly determined: when PA<0.5 and PC<0.5, and a Class D reward is obtained, the region is directly classified as Class D. The final class in the first round is determined by taking the maximum value of (PA, PB, PC, PD). Second round of classification: For connected components classified as class B in the first round, Union-Find merging is performed (nearest point distance ≤ 18μm and orientation difference ≤ 30°). After merging, features are re-extracted, and second round XGBoost classification is performed only on regions B, using differentiated reward / penalty coefficients (class B penalty × 0.012, class B reward × 15, class A reward × 1.2, class D reward × 55). The final class in the second round is determined by taking the maximum value from (PA, PB, PC, PD). After the second round of classification, rule processing is performed: if the maximum Feret diameter of the largest particle in region B is < 2μm, all inclusions in region B are forcibly downgraded to class D; then, DD collinear merging and BD principal axis sampling point merging are performed. The conditions for merging collinear D-groups are: the number of D-groups is ≥3, the distance between any two D-group centers is ≤16μm, and the perpendicular distance from the center of the third D-group to the line connecting the two points is ≤6μm. If these conditions are met, the D-group is merged into the B-group. Pre-filtering is performed before merging; if the C-value of all D-groups is <0.18 and the grayscale value is >175, merging is skipped. The BD-group principal axis sampling points are synthesized by taking points along the B-group principal axis from -L / 2 to +L / 2 in 5μm increments. For each location, points are simultaneously placed on the principal axis, 5μm above the principal axis, and 5μm below the principal axis, forming a 3xN grid of sampling points. D-groups whose center is <16μm from the nearest sampling point are merged into the B-group. D-groups with C-value <0.18 and grayscale value >175 are skipped. Finally, the envelope rule is applied: if the connected components of A / C are enveloped by the B-group rating box at least 80%, they are assigned to B; if the connected components of B are enveloped by the A-group rating box at least 90%, they are assigned to A. This yields the final A / B / C / D regions for the classification stage. Figure 1 and Figure 2 As shown, Figure 1 and Figure 2 In the image, the left side represents the input images, and the right side represents the output images. Figure 1 In the diagram, point a represents class A, point c represents class C, and point d represents class D. Figure 2 In the diagram, point b represents type B and point d represents type D. In the actual implementation of the visualization interface, different colors or different shades of gray can be selected to represent different impurity categories.

[0027] Step 5) Morphological optimization and rating. To ensure the integrity of each discontinuous inclusion, morphological optimization is performed on the classification results, which specifically includes: Step 5-1) Post-processing of Class A (sulfide). Class A sulfide inclusions are in the shape of long strips, and the processing methods are as follows: (a) Calculate the minimum bounding rectangle for each Class A connected domain; (b) Scan the binary mask along the main axis direction of the rectangle, multiply the maximum continuous pixel width by the pixel size to obtain the actual maximum width of the particle along the main axis direction; (c) Set the extension amount according to the pixel size: extend 20μm in the length direction and 7.5μm in the width direction; (d) Perform multiple iterations of merging on the extended rectangles. If two extended rectangles intersect, they are merged into the same rectangle, and the width after merging takes the maximum value of the actual maximum widths of each original connected domain; (e) After merging, the inclusions are divided into fine series (2μm<max_w≤5μm), coarse series (5μm<max_w≤12μm) and oversize (max_w>12μm) according to the maximum width; (f) According to the GB / T 10561 standard, the grade index of Class A inclusions is calculated by formula (1): i=0.5617×log10(L)-1.182, where L is the sum of the lengths of all merged rectangles, and the fine series grade index and coarse series grade index of Class A inclusions are finally obtained.

[0028] Step 5-2) Post-processing of Class B. Class B alumina inclusions are distributed in clusters, which are composed of multiple small particles arranged along a certain direction. The processing methods are as follows: (a) Perform directional dilation on each Class B candidate region; (b) Judge whether adjacent particles belong to the same string according to the angle threshold of 30° and the distance threshold of 18μm; (c) After determining that the particles are in the same group, perform multiple iterations of rectangle merging (extend 20μm in the length direction and 7.5μm in the width direction) to integrate discrete regions; (d) Take the maximum width of the largest inclusion in the connected domain as the actual width; (e) Count the total length L (μm) of all merged Class B inclusions, substitute it into the rating formula (2): i=10^(0.4633×log10(L)-0.873), and output the fine series and coarse series grade indexes.

[0029] Step 5-3) Class C post-processing. The processing method for Class C silicate inclusions is similar to that for Class A, specifically: (a) Extract the minimum bounding rectangle of each connected domain; (b) Scan along the main axis direction of the rectangle to obtain the actual maximum width, with the method identical to that for Class A; (c) Set the extension amount according to the pixel size: extend 20μm in the length direction and 7.5μm in the width direction; (d) Perform multiple iterations of merging on the extended rectangles, and after merging, take the maximum value of the actual maximum width of each original connected domain as the merged width; (e) After merging, classify according to the maximum width: fine series (2μm<max_w≤5μm), coarse series (5μm<max_w≤12μm), and oversize (max_w>12μm); (f) In accordance with the GB / T 10561 standard, the rating index of Class C inclusions is calculated according to formula (3): i=0.4797×log10(L)-0.901, where L is the sum of the lengths of all merged rectangles (μm), and the fine series rating index and coarse series rating index of Class C inclusions are finally obtained.

[0030] Step 5-4) Class D / Ds (spherical oxide) post-processing. Class D spherical oxide inclusions are granular, and are divided into Class D (particle size ≤13μm) and Class Ds (particle size>13μm) according to particle size. Double filtering is performed before rating: (a) Contrast filtering: low-contrast bright domains with C<0.18 and grayscale>175 are removed from the Class D mask; (b) Size filtering: excessively small particles with a Feret diameter <2.0μm are removed from the Class D mask; (c) After filtering, calculate the Feret diameter of each Class D particle; (d) Particles with a particle size ≤13μm are classified as Class D, and particles with a particle size>13μm are classified as Class Ds; (e) Class D is divided into fine series (particle size ≤8μm) and coarse series (8μm<particle size ≤13μm) according to particle size; (f) In accordance with the GB / T 10561 standard, the rating index of Class D inclusions is calculated according to formula (4): i=0.5×log10(N)-0.301, where N is the number of Class D inclusions in each field of view; the rating index of Class Ds inclusions is calculated according to formula (5): i=3.3117×log10(d)-3.221, where d is the diameter of the inclusion (μm).

[0031] Step 6) Classification and Rating Test. To verify the accuracy of the classification and rating of non-metallic inclusions in this embodiment, metallographic images of non-metallic inclusions from various grades of steel were selected as test data. The test images included four categories: Class A sulfide inclusions, Class B alumina inclusions, Class C silicate inclusions, and Class D spherical oxide inclusions, totaling 147 images. Of these, 134 were correct, resulting in an accuracy rate of 91.16%. The test results for each category are as follows: Category A: Fine sulfide series (AT) samples: 25, 25 correct, accuracy rate 100%; Category A: Coarse sulfide series (AH) samples: 20, 17 correct, accuracy rate 85%; Category B: Fine alumina series (BT) samples: 24, 19 correct, accuracy rate 79.17%; Category B: Coarse alumina series (BH) samples: 3, 2 correct, accuracy rate 66.67%; Category C: Fine silicate series (CT) samples: 1, 1 correct, accuracy rate 100%; Category C: Coarse silicate series (CH) samples: 6, 5 correct, accuracy rate 83.33%; Category D: Fine spherical oxide series (DT) samples: 117, 117 correct, accuracy rate 100%; Category D: Coarse spherical oxide series (DH) samples: 22, 22 correct, accuracy rate 100%; Category Ds (DS) samples: 5, 5 correct, accuracy rate 100%. See details below. Figure 10 As shown in the test results above, this embodiment can effectively identify non-metallic inclusion regions in metallographic images and accurately classify them into four categories: A, B, C, and D. After post-processing optimization, the interference of fracture boundaries is removed, resulting in a clear inclusion classification image. After segmentation, classification, and optimization, various inclusions can be rated.

[0032] Example 2: Full Field of View Rating. Full field of view rating is an extension of single image rating. This example uses a 5×5 full field of view as an example.

[0033] Step 1) Import multiple metallographic images of adjacent fields of view. The system stitches the images according to a preset grid layout (rows × columns) to form a complete field of view for analysis. Batch stitching supports both Z-shaped and straight-line stitching methods, and the sliding window step size can be set (10%, 20%, 25%, 50%, which are proportions relative to the size of a single field of view) to accommodate the analysis needs of non-metallic inclusions of different sizes. Figure 4 The image shown is the full-field mosaic result of this embodiment. Figure 5 This is a preview diagram of the full-view puzzle categories.

[0034] Step 2) Sliding window scanning and worst-case field-of-view localization are the core steps of the full-field-of-view rating, used to identify the local area with the most severe inclusions in the entire field, conforming to the requirements of GB / T 10561 standard regarding "worst-case field of view". Specifically: For each type of inclusion and each category (fine / coarse / oversized), calculate the score for each original field of view, and select the top two fields of view with the highest scores as candidates; with each top two candidate field of view as the center, perform a sliding window scan within one field of view above, below, left, and right (i.e., a 3×3 neighborhood), with the sliding step size = single field of view size × step_ratio (default 20%, adjustable), and the scanning range is limited to the neighborhood of one field of view surrounding the candidate field of view; for the four types of masks within each sliding window, use the same post-processing algorithms as the single-image rating process to calculate the rating score, record the position of the window with the highest score and detailed rating data; from the three categories of fine / coarse / oversized inclusions, select the one with the highest score as the final "worst-case field of view" representative for that type of inclusion. Figures 6 to 9 As shown, this embodiment presents the worst field-of-view results for Class A fine series, Class A coarse series, Class D fine series, and Class D coarse series, respectively.

[0035] Step 3) Worst field of view labeling and result output. To achieve visualization, the worst field of view location for each type of inclusion is labeled on the stitched large image using different colored / grayscale boxes, with the category and score labeled above the box. Figure 3 In the diagram, point I represents the worst field of view for Class A inclusions, point II for Class C inclusions, and point III for Class D inclusions. The system visualizes the classification and rating results for each type of inclusion in image format, and simultaneously outputs a text file containing detailed rating parameters (including the quantity, size, and level of each type of inclusion), which is saved to a specified local directory.

[0036] In the full-field rating test, this embodiment can effectively identify non-metallic inclusion regions in metallographic images and accurately classify them into four categories: A, B, C, and D. After post-processing optimization, the interference of fracture boundaries is removed, resulting in a clear inclusion classification image. After image segmentation, classification, and optimization, various inclusions can be rated.

Claims

1. A method for automatic classification and rating of non-metallic inclusions in steel from a single image, characterized in that, include: Obtain the metallographic images to be graded; Multi-scale recognition of metallographic images is performed using whole-image recognition and local region recognition. The whole-image recognition results are filtered based on the local region recognition results, and the filtered results are fused with the local region recognition results to obtain candidate regions for non-metallic inclusions. Connected component labeling is performed on candidate regions of non-metallic inclusions, image features of each connected component are extracted, and the image features are input into a classifier to obtain the classification probabilities of type A sulfides, type B alumina, type C silicates and type D spherical oxides. The category weight coefficients are determined according to the particle distribution characteristics of the connected components, and the classification probabilities are weighted and adjusted. The first round of classification results are determined according to the weighted and adjusted classification probabilities. For candidate regions classified as B in the first round, they are merged according to the spatial relationship between adjacent particles. Image features are then re-extracted from the merged B-class candidate regions and a second round of classification is performed. Morphological optimization was performed on inclusions of types A, B, C, and D, and a grade index was calculated based on the length, width, grain size, or number of the optimized inclusions to obtain the rating result of the metallographic image.

2. The method according to claim 1, characterized in that, It also includes image preprocessing steps: The scale region in the metallographic image is located, the scale region is converted to the HSV color space, and the scale detection region is generated based on the red color range. The effective scale bar is determined based on the length, shape, and area of ​​the red connected region, and the pixel size is determined based on the actual length of the scale bar. Morphological connections are made to the scale bars that are obscured by the scale characters, and the actual span of the scale bars is determined based on the correspondence between the connected areas and the original red areas. The length value and unit information are extracted from the scale area using optical character recognition, and the scale bars and scale text in the metallographic image are removed based on the scale information. When the scale information is not recognized, the preset pixel size or the manually set pixel size is used.

3. The method according to claim 1, characterized in that, The whole image recognition uses the semantic segmentation network UNet to infer the whole metallographic image, and the local region recognition uses the slice fine-tuning model of UNet to infer the image. UNet sets attention gating at the jump connection between the encoder and the decoder, and sets a void space pyramid pooling bottleneck layer at the end of the encoder.

4. The method according to claim 3, characterized in that, The whole image recognition uses UNet as the whole image model, and combines it with the boundary dilation algorithm to generate the basic recognition results of the whole image; The local region recognition adopts a slice fine-tuning model, and the recognition results of multiple local regions are superimposed by window weighting to reduce splicing artifacts between local regions; Based on the support level and confidence level of the local identification results, the local candidate regions are filtered according to rules, and the filtered local candidate regions are fused with the basic identification results to obtain the non-metallic inclusion candidate regions.

5. The method according to claim 1, characterized in that, Calculate the normalized contrast for each connected component. When a connected component belongs to a low-contrast bright area, remove the connected component from the candidate region of non-metallic inclusions. Image features of connected components include geometric features, color features, grayscale features, texture features, and spatial features. Among them, geometric features include the bounding rectangle, area, perimeter, Freret diameter, Hu invariant moments, and PCA tip features; color features include BGR mean, standard deviation, extrema, channel ratio, and color richness; grayscale features include mean, variance, Michelson contrast, RMS contrast, and grayscale entropy; texture features include LBP histogram, edge density, and gradient; and spatial features include particle group distribution, centroid spacing, principal axis orientation, and collinearity.

6. The method according to claim 1, characterized in that, Determining the category weight coefficients and performing the second round of classification includes: The foreground region corresponding to the connected component in the metallographic image is segmented twice to obtain the distribution of independent particles within the connected component. Based on the number of independent particles, the ratio of particle area to connected component area, the aspect ratio of the connected component, and the arrangement relationship of particles, the category weight coefficient corresponding to each category is determined, and the classification probability of the corresponding category is multiplied by the category weight coefficient. The first-round classification results are determined based on the weighted adjusted classification probabilities, and connected regions that satisfy the distribution characteristics of Class B particle groups are identified as Class B candidate regions. Since the characteristics of Class B inclusions are particles that are discontinuously distributed in a row, adjacent particles with spatial proximity and directional consistency in the Class B candidate region are merged. After merging, image features are re-extracted, and a second round of classification is performed on the merged Class B candidate region. Based on the particle size characteristics of region B and the spatial arrangement relationship between region D and region B in the second round of classification results, the classification results are corrected.

7. The method according to claim 1, characterized in that, Morphological optimization and rating include: Calculate the minimum bounding rectangle for the connected regions of type A sulfides and type C silicates, determine the actual maximum width along the main axis of the minimum bounding rectangle, and extend and merge adjacent or intersecting bounding rectangles. Directional expansion is performed on Class B alumina inclusions. Particles in the same group are determined based on the directional and distance relationships between adjacent particles, and particles in the same group are merged. Contrast filtering and size filtering were performed on D-type spherical oxide inclusions, and they were classified into D-type or Ds-type according to the diameter of the filtered particles. The grade index is calculated based on the category, length, width, particle size, or quantity of various non-metallic inclusions, using the rating rules corresponding to the category.

8. A full-field automatic rating method for non-metallic inclusions in steel, characterized in that, include: Acquire multiple metallographic images of adjacent fields of view, and stitch them together according to a preset grid layout to form a complete field of view; From the full field of view, candidate local regions are identified for worst-case field of view search, and window scanning is performed on the candidate local regions; Multi-scale identification of non-metallic inclusions within each window is performed, and the identification results of the entire window are fused with the identification results of local areas to obtain candidate regions of non-metallic inclusions within the window. Connectivity features are extracted and classified for candidate regions of non-metallic inclusions within the window. The classification probability is adjusted according to the particle distribution characteristics of the connected regions. Spatial relationship merging and second-round classification are performed on candidate regions classified as class B in the first round. Morphological optimization was performed based on the morphological characteristics of inclusions of types A, B, C, and D, and the category rating results of the window were calculated based on the length, width, particle size, or number of the optimized inclusions. The rating results for each window are compared according to the category and series of non-metallic inclusions to determine the worst field of view representative for each type of non-metallic inclusion; It also outputs the location, category, and rating results of the worst field of view.

9. An automatic rating system for non-metallic inclusions in steel, characterized in that, include: An image input device for inputting a metallographic image to be graded or multiple metallographic images of adjacent fields of view; An image recognition device is used to perform whole-image recognition and local region recognition on metallographic images, and to fuse the whole-image recognition results and local region recognition results to obtain candidate regions for non-metallic inclusions. A classification device is used to classify connected regions into Class A sulfides, Class B alumina, Class C silicates, and Class D spherical oxides based on image features of connected regions in candidate regions of non-metallic inclusions, and to adjust the classification probability by weighting according to the particle distribution characteristics of the connected regions. A morphological optimization and rating device is used to perform morphological optimization on connected components according to classification results, and to calculate a level index based on the length, width, particle size or number of the optimized connected components. The result output device is used to output the classification and rating results of non-metallic inclusions. The image input device, image recognition device, classification device, morphological optimization and rating device, and result output device are connected to the processor, and the processor is connected to the memory.

Images