An Automatic Analysis Method and System for Non-metallic Inclusions Based on Computer Vision

Abstract

This invention discloses an automatic analysis method and system for non-metallic inclusions based on computer vision. Belonging to the interdisciplinary field of image processing, machine vision, and metallic material inspection, the method includes: sample preparation and image acquisition; image preprocessing; contour retrieval and feature extraction; rolling direction calculation; inclusion classification and contour merging; quantitative statistics and national standard grading; and result output. This invention achieves integrated execution from metallographic image input to standardized inspection report output through fully automated processing, eliminating the need for manual visual observation and measurement. It supports continuous analysis of batch samples, significantly shortening the detection time for single images and batches, and substantially improving detection efficiency. It fully meets the application needs of large-scale inspection and online real-time quality inspection on production lines in the metallurgical industry.

Description

Technical Field

[0001] This invention relates to the interdisciplinary fields of image processing, machine vision, and metal material detection, specifically to intelligent identification, feature extraction, classification, grading, and quantitative analysis techniques for non-metallic inclusions in metallographic images of metallic materials, and particularly to an automatic analysis method and system for non-metallic inclusions based on computer vision. Background Technology

[0002] In the smelting, casting, and forming processes of steel and non-ferrous metals, non-metallic inclusions such as oxides, sulfides, silicates, and nitrides are inevitably introduced. These inclusions exist within the metal matrix as hard and brittle phases, granular, spherical, or aggregated states, disrupting the continuity and integrity of the metal matrix and directly deteriorating the mechanical properties, fatigue performance, processing performance, and service life of the metal material. They are key indicators determining the grade of steel and product quality. According to the national standard GB / T10561-2023 "Microscopic Evaluation Method for Non-metallic Inclusions in Steel," non-metallic inclusions in metallographic images need to be classified into five categories based on morphology and distribution: Category A sulfides, Category B alumina, Category C silicates, Category D spherical oxides, and Category DS large-particle spherical oxides. Identification, classification, quantitative statistics, and grading are required. Accurate detection and standardized grading of non-metallic inclusions have become essential steps in the production, factory inspection, and quality control of high-quality steel and high-grade high-quality steel.

[0003] Currently, non-metallic inclusion detection and analysis methods in the industry are mainly divided into two categories: traditional manual detection methods and computer vision-based automatic / semi-automatic detection methods. Traditional manual detection methods rely on visual observation under a metallographic microscope. Professional inspectors identify inclusions in metallographic images with the naked eye, determine their type based on experience, and perform quantitative analysis by manual measurement and counting. The results are then graded according to the GB / T 10561 standard. This is a long-standing and conventional method in the industry and the existing basic technology closest to this invention. Computer vision-based automatic / semi-automatic detection methods typically use metallographic images as the main data source, combining target detection, image segmentation, traditional image processing algorithms, or deep learning models for inclusion identification. Some techniques can remove artifacts, scratches, and other interference, representing the mainstream research direction in the field of automated non-metallic inclusion detection.

[0004] Traditional manual inspection methods have significant shortcomings in practical applications. They are inefficient, cumbersome, and cannot meet the needs of large-scale sample testing and real-time online quality inspection on production lines. The inspection process is highly subjective, with inconsistent judgment standards among different inspectors, leading to poor repeatability and consistency, and a high risk of misjudgment and missed detection. Furthermore, prolonged microscopic observation can cause visual fatigue for inspectors, further reducing the accuracy of results. In addition, non-standardized data recording methods make it difficult to generate standardized inspection reports, hindering product quality traceability and data management. Existing computer vision-based automatic inspection technologies also have many drawbacks. Most of these technologies lack unified inspection and evaluation indicators aligned with national standards. Deep learning model outputs exhibit high uncertainty and low recall, failing to meet the high-precision inspection requirements of industrial scenarios. Deep learning applications rely on a large number of professionally annotated metallographic images, which are difficult to obtain, costly to annotate, and have long model training cycles, hindering large-scale application across industries. Moreover, deep learning models have typical "black box" characteristics; their internal decision-making processes cannot be intuitively explained, making on-site deployment and parameter optimization difficult, reducing their credibility in industrial applications. General image processing algorithms are not specifically optimized for the characteristics of metallographic images. They are weak in resisting interference from factors such as noise, shadows, scratches, uneven lighting, and complex and diverse inclusion morphologies. During processing, problems such as missegmentation of inclusions and extraction of false contours are prone to occur, affecting recognition accuracy. More importantly, existing automatic detection technologies generally lack classification and judgment rules that comply with the GB / T 10561-2023 standard and multi-contour merging mechanisms based on the rolling direction. They cannot accurately distinguish the five types of inclusions, especially the aggregated B-type inclusions, resulting in the final grading results not meeting national standards. In addition, related detection systems also suffer from poor stability, insufficient compatibility, complex operation, and detection speed that cannot meet the needs of industrial online detection, which limits the widespread application of the technology.

[0005] The main reason for the aforementioned defects is that traditional detection methods rely entirely on human experience for judgment and measurement, without incorporating intelligent auxiliary means. Existing automated detection technologies mostly employ general image processing frameworks or deep learning models, failing to develop targeted algorithms based on the grayscale, morphology, and distribution characteristics of metallographic inclusions, and also lacking a complete classification, merging, and grading logic strictly based on national standards. The main difficulties in solving these problems are as follows: metallographic images contain numerous interfering factors such as noise and sample preparation defects, making it difficult to completely preserve the true edge information of inclusions while effectively removing interference; national standards require merging discrete particles into a single inclusion line according to rolling direction, spacing, and distribution rules, which conventional image processing algorithms struggle to achieve regular contour merging; complex inclusions such as clustered type B inclusions exhibit significant morphological differences, making accurate differentiation impossible using general features; and practical industrial applications require simultaneous high precision, high stability, interpretability, low cost, and fast detection speed, all of which current technical solutions cannot simultaneously meet. Summary of the Invention

[0006] This invention addresses the numerous shortcomings of existing non-metallic inclusion detection technologies in steel by proposing an automated analysis method and system for non-metallic inclusions based on computer vision. It aims to solve the problems of low efficiency, high subjectivity, poor result consistency, and inability to meet the needs of batch and online detection in traditional manual inspection. Simultaneously, it overcomes the deficiencies of existing computer vision-based automated detection methods, such as weak anti-interference capabilities, insufficient inclusion segmentation and recognition accuracy, and difficulty in achieving accurate classification and contour merging determination according to the GB / T 10561-2023 national standard. It also solves the technical problems of deep learning models relying on large amounts of labeled data, high training costs, unexplainable decision-making mechanisms, and difficulties in field deployment, as well as the poor stability, insufficient adaptability, and inability of existing systems to meet industrial application requirements. Ultimately, it achieves fully automated, high-precision, high-stability identification, classification, quantitative analysis, and grading of non-metallic inclusions that conforms to national standards.

[0007] On the one hand, this invention provides an automatic analysis method for non-metallic inclusions based on computer vision, the scheme of which is as follows:

[0008] Step 1: Sample preparation and image acquisition: Complete steel sampling, sample preparation and metallographic image acquisition according to national standards to obtain raw digital images containing non-metallic inclusions;

[0009] Step 2: Image preprocessing: The original digital image is sequentially subjected to black border removal, shadow removal, bilateral filtering for noise reduction, scratch removal, and adaptive thresholding or OTSU thresholding binarization to obtain a binary image that retains only the intrusive target.

[0010] Step 3: Contour retrieval and feature extraction: Perform contour retrieval on the binary image, extract all independent connected component contours, and calculate geometric features and grayscale features for each contour;

[0011] Step 4: Rolling direction calculation: Select the elongated inclusion profile, and determine the overall rolling direction based on the direction angle of the elongated inclusion profile. Then, recalculate the projected length of each profile based on the rolling direction.

[0012] Step 5: Inclusion Classification and Contour Merging: Using a contour classification decision algorithm, a multi-level decision logic is employed based on the morphological definitions of the five types of inclusions according to the national standard. First, DS-type inclusions are identified based on their equivalent diameter. Second, elongated A / C / B-type inclusions, granular B / D-type inclusions, and other aggregated B-type inclusions are distinguished based on their aspect ratio. Then, multiple discrete contours satisfying the national standard rules are merged into a single inclusion based on the projected length, spacing, and distribution direction, and D-type inclusions are distinguished based on the number of contours and their aspect ratio. Finally, for the merged contours, the number and confidence level of elongated A / C / B-type inclusions, granular B / D-type inclusions, and aggregated B-type inclusions within each contour are determined. First, class B inclusions are identified; finally, class A and class C inclusions are distinguished by relative color to complete the final classification.

[0013] Step 6: Quantitative statistics and national standard grading: Statistically analyze the total length, quantity, maximum width, and equivalent diameter parameters of the inclusions, and complete the grading according to the national standards to generate grading results and test data;

[0014] Step 7: Output Results: Output inclusion classification images, quantitative data, rating results, and standardized test report.

[0015] Optionally, step 2: image preprocessing: the original digital image is sequentially subjected to black border removal, shadow removal, bilateral filtering for noise reduction, scratch removal, and adaptive thresholding or OTSU thresholding binarization to obtain a binary image that retains only the intrusive target, specifically including:

[0016] Step 2.1: Remove black borders

[0017] The original digital image is thresholded to extract the white background, the outermost contour of the image is obtained and the minimum bounding rectangle of the contour is calculated. The bounding rectangle area is used as the effective image area, and excess black edges of the image are removed.

[0018] Step 2.2: Shadow Removal

[0019] The obtained digital image is converted into a grayscale image, a morphological dilation operation is performed on the grayscale image, and then an approximate background distribution map is generated by median filtering. The approximate background distribution map and the grayscale image are then compared to eliminate the effects of uneven lighting, shadows and background gradations, resulting in a target image with uniform brightness.

[0020] The digital image includes the original digital image and the image after black border removal.

[0021] Step 2.3: Bilateral filtering for noise reduction

[0022] The target image after removing shadows is smoothed by bilateral filtering, which removes image noise and texture interference while preserving the edge information of the inclusions.

[0023] Step 2.4: Scratch Removal Treatment

[0024] Canny edge detection is used to extract image edges, and Hough line detection is used to identify the scratch lines generated during sample preparation. The identified scratch areas are then repaired using the Telea04 image restoration algorithm to blend the scratch areas with the background.

[0025] Step 2.5: Binarization

[0026] The preprocessed image is binarized using an adaptive thresholding method or the Otsu method, converting the preprocessed image into a binary image containing only black areas of inclusions and a white background.

[0027] Optionally, the geometric features include: area, aspect ratio, roundness, equivalent diameter, maximum projected length, width sequence, and profile orientation angle;

[0028] The grayscale features include: average grayscale inside the contour, average grayscale of the neighboring background, and relative grayscale.

[0029] Optionally, the relative grayscale is the ratio of the average pixel value inside the inclusion contour to the average pixel value of the neighboring background, used to distinguish inclusions from matrix tissue;

[0030] The formula for calculating relative gray level is:

[0031]

[0032] In the formula: These are the pixel values ​​inside the outline. The value represents the background pixel count, and N and M represent the number of internal and background pixels, respectively.

[0033] Optionally, in step 4, the rolling direction calculation involves selecting a long strip-shaped inclusion profile, statistically determining the overall rolling direction based on the direction angle of the long strip-shaped inclusion profile, and recalculating the projected length of each profile based on the rolling direction. Specifically, this includes:

[0034] Select the profile of the elongated inclusion, statistically analyze its angular radian value, and calculate the overall rolling direction average angle avgD using the following formula:

[0035]

[0036] In the formula: ND represents the direction angle of a single profile, and ND represents the number of profiles involved in the calculation.

[0037] Using the average angle avgD of the overall rolling direction as the material rolling direction, the projected length of each profile in the rolling direction is recalculated as the basis for profile merging.

[0038] Optionally, step 5: Inclusion classification and contour merging: Using a contour classification decision algorithm, a multi-level decision logic is employed based on the morphological definitions of the five types of inclusions according to the national standard. First, DS-type inclusions are identified based on their equivalent diameter. Second, elongated A / C / B-type inclusions, granular B / D-type inclusions, and other aggregated B-type inclusions are distinguished based on their aspect ratio. Then, multiple discrete contours satisfying the national standard rules are merged into a single inclusion based on the projected length, spacing, and distribution direction, and D-type inclusions are distinguished based on the number of contours and their aspect ratio. Next, for the merged contour, the number and confidence level of elongated A / C / B-type inclusions, granular B / D-type inclusions, and aggregated B-type inclusions contained in each contour are analyzed. First, class B inclusions are identified. Finally, class A and class C inclusions are distinguished by relative color to complete the final classification. This includes:

[0039] First, DS-type large-particle spherical oxides are identified according to their equivalent diameter;

[0040] Secondly, preliminary classification and initial contour screening are performed. Based on the aspect ratio, elongated A / C / B type inclusions, granular B / D type inclusions, and other aggregated B type inclusions are distinguished. The contours are specifically categorized as follows:

[0041] ①. Those with a larger length-to-width ratio are elongated A / C / B type inclusions;

[0042] ②. Inclusions with a smaller aspect ratio are granular B / D type;

[0043] ③. Other aggregated Class B inclusions;

[0044] Then, based on the rolling direction angle, it is further calculated whether they can be merged into one. If they can be merged into one, then multiple discrete contours that meet the national standard rules are merged into the same inclusion according to the projected length, spacing, and distribution direction; specifically:

[0045] After being merged into one,

[0046] A contour with only one outline and a small aspect ratio is classified as type D.

[0047] The remaining outlines are categories A, B, and C, which have already been merged into one.

[0048] Next, for the contours that have been merged into one, the category of the contour is calculated by comprehensively considering the relative color, the number of contours belonging to the initial screening (①②③) contained in the contour, and whether it contains clustered B-type contours. Specifically:

[0049] Based on the number of ①②③ contained in each item and the confidence level This allows for the accurate identification of aggregated B-type alumina inclusions, distinguishing between them.

[0050] If there are many outlines with small aspect ratios, or the confidence level is high If the value is larger, then this item is considered to be of category B;

[0051] Otherwise, it is classified as either Category A or Category C;

[0052] Finally, categories A and C are distinguished by their relative colors.

[0053] Optionally, in step 5, calculating the confidence score Bc for aggregated type B alumina inclusions and identifying aggregated type B alumina inclusions specifically includes:

[0054] The identification of aggregated Class B alumina inclusions is achieved by gradient evaluation of the width sequence of the contour in the direction of maximum projected length.

[0055] The calculation method for the clustered B-type confidence score Bc is as follows:

[0056]

[0057] In the formula: NT represents the width sequence of the contour along the maximum projection length, where NT is the total number of points.

[0058] On the other hand, the present invention also provides an automatic analysis system for non-metallic inclusions based on computer vision, the system comprising:

[0059] Image acquisition module: used to acquire metallographic images, obtain raw digital images containing non-metallic inclusions, and transmit them to the system;

[0060] Preprocessing module: Performs black border removal, shadow removal, filtering, scratch removal, and binarization on the acquired raw digital image to obtain a binary image that retains only the intrusive target;

[0061] Feature extraction module: performs contour retrieval on the obtained binary image and calculates geometric and grayscale features;

[0062] Rolling direction calculation module: Based on the selected elongated inclusion profile direction angle, the overall rolling direction is determined statistically, and the projection length of each profile is recalculated based on the rolling direction;

[0063] Classification and merging module: Performs classification and determination of inclusions and merging of discrete contours;

[0064] Grading module: Completes grading assessment and generates test reports according to national standards;

[0065] Result output module: Outputs the detection results.

[0066] Compared with the prior art, the beneficial technical effects of the present invention are as follows:

[0067] Compared with existing traditional manual inspection and computer vision automatic inspection technologies, this invention can bring significant benefits in many aspects, such as inspection efficiency, recognition accuracy, standard compliance, system stability, and industrial applicability.

[0068] This invention achieves integrated execution from metallographic image input to standardized test report output through fully automated processing. The entire process requires no manual visual observation or measurement, supports continuous analysis of batch samples, significantly shortens the detection time for a single image and a single batch of samples, and significantly improves detection efficiency. It can fully meet the application needs of large-scale testing and online real-time quality inspection of production lines in the metallurgical industry.

[0069] This invention employs a preprocessing workflow specifically optimized for metallographic images, which can effectively remove interference factors such as black edges, shadows, scratches, and uneven lighting. Combined with a self-developed contour classification decision algorithm and multi-feature fusion judgment logic, it can achieve high accuracy and high recall in the identification of five types of inclusions: A, B, C, D, and DS. The edge positioning is accurate, effectively avoiding false positives and false negatives. The detection results have good consistency and repeatability and are not affected by subjective factors such as human experience and visual fatigue.

[0070] The inclusion classification rules, multi-profile merging mechanism based on rolling direction, quantitative calculation method and grade evaluation logic of this invention strictly comply with the requirements of the national standard GB / T 10561-2023. It can accurately identify complex inclusions such as aggregated Class B inclusions. The evaluation results are standardized and universal, and can be directly used for steel quality evaluation and factory inspection, and have authoritative compliance.

[0071] This invention employs explicit decision-making logic, eliminating the need for large-scale labeled samples for model training and complex parameter tuning. It features low data acquisition costs, simple deployment and maintenance, strong algorithm interpretability, stable and reliable operation, and adaptability to metallographic images with different sample quality and shooting conditions, making it more suitable for industrial sites.

[0072] Meanwhile, this invention integrates complete functions such as image management, segmentation, classification, grading, and report export. It supports multi-magnification image input, and the operation process is simple and intuitive, which can significantly reduce the professional ability requirements of operators. It facilitates enterprises to carry out quality control, data storage and quality traceability. The overall technical solution has achieved significant improvements in accuracy, efficiency, stability, standard compliance and practicality, and has outstanding technological progress and wide promotion value.

[0073] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0074] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0075] Figure 1 A flowchart of an automatic analysis method for non-metallic inclusions based on computer vision provided in an embodiment of the present invention;

[0076] Figure 2 This is a flowchart of the image processing method for inclusions provided in an embodiment of the present invention;

[0077] Figure 3 A feature decision algorithm diagram provided for an embodiment of the present invention. Detailed Implementation

[0078] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0079] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0080] like Figures 1 to 3As shown, this embodiment provides an automatic analysis method for non-metallic inclusions based on computer vision. Specifically, it is an automatic analysis method for non-metallic inclusions based on computer vision and a self-developed contour classification decision algorithm. This method is applicable to the fully automatic identification, classification, quantitative calculation, and national standard grading of non-metallic inclusions in steel under the GB / T 10561-2023 standard. The implementation method is described in detail below with specific steps:

[0081] Step 1: Sample preparation and image acquisition: Complete steel sampling, sample preparation and metallographic image acquisition according to national standards to obtain raw digital images containing non-metallic inclusions.

[0082] Specifically, steel samples were taken, inlaid, polished, and etched in accordance with the national standard GB / T 10561-2023. Metallographic images were acquired using a metallographic microscope at magnifications of 100x, 200x, 400x, or 500x to obtain original digital images containing non-metallic inclusions.

[0083] Step 2: Image Preprocessing: The original digital image is sequentially subjected to black border removal, shadow removal, bilateral filtering for noise reduction, scratch removal, and adaptive thresholding or OTSU thresholding binarization to obtain a binary image that retains only the intrusive target. Specifically, this includes:

[0084] 2.1 Removing Black Borders

[0085] Specifically, the original digital image is thresholded to extract the white background, the outermost contour of the image is obtained, and the minimum bounding rectangle of the contour is calculated. The bounding rectangle area is used as the effective image area, and the excess black edges of the image are removed to avoid interfering with subsequent contour extraction and feature calculation.

[0086] 2.2 Shadow Removal Processing

[0087] Specifically, the obtained digital image is converted into a grayscale image, a morphological dilation operation is performed on the grayscale image, and then an approximate background distribution map is generated by median filtering; the approximate background distribution map and the original grayscale image are compared by a difference operation to eliminate the effects of uneven lighting, shadows and background gradations, and obtain a target image with uniform brightness.

[0088] The digital image includes the original digital image and the image after black border removal.

[0089] 2.3 Bilateral Filtering for Noise Reduction

[0090] Specifically, bilateral filtering is used to smooth the image after removing shadows, which retains the edge information of the inclusions while filtering out image noise and texture interference, thus improving the subsequent binarization effect.

[0091] 2.4 Scratch Removal Treatment

[0092] Specifically, Canny edge detection is used to extract image edges, and Hough line detection is used to identify the straight lines of scratches generated during sample preparation. The identified scratch areas are then repaired using the Telea04 image restoration algorithm to blend the scratch areas with the background and prevent scratches from being misdetected as inclusion contours.

[0093] 2.5 Binarization Processing

[0094] Specifically, the preprocessed image is binarized using an adaptive thresholding method or the Otsu method, converting the image into a binary image containing only the black area of ​​the inclusions and a white background, thus highlighting the target area of ​​the inclusions.

[0095] Step 3: Contour retrieval and feature extraction: Perform contour retrieval on the binary image, extract all independent connected component contours, and calculate geometric and grayscale features for each contour.

[0096] Specifically, contour retrieval is performed on the binary image to extract the contours of all independent connected components, and the following feature parameters are calculated for each contour:

[0097] Geometric features include: area, aspect ratio, roundness, equivalent diameter, maximum projected length, width sequence, and profile orientation angle;

[0098] Gray-scale features include: average gray-scale inside the contour, average gray-scale of the surrounding background, and relative gray-scale (rc).

[0099] It should be noted that the relative grayscale is the ratio of the average pixel value inside the inclusion contour to the average pixel value of the neighboring background, which is used to distinguish inclusions from the matrix and improve classification accuracy.

[0100] The formula for calculating relative gray level is as follows:

[0101]

[0102] In the formula: These are the pixel values ​​inside the outline. The value represents the background pixel count, and N and M represent the number of internal and background pixels, respectively.

[0103] Step 4: Rolling direction calculation: Select the elongated inclusion profile, and determine the overall rolling direction based on the direction angle of the elongated inclusion profile. Then, recalculate the projected length of each profile based on the rolling direction.

[0104] Specifically, select the profile of the elongated inclusion, statistically analyze its angular radius values, and calculate the overall rolling direction average angle avgD using the following formula:

[0105]

[0106] In the formula: ND represents the direction angle of a single profile, and ND represents the number of profiles involved in the calculation.

[0107] Using avgD as the rolling direction of the material, the projected length of each profile in the rolling direction is recalculated and used as the basis for profile merging.

[0108] Step 5: Inclusion Classification and Contour Merging: Using a self-developed contour classification decision algorithm, a multi-level decision logic is adopted based on the morphological definitions of the five types of inclusions according to the national standard. First, DS-type inclusions are identified based on equivalent diameter. Second, elongated A / C / B-type inclusions, granular B / D-type inclusions, and other aggregated B-type inclusions are distinguished based on aspect ratio. Then, multiple discrete contours that meet the national standard rules are merged into a single inclusion based on the projected length, spacing, and distribution direction, and D-type inclusions are distinguished based on the number of contours and aspect ratio. Next, for the merged contours, the number and confidence level of elongated A / C / B-type inclusions, granular B / D-type inclusions, and aggregated B-type inclusions in each contour are determined. First, class B inclusions are identified; finally, class A and class C inclusions are distinguished by relative color to complete the final classification.

[0109] Specifically, a self-developed multi-level contour classification decision algorithm is used to perform the following judgments based on the national standard morphology definition:

[0110] First, DS-type large-particle spherical oxides are identified according to their equivalent diameter;

[0111] Secondly, preliminary classification and initial contour screening are performed. Based on the aspect ratio, elongated A / C / B type inclusions, granular B / D type inclusions, and other aggregated B type inclusions are distinguished. The contours are specifically categorized as follows:

[0112] ①. Those with a larger length-to-width ratio are likely to be elongated inclusions (Class A, Class C, Class B);

[0113] ②. Inclusions with a smaller length-to-width ratio are granular (Class B and Class D);

[0114] ③. Other inclusions that may be aggregated type B;

[0115] Then, based on the rolling direction angle, it is further calculated whether they can be merged into one. If they can be merged into one, then multiple discrete contours that meet the national standard rules are merged into the same inclusion according to the projected length, spacing, and distribution direction; specifically:

[0116] After being merged into one,

[0117] A contour with only one outline and a small aspect ratio is classified as type D.

[0118] The remaining outlines may be of type A, type B, or type C that have already been merged into one;

[0119] Next, for the contours that have been merged into one, the category of the contour is calculated by comprehensively considering the relative color, the number of contours belonging to the initial screening (①②③) contained in the contour, and whether it contains clustered B-type contours. Specifically:

[0120] Based on the number of ①②③ contained in each item and the confidence level This allows for the accurate identification of aggregated B-type alumina inclusions, distinguishing between them.

[0121] The formula for calculating the clustered B-type confidence score Bc is as follows:

[0122]

[0123] In the formula: The width sequence of the contour along the maximum projection length, where NT is the total number of points;

[0124] Specifically:

[0125] If there are many outlines with small aspect ratios, or the confidence level is high If the value is larger, then this item is considered to be of category B;

[0126] Otherwise, it is classified as either Category A or Category C;

[0127] Finally, categories A and C are distinguished by their relative colors.

[0128] Step 6: Quantitative Statistics and National Standard Grading

[0129] Specifically, the total length, quantity, maximum width, and maximum diameter of various inclusions are statistically analyzed, and the level is assessed according to the GB / T10561-2023 standard to generate rating results and test data.

[0130] Step 7: Output Results

[0131] Specifically, it outputs images of inclusion classification markers, classification statistics tables, quantitative analysis results, and rating reports, and supports export in PDF / Excel format.

[0132] This invention also provides an automatic analysis system for non-metallic inclusions based on computer vision, comprising:

[0133] Image acquisition module: used to acquire metallographic images, obtain raw digital images containing non-metallic inclusions, and transmit them to the system;

[0134] Preprocessing module: Performs black border removal, shadow removal, filtering, scratch removal, and binarization on the acquired raw digital image to obtain a binary image that retains only the intrusive target;

[0135] Feature extraction module: performs contour retrieval on the obtained binary image and calculates geometric and grayscale features;

[0136] Rolling direction calculation module: Based on the selected elongated inclusion profile direction angle, the overall rolling direction is determined statistically, and the projection length of each profile is recalculated based on the rolling direction;

[0137] Classification and merging module: Performs classification and determination of inclusions and merging of discrete contours;

[0138] Grading module: Completes grading assessment and generates test reports according to national standards;

[0139] Result output module: Outputs the detection results.

[0140] The modules are connected sequentially, and the data flows in one direction, enabling fully automated analysis.

[0141] This invention proposes an automatic analysis method and system for non-metallic inclusions based on computer vision. It can achieve fully automatic, high-precision, and nationally compliant detection of non-metallic inclusions in steel. This invention is applicable to non-metallic inclusion detection scenarios in steel, non-ferrous metals, and other metallic materials. It can be widely used in industries such as metallurgical production, machinery manufacturing, material quality inspection, quality control, and performance evaluation, providing efficient, stable, and standardized technical support for metallic material quality inspection, factory evaluation, and process optimization.

[0142] Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. Furthermore, those skilled in the art will recognize that, based on the ideas of this invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. An automatic analysis method for non-metallic inclusions based on computer vision, characterized in that, Includes the following steps: Step 1: Sample preparation and image acquisition: Complete steel sampling, sample preparation and metallographic image acquisition according to national standards to obtain raw digital images containing non-metallic inclusions; Step 2: Image preprocessing: The original digital image is sequentially subjected to black border removal, shadow removal, bilateral filtering for noise reduction, scratch removal, and adaptive thresholding or OTSU thresholding for binarization to obtain a binary image that retains only the intrusive target. Step 3: Contour retrieval and feature extraction: Perform contour retrieval on the binary image, extract all independent connected component contours, and calculate geometric features and grayscale features for each contour; Step 4: Rolling direction calculation: Select the elongated inclusion profile, and determine the overall rolling direction based on the direction angle of the elongated inclusion profile. Then, recalculate the projected length of each profile based on the rolling direction. Step 5: Inclusion Classification and Contour Merging: Using a contour classification decision algorithm, a multi-level decision logic is employed based on the morphological definitions of the five types of inclusions according to the national standard. First, DS-type inclusions are identified based on their equivalent diameter. Second, elongated A / C / B-type inclusions, granular B / D-type inclusions, and other aggregated B-type inclusions are distinguished based on their aspect ratio. Then, multiple discrete contours satisfying the national standard rules are merged into a single inclusion based on the projected length, spacing, and distribution direction, and D-type inclusions are distinguished based on the number of contours and their aspect ratio. Finally, for the merged contours, the number and confidence level of elongated A / C / B-type inclusions, granular B / D-type inclusions, and aggregated B-type inclusions within each contour are determined. First, class B inclusions are identified; finally, class A and class C inclusions are distinguished by relative color to complete the final classification. Step 6: Quantitative statistics and national standard grading: Statistically analyze the total length, quantity, maximum width, and equivalent diameter parameters of the inclusions, and complete the grading according to the national standards to generate grading results and test data; Step 7: Output Results: Output inclusion classification images, quantitative data, rating results, and standardized test report.

2. The automatic analysis method for non-metallic inclusions based on computer vision according to claim 1, characterized in that, Step 2: Image preprocessing: The original digital image is sequentially subjected to black border removal, shadow removal, bilateral filtering for noise reduction, scratch removal, and adaptive thresholding or OTSU thresholding binarization to obtain a binary image that retains only the intrusive target. Specifically, this includes: Step 2.1: Remove black borders The original digital image is thresholded to extract the white background, the outermost contour of the image is obtained and the minimum bounding rectangle of the contour is calculated. The bounding rectangle area is used as the effective image area, and excess black edges of the image are removed. Step 2.2: Shadow Removal The obtained digital image is converted into a grayscale image, a morphological dilation operation is performed on the grayscale image, and then an approximate background distribution map is generated by median filtering. The approximate background distribution map and the grayscale image are then compared to eliminate the effects of uneven lighting, shadows and background gradations, resulting in a target image with uniform brightness. The digital image includes the original digital image and the image after black border removal. Step 2.3: Bilateral filtering for noise reduction The target image after removing shadows is smoothed by bilateral filtering, which removes image noise and texture interference while preserving the edge information of the inclusions. Step 2.4: Scratch Removal Treatment Canny edge detection is used to extract image edges, and Hough line detection is used to identify the scratch lines generated during sample preparation. The identified scratch areas are then repaired using the Telea04 image restoration algorithm to blend the scratch areas with the background. Step 2.5: Binarization The preprocessed image is binarized using an adaptive thresholding method or the Otsu method, converting the preprocessed image into a binary image containing only black areas of inclusions and a white background.

3. The automatic analysis method for non-metallic inclusions based on computer vision according to claim 1, characterized in that, The geometric features include: area, aspect ratio, roundness, equivalent diameter, maximum projected length, width sequence, and profile orientation angle; The grayscale features include: average grayscale inside the contour, average grayscale of the neighboring background, and relative grayscale.

4. The automatic analysis method for non-metallic inclusions based on computer vision according to claim 3, characterized in that, The relative gray level is the ratio of the average pixel value inside the inclusion contour to the average pixel value of the neighboring background, used to distinguish inclusions from the matrix structure; The formula for calculating relative gray level is: In the formula: These are the pixel values ​​inside the outline. The value represents the background pixel count, and N and M represent the number of internal and background pixels, respectively.

5. The automatic analysis method for non-metallic inclusions based on computer vision according to claim 4, characterized in that, In step 4, the rolling direction is calculated as follows: A long strip-shaped inclusion profile is selected, and the overall rolling direction is determined statistically based on the direction angle of the long strip-shaped inclusion profile. The projected length of each profile is then recalculated based on the rolling direction. Specifically, this includes: Select the profile of the elongated inclusion, statistically analyze its angular radian value, and calculate the overall rolling direction average angle avgD using the following formula: In the formula: ND represents the direction angle of a single profile, and ND represents the number of profiles involved in the calculation. Using the average angle avgD of the overall rolling direction as the material rolling direction, the projected length of each profile in the rolling direction is recalculated as the basis for profile merging.

6. The automatic analysis method for non-metallic inclusions based on computer vision according to claim 5, characterized in that, Step 5: Inclusion Classification and Contour Merging: Using a contour classification decision algorithm, a multi-level decision logic is adopted based on the morphological definitions of the five types of inclusions according to the national standard. First, DS-type inclusions are identified based on equivalent diameter. Second, long strip-shaped A / C / B-type inclusions, granular B / D-type inclusions, and other aggregated B-type inclusions are distinguished based on aspect ratio. Then, multiple discrete contours that meet the national standard rules are merged into a single inclusion based on the projected length, spacing, and distribution direction, and D-type inclusions are distinguished based on the number of contours and aspect ratio. Next, for the contours that have been merged into one, the number and confidence level of long strip-shaped A / C / B-type inclusions, granular B / D-type inclusions, and aggregated B-type inclusions in each contour are determined. First, class B inclusions are identified. Finally, class A and class C inclusions are distinguished by relative color to complete the final classification. This includes: First, DS-type large-particle spherical oxides are identified according to their equivalent diameter; Secondly, preliminary classification and initial contour screening are performed. Based on the aspect ratio, elongated A / C / B type inclusions, granular B / D type inclusions, and other aggregated B type inclusions are distinguished. The contours are specifically categorized as follows: ①. Those with a larger length-to-width ratio are elongated A / C / B type inclusions; ②. Inclusions with a smaller aspect ratio are granular B / D type; ③. Other aggregated Class B inclusions; Then, based on the rolling direction angle, it is further calculated whether they can be merged into one. If they can be merged into one, then multiple discrete contours that meet the national standard rules are merged into the same inclusion according to the projected length, spacing, and distribution direction; specifically: After being merged into one, A contour with only one outline and a small aspect ratio is classified as type D. The remaining outlines are categories A, B, and C, which have already been merged into one. Next, for the contours that have been merged into one, the category of the contour is calculated by comprehensively considering the relative color, the number of contours belonging to the initial screening (①②③) contained in the contour, and whether it contains clustered B-type contours. Specifically: Based on the number of ①②③ contained in each item and the confidence level This allows for the accurate identification of aggregated B-type alumina inclusions, distinguishing between them. If there are many outlines with small aspect ratios, or the confidence level is high. If the value is larger, then this item is considered to be of category B; Otherwise, it is classified as either Category A or Category C; Finally, categories A and C are distinguished by their relative colors.

7. The automatic analysis method for non-metallic inclusions based on computer vision according to claim 6, characterized in that, In step 5, the confidence score Bc for aggregated type B inclusions is calculated to identify aggregated type B alumina inclusions. Specifically, this includes: The identification of aggregated Class B alumina inclusions is achieved by gradient evaluation of the width sequence of the contour in the direction of maximum projected length. The calculation method for the clustered B-type confidence score Bc is as follows: In the formula: NT represents the width sequence of the contour along the maximum projection length, where NT is the total number of points.

8. A computer vision-based automatic analysis system for non-metallic inclusions, applied to the computer vision-based automatic analysis method for non-metallic inclusions as described in any one of claims 1-7, characterized in that, The system includes: Image acquisition module: used to acquire metallographic images, obtain raw digital images containing non-metallic inclusions, and transmit them to the system; Preprocessing module: Performs black border removal, shadow removal, filtering, scratch removal, and binarization on the acquired raw digital image to obtain a binary image that retains only the intrusive target; Feature extraction module: performs contour retrieval on the obtained binary image and calculates geometric and grayscale features; Rolling direction calculation module: Based on the selected elongated inclusion profile direction angle, the overall rolling direction is determined statistically, and the projection length of each profile is recalculated based on the rolling direction; Classification and merging module: Performs classification and determination of inclusions and merging of discrete contours; Grading module: Completes grading assessment and generates test reports according to national standards; Result output module: Outputs the detection results.

Images