A method for automatic identification and quality evaluation of inclusions in aluminum alloy melts

Abstract

The present invention discloses an automatic recognition and quality evaluation method for inclusions in an aluminum alloy melt, which relates to the technical fields of image processing and target recognition, and includes: obtaining metallographic images of inclusions in a K-mode section of an aluminum alloy melt and image preprocessing under a stereo microscope; constructing a deep learning model; training the deep learning model; and creating a visual interactive system; a deep learning model trained with K-mode microstructure images can automatically identify inclusions in a to-be-detected image after training, and calculate the length and area of ​​the inclusions, and then calculate the inclusion influence factor and inclusion rate, so as to achieve a rapid evaluation of the purity of the aluminum alloy melt; compared with traditional evaluation methods, the deep learning model constructed by the present invention has high accuracy and a wide range of applications, and the created visual interactive system has a simple interface and is easy to use, which can greatly reduce manual labor and improve the work efficiency of the production line.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and target recognition, and more specifically, to an automatic recognition and quality evaluation method for aluminum alloy melt inclusions. Background Art

[0002] Aluminum alloy is an important industrial alloy with advantages such as good casting performance, high specific strength and corrosion resistance. Low carbon and environmental protection have become the mainstream trend of the development of the global automotive industry in the 21st century, and automobile lightweighting has also received attention and development in my country. The use of aluminum alloy castings is one of the important ways to achieve automobile lightweighting. Common automotive parts such as wheels, control arms, steering knuckles and subframes are mostly aluminum alloy castings.

[0003] The high quality of the melt is the premise for ensuring the performance of castings. The purity of the melt is the most important indicator for evaluating the quality of the melt. At present, the most widely used method for testing the purity of the melt is the K-mold method. The characterization of K-mold inclusions in aluminum alloy melts is mainly done manually, which has problems such as being greatly affected by manual experience, long analysis time, missed detection, and inconsistent dimensional measurement standards. Moreover, the test results need to be manually integrated and processed, which cannot reflect the size distribution of inclusions in real time and cannot quickly evaluate the quality of the melt.

[0004] Therefore, it is necessary to invent a deep learning method that can automatically and accurately identify inclusions in aluminum alloy melts, and can judge and feedback the severity of inclusions in real time to achieve rapid evaluation of melt quality. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes an automatic identification and quality evaluation method for aluminum alloy melt inclusions, comprising:

[0006] A method for automatically identifying and evaluating the quality of aluminum alloy melt inclusions, characterized in that it comprises the following steps:

[0007] Collection and preprocessing of deep learning training data, obtaining a sufficient number of K-molds from actual production ladles, obtaining metallographic images of inclusions in the K-mold section of aluminum alloy melt under a stereo microscope and image preprocessing;

[0008] Building deep learning models;

[0009] Input the metallographic images of aluminum alloy melt inclusions in the training set into the trained deep learning model, and perform digital representation and statistics on the recognition targets segmented by the deep learning model;

[0010] Determine the ability and accuracy of the deep learning model for target positioning and classification based on the harmonic mean of positioning loss, confidence loss, precision and recall (F1 score), formulate targeted treatment methods when the evaluation indicators do not meet the preset standards, and retrain the deep learning model using the adjusted training parameters;

[0011] Evaluate the overfitting of the deep learning model, formulate targeted treatment methods when missed detection or false detection occurs, and retrain the deep learning model using the adjusted training parameters;

[0012] Create a visual interactive system to use the trained deep learning model to perform real-time detection of K-mode inclusions and display the recognition results, inclusion size distribution, inclusion influencing factors and inclusion rate.

[0013] Furthermore, the collection and preprocessing of the deep learning training data, obtaining a sufficient number of K molds from the actual production ladles, obtaining metallographic images of inclusions in the K mold section of the aluminum alloy melt under a stereo microscope and preprocessing the images, are specifically as follows:

[0014] The aluminum alloy melt is poured into a sampling mold to obtain a K pattern, and after the sample is cooled, it is broken along the groove to obtain multiple fracture surfaces;

[0015] Use a stereo microscope to observe the cross-section of each fracture and take photos to obtain the metallographic image of the inclusions in each cross-section;

[0016] Use the annotation tool to mark the outer contour of each inclusion target in the metallographic images of the training set.

[0017] Furthermore, the establishment of the deep learning model is specifically as follows:

[0018] Through convolutional networks, multi-level feature pyramids and spatial pyramid pooling, multi-scale feature extraction of input images is achieved;

[0019] Fusion and enhancement of multi-scale features;

[0020] The extracted features are classified and bounding box regressed, and the classification loss and regression loss are calculated.

[0021] The multi-scale feature fusion and enhancement processing is specifically as follows:

[0022] The high-resolution features are fused with the low-level features for multi-scale features, and the multi-scale features are enhanced through geometric transformation, pixel transformation or image stitching.

[0023] Furthermore, the aluminum alloy melt inclusion metallographic images in the training set are input into the trained deep learning model, and the recognition targets segmented by the deep learning model are digitally characterized and counted, specifically:

[0024] The area and length of the recognition targets segmented by the deep learning model are digitally represented and calculated in the following way: the area is the size of the closed geometric figure formed by the edge pixels of the recognition target, and the length is the maximum value of the Euclidean distance calculated between the extreme points of the closed geometric figure formed by the edge pixels of the target.

[0025] Furthermore, when the evaluation indicators do not meet the preset standards, a targeted treatment method is formulated, specifically:

[0026] Adjust the number of iterations and batches, and add a small object detection layer.

[0027] Furthermore, when missed detection or false detection occurs, a targeted treatment method is formulated, specifically:

[0028] Regularization, adjust the weight parameters.

[0029] Furthermore, the recognition targets segmented by the deep learning model are divided into XS, S, M, L, and XL length levels, and the number of inclusions of each level in all detection images is counted. n 等级 , specifically:

[0030] XS grade: XS<100μm;

[0031] S grade: 100μm≤S<500μm;

[0032] M grade: 500μm≤M<800μm;

[0033] L grade: 800μm≤L<1200μm;

[0034] XL grade: XL ≥ 1200 μm.

[0035] Furthermore, the calculation formula of the inclusion influence factor is:

[0036] IF = n XS ×0.1 + n S ×0.5 + n M ×1 + n L ×1.5 + n xL ×2.

[0037] Furthermore, the calculation formula of the inclusion rate is:

[0038] Q = IF / 5.

[0039] The beneficial effects of the present invention are as follows:

[0040] (1) The deep learning model created adopts a single-stage target segmentation method with high target recognition efficiency, which solves the problem that inclusion defects in aluminum alloy melt cannot be automatically detected and accelerates the melt quality inspection process;

[0041] (2) By quantitatively characterizing inclusions, the number, length, and area of ​​inclusions are counted and calculated, and the defect conditions of inclusions are described in detail;

[0042] (3) Calculate the inclusion influence factor based on the characterization results and then obtain the inclusion rate, making the results of the evaluation method more objective and accurate;

[0043] (4) The established user interaction system is intuitive and efficient, which is convenient for users to use and quickly establish an evaluation system;

[0044] (5) The created deep learning model has no standardized requirements for the size of the image to be inspected, has strong universality, and can be used in the production line inspection process to evaluate the melt quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flowchart of the steps of the automatic segmentation and evaluation method of aluminum alloy melt inclusions implemented in the present invention;

[0046] Figure 2 The Yolo deep learning model structure diagram is implemented in the present invention;

[0047] Figure 3 A flowchart for training and applying a deep learning model for the present invention;

[0048] Figure 4 This is a schematic diagram of the digital representation of the area and length of the recognition target segmented by the deep learning model in the present invention.

[0049] Figure 5 This is a curve chart comparing training results for different numbers of iterations during the deep learning model training process of the present invention.

[0050] Figure 6 It is a schematic diagram of metallographic omission and misdetection of inclusions in the test set of the present invention.

[0051] Figure 7 This is a comparison curve of the generalization capabilities of different weight parameters used in the application test set of the present invention. DETAILED DESCRIPTION

[0052] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0053] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0054] like Figure 1-3 As shown, a method for automatically identifying and evaluating inclusions in aluminum alloy melts comprises:

[0055] S1: Collection and preprocessing of deep learning training data, obtaining a sufficient number of K-molds from the actual production ladles; using a stereo microscope, observing the cross section of the K-mold fracture at 16 times and taking pictures to obtain the metallographic images of inclusions in each cross section; using annotation tools (such as Labelme, but not limited to this tool), contour marking each inclusion target in the metallographic image in the training set and assigning a category name;

[0056] S2: Build a deep learning model based on the Ultralytics Yolo algorithm framework, and achieve multi-scale feature extraction, feature fusion and enhancement through the collaborative work of various modules, improve the model's ability to detect targets of different scales, and finally use GPU to train the deep learning model until high-precision target segmentation and classification are achieved;

[0057] S3: Input the metallographic images of aluminum alloy melt inclusions in the training set into the trained deep learning model, and perform digital representation and statistics on the recognition targets segmented by the deep learning model;

[0058] S4: Determine the ability and accuracy of the deep learning model for target positioning and classification based on the harmonic mean of positioning loss, confidence loss, precision and recall (F1 score), and formulate targeted treatment methods and retrain the deep learning model using adjusted training parameters when the evaluation indicators do not meet the preset standards;

[0059] S5: Evaluate the overfitting of the deep learning model, formulate targeted treatment methods when missed detection or false detection occurs, and retrain the deep learning model using the adjusted training parameters;

[0060] S6: Create a visual interactive system based on Streamlit, use the trained deep learning model to perform real-time detection of inclusions in aluminum alloy melts and display image segmentation results, inclusion size distribution, inclusion influencing factors and inclusion rate.

[0061] As a specific implementation of this embodiment, in S1, the microscopic metallographic images in the training set contain at least 500 images in total, including a sufficient number (80-100 images) of small-size inclusion images to facilitate the deep learning model to learn and extract the features of small-scale inclusions; when using Labelme to annotate the inclusions in the metallographic images, the generated Json annotation file needs to be converted into a TXT file in Yolo format.

[0062] In S2, a standardized Yolo image segmentation model is built, including image homogenization processing, that is, the pixel matrix is ​​divided by 255, so that the value of the image matrix is ​​between [0,1], and online data enhancement, randomly cropping, flipping, rotating, translating and mirroring the image through geometric transformation methods to expand the data, and randomly adjusting the HSV contrast, brightness, saturation, histogram homogenization, etc. through pixel transformation methods to expand the data to simulate the metallographic image method under different shooting conditions, so as to improve the performance and robustness of the deep learning model. This embodiment needs to be explained here, such as Figure 2 As shown in the figure, the construction of the deep learning model is based on the Ultralytics Yolo algorithm framework. The deep learning network includes three main modules: the Backbone module is responsible for feature extraction of the input image, and the feature map is decomposed through the convolutional network CBS module (convolutional layer (3×3) + normalization + SiLu activation) and C2F module (convolutional layer (1×1) + Bottleneck + convolutional layer (1×1)) to generate a multi-level feature pyramid, where Bottleneck: convolutional layer (1×1) + convolutional layer (3×3), and then spatial pyramid pooling SPPF (convolutional layer (1×1) + maximum pooling (multiple) + convolutional layer (1×1)) is used to achieve multi-scale feature extraction. This CSPDarknet-like structure can effectively reduce the amount of calculation and improve the efficiency of feature extraction; the Neck module is used to achieve feature fusion and enhancement, and the idea of ​​​​path aggregation network is used to fuse high-resolution features with low-level features to improve the model's detection ability for targets of different scales; the Head module is responsible for classifying the extracted features and regressing the bounding box. A decoupled head structure is adopted to decompose the classification and regression tasks into two branches, and BCE is used to calculate the classification loss and DFL+CIOU is used to calculate the regression loss.

[0063] In S3, the deep learning model needs to be iteratively trained multiple times. In each training, the images in the training set are sent to the model in batches for training; the segmented targets are digitally represented and counted, and the calculation method is that the area is the area of ​​the closed geometric figure formed by the edge pixel connection of the segmented target, and the length is the maximum value of the Euclidean distance calculated between the extreme points (the extreme north point (N), the extreme south point (S), the extreme east point (E) and the extreme west point (W)) of the closed geometric figure formed by the edge pixel connection of the target. Figure 4 The metallographic images of the inclusions shown are columns. The scale is 100 µm long and contains 41 pixels. The deep learning model identifies an inclusion with a characteristic area (the area enclosed by the white border in the output result image) = 155428.74 µm 2 , its characteristic length = maximum value (719.46µm, 184.13µm, 618.92µm, 605.47µm, 160.0µm, 539.41µm) = 719.46µm.

[0064] In S4, we now take the F1 score as an example to explain the iterative training process of deep learning models; the F1 score takes into account both precision and recall and is an important performance indicator for judging the classification and recognition capabilities of deep models. The closer the value is to 1.0, the better the comprehensive performance of the trained model in terms of precision and recall. The preset indicator is 0.85. If the trained deep model is lower than the prediction indicator, you can consider increasing the number of iterations or batches to improve the performance of the model. Figure 5 As shown in the figure, by increasing the number of iterations, both the F1 score and the confidence of target detection, i.e., the confidence, can be improved.

[0065] In S5, the images in the test set are used to judge the overfitting of the trained model. If the trained deep learning model does not have a strong generalization ability for object recognition in the images in the test set, missed detection and false detection may occur (such as Figure 6 As shown in ), you can consider reducing model parameters to improve the generalization ability of the trained deep learning model. Figure 7 As shown in the figure, under the same number of iterations and batches, the number of training weight parameters is reduced. Although the F1 score of the deep learning model that has completed training is slightly lower than that of the model with more weight parameters (0.85 VS 0.88), its F1 score still meets the prediction value without increasing the number of iterations or batches. Moreover, it has been tested that the deep learning model with fewer weight parameters still has a high accuracy rate for target recognition under the current conditions, and has better generalization ability.

[0066] In S6, the user loads the image to be tested in the visual interactive system and enters the ratio of the ruler length to the corresponding pixel value in the metallographic image; after the image is loaded, Streamlit will call the trained deep learning model to perform target image segmentation on the loaded image, and the data page will display the information of the identified inclusions, including their characteristic area, length, grade and size distribution, and then obtain the inclusion influence factor and inclusion rate. Taking a set of aluminum alloy melt K mold metallographic images as an example, if 15 inclusions are identified from 11 loaded images, including 14 S-level inclusions and 1 M-level inclusion, the inclusion influence factor = 14×0.5 + 1×1 = 8.0, and the inclusion rate = 8 / 5 = 1.6. The user then jumps to the image page to view the target segmentation results.

[0067] Through the above embodiments, the feasibility of the method of the present invention is verified, and the recognition speed is fast (it only takes 17.4 milliseconds to detect each picture) and the result is highly accurate.

[0068] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for automatic identification and quality evaluation of aluminum alloy melt inclusions, characterized in that: The following steps are involved: Collection and preprocessing of deep learning training data, obtaining a sufficient number of K-molds from actual production ladles, obtaining metallographic images of inclusions in the K-mold section of aluminum alloy melt under a stereo microscope and image preprocessing; Building deep learning models; Input the metallographic images of aluminum alloy melt inclusions in the training set into the trained deep learning model, and perform digital representation and statistics on the recognition targets segmented by the deep learning model; Determine the ability and accuracy of the deep learning model for target positioning and classification based on the harmonic mean of positioning loss, confidence loss, precision and recall (F1 score), formulate targeted treatment methods when the evaluation indicators do not meet the preset standards, and retrain the deep learning model using the adjusted training parameters; Evaluate the overfitting of the deep learning model, formulate targeted treatment methods when missed detection or false detection occurs, and retrain the deep learning model using the adjusted training parameters; Create a visual interactive system to use the trained deep learning model to perform real-time detection of K-mode inclusions and display the recognition results, inclusion size distribution, inclusion influencing factors, and inclusion rate; The aluminum alloy melt inclusion metallographic images in the training set are input into the trained deep learning model, and the recognition targets segmented by the deep learning model are digitally characterized and counted, specifically: The area and length of the recognition targets segmented by the deep learning model are digitally represented and calculated. The calculation method is as follows: the area is the area of ​​the closed geometric figure formed by the edge pixel connection of the recognition target, and the length is the maximum value of the Euclidean distance calculated between the extreme points of the closed geometric figure formed by the edge pixel connection of the target; The recognition targets segmented by the deep learning model are divided into XS, S, M, L, and XL length levels, and the number of inclusions of each level in all detection images is counted n 等级 , specifically: XS grade: XS<100μm; S grade: 100μm≤S<500μm; M grade: 500μm≤M<800μm; L grade: 800μm≤L<1200μm; XL grade: XL ≥ 1200 μm; The calculation formula of the inclusion influence factor is: IF = n XS ×0.1 + n S ×0.5 + n M ×1 + n L ×1.5 + n xL ×2; The calculation formula of the inclusion rate is: Q = IF / 5.

2. The method for automatic identification and quality evaluation of aluminum alloy melt inclusions according to claim 1, characterized in that: The collection and preprocessing of the deep learning training data, obtaining a sufficient number of K molds from the actual production ladles, obtaining metallographic images of inclusions in the K mold section of the aluminum alloy melt under a stereo microscope and preprocessing of the images, are specifically as follows: The aluminum alloy melt is poured into a sampling mold to obtain a K pattern, and after the sample is cooled, it is broken along the groove to obtain multiple fracture surfaces; Use a stereo microscope to observe the cross-section of each fracture and take photos to obtain the metallographic image of the inclusions in each cross-section; Use the annotation tool to mark the outer contour of each inclusion target in the metallographic images of the training set.

3. The method for automatic identification and quality evaluation of aluminum alloy melt inclusions according to claim 1, characterized in that: The establishment of the deep learning model is specifically as follows: Through convolutional networks, multi-level feature pyramids and spatial pyramid pooling, multi-scale feature extraction of input images is achieved; Fusion and enhancement of multi-scale features; The extracted features are classified and bounding box regressed, and the classification loss and regression loss are calculated.

4. The method for automatic identification and quality evaluation of aluminum alloy melt inclusions according to claim 3, characterized in that: The multi-scale feature fusion and enhancement processing is specifically as follows: The high-resolution features are fused with the low-level features for multi-scale features, and the multi-scale features are enhanced through geometric transformation, pixel transformation or image stitching.

5. The method for automatic identification and quality evaluation of aluminum alloy melt inclusions according to claim 1, characterized in that: When the evaluation indicators do not meet the preset standards, a targeted treatment method is formulated, specifically: Adjust the number of iterations and batches, and add a small object detection layer.

6. The method for automatic identification and quality evaluation of aluminum alloy melt inclusions according to claim 1, characterized in that: When missed detection or false detection occurs, a targeted treatment method is formulated, specifically: Regularization, adjust the weight parameters.

Images