Aluminum profile metallographic grading method and system, electronic equipment and storage medium

Abstract

The invention discloses an aluminum profile metallographic rating method and system, electronic equipment and a storage medium, and belongs to the technical field of computers, and the method comprises the steps: carrying out the data preprocessing of an original metallographic image of an aluminum profile, and obtaining a second metallographic image; performing grain region division and image processing on the second metallographic image to obtain a first impurity image and a grain boundary image; wherein the grain boundary image comprises a plurality of connected regions representing grains; subtracting the first impurity image from the original metallographic image to obtain a first background image; performing feature extraction on the original metallographic image, the first impurity image and the first background image according to a connected region of the grain boundary image to obtain a plurality of first features; performing correlation analysis to obtain correlation characteristics; and according to the correlation characteristics, performing classification prediction on the crystal grains through a pre-trained machine learning model to obtain the metallographic grade of the aluminum profile. According to the method, high-efficiency and high-precision metallographic grade evaluation can be carried out, diversified quality rating requirements are met, and the universality is high.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, system, electronic device and storage medium for metallographic grading of aluminum profiles. Background Art

[0002] The new generation of aluminum profiles used in products (mobile phone frames, buttons, back panels, etc.) have an extreme pursuit of a mirror-like gloss after oxidation of the material surface, which significantly increases the difficulty of manufacturing during the production process. Common quality defects include fogging and whitening of the oxidized surface, which are caused by subtle differences in the microstructure of the material, especially the complex regulation of precipitation behavior during the heat treatment stage. Metallographic testing is a key means to reveal these microscopic differences. For a long time, the determination of test results has relied on manual identification and lacked a unified quantitative standard, making it difficult to accurately adapt to more diverse quality requirements. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to provide a method, system, electronic device and storage medium for metallographic grading of aluminum profiles with high efficiency, high accuracy and high versatility.

[0004] To achieve the above-mentioned purpose, one aspect of an embodiment of the present application provides a metallographic rating method for aluminum profiles, the method comprising:

[0005] Performing data preprocessing on the original metallographic image of the aluminum profile to obtain a second metallographic image;

[0006] Performing grain region division and image processing on the second metallographic image to obtain a first impurity image and a grain boundary image; wherein the grain boundary image includes a plurality of connected regions representing grains;

[0007] Subtracting the first impurity image from the original metallographic image to obtain a first background image;

[0008] According to the connected area of ​​the grain boundary image, feature extraction is performed on the original metallographic image, the first impurity image and the first background image to obtain a plurality of first features;

[0009] Performing correlation analysis on the first feature to obtain a correlation feature;

[0010] According to the correlation characteristics, the grains are classified and predicted using a pre-trained machine learning model to obtain the metallographic grade of the aluminum profile.

[0011] In some embodiments, the data preprocessing includes at least one of contrast limited adaptive histogram image equalization processing, median filtering processing, and bilateral filtering processing.

[0012] In some embodiments, the step of performing grain region division and image processing on the second metallographic image to obtain a first impurity image and a grain boundary image comprises the following steps:

[0013] Performing a binarization process of an adaptive threshold on the second metallographic image to obtain a first impurity image reflecting impurity characteristics and a binarized image retaining grain characteristics;

[0014] The binary image is processed alternately by using connected domain analysis and morphological operation to obtain a grain boundary image containing connected regions.

[0015] In some embodiments, the feature extraction is performed on the original metallographic image, the first impurity image and the first background image according to the connected area of ​​the grain boundary image to obtain a plurality of first features, including:

[0016] Overlaying the connected area of ​​the grain boundary image onto the original metallographic image, the first impurity image and the first background image;

[0017] According to the connected areas, image features corresponding to each of the connected areas on the original metallographic image are extracted to obtain metallographic image features; wherein the metallographic image features include the original image grayscale average value, the original image grayscale standard deviation, the original image grayscale mode, the original image grayscale mode proportion, the original image entropy value, the original image grayscale maximum value, the original image grayscale minimum value, the original image grayscale median, the original image grayscale value of more than 180 pixels, the original image skewness and the original image kurtosis;

[0018] According to the connected regions, image features corresponding to each of the connected regions on the first impurity image are extracted to obtain impurity image features; the impurity image features include impurity proportions and impurity image entropy values;

[0019] According to the connected regions, image features corresponding to each of the connected regions on the first background image are extracted to obtain background image features; wherein the background image features include a background grayscale average value and a background grayscale standard deviation;

[0020] Extracting the area and perimeter of each grain in the grain boundary image;

[0021] The metallographic image feature, the impurity image feature, the background image feature, the area and the perimeter are used as first features.

[0022] In some embodiments, performing correlation analysis on the first feature to obtain a correlation feature comprises the following steps:

[0023] Calculating correlations between the first features in pairs;

[0024] The first features are screened according to the correlation to obtain correlation features; wherein the screening process includes retaining one of the first features whose correlation is higher than a preset threshold.

[0025] In some embodiments, the metallographic level is predicted according to the correlation feature by a pre-trained machine learning model to obtain the metallographic level, comprising the following steps:

[0026] Using a plurality of classifiers in a pre-trained random forest model, classify the grains in the original metallographic image according to the correlation features to obtain a plurality of classification results;

[0027] Voting on the classification results to obtain a final grain category;

[0028] The metallographic grade of the aluminum profile is obtained according to the grain type.

[0029] In some embodiments, the training step of the machine learning model includes:

[0030] Filter qualified and unqualified grain data from multiple original metallographic images to obtain a data set;

[0031] Dividing the data set into a training set and a test set according to a preset ratio;

[0032] Constructing a machine learning model to perform grain division on the training set to obtain training results;

[0033] The training results are corrected by an extreme gradient boosting decision tree to minimize the loss function and obtain a pre-trained machine learning model.

[0034] To achieve the above purpose, another aspect of the embodiment of the present application provides a metallographic grading system for aluminum profiles, the system comprising:

[0035] The first module is used to perform data preprocessing on the original metallographic image of the aluminum profile to obtain a second metallographic image;

[0036] The second module is used to perform grain region division and image processing on the second metallographic image to obtain a first impurity image and a grain boundary image; wherein the grain boundary image includes a plurality of connected regions representing grains;

[0037] A third module is used to obtain a first background image by subtracting the first impurity image from the original metallographic image;

[0038] A fourth module is used to extract features from the original metallographic image, the first impurity image and the first background image according to the connected area of ​​the grain boundary image to obtain a plurality of first features;

[0039] A fifth module is used to perform correlation analysis on the first feature to obtain a correlation feature;

[0040] The sixth module is used to classify and predict the grains according to the correlation characteristics through a pre-trained machine learning model to obtain the metallographic grade of the aluminum profile.

[0041] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above-mentioned method when executing the computer program.

[0042] To achieve the above objective, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0043] The embodiments of the present application include at least the following beneficial effects: The present application provides a method, system, electronic device and storage medium for metallographic grading of aluminum profiles, which obtains a second metallographic image by performing data preprocessing on the original metallographic image of the aluminum profile; performs grain area division and image processing on the second metallographic image to obtain a first impurity image and a grain boundary image; wherein the grain boundary image includes a number of connected areas representing grains; the first impurity image is subtracted from the original metallographic image to obtain a first background image; according to the connected areas of the grain boundary image, feature extraction is performed on the original metallographic image, the first impurity image and the first background image to obtain a number of first features; correlation analysis is performed on the first features to obtain correlation features; according to the correlation features, the grains are classified and predicted by a pre-trained machine learning model to obtain the overall steps of the metallographic grade of the aluminum profile, which can efficiently and accurately evaluate and grade the microstructural features of the aluminum profile, and because it is processed by computer vision, it is conducive to adapting to diverse quality grading needs and has high versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings are used to provide further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0045] Figure 1 It is a step diagram of a method for metallographic rating of aluminum profiles provided in an embodiment of the present application;

[0046] Figure 2 is a flow chart of obtaining a grain boundary image provided by an embodiment of the present application;

[0047] Figure 3is a schematic diagram of the original metallographic image provided in the embodiment of the present application;

[0048] Figure 4 is a schematic diagram of a contrast-limited adaptive histogram equalization result provided in an embodiment of the present application;

[0049] Figure 5 is a schematic diagram of the median filtering result provided in an embodiment of the present application;

[0050] Figure 6 is a schematic diagram of the bilateral filtering result provided in an embodiment of the present application;

[0051] Figure 7 is a schematic diagram of the processing result when the mean adaptive threshold parameter is (mean, 9, 1) provided in an embodiment of the present application;

[0052] Figure 8 This is a schematic diagram of the processing result of the connected domain analysis with a size of 100 provided in the embodiment of the present application.

[0053] Fig. 9 is a schematic diagram of a closing operation result provided in an embodiment of the present application;

[0054] Fig.10 This is a schematic diagram of the processing result of the connected domain analysis with a size of 500 provided in an embodiment of the present application;

[0055] Fig.11 It is a schematic diagram of the processing result provided by the embodiment of the present application when the opening operation is 1 and the closing operation is 2;

[0056] Fig.12 This is a schematic diagram of the processing result of the connected domain analysis with a size of 4000 provided in an embodiment of the present application;

[0057] Fig.13 is a schematic diagram of a processing result when the closing operation is 2 provided in an embodiment of the present application;

[0058] Fig.14 is a schematic diagram of a processing result of a connected domain analysis of an inverted image with a size of 1000 provided in an embodiment of the present application;

[0059] Fig.15 This is a schematic diagram of the divided grains marked according to an embodiment of the present application;

[0060] Fig.16 is a schematic diagram of the processing results when the adaptive threshold parameters provided in the embodiment of the present application are (mean, 9, 3);

[0061] Fig.17 is a background difference map provided in an embodiment of the present application;

[0062] Fig.18is a grain boundary image provided by an embodiment of the present application;

[0063] Fig.19 It is a schematic diagram of feature selection provided by an embodiment of the present application;

[0064] Fig. 20 It is a correlation matrix heat map provided in the embodiment of the present application;

[0065] Fig.21 This is a schematic diagram of random forest importance ranking provided in an embodiment of the present application;

[0066] Fig. 22 It is a flowchart of establishing and predicting a machine learning model provided in an embodiment of the present application;

[0067] Fig.23 It is the original image of the test metallographic phase 1 provided in the embodiment of the present application;

[0068] Fig.24 This is an image of the test metallographic phase 1 after the grains are divided provided in the embodiment of the present application;

[0069] Fig.25 It is the original image of the test metallographic phase 2 provided in the embodiment of the present application;

[0070] Fig.26 This is an image of the divided grains of the test metallographic phase 2 provided in the embodiment of the present application;

[0071] Fig. 27 It is a module schematic diagram of a metallographic rating system for aluminum profiles provided in an embodiment of the present application;

[0072] Fig.28 It is a schematic diagram of the system user interface provided in the embodiment of the present application;

[0073] Fig.29 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0075] Although the functional modules are divided in the system schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first / S100", "second / S200", etc. in the specification, claims and the above drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0076] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0077] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0078] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0079] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0080] Before describing the embodiments of the present application in detail, some nouns and terms involved in the embodiments of the present application are first described. The nouns and terms involved in the embodiments of the present application are subject to the following explanations.

[0081] (1) Adaptive threshold: The threshold can be automatically adjusted for image areas with different brightness or contrast, making threshold processing more flexible and effective. Compared with the global threshold, adaptive threshold processing can calculate the threshold for a small area (neighborhood) around each pixel of the image, thereby performing binarization processing based on local image features, which can effectively eliminate the problem of uneven illumination in metallographic images.

[0082] (2) Morphological operations: The opening operation, first erosion and then dilation, can eliminate small, isolated noise points in the image and smooth the contours of objects; the closing operation, first dilation and then erosion, can connect adjacent objects, fill small holes, and smooth the boundaries of objects.

[0083] (3) Connected domain analysis: The process of identifying and marking adjacent sets of pixels in a binary image that have the same pixel value (usually 1, representing the target). Its purpose is to segment the target object in the image into multiple independent connected regions.

[0084] In related technologies, with the new generation of products using aluminum profiles (mobile phone middle frames, buttons, back panels, etc.) pursuing the ultimate pursuit of a nearly mirror-like gloss after oxidation of the material surface, the management of each link in the production process faces unprecedented challenges, significantly increasing the difficulty of manufacturing. In this process, common quality defects such as fogging and whitening of the oxidized surface actually stem from subtle differences in the internal microstructure of the material, especially the complex regulation of precipitation behavior during the heat treatment stage. Metallographic testing, as a key means of revealing these microscopic differences, has long relied on manual identification and lacks a unified quantitative standard for the determination of test results, making it difficult to accurately adapt to diverse quality requirements.

[0085] In view of this, a method, system, electronic device and storage medium for metallographic grading of aluminum profiles are provided in the embodiments of the present application. The scheme combines computer vision and machine learning technology to grade the photos by distinguishing between "good grains" and "bad grains" in metallographic photos and calculating the area ratio of "good grains" in the metallographic photos, thereby achieving objective quantification of the detection results. According to different proportions, the metallographic grades are divided into several grades, such as five grades of A, B, C, D, and E, to eliminate interference from human factors and ensure accurate evaluation and grading of the microscopic structural characteristics of the material, thereby effectively improving the product quality control level and meeting the industry's stringent requirements for high-quality and high-precision mobile phone manufacturing.

[0086] The metallographic rating method of aluminum profiles provided in the embodiment of the present application relates to the field of computer technology. The metallographic rating method of aluminum profiles provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or it can be configured as a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements a metallographic rating method for aluminum profiles, etc., but is not limited to the above forms.

[0087] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0088] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0089] Figure 1 is an optional flow chart of a method for metallographic rating of aluminum profiles provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S100 to S600.

[0090] Step S100, performing data preprocessing on the original metallographic image of the aluminum profile to obtain a second metallographic image;

[0091] Step S200, performing grain region division and image processing on the second metallographic image to obtain a first impurity image and a grain boundary image; wherein the grain boundary image includes a plurality of connected regions representing grains;

[0092] Step S300, subtracting the first impurity image from the original metallographic image to obtain a first background image;

[0093] Step S400, performing feature extraction on the original metallographic image, the first impurity image and the first background image according to the connected area of ​​the grain boundary image to obtain a plurality of first features;

[0094] Step S500, performing correlation analysis on the first feature to obtain a correlation feature;

[0095] Step S600: Classify and predict the grains according to the correlation characteristics through a pre-trained machine learning model to obtain the metallographic grade of the aluminum profile.

[0096] Steps S100 to S600 shown in the embodiment of the present application can efficiently and accurately evaluate and grade the microstructural characteristics of aluminum profiles, and because they are processed through computer vision, they are conducive to adapting to diverse quality rating requirements and are highly versatile.

[0097] In step S100 of some embodiments, the data preprocessing includes at least one of contrast-limited adaptive histogram image equalization processing, median filtering processing, and bilateral filtering processing.

[0098] In some embodiments, step S200 includes but is not limited to the following steps S210 to S220:

[0099] Step S210, performing a binarization process of an adaptive threshold on the second metallographic image to obtain a first impurity image reflecting impurity characteristics and a binarized image retaining grain characteristics;

[0100] Step S220 , alternately processing the binary image using connected domain analysis and morphological operations to obtain a grain boundary image containing connected regions.

[0101] In some embodiments, step S400 includes but is not limited to the following steps S410 to S460:

[0102] Step S410, overlaying the connected area of ​​the grain boundary image onto the original metallographic image, the first impurity image and the first background image;

[0103] Step S420, extracting image features corresponding to each of the connected areas on the original metallographic image according to the connected areas, and obtaining metallographic image features; wherein the metallographic image features include the original image grayscale average value, the original image grayscale standard deviation, the original image grayscale mode, the original image grayscale mode ratio, the original image entropy value, the original image grayscale maximum value, the original image grayscale minimum value, the original image grayscale median, the original image grayscale ratio of pixels above 180 grayscale values, the original image skewness and the original image kurtosis;

[0104] Step S430, extracting image features corresponding to each of the connected regions on the first impurity image according to the connected regions to obtain impurity image features; the impurity image features include impurity proportions and impurity image entropy values;

[0105] Step S440, extracting image features corresponding to each of the connected regions on the first background image according to the connected regions to obtain background image features; wherein the background image features include a background grayscale average value and a background grayscale standard deviation;

[0106] Step S450, extracting the area and perimeter of each grain in the grain boundary image;

[0107] Step S460: taking the metallographic image feature, the impurity image feature, the background image feature, the area and the perimeter as the first feature.

[0108] In some embodiments, step S500 includes but is not limited to the following steps S510 to S520:

[0109] Step S510, calculating the correlation between the first features in pairs;

[0110] Step S520, screening the first features according to the correlation to obtain correlation features; wherein the screening includes retaining one of the first features whose correlation is higher than a preset threshold.

[0111] In some embodiments, step S600 includes but is not limited to the following steps S610 to S630:

[0112] Step S610, classifying the grains in the original metallographic image according to the correlation features through several classifiers in the pre-trained random forest model to obtain several classification results;

[0113] Step S620, voting on the classification results to obtain a final grain category;

[0114] Step S630, obtaining the metallographic grade of the aluminum profile according to the grain type.

[0115] In some embodiments, the training step of the machine learning model in step S600 includes the following steps a to d:

[0116] Step a, screening qualified and unqualified grain data from multiple original metallographic images to obtain a data set;

[0117] Step b, dividing a training set from the data set according to a preset ratio;

[0118] Step c, constructing a machine learning model to perform grain division on the training set to obtain training results;

[0119] Step d: Correct the training results by using an extreme gradient boosting decision tree to minimize the loss function and obtain a pre-trained machine learning model.

[0120] Below, in conjunction with a specific example of an application in a metallographic rating scenario of an aluminum profile, the solution of the embodiment of the present application is described in detail and explained:

[0121] In the embodiment of the present application, a metallographic grading method for aluminum profiles is provided, which can be applied, but not limited to, to accurately grading ultra-bright 3C aluminum profiles, effectively improving the level of product quality control. Specifically, it can be achieved by the following steps:

[0122] 1. Perform grain segmentation on the original metallographic image.

[0123] A metallographic picture is a microscopic photo taken by a metallographic microscope that can show the microstructural characteristics of the material. Get the metallographic picture of the aluminum profile and get the original metallographic image.

[0124] Further, the original metallographic image of the aluminum profile is subjected to at least one of contrast limited adaptive histogram equalization (CLAHE), median filtering and bilateral filtering to complete image preprocessing and obtain a second metallographic image. The original metallographic image (original image) and the schematic diagram of each preprocessing are shown in FIG. Figure 3 , Figure 4 , Figure 5 and Figure 6As shown. Contrast-limited adaptive histogram equalization processing is beneficial to improve image quality and image contrast, which helps to enhance the contrast between grains, making grain boundaries more prominent, and can also reduce noise amplification, which may interfere with accurate extraction. Median filtering is a nonlinear filtering technique that is often used to remove noise from images or videos, especially salt and pepper noise, while keeping edge information relatively good. Bilateral filtering can maintain edge clarity while removing noise. Grain boundaries, as important edge information, need to be well protected. This processing can effectively retain the clarity and details of grain boundaries while removing image noise.

[0125] Furthermore, based on the adaptive threshold, the second metallographic image is binarized with appropriate parameters to obtain a first impurity image that can reflect the impurity characteristics and a binarized image that retains most of the characteristics and is used to find the grain boundary.

[0126] Adaptive thresholding is an image processing method used to convert an image into a binary image, that is, the value of each pixel in the image is set to 0 or 255 (in an 8-bit image). The core idea of ​​adaptive thresholding is that instead of using a global fixed threshold to process the entire image, the threshold is dynamically calculated based on the local neighborhood information of each pixel in the image.

[0127] The grain boundaries on the binary image obtained above are discontinuous, and there are many points that do not belong to the grain boundaries. These points are regarded as impurity points in the process of extracting grain boundaries. Figure 2 First, we use the connected domain analysis to remove small impurity points, and then use morphological operations to eliminate small, isolated noise points and fill the voids between grain boundaries. We use the connected domain analysis and morphological operations alternately to gradually increase the threshold of the removed connected domain. Finally, we remove some small connected domains and get a picture of the divided grain marks, which is convenient for subsequent calculations and predictions. The schematic diagram of the processing results is shown in the figure. Figure 7 , Figure 8 , Fig. 9 , Fig.10 , Fig.11 , Fig.12 , Fig.13 and Fig.14 The schematic diagram after dividing the grain marking is shown as Fig.15 shown.

[0128] 2. Feature extraction.

[0129] Through image processing, three characteristic images can be obtained, namely the original metallographic image, the first impurity image (such as Fig.16 As shown, the binary image obtained by the adaptive threshold, the parameters are selected so that the impurity characteristics of the metallographic structure can be reflected) and the first background image (such as Fig.17The background difference image shown is an image obtained by subtracting the first impurity image from the original metallographic image. The grayscale value of the impurity point pixels is 255, and these pixels can be ignored in the calculation).

[0130] According to the connected area of ​​the obtained grain boundary image, the divided area (such as Fig.18 The grain boundary image shown in FIG. 1 is overlaid on the feature image; thus, for each connected region, all pixel values ​​of the region at the corresponding position on the feature image are extracted, and these values ​​are stored in a one-dimensional array.

[0131] The characteristics of the grains after division can be found from these characteristic images, and the characteristics useful for judging whether the grains are qualified can be selected.

[0132] In some embodiments, Fig.19 As shown in the figure, for the original metallographic image after dividing the area, the following features can be extracted:

[0133] ① Grayscale average of the original image: The grayscale average reflects the overall brightness level of the image.

[0134] ② Grayscale standard deviation of the original image: The grayscale standard deviation reflects the degree of discreteness of the grayscale distribution of the image. In the metallographic image of aluminum alloy, a large grayscale standard deviation may mean that the grayscale changes between grains or within grains are large.

[0135] ③ The grayscale value with the largest number of grayscale values ​​in the grain (the grayscale mode of the original image): This feature indicates which grayscale value appears most frequently in the grain area.

[0136] ④ The proportion of grayscale values ​​with the largest grayscale value in the grain (the proportion of grayscale mode in the original image): further quantifies the position of this grayscale value in the grain, which helps to analyze the uniformity and consistency of the internal structure of the grain. The better the grain, the larger the value.

[0137] ⑤ Entropy value of the original image: In the metallographic image of aluminum alloy, the size of the entropy may reflect the complexity of the grain structure, morphology and distribution in the image. The higher the entropy value, the greater the amount of information contained in the image and the more complex the grain structure may be.

[0138] ⑥ Maximum grayscale value of the original image: In the aluminum alloy metallographic image, the maximum grayscale value may be related to factors such as the reflection of the material surface, the degree of corrosion or local defects. It can help identify abnormal bright spots or highlighted areas in the image.

[0139] ⑦ Minimum grayscale value of the original image: It may reflect defects such as depressions, holes or corrosion pits on the surface of the material.

[0140] ⑧ Median grayscale of the original image: Evaluates the overall brightness level of the image and the symmetry of the grayscale distribution. In aluminum alloy metallographic images, the median grayscale may reflect the average grayscale level or main distribution range inside the grains.

[0141] ⑨The proportion of pixels with grayscale values ​​above 180 in the original image: The metallographic image is exposed to 180 grayscale when taken. This feature can reflect the brightness characteristics of the grains.

[0142] ⑩ Original image skewness: Skewness is a statistic that describes the data distribution pattern and is used to measure the symmetry of data distribution. In the grayscale distribution of aluminum alloy metallographic images, skewness can reflect the degree of asymmetry of the grayscale value distribution. Positive skewness means that the grayscale values ​​are mostly concentrated on the lower side; negative skewness means that the grayscale values ​​are mostly concentrated on the higher side. Skewness helps analyze the distribution of grayscale inside the grain.

[0143] Kurtosis of original image: Kurtosis is a statistic that describes the sharpness of data distribution. In the grayscale distribution of aluminum alloy metallographic images, kurtosis can reflect the sharpness or flatness of the grayscale value distribution. A high kurtosis indicates that the grayscale value distribution is relatively concentrated and sharp; a low kurtosis indicates that the grayscale value distribution is relatively dispersed and flat. Kurtosis helps to evaluate the concentration and distribution characteristics of the grayscale inside the grain.

[0144] In some embodiments, the following features can be extracted from the binary image obtained by the adaptive threshold after segmentation:

[0145] ① Impurity ratio: The impurity ratio directly reflects the content of impurities in the aluminum alloy grains. A lower impurity ratio means a purer grain.

[0146] ② Entropy of binary image: The higher the entropy value, the more complex and random the distribution of black and white pixels in the image is, which may mean that the grain structure is more complex or there are more subtle changes.

[0147] In some embodiments, the following features may be extracted from the divided background image:

[0148] ① Background grayscale average: removes the influence of impurities and better reflects the average brightness on the grain background.

[0149] ② Background grayscale standard deviation: After removing the influence of impurity points, it can better reflect the discrete degree of brightness on the grain background.

[0150] In some embodiments, the area and perimeter features of each grain boundary in the grain boundary image may also be extracted. The area is the number of pixels; the perimeter is the perimeter formed by drawing the boundary line of the connected region.

[0151] 3. Feature correlation screening.

[0152] Select some original metallographic images, divide the grains using the above method and calculate the above characteristics to obtain the characteristic data of the grains. After manually judging whether the grains are qualified, screen the characteristics of the grains and obtain the data set.

[0153] The features are screened based on relevance, which includes the following steps:

[0154] The features are selected by correlation method. Specifically, the training set is obtained by manually selecting the corresponding data of qualified and unqualified grains from multiple original metallographic images. The correlation matrix and the corresponding thermal map are obtained after correlation analysis on the training set. Fig. 20 shown. Fig. 20 In the figure, brightness represents the grayscale average value of the original image, brightness_std_dev represents the grayscale standard deviation of the original image, over_180_rate represents the proportion of pixels with grayscale values ​​above 180 in the original image, most_common_values ​​represents the grayscale value with the most grayscale values ​​in the grain (the grayscale mode of the original image), most_common_value_ratio represents the proportion of grayscale values ​​with the most grayscale values ​​in the grain (the proportion of the grayscale mode of the original image), median_value represents the median grayscale of the original image, kurt represents the kurtosis of the original image, and skewness represents the skewness of the original image.

[0155] For features with high correlation, only one of them can be retained, because the information contained in the features with high correlation is repeated. Retaining too many variables with the same explanation in the model will make the model more complicated and affect the generalization ability of the model. Therefore, one of these highly correlated features can be retained based on ease of understanding and observation, and the unretained features can be directly deleted. For example, the entropy of the binary graph is highly correlated with the impurity ratio. Obviously, impurities are easy to observe and understand intuitively, so the impurity ratio feature is retained, and the entropy of the binary graph is deleted. The results obtained after screening are shown in Table 1.

[0156] Table 1

[0157]

[0158] Features selected using correlation analysis: brightness, original image entropy, impurity ratio, skewness, maximum original image grayscale value, minimum original image grayscale value, brightness_std_dev, background grayscale standard deviation.

[0159] Then the embedding method is used to screen the features, and the threshold is set using the random forest feature importance method to screen out the features whose contribution rate is greater than the threshold. The importance threshold is set to 0.05. The results are as follows: Fig.21As shown in the figure, the final screened features are: Brightness Std Dev, background grayscale standard deviation, impurity ratio, original image entropy, original image grayscale minimum value, skewness. These features are used as the features for the final model fitting.

[0160] 4. Model establishment and metallographic grade evaluation.

[0161] Reference Fig. 22 , the processed data set is divided into training set and test set according to a certain ratio, and the random forest model and extreme gradient boosting decision tree model of machine learning ensemble learning model are selected to classify the grains in the metallographic image and divide the grains into qualified grains and unqualified grains. Ensemble learning is to combine the predictions of multiple weak learners to build a more powerful or more accurate model (strong learner).

[0162] The basic models of random forest and extreme gradient boosting decision tree are both decision tree models. Random forest is a typical algorithm of ensemble learning bagging method. Its idea is to use Bootstrap with replacement based on resampling to sample N times from the original data set containing N data and form a decision tree of a self-generated training data set. Repeating S times will build a random forest model containing S decision trees. Finally, the prediction results of multiple decision trees are voted to calculate the final result. The training results of random forest model are shown in Table 2. It can be seen that the model accuracy rate reaches 94%, and the prediction effect is good.

[0163] Table 2

[0164]

[0165]

[0166] Among them: Precision measures the proportion of samples predicted by the model as positive that are actually positive; Recall measures the proportion of samples that are correctly predicted as positive by the model among all samples that are actually positive; F1-Score is the harmonic mean of precision and recall, which is used to weigh the two; Support refers to the number of samples in each category. In the above data, there are 99 samples in category 0, 55 samples in category 1, and a total of 154 samples. Accuracy is the ratio of the number of samples correctly predicted by the model to the total number of samples; Macro Avg is a simple average of the indicators of each category (such as precision, recall, and F1-score), without considering the difference in support (i.e., the number of samples); Weighted Avg takes into account the support (i.e., the number of samples) of each category and gives higher weights to categories with better performance.

[0167] Another algorithm of ensemble learning is the boosting algorithm, which focuses on the samples that were misclassified by the previous model by continuously training the model, and gradually enhances the prediction ability of the model. The extreme gradient boosting decision tree (XGB) gradually builds models by minimizing the loss function. Each new model tries to correct the errors of the previous model. Its loss function not only calculates its own loss, but also adds a regularization term, which can effectively prevent the model from overfitting. The training results of the extreme gradient boosting decision tree are shown in Table 3. It can be seen that the model accuracy has reached 95%, and the prediction effect is good.

[0168] Table 3

[0169]

[0170] The two classifiers, XGBoost and random forest, are integrated to improve the prediction accuracy and model performance. A voting mechanism is used to determine the final classification result, that is, each classifier predicts the sample, and the final result is determined by the prediction results of all classifiers. The training results of the integrated model are shown in Table 4. It can be seen that the model accuracy has reached 96%, and the prediction accuracy has been further improved.

[0171] Table 4

[0172]

[0173] 5. Model verification.

[0174] Use two new metallographic images to test the model. Fig.23 The segmentation results are shown in Fig.24 As shown. The original image of the metallographic phase 2 is as follows Fig.25 , the segmentation results are as follows Fig.26 shown.

[0175] For the metallographic phase 1, the above method was used to divide the grains, calculate the characteristics, and use the model to predict the results. The qualified grains are [9, 14, 18, 33, 34, 35, 36], and the qualified grains account for 12.744%. Observing the picture, most of the grains are not clear, and the prediction results are relatively consistent.

[0176] For test metallographic phase 2, the above method is used to divide the grains, calculate the characteristics, and use the model to predict the results. The qualified grains are [2,3,4,5,10,11,13,14,16,17,20,21,25,29,30,31,33,36,38,39,41,42], and the qualified grains account for 57.791%. Observing the picture, the prediction result is good.

[0177] In summary, the embodiments of the present application have at least the following beneficial effects:

[0178] 1. Improve detection accuracy and consistency: Through computer vision technology, the system can automatically capture and analyze the features in the metallographic image to achieve objective quantification of the detection results. This eliminates the errors caused by subjective judgment in traditional manual detection, ensures the consistency and high accuracy of the detection results, and helps to accurately identify the glossiness and microstructural defects of the material surface.

[0179] 2. Improve detection efficiency: The automated detection process shortens manual operation time and improves detection efficiency, which means faster production speed and higher production capacity, helping to reduce production costs and quickly respond to market demand.

[0180] 3. Promote the standardization of quality management: Establish a unified quantitative standard to make the metallographic rating results comparable. This will help standardize the quality management within the enterprise.

[0181] See also Fig. 27 The embodiment of the present application also provides an aluminum profile metallographic rating system, which can implement the above-mentioned aluminum profile metallographic rating method, and the system includes:

[0182] The first module 101 is used to perform data preprocessing on the original metallographic image of the aluminum profile to obtain a second metallographic image;

[0183] The second module 102 is used to perform grain region division and image processing on the second metallographic image to obtain a first impurity image and a grain boundary image; wherein the grain boundary image includes a plurality of connected regions representing grains;

[0184] The third module 103 is used to obtain a first background image by subtracting the first impurity image from the original metallographic image;

[0185] A fourth module 104 is used to extract features from the original metallographic image, the first impurity image and the first background image according to the connected area of ​​the grain boundary image to obtain a plurality of first features;

[0186] The fifth module 105 is used to perform correlation analysis on the first feature to obtain a correlation feature;

[0187] The sixth module 106 is used to classify and predict the grains according to the correlation characteristics through a pre-trained machine learning model to obtain the metallographic level of the aluminum profile. For this system, the system operation interface can be designed as follows Fig.28 As shown, you can directly obtain image files or folders from the folder for processing, analyze and display the corresponding rating results and grain classification results with one click, and display specific characteristic parameters with a high degree of visualization.

[0188] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0189] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned aluminum profile metallographic rating method when executing the computer program. The electronic device can be any smart terminal including a tablet computer, a car computer, etc.

[0190] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0191] See also Fig.29 , Fig.29 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0192] The processor 201 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0193] The memory 202 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 202 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 202, and the processor 201 calls and executes a method for metallographic grading of aluminum profiles in the embodiment of this application;

[0194] Input / output interface 203, used to implement information input and output;

[0195] The communication interface 204 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0196] Bus 205 , which transmits information between various components of the device (e.g., processor 201 , memory 202 , input / output interface 203 , and communication interface 204 );

[0197] The processor 201 , the memory 202 , the input / output interface 203 and the communication interface 204 are connected to each other in communication within the device via the bus 205 .

[0198] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned aluminum profile metallographic grading method is implemented.

[0199] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0200] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0201] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0202] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0203] The system embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0204] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0205] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0206] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0207] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0208] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0209] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0210] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0211] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A method for metallographic grading of aluminum profiles, characterized in that: The following steps are involved: Performing data preprocessing on the original metallographic image of the aluminum profile to obtain a second metallographic image; Performing grain region division and image processing on the second metallographic image to obtain a first impurity image and a grain boundary image; wherein the grain boundary image includes a plurality of connected regions representing grains; Subtracting the first impurity image from the original metallographic image to obtain a first background image; According to the connected area of ​​the grain boundary image, feature extraction is performed on the original metallographic image, the first impurity image and the first background image to obtain a plurality of first features; Performing correlation analysis on the first feature to obtain a correlation feature; According to the correlation characteristics, the grains are classified and predicted using a pre-trained machine learning model to obtain the metallographic grade of the aluminum profile.

2. The method according to claim 1, characterized in that The data preprocessing includes at least one of contrast-limited adaptive histogram image equalization processing, median filtering processing, and bilateral filtering processing.

3. The method according to claim 1, characterized in that The step of dividing the second metallographic image into grain regions and performing image processing to obtain a first impurity image and a grain boundary image comprises the following steps: Performing a binarization process of an adaptive threshold on the second metallographic image to obtain a first impurity image reflecting impurity characteristics and a binarized image retaining grain characteristics; The binary image is processed alternately by using connected domain analysis and morphological operation to obtain a grain boundary image containing connected regions.

4. The method according to claim 1, characterized in that The method further comprises extracting features from the original metallographic image, the first impurity image and the first background image according to the connected area of ​​the grain boundary image to obtain a plurality of first features, including: Overlaying the connected area of ​​the grain boundary image onto the original metallographic image, the first impurity image and the first background image; According to the connected areas, image features corresponding to each of the connected areas on the original metallographic image are extracted to obtain metallographic image features; wherein the metallographic image features include the original image grayscale average value, the original image grayscale standard deviation, the original image grayscale mode, the original image grayscale mode proportion, the original image entropy value, the original image grayscale maximum value, the original image grayscale minimum value, the original image grayscale median, the original image grayscale value of more than 180 pixels, the original image skewness and the original image kurtosis; According to the connected regions, image features corresponding to each of the connected regions on the first impurity image are extracted to obtain impurity image features; the impurity image features include impurity proportions and impurity image entropy values; According to the connected regions, image features corresponding to each of the connected regions on the first background image are extracted to obtain background image features; wherein the background image features include a background grayscale average value and a background grayscale standard deviation; Extracting the area and perimeter of each grain in the grain boundary image; The metallographic image feature, the impurity image feature, the background image feature, the area and the perimeter are used as first features.

5. The method according to claim 1, characterized in that The step of performing correlation analysis on the first feature to obtain a correlation feature comprises the following steps: Calculating correlations between the first features in pairs; The first features are screened according to the correlation to obtain correlation features; wherein the screening process includes retaining one of the first features whose correlation is higher than a preset threshold.

6. The method according to claim 1, characterized in that The method of predicting the metallographic level by a pre-trained machine learning model according to the correlation feature to obtain the metallographic level includes the following steps: Using a plurality of classifiers in a pre-trained random forest model, classify the grains in the original metallographic image according to the correlation features to obtain a plurality of classification results; Voting on the classification results to obtain a final grain category; According to the grain type, the metallographic grade of the aluminum profile is obtained.

7. The method according to claim 1, characterized in that The training steps of the machine learning model include: Filter qualified and unqualified grain data from multiple original metallographic images to obtain a data set; Dividing a training set from the data set according to a preset ratio; Constructing a machine learning model to perform grain division on the training set to obtain training results; The training results are corrected by an extreme gradient boosting decision tree to minimize the loss function and obtain a pre-trained machine learning model.

8. A metallographic grading system for aluminum profiles, characterized in that: include: The first module is used to perform data preprocessing on the original metallographic image of the aluminum profile to obtain a second metallographic image; The second module is used to perform grain region division and image processing on the second metallographic image to obtain a first impurity image and a grain boundary image; wherein the grain boundary image includes a plurality of connected regions representing grains; A third module is used to obtain a first background image by subtracting the first impurity image from the original metallographic image; A fourth module is used to extract features from the original metallographic image, the first impurity image and the first background image according to the connected area of ​​the grain boundary image to obtain a plurality of first features; A fifth module is used to perform correlation analysis on the first feature to obtain a correlation feature; The sixth module is used to classify and predict the grains according to the correlation characteristics through a pre-trained machine learning model to obtain the metallographic grade of the aluminum profile.

9. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 7.

10. A computer storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to implement the method according to any one of claims 1 to 7 when executed by the processor.

Images