Intelligent metallographic structure analysis method and system based on feature fusion and active learning
By employing feature fusion and active learning methods, combined with lightweight machine learning and user-interactive annotation, the high cost, low efficiency, and low accuracy problems of metallographic analysis are solved. This enables automated metallographic analysis that is low-cost, high-efficiency, and high-accuracy, and is applicable to fields such as aerospace, machinery manufacturing, and transportation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HUAZHONG UNIV OF SCI & TECH
- Filing Date
- 2026-04-13
- Publication Date
- 2026-07-31
AI Technical Summary
Existing metallographic analysis techniques are costly, inefficient, and lack precision, failing to meet the rapid testing needs of large batches of samples in industrial settings. Furthermore, the measurement results rely on the personal experience and subjective judgment of technicians, making it difficult to establish an objective and standardized quality evaluation system.
A feature fusion and active learning approach is adopted. High-resolution images are preprocessed to extract texture, edge and multi-scale spatial features, construct high-dimensional feature vectors, and train them with lightweight machine learning classifiers such as random forests or support vector machines. An active learning loop is formed through user interactive annotation to achieve automated analysis of metallographic parameters.
It achieves low-cost, high-efficiency, and high-precision automated analysis of metallographic structures, adapts to various scenarios, has excellent cross-scenario generalization ability and model interpretability, is suitable for deployment on ordinary computers, and is applicable to fields such as aerospace, machinery manufacturing, and transportation.
Smart Images
Figure CN122493040A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of computer vision technology, and more specifically, relates to a method and system for intelligent analysis of metallographic structures based on feature fusion and active learning. Background Technology
[0002] Metallographic analysis is a core method for evaluating the microstructure of metallic materials, determining material properties, and assessing processing quality. It is widely used in aerospace, machinery manufacturing, and transportation industries. However, existing metallographic analysis procedures mainly rely on manual operation. Technicians must observe under a microscope, select specific areas within the field of view, and manually calculate key parameters such as grain size, secondary dendrite arm spacing (SDAS), and phase content using the truncation method or comparative method. This traditional approach is not only inefficient and time-consuming, failing to meet the rapid testing needs of large batches of samples in industrial settings, but also highly dependent on the personal experience and subjective judgment of technicians, resulting in poor data consistency among different personnel and making it difficult to establish an objective and standardized quality evaluation system.
[0003] With the advent of computer vision technology, traditional image processing techniques based on threshold segmentation, edge detection (such as Canny and Sobel operators), and watershed algorithms were once applied to the automation of metallographic analysis. These methods are computationally fast, but their robustness and generalization ability are poor, and they have extremely demanding requirements for the image acquisition environment. In actual industrial production, due to differences in sample preparation processes, metallographic images are often accompanied by interference such as uneven illumination, scratch noise, and varying degrees of corrosion. This causes traditional algorithms based on fixed thresholds or gradients to easily fail, resulting in fragmented segmentation results or numerous artifacts. They are also unable to handle non-ideal images with complex and varied tissue morphologies, and cannot meet the high-precision requirements of engineering applications.
[0004] In recent years, while deep learning technologies, represented by Convolutional Neural Networks (CNNs) (such as U-Net and Mask R-CNN), have achieved significant breakthroughs in segmentation accuracy, they have also introduced new technological barriers. First, deep learning models are typically "data-hungry" algorithms; training a highly usable model requires massive amounts of high-quality datasets with pixel-level precise annotations. However, annotating metallographic images requires a high level of expertise, making data acquisition extremely costly. Second, deep learning models have high demands on computing hardware resources, resulting in expensive deployment and maintenance. Furthermore, these models often exhibit "black box" characteristics, lacking interpretability. When misclassifications occur, engineers struggle to make intuitive corrections and optimizations, limiting their widespread application in small and medium-sized enterprises and real-time detection scenarios.
[0005] Therefore, how to overcome the shortcomings of existing technologies and achieve rapid, accurate, and low-cost automated analysis of metallographic structures is a technical problem that urgently needs to be solved. Summary of the Invention
[0006] In view of the shortcomings of the prior art, the purpose of this application is to solve the problems of high cost, low efficiency and low accuracy in the metallographic analysis of the prior art, and to realize the rapid, accurate and low-cost automated analysis of metallographic structures.
[0007] To achieve the above objectives, firstly, this application provides a method for intelligent analysis of metallographic structures based on feature fusion and active learning, comprising: Acquire a high-resolution image of the metallographic sample and preprocess the high-resolution image; Feature extraction is performed on the preprocessed image to obtain pixel-level features. The pixel-level features are then concatenated, fused, and standardized to construct a high-dimensional feature vector. The pixel-level features include texture features, edge features, and multi-scale spatial features. Receive user input and, in response to the user input, label different phase categories; use the high-dimensional feature vector as sample input and the phase category as sample label to train a lightweight machine learning classifier to obtain a trained and optimized image classification model; The image classification model is used to analyze the image to be analyzed, and a binary mask of the image to be analyzed is obtained. Then, a selected strategy is used to calculate the metallographic parameters based on the binary mask.
[0008] Optionally, the step of acquiring a high-resolution image of the metallographic sample and preprocessing the high-resolution image includes: Based on the imaging quality characteristics of the high-resolution image, image processing operations are selected and sorted from a preset algorithm library to construct a dynamically configured preprocessing pipeline; The high-resolution image is converted into a grayscale image and preprocessed through the preprocessing pipeline, which includes background correction.
[0009] Optionally, the background correction process specifically includes: The background illumination distribution map of the high-resolution image is estimated using large-scale median filtering; The corrected image is calculated using the following formula:
[0010] in, For the corrected image, This is the grayscale image corresponding to the high-resolution image. This is a background illumination distribution map. To prevent the minimum value of division by zero, This represents the mean value of the background illumination distribution map.
[0011] Optionally, feature extraction is performed on the preprocessed image to obtain pixel-level features. These pixel-level features are then concatenated, fused, and standardized to construct a high-dimensional feature vector, including: The local texture invariants of the corrected image are extracted using a local binary model to obtain the texture features; The first-order gradients in the X and Y directions of the corrected image are extracted using the Sobel operator, and the second-order edge gradients are extracted using the Laplacian operator. The edge features are obtained based on the extracted first-order and second-order gradients. Multi-scale spatial features are generated using Gaussian blurring with different kernel sizes; The texture features, edge features, and multi-scale spatial features are fused to obtain the pixel-level features, and the fused pixel-level features are normalized using the Z-Score normalization method to obtain the high-dimensional feature vector.
[0012] Optionally, the lightweight machine learning classifier is built based on the random forest algorithm or the support vector machine algorithm; When using the random forest algorithm, multiple decision trees are constructed through bootstrap sampling and feature random selection, and the prediction result is determined by the majority vote of the decision trees. When using the support vector machine algorithm, a radial basis function kernel is selected, and a one-to-many strategy is adopted to solve the multi-class segmentation task.
[0013] Optionally, the training process of the lightweight machine learning classifier further includes: When the model makes a misjudgment, it receives supplementary annotations from the user for the misjudged areas. Incremental data of the sample labels are obtained based on the supplementary annotation input, and the model is retrained based on the incremental data to form an active learning loop.
[0014] Optionally, the metallographic parameters include phase area ratio and secondary dendrite arm spacing; The method for calculating the phase area ratio includes: The total number of pixels belonging to the c-th type of organization in the binarized mask and the total number of effective pixels in the entire image are counted, and the phase area ratio is calculated based on the total number of pixels and the total number of effective pixels in the entire image; The method for calculating the secondary dendrite arm spacing includes at least one of the following: Morphological refinement is performed on the segmented dendritic region to obtain a single-pixel skeleton, the total dendrite area and estimated length are obtained, and the spacing between the secondary dendrite arms is calculated based on the area of the connected region and the skeleton length. Horizontal and vertical scan lines are automatically generated on the binarized mask. The number of intersections between the cut-off lines and the dendritic phase and the length of the cut-off lines are counted. The spacing between the secondary dendrite arms is calculated based on the number of intersections and the length of the cut-off lines. The grayscale intensity profile is extracted from the measurement line path defined by the user, and the number of dendrite arms is identified by smoothing and signal peak detection algorithm. The secondary dendrite arm spacing is calculated based on the measurement line length and the number of dendrite arms.
[0015] Optionally, it also includes: Component analysis is performed on the segmented connected domains to calculate the area, perimeter, roundness, and aspect ratio of the independent phases; An image stitching algorithm is used to stitch multiple local field-of-view images into a panoramic image, and macroscopic statistics are performed based on the panoramic image; the local field-of-view images are obtained by segmenting the high-resolution image.
[0016] Optionally, the image classification model implements image analysis through multi-process parallel processing; If the system is detected to have a graphics processing unit (GPU) hardware environment, it will automatically switch to the CUDA-based parallel computing module to perform convolution operations in feature computation. If no GPU hardware environment is detected in the system, the CPU multi-process pool will be used for computation.
[0017] Secondly, this application also provides a metallographic structure intelligent analysis method based on feature fusion and active learning, including: An image preprocessing module is used to acquire high-resolution images of metallographic samples and to preprocess the high-resolution images. The feature extraction module is used to extract features from the preprocessed image to obtain pixel-level features, and to perform concatenation, fusion and standardization of the pixel-level features to construct a high-dimensional feature vector; the pixel-level features include: texture features, edge features and multi-scale spatial features. The model training module is used to receive user input and, in response to the user input, label different phase categories; it uses the high-dimensional feature vector as sample input and the phase category as sample label to train a lightweight machine learning classifier to obtain a trained and optimized image classification model. The image analysis module is used to analyze the image to be analyzed using the image classification model, obtain a binary mask of the image to be analyzed, and perform calculations based on the binary mask using a selected strategy to obtain metallographic parameters.
[0018] Thirdly, this application provides an electronic device, comprising: at least one memory for storing a program; and at least one processor for executing the program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to execute the method described in the first aspect or any possible implementation thereof.
[0019] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible implementation thereof.
[0020] Fifthly, this application provides a computer program product that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible implementation thereof.
[0021] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.
[0022] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art: (1) By combining lightweight machine learning algorithms such as random forest / SVM with an interactive active learning mechanism, this application does not rely on massive labeled data and expensive computing power. It allows users to quickly start model training with a small amount of sparse labels and enables continuous iterative optimization of the model through interactive correction of misjudged regions. This application not only has excellent cross-scene generalization ability and can adapt to changes in lighting and materials, but also has extremely high model interpretability and low-cost deployment advantages, providing a practical and feasible technical path for achieving low-cost, high-efficiency, and high-precision automated analysis of metallographic structures throughout the entire process.
[0023] (2) This application can realize full-process intelligentization and improve efficiency and accuracy: In view of the problems that metallographic analysis relies on manual labor, has a long process and strong subjectivity, a full-process intelligent analysis system is constructed from image import, preprocessing, segmentation and annotation to measurement and statistics. The key parameters are accurately calculated through a multi-modal quantization engine (skeleton method, cut-off method, etc.).
[0024] (3) This application has strong generalization ability and is suitable for multiple scenarios. By customizing the preprocessing pipeline and interactive annotation mechanism, it effectively solves the problem of insufficient generalization ability of general AI models when facing different materials (such as aluminum alloy and steel) and different imaging conditions (uneven illumination and noise), and significantly improves the robustness of the system.
[0025] (4) This application can meet the low computing power requirements and is easy to deploy: This application uses classic machine learning algorithms such as random forest / SVM combined with multi-process / GPU acceleration. Compared with deep learning solutions, the requirements for hardware computing power and training data are greatly reduced, and the model is more interpretable. It is more suitable for deployment on ordinary computers in industrial sites and has extremely high practical value. Attached Figure Description
[0026] Figure 1 This is an overall flowchart of the metallographic structure intelligent analysis method according to the embodiments of this application; Figure 2 This is a schematic diagram of the kernel structure of the random forest machine learning model used in the embodiments of this application; Figure 3 This is a schematic diagram of the kernel structure of the support vector machine machine learning model used in the embodiments of this application; Figure 4 A diagram showing the secondary dendrite arm spacing measured using the cross-sectional peak detection method in this application; Figure 5 This is a diagram of the secondary dendrite arm spacing measured directly using the global intercept method in this application. Figure 6 This is a diagram of the secondary dendrite arm spacing measured using the skeleton method after the application was processed in a production line. Figure 7 This is a schematic diagram of the image segmentation results of this application; Figure 8 This is the second flowchart of the intelligent metallographic structure analysis method based on feature fusion and active learning provided in the embodiments of this application; Figure 9 This is a schematic diagram of the structure of the intelligent metallographic structure analysis system based on feature fusion and active learning provided in the embodiments of this application; Figure 10 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation
[0027] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0028] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.
[0029] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.
[0030] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.
[0031] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.
[0032] The embodiments of this application are described below with reference to the accompanying drawings.
[0033] Reference Figure 1 This application provides a method for intelligent analysis of metallographic structures based on feature fusion and active learning, including: S101. Acquire a high-resolution image of the metallographic sample and preprocess the high-resolution image; S102. Extract features from the preprocessed image to obtain pixel-level features. Concatenate, fuse, and standardize the pixel-level features to construct a high-dimensional feature vector. The pixel-level features include: texture features, edge features, and multi-scale spatial features. S103. Receive user input, and in response to the user input, label different phase categories; use the high-dimensional feature vector as sample input and the phase category as sample label to train a lightweight machine learning classifier to obtain a trained and optimized image classification model; When the model makes a misjudgment, it receives supplementary annotations from the user for the misjudged region; based on the supplementary annotations, it obtains incremental data of the sample labels; and retrains the model based on the incremental data to form an active learning loop. S104. The image to be analyzed is analyzed using the image classification model to obtain a binary mask of the image to be analyzed, and metallographic parameters are obtained by using a selected strategy based on the binary mask.
[0034] Specifically, firstly, before step S101, the test sample needs to be prepared: select representative parts and cut them into preset sizes, grind them level by level with sandpaper from coarse to fine, and obtain a mirror surface through mechanical polishing; after polishing, clean them, use an etchant (such as 0.5% HF aqueous solution) to briefly etch them to expose the tissue, and finally clean and dry them.
[0035] High-resolution images of metallographic samples are acquired via the S101. These images may be in color or grayscale format, with high pixel resolution, clearly revealing the detailed features of the microstructure. After image acquisition, appropriate image processing operations are dynamically selected and ordered from a pre-defined algorithm library based on the image's imaging quality characteristics, constructing an adaptive preprocessing pipeline.
[0036] Preprocessing includes background correction, specifically employing a background flattening algorithm based on division correction. This algorithm uses large-scale median filtering to estimate the background illumination distribution, performs pixel-level division between the original image and the background image, and normalizes the results to eliminate the vignetting effect caused by uneven microscope light sources.
[0037] It should be noted that the preprocessing in this embodiment also includes background flattening, grayscale conversion, Gaussian blurring, median filtering, brightness / contrast adjustment, sharpening, histogram equalization, CLAHE (adaptive equalization), binarization thresholding, Otsu automatic thresholding, erosion, dilation, opening and closing operations.
[0038] Secondly, adaptive feature fusion based on multi-scale perception is implemented through S102. Pixel-level features are extracted from the preprocessed image to construct a high-dimensional feature vector. Feature fusion includes: texture features: local texture invariants are extracted using a local binary mode to capture the microscopic texture differences between grains and the matrix; edge features: first-order and second-order edge gradient information is extracted using the Sobel operator (X / Y direction) and the Laplacian operator; multi-scale spatial features: multi-scale spatial information is generated through Gaussian blurring with different kernel sizes (5×5, 11×11). The above features are concatenated, fused, and standardized to form a pixel feature descriptor with strong discriminative power.
[0039] Furthermore, active learning model training is achieved through S103. Lightweight machine learning models such as Random Forest or Support Vector Machine (SVM) are used as classifiers. A small number of sparse lines are drawn on the image to label dendrites, precipitates, and the background. The system automatically extracts the fused features of the pixels in the labeled areas for model training.
[0040] Specifically, it includes the following processes: Sparse annotation: Users use a pen to mark a small number of red lines (representing Al) and green lines (representing eutectic Si) on the image.
[0041] Model Training: If Random Forest (RF) is selected: Set the number of decision trees (n_estimators=100), use Bootstrap sampling to build multiple decision trees, and determine the pixel category through majority voting. If Support Vector Machine (SVM) is selected: Use the RBF kernel function, the system automatically starts grid search, finds the optimal hyperparameter combination within the parameter range C∈[0.1, 100] and γ∈[0.001, 1], and uses 5-fold cross-validation for evaluation.
[0042] Active learning loop: After the initial training, the system automatically segments the current image. If a misclassification is found in a certain area (misclassifying tiny Si particles as background), the user can directly add a stroke to the misclassified area. The system quickly retrains the model based on incrementally labeled data, forming a closed loop of "labeling-prediction-correction-retraining". Engineering-grade accuracy can usually be achieved in 2-3 iterations.
[0043] When the model makes a misjudgment, it receives supplementary annotation input from the user for the misjudged area; based on the supplementary annotation input, it obtains incremental data of the sample label, and retrains the model based on the incremental data to form an active learning loop.
[0044] If the model makes a misjudgment, the misjudged area can be supplemented with annotations. The system can then be quickly retrained based on incremental data, forming an active learning loop of "annotation-training-feedback-correction" to achieve high-precision segmentation with small sample sizes.
[0045] It should be noted that in this embodiment, the system provides an interactive annotation interface, which receives sparse line annotations made by the user on the image using the drawing tool. The user can use lines of different colors to mark different phase categories.
[0046] For each pixel covered by the user-annotated line, the system extracts its standardized high-dimensional feature vector defined in step S102, and uses the user's annotation as the class label for that pixel, together forming a training sample set. Based on these samples, a lightweight machine learning classifier is trained, specifically using either a random forest algorithm or a support vector machine algorithm: when using a random forest, multiple decision trees are constructed through bootstrapping and random feature selection, and the final prediction result is determined by the majority vote of all decision trees; when using a support vector machine, a radial basis function kernel is selected and a one-to-many strategy is adopted to solve the multi-class segmentation problem.
[0047] After training, the model is applied to the current image for prediction. If the model is found to misclassify certain areas, the user can add annotations to the misclassified areas. The system will merge the newly added annotation data with the original data and perform incremental retraining on the model to form an active learning loop, continuously improving the segmentation accuracy of the model.
[0048] Finally, the image classification model trained and optimized in S103 is applied to a batch of metallographic images to be analyzed. A standardized high-dimensional feature vector, identical to that in step S102, is extracted from each pixel in the image to be analyzed. This vector is then input into the model for inference and prediction to determine the phase category to which each pixel belongs, thereby generating a binarized mask with the same size as the original image.
[0049] Based on the mask, the key metallographic parameter secondary dendrite arm spacing is calculated using a selected strategy: specifically, the global skeleton method, the global cut-off method, and the cross-sectional peak detection method can be selected, and the secondary dendrite arm spacing SDAS is finally output.
[0050] Optionally, the step of acquiring a high-resolution image of the metallographic sample and preprocessing the high-resolution image includes: Based on the imaging quality characteristics of the high-resolution image, image processing operations are selected and sorted from a preset algorithm library to construct a dynamically configured preprocessing pipeline; The high-resolution image is converted into a grayscale image and preprocessed through the preprocessing pipeline, which includes background correction.
[0051] Furthermore, the background correction process specifically includes: The background illumination distribution map of the high-resolution image is estimated using large-scale median filtering; The corrected image is calculated using the following formula:
[0052] in, For the corrected image, This is the grayscale image corresponding to the high-resolution image. This is a background illumination distribution map. To prevent the minimum value of division by zero, This represents the mean value of the background illumination distribution map.
[0053] Optionally, feature extraction is performed on the preprocessed image to obtain pixel-level features. These pixel-level features are then concatenated, fused, and standardized to construct a high-dimensional feature vector, including: The local texture invariants of the corrected image are extracted using a local binary model to obtain the texture features; The first-order gradients in the X and Y directions of the corrected image are extracted using the Sobel operator, and the second-order edge gradients are extracted using the Laplacian operator. The edge features are obtained based on the extracted first-order and second-order gradients. Multi-scale spatial features are generated using Gaussian blurring with different kernel sizes; The texture features, edge features, and multi-scale spatial features are fused to obtain the pixel-level features, and the fused pixel-level features are normalized using the Z-Score normalization method to obtain the high-dimensional feature vector.
[0054] Specifically, the feature extraction is implemented as follows: The system extracts pixel-level features from the preprocessed image and constructs a high-dimensional feature descriptor. Feature fusion vector. Specifically, it includes: Texture features Local Binary Model (LBP) is used, with the parameter set to the radius. R =1, sampling point P =8, mode is uniform, effectively capturing the micro-texture differences between the Al matrix and eutectic Si particles.
[0055] Edge features ( The Sobel operator is used to extract the first-order gradients in the X and Y directions, and the Laplacian operator is used to extract the second-order gradients, thus enhancing the grain boundary information.
[0056] Multi-scale spatial features Gaussian blur features with kernel sizes of 5×5 and 11×11 were calculated respectively, incorporating spatial scale information. Z-Score normalization was employed. The eigenvectors are normalized to eliminate dimensional differences.
[0057] Optionally, the lightweight machine learning classifier is built based on the random forest algorithm or the support vector machine algorithm; When using the random forest algorithm, multiple decision trees are constructed through bootstrap sampling and feature random selection, and the prediction result is determined by the majority vote of the decision trees. When using the support vector machine algorithm, a radial basis function kernel is selected, and a one-to-many strategy is adopted to solve the multi-class segmentation task.
[0058] Specifically, when using the random forest algorithm, the training process includes: setting the number of decision trees (n_estimators), the maximum tree depth (max_depth), and the maximum number of features to consider when splitting nodes (max_features); constructing decision trees through bootstrap sampling and a random feature selection mechanism; determining the prediction result by a majority vote of all decision trees; and evaluating the contribution of different image features to the segmentation result by calculating feature importance.
[0059] Reference Figure 2 , Figure 2This is a schematic diagram of the kernel structure of the random forest machine learning model used in this application embodiment; the figure shows the training data based on pixel features and label Y, which, after bootstrap sampling and feature subset selection, constructs N decision trees from decision tree 1 to decision tree N; each decision tree outputs category A (dendritic) or category B (matrix), and the prediction results of all decision trees are summarized by majority voting to obtain the final pixel prediction. At the same time, new input pixel feature vectors can also be used to complete pixel prediction through this process.
[0060] Reference Figure 3 , Figure 3 This is a schematic diagram of the kernel structure of the support vector machine machine learning model used in the embodiments of this application. In the figure, the left side is a low-dimensional, non-linear input space, in which pixel data is divided into two categories: dendrites (red dots) and matrix (blue crosses), and the data is distributed non-linearly. Through RBF kernel mapping, these data are mapped to the high-dimensional linear feature space on the right. In this space, the two types of data can be linearly divided by the optimal separating hyperplane. The support vectors and margins are marked in the figure, thereby achieving linear separability of pixel data categories.
[0061] As a further preferred embodiment of this application, in step S3, when the support vector machine (SVM) algorithm is used, the training process includes: selecting a radial basis function (RBF) kernel to handle nonlinear classification problems; adopting a "one-vs-Rest" strategy to solve multi-class segmentation tasks; and automatically optimizing the regularization parameter (C) and kernel parameter (γ) through grid search combined with cross-validation.
[0062] Optionally, the metallographic parameters include phase area ratio and secondary dendrite arm spacing; The method for calculating the phase area ratio includes: The total number of pixels belonging to the c-th type of organization in the binarized mask and the total number of effective pixels in the entire image are counted, and the phase area ratio is calculated based on the total number of pixels and the total number of effective pixels in the entire image; The method for calculating the secondary dendrite arm spacing includes at least one of the following: Morphological refinement is performed on the segmented dendritic region to obtain a single-pixel skeleton, the total dendrite area and estimated length are obtained, and the spacing between the secondary dendrite arms is calculated based on the area of the connected region and the skeleton length. Horizontal and vertical scan lines are automatically generated on the binarized mask. The number of intersections between the cut-off lines and the dendritic phase and the length of the cut-off lines are counted. The spacing between the secondary dendrite arms is calculated based on the number of intersections and the length of the cut-off lines. The grayscale intensity profile is extracted from the measurement line path defined by the user, and the number of dendrite arms is identified by smoothing and signal peak detection algorithm. The secondary dendrite arm spacing is calculated based on the measurement line length and the number of dendrite arms.
[0063] Specifically, phase area ratio: N is the total number of pixels N belonging to the c-th type of organization in the statistical mask. c With the number of effective pixels in the whole image ,calculate .
[0064] Secondary Dendrite Arm Spacing (SDAS) Calculation (Three Strategies): Strategy A: Global Skeleton Method. Morphological refinement of the Al dendrite region yields a single-pixel skeleton. Calculate the total dendrite area A and the total skeleton length. According to the formula Calculate the global average spacing. Strategy B: Global intercept method. Automatically generate scan meshes in the horizontal and vertical directions. Count the number P of intersections between the scan line and the dendrite interface and the total intercept length. ,calculate Strategy C: Profile Peak Detection Method. Draw lines in the user's area of interest and extract the grayscale profile. After Gaussian smoothing and detrending, use the scipy.signal.find_peaks algorithm to identify the number of peaks N, and calculate...
[0065] Reference Figures 4-6 , Figure 4 A diagram showing the secondary dendrite arm spacing measured using the profile peak detection method; Figure 5 This is a diagram showing the spacing between secondary dendrite arms measured directly using the global intercept method. Figure 6 This is a diagram showing the secondary dendrite arm spacing measured using the skeleton method after automated processing.
[0066] Grain size and morphological statistics: Analysis of connected regions to calculate the equivalent circle diameter of each individual grain. Roundness and aspect ratio.
[0067] Optionally, it also includes: The binarized mask is segmented to obtain the segmented connected components; Component analysis is performed on the segmented connected domains to calculate the area, perimeter, roundness, and aspect ratio of the independent phases; An image stitching algorithm is used to stitch multiple local field-of-view images into a panoramic image, and macroscopic statistics are performed based on the panoramic image; the local field-of-view images are obtained by segmenting the high-resolution image.
[0068] It should be noted that in this embodiment, a connected component labeling algorithm is applied to the binarized mask to determine the relationship between each pixel and its surrounding neighboring pixels: if the values of neighboring pixels are the same (or the difference is small), they are determined to belong to the same connected component; conversely, if the values of neighboring pixels are different (or the difference is large), they are determined to belong to different connected components. Through this process, all spatially continuous target pixels are merged into independent connected components.
[0069] For a large number of images within a folder, the system establishes a multi-process pool (ProcessPool). Each process is independently responsible for the "feature extraction-inference-statistics" process for one image. If the GPU is detected, convolution operations such as Gaussian blur are automatically migrated to the CUDA core for execution.
[0070] Image stitching: For large field-of-view requirements, an S-shaped or Z-shaped path is used to stitch the local field-of-view segmentation results into a panoramic distribution map based on the positional overlap rate. (Refer to...) Figure 7 , Figure 7 This is a schematic diagram of the image segmentation results according to an embodiment of this application; Output results: The final output is a CSV file containing detailed data for each image and a summary of statistics for the entire project, providing a quantitative basis for optimizing the heat treatment process of A356 aluminum alloy.
[0071] Optionally, the image classification model implements image analysis through multi-process parallel processing; If the system is detected to have a graphics processing unit (GPU) hardware environment, it will automatically switch to the CUDA-based parallel computing module to perform convolution operations in feature computation. If no GPU hardware environment is detected in the system, the CPU multi-process pool will be used for computation.
[0072] Specifically, the hardware and software environment of this application embodiment is as follows: Graphics Processing Unit (GPU, supporting CUDA acceleration) is used to accelerate feature extraction and matrix operations. Storage and Display: High-speed solid-state drives are used to store massive amounts of metallographic images and feature data; high-resolution display devices are used to display metallographic details and analysis charts. Image Acquisition Equipment: Connects to an optical microscope with a motorized stage, supporting automatic image stitching.
[0073] The software architecture and environment are developed based on the Python language. The backend integrates the OpenCV image processing engine, the Scikit-learn machine learning library, and the Matplotlib visualization library. The frontend uses Pyside2 to build the desktop application interface, realizing the separation of algorithm and interaction.
[0074] Reference Figure 8 , Figure 8This is a complete flowchart of an embodiment of this application, including the following steps: S1: Acquire high-resolution images of metallographic samples and construct a dynamically configurable adaptive preprocessing pipeline. C1: Perform background flattening based on background estimation and division correction to complete image preprocessing. Next, we proceed to S2: the adaptive feature fusion stage based on multi-scale perception, which extracts texture features (LBP) through D1, edge features (Sobel / Laplacian) through D2, and multi-scale spatial high-frequency features through D3. Finally, we form a high-dimensional pixel feature descriptor through E. Then, S3 is executed: the initial training of the active learning model based on sparse annotations. First, F1 receives a small amount of sparse pen annotations (dendritic / phase / background) from the user, and then F2 trains a lightweight machine learning classifier RF / SVM to complete the model training. After that, the judgment node G is entered: Is there any misjudgment in the model segmentation result evaluation? (Human-computer interaction verification): If the result meets the requirements (no), it enters the iterative optimization stage; if there is a misjudgment (yes), H is executed: interactive correction: the user supplements the annotations for the misjudged areas, and after forming an active learning closed loop, the evaluation is carried out again. The iterative optimization phase first uses I: rapid retraining of the model based on incremental data, which can be either G: rapid retraining of the model based on incremental data or LG: basic batch and shared classifier. Then, S4: multi-process parallel processing and fully automatic segmentation is executed. First, J1: detects the GPU environment and automatically switches to CUDA to accelerate inference, and then K: generates binary segmentation masks for batch images. Next, we enter the S5 stage: automated quantitative calculation of multimodal metallographic parameters, simultaneously executing L1: global skeleton method to calculate SDAS area / skeleton length, L2: global intercept method to automatically count SDAS, L3: profile peak detection method for local precise measurement, and L4: calculation of phase content area ratio. Subsequently, S6: Morphological Statistics and Image Stitching Analysis was performed, which was completed through M1: Connectivity Analysis: Grain Size / Roundness / Aspect Ratio Statistics and M2: Panoramic Stitching of the Dam Field Based on Location and Weight Rate. Finally, N: Output Metallographic Quantitative Analysis Report and Panoramic Distribution Map. Reference Figure 9 This application also provides a method for intelligent analysis of metallographic structures based on feature fusion and active learning, including: Image preprocessing module 910 is used to acquire high-resolution images of metallographic samples and preprocess the high-resolution images. The feature extraction module 920 is used to extract features from the preprocessed image to obtain pixel-level features, and to perform concatenation, fusion and standardization of the pixel-level features to construct a high-dimensional feature vector; the pixel-level features include: texture features, edge features and multi-scale spatial features. The model training module 930 is used to receive user input, and in response to the user input, label different phase categories; use the high-dimensional feature vector as sample input and the phase category as sample label to train a lightweight machine learning classifier to obtain a trained and optimized image classification model; The image analysis module 940 is used to analyze the image to be analyzed using the image classification model, obtain a binary mask of the image to be analyzed, and perform calculations based on the binary mask using a selected strategy to obtain metallographic parameters.
[0075] Specifically, the system in this application embodiment includes the following functions: Metallographic image import and management functions: Supports importing single or batch metallographic images in BMP, JPG, PNG, and TIFF formats. Project management: Supports building analysis projects, packaging image paths, specific preprocessing parameter configurations, user annotation information, and measurement data into a unified JSON format project file for easy saving, loading, and backtracking.
[0076] Configurable preprocessing capabilities: Provides a visual preprocessing pipeline construction panel. Algorithm library: Integrates algorithms such as background flattening, grayscale conversion, Gaussian blur, median filtering, brightness / contrast adjustment, sharpening, histogram equalization, CLAHE (adaptive equalization), binarization thresholding, Otsu's automatic thresholding, erosion, dilation, opening, and closing operations. Pipeline mechanism: Users can drag and drop the above algorithms to construct custom processing sequences (e.g., background flattening → Gaussian blur → CLAHE enhancement) and apply them to all images in the project with one click, achieving standardized preprocessing for multiple images.
[0077] Intelligent segmentation and interactive annotation: Provides a "human-computer loop" annotation and segmentation function. Interactive tools: Offers a brush tool with customizable color and thickness. Supports custom labels (e.g., defining red for Al matrix, green for eutectic Si, and blue for defects / precipitates). Sparse annotation: Users do not need to fill the entire area; they only need to draw a few lines on the image (sparse annotation), and the system can extract features for training. For areas with model prediction deviations, it supports "incremental annotation" and "erasure," balancing intelligence and flexibility.
[0078] Multimodal Quantization Measurement and Analysis Module: Ruler Calibration: Provides an interactive ruler setting tool to convert pixel distance to physical distance. SDAS Calculation Engine: Integrates three automatic statistical methods: Skeleton Analysis: Suitable for dendritic structures with good connectivity. Cross-Section Method: Simulates traditional manual cross-sections, automatically generating mesh cross-sections for statistical analysis. Profile Peak Detection Method: Based on grayscale intensity peak identification, used for precise local measurements. General Measurement: Supports length measurement between any two points and polygon region measurement.
[0079] The model training and statistical reporting module includes two lightweight algorithm kernels: Random Forest and SVM (Support Vector Machine). Dual-mode statistics support both "single-image statistics" (generating detailed reports for a single image) and "batch statistics" (multi-process concurrent processing of all images within a folder). Visualization supports simultaneous display of the original image and segmentation mask; automatically generates phase content and supports exporting CSV data reports.
[0080] Optionally, the step of acquiring a high-resolution image of the metallographic sample and preprocessing the high-resolution image includes: Based on the imaging quality characteristics of the high-resolution image, image processing operations are selected and sorted from a preset algorithm library to construct a dynamically configured preprocessing pipeline; The high-resolution image is converted into a grayscale image and preprocessed through the preprocessing pipeline, which includes background correction.
[0081] Optionally, the background correction process specifically includes: The background illumination distribution map of the high-resolution image is estimated using large-scale median filtering; The corrected image is calculated using the following formula:
[0082] in, For the corrected image, This is the grayscale image corresponding to the high-resolution image. This is a background illumination distribution map. To prevent the minimum value of division by zero, This represents the mean value of the background illumination distribution map.
[0083] Optionally, feature extraction is performed on the preprocessed image to obtain pixel-level features. These pixel-level features are then concatenated, fused, and standardized to construct a high-dimensional feature vector, including: The local texture invariants of the corrected image are extracted using a local binary model to obtain the texture features; The first-order gradients in the X and Y directions of the corrected image are extracted using the Sobel operator, and the second-order edge gradients are extracted using the Laplacian operator. The edge features are obtained based on the extracted first-order and second-order gradients. Multi-scale spatial features are generated using Gaussian blurring with different kernel sizes; The texture features, edge features, and multi-scale spatial features are fused to obtain the pixel-level features, and the fused pixel-level features are normalized using the Z-Score normalization method to obtain the high-dimensional feature vector.
[0084] Optionally, the lightweight machine learning classifier is built based on the random forest algorithm or the support vector machine algorithm; When using the random forest algorithm, multiple decision trees are constructed through bootstrap sampling and feature random selection, and the prediction result is determined by the majority vote of the decision trees. When using the support vector machine algorithm, a radial basis function kernel is selected, and a one-to-many strategy is adopted to solve the multi-class segmentation task.
[0085] Optionally, the training process of the lightweight machine learning classifier further includes: When the model makes a misjudgment, it receives supplementary annotations from the user for the misjudged areas. Incremental data of the sample labels are obtained based on the supplementary annotation input, and the model is retrained based on the incremental data to form an active learning loop.
[0086] Optionally, the metallographic parameters include phase area ratio and secondary dendrite arm spacing; The method for calculating the phase area ratio includes: The total number of pixels belonging to the c-th type of organization in the binarized mask and the total number of effective pixels in the entire image are counted, and the phase area ratio is calculated based on the total number of pixels and the total number of effective pixels in the entire image; The method for calculating the secondary dendrite arm spacing includes at least one of the following: Morphological refinement is performed on the segmented dendritic region to obtain a single-pixel skeleton, the total dendrite area and estimated length are obtained, and the spacing between the secondary dendrite arms is calculated based on the area of the connected region and the skeleton length. Horizontal and vertical scan lines are automatically generated on the binarized mask. The number of intersections between the cut-off lines and the dendritic phase and the length of the cut-off lines are counted. The spacing between the secondary dendrite arms is calculated based on the number of intersections and the length of the cut-off lines. The grayscale intensity profile is extracted from the measurement line path defined by the user, and the number of dendrite arms is identified by smoothing and signal peak detection algorithm. The secondary dendrite arm spacing is calculated based on the measurement line length and the number of dendrite arms.
[0087] Optionally, it also includes a statistical analysis and splicing module for: Component analysis is performed on the segmented connected domains to calculate the area, perimeter, roundness, and aspect ratio of the independent phases; An image stitching algorithm is used to stitch multiple local field-of-view images into a panoramic image, and macroscopic statistics are performed based on the panoramic image; the local field-of-view images are obtained by segmenting the high-resolution image.
[0088] Optionally, the image classification model implements image analysis through multi-process parallel processing; If the system is detected to have a graphics processing unit (GPU) hardware environment, it will automatically switch to the CUDA-based parallel computing module to perform convolution operations in feature computation. If no GPU hardware environment is detected in the system, the CPU multi-process pool will be used for computation.
[0089] Reference Figure 10 Based on the methods in the above embodiments, this application provides an electronic device that may include: a processor 11, a communications interface 12, a memory 13, and a communication bus 14, wherein the processor 11, the communications interface 12, and the memory 13 communicate with each other through the communication bus 14. The processor 11 may call logical instructions in the memory 13 to execute the methods in the above embodiments.
[0090] Furthermore, the logical instructions in the aforementioned memory 13 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.
[0091] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.
[0092] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.
[0093] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.
[0094] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.
[0095] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).
[0096] It is understood that the various numerical designations used in the embodiments of this application are merely for the convenience of description and are not intended to limit the scope of the embodiments of this application.
[0097] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.
Claims
1. A method for intelligent analysis of metallographic structures based on feature fusion and active learning, characterized in that, include: Acquire a high-resolution image of the metallographic sample and preprocess the high-resolution image; Feature extraction is performed on the preprocessed image to obtain pixel-level features. The pixel-level features are then concatenated, fused, and standardized to construct a high-dimensional feature vector. The pixel-level features include texture features, edge features, and multi-scale spatial features. The system receives user input and, in response to the user input, labels different phase categories; it uses the high-dimensional feature vector as sample input and the phase category as sample label to train a lightweight machine learning classifier, resulting in a trained and optimized image classification model; when the model predicts a misclassification, it receives supplementary annotation input from the user for the misclassified region; it obtains incremental data of the sample label based on the supplementary annotation input, and retrains the model based on the incremental data to form an active learning closed loop; The image classification model is used to analyze the image to be analyzed, and a binary mask of the image to be analyzed is obtained. Then, a selected strategy is used to calculate the metallographic parameters based on the binary mask.
2. The intelligent metallographic structure analysis method based on feature fusion and active learning according to claim 1, characterized in that, The process of acquiring a high-resolution image of the metallographic sample and preprocessing the high-resolution image includes: Based on the imaging quality characteristics of the high-resolution image, image processing operations are selected and sorted from a preset algorithm library to construct a dynamically configured preprocessing pipeline; The high-resolution image is converted into a grayscale image and preprocessed through the preprocessing pipeline, which includes background correction.
3. The intelligent metallographic structure analysis method based on feature fusion and active learning according to claim 2, characterized in that, The background correction process specifically includes: The background illumination distribution map of the high-resolution image is estimated using large-scale median filtering; The corrected image is calculated using the following formula: in, For the corrected image, This is the grayscale image corresponding to the high-resolution image. This is a background illumination distribution map. To prevent the minimum value of division by zero, This represents the mean value of the background illumination distribution map.
4. The intelligent metallographic structure analysis method based on feature fusion and active learning according to claim 3, characterized in that, Feature extraction is performed on the preprocessed image to obtain pixel-level features. These pixel-level features are then concatenated, fused, and standardized to construct a high-dimensional feature vector, including: The local texture invariants of the corrected image are extracted using a local binary model to obtain the texture features; The first-order gradients in the X and Y directions of the corrected image are extracted using the Sobel operator, and the second-order edge gradients are extracted using the Laplacian operator. The edge features are obtained based on the extracted first-order and second-order gradients. Multi-scale spatial features are generated using Gaussian blurring with different kernel sizes; The texture features, edge features, and multi-scale spatial features are fused to obtain the pixel-level features, and the fused pixel-level features are normalized using the Z-Score normalization method to obtain the high-dimensional feature vector.
5. The intelligent metallographic structure analysis method based on feature fusion and active learning according to claim 1, characterized in that, The lightweight machine learning classifier is built based on the random forest algorithm or the support vector machine algorithm; When using the random forest algorithm, multiple decision trees are constructed through bootstrap sampling and feature random selection, and the prediction result is determined by the majority vote of the decision trees. When using the support vector machine algorithm, a radial basis function kernel is selected, and a one-to-many strategy is adopted to solve the multi-class segmentation task.
6. The intelligent metallographic structure analysis method based on feature fusion and active learning according to claim 3, characterized in that, The metallographic parameters include phase area ratio and secondary dendrite arm spacing; The method for calculating the phase area ratio includes: The total number of pixels belonging to the c-th type of organization in the binarized mask and the total number of effective pixels in the entire image are counted, and the phase area ratio is calculated based on the total number of pixels and the total number of effective pixels in the entire image; The method for calculating the secondary dendrite arm spacing includes at least one of the following: Morphological refinement is performed on the segmented dendritic region to obtain a single-pixel skeleton, the total dendrite area and estimated length are obtained, and the spacing between the secondary dendrite arms is calculated based on the area of the connected region and the skeleton length. Horizontal and vertical scan lines are automatically generated on the binarized mask. The number of intersections between the cut-off lines and the dendritic phase and the length of the cut-off lines are counted. The spacing between the secondary dendrite arms is calculated based on the number of intersections and the length of the cut-off lines. The grayscale intensity profile is extracted from the measurement line path defined by the user, and the number of dendrite arms is identified by smoothing and signal peak detection algorithm. The secondary dendrite arm spacing is calculated based on the measurement line length and the number of dendrite arms.
7. The intelligent metallographic structure analysis method based on feature fusion and active learning according to claim 1, characterized in that, Also includes: The binarized mask is segmented to obtain the segmented connected components; Component analysis is performed on the segmented connected domains to calculate the area, perimeter, roundness, and aspect ratio of the independent phases in order to characterize the morphological features of individual dendrites. An image stitching algorithm is used to stitch multiple local field-of-view images into a panoramic image, and macroscopic statistics are performed based on the panoramic image; The local field-of-view image is obtained by segmenting the high-resolution image.
8. The intelligent metallographic structure analysis method based on feature fusion and active learning according to claim 1, characterized in that, The image classification model achieves image analysis through multi-process parallel processing; If the system is detected to have a graphics processing unit (GPU) hardware environment, it will automatically switch to the CUDA-based parallel computing module to perform convolution operations in feature computation. If no GPU hardware environment is detected in the system, the CPU multi-process pool will be used for computation.
9. A metallographic structure intelligent analysis system based on feature fusion and active learning, characterized in that, include: An image preprocessing module is used to acquire high-resolution images of metallographic samples and to preprocess the high-resolution images. The feature extraction module is used to extract features from the preprocessed image to obtain pixel-level features, and to perform concatenation, fusion and standardization of the pixel-level features to construct a high-dimensional feature vector; the pixel-level features include: texture features, edge features and multi-scale spatial features. The model training module is used to receive user input and, in response to the user input, label different phase categories; it uses the high-dimensional feature vector as sample input and the phase category as sample label to train a lightweight machine learning classifier to obtain a trained and optimized image classification model. The image analysis module is used to analyze the image to be analyzed using the image classification model, obtain a binary mask of the image to be analyzed, and perform calculations based on the binary mask using a selected strategy to obtain metallographic parameters.
10. An electronic device, characterized in that, include: At least one memory for storing computer programs; At least one processor is configured to execute a program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to perform the method as described in any one of claims 1-8.