AI-driven metallographic image analysis and classification system and method

Abstract

The invention provides an AI-driven metallographic image analysis and classification system and method, and the method comprises the steps: obtaining a high-resolution image of a metallographic sample through a microscope, and recording the basic information of the sample; denoising, graying, contrast enhancement and edge detection are carried out on the metallographic image; performing feature extraction on the processed metallographic image by adopting a convolutional neural network; classifying the extracted image features by using a support vector machine or a random forest; identifying and marking the abnormal image by using an isolated forest algorithm; all data are stored in a database, and a metallographic analysis report is automatically generated according to classification and detection results; and training and optimizing the deep learning model and a machine learning algorithm. According to the system, abnormal samples can be automatically identified and marked, the accuracy and timeliness of anomaly detection are ensured through a clustering algorithm and expert re-checking, comprehensive data storage and management functions are provided, and the safety and integrity of data are ensured through regular backup and optimization.

Description

Technical Field

[0001] The present invention relates to the field of material science and technology, and in particular to an AI-driven metallographic image analysis and classification system and method. Background Art

[0002] In the field of materials science and engineering, metallographic image analysis is a key technology used to evaluate the microstructure of metals and alloys to determine their mechanical properties and physical characteristics. Traditional metallographic image analysis usually relies on microscopic observation and manual evaluation, which requires experienced technicians to perform complex and time-consuming operations. In recent years, with the development of computer vision and artificial intelligence technology, automated metallographic image analysis systems have gradually been introduced to improve analysis efficiency and accuracy. Common techniques include image classification and feature extraction methods based on classical machine learning algorithms, as well as early Convolutional Neural Networks (CNN) models, which are used to achieve basic image classification and recognition tasks.

[0003] However, existing automated metallographic image analysis systems still face several problems. First, the limitations of feature extraction are obvious. Traditional feature extraction methods, such as edge detection and texture analysis, have difficulty capturing the complex morphological and texture features in metallographic images. Even early CNN models, limited by training data and network structure, have difficulty in fully extracting deep features. In addition, the generalization ability of the model is insufficient. Many models perform poorly when processing metallographic images of different types and morphologies. This is mainly due to the fact that the model is over-trained on a specific data set, resulting in insufficient generalization ability and difficulty in processing diverse metallographic images in practical applications. The low accuracy of anomaly detection is also a significant problem. Traditional methods are often inaccurate in identifying anomalies or defects in images, which may cause key defects to be overlooked and affect the final quality assessment of the material. In addition, the storage, management, and effective query of large-scale metallographic image data have always been a challenge. Traditional databases are not efficient in processing complex queries and real-time analysis.

[0004] Due to the existence of the above problems, traditional metallographic image analysis methods often lead to low analysis efficiency. Manual operations and inefficient automated tools lead to lengthy analysis processes, affecting production efficiency. The results are inconsistent, and since the analysis relies on human experience and judgment, the repeatability and consistency of the results are difficult to guarantee. Potential quality risks also increase, because untimely or inaccurate detection of anomalies and defects may lead to potential quality problems and safety hazards in the use of materials. In addition, due to improper data processing and management, resource waste is obvious in the data storage and retrieval process, and it is difficult to support rapid response to decisions. Therefore, the existing technology has obvious deficiencies in the field of metallographic image analysis, and a new AI-driven analysis and classification system is urgently needed to improve analysis efficiency, enhance generalization capabilities, improve anomaly detection accuracy, and optimize data processing and management processes. Summary of the invention

[0005] In order to overcome the shortcomings of the existing technology, the present invention proposes an AI-driven metallographic image analysis and classification system and method, which provides manufacturing companies with efficient, accurate and flexible material analysis solutions, significantly improving the quality control level and market competitiveness.

[0006] To achieve the above objectives, the present invention provides an AI-driven metallographic image analysis and classification method, comprising:

[0007] Step 1: Obtain high-resolution images of multiple metallographic samples through a microscope and record basic information of each sample, including sample type, processing technology, and shooting parameters.

[0008] Step 2: De-noise, grayscale, enhance contrast and detect edges on the metallographic image.

[0009] Step 3: Use convolutional neural network to extract features from the processed metallographic image.

[0010] Step 4: Use support vector machine or random forest to classify the extracted image features.

[0011] Step 5: Use the isolation forest algorithm to identify and mark abnormal images.

[0012] Step 6: All data are stored in the database and a metallographic analysis report is automatically generated based on the classification and test results.

[0013] Step 7: Train and optimize deep learning models and machine learning algorithms.

[0014] Furthermore, image preprocessing includes:

[0015] Performing multi-scale transformation on the image to enhance detail information at different scales; the multi-scale transformation includes Gaussian pyramid transformation and Laplace pyramid transformation to extract multi-level information in the image;

[0016] Separating the grains from the background in the image by using an adaptive threshold segmentation algorithm; the adaptive threshold segmentation algorithm automatically adjusts the threshold based on local contrast and brightness to achieve accurate segmentation;

[0017] Noise and artifacts in the image are removed by morphological operations (such as opening and closing operations); the morphological operations include erosion, dilation, opening operations and closing operations to optimize image quality.

[0018] Furthermore, feature extraction includes:

[0019] Using a pre-trained convolutional neural network model for transfer learning to extract deep features of the image; the transfer learning is based on a model pre-trained with the ImageNet dataset and is fine-tuned to adapt to metallographic image features;

[0020] The texture and morphological features are extracted by combining traditional image processing algorithms (such as Fourier transform and histogram equalization); the Fourier transform is used to extract frequency domain features, and the histogram equalization is used to enhance image contrast.

[0021] Further, image classification includes:

[0022] The feature data is standardized to eliminate the differences between different feature dimensions; the standardization includes mean normalization and maximum and minimum value normalization to unify the feature scale;

[0023] A cross-validation method is used to evaluate the performance of the classification model and adjust the model parameters to optimize the classification effect; the cross-validation method includes k-fold cross-validation and leave-one-out cross-validation to avoid overfitting and improve the generalization ability of the model.

[0024] Furthermore, anomaly detection includes:

[0025] Using a clustering algorithm (such as DBSCAN) to perform cluster analysis on feature data and identify abnormal clusters; the DBSCAN algorithm is based on a density-based clustering method and can identify abnormal clusters of any shape;

[0026] Abnormal images are subjected to secondary inspection and manual review to ensure the accuracy of abnormal marking; the secondary inspection includes cross-validation using multiple detection algorithms, and the manual review is confirmed by experts in the metallographic field.

[0027] Furthermore, data storage and management include:

[0028] Regularly back up and maintain the database to ensure data integrity and security; the backup includes incremental backup and full backup, and maintenance measures include data cleanup and optimization;

[0029] A flexible query interface is provided to support multi-dimensional and multi-condition image and data retrieval; the query interface is based on SQL and NoSQL database technologies to support complex queries and real-time data analysis.

[0030] Furthermore, the result output and report generation include:

[0031] Provide customizable report templates to support report generation for different user needs; the report templates include detailed reports and summary reports, and users can choose the appropriate template according to their needs;

[0032] The analysis results are displayed through a web interface, which supports online viewing and downloading of reports. The web interface adopts a responsive design and is adapted to different terminal devices, so users can access and download reports anytime and anywhere.

[0033] Furthermore, model optimization includes:

[0034] Retrain the model using the latest metallographic image data to update model parameters and weights; the retraining is based on online learning and incremental learning techniques to ensure that the model is always up to date;

[0035] In combination with feedback from domain experts, the model structure and algorithm process are continuously improved to enhance the overall analysis performance; the improvements include optimizing the neural network hierarchy, adjusting hyperparameters, and modifying rules in combination with expert knowledge.

[0036] An AI-driven metallographic image analysis and classification system, applicable to the AI-driven metallographic image analysis and classification method, comprising: a data acquisition module, an image preprocessing module, a feature extraction module, an image classification module, an anomaly detection module, a data storage and management module, a report generation module, and a user interface module;

[0037] The output end of the data acquisition module is signal-connected to the input end of the image preprocessing module;

[0038] The output end of the image preprocessing module is signal-connected to the input end of the feature extraction module, the output end of the feature extraction module is signal-connected to the input end of the image classification module, the output end of the image classification module is signal-connected to the input end of the abnormality detection module, the output end of the abnormality detection module is signal-connected to the input end of the data storage and management module, and the output end of the data storage and management module is signal-connected to the input end of the report generation module and the user interface module respectively.

[0039] Further,

[0040] Data acquisition module: used to obtain high-resolution images of multiple metallographic samples through a microscope and record basic information of each sample, including sample type, processing technology and shooting parameters.

[0041] Image preprocessing module: used to preprocess the collected metallographic images, the preprocessing includes denoising, graying, contrast enhancement and edge detection to obtain a processed image.

[0042] Feature extraction module: used to extract features from preprocessed metallographic images using convolutional neural networks, including deep features and traditional image processing features (such as Fourier transform and histogram equalization).

[0043] Image classification module: used to classify the extracted image features using support vector machine or random forest algorithm to obtain classification results.

[0044] Anomaly detection module: It is used to perform cluster analysis on feature data using clustering algorithms (such as DBSCAN), identify and mark abnormal clusters, and ensure the accuracy of abnormal marking through multiple detection algorithms and manual review.

[0045] Data storage and management module: used to store all metallographic images and their corresponding classification results, feature data and abnormal marks in the database, and regularly back up and maintain data to ensure data integrity and security.

[0046] Report generation module: used to automatically generate metallographic analysis reports based on classification results and abnormality detection results. The reports include detailed reports and summary reports. Users can select appropriate templates according to their needs. Reports can be output in PDF, HTML and other formats.

[0047] User interface module: used to display analysis results through a web interface, supporting online viewing and downloading of reports. The web interface adopts a responsive design and is adapted to different terminal devices. Users can access and download reports anytime and anywhere.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] 1. The present invention provides an AI-driven metallographic image analysis and classification system and method. Through automated image acquisition and analysis, the system significantly reduces the time of metallographic analysis, improves detection efficiency, and can quickly process a large number of samples. By using pre-trained convolutional neural networks and advanced image processing algorithms, the system can more accurately identify and classify metallographic structure types and reduce human errors.

[0050] 2. The present invention provides an AI-driven metallographic image analysis and classification system and method. The system can automatically identify and mark abnormal samples, ensure the accuracy and timeliness of abnormality detection through clustering algorithms and expert review, and reduce the risk of defective products entering the market; the system provides comprehensive data storage and management functions, ensures the security and integrity of data through regular backup and optimization, and provides a reliable basis for quality traceability.

[0051] 3. The present invention provides an AI-driven metallographic image analysis and classification system and method. The system supports multiple report templates. Users can generate detailed or summary reports as needed to help management quickly obtain required information to support decision-making; through a responsive Web interface, users can access and query analysis results anytime and anywhere, support multi-dimensional and complex condition retrieval, and improve work efficiency and convenience.

[0052] 4. The present invention provides an AI-driven metallographic image analysis and classification system and method. The system combines online learning and feedback from field experts to continuously update models and improve algorithms to ensure that the analysis capabilities always remain at the forefront and meet the ever-changing production needs. By reducing manual analysis and improving production line efficiency, the system helps companies reduce labor costs and operating expenses, and improves overall economic benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0054] Figure 1 It is a schematic diagram of the steps of the present invention DETAILED DESCRIPTION

[0055] The technical solution of the present invention will be more clearly and completely explained below through description of preferred embodiments of the present invention in combination with the accompanying drawings.

[0056] Terminology explanation:

[0057] CNN: Convolutional Neural Network, a deep learning model. DBSCAN): Density-based clustering algorithm used to identify clusters of arbitrary shapes and can identify noisy data. SQL: Structured Query Language. NoSQL: Non-relational database. Web: Refers to the World Wide Web. ImageNet: A large visual database. TA: Time Advance. AOA: Angle of Arrival, which refers to the angle of the direction from which the signal reaches the receiving device.

[0058] like Figure 1 As shown, the present invention is specifically:

[0059] Step 1: Obtain high-resolution images of metallographic samples through a microscope and record basic information of the samples, including sample type, processing technology and shooting parameters; Step 2: De-noise, grayscale, enhance contrast and detect edges of metallographic images; Step 3: Use convolutional neural networks to extract features from processed metallographic images; Step 4: Use support vector machines or random forests to classify the extracted image features; Step 5: Use the isolation forest algorithm to identify and mark abnormal images; Step 6: Store all data in a database and automatically generate a metallographic analysis report based on the classification and detection results; Step 7: Train and optimize deep learning models and machine learning algorithms.

[0060] As a specific implementation method, a pre-trained convolutional neural network model is used for transfer learning to extract deep features of an image as follows:

[0061] Choose a pre-trained model: Choose a convolutional neural network model pre-trained on the ImageNet dataset, such as VGG16, ResNet50, or InceptionV3. These models have been trained on large-scale image datasets and have powerful feature extraction capabilities.

[0062] Freeze bottom convolutional layers: Freeze the first few convolutional layers of the pre-trained model, which can extract common low-level features such as edges, textures, etc. Keep the weights of these layers unchanged, thereby reducing training time and data requirements.

[0063] Add custom layers: Add several new convolutional or fully connected layers on top of the pre-trained model to adapt to the specific features of metallographic images. These new layers will be specialized for learning high-level features in metallographic images, such as grain structure and defects.

[0064] Fine-tuning the model: The entire model is fine-tuned using the metallographic image dataset. When fine-tuning, a smaller learning rate is used so that the pre-trained weights can be slightly adjusted without destroying their original feature extraction capabilities. The weights of the newly added layers are updated through the back-propagation algorithm, while allowing the underlying frozen convolutional layers to be slightly adjusted to better fit the metallographic image features.

[0065] Combine traditional image processing algorithms to extract texture and morphological features

[0066] Fourier transform: Perform a Fourier transform on a metallographic image to convert the image from the spatial domain to the frequency domain. In the frequency domain, the different frequency components of the image can reveal the grain structure and other texture features. By analyzing the spectrum, texture features at specific frequencies, such as periodic structures or directional features, can be identified and extracted.

[0067] Histogram equalization: Apply histogram equalization to metallographic images to enhance the contrast of the image. Histogram equalization adjusts the grayscale distribution of the image to make the grayscale distribution of the image more uniform, thereby highlighting the detailed features. This is very useful for observing and analyzing tiny structures and defects in metallographic images.

[0068] The detailed step code example is as follows, select the pre-trained model:

[0069] from tensorflow.keras.applications import VGG16,ResNet50,InceptionV3

[0070] base_model=VGG16(weights='imagenet',include_top=False,input_shape=(224,224,3))

[0071] Freeze the bottom convolutional layers:

[0072] for layer in base_model.layers:

[0073] layer.trainable = False

[0074] Add a custom layer:

[0075]

[0076] Fine-tuning the model:

[0077] model.compile(optimizer='adam',loss='categorical_crossentropy',metrics=['accuracy'])

[0078] model.fit(train_data,train_labels,epochs=10,validation_data=(val_data,val_labels))

[0079] Fourier Transform:

[0080] import numpy as np

[0081] import cv2

[0082] image=cv2.imread('path_to_image',cv2.IMREAD_GRAYSCALE)

[0083] f_transform=np.fft.fft2(image)

[0084] f_shift=np.fft.fftshift(f_transform)

[0085] magnitude_spectrum=20*np.log(np.abs(f_shift))

[0086] Histogram Equalization:

[0087] equalized_image=cv2.equalizeHist(image)

[0088] Through these steps, deep learning and traditional image processing algorithms can be effectively used to extract the deep features, texture and morphological characteristics of metallographic images for accurate analysis and classification.

[0089] As a specific implementation method, the characteristic data is standardized as follows:

[0090] Standardized processing steps

[0091] Mean normalization: Calculate the mean and standard deviation of each feature. Subtract the mean from each feature value and divide it by its standard deviation so that the distribution of feature values ​​has a mean of 0 and a standard deviation of 1.

[0092] The code example is as follows:

[0093]

[0094] Maximum and minimum normalization: Calculate the minimum and maximum values ​​of each feature. Subtract the minimum value from each feature value and divide it by its range (maximum value minus minimum value) to scale the feature value to the range of [0,1].

[0095] The code example is as follows:

[0096] from sklearn.preprocessing import MinMaxScaler

[0097] scaler = MinMaxScaler()

[0098] normalized_features=scaler.fit_transform(features)

[0099] Use cross-validation to evaluate the performance of classification models

[0100] K-fold cross validation: Divide the data set into k equal parts. Each time, use k-1 parts as training sets and the remaining 1 part as validation set, and perform k training and validation. Calculate the performance index of each validation and take the average as the performance evaluation index of the model. The code example is as follows:

[0101]

[0102] Leave-one-out cross-validation: Divide the data set into n equal parts (n is the number of samples in the data set), use n-1 of them as training sets each time, and the remaining 1 as validation set, and perform n training and validation. Calculate the performance index of each validation, and take the average as the performance evaluation index of the model.

[0103] The code example is as follows:

[0104]

[0105]

[0106] Adjust model parameters to optimize classification performance

[0107] Grid Search: Define a parameter grid containing the parameter combinations to be tuned. During the cross-validation process, each parameter combination is trained and validated to select the parameter combination with the best performance. The code example is as follows:

[0108]

[0109] As a specific implementation method, clustering analysis is performed on feature data using a clustering algorithm (such as DBSCAN) to identify abnormal clusters as follows:

[0110] DBSCAN clustering analysis steps

[0111] Data preparation: Collect and standardize the processed feature data to ensure that the differences between different feature dimensions have been eliminated. The code example is as follows:

[0112] from sklearn.preprocessing import StandardScaler

[0113] features = np.array([...]) # Sample feature data

[0114] scaler = StandardScaler()

[0115] standardized_features=scaler.fit_transform(features)

[0116] Application of DBSCAN algorithm: Use DBSCAN algorithm to perform cluster analysis on the standardized feature data. Set two main parameters: eps (neighborhood radius) and min_samples (minimum number of core points). DBSCAN algorithm is a density-based clustering method that can identify abnormal clusters of any shape.

[0117] The code example is as follows:

[0118] from sklearn.cluster import DBSCAN

[0119] dbscan=DBSCAN(eps=0.5,min_samples=5)

[0120] clusters=dbscan.fit_predict(standardized_features)

[0121] #Add the clustering results to the feature data

[0122] import pandas as pd

[0123] data_with_clusters=pd.DataFrame(standardized_features, columns=['Feature1','Feature2','Feature3'])

[0124] data_with_clusters['Cluster']=clusters

[0125] Identify abnormal clusters: In the clustering results, samples with a label of -1 are considered noise points or abnormal points. These abnormal points are counted and analyzed to determine whether they represent abnormal clusters. The code example is as follows:

[0126] anomalies=data_with_clusters[data_with_clusters['Cluster']===-1]

[0127] print(f'Number of anomalies detected:{len(anomalies)}')

[0128] Perform secondary inspection and manual review on abnormal images; secondary inspection steps

[0129] Cross-validation using multiple detection algorithms: Use different anomaly detection algorithms to perform secondary detection on the identified abnormal images to ensure the reliability of the detection results. Commonly used anomaly detection algorithms include Isolation Forest and Local Outlier Factor (LOF).

[0130] The code example is as follows:

[0131] from sklearn.ensemble import IsolationForest

[0132] from sklearn.neighbors import LocalOutlierFactor

[0133] #Use Isolation Forest Algorithm

[0134] iso_forest=IsolationForest(contamination=0.1)

[0135] iso_forest.fit(standardized_features)

[0136] iso_forest_predictions=iso_forest.predict(standardized_features)

[0137] #Use the local outlier factor algorithm

[0138] lof=LocalOutlierFactor(n_neighbors=20,contamination=0.1)

[0139] lof_predictions=lof.fit_predict(standardized_features)

[0140] #Merge the results for cross validation

[0141] combined_anomalies=(iso_forest_predictions==-1)&(lof_predictions==-1)

[0142] Manual review: The abnormal images identified after cross-validation are submitted to metallographic experts for manual review. The experts confirm and mark the final abnormal images based on the image features and test results.

[0143] The code example is as follows:

[0144] confirmed_anomalies=data_with_clusters[combined_anomalies]

[0145] #Assume we have an expert review function expert_review(anomalies)

[0146] expert_review_results=expert_review(confirmed_anomalies)

[0147] Ensure accuracy of anomaly marking

[0148] Expert review process: Experts observe abnormal images and use their professional knowledge to confirm whether the abnormalities in the images are real. Experts can provide further comments and suggestions to help improve the automatic detection algorithm. The code example is as follows:

[0149]

[0150] Result recording and feedback: The final confirmed abnormal results are recorded in the database for subsequent analysis and model optimization. Expert feedback is incorporated into the model training process to improve the detection accuracy of the model. The code example is as follows:

[0151] #Store the confirmation results in the database

[0152] confirmed_anomalies.to_csv('confirmed_anomalies.csv',index=False)

[0153] Through these detailed steps, the DBSCAN algorithm can be effectively used for cluster analysis to identify and mark abnormal clusters in metallographic images, and the accuracy of abnormal marking can be ensured through secondary detection and manual review. This method combines automated detection with expert knowledge and can significantly improve the reliability and accuracy of anomaly detection.

[0154] As a specific implementation method, the database is backed up and maintained regularly as follows:

[0155] Backup strategy: Develop a detailed backup strategy, including the schedule for incremental and full backups. Incremental backups are usually performed once a day, backing up only the data that has changed since the last backup. Full backups are usually performed once a week or month, backing up the entire database.

[0156] The code example is as follows:

[0157] #Full backup example (assuming MySQL database is used)

[0158] mysqldump-u[username]-p[password][database_name]>full_backup_$(date+%F).sql

[0159] #Incremental backup example (assuming incremental backup using the file system)

[0160] rsync-av--delete / path / to / data / path / to / backup / incremental_backup_$(date+%F)

[0161] Data cleaning: Regularly clean up redundant and invalid data in the database. Delete data that exceeds the retention period by setting data retention policies. Optimize database indexes, delete unnecessary indexes, and rebuild fragmented indexes.

[0162] The code example is as follows:

[0163] --Delete data that exceeds the retention period

[0164] DELETE FROM table_name WHERE created_at <DATE_SUB(NOW(),INTERVAL1YEAR);

[0165] --Rebuild index

[0166] ALTER TABLE table_name DROP INDEX index_name,ADDINDEX index_name(column_name);

[0167] Database optimization: Optimize database query performance, including query rewriting, index optimization, and database parameter adjustment. Regularly evaluate database performance to identify and resolve performance bottlenecks.

[0168] The code example is as follows:

[0169] --Query rewrite example

[0170] EXPLAIN SELECT*FROM table_name WHERE column_name='value';

[0171] --Optimize database parameters (assuming MySQL is used)

[0172] SET GLOBAL innodb_buffer_pool_size=1024*1024*1024;#Set the InnoDB buffer pool size to 1GB

[0173] Provide flexible query interface; query interface implementation steps; select database technology: select appropriate SQL and NoSQL databases based on data characteristics and query requirements. SQL databases such as MySQL and PostgreSQL are suitable for structured data query and transaction processing. NoSQL databases such as MongoDB and Elasticsearch are suitable for unstructured data query and real-time data analysis.

[0174] Design query interface: Provide a flexible API interface to support users to query data through HTTP requests. The API interface can use RESTful or GraphQL architecture to achieve flexible data access. The code example is as follows:

[0175]

[0176]

[0177] Multi-dimensional and multi-condition query: The query interface is designed to support multi-dimensional and multi-condition combination query. It provides a variety of query conditions, including range query, fuzzy query and aggregate query. The code example is as follows:

[0178] from sqlalchemy import create_engine,Table,MetaData

[0179] from sqlalchemy.sql import select

[0180] engine=create_engine('postgresql: / / username:password@localhost / database_name')

[0181] connection = engine.connect()

[0182] metadata=MetaData()

[0183] table=Table('table_name',metadata,autoload=True,autoload_with=engine)

[0184] #Example query: Query by multiple conditions

[0185] stmt=select([table]).where(table.c.column1=='value1').where(table.c.column2.between(10,20))

[0186] results=connection.execute(stmt).fetchall()

[0187] Support complex queries and real-time data analysis: Based on NoSQL databases, such as Elasticsearch, it supports complex queries and real-time data analysis. Design and implement real-time data analysis functions, and provide real-time query and analysis interfaces. The code example is as follows:

[0188]

[0189]

[0190] As a specific implementation method, a customizable report template is provided to support report generation for different user needs as follows:

[0191] Determine the report template structure:

[0192] Detailed report template: includes all details of metallographic image analysis, such as sample information, image preprocessing steps, feature extraction and classification results, anomaly detection results, etc.

[0193] Summary report template: contains key results and conclusions, simplified information presentation, suitable for quick browsing.

[0194] The code example is as follows:

[0195]

[0196]

[0197]

[0198] Implement report generation: Use template engines (such as Jinja2) to generate reports, supporting user-defined content and format. Report generation can be output in PDF, HTML and other formats to meet different user needs. The code example is as follows:

[0199]

[0200] Display analysis results through the web interface, support online viewing and downloading of reports

[0201] Web Interface Design Steps

[0202] Building a Web application framework:

[0203] Use frameworks such as Flask or Django to build web applications and provide report display and download functions.

[0204] Design a responsive web interface to ensure good display on different terminal devices (PC, tablet, mobile phone). The code example is as follows:

[0205]

[0206] Implement responsive design: Use frameworks such as Bootstrap or Tailwind CSS to implement responsive design to ensure that the interface is adaptively displayed on different devices. The code example is as follows:

[0207]

[0208]

[0209] Report display and download: Display the report content in the web interface, and users can choose to view detailed reports or summary reports. Provide report download function, users can download the report in PDF format. The code example is as follows:

[0210]

[0211]

[0212] As a specific implementation method, the model retraining using the latest metallographic image data is as follows:

[0213] Collect the latest data: Continuously collect and store the latest metallographic image data to ensure timely update of the data set. The code example is as follows:

[0214]

[0215] Data preprocessing:

[0216] Preprocess the latest collected metallographic image data, including normalization, data enhancement, etc.

[0217] The code example is as follows:

[0218]

[0219]

[0220] Model retraining: Use the latest data to retrain the pre-trained convolutional neural network model. Use online learning and incremental learning techniques to ensure that the model's weights and parameters are updated in a timely manner. The code example is as follows:

[0221]

[0222] Combined with feedback from domain experts, we continuously improve the model structure and algorithm process

[0223] Collect expert feedback: Regularly communicate with domain experts to obtain feedback on model performance and improvement suggestions. The code example is as follows:

[0224]

[0225]

[0226] Optimize the neural network hierarchy: Based on expert feedback, adjust the neural network hierarchy, add or delete convolutional layers and fully connected layers. The code example is as follows:

[0227]

[0228] Adjust hyperparameters: Based on expert feedback and cross-validation results, adjust the model's hyperparameters (such as learning rate, batch size, etc.) to optimize the training effect. The code example is as follows:

[0229] from tensorflow.keras.optimizers import Adam

[0230] model.compile(optimizer=Adam(learning_rate=0.0001),

[0231] loss='categorical_crossentropy',metrics=['accuracy'])

[0232] model.fit(datagen.flow(new_images,new_labels,batch_size=64),epochs=10)

[0233] model.save('path_to_further_updated_model.h5')

[0234] Combine expert knowledge to modify rules: Based on the domain knowledge of experts, formulate and apply specific rules to improve feature extraction and classification. The code example is as follows:

[0235]

[0236]

[0237] Iterative improvement: Continue to collect new data and expert feedback, regularly retrain and optimize the model to ensure continuous improvement in model performance. The code example is as follows:

[0238]

[0239] As a specific implementation case, an automotive parts manufacturer needs to perform metallographic structure analysis on the steel parts it produces to ensure that the material quality meets the standards. Traditional manual analysis methods are time-consuming and easily affected by human factors. The steps of using an AI-driven metallographic image analysis and classification system are as follows:

[0240] Data acquisition module

[0241] The company installed an automated microscope with a high-resolution camera to collect metallographic images of steel samples. The image acquisition device is integrated at the end of the production line, which can automatically capture sample images of each batch of products and transmit them to the server in real time.

[0242] Image preprocessing module

[0243] The collected images are first processed by noise removal and grayscale. Histogram equalization technology is used to enhance the image contrast, and the Canny algorithm is used for edge detection to highlight the details of the metallographic structure.

[0244] Feature extraction module

[0245] Use pre-trained convolutional neural network models (such as ResNet) to extract deep features. Combine Fourier transform to extract frequency domain features, and use morphological operations to extract image morphological features. These features are used to characterize the texture and structural information of metallographic images.

[0246] Image Classification Module

[0247] The extracted features are fed into a tuned support vector machine model to classify different microstructure types (such as pearlite, bainite, ferrite, etc.). The classification results show the microstructure ratio of the batch of parts.

[0248] Anomaly Detection Module

[0249] The DBSCAN algorithm is used to perform cluster analysis on the classification results and identify abnormal images. For images marked as abnormal, the system will automatically initiate a secondary inspection and be reviewed by metallographic experts to ensure the accuracy of the abnormal marking.

[0250] Data storage and management module

[0251] All images, features, classification results, and anomaly detection information are stored in the company's internal database. The system is set to perform incremental and full backups regularly to ensure data security and integrity. The database is cleaned and optimized regularly to improve retrieval efficiency.

[0252] Report generation module

[0253] The reports generated by the system include detailed analysis reports and brief summary reports. The reports include the metallographic structure ratio of each batch of products, abnormal detection results and treatment suggestions. Users can choose different report formats for output according to their needs.

[0254] User Interface Module

[0255] Through the internal web interface, quality inspectors and management can view analysis results and reports in real time. The interface supports custom query functions, allowing users to retrieve analysis data within a specific batch or date range. Reports can be exported as PDF files for further sharing and archiving.

[0256] After implementing the system, the company's metallographic analysis efficiency increased by about 50%, and labor costs were significantly reduced. At the same time, automated analysis reduced human errors and improved the consistency and reliability of test results. Quality inspectors can spend more time dealing with anomalies and improving process flow rather than repetitive daily testing. Management is satisfied with the reports generated by the system and can make production decisions faster.

[0257] The above specific implementations are only descriptions of the preferred implementations of the present invention, and do not limit the protection scope of the present invention. Without departing from the design concept and spirit of the present invention, various modifications, substitutions and improvements made by ordinary technicians in this field to the technical solution of the present invention based on the text description and drawings provided by the present invention should all fall within the protection scope of the present invention. The protection scope of the present invention is determined by the claims.

Claims

1. AI-driven metallographic image analysis and classification method, characterized in that: include: Step 1: Obtain high-resolution images of metallographic samples through a microscope and record basic information of the samples, including sample type, processing technology, and shooting parameters; Step 2: De-noise, grayscale, enhance contrast and detect edges of the metallographic image; Step 3: Use convolutional neural network to extract features from the processed metallographic image; Step 4: Use support vector machine or random forest to classify the extracted image features; Step 5: Use the isolation forest algorithm to identify and mark abnormal images; Step 6: All data are stored in the database, and a metallographic analysis report is automatically generated based on the classification and test results; Step 7: Train and optimize deep learning models and machine learning algorithms.

2. The AI-driven metallographic image analysis and classification method according to claim 1, characterized in that: Steps 1 and 2 include: Performing multi-scale transformation on the image to enhance detail information at different scales; the multi-scale transformation includes Gaussian pyramid transformation and Laplace pyramid transformation to extract multi-level information in the image; Separating the grains from the background in the image by using an adaptive threshold segmentation algorithm; the adaptive threshold segmentation algorithm automatically adjusts the threshold based on local contrast and brightness to achieve accurate segmentation; Noise and artifacts in the image are removed by morphological operations; the morphological operations include erosion, dilation, opening and closing operations to optimize image quality.

3. The AI-driven metallographic image analysis and classification method according to claim 1, characterized in that: Step 3 includes: Using a pre-trained convolutional neural network model for transfer learning to extract deep features of the image; the transfer learning is based on a model pre-trained with the ImageNet dataset and is fine-tuned to adapt to metallographic image features; The traditional image processing algorithms, including Fourier transform and histogram equalization, are combined to extract texture and morphological features; the Fourier transform is used to extract frequency domain features, and the histogram equalization is used to enhance image contrast.

4. The AI-driven metallographic image analysis and classification method according to claim 1, characterized in that: Step 4 includes: Standardizing the feature data to eliminate differences between different feature dimensions; the standardization includes mean normalization and maximum and minimum value normalization to unify the feature scale; A cross-validation method is used to evaluate the performance of the classification model and adjust the model parameters to optimize the classification effect; the cross-validation method includes k-fold cross-validation and leave-one-out cross-validation to avoid overfitting and improve the generalization ability of the model.

5. The AI-driven metallographic image analysis and classification method according to claim 1, characterized in that: Step 5 includes: Using the clustering algorithm DBSCAN, the feature data is clustered and analyzed to identify abnormal clusters; the DBSCAN algorithm is based on a density-based clustering method and can identify abnormal clusters of any shape; Abnormal images are subjected to secondary inspection and manual review to ensure the accuracy of abnormal marking; the secondary inspection includes cross-validation using multiple detection algorithms, and the manual review is confirmed by experts in the metallographic field.

6. The AI-driven metallographic image analysis and classification method according to claim 1, characterized in that: Step 6 includes: Regularly back up and maintain the database to ensure data integrity and security; the backup includes incremental backup and full backup, and maintenance measures include data cleanup and optimization; A flexible query interface is provided to support multi-dimensional and multi-condition image and data retrieval; the query interface is based on SQL and NoSQL database technologies to support complex queries and real-time data analysis.

7. The AI-driven metallographic image analysis and classification method according to claim 1, characterized in that: Step 6 also includes: Provide report templates to support report generation for different user needs; the report templates include detailed reports and summary reports, and users can select appropriate templates according to their needs; The analysis results are displayed through a web interface, which supports online viewing and downloading of reports. The web interface adopts a responsive design and is adapted to different terminal devices, so that users can access and download reports anytime and anywhere.

8. The AI-driven metallographic image analysis and classification method according to claim 1, characterized in that: Step 7 includes: Retrain the model using the latest metallographic image data to update model parameters and weights; the retraining is based on online learning and incremental learning techniques to ensure that the model is always up to date; In combination with feedback from domain experts, the model structure and algorithm process are continuously improved to enhance the overall analysis performance; the improvements include optimizing the neural network hierarchy, adjusting hyperparameters, and modifying rules in combination with expert knowledge.

9. An AI-driven metallographic image analysis and classification system, applicable to the AI-driven metallographic image analysis and classification method according to any one of claims 1 to 8, characterized in that: include: Data acquisition module, image preprocessing module, feature extraction module, image classification module, anomaly detection module, data storage and management module, report generation module and user interface module; The output end of the data acquisition module is signal-connected to the input end of the image preprocessing module; The output end of the image preprocessing module is signal-connected to the input end of the feature extraction module, the output end of the feature extraction module is signal-connected to the input end of the image classification module, the output end of the image classification module is signal-connected to the input end of the abnormality detection module, the output end of the abnormality detection module is signal-connected to the input end of the data storage and management module, and the output end of the data storage and management module is signal-connected to the input end of the report generation module and the user interface module respectively.

10. The AI-driven metallographic image analysis and classification system according to claim 9, characterized in that: Data acquisition module: used to obtain high-resolution images of multiple metallographic samples through a microscope and record basic information of each sample, including sample type, processing technology and shooting parameters; Image preprocessing module: used to preprocess the collected metallographic images, the preprocessing includes denoising, graying, contrast enhancement and edge detection to obtain processed images; Feature extraction module: used to extract features from pre-processed metallographic images using convolutional neural networks, including deep features and traditional image processing features; Image classification module: used to classify the extracted image features using support vector machine or random forest algorithm to obtain classification results; Anomaly detection module: used to perform cluster analysis on feature data using clustering algorithms, identify and mark abnormal clusters, and ensure the accuracy of abnormal marking through multiple detection algorithms and manual review; Data storage and management module: used to store all metallographic images and their corresponding classification results, feature data and abnormal marks in the database, and regularly perform data backup and maintenance to ensure data integrity and security; Report generation module: used to automatically generate metallographic analysis reports based on classification results and abnormality detection results; the reports include detailed reports and summary reports. Users can select appropriate templates based on their needs. Reports can be output in PDF and HTML formats; User interface module: used to display analysis results through a web interface, supporting online viewing and downloading of reports. The web interface adopts a responsive design and is adapted to different terminal devices. Users can access and download reports anytime and anywhere.

Images