A method for constructing a corrosion damage multi-modal standard dataset

Abstract

The application discloses a kind of corrosion damage multi-modal standard data set construction methods, the method includes collecting multi-modal data, such as apparent damage image and microscopic metallographic image;Data cleaning and enhancement, corrosion feature labeling, feature extraction and selection, data modeling and verification, data set standardization and data set verification and the like processes, to construct high-quality, standardized corrosion damage data set.The standard data set constructed by the method provided in the application can be used for corrosion damage evaluation of industrial products in natural environment, significantly improve test efficiency and accuracy.

Description

Technical Field

[0001] This invention belongs to the field of industrial product testing technology, specifically relating to a method for constructing a multimodal standard dataset of corrosion damage. Background Technology

[0002] Long-term exposure of industrial products to natural environments can lead to surface damage (such as corrosion) and microstructural changes (such as metallographic corrosion), affecting their performance and safety. Traditional corrosion assessment methods rely on manual inspection, which suffers from high subjectivity and low efficiency. With the development of computer vision and materials analysis technologies, automated corrosion assessment based on multimodal data has become a trend. Existing dataset construction methods typically target only a single modality (such as surface or metallographic images) and lack dynamic correlation with environmental parameters, making it difficult for corrosion assessment models to reflect multi-scale damage mechanisms in real-world environments. Furthermore, a standardized solution for the spatiotemporal correspondence between surface and microscopic images has not yet been established, affecting the scientific validity and reusability of the datasets. Therefore, a standardized and operable method for constructing a multimodal standard dataset is urgently needed. Summary of the Invention

[0003] In view of this, the purpose of this invention is to provide a method for constructing a standard dataset of multimodal corrosion damage in industrial products. Through a systematic process of multimodal data acquisition, processing, annotation, and modeling, a high-quality, standardized dataset of corrosion damage is constructed.

[0004] The objective of this invention is achieved through the following technical solution: A method for constructing a standard dataset of multimodal corrosion damage includes: S1: Collect multimodal data in stages within the same physical region of the target sample, specifically including: In the natural environment stage, under preset climatic conditions, a high-resolution camera or industrial camera is used to take multi-angle pictures of the apparent damage area of ​​the target sample to obtain apparent damage images, and environmental parameters are recorded simultaneously, including temperature, humidity, and salt spray concentration. In the laboratory stage, the target sample is transferred to the laboratory, and after cutting and grinding the same apparent damage area, microscopic metallographic images are obtained using a metallographic microscope or scanning electron microscope. The apparent damage image, the corresponding microscopic metallographic image, and environmental parameters are bound and stored using a unique identifier. S2: Perform data cleaning and enhancement on the multimodal data to generate the first result; S3: Mark the corrosion damage on the first result and generate annotation data; S4: A deep learning model is used to extract corrosion features from the first result, and a feature importance evaluation method is used to screen the corrosion features. The screened corrosion features include macroscopic corrosion features corresponding to the apparent damage image and microscopic corrosion features corresponding to the microscopic metallographic image. S5: Construct a corrosion assessment model and train the corrosion assessment model using the selected corrosion features and corrosion damage annotations; S6: Standardize the dataset, including converting image data and annotation data into a uniform format and adding metadata about the image data; S7: Use statistical methods and machine learning models to verify the quality and consistency of the dataset; S8: Multimodal feature consistency verification.

[0005] Furthermore, the correspondence between the apparent damage image and the microscopic metallographic image in S1 is achieved in the following way: in the natural environment stage, the apparent damage area of ​​the target sample is marked with spatial coordinates; in the laboratory stage, the same area is located according to the spatial coordinate markings and microscopic images are acquired; and the spatiotemporal labels of the apparent image, microscopic image and environmental parameters are associated through the database.

[0006] Furthermore, in S2, data cleaning includes using image quality assessment algorithms and image filtering algorithms to remove blurry, overexposed, and underexposed images; data augmentation includes processing images through methods such as rotation, scaling, flipping, color adjustment, contrast enhancement, and sharpening, as well as synthesizing new images through generative adversarial networks.

[0007] Furthermore, corrosion damage annotation in S3 includes apparent damage annotation and microstructure annotation. Apparent damage annotation includes annotations such as corrosion type, corrosion rating and corrosion degree digital description for apparent damage images; microstructure annotation includes annotations such as corrosion depth and corrosion type for metallographic images.

[0008] Furthermore, training the corrosion assessment model in S5 includes optimizing the corrosion assessment model using cross-validation.

[0009] Furthermore, S7 specifically includes using statistical methods to examine the distribution and outliers of the data, and using machine learning models to verify the predictive power of the dataset.

[0010] Furthermore, in S8, a correlation analysis is performed on the macroscopic corrosion features extracted from the apparent damage image and the microscopic corrosion features extracted from the microscopic metallographic image; if the feature correlation is lower than a preset threshold, a manual re-inspection process is triggered.

[0011] Furthermore, the method also includes storing the dataset in a standardized database to support data querying and updating.

[0012] The beneficial effects of this invention are: The dataset construction method in this invention systematically integrates multimodal data such as apparent damage images, microscopic metallographic images, and environmental data, which can more comprehensively reflect the characteristics of corrosion damage and improve the accuracy and reliability of the assessment. Moreover, through the complementarity of multimodal data, the macroscopic and microscopic characteristics of corrosion can be better captured. This invention not only focuses on the apparent and microscopic characteristics of corrosion damage, but also introduces environmental data (such as temperature, humidity, and salt spray concentration) as auxiliary features. This allows for a better simulation of the impact of the natural environment on corrosion, providing more comprehensive data support for corrosion damage assessment. Through standardized processes such as data collection, cleaning and enhancement, feature annotation, modeling and validation, the high quality and reproducibility of the dataset are ensured, providing a solid foundation for subsequent research and applications. The dataset constructed by this method can be directly used to assess corrosion damage of industrial products in natural environments, significantly improving testing efficiency and accuracy.

[0013] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0014] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein: Figure 1 This is a schematic diagram illustrating the construction of a dataset according to an embodiment of this application. Detailed Implementation

[0015] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be understood that the preferred embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0016] Figure 1 This is a schematic diagram illustrating a method for constructing a standard dataset of multimodal corrosion damage, such as... Figure 1 As shown, it includes: S1: Collect multimodal data in stages within the same physical region of the target sample, specifically including: In the natural environment stage, under preset climatic conditions, a high-resolution camera or industrial camera is used to take multi-angle pictures of the apparent damage area of ​​the target sample to obtain apparent damage images, and environmental parameters are recorded simultaneously, including temperature, humidity, and salt spray concentration. In the laboratory stage, the target sample is transferred to the laboratory, and the same apparent damage area (i.e. the same physical area where damage exists) is cut and ground for pretreatment. Then, a metallographic image is obtained using a metallographic microscope or a scanning electron microscope. The apparent damage image, the corresponding microscopic metallographic image, and environmental parameters are bound and stored using a unique identifier (such as a QR code or laser mark). The target sample can be any industrial product; S2: Perform data cleaning and enhancement on the multimodal data to generate the first result; S3: Mark the corrosion damage on the first result and generate annotation data; S4: Use a deep learning model to extract corrosion features from the first result, and use a feature importance evaluation method to screen the corrosion features; S5: Construct a corrosion assessment model and train the corrosion assessment model using the selected corrosion features and corrosion damage annotations; S6: Standardize the dataset, including converting image data and annotation data into a uniform format and adding metadata about the image data; S7: Use statistical methods and machine learning models to verify the quality and consistency of the dataset; S8: Multimodal feature consistency verification.

[0017] In S1, the acquisition of multimodal data includes: under various climatic conditions (e.g., different temperatures, different humidity levels, different salt spray levels, etc.), using natural light or uniform artificial light sources, in natural environments (e.g., coastal areas, industrial areas, etc.), using high-resolution cameras or industrial cameras to capture images of the target sample from multiple angles to ensure image clarity and detail; in a laboratory environment, using metallographic microscopes or scanning electron microscopes to acquire microscopic metallographic images of the target sample, ensuring the resolution and detail of each metallographic image.

[0018] The correspondence between the apparent damage image and the micro metallographic image in S1 is achieved in the following way: In the natural environment stage, the apparent damage area of ​​the target sample is marked with spatial coordinates; in the laboratory stage, the same area (i.e., the apparent damage area) is located according to the spatial coordinate markings and micro images are acquired; the spatiotemporal labels of the apparent image, micro image (i.e., micro metallographic image) and environmental parameters are associated through the database.

[0019] For example, in the natural environment stage, high-precision positioning equipment (such as a laser coordinate system) can be used to spatially mark the apparent damage area of ​​the sample to ensure that the same area can be accurately located in the subsequent laboratory stage. In the laboratory stage, the sample is cut according to the marked position, leaving at least 5 mm of edge margin to avoid deformation of the damaged area, and the autofocus function of the metallurgical microscope is used to ensure image clarity. Then, the data from the two stages are linked, and the images and environmental parameters are stored using blockchain technology or a distributed database to ensure that the data is tamper-proof and traceable.

[0020] After acquiring multimodal data, it can be stored either locally or in the cloud to ensure data security.

[0021] In S2, data cleaning may include using image quality assessment algorithms and image filtering algorithms to remove blurry, overexposed, underexposed, and other defects from images in order to generate high-quality images.

[0022] For example, for images with apparent damage, image quality assessment algorithms (such as sharpness detection and brightness detection) can be used to remove low-quality images such as blur, overexposure, and underexposure, while retaining high-quality images; for microscopic metallographic images, image filtering algorithms (such as Gaussian filtering) can be used to remove low-quality images such as noise and blur.

[0023] After data cleaning, data augmentation can be performed. Data augmentation includes processing images through methods such as rotation, scaling, flipping, color adjustment, contrast enhancement, and sharpening, as well as synthesizing new images using generative adversarial networks to expand the dataset and achieve data diversity.

[0024] In S3, corrosion damage annotation includes surface damage annotation and microstructure annotation, which can be performed manually or semi-automatically using image annotation tools (such as LabelImg, VIA). Surface damage annotation includes digital descriptions of corrosion type, corrosion rating, and corrosion degree for surface damage images; microstructure annotation includes annotations of metallographic structure type, corrosion depth, and corrosion type for metallographic images.

[0025] For images of apparent damage, corrosion types can include uniform corrosion, pitting corrosion, cracks, etc.; corrosion rating can be determined based on corrosion area and corrosion depth, classifying corrosion levels into mild, moderate, and severe; digital descriptions of corrosion severity can include corrosion area percentage, corrosion depth value, etc.

[0026] For metallographic images, the corrosion depth can be measured using image analysis tools (such as ImageJ); the corrosion type can include grain boundary corrosion, intragranular corrosion, intergranular corrosion, etc.

[0027] In this step, annotation accuracy can be ensured through multi-person annotation and cross-validation.

[0028] In S4, a deep learning model is used to extract corrosion features, and these features are then screened using a feature importance evaluation method. For example, low-level features can be extracted from surface damage images using image processing techniques (such as edge detection and texture analysis), and then high-level features can be extracted using a deep learning model (such as a convolutional neural network). For microscopic metallographic images, image segmentation methods (such as U-Net) can be used to extract metallographic structures and corrosion regions, and then microscopic corrosion features can be extracted using a deep learning model. Next, the extracted corrosion features (including macroscopic corrosion features extracted from surface damage images and microscopic corrosion features extracted from microscopic metallographic images) are further evaluated for feature importance (such as random forest feature importance evaluation) to select the features most relevant to the corrosion rating and severity. The extracted features can be used for data distribution checks and subsequent model training and validation.

[0029] In S5, machine learning algorithms can be used to build corrosion assessment models, and the dataset can be divided into training and testing sets. Classification models (such as corrosion type classification models) and regression models (such as corrosion degree prediction models) are trained using the training set, and the corrosion assessment models are optimized through parameter tuning and cross-validation methods.

[0030] Next, the accuracy and reliability of the model are evaluated using a test set. Specifically, the model can be evaluated by calculating metrics such as classification accuracy and mean squared error, and the model accuracy can be improved through error analysis and iterative optimization.

[0031] In S6, dataset standardization can include data format standardization, data annotation standardization, and data storage standardization. Image data and annotation data can be converted into a unified format, and detailed metadata (such as data collection time, location, and environmental conditions) can be added to the dataset. The dataset can be stored in a standardized database to support efficient data querying and updating.

[0032] In S7, datasets are validated to ensure their quality and consistency. This includes using statistical methods to check for data distribution and outliers, and employing machine learning models to validate the dataset's predictive power. Validation reports are generated to document the dataset's accuracy and reliability.

[0033] In S8, a correlation analysis is performed on the macroscopic corrosion features (such as corrosion area and crack length) extracted from the surface damage image and the microscopic corrosion features (such as intergranular corrosion depth and metallographic structure type) extracted from the microscopic metallographic image; if the feature correlation is lower than the preset threshold, the manual re-inspection process is triggered.

[0034] Specifically, the multimodal feature consistency verification in S8 includes a comparison of macroscopic and microscopic corrosion features and a manual review rule. The comparison of macroscopic and microscopic corrosion features involves measuring the corrosion area ratio of the apparent damage image using ImageJ and performing linear regression analysis with the corrosion depth of the microscopic metallographic image (calculated through metallographic image grayscale analysis). The requirement is R² ≥ 0.85, where R², also known as the coefficient of determination, is a statistical indicator used to measure the fit of a model to the data and is used to judge the correlation between macroscopic and microscopic corrosion features. The condition for the manual review rule is that if the feature correlation does not meet the standard, at least two materials science experts must independently annotate the image, and the dataset must be updated based on the majority opinion.

[0035] After completing the above process, the constructed dataset can be published for research or development use.

[0036] The dataset described in this invention can be directly used for corrosion damage assessment of industrial products in natural environments, significantly improving testing efficiency and accuracy. Through automated data acquisition and model evaluation, it can greatly reduce labor and time costs. The constructed standard dataset will provide strong data support for corrosion damage assessment of industrial products, promoting the development of related research and applications. This dataset fills the gaps in existing corrosion damage datasets and provides an efficient and reliable solution for corrosion damage assessment of industrial products, possessing significant theoretical and practical value.

[0037] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for constructing a standard dataset of multimodal corrosion damage, characterized in that, include: S1: Collect multimodal data in stages within the same physical region of the target sample, specifically including: In the natural environment stage, under preset climatic conditions, a high-resolution camera or industrial camera is used to take multi-angle pictures of the apparent damage area of ​​the target sample to obtain apparent damage images, and environmental parameters, including temperature, humidity, and salt spray concentration, are recorded simultaneously; and the apparent damage area of ​​the target sample is marked with spatial coordinates. In the laboratory stage, the target sample is transferred to the laboratory, the same apparent damage area is located according to the spatial coordinate mark, and the same apparent damage area is pre-processed by cutting and grinding, and then a metallographic image is obtained using a metallographic microscope. The apparent damage image, the corresponding microscopic metallographic image, and environmental parameters are bound and stored using a unique identifier. S2: Perform data cleaning and enhancement on the multimodal data to generate the first result; S3: Mark the corrosion damage on the first result and generate annotation data; S4: A deep learning model is used to extract corrosion features from the first result, and a feature importance evaluation method is used to screen the corrosion features. The screened corrosion features include macroscopic corrosion features corresponding to the apparent damage image and microscopic corrosion features corresponding to the microscopic metallographic image. S5: Construct a corrosion assessment model and train the corrosion assessment model using the selected corrosion features and corrosion damage annotations; S6: Standardize the dataset, including converting image data and annotation data into a uniform format, adding metadata about the image data, and storing it. S7: Use statistical methods and machine learning models to verify the quality and consistency of the dataset; S8: Multimodal feature consistency verification, specifically including: performing correlation analysis on the macroscopic corrosion features corresponding to the apparent damage image and the microscopic corrosion features corresponding to the microscopic metallographic image; if the feature correlation is lower than the preset threshold, the manual re-inspection process is triggered. The correspondence between the apparent damage image and the microscopic metallographic image in S1 is achieved in the following way: In the natural environment stage, high-precision positioning equipment is used to spatially mark the apparent damage areas of the sample to ensure that the same area can be accurately located in the subsequent laboratory stage. In the laboratory stage, the sample is cut according to the marked position, leaving at least 5mm of edge allowance to avoid deformation of the damaged area, and the autofocus function of the metallurgical microscope is used to ensure image clarity. Then, the data from the two stages are correlated, and the images and environmental parameters are stored using blockchain technology or a distributed database to ensure that the data is tamper-proof and traceable. S4 specifically includes: extracting low-level features from surface damage images using image processing techniques, and then using a deep learning model to extract high-level features; for micro metallographic images, image segmentation methods can be used to extract metallographic structures and corrosion areas, and a deep learning model can be used to extract micro corrosion features; then, the extracted corrosion features are further evaluated for feature importance in order to select the features most relevant to the corrosion rating and degree.

2. The method for constructing a multimodal standard dataset of corrosion damage according to claim 1, characterized in that, In S2, data cleaning includes using image quality assessment algorithms and image filtering algorithms to remove blurry, overexposed, and underexposed images; data augmentation includes processing images through methods such as rotation, scaling, flipping, color adjustment, contrast enhancement, and sharpening, as well as synthesizing new images through generative adversarial networks.

3. The method for constructing a multimodal standard dataset of corrosion damage according to claim 1, characterized in that, Corrosion damage annotation in S3 includes surface damage annotation and microstructure annotation. Surface damage annotation includes annotations that digitally describe the corrosion type, corrosion rating, and corrosion degree for surface damage images. Microstructure annotation includes annotations that annotate the metallographic structure type, corrosion depth, and corrosion type for metallographic images.

4. The method for constructing a multimodal standard dataset of corrosion damage according to claim 1, characterized in that, Training the corrosion assessment model in S5 includes optimizing the corrosion assessment model using cross-validation.

5. The method for constructing a multimodal standard dataset of corrosion damage according to claim 1, characterized in that, S7 specifically includes using statistical methods to examine the distribution and outliers of the data, and using machine learning models to verify the predictive power of the dataset.

6. The method for constructing a multimodal standard dataset of corrosion damage according to claim 1, characterized in that, S8 specifically includes performing correlation analysis on the macroscopic corrosion features extracted from the surface damage image and the microscopic corrosion features extracted from the microscopic metallographic image; if the feature correlation is lower than a preset threshold, a manual re-inspection process is triggered.

7. The method for constructing a multimodal standard dataset of corrosion damage according to claim 1, characterized in that, The method also includes storing the dataset in a standardized database to support data querying and updating.

Images