Deep learning enabled automated detection and measurement system for anti-corrosion properties of coatings

Abstract

1. A system for evaluating corrosion resistance properties of a coating when applied to a corrosion-susceptible substrate, the system comprising: (i) an imaging unit configured to capture a plurality of grayscale images of the coating; (ii) a computer image processing unit configured to receive and reconstruct the plurality of captured grayscale images and output a reconstructed topographic image and a reconstructed color image; (iii) a data pre-processing unit configured to receive and combine the reconstructed topographic image and the reconstructed color image and output an image including high-dimensional data including one-dimensional height data and at least three-dimensional color data; and (iv) a corrosion detection unit configured to receive the high-dimensional data, recognize corrosion features, and output at least a location, classification, and area of ​​the corrosion features. a corrosion detection unit comprising a corrosion detection neural network having an input layer and an output layer, wherein input data to the input layer comprises high-dimensional data and output data from the output layer comprises at least location, classification, and area of ​​corrosion features; and (v) a computer-assisted data analysis unit configured to receive and analyze the data comprising at least location, classification, and area of ​​corrosion features and output a predicted grade of corrosion severity; a computer-implemented method for evaluating the corrosion resistance properties of a coating when applied to a corrosion-prone substrate; and a computing device having a computer image processing unit, a data pre-processing unit, a corrosion detection unit, and a computer-assisted data analysis unit disposed thereon.

Description

[Technical Field]

[0001] The present invention relates to a deep learning enabled automated detection and measurement system for corrosion damage, particularly suitable for the automated detection and quantitative assessment of the corrosion resistance properties of protective coatings. [Background technology]

[0002] Introduction Steel corrosion causes significant losses to the global economy and is a fundamental infrastructure issue. Protective coatings are widely used to prevent corrosion. Many protective coatings are offered to cover different corrosion resistance requirements, ranging from light industrial use to heavy-duty corrosion protection applications. This presents a challenge regarding how to quantify the corrosion resistance properties of protective coatings. The corrosion resistance properties of coatings are typically evaluated by first exposing coated panels to salt fog according to the ASTM B117-11 salt spray test to mimic a corrosive environment, and then visually inspecting and grading the corrosion damage by an operator. For example, rust and blister defects are graded by comparison with standard patterns defined in ASTM D610-08 and ASTM D714-02, respectively, and surface leakage defects are measured in millimeters according to ASTM D1654-08. Furthermore, such visual inspection and grading procedures performed by humans tend to be tedious, time-consuming, and subjective. Test results for corrosion resistance properties can also be easily misled by interference with the flow of the salt solution used in salt spray testing (also known as "rust bleed" or staining) and the insensitivity of the human eye to capturing small blisters and identifying their distribution.

[0003] Several imaging and machine vision systems and methods have been proposed for detecting corrosion defects using color digital camera imaging. However, none are directed to an autonomous and standardized process for detecting corrosion signatures and grading the corrosion-resistant properties of coatings when applied to metal substrates exposed to corrosive environments, such as salt spray. Therefore, it would be desirable to provide a system and method using deep learning for automated detection and quantitative evaluation of the corrosion-resistant properties of coatings when applied to corrosion-prone substrates. Summary of the Invention

[0004] The present invention employs a novel combination of computational imaging techniques, deep learning neural networks, and computer-assisted data analysis to provide a novel system and method for quantifying corrosion damage in coatings applied to corrosion-susceptible substrates (hereinafter "coated metal panels" or "coated panels") in the standardized ASTM B117-11 salt spray test using intelligent detection and automated imaging analysis. The system also enables a standardized approach for detecting and quantifying the corrosion-resistant properties of coatings on coated metal panels. The predicted corrosion severity ratings of the present invention have been validated by correlation with human grading results, which allows an autonomous process to predict accurate and reliable corrosion severity ratings compared to actual results obtained by visual inspection and human grading. The method of the present invention can effectively mitigate the interference of rust exudation on the surface caused by salt solution flow. The system of the present invention can also distinguish subtle differences between different corrosion features, such as rusting and blistering, that humans cannot distinguish through visual inspection. The method of the present invention can also significantly improve testing consistency and accuracy and reduce labor costs.

[0005] In a first aspect, the present invention provides a system for assessing the corrosion resistance properties of a coating when applied to a corrosion-prone substrate, comprising: (i) an imaging unit configured to capture a plurality of grayscale images of the coating; (ii) a computer image processing unit configured to receive and reconstruct the plurality of captured grayscale images and output a reconstructed topographic image and a reconstructed color image; (iii) a data pre-processing unit configured to receive and combine the reconstructed topographic image and the reconstructed color image and output an image comprising high-dimensional data including one-dimensional height data and at least three-dimensional color data; (iv) a corrosion detection unit configured to receive the high-dimensional data, recognize corrosion features, and output at least a location, a classification, and an area of ​​the corrosion features, the corrosion detection unit comprising a corrosion detection neural network having an input layer and an output layer, wherein input data to the input layer includes the high-dimensional data, and output data from the output layer includes at least a location, a classification, and an area of ​​the corrosion features; (v) a computer-aided data analysis unit configured to receive and analyze data including at least location, classification, and area of ​​corrosion features and output a predicted rating of corrosion severity.

[0006] In a second aspect, the present invention is a computer-implemented method for evaluating the corrosion resistance properties of a coating when applied to a corrosion-prone substrate, the method comprising: receiving a plurality of grayscale images of the coating; reconstructing the plurality of grayscale images by computer image processing and outputting a reconstructed topographic image and a reconstructed color image; combining the reconstructed topographic image and the reconstructed color image into an image comprising high-dimensional data including one-dimensional height data and at least three-dimensional color data; inputting high-dimensional data into a corrosion detection unit configured to recognize corrosion features and output at least a location, a classification, and an area of ​​the corrosion features, the corrosion detection unit comprising a corrosion detection neural network having an input layer that receives input data and an output layer that outputs output data, the input data including the high-dimensional data, and the output data including at least a location, a classification, and an area of ​​the corrosion features; receiving and analyzing data including at least the location, classification, and area of ​​the corrosion feature, and outputting a predicted rating of corrosion severity through computer-assisted data analysis.

[0007] In a third aspect, the present invention provides a computing device having disposed thereon a computer image processing unit, a data pre-processing unit, a corrosion detection unit, and a computer-assisted data analysis unit; a computer image processing unit configured to receive and reconstruct the captured grayscale images and output a reconstructed topographic image and a reconstructed color image; a data pre-processing unit configured to receive and combine the reconstructed topographic image and the reconstructed color image and output an image including high-dimensional data including one-dimensional height data and at least three-dimensional color data; a corrosion detection unit configured to receive the high-dimensional data, recognize corrosion features, and output at least a location, a classification, and an area of ​​the corrosion features, the corrosion detection unit comprising a corrosion detection neural network having an input layer and an output layer, wherein input data to the input layer includes the high-dimensional data, and output data from the output layer includes at least a location, a classification, and an area of ​​the corrosion features; The computer-aided data analysis unit is a computing device configured to receive and analyze data including at least the location, classification, and area of ​​the corrosion features and output a predicted rating of corrosion severity.

[0008] In a fourth aspect, the present invention is a process for training a neural network for corrosion detection, the process comprising: collecting a plurality of grayscale images of the set of coatings when applied to a corrodible substrate; reconstructing the captured grayscale images for each coating by computer processing and outputting a reconstructed topographic image and a reconstructed color image for each coating; combining the reconstructed topographic image and the reconstructed color image for each coating to output an image containing high-dimensional data for each coating, the high-dimensional data including one-dimensional height data and at least three-dimensional color data; obtaining at least the actual location, classification, and area of ​​corrosion features for each coating identified according to the quantified degree of rusting, degree of blistering, and maximum surface leakage; creating a training data set including a set of high-dimensional data and a set of data including at least actual locations, classifications, and regions of corrosion features for a set of coatings; training the neural network using the training data set, thereby obtaining a trained neural network. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 shows a schematic diagram of a system for evaluating the anti-corrosion properties of a coating when applied onto a substrate susceptible to corrosion. [Figure 2] 1 shows a flow chart of a method for evaluating the corrosion resistance properties of a coating on a corrosion-prone substrate according to one embodiment of the present invention. [Figure 3] 1 shows a schematic diagram of a multi-angle illumination imaging device according to an embodiment of the present invention. [Figure 4] 1 shows a schematic diagram of a multi-spectral illumination imaging device according to an embodiment of the present invention. [Figure 5]FIG. 1 is a schematic diagram of the principle of the shape-from-shading algorithm. [Figure 6] 1 shows a schematic flow chart for obtaining a topographic image from a multi-angle illumination image of a coated metal panel according to one embodiment of the present invention. [Figure 7] 1 shows a schematic diagram of color image reconstruction from multiple single-wavelength illumination images. [Figure 8] 1 shows a computer color image reconstructed from images of a coated metal panel captured by a multispectral illumination imaging device according to one embodiment of the present invention. [Figure 9] FIG. 1 shows a schematic diagram of high-dimensional data combined from the output of a computer image processing unit according to one embodiment of the present invention. [Figure 10] A schematic diagram of a typical U-Net neural network is shown. [Figure 11] 1 illustrates a model training process for a deep learning neural network according to one embodiment of the present invention. [Figure 12] 1 shows a schematic flow chart of a one-step deep learning neural network for corrosion recognition and classification according to an embodiment of the present invention. [Figure 13] 1 shows a schematic flow chart of a two-step deep learning neural network for corrosion recognition and classification according to an embodiment of the present invention. [Figure 14] 10 illustrates an example of a surface leakage calculation according to one embodiment of the present invention. [Figure 15] 1 illustrates an exemplary process for grading rust development on a coated metal panel according to one embodiment of the present invention. [Figure 16] 1 illustrates an exemplary process for grading rust development on a coated metal panel according to one embodiment of the present invention. [Figure 17] 1 illustrates a final output image from the inventive system according to one embodiment of the present invention. [Figure 18] 1 shows a schematic diagram of a cloud-based server cluster according to one embodiment of the present invention; [Figure 19] 1 shows the correlation results of a white paint coated metal panel evaluated by manual evaluation with the system of the present invention according to Example 1. [Figure 20] 1 shows the correlation results of a gray paint coated metal panel evaluated by manual evaluation with the system of the present invention according to Example 2. DETAILED DESCRIPTION OF THE INVENTION

[0010] Test methods, unless a date is given with the test method number, refer to the test method most recent as of the priority date of this document. Reference to a test method includes both a reference to the testing society and the test method number. The following test method abbreviations and identifiers apply herein: ASTM refers to ASTM International methods.

[0011] "Neural network" refers to an artificial neural network composed of artificial neurons or nodes and used to solve problems in artificial intelligence (AI). The connections of biological neurons are modeled in an artificial neural network as weights between nodes. Positive weights reflect excitatory connections, while negative values ​​imply inhibitory connections. All inputs are modified by the weights and summed. This activity is called a linear combination. Finally, an activation function controls the amplitude of the output.

[0012] "Machine learning" refers to a set of methods that "learn" from data to improve performance for a particular task. Machine learning algorithms build models based on historical data, also known as training data, to make predictions as the model output.

[0013] "Deep learning" is a type of machine learning in which a model is trained to perform classification tasks directly from images, text, or sounds. Deep learning is typically implemented using a neural network architecture. The term "deep" in deep learning refers to the number of layers in the network. The more layers, the deeper the network. Deep learning can involve more than two layers, or even hundreds of layers, in a neural network.

[0014] "Grayscale" in digital images means that each pixel's value represents only light intensity information. Grayscale images typically display only the darkest black to the brightest white. In other words, the image contains only black, white, and gray, with gray having multiple levels. In a grayscale image, each pixel has a value between 0 and 255, with zero corresponding to "black" and 255 corresponding to "white." The values ​​between 0 and 255 are various shades of gray, with values ​​closer to 0 being darker and values ​​closer to 255 being lighter.

[0015] "Image segmentation" refers to a technique used in digital image processing and analysis to divide an image into parts or regions, often based on the characteristics of the pixels in the image.

[0016] As used herein, "coating" (interchangeable with "coating film") refers to a film or coating film formed by applying a coating composition to a substrate and drying the coating composition or allowing it to dry.

[0017] As used herein, "coated metal panel" (interchangeably referred to as "coated panel") refers to a coating applied onto a corrosion-susceptible substrate. "Corrosion-susceptible substrate" refers to a substrate susceptible to corrosion, such as a metal substrate, preferably a steel substrate.

[0018] An image of a coating, coating sample, or coated panel refers to an image of the surface of the coating, coating sample, or coated panel to which the coating has been applied.

[0019] The "corrosion resistance properties" of a coating are typically characterized by one or more corrosion characteristics.

[0020] A "corrosion feature" (interchangeably with "corrosion defect") of a coating means a defect feature in the coating caused by corrosion of a corrosion-susceptible substrate to which the coating is applied. Corrosion features can include features that are useful for grading corrosion severity, including defect location, type (i.e., classification), size (e.g., length and / or width), number, density (e.g., distribution) of the defect. Classifications of corrosion features or corrosion defects herein include rust defects, blister defects, surface leaks, or combinations thereof.

[0021] The "corrosion severity rating" may include the severity of rusting, the severity of blistering, the maximum surface leakage, or a combination thereof; in particular, the ratings defined in the following ASTM standards for evaluating the corrosion severity of coatings when applied to corrosion-susceptible substrates. For example, the severity of rusting is based on the degree of rusting quantified, such as in accordance with ASTM D610-08 (Standard Practice for Evaluating the Degree of Rust on Painted Steel Surfaces), which may include a rust grade specified by the size of the rusted area (e.g., by percentage of the rusted surface area) and the type of rust distribution on the coating. The severity of blistering is based on the degree of blistering quantified, such as in accordance with ASTM D714-02 (Standard Test Method for Evaluating the Degree of Blistering of Paints), which may include the size and frequency (e.g., density) of blisters on the coating. Maximum surface leakage refers to the maximum corrosion width in millimeters (mm) from the scribe on a coated metal panel, for example, according to ASTM D1654-08 (Standard Test Method for Evaluation of Painted or Coated Specimens Exposed to Corrosive Environments).

[0022] Through a combination of various hardware and algorithms, the present invention can realize fully automated evaluation of the corrosion-resistant properties of coatings applied to corrosion-prone substrates (hereinafter "coated metal panels") in accordance with ASTM standards. The present invention significantly improves on the drawbacks associated with manual evaluation, such as accuracy and time consumption. The surface of a coated metal panel after exposure to a corrosive environment, such as salt spray, has a three-dimensional surface structure, but has little or no contrast between light and dark areas, resulting in visual noise. In particular, the onset of corrosion on a coated metal panel is difficult to detect by visual inspection. Compared to conventional machine vision systems for corrosion detection that directly use images acquired by a conventional digital color camera as input for machine learning, the present invention uses computer imaging processing of grayscale images to correctly reconstruct surface defects in the coating caused by corrosion. This allows for the acquisition of more accurate surface height maps and color information with higher discrimination during the data collection stage, which are essential for the subsequent recognition and quantification of corrosion features. The resulting reconstructed topographic and color images are then combined into an image containing high-dimensional data. Using such high-dimensional data as input for training a corrosion detection neural network improves the accuracy of corrosion feature detection and recognition, enabling the present invention to distinguish actual rust formation from rust exudation on the coating surface (thereby mitigating the interference of rust exudation with the corrosion rating), and can provide sufficient corrosion feature data for a quantified corrosion severity rating in subsequent computer-assisted data analysis. Thus, the systems and methods of the present invention enable an autonomous process to detect corrosion defects even in the early stages of corrosion damage and to provide a quantitative rating of corrosion severity that is verified by correlation with human grading results.

[0023] 1 shows a schematic diagram of a system for evaluating the corrosion resistance properties of a coating on a corrosion-prone substrate ("coated panel" or "coating sample") according to one embodiment of the present invention. The system comprises an imaging unit 101, a computer image processing unit 102, a data pre-processing unit 103, a corrosion detection unit 104, and a computer-aided data analysis unit 105. The imaging unit 101 acquires multiple grayscale images of the coating surface of the coated panel as input, and the computer-acid data analysis unit 105 outputs a corrosion severity rating.

[0024] FIG. 2 shows a flowchart of a method for evaluating the corrosion resistance properties of a coating when applied to a corrosion-susceptible substrate according to one embodiment of the present invention. The method includes image acquisition 201, computerized image processing 202, data preprocessing 203, corrosion detection neural network 204, and computer-aided data analysis 205. Image acquisition 201 includes multi-angle illumination imaging and multi-spectral illumination imaging, and the resulting images are processed through topographic image reconstruction and color image reconstruction, respectively, in computerized image processing 202. The images obtained after computerized image processing 202 are combined into an image containing high-dimensional data through data preprocessing 203. The high-dimensional data is then input into corrosion detection neural network 204. Corrosion detection neural network 204 is trained by model training to output segmentation results of corrosion defects, including the location, classification, and area of ​​corrosion features. These corrosion features are quantitatively analyzed in computer-aided data analysis 205, which then outputs a predicted rating of corrosion severity.

[0025] Imaging unit and image acquisition The system of the present invention includes an imaging unit useful for image acquisition. The imaging unit is configured to capture multiple images of a coating applied to a corrosion-prone substrate ("coated metal panel" or "coated panel"). The imaging unit typically enables the use of computer imaging, including, for example, multi-angle imaging, such as multi-angle illumination imaging, multispectral imaging, such as multi-spectral illumination imaging, or a combination thereof. The imaging unit includes an imaging device. The imaging device typically includes a programmed illumination device and a camera. For example, the imaging device may include a multi-angle illumination imaging device and a multispectral illumination imaging device. Alternatively, the imaging device may include a device capable of both multi-angle illumination imaging and multispectral illumination imaging. The camera in the imaging device may be any grayscale camera, such as a grayscale industrial camera with over 10 million pixels (e.g., 500 million pixels or more), and a data transfer and computer control interface. Desirably, the multiple grayscale images of the coating include images acquired through both multi-angle imaging and multispectral imaging, more desirably, images acquired through multispectral illumination imaging and images acquired through multi-angle illumination imaging.

[0026] Multi-angle lighting imaging Multi-angle or multiple directions means four or more directions, and can be five or more, six or more, or eight or more directions, preferably four to six directions. "Multi-angle illumination imaging" refers to imaging with a multi-angle illumination device. A multi-angle illumination device is a device that can illuminate a sample with light from multiple directions to cast directional shadows around raised or depressed features on the sample.

[0027] An imaging device with a multi-angle illumination device may also be referred to as a multi-angle illumination imaging device. An exemplary multi-angle illumination imaging device typically includes a single camera for capturing multiple images of a sample, i.e., a coating surface, illuminated by multiple light sources. The illumination sources may be a ring light with four 90-degree quadrants, an array of four bar lights, or any other configuration that produces multiple directional lighting.

[0028] A multi-angle illumination imaging device can be used to capture multiple images by projecting a segmented light array from multiple angles. The multiple images can then be used to generate a reconstructed topographic image in a computer image processing unit (described below) configured to obtain a shadow image in a process called "shape from shading." Multi-angle illumination imaging can highlight the three-dimensional surface structure of a sample, which is particularly suitable for detecting minute corrosion defects on the surface of a coating sample and for three-dimensional (3D) surface reconstruction. Suitable examples of multi-angle illumination imaging devices include the LSS-2404 available from CCS America Inc. and the CV-X series available from Keyence Corporation.

[0029] By using the multi-angle illumination imaging device in combination with a computer image processing unit (described below), visually noisy or highly reflective surfaces (such as glass) can be easily inspected. This is particularly effective for coating surfaces that have 3D structure but little or no contrast. FIG. 3 shows a schematic diagram of a multi-angle illumination imaging device 300 according to one embodiment of the present invention. The multi-angle illumination imaging device 300 with different illumination angles includes a camera and one or more illumination devices, e.g., Light 1, Light 2, Light 3, and Light 4, that emit light in different directions. The surface plane of the coating sample is illuminated one by one by lights from different directions so that multi-angle images of the coating sample with the same sequence are captured by the camera. Thus, multiple images of the surface of the coating sample (or coated panel) are acquired by the multi-angle illumination imaging device, which may also be referred to as "multi-angle illumination images."

[0030] Multispectral lighting imaging "Multispectral" refers to four or more wavelengths, for example, 4 to 16 wavelengths or 4 to 8 wavelengths. "Multispectral lighting imaging" refers to imaging using a multispectral lighting device. A multispectral lighting device is an illumination device with multiple light sources with different specified wavelengths. A multispectral lighting device with light from eight or more spectral channels can obtain more accurate color information than a conventional digital color camera and can be used to distinguish different classes of corrosion features on a coating surface based on such color information. "Digital color camera" refers to a common industrial and home color camera with a Bayer filter. A Bayer filter is a color filter array (CFA) that arranges RGB color filters on a square grid of a photosensor.

[0031] An imaging device equipped with a multispectral illumination device is also referred to as a multispectral illumination imaging device. Compared to hyperspectral imaging devices, such devices can provide a more simplified and practical solution for industrial imaging applications. A combination of light source wavelengths (e.g., LED sources) can be selected. For example, an LED light source with eight spectral channels and a monochrome grayscale camera can be used. Images captured by a multispectral illumination imaging device include four or more spectral image planes, representing four or more spectral channels, such as ultraviolet (405 nanometers (nm)), blue (457 nm), green (527 nm), orange (600 nm), red (660 nm), far-red (730 nm), infrared (860 nm), or white (600 nm), with all wavelength values ​​being approximate peak wavelength values. These images acquired by the multispectral illumination imaging device can be combined to produce a reconstructed color image in a computer image processing unit, as described below.

[0032] Under illumination by the light source of each spectral channel, a corresponding signal intensity image of the coating sample in the spectral region can be obtained. For example, when imaging is completed sequentially under illumination by eight spectral channels, eight response intensity images of the coating sample under the eight spectral channels are captured. The intensities of the eight spectral positions to the corresponding RGB or Lab color values ​​can then be reconstructed into a color image using a computer image processing unit, as described below. Thus, multiple images captured by a multispectral illumination imaging device can include response intensity images under the spectral channels. Alternatively, a grayscale camera can be used with a light source that includes multiple independently controlled spectral lighting channels. As the light source cycles through each individual spectral channel, subsequent images are captured. Each of these images corresponds to the reflectance of the coating for each individual spectral lighting. These images can then be combined and reconstructed into a single color image.

[0033] Using multispectral lighting imaging, designated corrosion features can also be distinguished from others on the image depending on how they respond to various spectral lighting. The image resolution of multispectral lighting imaging matches the full pixel resolution of the grayscale camera in the multispectral lighting imaging device. In contrast, when using conventional digital color camera imaging, multiple adjacent pixels are combined to separate colors, resulting in an image with relatively low resolution. Therefore, multispectral lighting imaging provides higher resolution at lower cost and less system complexity compared to digital color camera imaging. Suitable examples of multispectral lighting imaging devices include the HPR2 series available from CCS America Inc. and the CA-DRM10X available from Keyence Corporation.

[0034] 4 shows a schematic example of a multi-spectral illumination imaging device according to one embodiment of the present invention. The multi-spectral illumination imaging device 400 includes a camera and one or more illumination devices that emit light at different wavelengths, e.g., light 1, light 2, light 3, light 4, light 5, light 6, light 7, and light 8. Thus, multiple images of the coating sample are acquired by the multi-spectral illumination imaging device, which may also be referred to as a "multi-spectral illumination image."

[0035] Preferably, the plurality of grayscale images of the coating (used for computer image processing) for input to the computer image processing unit described below includes multi-angle illumination images and multi-spectral illumination images.

[0036] Computer image processing unit and computer image processing The system of the present invention also includes a computer image processing unit useful for computer image processing. The computer image processing unit is configured to reconstruct the captured grayscale images of the surface of the coating sample and output a reconstructed topographic image and a reconstructed color image. The computer image processing can reconstruct the images using computer vision techniques.

[0037] (A) Reconstruction of topographic images Multiple grayscale images of a coating sample ("original images") may include images captured from different illumination angles, e.g., images acquired by a multi-angle illumination imaging device, which may be reconstructed into a topographic image using surface height map information by a shape-from-shading algorithm. The shape-from-shading algorithm may employ a height-driven process, in which features without surface color or height are removed, and a surface image calculated based on the shading information is output. Reconstruction of a topographic image using the surface height map Z(x,y) may be performed in two steps by the shape-from-shading algorithm. In the first step, gradients in the x and y directions, denoted "p" and "q," respectively, may be calculated from the captured grayscale images. In the second step, the topographic image is derived by integrating the gradients.

[0038] Figure 5 shows a schematic diagram of the principle of the "shape-from-shading" algorithm. The intensity recorded by the camera depends primarily on the angle between the direction vector of the incident light ("s") and the normal vector ("n") of the observed surface element.

[0039] Generally, the z-axis of the real-world coordinate system can be chosen to coincide with the optical axis of the camera. The image plane is therefore parallel to the xy-plane of the coordinate system. The intensity I(x,y) registered by the camera depends only on the following parameters: the sensitivity of the camera sensor ("c"), the direction vector ("s") and intensity ("Q") of the incident telecentric light, and the fraction of light reflected towards the camera ("R(n,s)"); I(x,y)=c Q(x,y,z) R(n,s)

[0040] The reflectance map R(n,s) describes all the details of the reflection of light. R depends on the material- and position-dependent reflectance coefficients ("r") for diffuse and specular reflection, and on the angle ("θ") between the direction vector of the incident light ("s") and the unknown normal ("n") of the considered surface element. Therefore, considering only diffuse reflection (Lambert's law), we have: R(n,s)=r·cosθ, where cosθ=n·s

[0041] The result is independent of the camera's viewing direction. To suppress specular reflections, the camera and lamp must be positioned so that the specular light is refracted as far away from the camera lens as possible, following the rule "angle of incidence = angle of reflection".

[0042] The intensity recorded at each pixel of the camera corresponding to a volume element of the surface is therefore described by: I(x,y)=c·Q·r·n·s=ρ·n·s (where albedo ρ=c·Q·r)

[0043] Under these assumptions, the effects of c, Q, and r cannot be separated, so they are combined into a single variable known as albedo.

[0044] The normal vector "n" of a surface element is defined by the gradient of the surface Z(x,y) in the x and y directions, i.e., by the partial derivatives of Z(x,y) with respect to x and y.

[0045]

number

[0046] To eliminate ρ and calculate the gradients p and q, at least three independent equations, and therefore at least three photographs for different light directions, are required. Improved results and estimation of measurement errors can be achieved if more than three photographs are captured, thus leading to an overdetermined system of linear equations. After solving the system of equations using standard methods, the topographic image is calculated by integrating over p and q (for further details of the shape-from-shading algorithm, see B.K.P. Horn and M.J.Brookes (eds.), Shape from Shading, MIT Press 1989).

[0047] In corrosion resistance tests for coatings applied to corrosion-prone substrates (e.g., metals), corrosion often causes changes in the coating's surface morphology, such as uneven defects formed on the coating's originally flat surface. Computer image processing using shape-from-shading can rapidly characterize uneven corrosion defects on large-area samples in less time than 3D surface scanning tests, providing resulting topographic images with similar quality to traditional three-dimensional surface scanning tests (e.g., 3D laser scanning measurements, which can plot a 3D map of the sample surface step by step).

[0048] FIG. 6 shows a schematic flow chart for obtaining a topographic image from multi-angle illumination images of a coated metal panel according to one embodiment of the present invention. A coated steel panel 601 (i.e., a coating applied to a steel panel) after exposure to an ASTM B117-11 salt spray test is provided. Multiple grayscale images 602 are captured using a multi-angle illumination imaging device with lighting from different angles, where "Normal" represents all lights on, "Upper" represents the upper position lights on, "Left" represents the left position lights on, "Lower" represents the lower position lights on, and "Right" represents the right position lights on. The resulting multi-angle images are then processed by a shape-from-shading algorithm to obtain a reconstructed topographic image 603.

[0049] (B) Color image reconstruction The multiple grayscale images of the coating ("original images") can include images at different wavelengths, such as images acquired by the multispectral illumination imaging device described above. For example, eight images can be collected for eight different wavelengths. Reconstruction of a color image from the multispectral illumination images can be performed using a spectral reconstruction algorithm. The spectral reconstruction algorithm can be based on the following process: for each image at a specified wavelength, the grayscale level represents the intensity of reflected light; based on these data, a reflectance spectral curve can be generated for each pixel, a color value for each pixel can be calculated from the reflectance spectral curve, and a color image can then be reconstructed from the color value for each pixel.

[0050] The CIE 1931 color space, created by the International Commission on Illumination (CIE) in 1931, can be used for image reconstruction. For each pixel, the XYZ tristimulus values ​​can be calculated from the spectral data. According to the definition of tristimulus values, assuming that the spectral distribution function of the reflected light of a certain object is φ(λ), and the spectral tristimulus function is to decompose φ(λ) according to the spectral tristimulus values, we can obtain the tristimulus values ​​corresponding to each wavelength, and then integrate them over the entire visible light band to obtain the color tristimulus values ​​as shown in the following formula:

[0051]

number

[0052]

number

[0053] The resulting X, Y, Z values ​​are then substituted into the following equation to obtain the R, G, and B stimulus values ​​of the target reflectance spectrum as seen by the human eye:

[0054]

number

[0055] By calculating the chromaticity coordinates, the location of the color light in the CIE color space can be obtained (for further details, see http: / / www.brucelindbloom.com / index.html?Eqn_Spect_to_XYZ.html).

[0056] Figure 7 shows a schematic diagram of color image reconstruction from multiple single-wavelength illumination images. Eight grayscale images are generated from a multispectral illumination imaging device 702 having eight spectral channels 7011-7018 with wavelengths of 450 nm, 475 nm, 495 nm, 525 nm, 545 nm, 580 nm, 620 nm, and 670 nm, respectively. For each pixel (x, y) (x and y refer to the pixel's location on the image), such as pixel (0, 0), a reflectance spectral curve 702 with eight data points can be generated. The color value of such a pixel can then be calculated from its reflectance spectral curve. After the color values ​​for every pixel along with the location for each pixel are generated, a color image 703 is reconstructed with 24 colors, each of which represents a different color in a grid of 24 colors.

[0057] 8 shows a computerized color image reconstructed from images of a coated panel captured by a multispectral illumination imaging device according to one embodiment of the present invention. Eight grayscale images 801 from eight spectral lightings are processed in a computerized image processing unit using a spectral reconstruction algorithm to obtain one reconstructed color image 802.

[0058] The reconstructed color image allows for the distinction of smaller color differences present on the surface of the coated panel compared to images captured by a conventional digital color camera.

[0059] Preferably, a multi-angle illumination imaging device and a shape-from-shading algorithm are used to obtain a surface height map of the coated panel, thereby providing a reconstructed topographic image, and then a multi-spectral illumination imaging device and a spectral reconstruction algorithm are used to obtain color information on the sample surface, thereby providing a reconstructed color image.

[0060] Data Pre-processing Unit and Data Pre-processing The system of the present invention also includes a data pre-processing unit useful for pre-processing data output from the computer image processing unit. The data pre-processing unit is configured to combine the reconstructed topographic image and the reconstructed color image into an image comprising high-dimensional data. That is, the high-dimensional data is generated from the output of the computer image processing unit. The high-dimensional data is then used as input to the corrosion detection unit, which will be described later.

[0061] "High-dimensional data" herein refers to data with at least four dimensions. High-dimensional data in the present invention includes one-dimensional surface height data and at least three-dimensional color data (e.g., three or more color dimensions for each pixel on the sample image), and can have four or more, five or more, nine or more, or even ten or more color dimensions. The high-dimensional data, i.e., the combined image data, is used as a combined input for the corrosion detection unit described below. For each pixel on the sample image, a deep learning neural network in the corrosion detection unit described below uses the color data and surface height data together.

[0062] The reconstructed topographic image obtained from the computer processing unit may be processed by grayscale values ​​representing surface height values ​​and converted into one-dimensional data for surface height. The reconstructed color image obtained from the computer processing unit may be processed by reflective intensities of the sample when illuminated by different spectral wavelengths and converted into at least three-dimensional data for color. The at least three-dimensional color data and one-dimensional surface height data are then combined into high-dimensional data by a matrix addition operation.

[0063] 9 is a schematic diagram of high-dimensional data combined from the output of a computer image processing unit according to one embodiment of the present invention. A reconstructed color image 9011 is processed by reflectance intensities in red, green, and blue (RGB) spectral channels and converted into three-dimensional data of color 9021 (numbers representing reflectance intensities). A reconstructed topographic image 9012 is processed and converted into one-dimensional data of surface height 9022 (numbers representing reflectance intensities). The three-dimensional color data and one-dimensional surface height data are combined into four-dimensional data 903 (numbers representing reflectance intensities) by a matrix addition operation.

[0064] Combining the data in the reconstructed topographic image with the data in the reconstructed color image, thereby forming high-dimensional data, enables the present invention to distinguish actual rust formation from rust exudation on the coating surface, which refers to color staining on the coating surface caused by, for example, the flow of salt solution in the ASTM B117-11 salt spray test.

[0065] Corrosion detection unit and corrosion detection and recognition The system of the present invention also includes a corrosion detection unit useful for detecting corrosion features. The corrosion detection unit is configured to receive high-dimensional data, recognize corrosion features, and output at least the location, classification, and area of ​​the corrosion features. The corrosion detection unit may be a one-step deep learning unit including a corrosion detection neural network. The term "corrosion detection neural network" used herein refers to a neural network for recognizing corrosion defects. The corrosion detection neural network has an input layer and an output layer. Input data to the input layer includes the high-dimensional data obtained above. Output data from the output layer includes data on at least the location, classification, and area of ​​the corrosion features.

[0066] The corrosion detection neural network realizes image segmentation, particularly semantic segmentation, for different corrosion defects on images containing high-dimensional data. Image segmentation is an image processing method that divides an image into different parts according to its features and characteristics. Semantic segmentation is a type of image segmentation that assigns a class to every pixel in a given image. Compared to other types of image segmentation that aim to group similar regions of an image, semantic segmentation is particularly useful for quantifying corrosion features such as location, classification, or size using deep learning.

[0067] A U-Net neural network can be used to realize semantic segmentation functions for defined corrosion features within a corrosion detection neural network. (Further details of the U-Net neural network can be found in Ronneberger, Olaf, Philipp Fischer, and Thomas Brox, "U-net: Convolutional networks for biomedical image segmentation," International Conference on Medical image computing and computer-assisted intervention, Springer, Cham, 2015.) The U-Net neural network complements a regular contraction network with successive layers, where pooling operations are replaced by upsampling operators. These layers thus increase the resolution of the output. Successive convolutional layers can then be trained to assemble accurate outputs based on the results of the upsampling operations. Compared to fully convolutional networks, one modification in the U-Net neural network is that the upsampling section has many feature channels, which allows the network to propagate contextual information to higher-resolution layers. As a result, the expansion path is symmetric with respect to the contraction section, resulting in a U-shaped architecture. The U-Net neural network uses only the appropriate portion of each convolution without any fully connected layers. Figure 10 shows a schematic diagram of a typical U-Net neural network used in corrosion detection neural networks.

[0068] To further improve the prediction accuracy of the present invention, the corrosion detection unit can also include a region of interest (ROI) neural network before the corrosion detection neural network. An "ROI neural network" refers to a neural network for recognizing a region of interest. The ROI neural network is configured to receive high-dimensional data as input data and output an ROI recognition result. Based on the ROI recognition result, a boundary of the region of interest is extracted from the high-dimensional input data, thereby obtaining ROI-extracted data (i.e., high-dimensional data after ROI extraction). The ROI-extracted data is then input to the corrosion detection neural network, which can predict at least the location, classification, and area of ​​a corrosion defect as output.

[0069] Model training process The corrosion detection neural network may include a model training unit useful in a model training process that is used to train a deep learning neural network using a training dataset that uses training coated panels (also referred to as "training coatings").

[0070] The corrosion detection training dataset is used to train a corrosion detection model deployed on a corrosion detection neural network, thereby forming a trained corrosion detection model deployed on the trained corrosion detection neural network capable of predicting at least the location, classification, and area of ​​corrosion defects. The corrosion detection training dataset includes a set of high-dimensional data for training coated panels and a set of data containing corresponding actual corrosion features. The high-dimensional data for the training coated panels is obtained by (i) collecting multiple grayscale images of each training coated panel, (ii) reconstructing the captured grayscale images for each training coated panel and outputting a reconstructed topographic image and a reconstructed color image for each training coated panel by computer image processing, and (iii) combining the reconstructed topographic image and the reconstructed color image for each training coated panel, thereby forming an image containing high-dimensional data for each training coated panel, including one-dimensional height data and at least three-dimensional color data. Steps (i), (ii), and (iii) can be performed as described above in the sections on the imaging unit, the computer image processing unit, and the data pre-processing unit. Actual corrosion features, including their actual locations, classifications, and areas, on such training coated panels can be identified according to quantified rusting severity, blistering severity, and maximum surface leakage, or according to ASTM D610-08, ASTM D714-02, and ASTM D1654-08. For example, these corrosion features can be labeled by humans through visual inspection of the training coated panels. An experienced laboratory operator can use the LabelMe tool to manually label areas of corrosion features through visual inspection of the training coated panels, using different label classes for different classifications of corrosion defects, such as rusting, blistering, and surface leakage, according to ASTM D610-08, ASTM D714-02, and ASTM D1654-08, respectively.LabelMe is a graphic image annotation tool developed by the Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology (MIT), USA, located at http: / / labelme.csail.mit.edu. In one embodiment, surface leakage is labeled as one class, and blistering and rusting are labeled as separate classes for training coated panels.

[0071] The ROI training dataset is used to train an ROI model deployed on an ROI neural network, thereby forming a trained ROI model deployed on the trained ROI neural network that can predict regions of interest (ROIs). The ROI training dataset includes a set of high-dimensional data of training coated panels and a set of data including corresponding ROIs ("actual ROIs") of such training coated panels labeled by humans. The high-dimensional data for the training coated panels is obtained as described above. The regions of interest of the training coated panels are labeled by humans through visual inspection of the training coated panels, for example, using the LabelMe tool.

[0072] The size of the training dataset is important for the predictive accuracy of a neural network model. Generally, a large training dataset (more than 10,000 samples) can significantly improve the accuracy of model predictions. FIG. 11 illustrates a model training process for a deep learning neural network according to one embodiment of the present invention. The training process uses a training dataset that uses multiple training samples (e.g., 20 samples), referred to as the "original training dataset," which is a small training dataset 1101. The small training dataset 1101 includes a small image dataset (i.e., high-dimensional data) and a small labeled dataset (i.e., human-labeled corrosion features or human-labeled ROIs). The small training dataset 1101 can be further expanded to 1,000-2,000 times the number of datasets using a data augmentation algorithm 1102, which can then obtain a training dataset of tens of thousands of photos to form an augmented training dataset 1103, which includes the corresponding augmented image dataset and augmented labeled dataset. The augmented training dataset 1103 is used to train a neural network 1104 to obtain a trained neural network model 1105. Data augmentation in data analysis is a technique used to increase the amount of data by adding slightly modified copies of existing data or newly created synthetic data from existing data. Data augmentation can act as a regularization term and help reduce overfitting when training a deep learning model. Data augmentation can increase the diversity of data available for training a model without actually collecting new data. The training dataset used in the present invention can first be augmented using a data augmentation algorithm before training the neural network to further improve the accuracy of the model. Commonly used data augmentation techniques such as detexturing, decolorizing, cropping, extrapolation, and horizontal flipping can be used when training a neural network.

[0073] During the model training process, 10% to 30% of the original training dataset can be randomly selected as a cross-validation dataset, and the remainder is used as a new training dataset. This new training dataset is used to train the U-Net neural network. At the end of each iteration, the cross-validation dataset is used to validate the model. If the accuracy is not high enough (e.g., above 95%), the model training unit continues to optimize the next iteration. After a certain number of iterations (e.g., 300 to 500 iterations), if the model's prediction accuracy on the cross-validation dataset reaches a relatively high accuracy, e.g., above 95%, the training iterations can be stopped, and a trained model is produced.

[0074] Forecasting Process The corrosion detection unit of the present invention may also comprise a prediction unit, comprising one or more neural networks deployed with trained models, capable of predicting or outputting at least the location, classification, and size of corrosion defects and / or areas of concern upon receiving input high-dimensional data.

[0075] The trained corrosion detection model and the trained ROI model obtained from the training process can be deployed in a corrosion detection neural network and a trained ROI neural network, respectively. A neural network deployed using the trained corrosion detection model is also referred to as a "trained corrosion detection neural network." A neural network deployed using the trained ROI model is also referred to as a "trained ROI neural network." When receiving high-dimensional data of a coated panel, the trained corrosion detection neural network can recognize corrosion defects and output corrosion defect segmentation results (also as corrosion defect recognition results) including data including at least the location, classification, and area of ​​the corrosion defects for such coated panel. When receiving high-dimensional data of a coated panel, the trained ROI neural network can recognize regions of interest and output ROI segmentation results. The corrosion detection neural network and the region of interest neural network can each independently use a U-Net neural network.

[0076] When the system acquires new grayscale images, for example, images of a test coating, these new images are processed into high-dimensional data (as input data) including surface height map data and color data for these new images, and the high-dimensional data is input into a trained neural network, which can then generate predicted results within seconds and mark the predicted results on the image. The prediction process can be performed by a deep learning neural network, for example, a U-Net neural network.

[0077] FIG. 12 shows a schematic flowchart of a one-step deep learning neural network for corrosion recognition and classification according to one embodiment of the present invention. In this specification, a one-step deep learning neural network means that only one trained corrosion detection neural network is used. High-dimensional data 1202 for a new grayscale image is directly input into a trained corrosion detection neural network 1204 (i.e., a neural network developed using a trained corrosion detection model), which then outputs corrosion defect recognition results 1206, including location and area data 1206A for surface leaks and location and area data 1206B for rust and blister occurrences (output data). Therefore, the location, classification, and area of ​​corrosion defects can be predicted by the trained corrosion detection neural network.

[0078] The corrosion detection unit may or may not further include a trained ROI neural network. If there is a clear background around the coated panel when the grayscale image is captured, the system can use a one-step deep learning neural network without requiring an ROI neural network for region of interest recognition. Figure 13 shows a schematic flowchart of a two-step deep learning neural network for corrosion recognition and classification according to one embodiment of the present invention. For example, the corrosion detection unit 1300 includes an ROI neural network trained in an ROI recognition step 1301, followed by a corrosion detection neural network trained in a corrosion recognition step 1302. The trained ROI neural network receives input data and outputs ROI recognition results. Based on the ROI recognition results, the boundaries of the region of interest are extracted from the high-dimensional input data, thereby obtaining ROI-extracted data, e.g., high-dimensional data after ROI extraction. The ROI-extracted data is then input to the trained corrosion detection neural network as a second step, following the same procedure as the one-step deep learning neural network described above. The trained corrosion detection neural network then outputs a recognition result of the corrosion defect, which includes data (output data) including at least the location, classification, and area of ​​the corrosion defect, i.e., the output data from the output layer of the trained corrosion detection neural network includes at least the predicted location, classification, and area of ​​the corrosion feature.

[0079] Computer-Aided Data Analysis Unit The system of the present invention further comprises a computer-aided data analysis unit useful for receiving and analyzing output data from the corrosion detection unit. The computer-aided data analysis unit may also have the function of distinguishing between blistering defects and rusting defects in the output data. The computer-aided data analysis unit is configured to provide a predicted grade of corrosion severity by analyzing the corrosion feature data output from the corrosion detection neural network. The computer-aided data analysis may include image analysis and data statistics methods known in the art. The computer-aided data analysis may be performed using an automated measurement algorithm.

[0080] After the trained corrosion detection neural network recognizes corrosion features and, if present, the trained ROI neural network recognizes the boundaries of the region of interest, these output results for the corrosion features are further evaluated by a computer-aided data analysis unit to predict a corrosion severity grade. The output data for the corrosion severity grade may include defect location, classification with different colors, one-sided leak width, two-sided leak width, rust level, and blister level, or a combination thereof. "Leakage" refers to the width of corrosion at the scribe line on a coated panel. There are two main types of scribe marks in the industry for leak evaluation: single straight line and cross-line. Leakage grows from the scribe, so "one-sided leak" refers to the width of the corrosion area on one single side of the scribe, and "two-sided leak" means that the width of the corrosion area on both sides of the scribe is calculated as the total leak width.

[0081] Figure 14 shows an example of surface leakage calculation according to one embodiment of the present invention. Scribe mark "X" 1401, including scribe 1 and scribe 2 for a coated panel, is output from the corrosion detection neural network. The outlines of the corrosion areas in each scribe are extracted and shown in figures 1402A and 1402B, respectively, where the x direction is the direction of each scribe line and the y direction is the direction of corrosion growth. The maximum distance in the y axis between the two outlines for each scribe line is then calculated.

[0082] It is difficult to distinguish rust defects from blister defects through manual visual inspection. The difference between rust defects and blister defects is that rust defects cause discoloration on the coating surface of the coated panel, while blister defects do not. Therefore, the reconstructed color image obtained above can be used to help distinguish rust defects from blister defects based on the average color value for the defect. For each corrosion defect output from the corrosion detection neural network, the average color value within each corrosion defect is confirmed to determine whether it is rust. Because there is a superposition effect between the color of the coating and the color of rust on such a coating surface, the average color value criteria for determining rust on coatings with different colors may be different. For corrosion defects on gray and black coatings, the average RGB (red, green, and blue) color index used to determine whether it is rust is as follows: If a corrosion defect on a gray coating has R<82 and G<51 and B<51, it is classified as rust; If a corrosion defect on a black coating has R>7 and G>7 and B>7, it is classified as rust.

[0083] For white coatings, the reconstructed color image is first converted to its gray version with a gray value range of [0, 1]. Then, if the average gray value of a corrosion defect on the white coating is <0.5, it is classified as rust.

[0084] To automatically determine the color of the coating, the average RGB color index of all background areas within the region of interest is used. The background area means an area without surface leakage, rust occurrence, or blister occurrence. The following color index ranges are used for the determination of the colored coating: Black coating: R < 20 and G < 13 and B < 10; Gray coating: 62 < R < 133 and 59 < G < 122 and 48 < B < 112, and White coating: R > 195 and G > 174 and B > 126.

[0085] The degree of blister occurrence and the degree of rust occurrence can be graded based on analyzing the evaluation criteria according to ASTM D714 - 02 and ASTM D610 - 08, respectively. Regarding the characteristics of blister occurrence, the severity grade is first evaluated based on the size of the largest bubbles (e.g., the top 10 largest blisters within the area), and then the distribution of blister occurrence is evaluated based on the density of blisters (i.e., the number of blisters within a specific area). The blister occurrence evaluation can be carried out according to the ASTM D714 - 02 standard. Regarding the characteristics of rust occurrence, first, the severity level is calculated mainly based on the area ratio of the rust - occurred area (i.e., the percentage of the rust - occurred surface area to the total area), and then the distribution of rust is calculated mainly by different rust locations combined with various rust sizes. The rust occurrence evaluation can be carried out according to ASTM D610 - 08. The grading criteria for blister occurrence evaluation and rust occurrence evaluation can be quantified based on the analysis of photos and statistics in the reference standards shown in ASTM D714 - 02 and ASTM D610 - 08, respectively.

[0086] 15 shows an exemplary process for grading rust development on a coated panel according to one embodiment of the present invention. First, rust development defects 1503 are extracted from the output data from the corrosion detection neural network, where the rust development defects are determined and identified based on the average color value criteria described above. Then, the quantity and size distribution of the rust development defects are calculated and compared with the rust development criteria 1502. A final rust development grading result 1504 is automatically generated from the calculation. The rust development criteria 1502 is obtained as follows: Reference Standard 1501 was analyzed for the visual examples shown in Figures 1-3 of ASTM D610-08, and the rust distribution type (i.e., spot, full surface, and spot) was quantified by the number of rusted areas corresponding to each rust grade. The "% Area" is the percentage of rusted surface area relative to the total area, as shown in Table 1 of ASTM D610-08.

[0087] 16 shows a process for grading blistering on coated panels according to one embodiment of the present invention. First, blistering defects 1603 are extracted from the output data from the corrosion detection neural network after rusting defects have been filtered out. Then, the quantity and size distribution of blistering defects are calculated and compared with blistering criteria 1602. A final blistering grade result 1604 is automatically generated from the calculation. The blistering criteria 1602 is obtained as follows: The photographic reference standard 1601 for blister occurrence shown in Figures 1-4 of ASTM D714-02 is analyzed and quantified by the number of blister-occurring areas corresponding to size and density (low, medium, medium density, and high density). Blister "size" is measured in pixels (1 pixel = 0.035 mm). 2 ) and included in the blister occurrence criteria 1602.

[0088] FIG. 17 shows one type of final output image from the system of the present invention, according to one embodiment of the present invention. The final output image is obtained by overlaying the output data from the corrosion detection neural network and the computer-aided data analysis results on a reconstructed color image obtained in a computer processing unit. Image 1700 includes a boundary of a region of interest 1701. Within the boundary frame 1701 is the region of interest, and outside the boundary frame are regions of no interest, blisters 1702 (shown in red), rust 1703 (shown in blue), a surface leak area within an "X" scribe 1704, rust exudation 1705, and a predicted corrosion severity rating 1706. To highlight the blisters and rust on the image, some of the rust is marked within a solid-line square, and some of the blisters are marked within a dashed-line circle. The predicted corrosion severity rating includes the maximum one-sided and maximum two-sided surface leak in millimeters (mm), the extent of rust formation, and the extent of blister formation.

[0089] The present invention also relates to a computer-implemented method for evaluating the corrosion resistance properties of a coating when applied to a corrosion-prone substrate, i.e., the coated panel described above, which may include receiving a plurality of grayscale images of the coating, reconstructing the plurality of grayscale images by computer image processing and outputting a reconstructed topographic image and a reconstructed color image, combining the reconstructed topographic image and the reconstructed color image into an image comprising high-dimensional data including one-dimensional height data and at least three-dimensional color data, inputting the high-dimensional data into a corrosion detection unit configured to recognize corrosion features and output at least a location, a classification, and an area of ​​the corrosion features, the corrosion detection unit comprising a corrosion detection neural network having an input layer that receives input data and an output layer that outputs output data, wherein the input data comprises the high-dimensional data and the output data comprises at least a location, a classification, and an area of ​​the corrosion features, and analyzing at least the location, the classification, and the area of ​​the corrosion defects obtained from the corrosion detection unit and outputting a predicted grade of corrosion severity by computer-aided data analysis. The resulting predicted corrosion severity grade can be used to validate the coating composition (which can be labeled as "pass" or "fail" for corrosion resistance properties), modify the coating composition, or adjust the time interval for coating maintenance when using the coating composition on a corrosion-prone substrate (pre-dispute draft version). Each step in this method is as described for the corresponding unit of the system of the present invention above. Preferably, the multiple grayscale images include images acquired from both multi-angle imaging and multi-spectral imaging, more preferably images acquired through both multi-spectral illumination imaging and multi-angle illumination imaging. The present invention can demonstrate high prediction accuracy of corrosion severity grades for defined corrosion features, as indicated by a regression coefficient of >80% compared to human evaluation results. The regression coefficient R 2 is typically in the range of 0 to 1 and can be calculated according to equation (I) below:

[0090]

number

[0091]

number

[0092]

number

[0093]

number

[0094] The present invention also relates to a process for training a neural network to detect corrosion on coatings when applied to a substrate susceptible to corrosion, the process comprising: collecting a plurality of grayscale images of a set of coatings applied to a substrate susceptible to corrosion (i.e., training coated panels or the training coatings described above), computer-reconstructing the captured grayscale image for each coating to output a reconstructed topographic image and a reconstructed color image for each coating, combining the reconstructed topographic image and the reconstructed color image for each coating to output an image comprising high-dimensional data for each coating, including one-dimensional height data and at least three-dimensional color data, obtaining at least actual locations, classifications, and areas of corrosion features for each coating that can be identified according to quantified rusting level, blistering level, and maximum surface leakage, creating a training dataset comprising the set of high-dimensional data and a set of data comprising at least actual locations, classifications, and areas of corrosion features for the set of coatings, training a neural network using the training dataset, thereby obtaining the trained neural network, i.e., the corrosion detection neural network described above. Alternatively, the actual location, classification, and area of ​​corrosion features for each coating can be identified according to ASTM D610-08, ASTM D714-02, and ASTM D1654-08. For example, these corrosion features can be labeled by a human through visual inspection.

[0095] The present invention also relates to a computing device. A computing device useful in the present invention may include a processor and data storage that stores computer-executable instructions that, when executed by the processor, cause the computing device to perform the functions of a computer-implemented method for evaluating the corrosion resistance properties of a coating on a coated popanel.

[0096] The present invention also relates to a computing device having a computer image processing unit, a data pre-processing unit, a corrosion detection unit, and a computer-aided data analysis unit disposed thereon. The computing device may be a client device (e.g., a device actively operated by a user), a server device (e.g., a device that provides computational services to client devices), or some other type of computing platform. Some server devices can sometimes act as client devices to perform certain operations, and some client devices can incorporate server functionality.

[0097] Processors useful in the present invention may be one or more of any type of computer processing element, such as a central processing unit (CPU), a coprocessor (e.g., a mathematics, graphics, neural network, or cryptography coprocessor), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a network processor, and / or any form of integrated circuit or controller that performs processor operations.

[0098] The data storage may include one or more data storage arrays including one or more drive array controllers configured to manage read and write access to a group of hard disk drives and / or solid state drives.

[0099] In some embodiments, computing devices may be deployed to support a clustered architecture. The exact physical location, connectivity, and configuration of these computing devices may be unknown and / or unimportant to the client device. Thus, the computing devices may be referred to as "cloud-based" devices that may be housed in various remote data center locations, such as cloud-based server clusters. Preferably, the computing devices are cloud-based server clusters, and inputting cardinality data into the decision tree ensemble is performed via a web-based user interface accessible by a user.

[0100] FIG. 18 shows a schematic diagram of a cloud-based server cluster 1800 according to one embodiment of the present invention. Desirably, computing device operations may be distributed among server devices 1802, data storage 1804, and routers 1806, all of which may be connected by a local cluster network 308. The number of server devices 1802, data storage 1804, and routers 1806 in the server cluster 1800 may depend on the computing task(s) and / or applications assigned to the server cluster 1800. For example, the server devices 1802 may be configured to perform various computing tasks of the computing devices. Thus, computing tasks may be distributed among one or more of the server devices 1802. By way of example, the data storage 1804 may store any type of database, such as a structured query language (SQL) database or trained model checkpoints. Furthermore, any database in the data storage 304 may be monolithic or distributed across multiple physical devices. Router 1806 may include network equipment configured to provide internal and external communications for server cluster 300. For example, router 1806 may include one or more packet switching and / or routing devices (including switches and / or gateways) configured to provide (i) network communications between server device 1802 and data storage 1804 via cluster network 1808, and / or (ii) network communications between server cluster 1800 and other devices via communication link 1810 to network 1812. Server device 1802 may be configured to send and receive data to and from cluster data storage 1804. Additionally, server device 1802 may organize the received data into web page representations.Such representations may take the form of a markup language, such as hypertext markup language (HTML), extensible markup language (XML), or some other standardized or proprietary format. Additionally, server device 1802 may be capable of executing various types of computerized scripting languages, such as Perl, Python, PHP Hypertext Preprocessor (PHP), Active Server Pages (ASP), or JavaScript. Computer program code written in these languages ​​may facilitate the delivery of web pages to client devices, as well as client device interaction with the web pages. [Example]

[0101] Some embodiments of the present invention will now be described in the following examples. All parts and percentages are by weight unless otherwise indicated. The following standard analytical instruments and methods were used in determining the properties and characteristics described in the examples and herein below. OROTAN, KUAI YI, and ACRYSOL are trademarks of The Dow Chemical Company.

[0102] [Table 1]

[0103] Salt spray test Preparation of coated panels: The coating formulation was applied to a Q panel (cold-rolled steel) using a 150-micrometer applicator and dried first at 23 degrees Celsius (°C) and 50% relative humidity (RH) for 5 minutes (minutes, min), then at 60°C for 30 minutes, and finally at 23°C and 50% RH for 7 days. An "X"-shaped scribe mark was created by cutting the dried film on the resulting coated panel using a razor blade. The edges of the coated panel were sealed with 3M vinyl electrical tape so that all uncoated areas and a 5-mm-wide coating film from each edge of the panel were covered by the tape. The areas covered by the tape are collectively referred to as "areas of no interest."

[0104] The coated panels were then placed in a Q-Lab Corporation salt fog chamber, Q-FOG SSP-600, and exposed to a salt fog environment (5% sodium chloride fog) according to ASTM B117-11. The salt fog chamber simulates a corrosive environment. After a predetermined time, the panels were removed from the salt fog chamber for evaluation by the following manual and automated evaluation methods according to the present invention.

[0105] (A) Manual Evaluation: Coated panels were manually evaluated by three laboratory test operators through visual inspection. The degree of rusting, reported as rust grade and distribution, was evaluated according to ASTM D610-08. The degree of blistering, reported as blister size and distribution, was evaluated according to ASTM D714-02 (reapproved 2009), respectively. The maximum one-sided surface leakage, reported as "surface leakage width" in millimeters, was measured according to ASTM D1654-08.

[0106] (B) Automatic Evaluation: The coated panels were automatically evaluated using the system of the present invention. An imaging unit equipped with a Keyence CV-X, a CA-DRM10X ring light, and a CA-HX500M camera, all available from Keyence Corp., was placed on top of the coated panel holder to capture multi-angle and multispectral images of each coated panel. A computer equipped with a computer image processing unit, a corrosion detection unit, a data pre-processing unit, and a data analysis unit of the present invention was connected to the Keyence controller in the imaging unit using an Ethernet cable. This computer was used to control the imaging unit and process the resulting grayscale images according to the system of the present invention to automatically generate corrosion resistance evaluation results.

[0107] Example 1 To verify the effectiveness and accuracy of the system of the present invention, a white paint sample was used to compare the results from manual evaluation and automated evaluation using the system of the present invention. Table 2 shows the white paint formulation containing Binder 1 to form White-Coated Panel 1. OROTAN KUAI YI 731A dispersant, SURFYNOL TG, and TEGO Airex 902W were mixed with water under slow stirring to ensure all components were thoroughly dispersed. Ti-Pure R-706 was slowly added, and the rotation speed was adjusted accordingly to maintain the grounds in a "doughnut" shape. After the grounds were finer than 30 μm, additional water was added and uniformly mixed into the mixture. In the letdown stage, the binder latex, water, and aqueous ammonia were premixed, and then the grounds were added to the premix and gently added. Sodium nitrite (15%), Texanol, and ACRYSOL RM-8W thickener were then added to obtain the paint formulation.

[0108] The other white paint formulations for preparing white-coated Panels 2-22, containing Binders 2-22, respectively, were identical to the formulation for preparing Example 1 used to prepare white-coated Panel 1, except that the binder type, amount of binder, and amount of Texanol were as described below. The amount of each binder was adjusted based on the solids content (wt%) of the binder to ensure equal total solids for each paint formulation. The amount of Texanol was adjusted based on the minimum film forming temperature (MFFT) in °C of the binder used and can be calculated based on the following equation: Amount of Texanol = Weight of binder x Solid content x MFFT / 200

[0109] [Table 2]

[0110] The resulting white paint formulation was used to prepare white coated panels 1-22, which were further characterized according to the salt spray test described above.

[0111] After removing the coated panels from the salt spray chamber, manual and automated evaluations were performed on the coated panels. Three operators required two minutes to observe and evaluate each sample, and the entire process for evaluating 10 samples, from visual inspection to recording the grading results, took approximately 20 minutes. In contrast, the system of the present invention required only approximately 10 seconds from image capture to automated analysis for the evaluation of each sample, and the entire process for evaluating 10 samples, from image capture to result verification using the system of the present invention, took a maximum of two minutes. The system of the present invention demonstrated a ten-fold improvement in evaluation speed compared to the manual evaluation process. The corrosion severity grading results for these coated panels, obtained by manual operator evaluation and automated evaluation using the system of the present invention, are shown in Table 3. A comparison of these grades for several corrosion features is shown in Figure 19. As shown in Figure 19, the correlation results indicate a regression coefficient of 94.45% for the surface leakage width (19A), 89% for the rust grade (19B), and 81.5% for the blister size (19C). These results demonstrate that the novel automated system of the present invention can significantly improve evaluation efficiency while providing evaluation results that approximate those of a skilled laboratory operator, and also reduce potential bias and errors associated with manual inspection and evaluation by a laboratory operator.

[0112] [Table 3] NA - Not tested

[0113] Example 2 Gray paint formulations 1-15, containing binders 1-15, respectively, were prepared in Example 2 to verify the robustness of the system of the present invention for evaluating coatings of colors other than white paint. Example 2 was carried out according to the same procedure as Example 1, based on the formulations shown in Table 4. The resulting gray-coated panels 1-15 were characterized according to the salt spray test described above. The corrosion severity ratings of these coated panels, obtained by manual evaluation by an operator and automated evaluation using the system of the present invention, are shown in Table 5. A comparison of these ratings for several corrosion features is shown in Figure 20. As shown in Figure 20, the correlation results indicate that the regression coefficients are 90.07% for leak width (20A), 87.62% for rust grade (20B), and 83.46% for blister size (20C). These results demonstrate that the novel automated system of the present invention has the ability to approximate the skill of an experienced laboratory operator in rating corrosion features for gray paint.

[0114] [Table 4]

[0115] [Table 5]

Claims

1. 1. A system for evaluating the corrosion resistance properties of a coating when applied to a corrosion-prone substrate, comprising: (i) an imaging unit configured to capture a plurality of grayscale images of the coating; (ii) a computer image processing unit configured to receive and reconstruct the captured grayscale images and output a reconstructed topographic image and a reconstructed color image; (iii) a data pre-processing unit configured to receive and combine the reconstructed topographic image and the reconstructed color image and to output an image comprising high-dimensional data including one-dimensional height data and at least three-dimensional color data; (iv) a corrosion detection unit configured to receive the high-dimensional data, recognize corrosion features, and output at least a location, a classification, and an area of ​​the corrosion features, the corrosion detection unit comprising a corrosion detection neural network having an input layer and an output layer, wherein input data to the input layer includes the high-dimensional data, and output data from the output layer includes at least a location, a classification, and an area of ​​the corrosion features; and (v) a computer-aided data analysis unit configured to receive and analyze the data including at least the location, classification, and area of ​​the corrosion features and output a predicted rating of corrosion severity.

2. The system of claim 1 , wherein the plurality of grayscale images of the coating comprises images acquired through multi-spectral illumination imaging and images acquired through multi-angle illumination imaging.

3. 3. The system of claim 2, wherein the grayscale images acquired through multi-angle illumination imaging are reconstructed into the topographic image using surface height map information through a shape-from-shading algorithm.

4. The system of claim 2 , wherein the grayscale image acquired through multispectral illumination imaging is reconstructed into the color image using a spectral reconstruction algorithm.

5. The corrosion detection neural network uses a set of training coatings applied to corrosion-prone substrates to identify the following: collecting a plurality of grayscale images of each training coating; reconstructing the captured grayscale images for each training coating by computer processing and outputting a reconstructed topographic image and a reconstructed color image for each training coating; combining the reconstructed topographic image and the reconstructed color image for each training coating to output an image containing high-dimensional data for each training coating, the high-dimensional data including one-dimensional height data and at least three-dimensional color data; Obtaining at least the actual location, classification, and area of ​​corrosion features for each training coating identified according to the quantified degree of rusting, degree of blistering, and maximum surface leakage; creating a training data set including the set of high-dimensional data and a set of data including at least actual locations, classifications, and regions of the corrosion features for a set of coatings; and training the corrosion detection neural network using the training data set.

6. 2. The system of claim 1, wherein prior to the corrosion detection neural network, the corrosion detection unit also comprises a region of interest neural network configured to recognize boundaries of regions of interest and output the high-dimensional data after the regions of interest are extracted.

7. The system of claim 6 , wherein the corrosion detection neural network and the region of interest neural network each independently use a U-Net neural network.

8. 2. The system of claim 1, wherein in the data pre-processing unit, the reconstructed color image is processed using reflected intensities of red, green, and blue spectral channels and converted into three-dimensional color data, and the topographic image is processed using grayscale values ​​representing surface height values ​​and converted into one-dimensional height data.

9. The system of claim 1 , wherein the at least three-dimensional color data and the one-dimensional height data are combined into the high-dimensional data by a matrix addition operation.

10. The system of claim 1 , wherein the computer-aided data analysis unit is also configured to quantify grading criteria of the corrosion features and output a predicted quantified grade of corrosion severity.

11. The system of claim 1 , wherein the predicted corrosion severity rating comprises blistering severity, rusting severity, surface leakage, and combinations thereof.

12. 1. A computer-implemented method for evaluating the corrosion resistance properties of a coating when applied to a corrosion-susceptible substrate, comprising: receiving a plurality of grayscale images of the coating; reconstructing the plurality of grayscale images by computer image processing to output a reconstructed topographic image and a reconstructed color image; combining the reconstructed topographic image and the reconstructed color image into an image comprising high-dimensional data including one-dimensional height data and at least three-dimensional color data; inputting the high-dimensional data into a corrosion detection unit configured to recognize corrosion features and output at least a location, a classification, and an area of ​​the corrosion features, the corrosion detection unit comprising a corrosion detection neural network having an input layer that receives input data and an output layer that outputs output data, the input data including the high-dimensional data, and the output data including at least a location, a classification, and an area of ​​the corrosion features; receiving and analyzing the data including at least the location, classification, and area of ​​the corrosion features, and outputting a predicted rating of corrosion severity by computer-assisted data analysis.

13. The computer-implemented method of claim 1 , wherein the plurality of grayscale images are acquired through multi-spectral and multi-angle illumination imaging.

14. a computing device having a computer image processing unit, a data pre-processing unit, a corrosion detection unit, and a computer-aided data analysis unit disposed thereon; the computer image processing unit is configured to receive and reconstruct the captured grayscale images and output a reconstructed topographic image and a reconstructed color image; the data pre-processing unit is configured to receive and combine the reconstructed topographic image and the reconstructed color image, and output an image including high-dimensional data including one-dimensional height data and at least three-dimensional color data; the corrosion detection unit is configured to receive the high-dimensional data, recognize corrosion features, and output at least a location, a classification, and an area of ​​the corrosion features; the corrosion detection unit comprises a corrosion detection neural network having an input layer and an output layer, input data to the input layer includes the high-dimensional data, and output data from the output layer includes at least a location, a classification, and an area of ​​the corrosion features; a computing device, wherein the computer-aided data analysis unit is configured to receive and analyze the data including at least the location, classification, and area of ​​the corrosion features and output a predicted rating of corrosion severity.

15. 1. A process for training a neural network to detect corrosion, comprising: collecting a plurality of grayscale images of the set of coatings when applied to a corrodible substrate; reconstructing the captured grayscale images for each coating by computer processing and outputting a reconstructed topographic image and a reconstructed color image for each coating; combining the reconstructed topographic image and the reconstructed color image for each coating to output an image containing high-dimensional data for each coating, the high-dimensional data including one-dimensional height data and at least three-dimensional color data; Obtaining at least the actual location, classification, and area of ​​corrosion features for each coating identified according to the quantified degree of rusting, degree of blistering, and maximum surface leakage; creating a training data set including the set of high-dimensional data and a set of data including at least actual locations, classifications, and regions of the corrosion features for the set of coatings; training the neural network using the training data set, thereby obtaining a trained neural network.

Images