Analysis of a micrographic image of an inspection surface of a material

Abstract

The invention relates to an analysis system (100) comprising: - a segmentation model (102), previously trained by machine learning, for providing a segmentation (SEG) of the micrographic image (IMG) along the grain-boundary network so as to outline the grains; and - a characterization module (106) configured to determine a first evaluation (T1) of the grain size of the grains on the basis of the segmentation (SEG). The analysis system (100) further comprises: - a classification model (108), previously trained by machine learning, for determining a second evaluation (T2) of the grain size of the grains from the micrographic image (IMG); and - a comparison module (112) configured to compare the first and second evaluations (T1, T2) of grain size.

Description

Description TITLE: ANALYSIS OF A MICROGRAPHIC IMAGE OF AN EXAMINATION SURFACE OF A MATERIAL Technical field of the invention

[0001] The present invention relates to a system for analyzing a micrographic image of an examination surface of a material.

[0002] The present invention can in particular be used in each of the following two technical fields.

[0003] First, the present invention can be used in alloy research. For this purpose, the present invention allows for a deeper understanding of the microstructure of the alloy under study, in particular the links between its microstructure and its mechanical properties.

[0004] Furthermore, the present invention can be used to control microstructures on samples from forged parts and on pre-machined parts. In this context, the invention serves as a tool to aid in the evaluation and coding of microstructures, thus contributing to the validation and certification of parts, guaranteeing their conformity to the required specifications.

[0005] The present invention is, for example, intended to be used on micrographic images of microstructures of Inconel®718, a nickel-based austenitic superalloy widely used in the aerospace industry and in the manufacture of components subjected to extreme conditions.

[0006] A micrograph is a high-magnification image of a mirror-polished, chemically etched plane of material, obtained using an electron or optical microscope. It allows visualization and analysis of the material's microstructural characteristics, such as grain size and appearance, the constituents and phases present, and their distribution. Technological background

[0007] ASTM E112 defines several methods for determining the average grain size of metallic materials. These methods are essentially geometric, making them independent of the specific metal and alloy. Currently, of all the methods described in ASTM E112, only the comparative method is used by operators, as it remains the only one applicable within timeframes compatible with operational constraints. Although the robustness of this method is ensured by a quality process, including operator qualification, a human factor can still introduce some variability in the rating.

[0008] There are also automated measurement solutions, which for example use digital tools available on the market (for example, MIPAR software).

[0009] An automated measurement solution is described, for example, in the article by Shi et al., entitled "An improved U-Net image segmentation method and its application for metallic grain size statistics," published in 2022 in MPDI, Materials 2022, 15, 4417 (DOI 10.3390). This article describes a system for analyzing a micrograph of a material surface under examination. The micrograph represents a microstructure of the material with grains delimited by boundaries forming a network of boundaries. The analysis system comprises: a segmentation module previously trained by machine learning to provide segmentation of the micrograph image along the joint network in order to surround the grains; and a characterization module designed to determine a grain size assessment of grains from segmentation.

[0010] One problem with this known solution is that it is not possible to know whether the grain size assessment provided is reliable or not.

[0011] It may therefore be desirable to provide an analysis system that allows us to overcome at least some of the aforementioned problems and constraints. Summary of the invention

[0012] Therefore, a system is proposed for analyzing a micrograph of an examination surface of a material, the micrograph representing a microstructure of the material comprising grains delimited by boundaries forming a network of boundaries, the analysis system comprising: a segmentation model previously trained by machine learning to provide a segmentation of the micrograph along the network of boundaries in order to surround the grains; and a characterization module designed to determine an initial grain size assessment from segmentation; the analysis system being characterized in that it further comprises: a classification model previously trained by machine learning to determine a second grain size assessment of the grains from the micrograph image; and a comparison module designed to compare the first and second grain size assessments; and in that the characterization module is designed to complete the segmentation in order to obtain a completed segmentation, by identifying adjacent pixel groups of the segmentation forming respectively the grains of the segmentation, the characterization module being designed to determine the first grain size evaluation from the completed segmentation.

[0013] Thus, the second grain size assessment provides complementary advice to the first, allowing the comparison module to determine, in the event of a significant difference, that the reliability of the first assessment is low. Furthermore, initial training of the classification model, as well as any subsequent update training, is easier to perform than training the segmentation model. This is because training the classification model requires less expensive training data than training the segmentation model. Specifically, the training data for the classification model includes micrographs and, for each of these, a grain size assessment, for example, performed by domain experts.Such an evaluation, performed by experts, typically takes between a few seconds and a few minutes. In contrast, the training data for the segmentation model includes micrograph images and, for each image, a segmentation of that image. However, determining such a segmentation for a single image can take experts up to two weeks, despite existing preprocessing tools. This is because it is necessary to determine for each pixel of the micrograph whether it belongs to a grain or a boundary.

[0014] The invention may further include one or more of the following optional features, in any technically feasible combination.

[0015] Optionally, the comparison module is designed to calculate a difference between the first and second grain size assessments, and, if this difference is greater than a predefined threshold, to issue an alert.

[0016] Optionally, the analysis system also includes a display device designed to display the alert if the deviation is greater than the predefined threshold, the alert including, for example, the first and second grain size assessments.

[0017] Optionally, the analysis system also includes a display device designed to display a grain size derived from at least one of the two grain size assessments.

[0018] Optionally, to determine the first grain size assessment, the characterization module is designed to apply, to the completed segmentation, a mask delimiting a predefined area of ​​the completed segmentation, to determine a number of grains appearing, even partially, in the predefined area and to use a predefined equation giving a size index forming the first grain size assessment as a function of the number of grains appearing in the predefined area.

[0019] Optionally, the characterization module is also designed to provide a number N of grains present in the SEG segmentation.

[0020] Optionally, the characterization module is also designed to determine a number of grain populations, by partitioning the grains according to their surface area.

[0021] Optionally, the characterization module is also designed, for each grain, to fit an ellipse onto that grain and to provide a measurement of a grain shape comprising a ratio between a major axis and a minor axis of the ellipse.

[0022] Optionally, the characterization module is also designed to determine an equivalent diameter for each grain. Brief description of the figures

[0023] The invention will be better understood with the aid of the following description, given solely by way of example and made with reference to the accompanying drawings in which: Figure 1 is a functional view of an example of an analysis system according to the invention. Figure 2 shows two examples of micrograph images that can be received by the analysis system shown in Figure 1, and Figure 3 is an example of segmentation obtained from the left micrograph image of Figure 2. Detailed description of the invention

[0024] With reference to Figure 1, an example of an analysis system 100 according to the invention will now be described.

[0025] The 100 analysis system is designed to receive a micrograph of an examination surface of a material, for example, an alloy such as Inconel® 718. Preferably, this examination surface corresponds to a section of the material after it has been cut, this section subsequently being prepared by polishing and / or chemical etching by applying a reactive agent. The micrograph of the examination surface, i.e., the polished and / or chemically etched section, can be obtained using an electron or optical microscope.

[0026] The IMG micrograph thus represents the microstructure of the material. This microstructure comprises grains delimited by boundaries, which form a network of boundaries. Generally, a grain can consist of one or more crystallites, which are areas where the microstructure is perfectly uniform. The boundaries include, in particular, macie boundaries (also known as "crystal twins"), which are boundaries that coincide with a plane of mirror symmetry of the crystallites.

[0027] Two examples of micrograph images that can be received by the analysis system 100 are illustrated in Figure 2.

[0028] Returning to Figure 1, the analysis system 100 first includes a segmentation model 102 previously trained by machine learning to provide a SEG segmentation of the IMG micrograph image. This SEG segmentation follows the joint network to surround the grains represented on the IMG micrograph image. This makes it possible to distinguish the grains within the joint network, and thus to individually identify each grain of the microstructure.

[0029] Machine learning is specifically supervised and performed using a training dataset comprising labeled micrograph images, meaning each image is associated with a target segmentation. This labeling is, for example, developed manually, then validated and checked, preferably by metallurgy experts. Each target segmentation is, for example, in the form of a pixel image, in which the same first value is assigned to all pixels within the seams and the same second value is assigned to all pixels outside the seams.

[0030] It has been found that using machine learning to perform SEG segmentation is more efficient than methods using explicit algorithms such as thresholding, watersheds, etc. Indeed, the segmentations obtained by such explicit algorithms are not of sufficient quality to determine grain size.

[0031] The 102 segmentation model is preferably designed to disregard twinning. In other words, the provided SEG segmentation relies only on joints other than twinning. To achieve this property, the target segmentations are developed without considering twinning, so that, after training, the 102 segmentation model attempts to reproduce these target segmentations and thus segment without regard to twinning.

[0032] Furthermore, the 102 segmentation model is preferably designed to close seams that are not fully defined in the image. This property is important because the seams are more or less visible depending on the preparation of the examination surface, particularly the reagent used and its application. To achieve this property, the target segmentations are designed to have continuous and closed seams, adding, if necessary, seam portions that are barely visible due to image quality, but which are nonetheless present. Thus, after training, the 102 segmentation model attempts to reproduce these target segmentations and therefore close the seams.

[0033] Furthermore, the 102 segmentation model is preferably designed to remove certain artifacts from the micrograph image associated, for example, with polishing imperfections. To achieve this property, the target segmentations are designed to disregard these artifacts. In particular, in the target segmentations, only the grain interiors and grain boundaries are indicated, for example, by different pixel values ​​as previously described. Thus, after training, the 102 segmentation model aims to do the same.

[0034] An example of SEG segmentation obtained from the left micrograph image in Figure 2 is illustrated in Figure 3.

[0035] Returning to Figure 1, the analysis system 100 preferably includes a first verification model 104, previously trained by machine learning, to verify that the micrograph image IMG is close to initial reference micrograph images before the micrograph image is passed to the segmentation model 102. The first verification model 104, for example, includes a Siamese neural network. This first verification model 104 detects whether the micrograph image IMG is outside the distribution ("Out-Of-Distribution" or OOD) of the micrograph images used for machine learning of the segmentation model 102. The first verification model 104 is thus useful for improving the reliability and robustness of the segmentation model 102.Indeed, when a micrograph image differs considerably from those used for training the segmentation model 102, the latter may produce incorrect or unreliable predictions, so it is preferable not to transmit this micrograph image to the segmentation model 102.

[0036] The analysis system 100 also includes a characterization module 106 designed to determine, by SEG segmentation analysis, one or more characteristics of the material's microstructure.

[0037] In particular, this characterization module 106 is primarily designed to complete the SEG segmentation to obtain a completed SEG' segmentation, by identifying groups of adjacent pixels in the SEG segmentation that form the SEG segmentation grains, the number of which is denoted N. Thus, N groups of adjacent pixels are identified. This identification is implemented, for example, by assigning all pixels in the same group within the SEG segmentation the same identifier, which differs from one group to another.

[0038] Thus, the completed SEG' segmentation allows the characterization module 106 to obtain and provide the number N of grains present in the SEG segmentation and the number of pixels forming each grain.

[0039] The 106 characterization module is further designed to determine, from the completed segmentation SEG', a first grain size assessment T1 of the microstructure grains. This first grain size assessment T1 is, for example, in the form of a size index, such as that defined in ASTM E112. In this case, the 106 characterization module is designed to apply a predefined surface observation field to the completed segmentation SEG' (i.e., a mask delimiting a predefined area of ​​the SEG segmentation) and determine the number of grains appearing (even partially) within the surface observation field (i.e., within the area delimited by the mask). The 106 characterization module is then designed to use a predefined equation that gives the size index as a function of the number of grains appearing within the surface observation field.The size index can be expressed as the number of pixels per grain. Preferably, the characterization module 106 is designed to provide a confidence interval for the first grain size evaluation T1.

[0040] The number of pixels in each grain allows, for example, the characterization module 106 to deduce an area of ​​that grain, for example expressed in square millimeters, since each pixel covers a predefined area.

[0041] The characterization module 106 can then also be designed to determine a statistical distribution Ds of the grain surfaces.

[0042] The 106 characterization module can also be designed to determine an equivalent diameter for each grain, for example expressed in millimeters, i.e., the diameter of a circle with the same area as the grain. In this case, the 106 characterization module can also be designed to determine a DD statistical distribution of the equivalent grain diameters.

[0043] The 106 characterization module can also be designed to determine a measure of the shape of each grain. For example, the 106 characterization module is designed, for each grain, to fit an ellipse to that grain, that is, to find the ellipse closest to the grain's shape. As is known, the fitted ellipse has a major axis and a minor axis. Thus, the measure of the grain's shape can include a ratio between the major and minor axes of the ellipse. The grain is then considered non-equiaxed when the major axis is greater than twice the minor axis.

[0044] The characterization module 106 can also be designed to determine a number of grain populations, that is to say to carry out a partitioning (from the English "clustering") of the grains according to their surface area, consisting of searching for the optimal number of populations, each grouping grains with surfaces close to each other.

[0045] The analysis system 100 also includes a classification model 108 previously trained by machine learning to determine a second grain size evaluation T2 of the grains from the micrographic image IMG.

[0046] Machine learning is specifically supervised and performed using a training dataset comprising labeled micrograph images, meaning each image is associated with a target grain size. These target grain sizes are preferably validated and verified by metallurgy experts.

[0047] The analysis system 100 preferably includes a second verification model 110, previously trained using machine learning, to verify that the micrograph image IMG is close to second reference micrograph images before the micrograph is fed into the classification model 108. The second verification model 110 may, for example, include a Siamese neural network. This second verification model 110 detects whether the micrograph image IMG is outside the distribution (Out-Of-Distribution, or OOD) of the micrograph images used for machine learning of the classification model 108. The second verification model 110 is thus useful for improving the reliability and robustness of the classification model 108.Indeed, when a micrograph differs considerably from those used for training the classification model 108, the latter may produce incorrect or unreliable predictions, so it is preferable not to transmit this micrograph to the classification model 108.

[0048] The analysis system 100 also includes a comparison module 112 designed to compare the first and second grain size assessments T1, T2, in order to calculate a difference E between them. If this difference E exceeds a predefined threshold S, an alert is issued, for example, to a user of the analysis system 100, so that this user can choose the best assessment based on their expertise in the field.

[0049] The analysis system 100 further includes a display device 114 designed to display the characteristics determined by the characterization module 106, namely, for example, the number of grains N, the statistical distribution Ds of the grain areas, the statistical distribution DD of the equivalent grain diameters, and a grain size T derived from at least one of the two grain size assessments T1, T2, in particular if the difference E between the two grain size assessments T1, T2 is less than the threshold S. For example, the grain size T may be an average of the grain size assessments T1, T2. Conversely, when the difference E is greater than the threshold S, an alert is displayed, this alert including, for example, the two grain size assessments T1, T2.These characteristics N, Ds, DD, T, T1, T2 are displayed, for example, in numerical and / or graphical form, which allows for easy analysis and interpretation of the characteristics of the microstructure under study. Furthermore, the display device 114 is designed, for example, to display SEG segmentation, which makes it easy to distinguish grain boundaries.

[0050] In conclusion, it should also be noted that the invention is not limited to the embodiments described above; indeed, it will appear to a person skilled in the art that various modifications can be made to the embodiments described above, in light of the teaching which has just been disclosed to him.

[0051] In the detailed presentation of the invention given above, the terms used shall not be interpreted as limiting the invention to the embodiments set forth in this description, but shall be interpreted to include all equivalents which can be foreseen by a person skilled in the art by applying their general knowledge to the implementation of the teaching which has just been disclosed to them.

Claims

Demands [1] Analysis system (100) of a micrograph image (MIG) of an examination surface of a material, the micrograph image (MIG) representing a microstructure of the material comprising grains delimited by boundaries forming a boundary network, the analysis system (100) comprising: a segmentation model (102) previously trained by machine learning to provide a segmentation (SEG) of the micrograph image (IMG) along the joint network in order to surround the grains; and a characterization module (106) designed to determine a first grain size assessment (T1) of the grains from segmentation (SEG); the analysis system (100) being characterized in that it further comprises: a classification model (108) previously trained by machine learning to determine a second grain size evaluation (T2) of the grains from the micrograph image (IMG); and a comparison module (112) designed to compare the first and second grain size evaluations (T1, T2); and in that the characterization module (106) is designed to complete the segmentation (SEG) in order to obtain a completed segmentation, by identifying adjacent pixel groups of the segmentation (SEG) forming the grains of the segmentation (SEG), the characterization module (106) being designed to determine the first grain size evaluation (T1) from the completed segmentation. [2] Analysis system (100) according to claim 1, wherein the comparison module (112) is designed to calculate a difference (E) between the first and second grain size evaluations (T1, T2), and, if this difference (E) is greater than a predefined threshold (S), to issue an alert. [3] Analysis system (100) according to claim 2, further comprising a display device (114) designed to display the alert if the deviation (E) is greater than the predefined threshold (S), the alert comprising, for example, the first and second grain size assessments (T1, T2). [4] Analysis system (100) according to any one of claims 1 to 3, further comprising a display device (114) designed to display a grain size (T) derived from at least one of the two grain size assessments (T1, T2). [5] Analysis system (100) according to any one of claims 1 to 4, wherein, in order to determine the first grain size evaluation (T1), the characterization module (106) is designed to apply, on the completed segmentation, a mask delimiting a predefined area of ​​the completed segmentation, to determine a number of grains appearing, even partially, in the predefined area and to use a predefined equation giving a size index forming the first grain size evaluation (T1) as a function of the number of grains appearing in the predefined area. [6] Analysis system (100) according to any one of claims 1 to 5, wherein the characterization module (106) is designed to provide a number (N) of grains present in the segmentation (SEG). [7] Analysis system (100) according to any one of claims 1 to 6, wherein the characterization module (106) is designed to determine a number of grain populations, by performing a partitioning of the grains according to their surface area. [8] Analysis system (100) according to any one of claims 1 to 7, wherein the characterization module 106 is designed, for each grain, to fit an ellipse on that grain and to provide a measure of a grain shape comprising a ratio between a major axis and a minor axis of the ellipse. [9] Analysis system (100) according to any one of claims 1 to 8, wherein the characterization module (106) is designed to determine an equivalent diameter of each grain.

Images