Material analysis method

Abstract

The present invention relates to a material analysis method which may comprise the steps of: (a) preparing a specimen for structure analysis; (b) training an artificial intelligence model with training images labelled with structural aspects of arbitrary specimens; (c) analyzing a plurality of images for analysis not labelled with structural aspects of specimens by using the trained artificial intelligence model, and removing images for analysis classified as preset noise structural aspects; and (d) using the trained artificial intelligence model to analyze the images for analysis that remain after the images for analysis classified as the noise structural aspects are removed, and classifying the remaining images for analysis as at least one target structural aspect.

Description

Material analysis method

[0001] The present invention relates to a method for analyzing a material, and more specifically, to a method for analyzing the structure of a material using artificial intelligence.

[0002] In the blast furnace process of a steel mill, coal is dry-distilled at temperatures of 1,000 to 1,300 degrees Celsius to produce coke, making it suitable for use in a blast furnace. Coal, used as a raw material for coke, is a carbonaceous mineral composed of a multi-component mixture of maceral, a microstructural component.

[0003] Maceral is an organic component whose size and shape are distinguished mainly using a microscope, and is used as the minimum unit for identification when classifying the structure of coal. However, the classification standard for maceral according to the KS standard (KS E ISO 7404-3) relies on the visual analysis (point counting method) of the tester and also requires a minimum of 500 points to be counted, which not only causes analysis deviations between testers, but also has the problem of taking a long time.

[0004] The present invention aims to solve various problems, including the above-mentioned problems, by automating the visual analysis performed by a small number of skilled workers to classify and calculate coal maceral fractions using an analysis technology utilizing an artificial intelligence model based on deep learning, thereby providing a material analysis method that enables even unskilled workers to easily and quickly classify and calculate coal maceral fractions. However, these tasks are exemplary and the scope of the present invention is not limited thereby.

[0005] According to one embodiment of the present invention, a material analysis method is provided, which may include: (a) preparing a specimen for analyzing a tissue; (b) photographing an inspection area set on one surface of the specimen to obtain a plurality of specimen images; (c) preprocessing the plurality of specimen images to obtain a plurality of analysis images; and (d) analyzing the plurality of analysis images using a learned tissue classification artificial intelligence model to classify the plurality of analysis images into at least one target tissue pattern.

[0006] According to one embodiment of the present invention, the step (a) may include: (a-1) a step of crushing a material into particles having a predetermined size; and (a-2) a step of mixing the particles and a binder to produce the specimen.

[0007] According to one embodiment of the present invention, the material may be coal and the binder may be epoxy.

[0008] According to one embodiment of the present invention, the step (c) may be a step of extracting a focus area set for analysis on the sample image by an artificial intelligence model using a class activation map to obtain an image for analysis.

[0009] According to one embodiment of the present invention, the focus area may be an area corresponding to the central portion on the sample image.

[0010] According to one embodiment of the present invention, the step (c) may be a step of analyzing a plurality of specimen images in which the tissue pattern of the specimen is not labeled using a learned noise removal artificial intelligence model, and removing images for analysis classified as having a preset noise tissue pattern.

[0011] According to one embodiment of the present invention, the noise tissue pattern may be an image in which the tissue pattern of the specimen is classified as a binder by the noise removal artificial intelligence model.

[0012] According to one embodiment of the present invention, the noise removal artificial intelligence model may be an artificial intelligence model using a residual neural network or a mobilenet-based deep learning network.

[0013] According to one embodiment of the present invention, in the step (d), the tissue classification artificial intelligence model may be an artificial intelligence model using an Inception-based deep learning network.

[0014] According to one embodiment of the present invention, before step (c), the method may further include the step of (f) training a noise removal artificial intelligence model or a tissue classification artificial intelligence model with a labeled training image of the tissue aspect of any specimen.

[0015] According to one embodiment of the present invention, in the step (d), the target tissue pattern may be an image in which the tissue pattern of the specimen is classified by the second artificial intelligence model as at least one of vitrinite, exinite, fusinite, semi-fusinite, mineral, and a combination thereof.

[0016] According to one embodiment of the present invention, after step (d), the step (e) may further include a step of calculating the fraction of the tissue aspects of the classified specimen.

[0017] According to one embodiment of the present invention as described above, by setting a hexagonal inspection area, the characteristics of the entire circular cross-section can be sufficiently reflected, an efficient analysis time can be secured, and by utilizing artificial intelligence, there is an effect of being able to easily and quickly distinguish maceral structures that are difficult for general workers to distinguish.

[0018] Furthermore, by independently utilizing an AI model for classifying epoxy images and an AI model for classifying maceral structures, maceral structures can be classified with greater precision. Furthermore, by extracting the central region of the specimen image and conducting analysis, accuracy can be enhanced. Of course, the scope of the present invention is not limited by these effects.

[0019] Figure 1 is a flowchart showing each step of a material analysis method according to one embodiment of the present invention in order.

[0020] Figure 2 is a flowchart showing each step of a material analysis method according to one embodiment of the present invention in more detail.

[0021] FIG. 3 is an image schematically illustrating the structure of a noise removal artificial intelligence model according to one embodiment of the present invention.

[0022] Figure 4 is an image schematically showing the structure of a tissue classification artificial intelligence model according to one embodiment of the present invention.

[0023] FIG. 5 is a table and image showing a confusion matrix and maceral structure according to one embodiment of the present invention.

[0024] Figure 6 is a table showing the precision of each tissue classification artificial intelligence model by extracting the central area of ​​the sample.

[0025] Figure 7 is a confusion matrix image showing the degree of improvement in test accuracy when the central area of ​​the specimen is extracted.

[0026] Figure 8 is an image showing the focus area set by the artificial intelligence model for analysis on the above-mentioned sample image.

[0027] Figure 9 is a table showing the accuracy of noise removal artificial intelligence models according to their initial learning rates.

[0028] Figure 10 is a table showing the accuracy of the tissue classification artificial intelligence models according to their initial learning rates.

[0029] Figure 11 is a conceptual diagram schematically illustrating a material analysis device according to one embodiment of the present invention.

[0030] Fig. 12 is an image exemplarily showing a hexagonal inspection area according to one embodiment of the present invention.

[0031] Hereinafter, various preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

[0032] The embodiments of the present invention are provided to more fully explain the present invention to those skilled in the art. The following embodiments may be modified in various ways, and the scope of the present invention is not limited to the following embodiments. Rather, these embodiments are provided to more faithfully and completely explain the present disclosure and to fully convey the spirit of the present invention to those skilled in the art. In addition, the thickness and size of each layer in the drawings are exaggerated for convenience and clarity of explanation.

[0033] Hereinafter, embodiments of the present invention will be described with reference to drawings schematically illustrating ideal embodiments of the present invention. In the drawings, variations in the shapes depicted may be expected, for example, depending on manufacturing techniques and / or tolerances. Therefore, embodiments of the present invention should not be construed as limited to the specific shapes of the regions depicted herein, but should include, for example, variations in shapes resulting from manufacturing processes.

[0034] Figure 1 is a flowchart showing each step of a material analysis method according to one embodiment of the present invention in order.

[0035] As illustrated in FIG. 1, a material analysis method according to one embodiment of the present invention may include (a) a step of preparing a specimen for analyzing a tissue, (b) a step of photographing an inspection area set on one side of the specimen to obtain a plurality of specimen images, (c) a step of preprocessing the plurality of specimen images to obtain a plurality of analysis images, and (d) a step of analyzing the plurality of analysis images using a learned tissue classification artificial intelligence model to classify the plurality of analysis images into at least one target tissue pattern.

[0036] More specifically, as illustrated in FIG. 2, step (a) of preparing a specimen for analyzing a tissue may include a step of (a-1) crushing a material into particles having a predetermined size, and a step of (a-2) mixing the particles and a binder to prepare a specimen.

[0037] At this time, the material is coal, the binder is epoxy, and a step of crushing the coal into particles of a predetermined size, mixing the binder, and then manufacturing the entire specimen into a cylindrical shape through pressure molding can be performed.

[0038] Next, in step (b) of acquiring multiple analysis images by photographing an inspection area set on one side of the specimen, the inspection area may be an area in which multiple inspection points are set to form an overall hexagonal shape. This inspection area may be an area in which multiple inspection points are set to form an overall hexagonal shape, as illustrated in FIG. 9, which will be described later.

[0039] In addition, step (c) of preprocessing multiple specimen images to obtain multiple images for analysis may include a step of extracting a focus area set for analysis by an artificial intelligence model on the specimen images using a class activation map to obtain images for analysis. The focus area will be described in detail with reference to FIG. 8, which will be described later.

[0040] In addition, the step (c) of preprocessing multiple specimen images to obtain multiple analysis images may include a step of analyzing multiple specimen images in which the tissue pattern of the specimen is not labeled using a learned noise removal artificial intelligence model, and removing analysis images classified as a preset noise tissue pattern. The noise tissue pattern may be an image in which the tissue pattern of the specimen is classified as a binder, i.e., epoxy, by the noise removal artificial intelligence model.

[0041] At this time, more specifically, the noise removal artificial intelligence model can determine and classify an image in which the tissue pattern of the specimen is classified as a binder as a noise tissue pattern, and the tissue classification artificial intelligence model can determine and classify an image in which the tissue pattern of the specimen is classified as at least one of vitrinite, exinite, fusinite, semi-fusinite, mineral, and a combination thereof as a target tissue pattern.

[0042] In addition, as illustrated in FIG. 2, a material analysis method according to an embodiment of the present invention may further include, prior to step (c) of removing images classified as noise tissue patterns, step (f) of training an artificial intelligence model with training images labeled with the tissue patterns of any specimen.

[0043] Steps (a) to (e) illustrated in Fig. 2 may not necessarily be performed sequentially. For example, a sample for tissue analysis may be prepared in step (a) after training an artificial intelligence model with an arbitrary sample in step (f).

[0044] More specifically, the step (f) of training the AI ​​model with the image for training may include the step (f-1) of training the AI ​​model with the image for noise training, and the step (f-2) of training the AI ​​model with the image for target tissue training.

[0045] At this time, in the (f-1) step of training the noise learning image in the noise removal artificial intelligence model, the noise learning image may be an image labeled by classifying the tissue pattern of the specimen as a binder.

[0046] In addition, in the step (f-2) of training the target tissue learning image in the tissue classification artificial intelligence model, the target tissue learning image may be an image labeled by classifying the tissue aspect of the specimen into at least one of vitrinite, exinite, fusinite, semi-fusinite, mineral, and a combination thereof.

[0047] Hereinafter, steps (c) to (d) of the material analysis method according to one embodiment of the present invention will be described in detail.

[0048] According to one embodiment of the present invention, in step (c), an AI model can extract a focus area set for analysis on the specimen image by utilizing a class activation map to obtain an image for analysis.

[0049] According to one embodiment of the present invention, by removing images classified as a preset noise tissue pattern among a plurality of analysis images of a specimen in advance using a noise removal artificial intelligence model in step (c), the number of analysis images to be classified in step (d) can be reduced, and the remaining analysis images can be efficiently classified as a target tissue pattern using a tissue classification artificial intelligence model.

[0050] In step (c) of the material analysis method according to one embodiment of the present invention, the noise removal artificial intelligence model may be an artificial intelligence model using a residual neural network or a mobilenet-based deep learning network.

[0051] As illustrated in Figure 3, the residual neural network artificial intelligence model can solve the problem of information loss that occurs as the depth of the neural network increases by introducing skip connections to learn the difference between the previous layer and the next layer and calculating and learning the amount of change in each layer.

[0052] In addition, according to one embodiment of the present invention, through the skip connection structure of the residual neural network, the features of an epoxy image having a relatively simple shape compared to a complex maceral structure are effectively extracted and linked to the next layer, and even if the depth of each layer increases, the extracted features do not become blurred, so that the epoxy image can be classified with high precision.

[0053] In addition, although not illustrated in the drawing, MobileNet, which is designed to analyze images in real time on mobile devices, is an artificial intelligence model that guarantees high accuracy while maintaining a small model size and low computational cost, and can classify only epoxy images quickly and accurately.

[0054] In step (d) of the material analysis method according to one embodiment of the present invention, the tissue classification artificial intelligence model may be an artificial intelligence model utilizing an Inception-based deep learning network. More specifically, it may be an artificial intelligence model utilizing an Inception V3-based deep learning network.

[0055] As illustrated in Fig. 4, the Inception V3 AI model may have a structure including an Inception module composed of a combination of a convolution layer, an average pooling layer, a max pooling layer, concatenation, dropout, and softmax. In this case, each Inception module can extract features of an input image in multiple stages while reducing the amount of computation by calculating convolution layers divided into sizes such as 1x1, 3x3, and 5x5.

[0056] In addition, according to one embodiment of the present invention, more diverse characteristics can be found through the segmented convolutional layer, which is a characteristic of the inception structure, compared to an artificial intelligence model having a single layer, so that coal maceral tissue images containing various tissue aspects can be effectively classified.

[0057] Accordingly, when analyzing the image of the specimen using one artificial intelligence model, the accuracy of distinguishing epoxy was in the 40% range, which did not meet the required accuracy. However, by independently using a noise removal artificial intelligence model that classifies epoxy images and a tissue classification artificial intelligence model that classifies maceral tissue, epoxy could be distinguished with a high accuracy of 99% and maceral tissue could be classified with a high accuracy of 94%.

[0058] In addition, step (f) of the material analysis method according to one embodiment of the present invention may further include a step (f-3) of verifying the classification accuracy of a pre-trained artificial intelligence model using a confusion matrix. More specifically, step (f-3) may include a step of verifying the classification accuracy of a noise removal artificial intelligence model using a confusion matrix, and a step of verifying the classification accuracy of a tissue classification artificial intelligence model.

[0059] A confusion matrix can be visualized by displaying the results of an AI model's predictions and classification data as a matrix. For example, a confusion matrix is ​​a 2x2 matrix that combines the true and false values ​​of predicted values ​​with the true and false values ​​of actual values. It can obtain a combination of true positives (where the actual positive value is the same as the predicted positive value), false positives (where the negative value is incorrectly predicted as positive), true negatives (where the actual negative value is the same as the predicted negative value), and false negatives (where the positive value is incorrectly predicted as negative).

[0060] At this time, the confusion matrix according to one embodiment of the present invention uses a 5x5 matrix by extending the above-described 2x2 confusion matrix as illustrated in Fig. 5, so as to be able to classify the microstructural component of coal, Marceral, into five configurations. In this case, true positive values, where the actual positive value is the same as the predicted positive value, exist along the diagonal direction from the upper left to the lower right, and the remaining values ​​can be a combination of false positives or false negatives.

[0061] The maceral composition of coal can be divided into vitrinite, exinite, fusinite, semi-fusinite, and mineral, as shown in Fig. 5.

[0062] More specifically, vitrinite is a coal structure that exhibits coking properties and can be a structure derived from the woody part of plants and exhibits a smooth and homogeneous tissue surface. Exinite is a structure derived from the bark of fallen leaves or small twigs of plants used for coking properties control and has a high content of volatile matter and tar and can be a saw-shaped structure with an overall dark brown color. Inertinite is an inert component that does not soften or melt and can be a three-dimensional structure. The main types of inertinite include fusinites and semifusinites. Minerals are small amounts of minerals contained in coal and can be an overall dark brown tissue.

[0063] In the case of conventional visual analysis, it is very difficult for ordinary workers to distinguish maceral structures, and thus, it relies on the know-how of highly skilled workers. However, by applying the material analysis method according to one embodiment of the present invention, maceral structures that are difficult for ordinary workers to distinguish can be easily and quickly distinguished. For example, it is possible to easily distinguish semifuginite, which has the characteristics of vitrinite and fuginite mixed together, and also, for example, it is possible to easily distinguish between axinite and minerals that have a blackish brown color.

[0064] In addition, step (d) of classifying the image for analysis into a target tissue pattern can classify the remaining analysis images from which the epoxy image was removed as a noise tissue pattern in step (c) into one or more of the tissue patterns of vitrinite, exinite, fusinite, semi-fusinite, mineral, and combinations thereof.

[0065] In addition, the material analysis method according to one embodiment of the present invention may further include, after step (d), step (e) of calculating the fractions of the structural aspects of the classified specimen. For example, the maceral structure of the coal sample to be analyzed may be calculated as the proportion (%) of vitrinite, exinite, fusinite, semi-fusinite, and mineral with respect to the total volume.

[0066] FIG. 6 is a table comparing the results of inputting the SGDM or ADAM or RMSPROP parameter (Solver) and the results of inputting the existing entire specimen image in step (c) of the material analysis method according to one embodiment of the present invention, which extracts the focused area corresponding to the center portion of the specimen image.

[0067] As illustrated in Figure 6, the validation and test accuracies of the three types of deep learning networks were confirmed to have improved to 96%. In particular, as illustrated in Figure 7, when inputting the entire specimen image, 16 images that were previously incorrectly classified as minerals as exinite were confirmed to be effectively classified when the central region was extracted.

[0068] FIG. 8 is an image showing a focus area set for analysis by an artificial intelligence model on the specimen image using a class activation map in step (c) of a material analysis method according to one embodiment of the present invention.

[0069] It was found that, except for vitrinite, exinite, fusinite, semi-fusinite, and minerals, the artificial intelligence analyzed the area corresponding to the center. Therefore, in step (c) of the material analysis method according to one embodiment of the present invention, when a concentrated area corresponding to the center of the specimen image is extracted and an image for analysis is obtained and analyzed, a test accuracy of approximately 96% can be secured.

[0070] FIG. 9 is a table comparing the validation accuracy and test accuracy of the deep learning network ResNet 50 and MobileNet V2, which are artificial intelligence models of a material analysis method according to one embodiment of the present invention.

[0071] When the two types of deep learning networks described above were applied to the noise removal artificial intelligence model of the material analysis method according to one embodiment of the present invention to perform the step (c) of detecting noise, i.e., epoxy images, epoxy could be distinguished with very high accuracy, with a verification accuracy of 100% and a test accuracy of 99.86%, respectively.

[0072] FIG. 10 is a table comparing the validation accuracy and test accuracy of the Inception V3, a deep learning network of an artificial intelligence model of a material analysis method according to one embodiment of the present invention, and other types of deep learning networks.

[0073] As illustrated in Fig. 10, the validation accuracy and test accuracy were measured for each of the deep learning networks of Resnet, Inception-Resnet, Mobilenet, and Inception V3 applied to the present invention by applying an initial learning rate of 0.01 or 0.001 to three parameters (Solver). As a result of the measurement, it was confirmed that Inception V3 showed the highest validation accuracy (93.18%, 92.64%) and the highest test accuracy (91.25%, 94.11%) at an initial learning rate of 0.001.

[0074] That is, the applicant confirmed through repeated experiments that coal maceral textures can be classified with the highest accuracy when an initial learning rate of 0.001 is applied to the ADAM or RMSPROP parameters in a combination of a residual neural network artificial intelligence model and an Inception V3 artificial intelligence model, or a combination of a mobile net artificial intelligence model and an Inception V3 artificial intelligence model.

[0075] Hereinafter, a material analysis device (100) according to one embodiment of the present invention will be described with reference to FIGS. 11 and 12.

[0076] Fig. 11 is a conceptual diagram schematically illustrating a material analysis device (100) according to one embodiment of the present invention. As illustrated in Fig. 11, the material analysis device (100) according to one embodiment of the present invention may include a stage (110), a stage moving device (120), an image capturing device (130), and a tissue classification device (140).

[0077] More specifically, the stage moving device (120) may include a stage forward / backward moving device (121) that can move the stage (110) in the forward / backward direction, and a stage left / right moving device (122) that can move the stage (110) in the left / right direction.

[0078] In addition, the image capturing device (130) can capture an image of a specimen by setting an inspection area on one side of the specimen, as illustrated in FIG. 12. At this time, the inspection area may be an area in which a plurality of inspection points are set to form an overall hexagonal shape. This inspection area may be an area in which a plurality of inspection points are set to form an overall hexagonal shape, as illustrated in FIG. 12.

[0079] Accordingly, the stage moving device (120) can move the stage (110) so that the image capturing device (130) is positioned at the inspection point of one of the vertices of a hexagon, for example, and then move the stage (110) forward and backward or left and right so that the image capturing device (130) is positioned at the inspection point of another vertex adjacent thereto.

[0080] In the past, the inspection area was generally set so that the inspection points formed a rectangular shape overall, but in the present invention, by setting the inspection area as a hexagon as described above, the characteristics of the entire circular cross-section could be sufficiently reflected, and when the inspection area was set to form a polygonal shape larger than a hexagon, there was a problem that the time required for inspection was long due to an excessively large number of inspection points, so when the inspection area was set as a hexagon, the most efficient analysis could be performed.

[0081] In addition, the material analysis device (100) according to one embodiment of the present invention may include a deep learning unit (150). At this time, the deep learning unit (150) may include a noise removal deep learning unit (151) that learns and classifies noise tissue patterns so as to distinguish and classify noise appearing in a specimen, and a tissue classification deep learning unit (152) that learns and classifies target tissue patterns so as to distinguish and classify tissue patterns appearing in a specimen.

[0082] Accordingly, the images of the inspection points of the specimen captured by the image capturing device (130) can be classified into vitrinite, exinite, fusinite, semi-fusinite, and mineral using an artificial intelligence model learned using deep learning.

[0083] While the present invention has been described with reference to the embodiments illustrated in the drawings, these are merely exemplary, and those skilled in the art will appreciate that various modifications and equivalent alternative embodiments are possible. Therefore, the true scope of technical protection of the present invention should be determined by the technical spirit of the appended claims.

Claims

1. (a) Step of preparing a specimen for tissue analysis; (b) a step of obtaining multiple specimen images by photographing an inspection area set on one side of the specimen; (c) a step of preprocessing the plurality of specimen images to obtain a plurality of images for analysis; and (d) a step of analyzing the plurality of analysis images using a learned tissue classification artificial intelligence model and classifying them into at least one target tissue pattern; A material analysis method including:

2. In paragraph 1, Step (a) above, (a-1) a step of crushing the material into particles having a predetermined size; and (a-2) A step of manufacturing the specimen by mixing the particles and binder; A material analysis method including:

3. In paragraph 2, A material analysis method wherein the above material is coal and the above binder is epoxy.

4. In paragraph 1, Step (c) above, A material analysis method, which is a step of obtaining an image for analysis by extracting a focus area set for analysis from the above-mentioned specimen image by using a class activation map.

5. In paragraph 4, The above-mentioned concentrated area is a material analysis method, which is an area corresponding to the central portion in the above-mentioned specimen image.

6. In paragraph 1, Step (c) above, A material analysis method, which is a step of analyzing multiple specimen images whose tissue patterns are not labeled using a learned noise removal artificial intelligence model, and removing images for analysis classified as having a preset noise tissue pattern.

7. In paragraph 6, The above noise organization pattern is, A material analysis method, wherein the image is classified as a binder in terms of the tissue pattern of the specimen by the above noise removal artificial intelligence model.

8. In paragraph 6, The above noise removal artificial intelligence model is, A material analysis method using an artificial intelligence model that utilizes a residual neural network or a deep learning network based on MobileNet.

9. In paragraph 1, In the above step (d), The above organizational classification artificial intelligence model is, A material analysis method using an artificial intelligence model based on Inception-based deep learning networks.

10. In paragraph 1, Before step (c) above, (f) a step of training a noise removal artificial intelligence model or a tissue classification artificial intelligence model using a training image labeled with the tissue aspect of an arbitrary specimen; A material analysis method further comprising:

11. In paragraph 1, In the above step (d), The above target organizational aspect is, A material analysis method, wherein the image is classified by the second artificial intelligence model as having a texture pattern of at least one of vitrinite, exinite, fusinite, semi-fusinite, mineral, and a combination thereof.

12. In paragraph 1, After step (d) above, (e) a step of calculating the fraction of the tissue aspects of the classified specimen; A material analysis method further comprising:

Images