Metallographic structure phase classification method, metallographic structure phase classification device, metallographic structure phase learning method, metallographic structure phase learning device, metallographic structure phase prediction method and metallographic material material property prediction device

Abstract

The method for classifying the phases of metallographic structures includes: a feature value calculation step, in which one or more feature values ​​are calculated for pixels of an image obtained by photographing the metallographic structure of a metal material; and a phase classification step, in which, for a learned model that has been learned using feature values ​​of labels of multiple phases of the metallographic structure as input and labels of each phase as output, the feature value calculated in the feature value calculation step is input, and the label of the phase of the pixel corresponding to the input feature value is obtained, thereby classifying the phase of the metallographic structure of the image.

Description

Technical Field

[0001] The present invention relates to a metallographic phase classification method, a metallographic phase classification device, a metallographic phase learning method, a metallographic phase learning device, a metallographic material material property prediction method, and a metallographic material material property prediction device. Background Art

[0002] In recent years, exhaust gas regulations have been implemented to reduce the amount of pollutants discharged from the perspective of protecting the global environment. In addition, there is a strong demand for automobiles to improve fuel efficiency by reducing the weight of the vehicle body. One of the effective methods for reducing the weight of the vehicle body is to increase the strength of the thin steel plates used in the vehicle body, and the amount of high-strength steel plates used in automobiles is increasing year by year.

[0003] Generally, even if metal materials such as steel plates have the same composition, their properties strongly depend on the metallographic structure at the level of optical microscope or electron microscope (mm to μm level). Therefore, in the case of developing high-strength steel plates, methods of changing the composition such as solid solution strengthening methods using the addition of solid solution strengthening elements and precipitation strengthening methods using precipitates generated by the addition of precipitation strengthening elements can be used. In addition, in addition to these methods, methods of improving mechanical properties by changing the metallographic structure finally achieved by changing the heat treatment conditions under the same composition can also be used.

[0004] Thus, in order to develop high-strength steel sheets, it is important not only to control the composition but also the metallographic structure. Therefore, it is important to observe the metal material using an optical microscope or an electron microscope and to quantitatively evaluate its metallographic structure. In fact, at the site of material development, the organization observation of metal materials with changed heat treatment conditions is often carried out.

[0005] When observing the structure of a metal material such as a steel plate after performing sample conditioning using known grinding methods and etching methods, the phases can usually be distinguished due to the different contrasts in each phase. For example, when a representative steel plate composed of a ferrite phase and a pearlite phase in a steel material is sample conditioned using a known method and photographed using an optical microscope, the ferrite phase is observed with a gray contrast and the pearlite phase is observed with a black contrast. Therefore, the ferrite phase and the pearlite phase can be distinguished. Such material development is often carried out by repeatedly observing the metallographic structure using such a method, controlling the structure that changes due to heat treatment conditions, and attempting to achieve the material properties required as a raw material.

[0006] Here, in order to combine the material properties with the metallographic structure, it becomes important to quantitatively evaluate the observed metallographic structure. For example, in a representative dual-phase steel plate composed of a ferrite phase and a martensite phase, it is known that its tensile strength strongly depends on the phase fraction of the martensite phase. In the past, in order to quantitatively evaluate the phase fraction of these phases from the image obtained by shooting the metallographic structure, each phase was color-coded by hand painting, and the area of ​​each color after color classification was counted, thereby determining the phase fraction of each phase. However, in this method, the identification of the phase varies depending on the operator, so the error of each operator is large, and hand painting also requires a lot of time. Therefore, even for steel plates manufactured with the same composition and the same heat treatment conditions, the error is large, and a lot of time is required in the analysis of one image, so this method is almost not actually implemented.

[0007] As a quantitative evaluation method for metallographic structures that replaces such hand painting, which has large errors and requires a lot of time, the binarization of brightness values ​​can be cited. In this method, a threshold of the brightness value is determined for the image of the metallographic structure taken, and the image is converted using a computer or the like in a two-color manner, thereby extracting only a specific phase, and the phase fraction is determined by calculating the area of ​​each color. This method can accurately classify the phases when the brightness values ​​in each phase of the metallographic structure are significantly different. In addition, since only the threshold of the brightness value is determined as an operation, the metallographic structure can be quantitatively evaluated at a very fast speed compared to the above-mentioned hand painting. On the other hand, in metal materials, there are many cases where the difference in the brightness values ​​of each phase is unclear. When the difference in the brightness values ​​is unclear, the error becomes larger, and therefore there are many cases where the classification cannot be performed with good accuracy.

[0008] As computer performance has improved dramatically recently, a technology has been developed that uses more advanced computing technology rather than the above-mentioned hand painting or simple binarization of brightness values ​​to analyze images without human factors. For example, the following technology is disclosed in Patent Document 1. First, an image of a part of the human body surface is converted into an image in an opposite color space, and the opposite color space components of the image in the opposite color space are each decomposed into sub-band images of different spatial frequencies. Then, a feature value corresponding to the part of the human body surface is calculated for the sub-band image, and the appearance of the part of the human body surface is evaluated based on the feature value. By using this technology, the condition and texture of the skin can be objectively and quickly evaluated from an image of human skin.

[0009] In addition, the following technology is disclosed in Patent Document 2. First, a plurality of binarization processes are performed on a tissue image obtained by photographing the tissue while using different reference values ​​for binarization, thereby generating a plurality of binarized images. Next, the number of hole-like regions is calculated for each of the plurality of binarized images, a feature number is determined that characterizes the correspondence between the plurality of binarized reference values ​​and the number of hole-like regions, and output information corresponding to the feature number is generated.

[0010] Prior art literature

[0011] Patent Literature

[0012] Patent Document 1: Japanese Patent Application Publication No. 2018-121752

[0013] Patent Document 2: International Publication No. 2017 / 010397 Summary of the invention

[0014] Problems to be solved by the invention

[0015] However, the technology disclosed in Patent Document 1 is ultimately a technology for evaluating human skin, and therefore is difficult to apply to the tissue of metal materials such as steel plates. In addition, the technology disclosed in Patent Document 2 is also similar, because it is a technology for analyzing images of biological cells, and therefore is difficult to apply to the tissue of metal materials such as steel plates.

[0016] The present invention is completed in view of the above-mentioned problems, and its purpose is to provide a metallographic phase classification method, a metallographic phase classification device, a metallographic phase learning method, a metallographic phase learning device, a metallographic material material property prediction method and a metal material material property prediction device that can quantitatively and quickly evaluate important metallographic structures in metal materials.

[0017] Methods used to solve problems

[0018] In order to solve the above-mentioned problems and achieve the purpose, the method for classifying the phases of metallographic structures of the present invention is characterized in that it includes: a feature value calculation step, in which one or more feature values ​​are calculated for pixels of an image obtained by photographing the metallographic structure of a metal material; and a phase classification step, in which, for a learning completion model that has been learned using feature values ​​of labels of multiple phases of the above-mentioned metallographic structure as input and labels of each phase as output, the feature value calculated in the above-mentioned feature value calculation step is input, and the label of the phase of the pixel corresponding to the input feature value is obtained, thereby classifying the phase of the metallographic structure of the above-mentioned image.

[0019] Furthermore, the method for classifying phases of a metallographic structure of the present invention is characterized in that, in the above invention, it includes an image preprocessing step of adjusting the brightness values ​​of pixels of the above image before the above phase classification step.

[0020] Furthermore, the method for classifying phases of a metallographic structure of the present invention is characterized in that, in the above invention, in the image preprocessing step, a histogram of brightness values ​​of the image is equalized.

[0021] Furthermore, the method for classifying phases of a metallographic structure of the present invention is characterized in that, in the above invention, in the image preprocessing step, an average brightness value of the image is removed.

[0022] In order to solve the above-mentioned problems and achieve the purpose, in the method for classifying the phases of metallographic structures of the present invention, for multiple images obtained by photographing the metallographic structure of the above-mentioned metal material from a metal material, the phases of two or more metallographic structures are classified, and it is characterized in that it includes: an image preprocessing step, in which one or a specified number of images are selected from the above-mentioned multiple images, and the brightness values ​​of the pixels of the selected images are adjusted; a feature value calculation step, in which one or more feature values ​​are calculated for the pixels of the image preprocessed in the above-mentioned image preprocessing step; a phase classification step, in which the feature values ​​calculated in the above-mentioned feature value calculation step are input into a learning completion model that is learned by taking the feature values ​​of the multiple phases of the above-mentioned metallographic structure as input and the labels of each phase as output, and a phase classification step is obtained. The input feature value corresponds to the phase label of the pixel, thereby classifying the phase of the metallographic structure of the above image; an error evaluation value calculation step, wherein, for each phase of the metallographic structure classified in the above phase classification step, the correctness of each pixel is determined and the error evaluation value is calculated; a preprocessing adjustment step, wherein the preprocessing parameters of the above image preprocessing step are changed in a manner that the error evaluation value calculated in the above error evaluation value calculation step becomes smaller, the above feature value calculation step, the above phase classification step and the above error evaluation value calculation step are repeated to adjust the above preprocessing parameters, the preprocessing parameters adjusted in the above preprocessing adjustment step are determined as fixed values, and the above image preprocessing step, the above feature value calculation step and the above phase classification step are all performed on multiple images obtained by photographing the metallographic structure of the above metal material.

[0023] In addition, the phase classification method of the metallographic structure of the present invention is characterized in that, in the above invention, the above eigenvalue calculation step calculates one or more of the following eigenvalues: an identity eigenvalue representing the brightness value of the above image; an average (Mean) eigenvalue representing the average value of the brightness values ​​within a specified range of the above image; a Gaussian eigenvalue representing the average value of the brightness values ​​within the specified range of the above image, the closer to the center, the greater the weight; a median eigenvalue representing the center value of the brightness values ​​within the specified range of the above image; a maximum (Max) eigenvalue representing the maximum value of the brightness values ​​within the specified range of the above image; a minimum (Min) eigenvalue representing the minimum value of the brightness values ​​within the specified range of the above image; a derivative (Derivative) eigenvalue representing the differential value of the brightness value of the above image; a derivative addition eigenvalue obtained by adding the above derivative eigenvalues.

[0024] In addition, the method for classifying the phases of the metallographic structure of the present invention is characterized in that, in the above-mentioned invention, the above-mentioned specified range is a range that includes a size smaller than 1 / 2 of the above-mentioned grain size of the smaller grain size in one or more pre-specified phases of the above-mentioned metallographic structure and includes a size smaller than 1 / 2 of the above-mentioned grain size of the larger grain size.

[0025] Furthermore, the method for classifying phases of a metallographic structure of the present invention is characterized in that, in the above invention, the metal material is a dual-phase steel plate, and in the phase classification step, a ferrite phase and a martensite phase are classified.

[0026] In addition, the method for classifying the phase of a metallographic structure of the present invention is characterized in that, in the above invention, the above-mentioned learning completion model is a machine learning model including a decision tree, a random forest or a neural network.

[0027] In order to solve the above-mentioned problems and achieve the purpose, the phase classification device of the metallographic structure of the present invention is characterized in that it comprises: a feature value calculation unit, which calculates one or more feature values ​​for the pixels of the image obtained by photographing the metallographic structure of the metal material; and a phase classification unit, which inputs the feature value calculated by the above-mentioned feature value calculation unit into a learning completion model that has been learned with the feature value of one or more pre-specified phase labels of the above-mentioned metallographic structure as input and the label of each phase as output, obtains the phase label of the pixel corresponding to the input feature value, and thereby classifies the phase of the metallographic structure of the above-mentioned image.

[0028] In addition, the metallographic phase classification device of the present invention is characterized in that, in the above invention, it further includes an image preprocessing unit that adjusts the brightness value of the pixels of the above image and a preprocessing adjustment unit that adjusts the preprocessing parameters of the above image preprocessing unit.

[0029] In order to solve the above-mentioned problems and achieve the purpose, the method for learning the phase of metallographic structure of the present invention is characterized in that it includes: a phase designation step, wherein, for an image obtained by photographing the metallographic structure of a metal material, labels of each phase are assigned to one or more pixels corresponding to the pre-designated metallographic structure; a feature value calculation step, wherein one or more feature values ​​are calculated for pixels assigned with labels of each phase; and a model generation step, wherein learning is performed with the above-mentioned feature values ​​assigned with labels of each phase as input and the labels of each phase as output, thereby generating a learning completion model.

[0030] In order to solve the above-mentioned problems and achieve the purpose, the phase learning device of the metallographic structure of the present invention is characterized in that it comprises: a phase designation unit, which assigns labels of each phase to one or more pixels corresponding to a pre-designated one or more pixels of the above-mentioned metallographic structure for an image obtained by photographing the metallographic structure of a metal material; a feature value calculation unit, which calculates one or more feature values ​​for pixels assigned with labels of each phase; and a model generation unit, which performs learning with the above-mentioned feature values ​​assigned with labels of each phase as input and with the labels of each phase as output, thereby generating a learning completion model.

[0031] In order to solve the above problems and achieve the purpose, the material property prediction method of the metal material of the present invention is a material property prediction method of the metal material for predicting the material properties of the metal material, wherein, after the above-mentioned metallographic structure phase classification method, it includes: a quantitative evaluation step, wherein the quantitative evaluation value of the metallographic structure is calculated by calculating the size, area ratio or shape of each phase after classification; a data selection step, wherein data used in the prediction of the material properties of the above-mentioned metal material are selected from the above-mentioned quantitative evaluation value and the material properties of the above-mentioned metal material prepared in advance; a model generation step, wherein the selected data is used to generate a prediction model for predicting the material properties of the above-mentioned metal material; and a material property prediction step, wherein the material properties of the above-mentioned metal material are predicted using the generated prediction model.

[0032] In order to solve the above-mentioned problems and achieve the purpose, the material property prediction device of the metal material of the present invention is a material property prediction device of the metal material for predicting the material properties of the metal material, wherein it comprises: an input part, which inputs an image of the phase classification of the metallographic structure; a quantitative evaluation part, which calculates the quantitative evaluation value of the metallographic structure by calculating the size, area ratio or shape of each phase after classification; a data recording part, which records the above-mentioned quantitative evaluation value in a database; a data selection part, which selects data used in the prediction of the material properties of the above-mentioned metal material from the above-mentioned quantitative evaluation value recorded in the above-mentioned database and the material properties of the above-mentioned metal material; a model generation part, which uses the selected data to generate a prediction model for predicting the material properties of the above-mentioned metal material; a material property prediction part, which uses the generated prediction model to predict the material properties of the above-mentioned metal material; and an output part, which outputs the predicted material properties of the above-mentioned metal material.

[0033] Effects of the Invention

[0034] According to the metallographic structure phase learning method, metallographic structure phase classification method, metallographic structure phase learning device, and metallographic structure phase classification device of the present invention, the characteristic value of each phase is obtained from the metallographic structure image, and the corresponding relationship between the characteristic value and each phase is learned, thereby enabling quantitative and high-speed evaluation of important metallographic structures in metal materials.

[0035] In addition, according to the metallographic structure phase learning method, metallographic structure phase classification method, metallographic structure phase learning device, and metallographic structure phase classification device of the present invention, the brightness level of the structure image can be optimized even for different metal materials, thereby enabling the structure image phase classification to be performed with high precision.

[0036] In addition, according to the material property prediction method and the material property prediction device of the metal material of the present invention, it is possible to efficiently implement quantitative evaluation based on the classification results of the phases of the metallographic structure. Therefore, by deriving the correlation between the quantitative evaluation value and the material property of the metal material, the material property of the metal material can be accurately predicted. Thus, while observing the image of the metallographic structure, the material property of the metal material can be grasped, thereby improving the efficiency of the development of metal materials (such as steel plates). BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a block diagram showing a schematic structure of a metallographic phase learning device according to an embodiment of the present invention.

[0038] Figure 2 This is a block diagram showing a schematic structure of a metallographic phase classification device according to the first embodiment of the present invention.

[0039] Figure 3It is a block diagram showing a schematic structure of a phase classification device of a metallographic structure according to a second embodiment of the present invention.

[0040] Figure 4 This is a flowchart showing the flow of a method for learning a phase of a metallographic structure according to an embodiment of the present invention.

[0041] Figure 5 This is a flowchart showing the flow of the method for classifying phases of a metallographic structure according to the first embodiment of the present invention.

[0042] Figure 6 This is a flowchart showing the flow of a method for classifying phases of a metallographic structure according to the second embodiment of the present invention.

[0043] Figure 7 It is a block diagram showing a schematic structure of a material property prediction device for a metal material according to an embodiment of the present invention.

[0044] Figure 8 This is a flowchart showing the flow of a method for predicting material properties of a metal material according to an embodiment of the present invention.

[0045] Fig. 9 An embodiment of the method for learning the phase of a metallographic structure of the present invention is a diagram showing an image captured by a scanning electron microscope in the image input step and a spectrum profile of the brightness value of the image.

[0046] Fig.10 This is an embodiment of the method for learning the phase of a metallographic structure of the present invention, and is a diagram showing an image after background removal in an image preprocessing step and a spectrum profile of the brightness value of the image.

[0047] Fig.11 An embodiment of the method for learning the phase of the metallographic structure of the present invention is a diagram showing the ferrite phase and the martensite phase specified in the phase specifying step.

[0048] Fig.12 This is an embodiment of the method for learning the phase of the metallographic structure of the present invention, and is a graph showing the identical eigenvalue, Gaussian eigenvalue, and average eigenvalue calculated in the eigenvalue calculation step.

[0049] Fig.13 This is a diagram showing a decision tree generated in the model generation step, which is an embodiment of the method for learning the phase of a metallographic structure of the present invention.

[0050] Fig.14 This is an embodiment of the method for classifying phases of a metallographic structure of the present invention, and is a diagram showing the identity eigenvalue, Gaussian eigenvalue, and average eigenvalue calculated in the eigenvalue calculation step.

[0051] Fig.15This is an example of the method for classifying phases of a metallographic structure of the present invention, and is a diagram showing an original image, classification results of the example, and classification results of a comparative example.

[0052] Fig.16 This is an example of the method for classifying phases of a metallographic structure of the present invention, and is a diagram showing an original image, an image after background removal, classification results of the example, and classification results of a comparative example.

[0053] Fig.17 The embodiment of the material property prediction method of the metal material of the present invention is to show that according to Fig.15 (b) and Fig.16 (c) is a diagram of a histogram of the roundness of the ferrite phase calculated from the structure image.

[0054] Fig.18 This is an embodiment of the material property prediction method of a metal material according to the present invention, and is a diagram showing prediction results of tensile strength obtained using a prediction model (neural network model) generated by a prediction model generation unit. DETAILED DESCRIPTION

[0055] With reference to the accompanying drawings, a metallographic structure phase classification method, a metallographic structure phase classification device, a metallographic structure phase learning method, a metallographic structure phase learning device, a metal material material property prediction method and a metal material material property prediction device according to an embodiment of the present invention are described.

[0056] The metallographic structure phase learning method and learning device of the present embodiment are methods and devices for learning the metallographic structure phases which are important information in controlling the characteristics of metal materials used as raw materials for various products such as structural components and automotive components. In addition, the metallographic structure phase classification method and classification device of the present embodiment are methods and devices for classifying the metallographic structure phases according to the learning results of the metallographic structure phase learning method and learning device. It should be noted that the metal material for which the phase learning and classification are performed in the present embodiment is, for example, a dual-phase steel plate (DP steel plate) composed of a ferrite phase and a martensite phase. Below, the structure of the metallographic structure phase learning device and classification device is described, and then the learning method and classification method using these devices are described. Then, the structure of the material property prediction device of the metal material is described, and then the material property prediction method using the device is described.

[0057] (Metallurgical structure phase learning device)

[0058] Reference Figure 1 A metallographic phase learning device (hereinafter referred to as “learning device”) 1 will be described. The learning device 1 includes an input unit 10 , an output unit 20 , and a computing unit 30 .

[0059] The input unit 10 is an input unit for inputting an image obtained by photographing the metallographic structure of a metal material (hereinafter referred to as a "tissue image") to the calculation unit 30. The tissue image can be obtained by a well-known photographing device such as an optical microscope or a scanning electron microscope that is widely used in photographing the tissue image.

[0060] Usually, sample conditioning is performed when the tissue image is taken, but the sample conditioning can use a known method. For example, in the case of taking a picture using a scanning electron microscope, the following method can be listed as a known sample conditioning. First, the sample piece is roughly ground using sandpaper, and then polished using 0.5μm to 2.0μm aluminum oxide abrasive until the sample surface becomes a mirror surface. Then, erode in a dilute corrosive solution such as 1% nitric acid ethanol solution for about 1 second. The implementation of this sample conditioning is not necessary, but it is preferred to perform mirror polishing until the pre-existing scratches on the sample surface disappear.

[0061] In addition, the shooting conditions such as the magnification when shooting the tissue image, the intensity of the light source under the optical microscope, the accelerating voltage under the electron microscope, etc. are not particularly limited, and the operator can set them according to the metallographic structure that becomes the analysis object. In addition, when shooting the tissue image, it is preferably shot after adjusting the contrast in a manner that makes the contrast of each phase clear. It should be noted that the tissue image is not limited to being obtained by a shooting device, and can also be obtained by an external or internal recording medium (such as a hard disk, a USB memory, etc.) that is shot and saved in advance, and input into the input unit 10.

[0062] The output unit 20 is an output unit that outputs the calculation results obtained by the calculation unit 30. The output unit 20 is composed of, for example, a display, a printer, or a smart phone. The output unit 20 outputs, for example, a tissue image preprocessed by the image preprocessing unit 31, the characteristic value of the phase of the metallographic structure calculated by the characteristic value calculation unit 33, and the learning results of each phase of the metallographic structure obtained by the model generation unit 34. It should be noted that the output form based on the output unit 20 is not particularly limited, and for example, it can be output in the form of data such as a text file or an image file, or projected to an output device.

[0063] The operation unit 30 is implemented, for example, by a processor including a CPU (Central Processing Unit) and a memory (main storage unit) including a RAM (Random Access Memory) and a ROM (Read Only Memory). The operation unit 30 loads the program into the working area of ​​the main storage unit and executes it, and controls each component by executing the program, thereby realizing functions consistent with the prescribed purpose. In addition, the operation unit 30 functions as an image preprocessing unit 31, a phase specifying unit 32, a eigenvalue calculating unit 33, and a model generating unit 34 by executing the above-mentioned program. It should be noted that the details of each unit will be described later (refer to Figure 4 ).

[0064] (Metallurgical Structure Phase Classification Device (Embodiment 1))

[0065] Reference Figure 2 A metallographic phase classification device (hereinafter referred to as “classification device”) 2 according to Embodiment 1 will be described. The classification device 2 includes an input unit 40 , an output unit 50 , and a calculation unit 60 .

[0066] The input unit 40 is an input unit for inputting the tissue image of the metal material to the operation unit 60. The tissue image can be obtained by a well-known photographing device such as an optical microscope or a scanning electron microscope that is widely used in photographing the tissue image. The tissue image input to the input unit 40 is subjected to sample adjustment in the same manner as the tissue image input to the input unit 10. The method of sample adjustment is the same as the method of sample adjustment for the tissue image input to the input unit 10. In addition, the shooting conditions, contrast adjustment, and acquisition of the tissue image from the recording medium when photographing the tissue image input to the input unit 40 are also the same as those for the tissue image input to the input unit 10.

[0067] The output unit 50 is an output unit that outputs the calculation result obtained by the calculation unit 60. The output unit 50 is composed of, for example, a display, a printer, or a smart phone. The output unit 50 outputs, for example, a tissue image preprocessed by the image preprocessing unit 61, the characteristic value of each phase of the metallographic structure calculated by the characteristic value calculation unit 62, and the classification result of the phase of the metallographic structure obtained by the phase classification unit 63. It should be noted that the output form based on the output unit 50 is not particularly limited, and for example, it can be output in the form of data such as a text file, an image file, or projected to an output device.

[0068] The operation unit 60 is implemented, similarly to the operation unit 30, by a processor including a CPU and a memory including a RAM, a ROM, etc. The operation unit 60 loads the program into the working area of ​​the main storage unit and executes it, and controls each component by executing the program, thereby realizing functions consistent with the prescribed purpose. In addition, the operation unit 60 functions as an image preprocessing unit 61, a feature value operation unit 62, and a phase classification unit 63 by executing the above program. It should be noted that the details of each unit will be described later (see Figure 5 ).

[0069] (Metallurgical Structure Phase Classification Device (Embodiment 2))

[0070] Reference Figure 3 A metallographic phase classification device (hereinafter referred to as “classification device”) 3 according to Embodiment 2 will be described. The classification device 3 includes an input unit 70 , an output unit 80 , and a calculation unit 90 .

[0071] The input unit 70 is an input unit for inputting a tissue image of a metal material to the operation unit 90. The tissue image can be obtained by a known photographing device such as an optical microscope or a scanning electron microscope that is widely used in photographing the tissue image. The tissue image input to the input unit 70 is subjected to sample adjustment in the same manner as the tissue image input to the input units 10 and 40. The method of sample adjustment is the same as the method of sample adjustment performed on the tissue image input to the input units 10 and 40. In addition, the photographing conditions, contrast adjustment, and acquisition of the tissue image from the recording medium when photographing the tissue image input to the input unit 70 are also the same as those for the tissue image input to the input units 10 and 40.

[0072] The output unit 80 is an output unit that outputs the calculation results obtained by the calculation unit 90. The output unit 80 is composed of, for example, a display, a printer, or a smart phone. The output unit 80 outputs, for example, a tissue image preprocessed by the image preprocessing unit 91, the characteristic values ​​of each phase of the metallographic structure calculated by the characteristic value calculation unit 92, and the classification result of the phase of the metallographic structure obtained by the phase classification unit 93. In addition, the output unit 80 also outputs the error evaluation value calculated by the error evaluation value calculation unit 94, the preprocessing conditions (preprocessing parameters) adjusted by the preprocessing adjustment unit 95, etc. It should be noted that the output form based on the output unit 80 is not particularly limited, for example, it can be output in the form of data such as a text file, an image file, etc., projected to an output device.

[0073] The operation unit 90 is implemented, similarly to the operation units 30 and 60, by a processor including a CPU and a memory including a RAM, a ROM, etc. The operation unit 90 loads the program into the working area of ​​the main storage unit and executes it, and controls each component by executing the program, thereby realizing functions consistent with the prescribed purpose. In addition, the operation unit 90 functions as an image preprocessing unit 91, a feature value calculation unit 92, a phase classification unit 93, an error evaluation value calculation unit 94, and a preprocessing adjustment unit 95 by executing the above program. It should be noted that the details of each unit will be described later (see Figure 6 ).

[0074] (Method of studying the phases of metallographic structures)

[0075] Reference Figure 4 A method for learning the phase of a metallographic structure using the learning device 1 is described. The method for learning the phase of a metallographic structure sequentially performs an image input step, an image preprocessing step, a phase specifying step, a feature value calculating step, and a model generating step. It should be noted that, as required, a sample conditioning step such as grinding the surface of the sample using a known method and a photographing step of photographing the structure image of the metal material using a known photographing device such as an optical microscope or a scanning electron microscope can be performed before the image preprocessing step.

[0076] <Image input steps>

[0077] In the image input step, the input unit 10 inputs the tissue image subjected to sample adjustment as necessary into the calculation unit 30 (step S1 ).

[0078] <Image preprocessing steps>

[0079] In the image preprocessing step, the image preprocessing unit 31 performs preprocessing of the tissue image (step S2). Here, for the tissue image (original image) input by the input unit 10, due to the difference in the sample adjustment method before shooting, and the intensity and contrast adjustment of the light source during shooting, even when the same sample is photographed using the same shooting device, there are many cases where the contrast in each captured image is different. In addition, for the tissue image input by the input unit 10, even if it is the same phase in the same image, there is a situation where the brightness value is different depending on the position in the image due to the difference in the position of the light source in the case of an optical microscope, and due to the difference in the position of the detector in the case of an electron microscope. When the brightness values ​​are different like this, not only the learning efficiency is reduced, but it also becomes a cause of error in the classification of the phase. Therefore, in the image preprocessing step, the tissue image is preprocessed by adjusting the brightness value of the pixels of the tissue image.

[0080] As preprocessing of tissue images, for example, the following processes (1) to (3) can be listed. The following processes (1) to (3) can be combined to perform a plurality of preprocessing, or only one (for example, only identity processing) can be performed as preprocessing. However, in the case of photographing metallographic structures, since there are many cases where the brightness values ​​in each tissue image are different or the brightness values ​​are different depending on the position in the same image, it is preferable to implement at least the background removal process in the following processes (1) to (3). In addition to the following processes (1) to (3), a process of removing the maximum and minimum values ​​of the overall brightness value by an amount within a specified range can also be performed.

[0081] (1) Identical processing

[0082] In the identity process, the input tissue image itself is returned as output.

[0083] (2) Background removal processing

[0084] In the background removal process, the average brightness value (background) of the tissue image is removed. There is a situation in which the tissue image as a whole has a certain brightness value or more, and the absolute value of the brightness value of each tissue image is greatly different when the tissue image is photographed using an optical microscope or an electron microscope. In addition, there is also the following situation: due to the different positions of the light source or the detector, a part looks brighter and a part looks darker. Therefore, in the background removal process, a rectangular area of ​​x pixels × y pixels is predetermined, in which the minimum value of the brightness value is obtained, and an appropriate function (such as a linear function, etc.) is used to interpolate between its pixels, and the brightness value of all pixels is subtracted by the interpolation function, thereby adjusting the brightness value. It should be noted that "the number of pixels x" and "the number of pixels y" can be arbitrarily set by the operator for each tissue image, but it is preferably set to the size of the crystal grain size of the smaller crystal grain size in the multiple phases of the metallographic structure, for example. In addition, in the case of anisotropy, it is preferably set to the size of the grain size in each x direction and y direction. In addition, the region of x pixels×y pixels does not need to be a rectangle. For example, when the tissue image is spherical, the region of x pixels×y pixels is also preferably set to be spherical. It should be noted that the details of the background removal process will be described later (see Fig. 9 and Fig.10 ).

[0085] (3) Histogram equalization

[0086] In the histogram equalization process, the histogram of the brightness value of the tissue image is equalized. The distribution of the brightness value of the tissue image photographed by an optical microscope or an electron microscope often has a significantly different average value and standard deviation in each tissue image. Therefore, standardization is performed to make the average value and standard deviation of the histogram consistent, so that the average value and deviation of the brightness value of the tissue image are uniform.

[0087] <Phase designation procedure>

[0088] In the phase designation step, the phase designation unit 32 assigns labels of each phase to pixels corresponding to a pre-designated or specific pixel of the metallographic structure for the structure image obtained by photographing the metallographic structure of the metal material (step S3). It should be noted that, for example, in the case of DP steel plate, "assigning labels of each phase to pixels of each phase in the structure image" means determining pixels corresponding to the ferrite phase and martensite in the structure image, so that the pixels of the structure image are associated with the ferrite phase and the martensite phase. It should be noted that the details of the phase designation step will be described later (refer to Fig.11 ).

[0089] <Eigenvalue calculation steps>

[0090] In the feature value calculation step, the feature value calculation unit 33 calculates one or more feature values ​​for the pixels labeled with each phase in the phase specifying step (step S4). In the feature value calculation step, for example, one or more feature values ​​(1) to (8) below are calculated.

[0091] (1) Identity eigenvalue

[0092] The identity eigenvalue is a eigenvalue representing the brightness value of the tissue image itself.

[0093] (2) Average eigenvalue

[0094] The average characteristic value is a characteristic value representing the average value of the brightness value within the specified range of the tissue image. That is, the average characteristic value is a value obtained by taking out the specified range "number of pixels x × number of pixels y" from each phase of the tissue image and averaging the brightness values ​​therein. "Number of pixels x" and "number of pixels y" can be the same size or different sizes. In addition, "number of pixels x" and "number of pixels y" are preferably set to a range of, for example, a size greater than the noise contained in the tissue image and less than 1 / 2 of the size of the crystal grain size of the smaller side of the multiple phases of the metallographic structure. In addition, in the case of anisotropy, it is preferably set to the size of the grain size in each x direction and y direction. In addition, the area of ​​x pixels × y pixels does not need to be a rectangle. For example, in the case where the tissue image is spherical, the area of ​​x pixels × y pixels is also preferably set to a spherical shape. In addition, the average characteristic value can also be calculated for multiple pixel numbers x and y. It should be noted that when the range of pixels is too large, it is affected by the grain boundary and other adjacent phases. Therefore, the range of pixels is preferably set to include a range having a size smaller than 1 / 2 of the size of the larger crystal grain.

[0095] Here, “noise included in the tissue image” refers to, for example, a portion in the tissue image where the brightness value suddenly increases (for example, see Fig.10 In addition, “making the number of pixels x, y larger than the noise” means making the number of pixels larger than the width of the noise (see part B of (b) of the same figure).

[0096] (3) Gaussian eigenvalue

[0097] The Gaussian eigenvalue is a eigenvalue representing the average value of the brightness value with a greater weight as it approaches the center within a specified range of the tissue image. That is, the Gaussian eigenvalue is a value obtained by taking out a specified range of "number of pixels x × number of pixels y" from each phase of the tissue image, and taking out an average value with a greater weight as the pixel is closer to the center. "Number of pixels x" and "number of pixels y" can be the same size or different sizes. In addition, "number of pixels x" and "number of pixels y" are preferably set to a range that is, for example, larger than the noise contained in the tissue image and smaller than 1 / 2 of the size of the crystal grain size of the smaller crystal grain size in multiple phases of the metallographic structure. In addition, in the case of anisotropy, it is preferably set to the size of the grain size in each x direction and y direction. In addition, the area of ​​x pixels × y pixels does not need to be a rectangle. For example, in the case where the tissue image is spherical, the area of ​​x pixels × y pixels is also preferably set to a spherical shape. In addition, the Gaussian eigenvalue can also be calculated for multiple numbers of pixels x and y. It should be noted that when the range of pixels is too large, it is affected by the grain boundary and other adjacent phases. Therefore, the range of pixels is preferably set to include a range having a size smaller than 1 / 2 of the size of the larger crystal grain.

[0098] When calculating the Gaussian eigenvalue, the operator can arbitrarily set the degree of weighting to be given to the center pixel, but it is preferable to use the Gaussian function represented by the following equation (1).

[0099] A·exp(-(Δx 2 +Δy 2 )) …(1)

[0100] It should be noted that Δx and Δy in the above formula (1) can be expressed as the following formulas (2) and (3).

[0101] Δx=x i -x c …(2)

[0102] Δy=y i -y c …(3)

[0103] Among them, x c ,y c : Rectangle center coordinates

[0104] x i ,y i : Rectangle position coordinates

[0105] (4) Intermediate eigenvalues

[0106] The intermediate eigenvalue is a eigenvalue representing the central value of the brightness value within the specified range of the tissue image. That is, the intermediate eigenvalue is a value obtained by taking out the specified range "number of pixels x × number of pixels y" from each phase of the tissue image and taking out the central value from the brightness value therein. "Number of pixels x" and "number of pixels y" can be the same size or different sizes. In addition, "number of pixels x" and "number of pixels y" are preferably set to a range of, for example, a size greater than the noise contained in the tissue image and less than 1 / 2 of the size of the crystal grain size of the smaller side of the multiple phases of the metallographic structure. In addition, in the case of anisotropy, it is preferably set to the size of the grain size in each x direction and y direction. In addition, the area of ​​x pixels × y pixels does not need to be a rectangle. For example, in the case where the tissue image is spherical, the area of ​​x pixels × y pixels is also preferably set to a spherical shape. In addition, the intermediate eigenvalue can also be calculated for multiple numbers of pixels x and y. It should be noted that when the range of pixels is too large, it is affected by the grain boundary and other adjacent phases. Therefore, the range of pixels is preferably set to include a range having a size smaller than 1 / 2 of the size of the larger crystal grain.

[0107] (5) Maximum eigenvalue

[0108] The maximum eigenvalue is a eigenvalue representing the maximum value of the brightness value within the prescribed range of the tissue image. That is, the maximum eigenvalue is a value obtained by taking out the prescribed range "number of pixels x × number of pixels y" from each phase of the tissue image and taking out the maximum value from the brightness value therein. "Number of pixels x" and "number of pixels y" can be the same size or different sizes. In addition, "number of pixels x" and "number of pixels y" are preferably set to a range that is, for example, larger than the noise contained in the tissue image and smaller than 1 / 2 of the size of the crystal grain size of the smaller crystal grain size in multiple phases of the metallographic structure. In addition, in the case of anisotropy, it is preferably set to the size of the grain size in each x direction and y direction. In addition, the area of ​​x pixels × y pixels does not need to be a rectangle. For example, in the case where the tissue image is spherical, the area of ​​x pixels × y pixels is also preferably set to be spherical. In addition, the maximum eigenvalue can also be calculated for multiple numbers of pixels x and y. It should be noted that when the range of pixels is too large, it is affected by the grain boundary and other adjacent phases. Therefore, the range of pixels is preferably set to include a range having a size smaller than 1 / 2 of the size of the larger crystal grain.

[0109] (6) Minimum eigenvalue

[0110] The minimum eigenvalue is a eigenvalue representing the minimum value of the brightness value within the prescribed range of the tissue image. That is, the minimum eigenvalue is a value obtained by taking out the prescribed range "number of pixels x × number of pixels y" from each phase of the tissue image and taking out the minimum value from the brightness value therein. "Number of pixels x" and "number of pixels y" can be the same size or different sizes. In addition, "number of pixels x" and "number of pixels y" are preferably set to a range of, for example, a size greater than the noise contained in the tissue image and less than 1 / 2 of the size of the crystal grain size of the smaller side of the multiple phases of the metallographic structure. In addition, in the case of anisotropy, it is preferably set to the size of the grain size in each x direction and y direction. In addition, the area of ​​x pixels × y pixels does not need to be a rectangle. For example, in the case where the tissue image is spherical, the area of ​​x pixels × y pixels is also preferably set to be spherical. In addition, the minimum eigenvalue can also be calculated for multiple numbers of pixels x and y. It should be noted that when the range of pixels is too large, it is affected by the grain boundary and other adjacent phases. Therefore, the range of pixels is preferably set to include a range having a size smaller than 1 / 2 of the size of the larger crystal grain.

[0111] (7) Derivative eigenvalue

[0112] The derivative characteristic value is a value obtained by taking out a specified range of "pixel number x × pixel number y" from each phase of the tissue image and calculating the differential values ​​of the pixels at the end thereof in the x direction and the y direction. "Pixel number x" and "pixel number y" can be the same size or different sizes. In addition, "pixel number x" and "pixel number y" are preferably set to a range that is, for example, larger than the noise contained in the tissue image and includes a size that is smaller than 1 / 2 of the grain size of the smaller grain size in multiple phases of the metallographic structure. In addition, in the case of anisotropy, it is preferably set to the size of the grain size in each x direction and y direction. In addition, the area of ​​x pixels × y pixels does not need to be rectangular. For example, in the case where the tissue image is spherical, the area of ​​x pixels × y pixels is also preferably set to be spherical. In addition, the derivative characteristic value can be calculated for multiple pixel numbers x and y. It should be noted that when the range of pixels is too large, it is affected by the grain boundary and other adjacent phases. Therefore, the range of pixels is preferably set to a range that includes a size that is smaller than 1 / 2 of the grain size of the larger grain size.

[0113] (8) Derivative addition eigenvalue

[0114] The derivative addition characteristic value is the value obtained by convolving the derivative characteristic value by calculating the above-mentioned average characteristic value, Gaussian characteristic value, intermediate characteristic value, maximum characteristic value and minimum characteristic value of the above-mentioned derivative characteristic value. The above-mentioned "number of pixels x" and "number of pixels y" can be the same size or different sizes. In addition, "number of pixels x" and "number of pixels y" are preferably set to a range of, for example, a size greater than the noise contained in the tissue image and less than 1 / 2 of the size of the crystal grain size of the smaller side of the multiple phases of the metallographic structure. In addition, in the case of anisotropy, it is preferably set to the size of the grain size in each x direction and y direction. In addition, the area of ​​x pixels × y pixels does not need to be a rectangle. For example, in the case where the tissue image is spherical, the area of ​​x pixels × y pixels is also preferably set to a spherical shape. In addition, the derivative addition characteristic value can also be calculated for multiple pixel numbers x and y. It should be noted that when the range of pixels is too large, it is affected by the grain boundary and other adjacent phases. Therefore, the range of pixels is preferably set to include a range having a size smaller than 1 / 2 of the size of the larger crystal grain.

[0115] Here, since the above-mentioned eigenvalues ​​(1) to (6) are calculated for multiple pixels of each phase, even the same phase has different eigenvalues, and a histogram of eigenvalues ​​can be made for each phase. In addition, all of the above-mentioned eigenvalues ​​(1) to (6) can be calculated, or only a part of the eigenvalues ​​can be calculated. In addition, eigenvalues ​​that combine the calculations of each eigenvalue can be added, and eigenvalues ​​not listed above can be added as needed. These selections are preferably made by the operator in a manner to improve the learning accuracy in the learning method, and eigenvalues ​​with large differences in the eigenvalues ​​of each phase are preferably used. For example, in the histogram (frequency distribution) obtained for the eigenvalues ​​of each phase, the difference in the value of the eigenvalue with the largest frequency of the eigenvalues ​​of each phase (set as distance D) is normalized using the difference between the maximum and minimum values ​​of the corresponding eigenvalue. Then, consider comparing the normalized distance D for each eigenvalue, and adopting multiple from the large normalized distance D.

[0116] In addition to the eigenvalues ​​(1) to (6), the eigenvalues ​​(7) and (8) described above may be calculated, or an additional eigenvalue combining the eigenvalues ​​may be added. Since the eigenvalues ​​(7) and (8) are calculated for a plurality of pixels of each phase in the same manner as the eigenvalues ​​(1) to (6) described above, even the same phase has different eigenvalues, and a histogram of the eigenvalues ​​can be prepared for each phase.

[0117] It should be noted that when calculating the above-mentioned eigenvalues ​​(1) to (8), the specified range "number of pixels x × number of pixels y" is taken out to calculate the eigenvalue, but the eigenvalue obtained by convolving the pixels at the center of the "number of pixels x × number of pixels y" is calculated. Then, the "number of pixels x × number of pixels y" is moved on the tissue image, and the eigenvalues ​​of each position are calculated at the same time. In addition, when the "number of pixels x × number of pixels y" is located at the end (upper, lower, left, and right ends) on the tissue image, a boundary condition is set or the number of pixels is limited to the range from the center to the end to calculate the eigenvalue. In addition, as a boundary condition, for the pixels from the center to the outside of the "number of pixels x × number of pixels y", the same eigenvalue as the center of the "number of pixels x × number of pixels y" is set. Alternatively, the eigenvalue is calculated by extrapolating from the center to the outside using an interpolation function such as a linear function, an exponential function, or a spline function.

[0118] <Model creation steps>

[0119] In the model generation step, the model generation unit 34 performs learning (machine learning) on ​​the pixels assigned with the labels of each phase, using the feature values ​​calculated in the feature value calculation step as input and the labels of each phase as output, thereby generating a learning completed model (step S5). In the model generation step, a decision tree is generated in which the feature values ​​are set as branch conditions. It should be noted that the machine learning method in the model generation step is not limited to the decision tree, for example, it can be a random forest or a neural network, but in this embodiment, a decision tree is used as an example for explanation.

[0120] Specifically, in the model generation step, binarization is repeated multiple times based on the characteristic values ​​of each phase calculated in the characteristic value calculation step, thereby classifying the phases of the metallographic structure. In this case, the degree of accuracy with which each phase is classified is set in advance based on the phase specified by the operator and the characteristic values ​​of each phase calculated in the characteristic value calculation step, and the learning of the branch based on the binarization is performed based on the set numerical information.

[0121] For example, in the case where the branch is set in a manner that implements binarization with an accuracy of 80%, the feature value is repeatedly binarized multiple times according to the specified phase and its feature value in a manner that implements the phase classification with a probability of more than 80% to perform learning, thereby making a decision tree. The above-mentioned accuracy setting can be set arbitrarily by the operator, but it is preferable to set the lower limit to more than 80%. When the accuracy is set to less than 80%, the classification accuracy decreases. In addition, on the contrary, when the accuracy is too high, the classification accuracy in the image classification after learning becomes worse due to over-learning. Therefore, it is preferable to set the upper limit of the accuracy to less than 99%.

[0122] In the model generation step, the order of binarization of each eigenvalue when binarization is performed multiple times (the order of branches) can be specified in advance by the operator, or can be randomly determined using random numbers. Since the optimal order of binarization of each eigenvalue is often unclear in advance, it is preferred to use random numbers and use a computer to explore the order of binarization of each eigenvalue that can be classified with the above accuracy or above. Similarly, since the optimal number of binarizations of each eigenvalue is often unclear in advance, it is preferred to use a computer to explore the number of binarizations of each eigenvalue that can be classified with the above accuracy or above. In addition, the eigenvalue used as a branch condition during binarization can be used as a branch condition multiple times. It should be noted that the details of the model generation step will be described later (refer to Fig.13 ). In addition, in the selection of the characteristic values ​​that become the input of the model, it is also possible to select from the distances with the largest distance D based on the distance D (including the normalized distance) between the maximum values ​​of the frequencies of the characteristic values ​​of the above-mentioned phases, so as to select in a manner that maximizes the model accuracy.

[0123] Here, after the feature value calculation step is implemented in step S4, the calculation result of the feature value can be output from the output unit 20. The output format at this time is not particularly limited, and it can be output in the form of a text file or an image file. In this way, by outputting the calculation result of the feature value in the form of a text file or an image file, the classification of each feature value can be confirmed before the phase learning is carried out. Similarly, after the learning completion model generation step is implemented in step S5, the generation result of the learning completion model (such as a decision tree) can be output from the output unit 20. The output format at this time is not particularly limited, and it can be output in the form of a text file or an image file.

[0124] According to the metallographic phase learning device and metallographic phase learning method of the above-described implementation mode, the characteristic value of each phase is obtained from the image of the metallographic structure, and the correspondence between the characteristic value and each phase is learned, thereby enabling quantitative and high-speed evaluation of important metallographic structures in metal materials.

[0125] In addition, according to the phase learning device and phase learning method of the metallographic structure of the embodiment, the captured tissue image is appropriately preprocessed, the phase of the tissue image that has been preprocessed is specified, and the characteristic value of the specified phase is calculated and recorded. Then, the characteristic value of each recorded phase is repeatedly binarized appropriately, and the binarization step that is repeatedly performed is learned. Then, based on the phase learning result, the phase of any tissue image is automatically classified, and the phase fraction of each phase can be accurately measured.

[0126] (Method for Classifying Phases of Metallographic Structure (Embodiment 1))

[0127] Reference Figure 5 The metallographic phase classification method using the classification device 2 is described. The metallographic phase classification method sequentially performs an image input step, an image preprocessing step, a characteristic value calculation step, and a phase classification step. It should be noted that, as required, a sample conditioning step such as grinding the surface of the sample using a known method and a photographing step of photographing the structure image of the metal material using a known photographing device such as an optical microscope or a scanning electron microscope can be performed before the image preprocessing step.

[0128] <Image input steps>

[0129] In the image input step, the input unit 40 inputs the tissue image subjected to sample adjustment as necessary into the calculation unit 60 (step S11 ).

[0130] <Image preprocessing steps>

[0131] In the image preprocessing step, the image preprocessing unit 61 performs preprocessing of the tissue image (step S12). Here, for the tissue image (original image) input by the input unit 40, due to the difference in the sample adjustment method before shooting, and the intensity and contrast adjustment of the light source during shooting, even when the same sample is photographed using the same shooting device, there are many cases where the contrast in each captured image is different. In addition, for the tissue image input by the input unit 40, even if it is the same phase in the same image, there is a situation where the brightness value is different depending on the position in the image due to the difference in the position of the light source in the case of an optical microscope, and due to the difference in the position of the detector in the case of an electron microscope. When the brightness values ​​are different like this, not only the learning efficiency is reduced, but it also becomes a cause of error in the classification of the phase. Therefore, in the image preprocessing step, the tissue image is preprocessed by adjusting the brightness value of the pixels of the tissue image.

[0132] As a preprocessing of tissue images, the preprocessing step of the above learning method ( Figure 4 Step S2) can also include (1) identity processing, (2) background removal processing, and (3) histogram equalization processing. It should be noted that the contents of these processing are the same as those of the preprocessing steps of the above-mentioned learning method, so the description is omitted. The above-mentioned processing (1) to (3) can be combined as a plurality of preprocessing steps, or only one (for example, only the identity processing) can be performed as preprocessing. However, in the case of photographing metallographic structures, since there are many cases where the brightness values ​​in each tissue image are different, or the brightness values ​​are different depending on the position in the same image, it is preferred to implement at least the background removal processing in the above-mentioned processing (1) to (3).

[0133] <Eigenvalue calculation steps>

[0134] In the feature value calculation step, the feature value calculation unit 62 calculates one or more feature values ​​for each pixel of the tissue image input in the image input step (step S13). In the feature value calculation step, the feature value calculation step ( Figure 4 In step S4), one or more eigenvalues ​​among (1) the identical eigenvalue, (2) the average eigenvalue, (3) the Gaussian eigenvalue, (4) the intermediate eigenvalue, (5) the maximum eigenvalue, and (6) the minimum eigenvalue are calculated in the same manner.

[0135] In addition, in the feature value calculation step, the feature value calculation step ( Figure 4 In step S4), one or more eigenvalues ​​of eigenvalues ​​(1) to (8) can be calculated by adding (7) derivative eigenvalues ​​and (8) derivative-added eigenvalues. In addition, in the eigenvalue calculation step, in addition to eigenvalues ​​(1) to (6), eigenvalues ​​(7) and (8) can also be calculated.

[0136] Here, since the above-mentioned eigenvalues ​​(1) to (6) are calculated for multiple pixels of each phase, even the same phase has different eigenvalues, and a histogram of eigenvalues ​​can be made for each phase. In addition, all of the above-mentioned eigenvalues ​​(1) to (6) can be calculated, or only a part of the eigenvalues ​​can be calculated. In addition, eigenvalues ​​that combine the calculations of each eigenvalue can be added, and eigenvalues ​​not listed above can be added as needed. These selections are preferably made by the operator in a manner to improve the learning accuracy in the learning method, and it is preferred to use eigenvalues ​​with large differences in the eigenvalues ​​of each phase.

[0137] In addition, the eigenvalues ​​(7) and (8) are calculated for multiple pixels of each phase in the same manner as the eigenvalues ​​(1) to (6) described above, so even the same phase has different eigenvalues, and a histogram of the eigenvalues ​​can be made for each phase. In addition, all of the eigenvalues ​​(1) to (8) described above can be calculated, or only a portion of the eigenvalues ​​can be calculated. In addition, eigenvalues ​​that combine the calculations of each eigenvalue can be added, and eigenvalues ​​not listed above can be added as needed. These selections are preferably made by the operator in a manner that improves the learning accuracy in the learning method, and it is preferred to use eigenvalues ​​with large differences in the eigenvalues ​​of each phase.

[0138] It should be noted that when calculating the above-mentioned eigenvalues ​​(1) to (8), the specified range "number of pixels x × number of pixels y" is taken out to calculate the eigenvalue, but the eigenvalue obtained by convolving the pixels at the center of the "number of pixels x × number of pixels y" is calculated. Then, the "number of pixels x × number of pixels y" is moved on the tissue image, and the eigenvalues ​​of each position are calculated at the same time. In addition, when the "number of pixels x × number of pixels y" is located at the end (upper, lower, left, and right ends) on the tissue image, a boundary condition is set or the number of pixels is limited to the range from the center to the end to calculate the eigenvalue. In addition, as a boundary condition, for the pixels from the center to the outside of the "number of pixels x × number of pixels y", the same eigenvalue as the center of the "number of pixels x × number of pixels y" is set. Alternatively, the eigenvalue is calculated by extrapolating from the center to the outside using an interpolation function such as a linear function, an exponential function, or a spline function.

[0139] <Phase classification steps>

[0140] In the phase classification step, the phase classification unit 63 uses the learning model (e.g., decision tree) generated in the model generation step (step S5) of the above-mentioned learning method to classify the phase of the metallographic structure (step S14). That is, in the phase classification step, the feature value calculated in the feature value calculation step is input to the learning model that has been learned by taking the feature value of the label of one or more phases specified in advance of the metallographic structure as input and the label of each phase as output. Then, by obtaining the phase label of the pixel corresponding to the input feature value, the phase of the metallographic structure of the tissue image is classified.

[0141] Here, after the feature value calculation step is performed in step S13, the feature value calculation result can be output from the output unit 50. The output format at this time is not particularly limited, and can be a text file (such as a set of numerical values) or an image file (such as Fig.12 The histogram image of Fig.14 In this way, by outputting the calculation results of the characteristic values ​​in the form of a text file or an image file, the classification of each characteristic value can be confirmed before the phase classification. Similarly, after the phase classification step is performed in step S14, the phase classification result can be output from the output unit 50. The output format at this time is not particularly limited, and it can be output in the form of a text file or an image file.

[0142] According to the metallographic structure phase classification device and metallographic structure phase classification method of embodiment 1 as described above, the characteristic value of each phase is obtained from the image of the metallographic structure, and the correspondence between the characteristic value and each phase is learned, thereby enabling quantitative and high-speed evaluation of important metallographic structures in metal materials.

[0143] In addition, according to the metallographic structure phase classification device and metallographic structure phase classification method of embodiment 1, the captured tissue image is appropriately pre-processed, the phase of the tissue image that has been pre-processed is specified, and the characteristic value of the specified phase is calculated and recorded. Then, the characteristic value of each phase recorded is repeatedly binarized appropriately, and the binarization step that is repeatedly performed is learned. Then, based on the phase learning result, the phase of any tissue image is automatically classified, and the phase fraction of each phase can be accurately measured.

[0144] (Method for Classifying Phases of Metallographic Structure (Embodiment 2))

[0145] Reference Figure 6A method for classifying phases of a metallographic structure using the classification device 3 is described. In the method for classifying phases of a metallographic structure, for a plurality of images obtained by photographing the metallographic structure of a metal material from a metal material, two or more phases of the metallographic structure are classified. The method for classifying phases of a metallographic structure sequentially performs an image input step, an image preprocessing step, a feature value calculation step, a phase classification step, an error evaluation value calculation step, and a preprocessing adjustment step. It should be noted that, as needed, a sample adjustment step of grinding the surface of the sample using a known method and a shooting step of shooting the tissue image of the metal material using a known shooting device such as an optical microscope or a scanning electron microscope can be performed before the image preprocessing step.

[0146] <Image input steps>

[0147] In the image input step, the input unit 70 inputs a portion of the sample image to the operation unit 90 (step S21). Specifically, in the image input step, one or a predetermined number of tissue images are selected from a plurality of tissue images that are taken of a metal material and subjected to sample adjustment as needed, and the selected tissue images are input to the operation unit 90.

[0148] <Image preprocessing steps>

[0149] In the image preprocessing step, the image preprocessing unit 91 performs preprocessing of the tissue image (step S22). Here, for the tissue image (original image) input by the input unit 70, due to the difference in the sample adjustment method before shooting, and the intensity and contrast adjustment of the light source during shooting, even when the same sample is photographed using the same shooting device, there are many cases where the contrast in each captured image is different. In addition, for the tissue image input by the input unit 70, even if it is the same phase in the same image, there is a situation where the brightness value is different depending on the position in the image due to the difference in the position of the light source in the case of an optical microscope, and due to the difference in the position of the detector in the case of an electron microscope. When the brightness values ​​are different like this, not only the learning efficiency is reduced, but it also becomes a cause of error in the classification of the phase. Therefore, in the image preprocessing step, the tissue image is preprocessed by adjusting the brightness value of the pixels of the tissue image.

[0150] As a preprocessing of tissue images, the preprocessing step of the above learning method ( Figure 4Step S2) can also include (1) identity processing, (2) background removal processing, and (3) histogram equalization processing. It should be noted that the contents of these processing are the same as those of the preprocessing steps of the above-mentioned learning method, so the description is omitted. The above-mentioned processing (1) to (3) can be combined as a plurality of preprocessing steps, or only one (for example, only the identity processing) can be performed as preprocessing. However, in the case of photographing metallographic structures, since there are many cases where the brightness values ​​in each tissue image are different, or the brightness values ​​are different depending on the position in the same image, it is preferred to implement at least the background removal processing in the above-mentioned processing (1) to (3).

[0151] <Eigenvalue calculation steps>

[0152] In the feature value calculation step, the feature value calculation unit 92 calculates one or more feature values ​​for each pixel of the tissue image input in the image input step (step S23). In the feature value calculation step, the feature value calculation step ( Figure 4 In step S4), one or more eigenvalues ​​among (1) the identical eigenvalue, (2) the average eigenvalue, (3) the Gaussian eigenvalue, (4) the intermediate eigenvalue, (5) the maximum eigenvalue, and (6) the minimum eigenvalue are calculated in the same manner.

[0153] In addition, in the feature value calculation step, the feature value calculation step ( Figure 4 In step S4), one or more eigenvalues ​​of eigenvalues ​​(1) to (8) can be calculated by adding (7) derivative eigenvalues ​​and (8) derivative-added eigenvalues. In addition, in the eigenvalue calculation step, in addition to eigenvalues ​​(1) to (6), eigenvalues ​​(7) and (8) can also be calculated.

[0154] Here, since the above-mentioned eigenvalues ​​(1) to (6) are calculated for multiple pixels of each phase, even the same phase has different eigenvalues, and a histogram of eigenvalues ​​can be made for each phase. In addition, all of the above-mentioned eigenvalues ​​(1) to (6) can be calculated, or only a part of the eigenvalues ​​can be calculated. In addition, eigenvalues ​​that combine the calculations of each eigenvalue can be added, and eigenvalues ​​not listed above can be added as needed. These selections are preferably made by the operator in a manner to improve the learning accuracy in the learning method, and it is preferred to use eigenvalues ​​with large differences in the eigenvalues ​​of each phase.

[0155] In addition, the eigenvalues ​​(7) and (8) are calculated for multiple pixels of each phase in the same manner as the eigenvalues ​​(1) to (6) described above, so even the same phase has different eigenvalues, and a histogram of the eigenvalues ​​can be made for each phase. In addition, all of the eigenvalues ​​(1) to (8) described above can be calculated, or only a portion of the eigenvalues ​​can be calculated. In addition, eigenvalues ​​that combine the calculations of each eigenvalue can be added, and eigenvalues ​​not listed above can be added as needed. These selections are preferably made by the operator in a manner that improves the learning accuracy in the learning method, and it is preferred to use eigenvalues ​​with large differences in the eigenvalues ​​of each phase.

[0156] It should be noted that when calculating the above-mentioned eigenvalues ​​(1) to (8), the specified range "number of pixels x × number of pixels y" is taken out to calculate the eigenvalue, but the eigenvalue obtained by convolving the pixels at the center of the "number of pixels x × number of pixels y" is calculated. Then, the "number of pixels x × number of pixels y" is moved on the tissue image, and the eigenvalues ​​of each position are calculated at the same time. In addition, when the "number of pixels x × number of pixels y" is located at the end (upper, lower, left, and right ends) on the tissue image, a boundary condition is set or the number of pixels is limited to the range from the center to the end to calculate the eigenvalue. In addition, as a boundary condition, for the pixels from the center to the outside of the "number of pixels x × number of pixels y", the same eigenvalue as the center of the "number of pixels x × number of pixels y" is set. Alternatively, the eigenvalue is calculated by extrapolating from the center to the outside using an interpolation function such as a linear function, an exponential function, or a spline function.

[0157] <Phase classification steps>

[0158] In the phase classification step, the phase classification unit 93 uses the learning model (e.g., decision tree) generated in the model generation step (step S5) of the above-mentioned learning method to classify the phase of the metallographic structure (step S24). That is, in the phase classification step, the feature value calculated in the feature value calculation step is input to the learning model that has been learned by taking the feature value of the label of one or more phases specified in advance of the metallographic structure as input and the label of each phase as output. Then, by obtaining the phase label of the pixel corresponding to the input feature value, the phase of the metallographic structure of the tissue image is classified.

[0159] Here, after the feature value calculation step is performed in step S23, the feature value calculation result can be output from the output unit 80. The output format at this time is not particularly limited, and can be in the form of a text file (e.g., a set of numerical values) or an image file (e.g., Fig.12 The histogram image of Fig.14In this way, by outputting the calculation results of the characteristic values ​​in the form of a text file or an image file, the classification of each characteristic value can be confirmed before the phase classification is performed. Similarly, after the phase classification step is performed in step S24, the phase classification result can be output from the output unit 80. The output format at this time is not particularly limited, and it can be output in the form of a text file or an image file.

[0160] <Error evaluation value calculation steps>

[0161] In the error evaluation value calculation step, the error evaluation value calculation unit 94 determines the correctness of each pixel for each phase of the metallographic structure classified in the phase classification step, and calculates an error evaluation value (step S25 ).

[0162] <Pre-treatment adjustment steps>

[0163] In the preprocessing adjustment step, the preprocessing conditions (preprocessing parameters) of the image preprocessing unit 91 are changed in such a way that the error evaluation value calculated in the error evaluation value calculation step becomes smaller, and the feature value calculation step, phase classification step and error evaluation value calculation step are repeated to adjust the preprocessing conditions.

[0164] In the preprocessing adjustment step, first, the preprocessing adjustment unit 95 determines whether the phase classification accuracy is above the specified value based on the error evaluation value calculated in the error evaluation value calculation step (step S26). When it is determined that the phase classification accuracy is above the specified value (yes in step S26), the preprocessing condition is determined to be a fixed value, and other images of the sample, that is, multiple images obtained by photographing the metallographic structure of the metal material, are input (step S27). Then, all of these images are subjected to the image preprocessing step (step S28), the feature value calculation step (step S29) and the phase classification step (step S30).

[0165] On the other hand, when it is determined that the phase classification accuracy is less than the prescribed value (No in step S26), the preprocessing adjustment unit 95 changes the preprocessing conditions (step S31) and returns to step S22. Then, until it is determined in step S26 that the phase classification accuracy is greater than the prescribed value, the image preprocessing step (step S22), the feature value calculation step (step S23), the phase classification step (step S24), and the error evaluation value calculation step (step S25) are performed.

[0166] Here, when the phase classification is performed on the organizational image of the metal material of the same steel grade or the same system after temporarily capturing the organizational image of the metal material, preparing the sample, and learning the phase classification, the classification accuracy may be poor. This is because even for the same steel grade or the same system, the contrast of the captured image may change due to differences in the etching conditions and imaging conditions of the sample.

[0167] The contrast of the tissue image can be adjusted to some extent in the above-mentioned image preprocessing step (step S22), but the preprocessing conditions (preprocessing parameters) also include parameters that can be changed. Therefore, by adjusting the preprocessing conditions according to the material, it is expected that the accuracy of phase classification can be improved.

[0168] Therefore, when multiple tissue images are collected for the same sample and the phases are classified, the processing shown in the above steps S21 to S25 is performed. That is, one or a predetermined number of tissue images are selected from the multiple tissue images and input (step S21), and image preprocessing (step S22), characteristic value calculation (step S23), and phase classification of the metallographic structure based on the previously learned learning model (step S24) are performed. Then, the results are evaluated (i.e., the error evaluation value is calculated (step S25)).

[0169] In the above step S25, for example, the phase classification result performed by a person is compared with the phase classification result in step S24. In this case, the phase classification result performed by a person is set to be positive, and the phase classification result obtained in step S24 is judged to be correct or not in pixel units, and the error is numerically quantified. For example, by calculating the total value by setting the positive pixel to "0" and the wrong pixel to "1", the error evaluation value can be obtained.

[0170] Then, in the above-mentioned step S31, the parameters of the brightness adjustment as the preprocessing conditions are changed so that the error evaluation value calculated in step S25 becomes smaller and the classification accuracy becomes equal to or higher than a predetermined threshold value (see step S26).

[0171] The preprocessing parameters changed in step S31 include, for example, the width of the average brightness value for background removal. In addition, in the homogenization of the histogram of the brightness value of the tissue image, since it is not clear what kind of average and standard deviation are desired, it is considered to also search for the value of the average and standard deviation as a reference and change the histogram of the brightness value. In the search for these parameters, a general optimization method can be used.

[0172] Then, the pre-processing conditions are optimized based on the results of the phase classification of one or a predetermined number of tissue images, and then the metallographic phase classification is performed on other tissue images of the same sample as shown in the above steps S27 to S30. In this case, basically, the pre-processing conditions can be optimized using one tissue image as a sample, but when tissue images within a few % of the entire evaluation images are used as samples, further improvement in classification accuracy can be expected.

[0173] According to the metallographic structure phase classification device and metallographic structure phase classification method of the second embodiment as described above, the brightness level of the structure image can be optimized even for different metal materials, so the phases of the structure image can be classified with high accuracy.

[0174] (Material property prediction device for metal materials)

[0175] Reference Figure 7 A material property prediction device for a metal material (hereinafter referred to as “material property prediction device”) 4 will be described. The material property prediction device 4 includes an input unit 100 , an output unit 110 , a calculation unit 120 , and a storage unit 130 .

[0176] The input unit 100 is an input unit for inputting an image in which the metallographic phases are classified (hereinafter referred to as a "classified image") to the operation unit 120. The classified image is an image in which the metallographic phases are classified by the above-mentioned metallographic phase classification method, or an image in which the metallographic phases are classified by other methods such as binarization of brightness values, etc.

[0177] The output unit 110 is an output unit that outputs the calculation results obtained by the calculation unit 120. The output unit 110 is composed of, for example, a display, a printer, or a smart phone. The output unit 110 outputs, for example, the quantitative evaluation value of the metallographic structure calculated by the quantitative evaluation unit 121, the data recorded in the database of the storage unit 130, the prediction result (material properties of the metal material) obtained by the material property prediction unit 125, etc. It should be noted that the output form based on the output unit 110 is not particularly limited, for example, it can be output in the form of data such as a text file, an image file, etc., or projected to an output device.

[0178] The operation unit 120 is implemented, similarly to the operation units 30, 60, and 90, by a processor including a CPU and a memory including a RAM, a ROM, and the like. The operation unit 120 loads the program into the working area of ​​the main storage unit and executes it, and controls each component by executing the program, thereby realizing functions consistent with the prescribed purpose. In addition, the operation unit 120 functions as a quantitative evaluation unit 121, a data recording unit 122, a data selection unit 123, a model generation unit 124, and a material property prediction unit 125 by executing the above-mentioned program. It should be noted that the details of each unit will be described later (see Figure 8 ).

[0179] The storage unit 130 is composed of recording media such as EPROM (Erasable Programmable ROM), hard disk drive (HDD), solid state drive (SSD), and removable media. Examples of removable media include USB (Universal Serial Bus) memory, CD (Compact Disc), DVD (Digital Versatile Disc), and optical disk recording media such as BD (Blu-ray (registered trademark) Disc). In addition, the storage unit 130 can store an operating system (OS), various programs, various tables, various databases, and the like.

[0180] The storage unit 130 stores, for example, a database recording predetermined data, a prediction model (learning completed model) generated by the model generation unit 124, and the like. The database records, for example, the quantitative evaluation value of the metallographic structure calculated by the quantitative evaluation unit 121, the material property value (steel plate data) of the metal material obtained in advance through mechanical testing, etc., the component composition of the metal material, the metal material, and the like.

[0181] (Material property prediction method for metal materials)

[0182] Reference Figure 8 The metallographic phase learning method using the material property prediction device 4 is described. In the metallographic phase learning method, after the above-mentioned metallographic phase classification method is performed, the classification result (classification image) is used for implementation. The metallographic phase learning method sequentially performs an image input step, a quantitative evaluation step, a data recording step, a data selection step, a model generation step, a material property prediction step, and a prediction result output step.

[0183] <Image input steps>

[0184] In the image input step, the input unit 100 inputs the classification image to the calculation unit 120 (step S41 ).

[0185] <Quantitative evaluation steps>

[0186] In the quantitative evaluation step, the quantitative evaluation unit 121 calculates the quantitative evaluation value of the metallographic structure by performing quantitative evaluation on each phase included in the classified image (step S42). In the quantitative evaluation step, for example, the following quantitative evaluation values ​​(1) to (5) are calculated.

[0187] (1) Area ratio

[0188] By finding the area of ​​the classified phases, the area ratio of the phases is calculated.

[0189] (2) Major diameter, minor diameter, and aspect ratio

[0190] By performing elliptical approximation on the shape of each crystal grain of the classified phase, the major axis, minor axis or aspect ratio of the ellipsoid is calculated.

[0191] (3) Feret diameter

[0192] A straight line is drawn from the interface of each crystal grain of the classified phase, and the Feret diameter at which the straight line has the maximum distance is calculated.

[0193] (4) Average diameter

[0194] The area of ​​each crystal grain of the classified phase is calculated, and the square root of the area is taken to derive the average diameter of the crystal grains.

[0195] (5) Roundness

[0196] The area and perimeter of each crystal grain of the classified phase are obtained, and the roundness of the crystal grain is calculated according to the following formula (4). It should be noted that when the crystal grain is a perfect circle, the roundness is 1.0, and conversely, the further the crystal grain is from a perfect circle, the smaller the roundness is.

[0197]

[0198] Where, C: roundness

[0199] S: Area

[0200] P: circumference

[0201] Here, since the quantitative evaluation values ​​(2) to (5) are calculated for each crystal grain, a plurality of numerical values ​​can be obtained even for one structure image, and a histogram of each quantitative evaluation value can be created.

[0202] <Data recording procedure>

[0203] In the data recording step, the data recording unit 122 records the quantitative evaluation value of the metallographic structure calculated in the quantitative evaluation step in the database of the storage unit 130 (step S43 ).

[0204] <Data selection steps>

[0205] In the data selection step, the data selection unit 123 selects (extracts) data for predicting the material properties of the metal material from the data of the quantitative evaluation values ​​of the metallographic structure and the material properties of the metal material recorded in the database (step S44 ).

[0206] It should be noted that, since the quantitative evaluation value is calculated for each crystal grain, multiple values ​​can be obtained for one tissue image. For example, since the average diameter of the quantitative evaluation value (4) obtains an average diameter for each crystal grain, multiple values ​​are obtained for one image. Among these multiple numerical information, the average value can be calculated, and only the average value is used for the prediction of material properties, or the standard deviation can be used for the prediction of material properties. In addition, in the data selection step, in the prediction model generation step described later, it is preferred to select a quantitative evaluation value with good prediction accuracy of material properties. Therefore, for example, in the material property prediction step described later, when the prediction accuracy is poor, it is preferred to return to this step and implement data selection again. It should be noted that in addition to the quantitative evaluation value of the metallographic structure, the component composition and heat treatment conditions of the metal material can also be input.

[0207] <Prediction model generation steps>

[0208] In the prediction model generation step, the model generation unit 124 generates a prediction model for predicting the material properties of the metal material using the data selected in the data selection step (step S45). In the prediction model generation step, specifically, the prediction model for predicting the material properties is generated using the quantitative evaluation value, the component composition of the metal material, the heat treatment conditions, etc. selected in the data selection step. At this time, the prediction model is also generated using the data of the material properties of the metal material selected in the data selection step.

[0209] The prediction model can be generated using models such as neural network, support vector regression, Gaussian process regression, etc., or it can be generated as a simple regression equation such as linear regression. In addition, in the generation of the prediction model, it is preferred to use multiple prediction models to predict material properties and use the prediction model with the highest prediction accuracy.

[0210] Regarding the prediction accuracy of material properties, for example, the X-axis takes the measured value and the Y-axis takes the predicted value, and a two-dimensional graph is used to confirm the degree of consistency of the data of the two, or the parameters obtained by adding the prediction errors of each data and dividing it by the number of data are used for evaluation. In addition, the prediction accuracy of material properties is preferably evaluated according to the steps described below. First, the data selected from the database are divided into data (training data) for parameter fitting in the prediction model and data (test data) not used for fitting. Then, the prediction accuracy of material properties is evaluated according to the consistency of the predicted value and the measured value of the test data. It should be noted that when the data selected from the database are divided into training data and test data, it can be selected and divided by the operator, or it can be randomly determined using random numbers after determining the ratio of training data and test data.

[0211] <Material property prediction steps>

[0212] In the material property prediction step, the material property prediction unit 125 predicts the material property of the metal material using the prediction model generated in the prediction model generation step (step S46). In the material property prediction step, specifically, the material property of the metal material is predicted by inputting the quantitative evaluation value selected in the data selection step into the prediction model generated in the prediction model generation step.

[0213] <Prediction result output steps>

[0214] In the prediction result output step, the output unit 110 outputs the prediction result in the material property prediction step, that is, the material property of the metal material (step S47 ).

[0215] Here, in the previous methods, it is difficult to perform efficient classification of metallographic phases and quantitative evaluation of metallographic images using the classified phases, so it is difficult to accurately predict material properties from the tissue image. On the other hand, according to the material property prediction method and material property prediction device of the metal material of the present embodiment, it is possible to efficiently implement quantitative evaluation based on the classification results of the metallographic phases. Therefore, by deriving the correlation between the quantitative evaluation value and the material property of the metal material, the material property of the metal material can be predicted with good accuracy. As a result, while observing the image of the metallographic structure, the material property of the metal material can be grasped, so the efficiency of the development of metal materials (such as steel plates) can be improved.

[0216] In addition, according to the material property prediction method and the material property prediction device of the metal material of the present embodiment, by efficiently segmenting the tissue image, it is possible to efficiently perform quantitative evaluation of the metallographic structure, unlike in the past. In addition, by using quantitative indicators in this way, it is possible to accurately predict the material properties of the photographed tissue image.

[0217] [Example]

[0218] (Example 1)

[0219] Reference Figures 9 to 16 Embodiment 1 of the method for learning the phase of a metallographic structure and the method for classifying the phase of a metallographic structure of the present invention is described. In this embodiment, as the method for learning the phase of a metallographic structure, a sample adjustment step, a shooting step, an image input step, an image preprocessing step, a phase specifying step, a feature value calculation step, and a model generation step are performed. In addition, as the method for classifying the phase of a metallographic structure, an image input step, a feature value calculation step, and a phase classification step are performed.

[0220] First, the DP steel sheet (steel sheet composed of a ferrite phase and a martensite phase) was mirror-polished and etched with nital (sample conditioning step), and the metallographic structure was observed using a scanning electron microscope (imaging step). Fig. 9 (a) shows a tissue image (original image) taken using a scanning electron microscope. Fig. 9 (b) shows the spectrum profile of the brightness value in the central part (position of the L1 line) of the tissue image.

[0221] Next, the tissue image is input to the learning device 1 (image input step), and an image preprocessing step is performed. Fig.10 (a) is a tissue image after background removal in the image preprocessing step. Fig.10 (b) shows the spectrum profile of the brightness value at the center of the tissue image (the position of the L2 line). In the image preprocessing step, the minimum brightness value is obtained for every 50 pixels to remove the background.

[0222] Next, a phase specifying step of specifying a ferrite phase and a martensite phase is performed on the structure image subjected to the pre-processing of background removal. Fig.11 The ferrite phase and martensite phase specified in the phase specifying step are shown. In this figure, the area surrounded by the solid line is the ferrite phase, and the area surrounded by the dotted line is the martensite phase. In this embodiment, three ferrite phases and four martensite phases are specified, but the number of specified phases can be determined according to mechanical specifications, etc.

[0223] Next, the eigenvalues ​​of the ferrite phase and the martensite phase are calculated (eigenvalue calculation step). In the eigenvalue calculation step, for the ferrite phase and the martensite phase, the identity eigenvalue, the average eigenvalue of 4 pixels, 8 pixels, and 16 pixels, the Gaussian eigenvalue, the intermediate eigenvalue, the maximum eigenvalue, and the minimum eigenvalue are calculated respectively. Among these 16 eigenvalues, Fig.12 (a) shows a histogram representing the identity eigenvalue, Fig.12 (b) shows a histogram representing the Gaussian eigenvalues ​​calculated for a size of 4 pixels, Fig.12 (c) shows a histogram representing the average feature value calculated for a size of 4 pixels. It should be noted that Fig.12 The numbers in parentheses of the eigenvalues ​​in (b) and (c) indicate the pixel size when the eigenvalues ​​were calculated.

[0224] It should be noted that if Fig.12As shown, when multiple eigenvalues ​​are calculated by the eigenvalue calculation step, the histogram of each eigenvalue can be confirmed, and the eigenvalues ​​learned in the model generation step of the latter section can be selected. In this case, as shown in the figure, in the distribution of the eigenvalues ​​of each phase, it is preferred to select the eigenvalues ​​whose distance D between the values ​​of the high-frequency part is as far as possible as the eigenvalues ​​learned in the model generation step. Since the distance D between the values ​​of the high-frequency part is far away, it means that the eigenvalues ​​of each phase are easy to separate, by learning the eigenvalues ​​whose distance D is as far as possible, a learning completion model with good accuracy can be generated. For example, in the histogram (frequency distribution) obtained for the eigenvalues ​​of each phase, the difference in the value of the eigenvalue with the largest frequency of the eigenvalues ​​of each phase (set as distance D) is normalized using the difference between the maximum and minimum values ​​of the corresponding eigenvalues. Then, consider comparing the normalized distance D for each eigenvalue, and adopting multiple from the large normalized distance D.

[0225] After the characteristic values ​​are calculated in this way, binarization is repeatedly performed multiple times in the model generation step to perform phase learning. In this embodiment, for test data randomly extracted from the distribution of characteristic values ​​of the ferrite phase and the martensite phase, binarization is repeatedly performed multiple times to perform learning in a manner that classification is achieved with a probability of more than 90%. Fig.13 FIG. 4 shows a part of the decision tree generated by the learning result. It should be noted that Fig.13 The numbers in parentheses of the eigenvalues ​​set in the branch conditions of represent the pixel size when calculating the eigenvalue. In this learning example, when the average eigenvalue calculated for 16 pixels is less than 31.5 and the identity eigenvalue is less than 12.5, the pixel is determined to be ferrite.

[0226] Next, according to Fig.13 First, the tissue image (refer to Fig.10 ) is input into the classification device 2 (image input step), and the characteristic values ​​of the ferrite phase and the martensite phase are calculated (characteristic value calculation step).

[0227] In the eigenvalue calculation step, the identity eigenvalue, the average eigenvalue of 4 pixels, 8 pixels, and 16 pixels, the Gaussian eigenvalue, the intermediate eigenvalue, the maximum eigenvalue, and the minimum eigenvalue are calculated for the ferrite phase and the martensite phase, respectively. Among these 16 eigenvalues, Fig.14 (a) shows an image representing the identity eigenvalue, Fig.14 (b) shows an image representing the Gaussian eigenvalues ​​calculated for a size of 16 pixels, Fig.14 (c) shows an image showing the average feature value calculated for a size of 16 pixels. It should be noted that Fig.14The numbers in parentheses of the eigenvalues ​​in (b) and (c) indicate the pixel size when the eigenvalues ​​were calculated.

[0228] Next, the eigenvalues ​​calculated in the eigenvalue calculation step are classified using the decision tree (branch diagram of the ferrite phase and the martensite phase) generated by the learning device 1. The results are shown in Fig.15 . Fig.15 (a) is the original image before preprocessing. Fig.15 (b) is the classification result of the embodiment, Fig.15 (c) is a classification result of a comparative example. The classification result of the comparative example shows the result of classifying the feature values ​​of the tissue image using the conventional binarization method.

[0229] In the classification results of the comparative example, Fig.15 As shown in (c), there are many cases where the phase is classified as ferrite phase although it is a martensite phase, and vice versa, the phase is classified as martensite phase although it is a ferrite phase, and the classification accuracy is low. Therefore, even if the areas of the white part and the black part of the binary image are calculated, the phase fraction of the ferrite phase and the martensite phase cannot be accurately calculated.

[0230] On the other hand, in the classification results of the embodiment, if Fig.15 As shown in (b), the accuracy is higher than that of the comparative example, and the ferrite phase and the martensite phase can be classified very efficiently compared with the method of distinguishing the colors of the phases by hand painting and classifying them by manpower.

[0231] (Example 2)

[0232] Reference Fig.16 Embodiment 2 of the method for classifying the phase of a metallographic structure of the present invention is described. In this embodiment, feature values ​​are calculated for a tissue image that has undergone an image preprocessing step (background removal) and a tissue image that has not undergone an image preprocessing step (background removal), and the phase of the metallographic structure is classified using the learning result (e.g., decision tree) of the above-mentioned embodiment 1. That is, in this embodiment, the difference in the classification accuracy of the phase based on the presence or absence of the image preprocessing step is studied.

[0233] First, the DP steel sheet (steel sheet composed of a ferrite phase and a martensite phase) was mirror-polished and etched with nital (sample conditioning step), and the metallographic structure was observed using a scanning electron microscope (imaging step). Fig.16 (a) shows a tissue image taken by a scanning electron microscope. Next, the tissue image is input to the classification device 2 (image input step), and an image preprocessing step is performed. Fig.16 (b) shows the tissue image after background subtraction is performed in the image preprocessing step.

[0234] Next, the eigenvalues ​​of the ferrite phase and the martensite phase are calculated for the tissue image after background removal in the image preprocessing step. In the eigenvalue calculation step, the constant eigenvalue, the average eigenvalue of 4 pixels, 8 pixels, and 16 pixels, the Gaussian eigenvalue, the intermediate eigenvalue, the maximum eigenvalue, and the minimum eigenvalue are calculated for the ferrite phase and the martensite phase, respectively.

[0235] Next, each eigenvalue calculated in the eigenvalue calculation step is classified using the decision tree (branch diagram of the ferrite phase and the martensite phase) generated in the first embodiment described above. Fig.16 (c) is the classification result of the embodiment, Fig.16 (d) is the classification result of the comparative example. The classification result of the embodiment shows the result of classifying the feature values ​​of the tissue image with background removed using the decision tree that learns the feature values ​​of the tissue image with background removed. On the other hand, the classification result of the comparative example shows the result of classifying the feature values ​​of the tissue image without background removed using the decision tree that learns the feature values ​​of the tissue image without background removed.

[0236] In the classification results of the embodiment, Fig.16 As shown in (c), the classification accuracy is high, and the ferrite phase and the martensite phase can be classified very efficiently compared with the method of distinguishing the colors of the phases by hand painting and classifying them by manpower.

[0237] On the other hand, in the classification results of the comparative example, Fig.16 As shown in (d), there are many cases where the phase is classified as ferrite phase although it is a martensite phase, and vice versa, the phase is classified as martensite phase although it is a ferrite phase, and the classification accuracy is low. Therefore, even if the areas of the white part and the black part of the binary image are calculated, the phase fraction of the ferrite phase and the martensite phase cannot be accurately calculated.

[0238] (Example 3)

[0239] Reference Fig.17 and Fig.18 A third embodiment of the material property prediction method and the material property prediction device of a metal material according to the present invention will be described.

[0240] In this embodiment, the classification result of the above-mentioned embodiment 1 is first used (refer to Fig.15 (b)) and the classification results of Example 2 (refer to Fig.16 (c)) performs quantitative evaluation of the metallographic structure (quantitative evaluation step). At this time, histograms of the area ratio and circularity are calculated for the ferrite phase and the martensite phase. Fig.17 (a) shows the classification result of Example 1 (refer to Fig.15(b)) Histogram of the calculated roundness of the ferrite phase, Fig.17 (b) shows the classification result of Example 2 (refer to Fig.16 (c)) Histogram of the calculated roundness of the ferrite phase.

[0241] Next, in addition to the average values ​​and quantitative evaluation values ​​of the area ratios and circularity of the ferrite phase and martensite phase among the calculated quantitative evaluation values, the component composition of the metal material is selected (data selection step), and these data are used for prediction of material properties.

[0242] Next, data of 100 steel types were randomly extracted from the database of DP steel plates including the structure image, the component composition of the metal material, and the tensile strength. Then, the extracted data were similarly classified into phases, and the average values ​​of the area ratio and roundness of the ferrite phase and martensite phase were calculated.

[0243] Next, a prediction model for predicting tensile strength is generated from the above quantitative evaluation value and the component composition of the metal material (prediction model generation step). It should be noted that here, the extracted data is randomly divided into training data and test data at a ratio of 9:1. In addition, a prediction model for predicting tensile strength is generated using a neural network model.

[0244] Next, in order to verify the prediction accuracy of the prediction model, the measured values ​​and predicted values ​​of tensile strength were compared. Fig.18 The predicted results of tensile strength obtained by using the neural network model generated by the model generation unit are shown. In the figure, the horizontal axis represents the measured value of the tensile strength obtained by standardization using the average value and standard deviation of the tensile strength extracted from the database. In addition, the vertical axis represents the predicted value of the tensile strength obtained by standardization using the average value and standard deviation of the tensile strength extracted from the database. In addition, in the figure, the circular scatter points represent the predicted results of the tensile strength of the samples (training data) used for parameter adjustment in the neural network model. In addition, the square scatter points represent the predicted results of the tensile strength of the samples (test data) not used for parameter adjustment.

[0245] like Fig.18 As shown, the prediction accuracy of material properties is good for both training data and test data, and the tensile strength can be predicted with good accuracy by using the area ratio of ferrite phase and martensite phase, the average value of roundness, and the component composition of the metal material. In this way, by using the method of the present invention, the material properties can be predicted with good accuracy from the tissue image and the component composition of the metal material, thereby improving the efficiency of steel plate development.

[0246] The above specifically describes the metallographic phase classification method, metallographic phase classification device, metallographic phase learning method, metallographic phase learning device, metallographic material material property prediction method and metal material material property prediction device of the present invention through specific implementation methods and examples, but the subject matter of the present invention is not limited to these records and should be interpreted broadly based on the records of the claims. In addition, it goes without saying that the schemes obtained by making various changes and modifications based on these records are also included in the subject matter of the present invention.

[0247] Here, the method for learning the phase of the metallographic structure, the method for classifying the phase of the metallographic structure, and the method for predicting the material properties of the metal material of the present invention can also be implemented by importing the software installed with the method into a general commercially available computer. A general commercially available computer is, for example, a CPU having a command to execute a program as software for realizing various functions, a recording medium (such as a hard disk, a USB memory) having the above-mentioned software and various data recorded in a form readable by a computer (or CPU), a RAM for expanding the above-mentioned program, and a computing unit such as a GPU as a processor dedicated to image processing. In addition, it can be implemented not only by a commercially available computer, but also by importing software into a cloud computer on the network.

[0248] In addition, if Figure 1 , Figure 2 , Figure 3 and Figure 7 As shown, the learning device 1, the classification devices 2, 3, and the material property prediction device 4 are described as different structures, but they can also be implemented by different devices, or can be implemented by one device. In the case where the learning device 1, the classification devices 2, 3, and the material property prediction device 4 are implemented by one device, the input unit 10, the input unit 40, the input unit 70 and the input unit 100, the output unit 20, the output unit 50, the output unit 80 and the output unit 110, and the operation unit 30, the operation unit 60, the operation unit 90 and the operation unit 120 can be physically the same.

[0249] In addition, in the present embodiment, although the dual-phase steel plate is described as an example, the present invention is also applicable to steel plates having three or more phases.

[0250] Explanation of symbols

[0251] 1 Learning device

[0252] 10 Input section

[0253] 20 Output section

[0254] 30 Operation unit

[0255] 31 Image preprocessing unit

[0256] 32 Phase designation unit

[0257] 33 Eigenvalue calculation unit

[0258] 34 Model Generation Department

[0259] 2 Classification device

[0260] 40 Input section

[0261] 50 Output

[0262] 60 Operation unit

[0263] 61 Image Preprocessing Unit

[0264] 62 Eigenvalue calculation unit

[0265] 63 Phase Classification

[0266] 3 Classification device

[0267] 70 Input section

[0268] 80 Output

[0269] 90 Operation unit

[0270] 91 Image Preprocessing Unit

[0271] 92 Eigenvalue calculation unit

[0272] 93 Phase Classification

[0273] 94 Error evaluation value calculation unit

[0274] 95 Pre-treatment adjustment unit

[0275] 4 Material property prediction device

[0276] 100 Input section

[0277] 110 Output section

[0278] 120 Operation unit

[0279] 121 Quantitative Evaluation Department

[0280] 122 Data Recording Department

[0281] 123 Data Selection Department

[0282] 124 Model Generation Department

[0283] 125 Material Properties Prediction Department

[0284] 130 Storage

Claims

1. A method for classifying phases of a metallographic structure, which is a method for classifying phases of a metallographic structure of a metal material by classifying two or more phases of the metallographic structure from a plurality of images obtained by photographing the metallographic structure of the metal material, characterized in that: include: An image preprocessing step, wherein one or a specified number of images are selected from the plurality of images, and brightness values ​​of pixels of the selected images are adjusted; a feature value calculation step, wherein one or more feature values ​​are calculated for pixels of the image preprocessed in the image preprocessing step; a phase classification step, wherein the feature value calculated in the feature value calculation step is input to a learning model that has been learned by taking feature values ​​assigned to labels of a plurality of phases of the metallographic structure as input and taking labels of each phase as output, and the phase of the metallographic structure of the image is classified by obtaining the label of the phase of the pixel corresponding to the input feature value; an error evaluation value calculation step, wherein, for each phase of the metallographic structure classified in the phase classification step, the correctness of each pixel is determined and an error evaluation value is calculated; and a preprocessing adjustment step, wherein the preprocessing parameters of the image preprocessing step are changed in such a way that the error evaluation value calculated in the error evaluation value calculation step becomes smaller, and the preprocessing parameters are adjusted by repeating the feature value calculation step, the phase classification step and the error evaluation value calculation step, The preprocessing parameters adjusted in the preprocessing adjustment step are determined as fixed values, and the image preprocessing step, the feature value calculation step and the phase classification step are all performed on a plurality of images obtained by photographing the metallographic structure of the metal material.

2. The method for classifying phases of a metallographic structure according to claim 1, characterized in that: In the eigenvalue calculation step, one or more eigenvalues ​​of the following are calculated: An identity eigenvalue representing the brightness value of the image; an average characteristic value representing an average value of brightness values ​​within a specified range of the image; A Gaussian eigenvalue representing an average value of brightness values ​​within a specified range of the image, with the values ​​being weighted more as they are closer to the center; An intermediate characteristic value representing a central value of brightness values ​​within a specified range of the image; a maximum eigenvalue representing the maximum value of brightness values ​​within a specified range of the image; A minimum eigenvalue representing the minimum value of brightness values ​​within a specified range of the image; A derivative eigenvalue representing a differential value of a brightness value of the image; A derivative addition eigenvalue is obtained by adding the derivative eigenvalues.

3. The method for classifying phases of a metallographic structure according to claim 2, characterized in that: The specified range is a range that includes a size smaller than 1 / 2 of the size of the smaller grain size in one or more phases specified in advance in the metallographic structure and a size smaller than 1 / 2 of the size of the larger grain size.

4. The method for classifying phases of a metallographic structure according to any one of claims 1 to 3, characterized in that: The metal material is a dual-phase steel plate, In the phase classification step, the ferrite phase and the martensite phase are classified.

5. The method for classifying phases of a metallographic structure according to any one of claims 1 to 3, characterized in that: The learning completion model is a machine learning model including a decision tree, a random forest or a neural network.

6. A metallographic phase classification device, which is a metallographic phase classification device for classifying two or more metallographic phases based on a plurality of images obtained by photographing the metallographic structure of a metal material, characterized in that: have: An image preprocessing unit, which selects one or a specified number of images from the plurality of images and adjusts the brightness values ​​of pixels of the selected images; a feature value calculation unit that calculates one or more feature values ​​for pixels of the image preprocessed by the image preprocessing unit; a phase classification unit that inputs the feature value calculated by the feature value calculation unit into a learning model that has been learned by taking the feature value of the plurality of phases of the metallographic structure as input and taking the label of each phase as output, obtains the label of the phase of the pixel corresponding to the input feature value, and thereby classifies the phase of the metallographic structure of the image; an error evaluation value calculation unit that determines the correctness of each pixel for each phase of the metallographic structure classified by the phase classification unit and calculates an error evaluation value; as well as a preprocessing adjustment unit that changes a preprocessing parameter used in the image preprocessing unit so as to reduce the error evaluation value calculated by the error evaluation value calculation unit, and adjusts the preprocessing parameter by repeatedly calculating the characteristic value, classifying the phase, and calculating the error evaluation value, The preprocessing parameters adjusted by the preprocessing adjustment unit are determined as fixed values, and the image preprocessing, the characteristic value calculation, and the phase classification are performed on all of the plurality of images obtained by photographing the metallographic structure of the metal material.

7. The phase classification device of metallographic structure according to claim 6, characterized in that: Also available: a phase specifying unit that assigns labels of each phase to pixels corresponding to one or more pixels specified in advance in the metallographic structure of the metal material in an image obtained by photographing the metallographic structure of the metal material; a feature value calculation unit that calculates one or more feature values ​​for pixels to which labels of each phase are assigned; and The model generation unit generates a learned model by performing learning using the feature value to which the label of each phase is assigned as an input and the label of each phase as an output.

8. A method for predicting material properties of a metal material, which is a method for predicting material properties of a metal material, in, The method for classifying phases of a metallographic structure according to any one of claims 1 to 5, comprising: A quantitative evaluation step, wherein the quantitative evaluation value of the metallographic structure is calculated by calculating the size, area ratio or shape of each phase after classification; a data selection step, wherein data used in predicting the material properties of the metal material are selected from the quantitative evaluation value and the material properties of the metal material prepared in advance; a model generating step, wherein the selected data is used to generate a prediction model for predicting material properties of the metal material; and The step of predicting material properties includes predicting the material properties of the metal material using the generated prediction model.

9. A material property prediction device for a metal material, which is a material property prediction device for a metal material for predicting the material property of the metal material, comprising: The phase classification device of the metallographic structure according to claim 6 or 7; An input unit that inputs an image of a metallographic structure phase classified by the metallographic structure phase classification device; A quantitative evaluation unit that calculates a quantitative evaluation value of a metallographic structure by calculating a size, an area ratio, or a shape of each phase after classification; a data recording unit, which records the quantitative evaluation value in a database; a data selection unit that selects data used in predicting the material properties of the metal material from among the quantitative evaluation values ​​and the material properties of the metal material recorded in the database; a model generating unit that generates a prediction model for predicting material properties of the metal material using the selected data; a material property prediction unit, which uses the generated prediction model to predict the material property of the metal material; as well as An output unit outputs the predicted material properties of the metal material.

Images