Physical property estimation device, physical property estimation system, physical property estimation method, program, and recording medium

Abstract

This technology is advantageous for improving the accuracy of estimating the physical properties of a sample. [Solution] The physical property estimation device includes a processor. The processor receives inputs: a first image in which a first imaging range of the sample is captured at a first magnification, a first physical property value in the first range of the sample, and a second image in which a second imaging range wider than the first imaging range of the sample is captured at a second magnification lower than the first magnification. The processor uses at least the first image, the first physical property value, and the second image to estimate the second physical property value in the second range wider than the first range of the sample.

Description

Technical Field

[0001] The present disclosure relates to a physical property estimation device, a physical property estimation system, a physical property estimation method, a program, and a recording medium.

Background Art

[0002] In material development, raw materials to be used are selected, the blending amounts of the materials are determined, and samples are prepared using various processing processes such as stirring, kneading, and heating. As an evaluation of the prepared samples, physical property values of the samples are obtained. By controlling microscopic structures such as the structure, phase, and distribution of the blended fillers of the materials in these material blending and production processes, materials satisfying desired physical property values are searched for. In such material development, data-driven material development is carried out, and a device for estimating physical property values using machine learning based on information on material structures such as the spectra, images, or graphs of materials has been reported.

[0003] By the way, among physical properties, the physical property values obtained in the microscopic region and those obtained in the macroscopic region may be different. Also, although it is possible to measure physical properties in the microscopic region, it may be difficult to measure physical properties in the macroscopic region.

[0004] Patent Document 1 discloses a device that estimates a value related to the ductility of a steel material using an estimation model created by machine learning with the feature amounts extracted from a plurality of images of the steel material taken with different magnifications as inputs.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, the method described in Patent Document 1 requires that the image contain microscopic material structure information that exhibits the physical properties, and the accuracy of property estimation was not always sufficient when applied to other samples. [Means for solving the problem]

[0007] This disclosure provides a technique that is advantageous for improving the accuracy of estimating the physical properties of a sample.

[0008] A first aspect of the present disclosure is a physical property estimation device comprising a processor, wherein the processor receives inputs of a first image in which a first imaging range of a sample is imaged at a first magnification, a first physical property value in the first range of the sample, and a second image in which a second imaging range of the sample, which is wider than the first imaging range, is imaged at a second magnification lower than the first magnification, and estimates a second physical property value in the second range of the sample, which is wider than the first range, using at least the first image, the first physical property value, and the second image.

[0009] A second aspect of this disclosure is a method for estimating physical properties using a processor, characterized in that the processor receives inputs of a first image in which a first imaging range of a sample is imaged at a first magnification, a first physical property value in the first range of the sample, and a second image in which a second imaging range of the sample, which is wider than the first imaging range, is imaged at a second magnification lower than the first magnification, and the processor estimates a second physical property value in the second range of the sample, which is wider than the first range, based on the first image, the first physical property value, and the second image. [Brief explanation of the drawing]

[0010] [Figure 1] This is an explanatory diagram showing the schematic configuration of the physical property estimation system according to Embodiment 1. [Figure 2] (a) is a functional block diagram of the CPU of the physical property estimation device according to Embodiment 1. (b) is an explanatory diagram of an example of training data according to Embodiment 1. (c) is an explanatory diagram of an example of input data and output data which are inference results used for inference according to Embodiment 1. [Figure 3] (a) is a schematic diagram of a micro-test specimen according to Embodiment 1. (b) is a schematic diagram of an image obtained by imaging the imaging range of the test specimen according to Embodiment 1. [Figure 4] (a) is a schematic diagram of a standard test specimen according to Embodiment 1. (b) is a schematic diagram of an image obtained by imaging the imaging range of the standard test specimen according to Embodiment 1. [Figure 5] This is an explanatory diagram of machine learning in Embodiment 1. [Figure 6] (a) is a schematic diagram of a sample according to Embodiment 1. (b) is a schematic diagram of a first image obtained by imaging a first imaging range of the sample according to Embodiment 1. (c) is a schematic diagram of a second image obtained by imaging a second imaging range of the sample according to Embodiment 1. [Figure 7] (a) is an explanatory diagram of an example of training data according to Embodiment 2. (b) is an explanatory diagram of an example of input data and output data which are inference results used for inference according to Embodiment 2. [Figure 8] (a) is a schematic diagram of a micro-test specimen according to Embodiment 2. (b) is a schematic diagram of the imaging range of a micro-test specimen according to Embodiment 2 and the image obtained when the imaging range is imaged. (c) is a schematic diagram of the imaging range of a micro-test specimen according to Embodiment 2 and the image obtained when the imaging range is imaged. [Figure 9] (a) is a schematic diagram of a standard test specimen according to Embodiment 2. (b) is a schematic diagram of an image obtained by imaging the imaging range of the standard test specimen according to Embodiment 2. [Figure 10] This is an explanatory diagram of machine learning in Embodiment 2. [Figure 11] (a) is a schematic diagram of the sample according to Embodiment 2. (b) is a schematic diagram of the first image obtained by imaging the first imaging range of the sample according to Embodiment 2. (c) is a schematic diagram of the third image obtained by imaging the third imaging range of the sample according to Embodiment 2. (d) is a schematic diagram of the second image obtained by imaging the second imaging range of the sample according to Embodiment 2. [Figure 12] (a) is an explanatory diagram of an example of learning data according to Embodiment 3. (b) is an explanatory diagram of an example of input data used for inference according to Embodiment 3 and output data which is an inference result. [Figure 13] (a) is a schematic diagram of a micro test piece according to Embodiment 3. (b) is a schematic diagram of an image obtained by imaging the imaging range of the micro test piece according to Embodiment 3. [Figure 14] (a) is a schematic diagram of a standard test piece according to Embodiment 3. (b) is a schematic diagram of an image obtained by imaging the imaging range of the standard test piece according to Embodiment 3. [Figure 15] It is an explanatory diagram of machine learning in Embodiment 3. [Figure 16] (a) is a schematic diagram of a sample according to Embodiment 3. (b) is a schematic diagram of a first image obtained by imaging the first imaging range of the sample according to Embodiment 3. (c) is a schematic diagram of a second image obtained by imaging the second imaging range of the sample according to Embodiment 3. [Figure 17] (a) is an explanatory diagram of an example of learning data according to Embodiment 4. (b) is an explanatory diagram of an example of input data used for inference according to Embodiment 4 and output data which is an inference result. [Figure 18] It is an explanatory diagram of machine learning in Embodiment 4.

Mode for Carrying Out the Invention

[0011] The embodiments shown below will be described with reference to the accompanying drawings. Note that the embodiments shown below are merely examples, and for example, those skilled in the art can appropriately change the detailed configurations without departing from the spirit of the present disclosure.

[0012] [Embodiment 1] FIG. 1 is an explanatory diagram showing a schematic configuration of a physical property estimation system 1000 according to Embodiment 1. The physical property estimation system 1000 includes a physical property estimation device 100, an imaging device 200, an imaging device 250, a display device 300, an input device 400, and a measurement device 500. The physical property estimation device 100 is configured to have one or more computers. Hereinafter, a case where the physical property estimation device 100 is configured by one computer will be described as an example.

[0013] The physical property estimation device 100 is an example of an information processing device. The imaging device 200, the imaging device 250, the display device 300, the input device 400, and the measurement device 500 are connected to the physical property estimation device 100. The measurement device 550 can be connected to the physical property estimation device 100, but is used in a learning phase of the physical property estimation device 100 described later and is not used in an inference phase.

[0014] The imaging device 200 is an example of a first imaging device. The imaging device 200 can be, for example, a microscope, an optical microscope, a scanning electron microscope (SEM), a transmission electron microscope (TEM), or an X-ray computed tomography (CT). In Embodiment 1, a case where the imaging device 200 is a microscope will be described as an example.

[0015] The imaging device 250 is an example of a second imaging device. The imaging device 250 can be, for example, an optical microscope, an SEM, or an X-ray CT. Hereinafter, a case where the imaging device 250 is an optical microscope will be described as an example. The imaging device 200 and the imaging device 250 are used to image a subject. The imaging device 200 can image the subject at a higher magnification than the imaging device 250.

[0016] The display device 300 is, for example, a display. The input device 400 is, for example, a keyboard and mouse. The measuring device 500 is, for example, a scanning probe microscope (SPM), which can be used to measure electrical resistance in the micro-region as a physical property. The measuring device 550 can be used to measure electrical resistance in the macro-region, which is a wider range than the range measured by the measuring device 500, as a physical property.

[0017] The physical property estimation device 100 may be any type of computer, such as a desktop computer, a tablet computer, or a laptop computer. Furthermore, the physical property estimation device 100 may be a general-purpose computer or a dedicated computer. Also, the physical property estimation device 100 may be a computer in which the display device 300 and input device 400 are integrated into the main unit. Alternatively, the display device 300 and input device 400 may consist of a touch panel display or the like that combines display and input functions.

[0018] The physical property estimation device 100 includes a CPU (Central Processing Unit) 101, which is an example of a processor. The CPU 101 is an information processing unit. The physical property estimation device 100 also includes a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, and an SSD (Solid State Drive) 104 as storage units. The physical property estimation device 100 also includes a recording disk drive 105 and an input / output interface 106. The CPU 101, ROM 102, RAM 103, SSD 104, recording disk drive 105, and input / output interface 106 are connected to each other by a bus to enable data transmission. The imaging device 200, imaging device 250, display device 300, input device 400, and measuring device 500 are connected to the input / output interface 106.

[0019] ROM 102 stores the basic programs related to the operation of the computer. RAM 103 is a memory device that temporarily stores various data, such as the results of calculations performed by CPU 101. SSD 104 records the results of calculations performed by CPU 101 and various data acquired from external sources, as well as a program 161 that causes CPU 101 to execute various processes. Program 161 includes application software that CPU 101 can execute.

[0020] The CPU 101 executes the program 161 recorded on the SSD 104, thereby performing the information processing shown below. The recording disk drive 105 can read various data and programs recorded on the recording disk 162. The recording disk drive 105 can read data recorded on the recording disk 162, which is an example of a recording medium. The CPU 101 also accepts information input from the user via the input device 400.

[0021] In Embodiment 1, the non-temporary computer-readable recording medium is the SSD 104, and the program 161 is recorded on the SSD 104, but this is not the only possible representation. The program 161 may be recorded on any recording medium that is non-temporary computer-readable. Examples of recording media that can supply the program 161 to a computer include flexible disks, hard disks, optical disks, magneto-optical disks, magnetic tapes, and non-volatile memory. Examples of non-volatile memory include USB memory and SD cards. The program 161 may also be obtained from a network (not shown).

[0022] Furthermore, the physical property estimation device 100 having a processor may be configured in ways other than those described above, such as a PLD (Programmable Logic Device) including an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), a general-purpose or dedicated computer with a program installed, or a combination of all or part of these.

[0023] Figure 2(a) is a functional block diagram of the CPU 101 of the physical property estimation device 100 according to Embodiment 1. In Embodiment 1, the CPU 101 functions as a learning unit 1 and an estimation unit 2 by executing program 161. Specifically, the CPU 101 functions as a learning unit 1 in the learning phase and as an estimation unit 2 in the inference phase. The learning unit 1 executes a learning method, and the estimation unit 2 executes a physical property estimation method. Hereafter, the object to be inferred will be referred to as a "sample," and the object used to acquire learning data will be referred to as a "test piece." A sample is, for example, a prototype.

[0024] In Embodiment 1, the functions of the learning unit 1 and the estimation unit 2 are described as being implemented on a single computer, but the invention is not limited to this. The functions of the learning unit 1 and the estimation unit 2 may be implemented on multiple computers. For example, the functions of the learning unit 1, i.e., machine learning, may be implemented on a computer other than the physical property estimation device 100, and the functions of the estimation unit 2 may be implemented on the physical property estimation device 100. In this case, the physical property estimation device 100 can acquire a trained machine learning model from the other computer.

[0025] The learning unit 1 performs supervised learning as a machine learning method during the learning phase. The learning unit 1 performs supervised machine learning using training data T1 which has multiple datasets S1 of input data IN1 and ground truth data A1, and generates a trained machine learning model M1. The machine learning model M1 is stored, for example, in SSD104 in Figure 1. In the inference phase, the estimation unit 2 performs inference on input data IN2 using the trained machine learning model M1 and outputs output data OUT2, which is the inference result. In the following explanation, "estimation" and "inference" are synonymous.

[0026] Figure 2(b) is an explanatory diagram of an example of training data T1 according to Embodiment 1. Each of the multiple datasets S1 includes, as input data IN1, at least one image IM11 of the micro-region of the test specimen, at least one physical property value P11 of the physical properties of the micro-region of the test specimen, and at least one image IM12 of the macro-region of the test specimen. In addition, each of the multiple datasets S1 includes, as ground truth data A1, the physical property value P12 of the physical properties of the macro-region of the test specimen. Here, physical properties refer to properties such as mechanical properties, electrical properties, thermal properties, magnetic properties, and optical properties. Physical property values ​​refer to values ​​that represent these properties. For example, if the electrical property is electrical resistance, the physical property value is the electrical resistance value (Ω).

[0027] Image IM11 is digital data such as an image acquired by the imaging device 200 when it images the test specimen at a first magnification. Image IM12 is digital data such as an image acquired by the imaging device 250 when it images the test specimen at a second magnification, which is lower than the first magnification. The imaging device 200 images the test specimen at a higher magnification than the imaging device 250. Both images IM11 and IM12 are images acquired at a magnification higher than 1x.

[0028] Figure 2(c) is an explanatory diagram of an example of input data IN2 and output data OUT2, which are the inference results, used for inference according to Embodiment 1. The input data IN2 includes at least one image IM1 of the micro-region of the sample, at least one physical property value P1 of the physical properties of the sample in the micro-region, and at least one image IM2 of the macro-region of the sample. The output data OUT2 is the physical property value P2 of the physical properties of the sample in the macro-region.

[0029] Image IM1 is digital data such as an image acquired by the imaging device 200 when it images the sample at a first magnification. Image IM2 is digital data such as an image acquired by the imaging device 250 when it images the sample at a second magnification. The imaging device 200 images the sample at a higher magnification than the imaging device 250. Both images IM1 and IM2 are images acquired at a magnification higher than 1x.

[0030] The number of images IM1 is the same as the number of images IM11 in one dataset S1. The number of physical property values ​​P1 is the same as the number of physical property values ​​P11 in one dataset S1. The number of images IM2 is the same as the number of images IM12 in one dataset S1.

[0031] Figure 3(a) is a schematic diagram of a micro-test piece 51 according to Embodiment 1. Figure 3(b) is a schematic diagram of an image IM11 obtained by imaging the imaging range 21 of the micro-test piece 51 according to Embodiment 1. In Embodiment 1, the test piece used for learning and the sample used for inference are composite members obtained by dispersing a metal filler 6 having a predetermined particle size distribution in a predetermined ratio in a thermosetting resin material 5, and then thermosetting the thermosetting resin material 5. The estimation unit 2 estimates the electrical resistance value of the sample as a physical property value of the sample. The metal filler 6 may be, for example, silver (Ag) particles or copper (Cu) particles of 0.1 to 10 μm. In addition to the electrical resistance value, other physical property values ​​estimated by the physical property estimation device 100 include, for example, the electrical conductivity value, the thermal conductivity value, the thermal diffusivity value, and the elastic modulus value. Furthermore, the test pieces and samples may be made of materials other than resin, such as metal, ceramics, or rubber, and may also be composite materials containing fillers such as carbon, metal, oxide, or nitride.

[0032] The following describes how to obtain the training data T1 used for machine learning. The microscopic test specimen 51 shown in Figure 3(a) is obtained by cutting a sheet-shaped composite material into thin slices using a microtome, and is a test specimen with an area of ​​1 mm × 0.3 mm and a thickness of 1 μm.

[0033] Figure 4(a) is a schematic diagram of the standard test specimen 52 according to Embodiment 1. Figure 4(b) is a schematic diagram of the image IM12 obtained by imaging the imaging range 22 of the standard test specimen 52 according to Embodiment 1. Note that the micro test specimen 51 shown in Figure 3(a) and the standard test specimen 52 shown in Figure 4(a) may be the same test specimen, or they may be substantially identical test specimens made from the same material and using the same manufacturing method. In this example, the standard test specimen 52 is made of substantially the same material as the micro test specimen 51 and is a test specimen with an area of ​​100 mm × 50 mm and a thickness of 1 mm.

[0034] Image IM11 shown in Figure 3(b) is an image taken at a first magnification of 500x using a microscope, which is an example of an imaging device 200, of one of the multiple (e.g., three) imaging ranges 21 of the micro-test specimen 51 shown in Figure 3(a).

[0035] Each imaging area 21 of the micro specimen 51 is, for example, 100 μm × 100 μm. To reduce the influence of variations in the physical properties of the micro specimen 51 from place to place, three imaging areas 21 are imaged by the imaging device 200, thereby obtaining multiple (for example, three) images IM11. In addition to the microscope, the imaging areas 21 may also be imaged using, for example, an optical microscope, a scanning electron microscope (SEM), a transmission electron microscope (TEM), or an X-ray CT scanner.

[0036] The imaging magnification can be arbitrarily set according to the material, and may be a magnification that reveals structures contributing to the expression of physical properties, such as the microstructure, phase, and distribution of compounded fillers of the material, for example, around 500 to 5000 times. In addition, the micro-test piece 51 may be polished, etched, or plasma treated before imaging. Furthermore, a conductive film, which is generally applied to the observation surface when observing insulating materials with SEM, may be thinly deposited on the surface of the micro-test piece 51.

[0037] Image IM12 shown in Figure 4(b) is an image taken using an optical microscope, which is an example of an imaging device 250, at a second magnification of 50x, which is lower than the first magnification, of one of several (e.g., three) imaging ranges 22 of the standard test specimen 52 shown in Figure 4(a).

[0038] The imaging range 22 of the standard test specimen 52 is, for example, 1 mm × 1 mm. To reduce the influence of localized variations in the standard test specimen 52, multiple (e.g., three) imaging ranges 22 are imaged by the imaging device 250, thereby obtaining multiple images IM12. In addition to an optical microscope, the imaging range 22 may also be imaged using, for example, a scanning electron microscope (SEM) or X-ray CT. Furthermore, the imaging conditions and imaging device for image IM12 may be the same as those for image IM11. That is, image IM11 and image IM12 may be acquired using the same imaging device.

[0039] The first magnification can be between 5 and 20 times the second magnification. If the second magnification is too close to the first magnification, less information is obtained, and if the second magnification is too low, the distribution of tissue and the arrangement of fillers cannot be obtained, reducing the accuracy of the estimation of physical properties.

[0040] Next, the physical properties P11 of the micro specimen 51 are measured using a measuring device 500 in the range 11 associated with the image IM11 obtained by imaging the imaging range 21. In Embodiment 1, the electrical properties of the range 11 of the micro specimen 51, such as the electrical resistance of the range 11, are measured using a scanning probe microscope (SPM) as the measuring device 500, and the electrical resistance value is obtained as the physical property value P11 of the range 11. Specifically, the current image distribution of the range 11 of the micro specimen 51 is determined using the SPM, the resistance distribution of the range 11 is determined from the current image distribution of the range 11, and the electrical resistance value of the range 11 is obtained as the physical property value P11 by averaging the resistance distribution of the range 11.

[0041] Range 11 overlaps with imaging range 21. More specifically, part or all of range 11 overlaps with part or all of imaging range 21. In the examples of Figures 3(a) and 3(b), range 11 and imaging range 21 are rectangular areas of the same size, and are slightly offset from each other, with part of range 11 overlapping part of imaging range 21. In Embodiment 1, since there are multiple (e.g., three) imaging ranges 21 for the micro specimen 51, a range 11, which is the measurement range for physical properties, is defined for each of the multiple (e.g., three) imaging ranges 21, and physical properties are measured in each of the multiple (e.g., three) ranges 11, and multiple (e.g., three) physical property values ​​P11 are obtained.

[0042] The range 11 may be the range of magnification from the first magnification to the second magnification, that is, a range greater than or equal to the width of the imaging range 21 and less than or equal to the width of the imaging range 22.

[0043] Alternatively, range 11 may be approximately the same size as imaging range 21, that is, 0.9 times or more and 1.1 times or less of imaging range 21.

[0044] In Embodiment 1, the imaging range 21 of the micro specimen 51 is imaged using a microscope attached to the SPM, and the electrical resistance value is obtained from the current image distribution acquired at the same location. Therefore, range 11 is substantially the same as the imaging range 21.

[0045] The positions of imaging range 21 and range 11 may be determined using an imaging device 200 such as a microscope or microscope attached to the measuring device 500, or they may be determined by forming a mark on the minute test piece 51 in advance, such as by scratching it.

[0046] In Embodiment 1, the physical property value P12, which is the correct data A1 of the training data T1, is acquired by a measuring device 550 different from the measuring device 500. The physical property value P12 is acquired by measuring a range 12 of the standard test piece 52 with the measuring device 550. Range 12 is a wider range than range 11. In this example, the physical property value P12 is the electrical resistance value of the standard test piece 52 in range 12.

[0047] The measuring device 550 includes two electrodes attached to both ends of the standard test piece 52 in the longitudinal direction, a voltage source that applies a predetermined voltage between the two electrodes, and an ammeter that measures the current flowing between the two electrodes to which the predetermined voltage is applied. An electrical resistance value, which is the electrical resistance value in the range 12 of the standard test piece 52, is obtained from the current value measured by the ammeter. In Embodiment 1, the range 12 of the standard test piece 52 is the entire standard test piece 52.

[0048] As in Embodiment 1, when the size of range 12 is significantly larger than the size of range 11, it is often difficult to measure the physical properties with the same measuring device 500, and the same physical properties (e.g., electrical resistance) are measured in different ways using different measuring devices 500 and 550 in ranges 11 and 12.

[0049] In the learning phase, images IM11, IM12 and physical properties P11, P12 are obtained from multiple micro specimens 51 and multiple standard specimens 52, each with different material preparation processes such as material composition, filler content, mixing conditions, and heat treatment conditions. Multiple datasets S1 are then created, each containing image IM11, physical property P11, and image IM12 as input data IN1, and physical property P12 as ground truth data A1. The learning data T1 is then prepared using these multiple datasets S1.

[0050] In a single dataset S1, the image IM11, physical property values ​​P11, and image IM12 included in the input data IN1 may each be one or multiple (e.g., three). As mentioned above, in order to reduce variability within the material, multiple (e.g., three) images IM11 and multiple (e.g., three) physical property values ​​P11 are acquired for one micro specimen 51, and multiple (e.g., three) images IM12 are acquired for one standard specimen 52. These multiple images IM11, multiple physical property values ​​P11, and multiple images IM12 are used to create the input data IN1 in a single dataset S1. This input data IN1 is then subjected to supervised learning using physical property values ​​P12 measured within range 12 as the ground truth data A1 to obtain a machine learning model M1 that estimates physical property values.

[0051] The learning method of learning unit 1 will be explained with a specific example. For machine learning in learning unit 1, a regression model using a convolutional neural network (CNN) is used, for example. Figure 5 is an explanatory diagram of machine learning in embodiment 1. Figure 5 shows the input layer and output layer of the regression model using a CNN. The input layer includes three images IM11, three physical property values ​​P11, and three images IM12. By having learning unit 1 learn through multiple hidden layers so that one physical property value P12 is output, a machine learning model M1, which is an estimation model, is created. Note that the machine learning method is not limited to CNN. Also, in order to improve the accuracy of estimation of physical properties, the number of datasets S1 can be, for example, 50 or more, or for example, 100 or more.

[0052] By inputting this information into the learning unit 1 and performing machine learning on the learning unit 1, a machine learning model M1 is created that can accurately estimate physical properties in the macro region, even when the distribution of physical properties in the micro region of a material contributes to the physical properties in the macro region, which has a wider range than the micro region.

[0053] Although the explanation described the case where the physical properties measured in range 11 and range 12 are electrical properties, the explanation is not limited to this, and the physical properties measured in range 11 and range 12 may be, for example, mechanical properties or thermal properties. If the physical properties to be measured are mechanical properties, for example, the measuring device 500 may be a nanoindenter or a micro-Vickers tester, and may measure hardness or elastic modulus. If the physical properties to be measured are thermal properties, for example, the measuring device 500 may measure thermal diffusivity using a laser flash method in a micro-region. If the physical properties to be measured are electrical properties, for example, the measuring device 500 may measure dielectric constant.

[0054] The created machine learning model M1 is stored in a memory unit such as SSD104 and used for inference processing in estimation unit 2 during the inference phase.

[0055] Next, the inference phase will be explained. Figure 6(a) is a schematic diagram of the sample 61 according to Embodiment 1. Figure 6(b) is a schematic diagram of the image IM1 obtained by imaging the imaging range 41 of the sample 61 according to Embodiment 1. Figure 6(c) is a schematic diagram of the image IM2 obtained by imaging the imaging range 42 of the sample 61 according to Embodiment 1.

[0056] Image IM1 shown in Figure 6(b) is an image obtained by capturing one of the multiple (e.g., three) imaging ranges 41 of the sample 61 shown in Figure 6(a) at a first magnification of 500x using a microscope, which is an example of the imaging device 200.

[0057] The imaging range 41 is an example of the first imaging range. Image IM1 is an example of the first image. Each imaging range 41 of the sample 61 is, for example, 100 μm × 100 μm. To reduce the effect of locational variations in the sample 61, multiple (for example, three) images IM1 are obtained by imaging three imaging ranges 41 with the imaging device 200. In addition to the microscope, the imaging range 41 may also be imaged using, for example, an optical microscope, SEM, TEM, or X-ray CT.

[0058] The imaging magnification can be arbitrarily set according to the material, and may be a magnification that shows structures contributing to the expression of physical properties, such as the microstructure, phase, and distribution of compounded fillers of the material, for example, around 500 to 5000 times. In addition, the sample 61 may be polished, etched, or plasma treated before imaging. Furthermore, a conductive film, which is generally applied to the observation surface when observing insulating materials with SEM, may be thinly deposited on the surface of the sample 61. In this way, the estimation unit 2 receives input of an image IM1 in which the imaging range 41 of the sample 61 is captured at the first magnification.

[0059] Image IM2, shown in Figure 6(c), is an image taken using an optical microscope, which is an example of the imaging device 250, at a second magnification of 50x, which is lower than the first magnification, of one of the multiple (e.g., three) imaging ranges 42 of the sample 61 shown in Figure 6(a).

[0060] Image range 42 is an example of a second imaging range. Image IM2 is an example of a second image. The imaging range 42 of sample 61 is, for example, 1 mm × 1 mm. To reduce the effect of localized variations in sample 61, multiple (e.g., three) imaging ranges 42 are imaged by the imaging device 250, thereby obtaining multiple images IM2. In addition to an optical microscope, the imaging range 42 may also be imaged using, for example, a scanning electron microscope (SEM) or X-ray CT. Furthermore, image IM2 may be obtained using the same imaging conditions and imaging device as image IM1. That is, image IM1 and image IM2 may be acquired using the same imaging device. Also, when acquiring image IM2, the imaging range 41 of image IM1 may be superimposed on the imaging range 42 so that the user can view it on the display device 300. That is, the CPU 101 may superimpose the imaging range 41 of image IM1 onto image IM2 and display it on the display device 300. Thus, the estimation unit 2 receives an input image IM2 in which an imaging range 42 wider than the imaging range 41 of the sample 61 is captured at a second magnification lower than the first magnification.

[0061] Even in the inference phase, the first magnification can be between 5 and 20 times the second magnification. If the second magnification is too close to the first, less information is obtained, and if the second magnification is too low, the distribution of tissue and the arrangement of fillers cannot be obtained, reducing the accuracy of the estimation of physical properties.

[0062] Next, the physical property value P1 of the sample 61 is measured using the measuring device 500 in range 31 associated with the image IM1 obtained by imaging range 41. Range 31 is an example of a first range. Physical property value P1 is an example of a first physical property value. In Embodiment 1, as a physical property of the sample 61, the electrical properties of range 31 of the sample 61, for example, the electrical resistance of range 31, are measured using SPM as the measuring device 500, and the electrical resistance value is obtained as the physical property value P1 of range 31. Specifically, the current image distribution of range 31 of the sample 61 is determined using SPM, the resistance distribution of range 31 is determined from the current image distribution of range 31, and the electrical resistance value of range 31 is obtained as the physical property value P1 by averaging the resistance distribution of range 31. In this way, the estimation unit 2 receives input of the physical property value P1 of the physical properties of the sample 61 in range 31.

[0063] Range 31 overlaps with imaging range 41. More specifically, part or all of range 31 overlaps with part or all of imaging range 41. In the examples of Figures 6(a) and 6(b), range 31 and imaging range 41 are rectangular areas of the same size, and are slightly offset from each other, with part of range 31 overlapping part of imaging range 41. In Embodiment 1, since there are multiple (e.g., three) imaging ranges 41 for the sample 61, a range 31, which is the measurement range for physical properties, is defined for each of the multiple (e.g., three) imaging ranges 41, and physical properties are measured in each of the multiple (e.g., three) ranges 31, and multiple (e.g., three) physical property values ​​P1 are obtained. Furthermore, when determining range 31, the imaging range 41 may be made visible to the user on the display device 300.

[0064] Then, the estimation unit 2 estimates the physical property value P2 of the sample 61 in a range 32 that is wider than range 31, using at least image IM1, physical property value P1, and image IM2. Range 32 is an example of a second range. Physical property value P2 is an example of a second physical property value.

[0065] The range 31 may be the range of magnification from the first magnification to the second magnification, that is, a range greater than or equal to the width of the imaging range 41 and less than or equal to the width of the imaging range 42.

[0066] Alternatively, range 31 may be approximately the same size as imaging range 41, that is, 0.9 times or more and 1.1 times or less of imaging range 41.

[0067] In Embodiment 1, the imaging range 41 of the sample 61 is imaged using a microscope attached to the SPM, and the electrical resistance value is obtained from the current image distribution acquired at the same location. Therefore, range 31 is substantially the same as the imaging range 41.

[0068] The positions of imaging range 41 and range 31 may be determined using an imaging device 200 such as a microscope or microscope attached to the measuring device 500, as in Embodiment 1, or they may be determined by forming a mark on the sample 61 beforehand, such as by scratching.

[0069] In Embodiment 1, the physical property value P2 in range 32 is estimated by the inference process of the estimation unit 2. That is, the estimation unit 2 uses a pre-trained machine learning model M1 to estimate the physical property value P2, with image IM1, physical property value P1, and image IM2 as input data IN2, and the physical property value P2 as output data OUT2. In this example, the physical property value P2 is the electrical resistance value of the sample 61 in range 32.

[0070] In Embodiment 1, it is not necessary to prepare a measuring device 550. The inference processing of the estimation unit 2 obtains the electrical resistance value, which is the value of the electrical resistance in the range 32 of the sample 61. In Embodiment 1, the range 32 of the sample 61 is the entire sample 61.

[0071] As in Embodiment 1, when the size of range 32 is significantly larger than the size of range 31, it is often difficult to measure the physical properties with the measuring device 500, and the electrical resistance value, which is the physical property value P2 of the physical properties in range 32, is estimated by the estimation unit 2.

[0072] In the inference phase, the images IM1 and IM2 obtained in this way, along with the physical property value P1, are included in the input data IN2, and the physical property value P2 is output as output data OUT2.

[0073] The input data IN2 may contain only one image IM1, one physical property value P1, and one image IM2, or multiple images (e.g., three) of each. As mentioned above, to reduce variability within the material, multiple images (e.g., three) of image IM1, multiple physical property values ​​(e.g., three) of image IM2, and multiple images (e.g., three) of image IM2 are acquired for one sample 61, and the input data IN2 is created from these multiple images IM1, multiple physical property values ​​P1, and multiple images IM2. From this input data IN2, physical property values ​​P2 within a range of 32 are obtained from the trained machine learning model M1.

[0074] Although the explanation has described the case where the physical properties in ranges 31 and 32 are electrical properties, the explanation is not limited to this, and the physical properties in ranges 31 and 32 may be, for example, mechanical properties or thermal properties. If the physical property measured in range 31 is a mechanical property, for example, the measuring device 500 may be a nanoindenter or a micro-Vickers tester, and may measure hardness or elastic modulus. If the physical property measured in range 31 is a thermal property, for example, the measuring device 500 may measure thermal diffusivity using a laser flash method in a micro-region. If the physical property measured in range 31 is an electrical property, for example, the measuring device 500 may measure dielectric constant.

[0075] Thus, in the inference phase, the estimation unit 2 acquires input data IN2, which includes image IM1, physical property values ​​P1 measured in range 31, and image IM2 of the sample 61 whose physical properties are to be estimated, in the same manner as in the learning phase, and stores it in the SSD 104. Then, the estimation unit 2 uses the input data IN2 stored in the SSD 104 to infer the physical property values ​​P2, which are output data OUT2. There may be multiple images IM1, physical property values ​​P1, and images IM2, but the number of images IM1, physical property values ​​P1, and images IM2 must match the number of images IM11, physical property values ​​P11, and images IM12 that were input when the machine learning model M1, which is the estimation model, was trained. That is, the number of images IM1, the number of physical property values ​​P1, and the number of images IM2 must match the number of images IM11, physical property values ​​P11, and images IM12 included in one dataset S1 during the learning phase. In this way, the estimation unit 2 can estimate the physical properties of the range 32 of the sample 61 from the trained machine learning model M1 that was created in advance during the learning phase, based on the input data IN2 that was received.

[0076] In Embodiment 1, the estimation unit 2 receives input data IN2 consisting of three images IM1, which are examples of at least one first image; three physical property values ​​P1, which are examples of at least one first physical property value; and three images IM2, which are examples of at least one second image. Based on the received images IM1, physical property values ​​P1, and images IM2, the estimation unit 2 estimates the physical property value P2.

[0077] Thus, according to Embodiment 1, in the inference phase, the estimation unit 2 estimates the physical property value P2, eliminating the need to prepare the measuring device 550. In other words, the effort of setting up the measuring device 550 to evaluate the prototyped sample 61 each time a prototype sample 61 is fabricated is eliminated, thus improving the efficiency of evaluating the sample 61. Therefore, the estimation accuracy of the physical property value P2 in the macroscopic region of the sample is improved. Furthermore, even if it is possible to measure the physical property value P2 with the measuring device 500, the step of measuring the physical property value P2 with the measuring device 500 can be omitted, thus improving the efficiency of evaluating the sample 61.

[0078] [Embodiment 2] Embodiment 2 will now be described. Hereinafter, elements denoted by the same reference numerals as those in Embodiment 1 will have substantially the same configuration and function as those described in Embodiment 1 unless otherwise specified. The differences from Embodiment 1 will be the main focus of this description.

[0079] Since the hardware configuration of the physical property estimation system in Embodiment 2 is substantially the same as the hardware configuration of the physical property estimation system 1000 in Embodiment 1 shown in Figure 1, a description of the hardware configuration of the physical property estimation system in Embodiment 2 will be omitted. In Embodiment 2, in the learning phase and the inference phase, an image captured at a third magnification between the first and second magnification, and estimated values ​​of physical properties in the range corresponding to the image are used as input data.

[0080] In Embodiment 2, the CPU 101 shown in Figure 1 functions as the learning unit 1 and estimation unit 2 shown in Figure 2(a) by executing the program 161. Specifically, the CPU 101 functions as the learning unit 1 in the learning phase and as the estimation unit 2 in the inference phase. The learning unit 1 executes the learning method, and the estimation unit 2 executes the physical property estimation method. The object to be inferred is referred to as a "sample," and the object used to acquire learning data is referred to as a "test piece." The sample is, for example, a prototype.

[0081] Figure 7(a) is an explanatory diagram of an example of training data T1 according to Embodiment 2. Figure 7(b) is an explanatory diagram of an example of input data IN2 and output data OUT2, which are the inference results, used for inference according to Embodiment 2.

[0082] The learning unit 1 performs supervised learning as a machine learning method during the learning phase. The learning unit 1 performs supervised machine learning using training data T1 which has multiple datasets S1 of input data IN1 and ground truth data A1, and generates a trained machine learning model M1 as shown in Figure 2(a). The machine learning model M1 is stored, for example, in SSD104 in Figure 1. In the inference phase, the estimation unit 2 performs inference on input data IN2 using the trained machine learning model M1 and outputs output data OUT2 which is the inference result.

[0083] Each of the multiple datasets S1 includes, as input data IN1, at least one image IM11 of the micro-region of the specimen, at least one physical property value P11 of the physical properties of the micro-region of the specimen, at least one image IM13 of the intermediate region between the area of ​​the micro-region and the area of ​​the macro-region of the specimen, at least one physical property value P13 of the physical properties of the intermediate region of the specimen, and at least one image IM12 of the macro-region of the specimen. In addition, each of the multiple datasets S1 includes, as ground truth data A1, the physical property value P12 of the physical properties of the macro-region of the specimen.

[0084] Image IM11 is digital data such as an image acquired by the imaging device 200 when it images the test specimen at a first magnification. Image IM13 is digital data such as an image acquired by the imaging device 200 when it images the test specimen at a third magnification, which is lower than the first magnification and higher than the second magnification. The third magnification is a magnification between the first and second magnifications. Image IM12 is digital data such as an image acquired by the imaging device 250 when it images the test specimen at a second magnification, which is lower than both the first and third magnifications. The imaging device 200 images the test specimen at a higher magnification than the imaging device 250. All of images IM11, IM12, and IM13 are images acquired at a magnification higher than 1x.

[0085] The input data IN2 includes at least one image IM1 of the micro-region of the sample, at least one physical property value P1 of the physical properties of the sample in the micro-region, at least one image IM3 of the intermediate region of the sample, at least one physical property value P3 of the physical properties of the sample in the intermediate region, and at least one image IM2 of the macro-region of the sample. The output data OUT2 is the physical property value P2 of the physical properties of the sample in the macro-region.

[0086] Image IM1 is digital data such as an image acquired by the imaging device 200 when it images the sample at the first magnification. Image IM3 is digital data such as an image acquired by the imaging device 200 when it images the sample at the third magnification. Image IM2 is digital data such as an image acquired by the imaging device 250 when it images the sample at the second magnification. The imaging device 200 images the sample at a higher magnification than the imaging device 250. All of the images IM1, IM2, and IM3 are images acquired at a magnification higher than 1x.

[0087] The number of images IM1 is the same as the number of images IM11 in one dataset S1. The number of physical property values ​​P1 is the same as the number of physical property values ​​P11 in one dataset S1. The number of images IM3 is the same as the number of images IM13 in one dataset S1. The number of physical property values ​​P3 is the same as the number of physical property values ​​P13 in one dataset S1. The number of images IM2 is the same as the number of images IM12 in one dataset S1.

[0088] Figure 8(a) is a schematic diagram of a micro-test specimen 51 according to Embodiment 2. Figure 8(b) is a schematic diagram of the imaging range 21 of the micro-test specimen 51 according to Embodiment 2 and the image IM11 obtained when the imaging range 21 is imaged. Figure 8(c) is a schematic diagram of the imaging range 23 of the micro-test specimen 51 according to Embodiment 2 and the image IM13 obtained when the imaging range 23 is imaged.

[0089] In Embodiment 2, the test piece used for learning and the sample used for inference are composite materials in which a magnesium oxide (MgO) filler 60 having a predetermined particle size distribution is dispersed in a predetermined ratio within a resin material 50. The estimation unit 2 then estimates the value of the thermal diffusivity of the sample as a physical property value of the sample.

[0090] The method for preparing test specimens and samples will now be described. After mixing MgO filler with a liquid resin material in a predetermined ratio and stirring, the liquid mixture is spread in a sheet-like manner onto one of a pair of electrodes, and the other electrode is placed on top of the liquid mixture, sandwiching the liquid mixture between the pair of electrodes. An electric field is then applied in the thickness direction to promote the orientation of the filler. Subsequently, the liquid mixture is cured by heating, and the cured composite material is peeled off the electrodes to obtain a sheet-like composite material.

[0091] The following describes how to obtain the training data T1 used for machine learning. The micro-test specimen 51 shown in Figure 8(a) is obtained by cutting a sheet-shaped composite material into thin slices using a microtome, and is a test specimen with an area of ​​1 mm × 1 mm and a thickness of 1 μm.

[0092] Figure 9(a) is a schematic diagram of the standard test specimen 52 according to Embodiment 2. Figure 9(b) is a schematic diagram of the image IM12 obtained when the imaging range 22 of the standard test specimen 52 according to Embodiment 2 is imaged. The micro test specimen 51 shown in Figure 8(a) and the standard test specimen 52 shown in Figure 9(a) may be the same test specimen, or they may be substantially identical test specimens made from the same material and using the same manufacturing method. In this example, the standard test specimen 52 is made of substantially the same material as the micro test specimen 51 and is a test specimen with an area of ​​30 mm × 15 mm and a thickness of 1 mm.

[0093] Image IM11 shown in Figure 8(b) is an image obtained by capturing one of several (e.g., three) imaging areas 21 of the micro specimen 51 shown in Figure 8(a) using a scanning electron microscope (SEM), which is an example of an imaging device 200, at a first magnification of 500x. Each imaging area 21 of the micro specimen 51 is, for example, 100 μm × 100 μm. To reduce the effect of location-specific variations in the micro specimen 51, three imaging areas 21 are captured by the imaging device 200, thereby obtaining multiple (e.g., three) images IM11.

[0094] Image IM13 shown in Figure 8(c) is an image obtained by capturing one of several (e.g., two) imaging areas 23 of the micro specimen 51 shown in Figure 8(a) at a third magnification of 100x using a scanning electron microscope (SEM), which is an example of an imaging device 200. Each imaging area 23 of the micro specimen 51 is, for example, 500 μm × 500 μm. To reduce the effect of localized variability in the micro specimen 51, multiple (e.g., two) images IM13 are obtained by imaging two imaging areas 23 with the imaging device 200. The imaging area 23 of image IM13 may or may not include the imaging area 21 of image IM11. In addition to the SEM, the imaging area 21 or imaging area 23 may also be imaged using, for example, an optical microscope, microscope, TEM, or X-ray CT. Furthermore, although it has been explained that image IM11 and image IM13 are obtained using the same imaging device 200, this is not limited to this, and image IM11 and image IM13 may be obtained using different imaging devices.

[0095] Image IM12 shown in Figure 9(b) is an image taken using an optical microscope, which is an example of an imaging device 250, at a second magnification of 25x, which is lower than the first and third magnifications, of one of several (e.g., three) imaging ranges 22 of the standard test specimen 52 shown in Figure 9(a).

[0096] The imaging range 22 of the standard test specimen 52 is, for example, 2 mm × 2 mm. To reduce the influence of localized variations in the standard test specimen 52, multiple (e.g., three) imaging ranges 22 are imaged by the imaging device 250, thereby obtaining multiple images IM12. In addition to an optical microscope, the imaging range 22 may also be imaged using, for example, a scanning electron microscope (SEM) or X-ray CT. Furthermore, the imaging conditions and imaging device for image IM12 may be the same as those for images IM11 and IM13. That is, image IM11 and images IM12 and IM13 may be acquired using the same imaging device.

[0097] The first magnification can be between 5 and 20 times the second magnification. If the second magnification is too close to the first magnification, less information is obtained, and if the second magnification is too low, the distribution of tissue and the arrangement of fillers cannot be obtained, reducing the accuracy of the estimation of physical properties.

[0098] Next, the physical property value P11 is measured using the measuring device 500 in area 11 of the micro specimen 51 associated with the image IM11 obtained by imaging area 21, and the physical property value P13 is measured using the measuring device 500 in area 13 of the micro specimen 51 associated with the image IM13 obtained by imaging area 23.

[0099] In Embodiment 2, the thermal properties of the micro specimen 51, specifically the thermal diffusivity of the ranges 11 and 13, are measured using a measuring device 500, and the thermal diffusivity values ​​are obtained as the physical property values ​​P11 and P13 for the ranges 11 and 13.

[0100] Specifically, for the physical properties in range 11 and range 13, the periodic laser heating method (Kazuya Okamoto, Yamaguchi Tokyo University of Science Bulletin, 2018, (1), 61-65), a known technique for measuring thermal diffusivity in thin films, can be used. The ranges 11 and 13 for measuring thermal diffusivity are, for example, φ100 μm and φ500 μm, respectively.

[0101] Range 11 overlaps with imaging range 21. More specifically, part or all of range 11 overlaps with part or all of imaging range 21. Range 13 overlaps with imaging range 23. More specifically, part or all of range 13 overlaps with part or all of imaging range 23.

[0102] In the example in Figure 8(b), the imaging range 21 is rectangular, and range 11 is circular, with range 11 located inside imaging range 21. Similarly, in the example in Figure 8(c), the imaging range 23 is rectangular, and range 13 is circular, with range 13 located inside imaging range 23.

[0103] In Embodiment 2, since there are multiple (e.g., three) imaging ranges 21 for the micro specimen 51, a range 11 which is the measurement range for physical properties is defined for each of the multiple (e.g., three) imaging ranges 21, and physical properties are measured in each of the multiple (e.g., three) ranges 11, and multiple (e.g., three) physical property values ​​P11 are obtained. Similarly, since there are multiple (e.g., two) imaging ranges 23 for the micro specimen 51, a range 13 which is the measurement range for physical properties is defined for each of the multiple (e.g., two) imaging ranges 23, and physical properties are measured in each of the multiple (e.g., two) ranges 13, and multiple (e.g., two) physical property values ​​P13 are obtained.

[0104] The range 11 may be the range of magnification from the first magnification to the second magnification, that is, a range greater than or equal to the width of the imaging range 21 and less than or equal to the width of the imaging range 22.

[0105] Alternatively, range 11 may be approximately the same size as imaging range 21, that is, 0.9 times or more and 1.1 times or less of imaging range 21.

[0106] In Embodiment 2, the physical property value P12, which is the ground truth data A1 of the training data T1, is acquired by a measuring device 550 different from the measuring device 500. The physical property value P12 is acquired by measuring a range 12 of the standard test piece 52 with the measuring device 550. Range 12 is wider than both range 11 and range 13. In this example, the physical property value P12 is the value of the thermal diffusivity in range 12 of the standard test piece 52.

[0107] The thermal diffusivity value, which is a physical property of the range 12 of the standard test specimen 52, is calculated using the temperature gradient method from the thermal conductivity obtained from the temperature gradient between one end and the other end in the direction of the long side (30 mm) of the standard test specimen 52, and the specific heat measured separately. In Embodiment 2, the range 12 of the standard test specimen 52 is the entire standard test specimen 52.

[0108] As in Embodiment 2, when the size of range 12 is significantly larger than the size of range 11, it is often difficult to measure the physical properties with the same measuring device 500, and the same physical properties (e.g., thermal diffusivity) are measured in different ways using different measuring devices 500 and 550 in ranges 11 and 12.

[0109] In the learning phase, images IM11, IM12, IM13 and physical properties P11, P12, P13 are obtained from multiple micro specimens 51 and multiple standard specimens 52 with different material preparation processes, including filler content, mixing conditions, orientation voltage, and heat treatment conditions. Multiple datasets S1 are created, each containing image IM11, physical property P11, image IM13, physical property P13, and image IM12 as input data IN1, and physical property P12 as ground truth data A1. The learning data T1 is then prepared using these multiple datasets S1.

[0110] In a single dataset S1, the image IM11, physical property value P11, image IM13, physical property value P13, and image IM12 included in the input data IN1 may each be one or multiple. As mentioned above, in order to reduce variability within the material, multiple (e.g., 3) images IM11, multiple (e.g., 3) physical property value P11, multiple (e.g., 2) images IM13, and multiple (e.g., 2) physical property value P13 are obtained for one micro specimen 51, and multiple (e.g., 3) images IM12 are obtained for one standard specimen 52. The input data IN1 in a single dataset S1 is created from these multiple images IM11, multiple physical property value P11, multiple images IM13, multiple physical property value P13, and multiple images IM12. A machine learning model M1 that estimates physical properties is obtained by performing supervised learning on this input data IN1, using physical property values ​​P12 measured within range 12 as the ground truth data A1.

[0111] The learning method of learning unit 1 will be explained with a specific example. Similar to Embodiment 1, a regression model using a convolutional neural network (CNN) is used for machine learning in learning unit 1. Figure 10 is an explanatory diagram of machine learning in Embodiment 2. Figure 10 shows the input layer and output layer of a regression model using a CNN. The input layer includes three images IM11, three physical property values ​​P11, two images IM13, two physical property values ​​P13, and three images IM12. By having learning unit 1 learn through multiple hidden layers so that one physical property value P12 is output, a machine learning model M1, which is an estimation model, is created. Note that the machine learning method is not limited to CNN. Also, in order to improve the accuracy of estimation of physical properties, the number of datasets S1 can be, for example, 50 or more, or for example, 100 or more.

[0112] In this way, by gradually changing the magnification, the relationship between the structure and physical properties of the micro-test specimen 51 is input in more detail at each magnification, making it possible to estimate the physical properties of the sample with high accuracy. For example, in the case of a material containing fillers of different sizes, the arrangement information of small and large fillers may not be sufficient in a single image. In such cases, by inputting images with gradually changed magnification according to the filler size, and the physical property values ​​measured within the range associated with those images, information on the arrangement of fillers of each size and the physical properties obtained from that arrangement is added, and a machine learning model M1 capable of estimating physical properties with even higher accuracy is created.

[0113] The created machine learning model M1 is stored in a memory unit such as SSD104 and used for inference processing in estimation unit 2 during the inference phase.

[0114] Next, the inference phase will be explained. Figure 11(a) is a schematic diagram of the sample 61 according to Embodiment 2. Figure 11(b) is a schematic diagram of image IM1 obtained by imaging the imaging range 41 of the sample 61 according to Embodiment 2. Figure 11(c) is a schematic diagram of image IM3 obtained by imaging the imaging range 43 of the sample 61 according to Embodiment 2. Figure 11(d) is a schematic diagram of image IM2 obtained by imaging the imaging range 42 of the sample 61 according to Embodiment 2.

[0115] Image IM1 shown in Figure 11(b) is an image obtained by capturing one of the multiple (e.g., three) imaging ranges 41 of the sample 61 shown in Figure 11(a) using an SEM, which is an example of an imaging device 200, at a first magnification of 500x.

[0116] The imaging range 41 is an example of the first imaging range. Image IM1 is an example of the first image. Each imaging range 41 of the sample 61 is, for example, 100 μm × 100 μm. To reduce the effect of localized variability in the sample 61, three imaging ranges 41 are imaged by the imaging device 200, thereby obtaining multiple (for example, three) images IM1. In addition to SEM, the imaging range 41 may also be imaged using, for example, an optical microscope, a microscope, TEM, or X-ray CT.

[0117] The imaging magnification can be arbitrarily set according to the material, and may be a magnification that reveals structures that contribute to the expression of physical properties, such as the microstructure, phase, and distribution of compounded fillers of the material, for example, around 500 to 5000 times. Thus, the estimation unit 2 receives input of an image IM1 in which the imaging range 41 of the sample 61 is captured at the first magnification.

[0118] Image IM3, shown in Figure 11(c), is an image obtained by capturing one of the multiple (e.g., two) imaging ranges 43 of the sample 61 shown in Figure 11(a) at a third magnification of 100x using an SEM, which is an example of an imaging device 200.

[0119] The imaging range 43 of sample 61 is an example of a third imaging range. Image IM3 is an example of a third image. Each imaging range 43 of sample 61 is, for example, 500 μm × 500 μm. To reduce the effect of locational variations in sample 61, multiple (for example, two) images IM3 are obtained by imaging two imaging ranges 43 with the imaging device 200. The imaging range 43 of image IM3 may or may not include the imaging range 41 of image IM1. In addition to SEM, imaging ranges 41 or 43 may be imaged using, for example, an optical microscope, microscope, TEM, or X-ray CT. Furthermore, although images IM1 and IM3 have been described as being obtained with the same imaging device 200, this is not limited to this, and images IM1 and IM3 may be obtained with different imaging devices. Thus, the estimation unit 2 receives input of an image IM3 captured at a third magnification between the first and second magnifications, within an imaging range 43 that is wider than the imaging range 41 of the sample 61 and narrower than the imaging range 42.

[0120] Image IM2, shown in Figure 11(d), is an image taken using an optical microscope, which is an example of the imaging device 250, at a second magnification of 25x, which is lower than the first magnification, of one of the multiple (e.g., three) imaging ranges 42 of the sample 61 shown in Figure 11(a).

[0121] Image range 42 is an example of a second imaging range. Image IM2 is an example of a second image. The imaging range 42 of sample 61 is, for example, 2 mm × 2 mm. To reduce the effect of localized variations in sample 61, multiple (e.g., three) imaging ranges 42 are imaged by the imaging device 250, thereby obtaining multiple images IM2. In addition to an optical microscope, the imaging range 42 may also be imaged using, for example, a scanning electron microscope (SEM) or X-ray CT. Furthermore, image IM2 may be obtained using the same imaging conditions and imaging device as images IM1 and IM3. That is, images IM1 and images IM2 and IM3 may be acquired using the same imaging device. In this way, the estimation unit 2 receives input of image IM2 in which an imaging range 42 wider than the imaging range 41 of sample 61 is imaged at a second magnification lower than the first and third magnifications.

[0122] Even in the inference phase, the first magnification can be between 5 and 20 times the second magnification. If the second magnification is too close to the first, less information is obtained, and if the second magnification is too low, the distribution of tissue and the arrangement of fillers cannot be obtained, reducing the accuracy of the estimation of physical properties.

[0123] Next, the physical property value P1 is measured using the measuring device 500 in range 31 of the sample 61 associated with image IM1 obtained by imaging range 41, and the physical property value P3 is measured using the measuring device 500 in range 33 of the sample 61 associated with image IM3 obtained by imaging range 43. Range 31 is an example of the first range. Physical property value P1 is an example of the first physical property value. Range 33 is an example of the third range. Physical property value P3 is an example of the third physical property value. Range 33 is wider than range 31 and narrower than range 32.

[0124] In Embodiment 2, the thermal properties of the sample 61, specifically the thermal diffusivity in ranges 31 and 33, are measured using the measuring device 500, and the values ​​of thermal diffusivity are obtained as physical property values ​​P1 and P3 for ranges 31 and 33. The method for measuring thermal diffusivity is the same as the method for obtaining physical property values ​​P11 and P13. The ranges 31 and 33 for measuring thermal diffusivity are, for example, φ100 μm and φ500 μm, respectively. Thus, the estimation unit 2 receives input of the physical property value P1 for the physical properties of the sample 61 in range 31 and the physical property value P3 for the physical properties of the sample 61 in range 33.

[0125] Range 31 overlaps with imaging range 41. More specifically, part or all of range 31 overlaps with part or all of imaging range 41. Range 33 overlaps with imaging range 43. More specifically, part or all of range 33 overlaps with part or all of imaging range 43.

[0126] In the example in Figure 11(b), the imaging range 41 is rectangular, and range 31 is circular, with range 31 located inside the imaging range 41. That is, range 31 is included in the imaging range 41. Similarly, in the example in Figure 11(c), the imaging range 43 is rectangular, and range 33 is circular, with range 33 located inside the imaging range 43. That is, range 33 is included in the imaging range 43.

[0127] In Embodiment 2, since there are multiple (e.g., three) imaging ranges 41 for the sample 61, a range 31 which is the measurement range for physical properties is defined for each of the multiple (e.g., three) imaging ranges 41, and physical properties are measured in each of the multiple (e.g., three) ranges 31, and multiple (e.g., three) physical property values ​​P1 are obtained. Similarly, since there are multiple (e.g., two) imaging ranges 43 for the sample 61, a range 33 which is the measurement range for physical properties is defined for each of the multiple (e.g., two) imaging ranges 43, and physical properties are measured in each of the multiple (e.g., two) ranges 33, and multiple (e.g., two) physical property values ​​P3 are obtained.

[0128] Then, the estimation unit 2 uses image IM1, physical property value P1, image IM3, physical property value P3, and image IM2 to estimate the physical property value P2 of the sample 61 in a range 32 that is wider than the ranges 31 and 33. Range 32 is an example of a second range. Physical property value P2 is an example of a second physical property value.

[0129] The range 31 may be the range of magnification from the first magnification to the second magnification, that is, a range greater than or equal to the width of the imaging range 41 and less than or equal to the width of the imaging range 42.

[0130] Alternatively, range 31 may be approximately the same size as imaging range 41, that is, 0.9 times or more and 1.1 times or less of imaging range 41.

[0131] In Embodiment 2, the physical property value P2 in range 32 is estimated by the inference process of the estimation unit 2. Specifically, the estimation unit 2 uses a pre-trained machine learning model M1 to estimate the physical property value P2, with image IM1, physical property value P1, image IM3, physical property value P3, and image IM2 as input data IN2, and the physical property value P2 as output data OUT2. In this example, the physical property value P2 is the value of the thermal diffusivity of the sample 61 in range 32.

[0132] In Embodiment 2, it is not necessary to prepare the measuring device 550. The inference processing of the estimation unit 2 obtains the value of the thermal diffusivity in the range 32 of the sample 61. In Embodiment 2, the range 32 of the sample 61 is the entire sample 61.

[0133] As in Embodiment 2, when the size of range 32 is significantly larger than the size of range 31, it is often difficult to measure the physical properties with the measuring device 500, and the value of the thermal diffusivity, which is the physical property value P2 of the physical properties in range 32, is estimated by the estimation unit 2.

[0134] In the inference phase, the images IM1, IM2, and IM3 obtained in this way, along with the physical properties P1 and P3, are included in the input data IN2, and the physical property P2 is output as the output data OUT2.

[0135] The input data IN2 may contain only one image IM1, physical property P1, image IM3, physical property P3, and image IM2, or multiple images of each. As mentioned above, in order to reduce variations within the material, multiple images IM1, multiple physical property P1, multiple images IM3, multiple physical property P3, and multiple images IM2 are acquired for one sample 61, and the input data IN2 is created from these multiple images IM1, multiple physical property P1, multiple images IM3, multiple physical property P3, and multiple images IM2. From this input data IN2, physical property P2 within a range of 32 is obtained from the trained machine learning model M1.

[0136] Thus, in the inference phase, the estimation unit 2 acquires input data IN2, which includes image IM1, physical property value P1 measured in range 31, image IM3, physical property value P3 measured in range 33, and image IM2 of the sample 61 whose physical properties are to be estimated, in the same manner as in the learning phase, and stores it in the SSD 104. Then, the estimation unit 2 uses the input data IN2 stored in the SSD 104 to infer the physical property value P2, which is the output data OUT2. There may be multiple images IM1, physical property value P1, image IM3, physical property value P3, and image IM2, but the number of images IM1, physical property value P1, image IM3, physical property value P3, and image IM2 must match the number of images IM11, physical property value P11, image IM13, physical property value P13, and image IM12 that were input when the machine learning model M1, which is the estimation model, was trained. In other words, the number of images IM1, the number of physical property values ​​P1, the number of images IM3, the number of physical property values ​​P3, and the number of images IM2 must match the number of images IM11, the number of physical property values ​​P11, the number of images IM13, the number of physical property values ​​P13, and the number of images IM12 included in a single dataset S1 during the learning phase. In this way, the estimation unit 2 can estimate the physical properties of the range 32 of the sample 61 from the trained machine learning model M1 that was created in advance during the learning phase, based on the input data IN2.

[0137] In Embodiment 2, the estimation unit 2 receives the following inputs as input data IN2: three images IM1 which are examples of at least one first image; three physical property values ​​P1 which are examples of at least one first physical property value; two images IM3 which are examples of at least one third image; two physical property values ​​P3 which are examples of at least one third physical property value; and three images IM2 which are examples of at least one second image. Based on the received inputs, images IM1, physical property value P1, image IM3, physical property value P3, and image IM2, the estimation unit 2 estimates the physical property value P2.

[0138] Thus, according to Embodiment 2, in the inference phase, the estimation unit 2 estimates the physical property value P2, eliminating the need to prepare the measuring device 550. In other words, the effort of setting up the measuring device 550 to evaluate the prototyped sample 61 each time a prototype sample 61 is fabricated is eliminated, thus improving the efficiency of evaluating the sample 61. Therefore, the estimation accuracy of the physical property value P2 in the macroscopic region of the sample is improved. Furthermore, even if it is possible to measure the physical property value P2 with the measuring device 500, the step of measuring the physical property value P2 with the measuring device 500 can be omitted, thus improving the efficiency of evaluating the sample 61.

[0139] [Embodiment 3] Embodiment 3 will now be described. Hereinafter, elements denoted by reference numerals common to the various embodiments described above will have substantially the same configuration and function as those described in Embodiment 1 unless otherwise specified. The differences from Embodiment 1 will be the main focus of this description.

[0140] Since the hardware configuration of the physical property estimation system in Embodiment 3 is substantially the same as the hardware configuration of the physical property estimation system 1000 in Embodiment 1 shown in Figure 1, a description of the hardware configuration of the physical property estimation system in Embodiment 3 will be omitted. In Embodiment 3, multiple types of physical property values ​​that are different from each other in the first range are used as input data in the learning phase and the inference phase.

[0141] In Embodiment 3, the CPU 101 shown in Figure 1 functions as the learning unit 1 and estimation unit 2 shown in Figure 2(a) by executing the program 161. Specifically, the CPU 101 functions as the learning unit 1 in the learning phase and as the estimation unit 2 in the inference phase. The learning unit 1 executes the learning method, and the estimation unit 2 executes the physical property estimation method. The object to be inferred is referred to as a "sample," and the object used to acquire learning data is referred to as a "test piece." The sample is, for example, a prototype.

[0142] Figure 12(a) is an explanatory diagram of an example of training data T1 according to Embodiment 3. Figure 12(b) is an explanatory diagram of an example of input data IN2 and output data OUT2, which are the inference results, used for inference according to Embodiment 3.

[0143] The learning unit 1 performs supervised learning as a machine learning method during the learning phase. The learning unit 1 performs supervised machine learning using training data T1 which has multiple datasets S1 of input data IN1 and ground truth data A1, and generates a trained machine learning model M1 as shown in Figure 2(a). The machine learning model M1 is stored, for example, in SSD104 in Figure 1. In the inference phase, the estimation unit 2 performs inference on input data IN2 using the trained machine learning model M1 and outputs output data OUT2 which is the inference result.

[0144] In the third embodiment, the physical property corresponding to the physical property value P11 described in the first embodiment is referred to as the first physical property. Each of the multiple datasets S1 includes, as input data IN1, at least one image IM11 of the micro region of the test specimen, at least one physical property value P11 of the first physical property in the micro region of the test specimen, at least one physical property value P14 of the second physical property in the micro region of the test specimen, and at least one image IM12 of the macro region of the test specimen.

[0145] The second physical property is a different type of physical property from the first physical property. As explained in the first embodiment, physical properties include mechanical properties, electrical properties, thermal properties, magnetic properties, and optical properties. The first and second physical properties may be different properties. For example, the first physical property may be a mechanical property and the second physical property may be an electrical property. Also, the first and second physical properties may be the same property but have different indices. For example, both the first and second physical properties may be mechanical properties and correlated, but the first physical property may be Young's modulus and the second physical property may be hardness. Examples of hardness include Vickers hardness, Brinell hardness, and Rockwell hardness. In other words, the physical property value P14 corresponds to a different type of physical property than the first physical property corresponding to the physical property value P11.

[0146] Furthermore, each of the multiple datasets S1 includes, as ground truth data A1, a physical property value P12 of the physical properties in the macro region of the test specimen. The physical property corresponding to the physical property value P12 may be the first physical property.

[0147] Image IM11 is digital data such as an image acquired by the imaging device 200 when it images the test specimen at a first magnification. Image IM12 is digital data such as an image acquired by the imaging device 250 when it images the test specimen at a second magnification, which is lower than the first magnification. The imaging device 200 images the test specimen at a higher magnification than the imaging device 250. Both images IM11 and IM12 are images acquired at a magnification higher than 1x.

[0148] The input data IN2 includes at least one image IM1 of the sample in the micro region, at least one physical property value P1 of the first physical property in the sample in the micro region, at least one physical property value P4 of the second physical property in the sample in the micro region, and at least one image IM2 of the sample in the macro region. Physical property value P1 is an example of the first physical property value. Physical property value P4 is an example of the fourth physical property value. The output data OUT2 is the physical property value P2 of the first physical property in the macro region. Physical property value P2 is an example of the second physical property value.

[0149] Image IM1 is digital data such as an image acquired by the imaging device 200 when it images the sample at a first magnification. Image IM2 is digital data such as an image acquired by the imaging device 250 when it images the sample at a second magnification. The imaging device 200 images the sample at a higher magnification than the imaging device 250. Both images IM1 and IM2 are images acquired at a magnification higher than 1x.

[0150] The number of images IM1 is the same as the number of images IM11 in one dataset S1. The number of physical property values ​​P1 is the same as the number of physical property values ​​P11 in one dataset S1. The number of physical property values ​​P4 is the same as the number of physical property values ​​P14 in one dataset S1. The number of images IM2 is the same as the number of images IM12 in one dataset S1.

[0151] Figure 13(a) is a schematic diagram of a micro-test specimen 51 according to Embodiment 3. Figure 13(b) is a schematic diagram of an image IM11 obtained by imaging the imaging range 21 of the micro-test specimen 51 according to Embodiment 3. The micro-test specimen 51 is a specimen with an area of ​​1 mm × 0.3 mm and a thickness of 50 μm.

[0152] Figure 14(a) is a schematic diagram of the standard test specimen 52 according to Embodiment 3. The standard test specimen 52 is a 3 mm thick dumbbell test specimen conforming to JIS Z 2241 as defined by the Japanese Industrial Standards. Figure 14(b) is a schematic diagram of the image IM12 obtained by imaging the imaging range 22 of the standard test specimen 52 according to Embodiment 3.

[0153] Image IM11 shown in Figure 13(b) is an image taken at a first magnification of 500x using a microscope, which is an example of an imaging device 200, to capture one of the multiple imaging ranges 21 of the micro specimen 51 shown in Figure 13(a), similar to Embodiment 1. Image IM12 shown in Figure 14(b) is also an image taken at a second magnification of 50x, which is lower than the first magnification, using an optical microscope, which is an example of an imaging device 250, to capture one of the multiple (e.g., three) imaging ranges 22 of the standard specimen 52 shown in Figure 14(a), similar to Embodiment 1.

[0154] Next, in the range 11 of the micro specimen 51 associated with the image IM11 obtained by imaging the imaging range 21, two different types of physical properties P11 and P14 are measured by the measuring device 500. In Embodiment 3, the physical property P11 in the range 11 of the micro specimen 51 is obtained as the value of the mechanical property of the range 11 of the micro specimen 51, for example, the Young's modulus of the range 11, and the physical property P14 of the second physical property in the range 11 of the micro specimen 51 is obtained as the value of the mechanical property of the range 11 of the micro specimen 51, for example, the hardness of the range 11. That is, the first physical property is, for example, Young's modulus, and the second physical property is, for example, hardness.

[0155] The physical properties P11 and P14 are obtained by measuring range 11 using a nanoindenter as the measuring device 500. Specifically, a Berkovich indenter indentation test is performed using the nanoindenter, the relationship between load and indentation amount is obtained, and the Oliver-Pharr method is used to analyze it and obtain the Young's modulus value as physical property P11 and the hardness value as physical property P14. Range 11 is essentially the size of the indentation. Although the physical property P14 was explained using the measuring device 500 used to obtain physical property P11 as an example, it is not limited to this, and physical property P14 may be obtained using a different device than the measuring device 500.

[0156] In Embodiment 3, the physical property value P12, which is the ground truth data A1 of the training data T1, is acquired by a measuring device 550 different from the measuring device 500. The physical property value P12 is acquired by measuring a range 12 of the standard test piece 52 with the measuring device 550. Range 12 is a wider range than range 11. In this example, the physical property value P12 is the Young's modulus value in range 12 of the standard test piece 52.

[0157] The measuring device 550 is, for example, a tensile testing machine. In accordance with JIS Z 2241, defined by the Japanese Industrial Standards, both ends of the standard test specimen 52 are stretched, and the Young's modulus, as the first physical property, is measured from the load and elongation. The range 12 of the standard test specimen 52 is the entire standard test specimen 52.

[0158] In the learning phase, images IM11, IM12 and physical properties P11, P14, P12 are obtained from multiple micro specimens 51 and multiple standard specimens 52, each with different material preparation processes such as material composition, filler content, mixing conditions, and heat treatment conditions. Multiple datasets S1 are created, each containing image IM11, physical properties P11, P14, and image IM12 as input data IN1, and physical property P12 as ground truth data A1. The learning data T1 is then prepared using these multiple datasets S1. Machine learning is performed using the same method as in Embodiment 1, with the multiple datasets S1 as input, to create a machine learning model M1.

[0159] Figure 15 is an explanatory diagram of machine learning in Embodiment 3. Figure 15 shows the input and output layers of a regression model using a CNN. The input layer includes three images IM11, three physical properties P11, three physical properties P14, and three images IM12. By having the learning unit 1 perform machine learning through multiple intermediate layers so that one physical property P12 is output, a machine learning model M1, which is an estimated model, is created.

[0160] Next, the inference phase will be explained. Figure 16(a) is a schematic diagram of the sample 61 according to Embodiment 3. Figure 16(b) is a schematic diagram of the image IM1 obtained by imaging the imaging range 41 of the sample 61 according to Embodiment 3. Figure 16(c) is a schematic diagram of the image IM2 obtained by imaging the imaging range 42 of the sample 61 according to Embodiment 3.

[0161] Image range 41 is an example of the first image range. Image IM1 is an example of the first image. Image range 41 is imaged by the imaging device 200 at the same magnification as in Embodiment 1, similar to Embodiment 1. Image range 42 is an example of the second image range. Image IM2 is an example of the second image. Image range 42 is imaged by the imaging device 250 at the same magnification as in Embodiment 1, similar to Embodiment 1.

[0162] Next, the physical property values ​​P1 for the first physical property and P4 for the second physical property are obtained in range 31 of the sample 61 associated with the image IM1 obtained by imaging range 41. Range 31 is an example of the first range. In Embodiment 3, the first physical property is Young's modulus, and the physical property value P1 is a value indicating Young's modulus. Also in Embodiment 3, the second physical property is hardness, and the physical property value P4 is a value indicating hardness. Both Young's modulus and hardness are examples of mechanical properties. The values ​​of Young's modulus and hardness are obtained by measuring range 31 using a nanoindenter.

[0163] The estimation unit 2 uses at least image IM1, physical property value P1, physical property value P4, and image IM2 to estimate the physical property value P2 of the first physical property in a range 32 that is wider than range 31 of the sample 61. Range 32 is an example of a second range.

[0164] In Embodiment 3, the first physical property value P2 in range 32 is estimated by the inference process of the estimation unit 2. That is, the estimation unit 2 uses a pre-trained machine learning model M1 to estimate the physical property value P2, with image IM1, physical property value P1, physical property value P4, and image IM2 as input data IN2, and physical property value P2 as output data OUT2. In this example, the first physical property in range 32 is the Young's modulus of sample 61.

[0165] In the inference phase, images IM1 and IM2 and physical properties P1 and P4 are included in the input data IN2, and the physical property P2 is output as output data OUT2. Thus, in Embodiment 3, in the inference phase, there is no need to prepare the sample 61 in accordance with Japanese Industrial Standards, and the cost and time required for processing the sample 61 can be reduced. Furthermore, the accuracy of estimating the physical property P2 is improved by increasing the number of physical properties used for estimation.

[0166] [Embodiment 4] Embodiment 4 will now be described. Hereinafter, elements denoted by reference numerals common to the various embodiments described above will have substantially the same configuration and function as those described in Embodiment 1 unless otherwise specified. The differences from Embodiment 1 will be the main focus of this description.

[0167] The hardware configuration of the material property estimation system in Embodiment 4 is substantially the same as the hardware configuration of the material property estimation system 1000 in Embodiment 1 shown in Figure 1, therefore, a description of the hardware configuration of the material property estimation system in Embodiment 4 will be omitted. In Embodiment 4, the CPU 101 estimates a second material property that is correlated with the first material property. Here, correlated material properties are material properties that are related by physical factors, such as hardness and strength. Hard materials have strong bonds between atoms, are difficult to deform, and therefore tend to be strong against external forces and have high strength, so hardness and strength are considered to be related by physical factors.

[0168] In Embodiment 4, the CPU 101 shown in Figure 1 functions as the learning unit 1 and estimation unit 2 shown in Figure 2(a) by executing the program 161. Specifically, the CPU 101 functions as the learning unit 1 in the learning phase and as the estimation unit 2 in the inference phase. The learning unit 1 executes the learning method, and the estimation unit 2 executes the physical property estimation method. The object to be inferred is referred to as a "sample," and the object used to acquire learning data is referred to as a "test piece." The sample is, for example, a prototype.

[0169] Figure 17(a) is an explanatory diagram of an example of training data T1 according to Embodiment 4. Figure 17(b) is an explanatory diagram of an example of input data IN2 and output data OUT2, which are the inference results, used for inference according to Embodiment 4.

[0170] The learning unit 1 performs supervised learning as a machine learning method during the learning phase. The learning unit 1 performs supervised machine learning using training data T1 which has multiple datasets S1 of input data IN1 and ground truth data A1, and generates a trained machine learning model M1 as shown in Figure 2(a). The machine learning model M1 is stored, for example, in SSD104 in Figure 1. In the inference phase, the estimation unit 2 performs inference on input data IN2 using the trained machine learning model M1 and outputs output data OUT2 which is the inference result.

[0171] In the learning phase, the learning unit 1 takes image IM11, physical property value P11, and image IM12 as input, and uses physical property value P15, which is correlated with physical property value P11, as the ground truth data A1 to create a machine learning model M1. In the inference phase, the estimation unit 2 takes image IM1, physical property value P1, and image IM2 as input data IN2, and estimates the physical property value P5 using the created machine learning model M1.

[0172] Each of the multiple datasets S1 includes, as input data IN1, at least one image IM11 of the micro-region of the test specimen, at least one physical property value P11 of the physical properties of the micro-region of the test specimen, and at least one image IM12 of the macro-region of the test specimen. In addition, each of the multiple datasets S1 includes, as ground truth data A1, a physical property value P15 that is correlated with the physical property value P11 of the physical properties of the macro-region of the test specimen.

[0173] Image IM11 is digital data such as an image acquired by the imaging device 200 when it images the test specimen at a first magnification. Image IM12 is digital data such as an image acquired by the imaging device 250 when it images the test specimen at a second magnification, which is lower than the first magnification. The imaging device 200 images the test specimen at a higher magnification than the imaging device 250. Both images IM11 and IM12 are images acquired at a magnification higher than 1x.

[0174] The input data IN2 includes at least one image IM1 of the sample in the micro region, at least one physical property value P1 of the sample in the micro region, and at least one image IM2 of the sample in the macro region. The output data OUT2 is a physical property value P5 that is correlated with the physical property value P1 in the macro region. Physical property value P1 is an example of a first physical property value. Physical property value P5 is an example of a second physical property value.

[0175] Image IM1 is digital data such as an image acquired by the imaging device 200 when it images the sample at a first magnification. Image IM2 is digital data such as an image acquired by the imaging device 250 when it images the sample at a second magnification. The imaging device 200 images the sample at a higher magnification than the imaging device 250. Both images IM1 and IM2 are images acquired at a magnification higher than 1x.

[0176] The number of images IM1 is the same as the number of images IM11 in one dataset S1. The number of physical property values ​​P1 is the same as the number of physical property values ​​P11 in one dataset S1. The number of images IM2 is the same as the number of images IM12 in one dataset S1.

[0177] The micro-test specimen 51 according to Embodiment 4 is the same as the micro-test specimen 51 shown in Figure 13(a) of Embodiment 3, and the image IM11 shown in Figure 13(b) is obtained by imaging the imaging range 21. A detailed description of the micro-test specimen 51 is omitted. Similarly, the standard test specimen 52 according to Embodiment 4 is the same as the standard test specimen 52 shown in Figure 14(a) of Embodiment 3, and the image IM12 shown in Figure 14(b) is obtained by imaging the imaging range 22. The micro-test specimen 51, standard test specimen 52, image IM11, and image IM12 are as described in Embodiment 3, and a detailed description of them is omitted. In Embodiment 4, the physical properties in the macro region of the micro-test specimen 51 and standard test specimen 52 differ from those in Embodiment 3.

[0178] The physical property value P11 of the micro specimen 51 in range 11, associated with the image IM11 obtained by imaging range 21, is acquired using the measuring device 500. In Embodiment 4, the physical property value P11 in range 11 of the micro specimen 51 is a value indicating the mechanical properties in range 11 of the micro specimen 51, for example, a value indicating Young's modulus in range 11.

[0179] The physical property value P11 is obtained by measuring range 11 using a nanoindenter as the measuring device 500. Specifically, a Berkovich indenter indentation test is performed with the nanoindenter, the relationship between load and indentation amount is obtained, and the Young's modulus value is obtained by analyzing it using the Oliver-Pharr method. Range 11 is essentially the size of the indentation.

[0180] In Embodiment 4, the physical property value P15, which is the correct data A1 of the learning data T1, is acquired by a measuring device 550 different from the measuring device 500. The physical property value P15 is acquired by measuring the range 12 of the standard test piece 52 shown in Figure 14(a) with the measuring device 550. Range 12 is a wider range than range 11. In this example, the physical property corresponding to the physical property value P15 is the tensile strength in range 12 of the standard test piece 52. The measuring device 550 is, for example, a tensile testing machine. In accordance with JIS Z 2241 specified in the Japanese Industrial Standards, the physical property value P15 in range 12 of the standard test piece 52 is acquired by pulling both ends of the standard test piece 52 and measuring the tensile strength from the maximum load and the cross-sectional area of ​​the standard test piece 52.

[0181] In the learning phase, images IM11, IM12 and physical properties P11, P15 are obtained from multiple micro specimens 51 and multiple standard specimens 52 with different material preparation processes, including material composition, filler content, mixing conditions, and heat treatment conditions. Multiple datasets S1 are created, each containing image IM11, physical property P11, and image IM12 as input data IN1, and physical property P15 as ground truth data A1. The learning data T1 is then prepared using these multiple datasets S1. Machine learning is performed using the same method as in Embodiment 1, with the multiple datasets S1 as input, to create a machine learning model M1.

[0182] Figure 18 is an explanatory diagram of machine learning in Embodiment 4. Figure 18 shows the input and output layers of a regression model using a CNN. The input layer includes three images IM11, three physical properties P11, and three images IM12. By having the learning unit 1 perform machine learning through multiple hidden layers so that one physical property P15 is output, a machine learning model M1, which is an estimation model, is created.

[0183] Next, the inference phase will be described. In the inference phase of Embodiment 4, the sample 61 shown in Figure 16(a) will also be used. Then, image IM1 shown in Figure 16(b), which is an image of the imaging range 41, and image IM2 shown in Figure 16(c), which is an image of the imaging range 42, will be obtained. The sample 61 and images IM1 and IM2 are the same as those described in Embodiment 3, so a detailed description of them will be omitted. In Embodiment 4, the physical properties of the sample 61 in the macro region differ from those in Embodiment 3.

[0184] In the range 31 of the sample 61 associated with the image IM1 obtained by imaging the imaging range 41, the physical property value P1 is acquired. Range 31 is an example of a first range. In Embodiment 4, the physical property value P1 in range 31 of the sample 61 is a value that indicates the mechanical properties of the sample 61 in range 31, for example, a value that indicates the Young's modulus in range 31.

[0185] Then, the estimation unit 2 estimates the physical property value P5 of the sample 61 in a range 32 that is wider than range 31, using at least image IM1, physical property value P1, and image IM2. Range 32 is an example of a second range.

[0186] In Embodiment 4, the physical property value P5 in range 32 is estimated by the inference process of the estimation unit 2. That is, the estimation unit 2 uses a pre-trained machine learning model M1 to estimate the physical property value P5, with image IM1, physical property value P1, and image IM2 as input data IN2, and the physical property value P5 as output data OUT2. In this example, the physical property corresponding to the physical property value P5 is the tensile strength of the sample 61 in range 32.

[0187] In the inference phase, images IM1 and IM2 and physical property value P1 are included in the input data IN2, and the physical property value P5 is output as output data OUT2. Thus, in Embodiment 4, in the inference phase, there is no need to prepare sample 61 in accordance with Japanese Industrial Standards, and the cost and time required for processing sample 61 can be reduced. Furthermore, it becomes possible to accurately estimate physical properties that are related due to physical factors, even if they are not the same physical properties.

[0188] This disclosure is not limited to the embodiments described above, and many modifications are possible within the technical concept of this disclosure. Furthermore, the effects described in this embodiment are merely a list of the most preferred effects arising from the embodiments of this disclosure, and are not limited to those described in this embodiment.

[0189] In the above-described embodiment, it was explained that it is preferable for the estimation unit 2 to estimate the physical property value P2 using a pre-trained machine learning model M1. However, the method for estimating the physical property value P2 is not limited to machine learning, and the physical property value P2 may be estimated by performing predetermined calculation processing.

[0190] In summary, this disclosure provides a technique that is advantageous for improving the accuracy of estimating the physical properties of a sample.

[0191] (Other embodiments) This disclosure can also be implemented by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be implemented by a circuit (e.g., an ASIC) that implements one or more functions.

[0192] The above disclosure of embodiments includes the following sections.

[0193] (Section 1) A physical property estimation device equipped with a processor, The aforementioned processor, The system accepts inputs: a first image in which a first imaging range of the sample is captured at a first magnification; a first physical property value in the first range of the sample; and a second image in which a second imaging range of the sample, which is wider than the first imaging range, is captured at a second magnification lower than the first magnification. Using at least the first image, the first physical property, and the second image, the second physical property in a second range wider than the first range of the sample is estimated. A physical property estimation device characterized by the following features.

[0194] (Section 2) The first range overlaps with the first imaging range. The physical property estimation device according to item 1, characterized in that

[0195] (Section 3) The first range is a range that is greater than or equal to the width of the first imaging range and less than or equal to the width of the second imaging range. The physical property estimation device according to item 2, characterized in that

[0196] (Section 4) The first range is 0.9 times or more and 1.1 times or less the width of the first imaging range. The physical property estimation device according to item 2, characterized in that

[0197] (Section 5) The first magnification is between 5 and 20 times the second magnification. A physical property estimation device according to any one of items 1 to 4, characterized by the above.

[0198] (Section 6) The processor overlays the first imaging range of the first image onto the second image and displays it on the display device. A physical property estimation device according to any one of items 1 to 5, characterized by the above.

[0199] (Section 7) The processor uses a pre-trained machine learning model that takes the first image, the first physical property, and the second image as input data and the second physical property as output data to estimate the second physical property. A physical property estimation device according to any one of items 1 to 6, characterized by the above.

[0200] (Section 8) Each of the physical properties corresponding to the first physical property value and the physical property corresponding to the second physical property value is an electrical property, a thermal property, a mechanical property, or an optical property. A physical property estimation device according to any one of items 1 to 7, characterized by the above.

[0201] (Section 9) The aforementioned processor, The third imaging range of the sample, which is wider than the first imaging range and narrower than the second imaging range, accepts input of a third image captured at a third magnification between the first and second magnifications, and a third physical property value in the third range, which is wider than the first range and narrower than the second range. The second physical property is estimated using the first image, the first physical property, the second image, the third image, and the third physical property. A physical property estimation device according to any one of items 1 to 8, characterized by the above.

[0202] (Section 10) The aforementioned processor, The system accepts input of a fourth physical property of a different type from the first physical property within the first range, The second physical property is estimated using at least the first image, the first physical property, the fourth physical property, and the second image. A physical property estimation device according to any one of items 1 to 9, characterized by the above.

[0203] (Section 11) The second physical property is correlated with the first physical property. A physical property estimation device according to any one of items 1 to 10, characterized by the features described above.

[0204] (Section 12) A physical property estimation device described in any one of items 1 to 11, A first imaging device for imaging the first imaging range of the sample and acquiring the first image, A second imaging device for imaging the second imaging range of the sample and acquiring the second image, The device comprises a measuring device that measures the physical properties of the sample within the first range and obtains the first physical property value. A material property estimation system characterized by the following:

[0205] (Section 13) A physical property estimation device described in any one of items 1 to 11, An imaging device that captures a first imaging range of the sample to acquire a first image, and captures a second imaging range of the sample to acquire a second image, The device comprises a measuring device that measures the physical properties of the sample within the first range and obtains the first physical property value. A material property estimation system characterized by the following:

[0206] (Section 14) A method for estimating physical properties using a processor, The processor receives inputs: a first image in which a first imaging range of the sample is captured at a first magnification; a first physical property value in the first range of the sample; and a second image in which a second imaging range of the sample, which is wider than the first imaging range, is captured at a second magnification lower than the first magnification. The processor estimates a second physical property value in a second range that is wider than the first range of the sample, based on the first image, the first physical property value, and the second image. A method for estimating physical properties characterized by the above.

[0207] (Section 15) The first range overlaps with the first imaging range. The method for estimating physical properties according to item 14, characterized by the features described above.

[0208] (Section 16) The first range is a range that is greater than or equal to the width of the first imaging range and less than or equal to the width of the second imaging range. The method for estimating physical properties according to item 15, characterized by the features described herein.

[0209] (Section 17) The first range is 0.9 times or more and 1.1 times or less the width of the first imaging range. The method for estimating physical properties according to item 15, characterized by the features described herein.

[0210] (Section 18) The first magnification is between 5 and 20 times the second magnification. A method for estimating physical properties according to any one of items 14 to 17, characterized by the features described above.

[0211] (Section 19) The processor overlays the first imaging range of the first image onto the second image and displays it on the display device. A method for estimating physical properties according to any one of items 14 to 18, characterized by the features described above.

[0212] (Section 20) The processor uses a pre-trained machine learning model that takes the first image, the first physical property, and the second image as input data and the second physical property as output data to estimate the second physical property. A method for estimating physical properties according to any one of items 14 to 19, characterized by the features described above.

[0213] (Section 21) Each of the physical properties corresponding to the first physical property value and the physical property corresponding to the second physical property value is an electrical property, a thermal property, a mechanical property, or an optical property. A method for estimating physical properties according to any one of items 14 to 20, characterized by the above.

[0214] (Section 22) The processor receives input of a third image captured at a third magnification between the first and second magnifications, and third physical property values ​​in the third range, which is wider than the first range and narrower than the second range, within a third imaging range that is wider than the first range and narrower than the second range. The processor estimates the second physical property using the first image, the first physical property, the second image, the third image, and the third physical property. A method for estimating physical properties according to any one of items 14 to 21, characterized by the features described above.

[0215] (Section 23) The aforementioned processor, The system accepts input of a fourth physical property of a different type from the first physical property in the first range, The second physical property is estimated using at least the first image, the first physical property, the fourth physical property, and the second image. A method for estimating physical properties according to any one of items 14 to 22, characterized by the features described above.

[0216] (Section 24) The second physical property is correlated with the first physical property. A method for estimating physical properties according to any one of items 14 to 23, characterized by the features described above.

[0217] (Section 25) A program for causing a computer to execute the physical property estimation method described in any one of items 14 to 24.

[0218] (Section 26) A computer-readable recording medium on which the program described in item 25 is recorded. [Explanation of Symbols]

[0219] IM1…Image (First Image), IM2…Image (Second Image), P1…Physical Property Value (First Physical Property Value), P2…Physical Property Value (Second Physical Property Value), 2…Estimation Unit (Processor), 31…Range (First Range), 32…Range (Second Range), 41…Imaging Range (First Imaging Range), 42…Imaging Range (Second Imaging Range), 61…Sample, 100…Physical Property Estimation Device, 200…Imaging Device (First Imaging Device), 250…Imaging Device (Second Imaging Device), 500…Measurement Device, 1000…Physical Property Estimation System

Claims

1. A physical property estimation device equipped with a processor, The aforementioned processor, The system accepts inputs: a first image in which a first imaging range of the sample is captured at a first magnification; a first physical property value in the first range of the sample; and a second image in which a second imaging range, which is wider than the first imaging range of the sample, is captured at a second magnification lower than the first magnification. Using at least the first image, the first physical property, and the second image, the second physical property in a second range wider than the first range of the sample is estimated. A physical property estimation device characterized by the following features.

2. The first range overlaps with the first imaging range. The physical property estimation apparatus according to feature 1.

3. The first range is a range that is greater than or equal to the width of the first imaging range and less than or equal to the width of the second imaging range. The physical property estimation device according to feature 2.

4. The first range is 0.9 times or more and 1.1 times or less the width of the first imaging range. The physical property estimation device according to feature 2.

5. The first magnification is 5 times or more and 20 times or less the second magnification. The physical property estimation apparatus according to feature 1.

6. The processor overlays the first imaging range of the first image onto the second image and displays it on the display device. The physical property estimation apparatus according to feature 1.

7. The processor uses a pre-trained machine learning model that takes the first image, the first physical property, and the second image as input data and the second physical property as output data to estimate the second physical property. The physical property estimation apparatus according to feature 1.

8. Each of the physical properties corresponding to the first physical property value and the physical property corresponding to the second physical property value is an electrical property, a thermal property, a mechanical property, or an optical property. The physical property estimation apparatus according to feature 1.

9. The aforementioned processor, The third imaging range of the sample, which is wider than the first imaging range and narrower than the second imaging range, receives input of a third image captured at a third magnification between the first and second magnifications, and a third physical property value in the third range, which is wider than the first range and narrower than the second range. The second physical property is estimated using the first image, the first physical property, the second image, the third image, and the third physical property. The physical property estimation apparatus according to feature 1.

10. The aforementioned processor, The system accepts input of a fourth physical property of a different type from the first physical property within the first range, The second physical property is estimated using at least the first image, the first physical property, the fourth physical property, and the second image. The physical property estimation apparatus according to feature 1.

11. The second physical property is correlated with the first physical property. The physical property estimation apparatus according to feature 1.

12. A physical property estimation device according to any one of claims 1 to 11, A first imaging device for imaging the first imaging range of the sample and acquiring the first image, A second imaging device for acquiring a second image by imaging the second imaging range of the sample, The device comprises a measuring device that measures the physical properties of the sample within the first range and obtains the first physical property value. A material property estimation system characterized by the following:

13. A physical property estimation device according to any one of claims 1 to 11, An imaging device that captures a first imaging range of the sample to acquire a first image, and captures a second imaging range of the sample to acquire a second image, The device comprises a measuring device that measures the physical properties of the sample within the first range and obtains the first physical property value. A material property estimation system characterized by the following:

14. A method for estimating physical properties using a processor, The processor receives inputs: a first image in which a first imaging range of the sample is captured at a first magnification; a first physical property value in the first range of the sample; and a second image in which a second imaging range of the sample, which is wider than the first imaging range, is captured at a second magnification lower than the first magnification. The processor estimates a second physical property value in a second range that is wider than the first range of the sample, based on the first image, the first physical property value, and the second image. A method for estimating physical properties characterized by the above.

15. The first range overlaps with the first imaging range. The method for estimating physical properties according to feature 14.

16. The first range is a range that is greater than or equal to the width of the first imaging range and less than or equal to the width of the second imaging range. The method for estimating physical properties according to feature 15.

17. The first range is 0.9 times or more and 1.1 times or less the width of the first imaging range. The method for estimating physical properties according to feature 15.

18. The first magnification is 5 times or more and 20 times or less the second magnification. The method for estimating physical properties according to feature 14.

19. The processor overlays the first imaging range of the first image onto the second image and displays it on the display device. The method for estimating physical properties according to feature 14.

20. The processor uses a pre-trained machine learning model that takes the first image, the first physical property, and the second image as input data and the second physical property as output data to estimate the second physical property. The method for estimating physical properties according to feature 14.

21. Each of the physical properties corresponding to the first physical property value and the physical property corresponding to the second physical property value is an electrical property, a thermal property, a mechanical property, or an optical property. The method for estimating physical properties according to feature 14.

22. The processor receives input of a third image captured at a third magnification between the first and second magnifications, and third physical property values ​​in the third range, which is wider than the first range and narrower than the second range, within a third imaging range that is wider than the first range and narrower than the second range. The processor estimates the second physical property using the first image, the first physical property, the second image, the third image, and the third physical property. The method for estimating physical properties according to feature 14.

23. The aforementioned processor, The system accepts input of a fourth physical property of a different type from the first physical property in the first range, The second physical property is estimated using at least the first image, the first physical property, the fourth physical property, and the second image. The method for estimating physical properties according to feature 14.

24. The second physical property is correlated with the first physical property. The method for estimating physical properties according to feature 14.

25. A program for causing a computer to execute the physical property estimation method described in any one of claims 14 to 24.

26. A computer-readable recording medium having the program described in claim 25 recorded on it.

Images