Information processing device, information processing method, and recording medium recording information processing program

Abstract

The present invention provides an information processing device, an information processing method, and a recording medium having an information processing program recorded thereon, which can easily judge different regions of crystallinity of a material on an image captured by a transmission electron microscope (TEM). An acquisition unit (142) of a server (14) acquires an image captured by a transmission electron microscope. Next, a calculation unit (144) calculates the difference in pixel values ​​of pixels contained in each of the local regions of the acquired image. A judgment unit (146) judges the height of crystallinity of each of the local regions of the image based on the difference in pixel values ​​of the calculated local regions.

Description

Technical Field

[0001] The present disclosure relates to an information processing device, an information processing method, and a recording medium having an information processing program recorded thereon. Background Art

[0002] Conventionally, there is a known technique for analyzing images captured by a transmission electron microscope (TEM) (hereinafter, simply referred to as "TEM images"). For example, in HREM Research Inc., "GPA for Digital Micrograph",<https: / / www.hremresearch.com / Eng / download / documents / gpa4dm.pdf> A general image analysis technique for TEM images is disclosed.

[0003] In addition, in Jacob Madsen et al.’s “A Deep Learning Approach to Identify LocalStructures in Atomic-Resolution Transmission Electron Microscopy Images”, <https: / / onlinelibrary.wiley.com / doi / abs / 10.1002 / adts.201800037@10.1002 / (ISSN)25130390.Machine Learning> In the paper, a technology is disclosed that uses a deep neural network to analyze the local structure of a substance reflected in a TEM image. Summary of the invention

[0004] Technical problem to be solved by the invention

[0005] In addition, there are cases where it is desired to evaluate the crystallinity of a material reflected in a TEM image. Figure 7 An example of a TEM image is shown in FIG. Figure 7 As shown, the portions that appear as patterns on the TEM images IM1 and IM2 are portions with high crystallinity, and the other portions are portions with low crystallinity.

[0006] In order to Figure 7 When evaluating the crystallinity of the TEM images IM1 and IM2 shown in FIG. , for example, Figure 8 The crystallinity was evaluated by the brightness value of the image as shown.

[0007] However, if Figure 8 As shown in FIG. 1 , since there are many areas with roughly the same brightness in the high-crystallinity area L1 and the high-crystallinity area L2 in the TEM image IM, the crystallinity cannot be evaluated only by the brightness value of the image. When only threshold processing is performed on the brightness value of the TEM image IM, an image with Figure 8 The image IMb shown. However, Figure 8 The image IMb extracts only the portion with a higher brightness value in pixel units, but does not extract the portion with different crystallinity in area units.

[0008] Therefore, it is impossible to judge the degree of crystallinity of a material reflected on a TEM image simply by, for example, image analysis such as that disclosed in “GPA for Digital Micrograph” by HREM Research Inc. mentioned above.

[0009] In addition, in the case of evaluating the crystallinity of the material reflected in the TEM image by neural networks, etc., as disclosed in "A Deep Learning Approach to Identify Local Structures in Atomic-Resolution Transmission Electron Microscopy Images" by Jacob Madsen et al., it is necessary to prepare a large amount of labeled learning data. The accuracy of the learned model such as neural networks depends on the quality and quantity of the learning data. In the case of generating a learned model with good accuracy, a large amount of learning data needs to be prepared, which will require a huge amount of time and effort in the preparation.

[0010] Therefore, the related technologies of the above-mentioned HREM Research Inc., "GPA for Digital Micrograph" and Jacob Madsen et al., "A Deep Learning Approach to Identify Local Structures in Atomic-Resolution Transmission Electron Microscopy Images" have the problem that the crystallinity of the material reflected in the TEM image cannot be easily judged.

[0011] The present disclosure considers the above-mentioned facts and simply determines the crystallinity of a material reflected on a TEM image.

[0012] Methods used to solve technical problems

[0013] An information processing device of the first mode is an information processing device comprising a memory and a processor connected to the memory, wherein the processor is configured to obtain an image captured by a transmission electron microscope, calculate the difference in pixel values ​​of pixels contained in each local area of ​​the acquired image, and judge the height of crystallinity of each local area of ​​the image based on the calculated difference in pixel values ​​of the local area.

[0014] The processor of the information processing device of the second aspect calculates the standard deviation of the pixel values ​​of the pixels included in the local area as the difference.

[0015] The processor of the information processing device of the third mode calculates the difference in the pixel value for each of the local areas representing the image within a predetermined range centered on the central pixel, and when judging the height of the crystallinity of the local area, the processor judges the height of the crystallinity of the central pixel in the local area based on the difference calculated for each of the local areas, thereby judging the height of the crystallinity of the local area. Thus, the height of the crystallinity can be judged for each pixel in the image.

[0016] The processor of the information processing device of the fourth mode generates a crystallinity image showing the height of crystallinity by assigning discrete pixel values ​​according to the height of crystallinity of the central pixel based on the judgment result of each of the central pixels, thereby making it possible to visualize the crystallinity of the material projected on the TEM image.

[0017] The processor of the information processing device of the fifth mode determines, based on the difference calculated for each of the local regions and a predetermined value range, that the local region is a region having a high degree of crystallinity corresponding to the value range when the difference is within the value range.

[0018] The information processing method of the sixth aspect is to execute the following processing by a computer, namely:

[0019] An image captured by a transmission electron microscope is obtained, and for each of the local areas of the image, a difference in pixel values ​​of pixels contained in the local area is calculated; and for each of the local areas of the image, a height of crystallinity of the local area is determined based on the calculated difference in pixel values ​​of the local area.

[0020] The seventh method of recording a recording medium having an information processing program recorded thereon is a recording medium having an information processing program recorded thereon for causing a computer to execute the following processing, wherein the processing is: obtaining an image captured by a transmission electron microscope, calculating the difference in pixel values ​​of pixels contained in each of the local areas of the image obtained, and, for each of the local areas of the image, judging the height of crystallinity of the local area based on the calculated difference in pixel values ​​of the local area.

[0021] Effects of the Invention

[0022] As described above, according to the present disclosure, there is an effect that the crystallinity of a material reflected on a TEM image can be easily determined. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a schematic block diagram of an information processing system according to an embodiment of the present invention.

[0024] Figure 2 A diagram for explaining a local area on a TEM image.

[0025] Figure 3 This is a diagram for explaining the relationship between the standard deviation of the pixel values ​​of pixels included in the local area and the degree of crystallinity.

[0026] Figure 4 This is a diagram showing an example of a crystallinity image.

[0027] Figure 5 It is a diagram showing a configuration example of a user terminal and a server computer according to an embodiment.

[0028] Figure 6 This is a flowchart showing an example of information processing performed by a server according to an embodiment.

[0029] Figure 7 A diagram showing an example of a TEM image.

[0030] Figure 8 This figure shows an example of an image obtained by performing threshold processing on a TEM image. DETAILED DESCRIPTION

[0031] <Implementation Method>

[0032] Figure 1 1 is a block diagram showing an example of the functional structure of the information processing system 10 according to the embodiment. Figure 1 As shown, the information processing system 10 includes a user terminal 12 and a server 14 as an example of an information processing device. The user terminal 12 and the server 14 are connected via a network 13 such as the Internet.

[0033] The information processing system 10 according to the embodiment analyzes a TEM image, which is an image captured by a transmission electron microscope, to determine the crystallinity of a material reflected on the TEM image. Specifically, the server 14 of the information processing system 10 calculates the difference in pixel values ​​of pixels included in each local area of ​​the TEM image sent from the user terminal 12, and determines the height of the crystallinity of the local area based on the difference. Thus, the crystallinity of a material reflected on the TEM image can be easily determined.

[0034] Hereinafter, this will be described in detail.

[0035] (User Terminal)

[0036] The user inputs a TEM image of a material to be analyzed to the user terminal 12 operated by the user. The TEM image is an image of a material captured by a transmission electron microscope.

[0037] The user terminal 12 transmits the TEM image input by the user to the server 14 via the network 13. The TEM image transmitted from the user terminal 12 is analyzed in the server 14 described later.

[0038] (server)

[0039] like Figure 1 As shown, the server 14 includes a transmitting and receiving unit 140 , an acquiring unit 142 , a calculating unit 144 , a determining unit 146 , and an image generating unit 148 .

[0040] The transmission and reception unit 140 receives the TEM image transmitted from the user terminal 12 .

[0041] The acquisition unit 142 acquires the TEM image received by the transmission and reception unit 140 .

[0042] The calculation unit 144 calculates the difference in pixel values ​​of pixels included in each local area of ​​the TEM image acquired by the acquisition unit 142. Figure 2, a diagram for explaining a local area on a TEM image is shown.

[0043] Specifically, if Figure 2 As shown, the calculation unit 144 sets a plurality of local regions R in the TEM image IM for expressing an image within a predetermined range centered on the central pixel P. For example, the calculation unit 144 sets each of the local regions R by sequentially setting each pixel constituting the TEM image IM as the central pixel P. Furthermore, the calculation unit 144 calculates the standard deviation of the pixel value as an example of the difference in the pixel values ​​of the pixels included in the local region for each of the plurality of local regions.

[0044] Figure 3 1 is a diagram for explaining the relationship between the standard deviation of the pixel values ​​of the pixels included in the local area and the degree of crystallinity. Figure 3 As shown in FIG. 1 , the local region R1 centered around the central pixel P1 in the TEM image IM corresponds to a portion with high crystallinity. Figure 3 As shown in FIG. 1 , the local area R2 centered at the central pixel P2 in the TEM image IM corresponds to a site with low crystallinity. Figure 3 As shown, when a graph is generated with the horizontal axis representing pixel value and the vertical axis representing the frequency of pixel value, the standard deviation of the pixel values ​​of pixels included in the local area R1 is large, and the standard deviation of the pixel values ​​of pixels included in the local area R2 is small.

[0045] Therefore, the determination unit 146 determines whether the central pixel in each of the plurality of local regions is a pixel with high crystallinity based on the standard deviation of the local region calculated by the calculation unit 144. In this way, it is determined whether the local region is a region with high crystallinity.

[0046] Specifically, the determination unit 146 determines that the central pixel of the local area is located in a region with high crystallinity based on the standard deviation of the local area and a preset threshold value for each of the plurality of local areas, if the standard deviation is larger than the threshold value. On the other hand, if the standard deviation is less than the threshold value, the determination unit 146 determines that the central pixel of the local area is not located in a region with high crystallinity.

[0047] The image generation unit 148 generates a crystalline image showing the height of crystallinity by assigning discrete pixel values ​​according to the height of crystallinity of the center pixel based on the judgment result of the judgment unit 146. Specifically, when it is judged that the crystallinity of the center pixel is high, the image generation unit 148 assigns a first pixel value (for example, a pixel value showing white) to the pixel at the position corresponding to the center pixel. On the other hand, when it is judged that the crystallinity of the center pixel is low based on the judgment result of the judgment unit 146, the image generation unit 148 assigns a second pixel value (for example, a pixel value showing black) to the pixel at the position corresponding to the center pixel. The image generation unit 148 generates a crystalline image showing the height of crystallinity in this way.

[0048] exist Figure 4 , there is shown an example of a TEM image IM, a crystallinity image IMb generated by conventional techniques, and a crystallinity image IMc generated by processing of the server 14 of this embodiment. The crystallinity image IMb generated by conventional techniques is an image obtained by evaluating the crystallinity of the image by thresholding the brightness value of the image.

[0049] In the crystalline image IMc generated by the processing of the server 14 of this embodiment, white is assigned when the central pixel is determined to have high crystallinity, and black is assigned when the central pixel is determined to have low crystallinity. Figure 4 As shown, the crystallinity image IMb obtained by the existing method is an image obtained only by threshold processing based on pixel values, and although the part with higher crystallinity is extracted in pixel units, it is not extracted in area units. In contrast, it can be seen that the crystallinity image IMc extracts the part with higher crystallinity with good accuracy in area units.

[0050] The transmitting and receiving unit 140 transmits the crystallinity image generated by the image generating unit 148 to the user terminal 12 .

[0051] The user terminal 12 receives the crystallinity image transmitted from the server 14. The user confirms the crystallinity image displayed on the display unit of the user terminal 12.

[0052] The user terminal 12 and the server 14 can, for example, Figure 5The computer 50 for realizing the user terminal 12 and the server 14 includes a CPU 51, a memory 52 as a temporary storage area, and a non-volatile storage unit 53. In addition, the computer includes an input / output interface (I / F) 54 for connecting to an input / output device (omitted from the figure), and a read / write (R / W) unit 55 for controlling the reading and writing of data to a recording medium 59. In addition, the computer includes a network I / F 56 for connecting to a network such as the Internet. The CPU 51, the memory 52, the storage unit 53, the input / output I / F 54, the R / W unit 55, and the network I / F 56 are connected to each other via a bus 57.

[0053] The storage unit 53 can be implemented by a Hard Disk Drive (HDD), a Solid State Drive (SSD), a flash memory, etc. In the storage unit 53 as a storage medium, a program for making the computer function is stored. The CPU 51 reads the program from the storage unit 53 and expands it in the memory 52, and sequentially executes the processes of the program.

[0054] Next, the operation of the information processing system 10 according to the embodiment will be described.

[0055] When a TEM image is input to the user terminal 12, the user terminal 12 sends the TEM image to the server 14. When the server 14 receives the TEM image, the server 14 executes Figure 6 The information processing routine shown.

[0056] In step S100 , the CPU 51 functions as the acquisition unit 142 to acquire the TEM image received by the transmission / reception unit 140 .

[0057] In step S102 , the CPU 51 as the calculation unit 144 calculates the standard deviation of the pixel values ​​of the pixels included in each of the local regions of the TEM image acquired in step S100 .

[0058] In step S104 , the CPU 51 as the determination unit 146 determines the degree of crystallinity of the central pixel in each of the plurality of local areas based on the standard deviation of the local area calculated in step S102 .

[0059] In step S106, the CPU 51 as the image generating unit 148 assigns a first pixel value (for example, a pixel value representing white) to the pixel at the position corresponding to the center pixel when it is judged that the center pixel has high crystallinity, and assigns a second pixel value (for example, a pixel value representing black) to the pixel at the position corresponding to the center pixel when it is judged that the center pixel has low crystallinity, based on the judgment result obtained in the above step S104, thereby generating a highly crystalline image representing crystallinity.

[0060] In step S108 , the CPU 51 functions as the transmission and reception unit 140 and transmits the crystalline image generated in step S106 as a result to the user terminal 12 .

[0061] As described above, the server 14 of the information processing system 10 involved in the embodiment calculates the difference of the pixel values ​​of the pixels contained in each of the local areas of the TEM image as an image taken by a transmission electron microscope. The server 14 determines the height of the crystallinity of the local area based on the difference of the pixel values ​​of the pixels contained in the local area for each of the multiple local areas. Thus, the crystallinity of the material mapped on the TEM image can be easily determined. In addition, since there is no need to use a learning model such as a neural network, the crystallinity of each local area in the TEM image can be determined without having to prepare learning data with labels. In addition, compared with the case where only the brightness value and the threshold value are used, the height of the crystallinity can be determined with good accuracy.

[0062] In addition, although the processing performed by each device in the above-mentioned embodiment is described as software processing performed by executing a program, it can also be performed as processing performed by hardware. Alternatively, it can also be performed as processing that combines both software and hardware. In addition, the program stored in the ROM can also be stored in various storage media and circulated.

[0063] Furthermore, the present disclosure is not limited to the above contents, and it is apparent that the present disclosure can be implemented with various modifications other than the above embodiments without departing from the gist of the present disclosure.

[0064] For example, although the above embodiment describes the case where the standard deviation is calculated as an example of the difference in pixel values ​​of pixels included in a local area in a TEM image, the present invention is not limited thereto. The difference in pixel values ​​of pixels included in a local area may also be evaluated by other indicators.

[0065] In addition, although the judgment unit 146 of the above-mentioned embodiment is based on the standard deviation and threshold of the local area, and when the standard deviation of the local area is larger than the threshold, it is judged that the central pixel of the local area is located in the area with higher crystallinity, and when the standard deviation of the local area is below the threshold, it is judged that the central pixel of the local area is located in the area with lower crystallinity as an example, it is not limited to this. For example, the following method can also be adopted, that is, the judgment unit 146 judges the height of crystallinity based on the standard deviation of the local area and the pre-set value range. Specifically, the following method can also be adopted, that is, when the standard deviation of the local area is within the value range, the judgment unit 146 judges that the local area is a region with a height of crystallinity corresponding to the value range. For example, when the standard deviation of the local area is within the value range T1 to T2, the judgment unit 146 judges that the local area is a region where the crystallinity of the central pixel of the local area has a crystallinity height C (or C1 to C2) corresponding to the value range T1 to T2. Thus, the height of crystallinity can be evaluated continuously, rather than just evaluating the height of crystallinity.

Claims

1. An information processing device, comprising a memory and a processor connected to the memory, wherein: The processor is configured to: Obtain images taken using a transmission electron microscope. For each of the local regions of the image obtained, a difference in pixel values ​​of pixels included in the local region is calculated, and for each of the local regions of the image, determining the degree of crystallinity of the local region based on the calculated difference in the pixel values ​​of the local region, When the processor calculates the difference in the pixel values, The difference is calculated for each of the local areas of the image within a predetermined range centered on the central pixel, and When determining the height of crystallinity of the local region, the height of crystallinity of the central pixel in the local region is determined based on the difference calculated for each of the local regions, thereby determining the height of crystallinity of the local region.

2. The information processing device according to claim 1, wherein: The processor calculates a standard deviation of pixel values ​​of pixels included in the local area as the difference.

3. The information processing device according to claim 1, wherein: The processor generates a crystallinity image expressing the degree of crystallinity by assigning discrete pixel values ​​according to the degree of crystallinity of the central pixel based on the determination result of each of the central pixels.

4. The information processing device according to any one of claims 1 to 3, wherein: The processor determines, based on the difference calculated for each of the local regions and a preset value range, that the local region is a region having a high degree of crystallinity corresponding to the value range when the difference is within the value range.

5. An information processing method, wherein a processor performs the following processing, namely: Obtain images taken using a transmission electron microscope. For each of the local regions of the image obtained, a difference in pixel values ​​of pixels included in the local region is calculated, and for each of the local regions of the image, determining the degree of crystallinity of the local region based on the calculated difference in the pixel values ​​of the local region, When calculating the difference in the pixel values, The difference is calculated for each of the local areas of the image within a predetermined range centered on the central pixel, and When determining the height of crystallinity of the local region, the height of crystallinity of the central pixel in the local region is determined based on the difference calculated for each of the local regions, thereby determining the height of crystallinity of the local region.

6. A recording medium having an information processing program recorded thereon, wherein the information processing program is used to cause a processor to execute the following processing, namely: Obtain images taken using a transmission electron microscope. For each of the acquired local regions of the image, a difference in pixel values ​​of pixels included in the local region is calculated, and for each of the local regions of the image, determining the degree of crystallinity of the local region based on the calculated difference in the pixel values ​​of the local region, When calculating the difference in the pixel values, The difference is calculated for each of the local areas of the image within a predetermined range centered on the central pixel, and When determining the height of crystallinity of the local region, the height of crystallinity of the central pixel in the local region is determined based on the difference calculated for each of the local regions, thereby determining the height of crystallinity of the local region.

Images