Information processing device, information processing method, and information processing program

Abstract

To provide an information processing device capable of visualizing and analyzing analysis data of a material or a component.SOLUTION: An information processing device comprises: a first acquisition unit which acquires a power spectrum of a plurality of two-dimensional data or three-dimensional data representing a state of a material or a component; a second acquisition unit which acquires a main component of a frequency component of the power spectrum and a main component score value by analyzing the main components of a plurality of power spectral; and a reconfiguration unit which generates a reconfiguration image, which is an image generated by visualizing a frequency region and an intensity on the basis of the main component, the main component score value, and the power spectrum.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] This disclosure relates to an information processing apparatus, an information processing method, and an information processing program.

Background Art

[0002] Patent Document 1 discloses a technique for easily determining the crystallinity of a material shown in an image captured by a transmission electron microscope (TEM).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the analysis of materials, it is necessary to consider which regions in the image affect the performance. In this case, it is necessary to use an extremely large number of image and performance datasets as learning data, which takes time to collect the data. Therefore, a technique for visualizing the regions in an image that affect performance even with a small dataset has been desired.

[0005] An object of this disclosure is to provide an information processing apparatus, an information processing method, and an information processing program capable of visualizing and analyzing analysis data of materials and components.

Means for Solving the Problems

[0006] The information processing apparatus according to claim 1 is The system includes: a first acquisition unit that acquires power spectra of multiple two-dimensional or three-dimensional data representing the state of a material or part; a second acquisition unit that acquires principal components and principal component score values ​​of the frequency components of the power spectra by performing principal component analysis on the multiple power spectra; and a reconstruction unit that generates a reconstructed image, which is an image that visualizes the frequency domain and intensity, based on the principal components, the principal component score values, and the power spectra.

[0007] The information processing device described in claim 1 performs principal component analysis of data and visualizes the frequency domain and intensity. This makes it possible to visualize and analyze analysis data of materials and components.

[0008] The information processing device according to claim 2 is the information processing device according to claim 1, wherein the first acquisition unit generates a plurality of one-dimensional spectra by integrating a plurality of power spectra in the azimuthal direction in the case of two-dimensional data, and by integrating them in the azimuthal and polar angle directions in the case of three-dimensional data, and the second acquisition unit obtains the principal components of the frequency components of the power spectra and the principal component score values ​​by performing principal component analysis on the generated plurality of one-dimensional spectra.

[0009] According to the information processing device described in claim 2, high-precision analysis can be achieved from the generated one-dimensional spectrum.

[0010] The information processing apparatus according to claim 3 is an information processing apparatus according to claim 1, wherein the second acquisition unit further includes an analysis unit that outputs the principal components and the principal component score values ​​as results of principal component analysis to a predetermined user terminal, accepts a specification of a change in the principal component score value for a predetermined principal component, and generates reconstructed spectral data when the principal component score value is changed. According to the information processing apparatus according to claim 3, the user can easily check the changes in the principal component analysis by changing the values.

[0011] The information processing method described in claim 4 involves a computer performing a process to obtain power spectra of a plurality of two-dimensional or three-dimensional data representing the state of a material or part, to obtain principal components and principal component scores of the frequency components of the power spectra by performing principal component analysis on the plurality of power spectra, and to generate a reconstructed image which is an image that visualizes the frequency domain and intensity based on the principal components, the principal component scores, and the power spectra.

[0012] The information processing program described in claim 5 causes a computer to perform the following processes: acquire power spectra of a plurality of two-dimensional or three-dimensional data representing the state of a material or part; perform principal component analysis on the plurality of power spectra to acquire the principal components of the frequency components of the power spectra and the values ​​of the principal component scores; and generate a reconstructed image, which is an image that visualizes the frequency domain and intensity, based on the principal components, the values ​​of the principal component scores, and the power spectra. [Effects of the Invention]

[0013] The technology disclosed herein enables the visualization and analysis of analytical data for materials and components. [Brief explanation of the drawing]

[0014] [Figure 1] Figure 1 is a diagram showing the configuration of an information processing system. [Figure 2] Figure 2 shows an example of multiple images generated by specifying the interface state and aspect ratio. [Figure 3] Figure 3 is a block diagram showing the hardware configuration of the information processing device. [Figure 4] Figure 4 shows an image illustrating the flow of acquiring spectral data and principal components. [Figure 5A] Figure 5A shows an example of the analysis interface displaying the results of principal component analysis. [Figure 5B] Figure 5B shows an example of the analysis interface displaying the results of principal component analysis. [Figure 5C] FIG. 5C is an example of a display of an analysis interface showing the results of principal component analysis. [Figure 6] FIG. 6 is a diagram showing an example of a reconstructed image. [Figure 7] FIG. 7 is a flowchart of an information processing method executed by the information processing apparatus.

MODE FOR CARRYING OUT THE INVENTION

[0015] Embodiments of the present invention will be described. In the present embodiment, as an example, a plurality of images generated by specifying changes in the aspect ratio and the interface state are generated, and principal component analysis is performed to output, as an analysis result, a reconstructed image that is an image obtained by analyzing the frequency domain and intensity.

[0016] FIG. 1 is a diagram showing the configuration of an information processing system 100. As shown in FIG. 1, in the information processing system 100, a user terminal 102 and an information processing apparatus 110 are connected via a network N such as the Internet.

[0017] The user terminal 102 is a terminal that inputs information related to image analysis. The user terminal 102 includes a control unit 104 and a display unit 106. The control unit 104 of the user terminal 102 inputs a specification for image generation. Further, the control unit 104 of the user terminal 102 causes the display unit 106 to display a principal component analysis interface described later. Further, the user terminal 102 receives the reconstructed image analyzed by the information processing system 100 and causes the display unit 106 to display it.

[0018] In image generation, it is assumed that the performance of a material is modeled assuming a microscopic image. Therefore, in the specification of image generation, changes in the aspect ratio and the interface state are specified. FIG. 2 is an example of a plurality of images generated by specifying the interface state and the aspect ratio. The example shown in FIG. 2 is an example in which, imitating a microscopic image of a material, a plurality of images in which particles having different aspect ratios, particle sizes, and interface states are distributed are generated. By changing the aspect ratio in this way, images with different patterns can be easily generated.

[0019] FIG. 3 is a block diagram showing the hardware configuration of the information processing apparatus 110. As shown in FIG. 2, the information processing apparatus 110 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage 14, an input unit 15, a display unit 16, and a communication interface (I / F) 17. Each component is connected so as to be communicable with each other via a bus 19. Note that the user terminal 102 may have a similar hardware configuration.

[0020] The CPU 11 is a central processing unit that executes various programs and controls each unit. That is, the CPU 11 reads a program from the ROM 12 or the storage 14 and executes the program using the RAM 13 as a work area. The CPU 11 performs control of each of the above components and various arithmetic processes according to a program stored in the ROM 12 or the storage 14. In the present embodiment, an information processing program is stored in the ROM 12 or the storage 14.

[0021] The ROM 12 stores various programs and various data. The RAM 13 temporarily stores a program or data as a work area. The storage 14 is composed of a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and stores various programs including an operating system and various data.

[0022] The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used to perform various inputs.

[0023] The display unit 16 is, for example, a liquid crystal display, and displays various information. The display unit 16 may adopt a touch panel method and function as the input unit 15.

[0024] The communication interface 17 is an interface for communicating with other devices such as terminals. For such communication, a wired communication standard such as Ethernet® or FDDI, or a wireless communication standard such as 4G, 5G, or Wi-Fi® may be used.

[0025] The functional configuration of the information processing device 110 shown in Figure 1 will now be explained. Functionally, the information processing device 110 consists of a storage unit 112, an image generation unit 120, a first acquisition unit 122, a second acquisition unit 124, an analysis unit 126, and a reconstruction unit 128. Each functional configuration is realized by the CPU 11 reading an information processing program stored in the ROM 12 or storage unit 14, expanding it into the RAM 13, and executing it. Note that multiple images may be acquired in advance and stored in the storage unit 112, and the processing of the image generation unit 120 may be omitted. The processing of the analysis unit 126 may also be omitted. Furthermore, the user terminal 102 may be integrated with the information processing device 110.

[0026] The memory unit 112 stores the results of the principal component analysis. The principal component analysis results include the values ​​of each principal component and the principal component score. The memory unit 112 also stores the reconstructed image obtained by analyzing and reconstructing the image.

[0027] The image generation unit 120 accepts the specification of the aspect ratio and interface state and generates multiple images. The multiple images should be generated as images representing the state of the material or part, and are assumed to be microscope images, which are images taken with a microscope. The images may be generated, for example, using a trained model that has learned the generation pattern of microscope images from the specification of the aspect ratio and interface state. In addition, the specification of the aspect ratio and interface state may be actual microscope images. Furthermore, not only two-dimensional images but also three-dimensional images may be used. Examples of three-dimensional images include X-ray CT images and 3D-SEM images. Note that the two-dimensional image and three-dimensional image are examples of two-dimensional data and three-dimensional data in this disclosure. In the following description, processing using images will be explained as an example, but it is not limited to images and can be applied to data that represents the state of an object. For example, it can be applied to three-dimensional data that can be represented as voxels by stacking images.

[0028] Figure 4 shows an image illustrating the flow of acquiring spectral data and principal components. (a) is an image generated by the image generation unit 120. (b1) is an example of a two-dimensional power spectrum, obtained by performing a Fourier transform on the image and squaring the amplitude spectrum resulting from the transformation. One power spectrum data is obtained from one image. In the frequency space, the two-dimensional power spectrum is represented with the center as the origin and the power spectrum spreading outwards from the center. (b2) is an example of a one-dimensional power spectrum obtained by integrating the two-dimensional spectrum in the azimuthal direction. The one-dimensional power spectrum is represented as a graph with scattering intensity on the vertical axis and frequency on the horizontal axis. (PS) indicates that spectral data is obtained as a spectrum group by acquiring each one-dimensional power spectrum from each of multiple images. (c) is a graph showing the scores of each principal component in the principal component analysis. The horizontal and vertical axes of the graph represent different principal components. Each point plots the value of the score of each principal component of the one-dimensional power spectrum.

[0029] The first acquisition unit 122 acquires power spectra of multiple two-dimensional or three-dimensional data representing the state of a material or part. An example is given of generating a power spectrum using an image generated by the image generation unit 120. For example, based on the input of each of multiple images (two-dimensional data), each image is Fourier transformed, and the intensity of the power spectrum obtained by squaring the amplitude spectrum resulting from the transformation is integrated in the azimuthal direction to obtain a one-dimensional power spectrum (the spectrum group described above). The first acquisition unit 122 may also perform principal component analysis on the power spectrum itself. The first acquisition unit 122 may also convert three-dimensional data into a one-dimensional spectrum. The first acquisition unit 122 can generate a one-dimensional power spectrum by integrating the three-dimensional power spectrum in the azimuthal and polar angle directions with respect to two types of angular parameters in three-dimensional polar coordinates. As described above, the first acquisition unit 112 can generate multiple one-dimensional spectra by integrating multiple power spectra in the azimuthal direction in the case of two-dimensional data, and in the azimuthal and polar angle directions in the case of three-dimensional data.

[0030] The second acquisition unit 124 performs principal component analysis on multiple power spectra to obtain the principal components of the frequency components of the power spectra and the principal component scores as results of the principal component analysis. Principal components are components obtained by reducing the numerous explanatory variables included in the object of analysis. Principal components are represented, for example, as pc1, pc2, etc., where "pc1" represents the first principal component and "pc2" represents the second principal component. The principal component score is the value of the principal component feature obtained as a result of the principal component analysis. For each image corresponding to the power spectrum, a principal component score is obtained for each principal component.

[0031] The analysis unit 126 outputs the results of the principal component analysis to the user terminal 102. The analysis unit 126 also accepts a specification of the change in the principal component score value for a predetermined principal component and generates reconstructed spectral data (reconstructed power spectrum) when the principal component score value is changed.

[0032] On the user terminal 102, the display unit 106 displays content corresponding to the principal component analysis results on the analysis interface. The user terminal 102 accepts user input and specifications from the analysis interface. The display of the analysis interface will be explained below using Figure 5 (Figures 5A to 5C) as an example. Figure 5 is an example of the display of the analysis interface showing the results of principal component analysis. On the user terminal 102, the displays of Figures 5A, 5B, and 5C can be selected or switched on the analysis interface.

[0033] Figure 5A is an example of a graph plotting the values ​​of each principal component score. The graph assigns a performance level to each point. The intensity of the color of a plotted point represents the performance of the material corresponding to that point. Specifically, a white plotted point indicates poor performance of the material, while a black plotted point indicates good performance. Performance levels can be determined using pre-set material performance values ​​(e.g., mechanical strength, surface reflectivity, and transparency). Figure 5B is an example of principal component score values ​​for each principal component displayed as bars. On the left side of Figure 5B, ten PC values ​​from pc1 to pc10 are displayed as bars. The user can change each PC value by manipulating the bars. Functions may also be provided to display the power spectrum with all PC values ​​set to zero (= the average of the input image power spectrum), and to reset the ten changed PC values ​​from pc1 to pc10 back to their original values.

[0034] Figures 5A and 5B show arrows indicating a change made by the user to decrease the principal component score of pc1. In Figure 5B, the arrow for pc1 indicates that the value of pc1 was changed from 0.71 to -1.37. The analysis unit 126 of the information processing device 110 receives such a specification of change in the principal component score and generates a reconstructed power spectrum, which is a one-dimensional power spectrum that reflects the change.

[0035] Figure 5C shows an example of a one-dimensional power spectrum displayed in the analysis interface. The dotted line (g) represents the one-dimensional power spectrum of the sample acquired from the original image, and the solid line (h) represents the reconstructed power spectrum, which is a one-dimensional power spectrum reflecting the specified change. By lowering pc1, the intensity of some frequencies is amplified in (h) compared to (g). In this way, the analysis unit 126 makes it easy to check the changes in spectral data when the principal component score values ​​are changed.

[0036] The reconstruction unit 128 generates a reconstructed image, which is an image that visualizes the frequency domain and intensity, based on the principal components, the principal component score values, and the power spectrum. More specifically, the reconstruction unit 128 first generates a reconstructed power spectrum by multiplying the power spectrum of each image by an intensity filter created using principal component score values ​​modified by the user. Then, the reconstruction unit 128 generates a reconstructed image by performing an inverse Fourier transform on the reconstructed power spectrum using the phase information of the original image. Figure 6 shows an example of a reconstructed image. The reconstructed image is drawn so that the intensity in the frequency domain is distinguished by a gradient of positive and negative components.

[0037] The reconstruction unit 128 outputs the generated reconstructed image to the user terminal 102. The user terminal 102 displays the reconstructed image on the display unit 106. The reconstructed image may be displayed in parallel with the analysis interface display, or it may be displayed by switching between them using the display button.

[0038] (Control flow) The processing flow of the information processing method executed by the information processing device 110 of this embodiment will be explained using the flowchart in Figure 7. Figure 7 is a flowchart of the information processing method executed by the information processing device 110. The processing in the information processing device 110 is performed by the CPU 11 functioning as each part. It is assumed that the aspect ratio and interface state have been specified in advance from the user terminal 102, and that multiple images have been generated according to the specified aspect ratio and interface state. The information processing device 110 receives input for each of the generated multiple images and performs the following processing.

[0039] In step S100, the CPU 11 acquires the power spectra of multiple two-dimensional or three-dimensional data representing the state of the material or component.

[0040] In step S102, the CPU 11 performs principal component analysis on multiple power spectra to obtain the principal components of the frequency components of the power spectra and the principal component scores as results of the principal component analysis.

[0041] In step S104, the CPU 11 outputs the principal component analysis results to the user terminal 102. The user terminal 102 displays the principal component analysis results on the analysis interface of the display unit 106.

[0042] In step S106, the CPU 11 receives a specification of a change in the principal component score value for a predetermined principal component and generates reconstructed spectral data (reconstructed power spectrum) when the principal component score value is changed.

[0043] In step S108, the CPU 11 outputs the reconstructed spectral data (reconstructed power spectrum) to the user terminal 102. The user terminal 102 displays the reconstructed spectral data on the analysis interface of the display unit 106.

[0044] In step S110, the CPU 11 generates a reconstructed image, which is an image that visualizes the frequency domain and intensity, based on the principal components, the principal component score values, and the spectral data.

[0045] In step S112, the CPU 11 outputs the generated reconstructed image to the user terminal 102. The user terminal 102 displays the received reconstructed image on the analysis interface of the display unit 106.

[0046] In summary, the information processing device 110 of this embodiment can visualize and analyze material and component analysis data without collecting a large amount of data.

[0047] Furthermore, while the above-described embodiment explained an example where the user manipulates the bars of each PC value in the analysis interface to specify changes in the principal component score values, it is not limited to this. For example, the values ​​of each principal component score may be changed automatically, and the results reflecting these automatic changes may be displayed on the analysis interface of the user terminal 102.

[0048] In addition, the various processes that the CPU 11 reads and executes in the above embodiment may be executed by various processors other than the CPU. Examples of such processors include PLDs (Programmable Logic Devices) such as FPGAs (Field-Programmable Gate Arrays) whose circuit configuration can be changed after manufacturing, and dedicated electrical circuits that are processors with circuit configurations specifically designed to execute specific processes, such as ASICs (Application Specific Integrated Circuits). Furthermore, each of the above processes may be executed by one of these various processors, or by a combination of two or more processors of the same or different types (for example, multiple FPGAs, and a combination of a CPU and an FPGA). More specifically, the hardware structure of these various processors is an electrical circuit that combines circuit elements such as semiconductor elements.

[0049] Furthermore, in the above embodiment, the information processing program was described as being pre-stored (installed) on a computer-readable non-temporary recording medium. For example, the information processing program is pre-stored on ROM 12 or storage 14. However, it is not limited to this, and each program may be provided in a form recorded on a non-temporary recording medium such as CD-ROM (Compact Disc Read Only Memory), DVD-ROM (Digital Versatile Disc Read Only Memory), and USB (Universal Serial Bus) memory. Alternatively, the information processing program may be downloaded from an external device via a network.

[0050] The processing flow described in the above embodiment is just one example, and unnecessary steps may be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose. [Explanation of Symbols]

[0051] 100 Information Processing Systems 102 User terminals 110 Information Processing Device 112 Storage section 120 Image generation unit 122 First acquisition part 124 Second acquisition part 126 Analysis Department 128 Reconstruction part

Claims

1. A first acquisition unit that acquires the power spectrum of multiple two-dimensional or three-dimensional data representing the state of a material or part, A second acquisition unit obtains the principal components of the frequency components of the power spectra and the principal component score values ​​by performing principal component analysis on multiple power spectra, A reconstruction unit generates a reconstructed image, which is an image that visualizes the frequency domain and intensity, based on the principal components, the principal component score values, and the power spectrum. Information processing device including

2. The first acquisition unit generates multiple one-dimensional spectra by integrating the multiple power spectra in the azimuth direction in the case of two-dimensional data, and by integrating them in the azimuth and polar angle directions in the case of three-dimensional data. The information processing apparatus according to claim 1, wherein the second acquisition unit acquires the principal components of the frequency components of the power spectrum and the principal component score values ​​by performing principal component analysis on the plurality of generated one-dimensional spectra.

3. The second acquisition unit outputs the principal components and the principal component scores as the results of principal component analysis to a predetermined user terminal. The information processing apparatus according to claim 1, further comprising an analysis unit that receives a specification of a change in the value of the principal component score for a predetermined principal component and generates reconstructed spectral data when the value of the principal component score is changed.

4. By acquiring the power spectra of multiple two-dimensional or three-dimensional data representing the state of a material or component, By performing principal component analysis on multiple power spectra, the principal components of the frequency components of the power spectra and the principal component scores are obtained. Based on the principal components, the principal component scores, and the power spectrum, a reconstructed image is generated, which is an image that visualizes the frequency domain and intensity. An information processing method in which a computer performs the processing.

5. By acquiring the power spectra of multiple two-dimensional or three-dimensional data representing the state of a material or component, By performing principal component analysis on multiple power spectra, the principal components of the frequency components of the power spectra and the principal component scores are obtained. Based on the principal components, the principal component scores, and the power spectrum, a reconstructed image is generated, which is an image that visualizes the frequency domain and intensity. An information processing program that instructs a computer to perform a task.

Images