Information processing device, information processing method, and information processing program

Abstract

To provide an information processing device that can acquire an image that enables highly accurate analysis of a material and a component.SOLUTION: An information processing device includes: a division unit that accepts input of an image of a state of a material or a component and divides the same into meshes; a conversion unit that converts the divided image into spectral information for each divided mesh; a calculation unit that calculates, on the basis of predetermined reference information and the spectrum information for each of the meshes, a degree of similarity of mesh distributions indicating a degree of similarity for each of the meshes; and a discrimination unit that outputs, on the basis of the degree of similarity of the mesh distributions, a portion of the mesh obtained by dividing the image, according to the degree of similarity.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] Patent Document 1 discloses a technology for dividing a target area into a plurality of partial areas, and for each partial area, comparing the reflectance spectrum within the partial area with a reference spectrum prepared in advance to identify the plant species within the partial area. In addition, this technology excludes spectral data that does not have a reflectance exceeding a predetermined reflectance from the target of the identification process. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-133966 Summary of the Invention [Problem to be solved by the invention]

[0004] Prior art such as Patent Document 1 proposes a technique for identifying images by comparing them with a standard, but no technique has been proposed to date for identifying foreign objects contained in images of materials or parts.

[0005] An object of the present disclosure is to provide an information processing device, an information processing method, and an information processing program that are capable of acquiring images that enable highly accurate analysis of materials and parts. [Means for solving the problem]

[0006] The information processing device according to claim 1 includes a division unit that receives an input of an image of a state of a material or a part and divides it into meshes; a conversion unit that converts the divided image into spectral information for each divided mesh; a calculation unit that calculates a similarity of mesh distribution indicating a degree of similarity for each mesh based on predetermined reference information and the spectral information for each mesh; and a discrimination unit that outputs a portion of the meshes obtained by dividing the image according to the similarity of the mesh distribution. The conversion unit converts the spectral information for each mesh into a predetermined power spectrum distribution and a predetermined size distribution, and the size distribution is shown as a graph of particle size and frequency obtained by applying an EM algorithm, which is a predetermined machine learning method, to the power spectrum distribution. The calculation unit calculates the similarity between the power spectrum distribution and the size distribution as the similarity of the mesh distribution, and the discrimination unit discriminates foreign matter in the image for each of the power spectrum distribution and the size distribution.

[0007] The information processing device described in claim 1 calculates the similarity based on the spectral information for each mesh and outputs the portion of the mesh that corresponds to the similarity, thereby obtaining images that enable highly accurate analysis of materials and parts. In addition, the similarity can be determined from the different characteristics of the power spectrum and size distribution that represent the characteristics of the material.

[0010] Claim 2 In the information processing device according to claim 1, the reference information uses an image average of the image, a predetermined reference image, or a predetermined one-dimensional power spectrum. 2 According to the information processing device described in the above, reference information can be designated depending on the situation in which it is available.

[0011] In addition, in the information processing device according to claim 2, the conversion unit may perform processing within a range corresponding to the reference information on a two-dimensional power spectrum obtained by Fourier transforming the divided image, obtain a distribution of the power spectrum, and apply a predetermined machine learning method to the distribution of the power spectrum to obtain the size distribution.

[0012] Claim 3The information processing method described in the item (1) includes: inputting an image of a photographed state of a material or part, dividing the image into meshes, converting the divided image into spectral information for each divided mesh, calculating a mesh distribution similarity indicating the degree of similarity for each mesh based on predetermined reference information and the spectral information for each mesh, and outputting a portion of the meshes into which the image is divided that corresponds to the similarity based on the mesh distribution similarity. In the conversion, spectral information for each mesh is converted into a predetermined power spectrum distribution and a predetermined size distribution, and the size distribution is shown as a graph of particle size and frequency obtained by applying an EM algorithm, which is a predetermined machine learning method, to the power spectrum distribution. In the calculation of the mesh distribution similarity, the similarity of each of the power spectrum distribution and the size distribution is calculated as the mesh distribution similarity, and in the output of the portion according to the similarity, foreign matter in the image is discriminated for each of the power spectrum distribution and the size distribution. The processing is performed by a computer.

[0013] Claim 4 The information processing program described in the item (1) inputs an image of a photographed state of a material or part, divides it into meshes, converts the divided image into spectral information for each divided mesh, calculates a mesh distribution similarity indicating the degree of similarity for each mesh based on predetermined reference information and the spectral information for each mesh, and outputs a portion of the meshes into which the image is divided that corresponds to the similarity based on the mesh distribution similarity. In the conversion, spectral information for each mesh is converted into a predetermined power spectrum distribution and a predetermined size distribution, and the size distribution is shown as a graph of particle size and frequency obtained by applying an EM algorithm, which is a predetermined machine learning method, to the power spectrum distribution. In the calculation of the mesh distribution similarity, the similarity of each of the power spectrum distribution and the size distribution is calculated as the mesh distribution similarity, and in the output of the portion according to the similarity, foreign matter in the image is discriminated for each of the power spectrum distribution and the size distribution. , and have the computer execute the processing. [Effects of the Invention]

[0014] The techniques disclosed herein make it possible to obtain images that enable highly accurate analysis of materials and parts. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram illustrating a configuration of an information processing system. [Figure 2] FIG. 2 is a block diagram showing the hardware configuration of the information processing device. [Figure 3] FIG. 3 is a diagram showing an example of dividing an image into meshes. [Figure 4] FIG. 4 shows an example of a two-dimensional power spectrum and a one-dimensional power spectrum. [Figure 5] FIG. 5 shows an example of the power spectrum distribution and size distribution. [Figure 6]FIG. 6 is a diagram showing an example of the representation of the power spectrum distribution and the size distribution. [Figure 7] FIG. 7 is a diagram showing an example of a mask image of the discrimination result. [Figure 8] FIG. 8 is a flowchart of an information processing method executed by the information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0016] An embodiment of the present invention will be described. In this embodiment, a method is proposed for identifying outliers, i.e., parts determined to be foreign matter, contained in an image of the state of a material or part, and improving analysis accuracy. By excluding parts of the image that contain foreign matter and utilizing them, it is expected that analysis accuracy will be improved.

[0017] Fig. 1 is a diagram showing the configuration of an information processing system 100. As shown in Fig. 1, in the information processing system 100, an image capturing device 102, a user terminal 104, and an information processing device 110 are connected via a network N such as the Internet.

[0018] The image capturing device 102 is, for example, a microscope, and outputs a captured image. In this embodiment, the image output is an image of material particles. Note that the image may be any image that captures the state of the surface or cross section of a material or component, and a camera or the like may be used as the image capturing device 102.

[0019] The user terminal 104 transmits the image captured by the imaging device 102 to the information processing device 110. The user terminal 104 also specifies reference information and outputs the reference information to the information processing device 110. The reference information is specified by specifying whether to use the image average of the image, a specified reference image, or a specified one-dimensional power spectrum. If the reference image or one-dimensional power spectrum is specified, these are included in the reference information. The image average means that the image average of the captured image is used, and therefore no other reference information is specified. In addition, the user terminal 104 transmits the number of mesh divisions, the pixel sizes of the reference side and the analysis side, and the discrimination threshold to the information processing device 110. The discrimination threshold is a similarity threshold for determining a foreign object, and is prepared for each of the power spectrum distribution and the size distribution.

[0020] Fig. 2 is a block diagram showing the hardware configuration of the information processing device 110. As shown in Fig. 2, the information processing device 110 has a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage 14, an input unit 15, a display unit 16, and a communication interface (I / F) 17. Each component is connected to each other via a bus 19 so as to be able to communicate with each other.

[0021] The CPU 11 is a central processing unit that executes various programs and controls each part. That is, the CPU 11 reads a program from the ROM 12 or the storage 14 and executes the program using the RAM 13 as a work area. The CPU 11 controls each of the above components and performs various arithmetic processing in accordance with the program stored in the ROM 12 or the storage 14. In this embodiment, an information processing program is stored in the ROM 12 or the storage 14.

[0022] The ROM 12 stores various programs and various data. The RAM 13 temporarily stores programs or data as a working area. The storage 14 is configured with a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive) and stores various programs including the operating system and various data.

[0023] The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used to perform various inputs.

[0024] The display unit 16 is, for example, a liquid crystal display, and displays various information. The display unit 16 may function as the input unit 15 by adopting a touch panel system.

[0025] The communication interface 17 is an interface for communicating with other devices such as terminals, etc. For this communication, for example, a wired communication standard such as Ethernet (registered trademark) or FDDI, or a wireless communication standard such as 4G, 5G, or Wi-Fi (registered trademark) is used.

[0026] The following describes each functional configuration of the information processing device 110 in Fig. 1. Functionally, the information processing device 110 is configured to include a storage unit 112, a division unit 120, a conversion unit 122, a calculation unit 124, and a determination unit 126. Each functional configuration is realized by the CPU 11 reading out an information processing program stored in the ROM 12 or storage 14, expanding it in the RAM 13, and executing it.

[0027] The storage unit 112 stores images of the state of materials or parts received from the user terminal 104. The storage unit 112 also stores reference information, the number of mesh divisions, pixel sizes on the reference side and analysis side, and a discrimination threshold.

[0028] The dividing unit 120 inputs an image and divides it into meshes according to the number of divisions of the mesh. Figure 3 is a diagram showing an example of dividing an image into meshes. (a1) is an image of a photograph of material particles, and the image (a1) is divided into meshes as shown in (a2).

[0029] The conversion unit 122 converts the divided image into spectral information for each divided mesh. First, the conversion unit 122 converts the divided image into a two-dimensional power spectrum for each mesh using a Fourier transform. The conversion unit 122 then converts the two-dimensional power spectrum into a one-dimensional power spectrum, which is used as a power spectrum distribution. The conversion unit 122 also obtains a size distribution from the power spectrum distribution. In this way, the conversion unit 122 converts into a predetermined power spectrum distribution and a predetermined size distribution as spectral information for each mesh. The size distribution will be described later.

[0030] Figure 4 shows examples of two-dimensional and one-dimensional power spectra. (b1) is an example of the two-dimensional power spectrum of a certain divided mesh, and shows that the power spectrum spreads from the center, with the center being the origin of spatial frequency. (b2) is an example of a one-dimensional power spectrum (power spectrum distribution) converted from the two-dimensional power spectrum of a certain mesh, and is shown as a graph with the vertical axis representing scattering intensity and the horizontal axis representing frequency.

[0031] The conversion in the conversion unit 122 involves deleting the q range and applying the EM algorithm. Deleting the q range means that when the range of the x-axis of the power spectrum obtained from the divided image differs from the reference information, the range is adjusted to the shorter x-axis, and the range outside the adjusted x-axis is not used for analysis. In this way, the conversion unit 122 processes the range of the two-dimensional power spectrum according to the reference information, and obtains the distribution of the power spectrum, which is a one-dimensional power spectrum. By deleting the q range, the distribution of the power spectrum, which is a one-dimensional power spectrum, can be obtained as spectral information from the two-dimensional power spectrum.

[0032] The EM algorithm is a method for predicting a Gaussian mixture distribution. As spectral information, the size distribution is obtained from the power spectrum distribution using the EM algorithm. In this way, the conversion unit 122 applies the EM algorithm, which is a predetermined machine learning method, to the power spectrum distribution to obtain the size distribution.

[0033] Figure 5 shows an example of the power spectrum distribution and size distribution. (c1) is the power spectrum distribution, and like (b2) above, it is represented as a graph with the vertical axis representing scattering intensity and the horizontal axis representing frequency. (c2) is the size distribution, and is represented as a graph with the vertical axis representing particle size and the horizontal axis representing frequency.

[0034] The calculation unit 124 calculates the similarity of mesh distributions, which indicates the degree of similarity for each mesh, based on predetermined reference information and the spectral information for each mesh. The similarity of mesh distributions is the similarity between the power spectrum distribution and the size distribution. The mesh distribution is expressed as a distinguishable distribution using color gradation or the like. FIG. 6 is a diagram showing an example of the expression of the power spectrum distribution and the size distribution. (d1) is an example of the power spectrum distribution, and (d2) is an example of the size distribution, which show the similarity for each divided mesh. In both examples, areas with low numerical values ​​and dark gradations indicate areas with low similarity. The expression of the power spectrum distribution and the size distribution is merely an example, and color patterns or the like may also be used.

[0035] The discrimination unit 126 discriminates foreign matter in the image for each of the power spectrum distribution and the size distribution based on the similarity of the mesh distribution and a discrimination threshold. Foreign matter in the image is assumed to be foreign matter particles that are outliers of the materials or components present in the image, or foreign matter particles due to objects other than the materials or raw materials. The discrimination unit 126 outputs mask images in which portions of the power spectrum distribution and the size distribution that have been discriminated as foreign matter are masked. FIG. 7 shows an example of a mask image of the discrimination result. (e1) is a mask image of the discrimination result for the power spectrum distribution, and (e2) is a mask image of the discrimination result for the size distribution. Discrimination results are obtained for each mesh in the mask image, and meshes that have been discriminated as foreign matter are masked in black. Because the power spectrum distribution and the size distribution focus on different features, differences arise in the masked portions.

[0036] (Flow of Control) The flow of processing as an information processing method executed by the information processing device 110 of this embodiment will be described with reference to the flowchart in Fig. 8. Fig. 8 is a flowchart of the information processing method executed by the information processing device 110. The processing in the information processing device 110 is executed by the CPU 11 functioning as each unit.

[0037] In step S100, the CPU 11 acquires an image of the state of the material or part, and reference information.

[0038] In step S102, CPU 11 executes processing according to the reference information to create a reference for similarity. If the reference information is an image average, an image average is created from the acquired images and used as the reference for similarity. If the reference information is a reference image, the reference image used as the reference for similarity is read and spectral information is created. The spectral information can be created by performing processing similar to steps S106 to S110, which will be described later. If the reference information is a one-dimensional power spectrum, the one-dimensional power spectrum used as the reference for similarity is read as spectral information.

[0039] In step S104, the CPU 11 inputs an image and divides it into meshes according to the number of divisions into meshes.

[0040] In step S106, the CPU 11 performs a Fourier transform on the divided image for each divided mesh, converting it into a two-dimensional power spectrum.

[0041] In step S108, the CPU 11 performs processing on each two-dimensional power spectrum of the mesh within a range corresponding to the reference information, and converts it into a power spectrum distribution that is a one-dimensional power spectrum.

[0042] In step S110, the CPU 11 converts the power spectrum distribution of each mesh into a size distribution using a predetermined machine learning method. As a result, the image for each mesh is converted into a power spectrum distribution and a size distribution as spectral information for each mesh.

[0043] In step S112, the CPU 11 calculates the mesh distribution similarity, which indicates the degree of similarity for each mesh, based on the similarity standard created from the reference information in step S102 and the spectrum information for each mesh. The mesh distribution similarity is the similarity between the power spectrum distribution and the size distribution.

[0044] In step S114, the CPU 11 determines foreign matter in the image for each of the power spectrum distribution and the size distribution based on the mesh distribution.

[0045] In step S116, the CPU 11 outputs a mask image in which foreign matter in the image is discriminated for each mesh for each of the power spectrum distribution and the size distribution.

[0046] As described above, the information processing device 110 of this embodiment can acquire images that enable highly accurate analysis of materials and parts.

[0047] In the above embodiment, various processes executed by the CPU 11 after reading software (programs) may be executed by various processors other than a CPU. Examples of such processors include programmable logic devices (PLDs) (such as field-programmable gate arrays (FPGAs)) whose circuit configuration can be changed after fabrication, and dedicated electrical circuits, such as application-specific integrated circuits (ASICs), which are processors with circuit configurations specifically designed to execute specific processes. Each of the above processes may be executed by one of these various processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of a CPU and an FPGA). The hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor devices.

[0048] In the above embodiment, the information processing program has been described as being pre-stored (installed) in a computer-readable non-transitory recording medium. For example, the information processing program is pre-stored in the ROM 12 or the storage 14. However, the present invention is not limited to this. Each program may be provided in a form recorded on a non-transitory recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. The information processing program may also be downloaded from an external device via a network.

[0049] The processing flow described in the above embodiment is an example, and unnecessary steps may be deleted, new steps may be added, or the processing order may be changed within the scope of the gist of the invention. [Explanation of symbols]

[0050] 100 Information Processing Systems 102 Imaging equipment 104 User terminal 110 Information processing equipment 112 Storage section 120 Division 122 Conversion Unit 124 Calculation Unit 126 Discrimination part

Claims

1. a division unit that receives an image of the state of a material or part and divides it into meshes; a conversion unit that converts the divided image into spectral information for each divided mesh; a calculation unit that calculates a mesh distribution similarity indicating a degree of similarity for each mesh based on predetermined reference information and spectrum information for each mesh; a discrimination unit that outputs a portion of the meshes obtained by dividing the image according to the similarity of the mesh distribution; Including, The conversion unit converts the spectral information for each mesh into a predetermined power spectrum distribution and a predetermined size distribution, The size distribution is shown as a graph of particle size and frequency obtained by applying an EM algorithm, which is a predetermined machine learning method, to the distribution of the power spectrum; the calculation unit calculates a similarity between the power spectrum distribution and the size distribution as the similarity between the mesh distributions, The information processing device, wherein the discrimination unit discriminates foreign matter in the image for each of the power spectrum distribution and the size distribution.

2. The information processing apparatus according to claim 1 , wherein the reference information uses an image average of the image, a predetermined reference image, or a predetermined one-dimensional power spectrum.

3. Input an image of the state of the material or part, divide it into meshes, The divided image is converted into spectral information for each divided mesh; Calculating a mesh distribution similarity indicating a degree of similarity for each mesh based on predetermined reference information and the spectrum information for each mesh; outputting a portion of the meshes obtained by dividing the image according to the similarity of the mesh distribution; In the conversion, the spectral information for each mesh is converted into a predetermined power spectrum distribution and a predetermined size distribution, The size distribution is shown as a graph of particle size and frequency obtained by applying an EM algorithm, which is a predetermined machine learning method, to the distribution of the power spectrum; In the calculation of the mesh distribution similarity, a similarity between the power spectrum distribution and the size distribution is calculated as the mesh distribution similarity; In the output of the portion according to the similarity, a foreign object in the image is identified for each of the power spectrum distribution and the size distribution. An information processing method in which processing is performed by a computer.

4. Input an image of the state of the material or part, divide it into meshes, The divided image is converted into spectral information for each divided mesh; Calculating a mesh distribution similarity indicating a degree of similarity for each mesh based on predetermined reference information and the spectrum information for each mesh; outputting a portion of the meshes obtained by dividing the image according to the similarity of the mesh distribution; In the conversion, the spectral information for each mesh is converted into a predetermined power spectrum distribution and a predetermined size distribution, The size distribution is shown as a graph of particle size and frequency obtained by applying an EM algorithm, which is a predetermined machine learning method, to the distribution of the power spectrum; In the calculation of the mesh distribution similarity, a similarity between the power spectrum distribution and the size distribution is calculated as the mesh distribution similarity; In the output of the portion according to the similarity, a foreign object in the image is identified for each of the power spectrum distribution and the size distribution. An information processing program that causes a computer to execute a process.

Images