Image generating device, rubber composition blending estimation device, and learning device

Abstract

To provide a technique for estimating the correlation between the structure and blend of a rubber composition.SOLUTION: An image generation device includes a data acquisition part and an image generation part. The data acquisition part acquires blend data representing the blend of a material constituting the rubber composition. The image generation part inputs the acquired blend data to a machine learning model together with a noise and generates a simulation microscopic image simulating a microscopic image obtained by imaging the rubber composition by a microscope as the output of the machine learning model.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an image generating device, a rubber composition blending estimation device, and a learning device. [Background technology]

[0002] Japanese Patent No. 6609387 (Patent Document 1) discloses a method for estimating the properties of a rubber material from an index showing characteristics of an image of the rubber material captured by a microscope and the compounding of the rubber material. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6609387 Summary of the Invention [Problem to be solved by the invention]

[0004] According to Patent Document 1, the structure of a rubber material (represented in an image), the composition of the rubber material, and the properties of the rubber material are correlated with each other. However, Patent Document 1 does not disclose a method for estimating the correlation between the structure and composition of a rubber material.

[0005] The present invention has been made to solve such problems, and an object of the present invention is to provide a technique for estimating the correlation between the structure and compounding of a rubber composition depicted in an image. [Means for solving the problem]

[0006] According to one aspect of the present invention, there is provided an image generating device including a data acquiring unit and an image generating unit. The data acquiring unit acquires blending data representing a blending of materials constituting a rubber composition. The image generating unit inputs the acquired blending data together with noise into a machine learning model, and generates, as an output of the machine learning model, a simulated microscope image that simulates a microscope image of the rubber composition captured by a microscope.

[0007] According to the image generating device, when the compounding data of the rubber composition is given, a simulated microscope image for estimating the structure of the rubber composition is obtained, which makes it possible to predict the structure of the rubber composition obtained from the compounding data.

[0008] In the image generating apparatus, the microscope may be an electron microscope.

[0009] A rubber composition blending estimation device according to one aspect of the present invention includes an image acquisition unit and a blending estimation unit. The image acquisition unit acquires a microscopic image of a rubber composition captured by a microscope or a simulated microscopic image that simulates the image. The blending estimation unit inputs the acquired microscopic image or the acquired simulated microscopic image to a machine learning model and estimates the blending of materials that constitute the rubber composition as an output of the machine learning model.

[0010] According to the rubber composition blending estimation device, the blending of materials constituting a rubber composition can be estimated based on an image that represents the structure of a rubber composition whose material blending is unknown or an image that predicts the structure of the rubber composition.

[0011] In the above blending estimation device, the microscope may be an electron microscope.

[0012] A method for generating a trained model according to one aspect of the present invention comprises the following steps. A step of preparing learning data in which a microscopic image of a rubber composition taken by a microscope is combined with formulation data representing the formulation of materials constituting the rubber composition. A step of training a first machine learning model that, when the blending data and noise are input, generates a simulated microscope image that simulates the microscope image based on the input blending data and noise, and a second machine learning model that, when the microscope image or the simulated microscope image is input, outputs an identification result indicating whether the input image is the microscope image or not. The learning step alternately repeats the following steps. A step of training the second machine learning model so that, when the microscope image or the simulated microscope image is input to the second machine learning model, the classification result output by the second machine learning model has a high probability of being correct. A step of training the first machine learning model so that, when the simulated microscope image is input to the second machine learning model, the probability that the classification result output by the second machine learning model is correct is reduced.

[0013] In the method for generating a trained model, the first machine learning model may include a generation layer that generates data larger than the input data by upsampling the input data and outputs the data as the simulated microscope image of a predetermined size. The second machine learning model may include an identification layer that is paired with the generation layer and that outputs an identification result indicating whether the input image is the microscope image or not when the simulated microscope image output by the generation layer or the microscope image having the same size as the simulated microscope image is input. The method for generating a trained model may further include a step of adding one new generation layer to the first machine learning model after the training step, and adding one new identification layer to the second machine learning model that is paired with the added generation layer, and a step of further training the first machine learning model and the second machine learning model after the adding step, and the adding step and the further training step may be alternately repeated, and the size of the simulated microscope image output by the added generation layer may be gradually increased each time the adding step is repeated.

[0014] In the above trained model generation method, the second machine learning model may further output data estimating blending data corresponding to the input microscopic image or the simulated microscopic image. The step of training the second machine learning model may include a step of training the second machine learning model such that, when the microscopic image is input to the second machine learning model, a difference between the data output by the second machine learning model and the blending data becomes small. The step of training the first machine learning model may include a step of training the first machine learning model such that, when the simulated microscopic image is input to the second machine learning model, a difference between the data output by the second machine learning model and the blending data input to the first machine learning model becomes small.

[0015] In the above-mentioned method for generating a trained model, the microscope may be an electron microscope. Effect of the Invention

[0016] According to the present invention, there is provided a technique for estimating the correlation between the structure and compounding of a rubber composition. [Brief description of the drawings]

[0017] [Figure 1] 3 is an example of an image of a rubber composition taken by an electron microscope. [Diagram 2] FIG. 2 is a conceptual diagram of a machine learning model of the learning device according to the first embodiment. [Diagram 3] FIG. 2 is a block diagram showing the electrical configuration of the learning device according to the first embodiment. [Figure 4] 11 is a flowchart showing the flow of the learning process of the second machine learning model. [Diagram 5] 1 is a flowchart showing the flow of the learning process of the first machine learning model. [Figure 6] FIG. 1 is a block diagram showing the electrical configuration of an image generating apparatus according to a first embodiment. [Figure 7] 5 is a flowchart showing the flow of image generation processing according to the first embodiment. [Figure 8]FIG. 1 is a block diagram showing an electrical configuration of an estimation device according to a first embodiment. [Figure 9] 5 is a flowchart showing a flow of an estimation process according to the first embodiment. [Figure 10] FIG. 11 is a block diagram showing the electrical configuration of a learning device according to a second embodiment. [Figure 11] FIG. 11 is a conceptual diagram of a machine learning model of a learning device according to a second embodiment. [Figure 12] A diagram explaining the configuration of the third machine learning model. [Figure 13] 10 is a flowchart showing the flow of a learning process according to the second embodiment. [Figure 14] FIG. 11 is a diagram for explaining a learning method according to the second embodiment. [Figure 15] FIG. 11 is a block diagram showing the electrical configuration of an image generating apparatus according to a second embodiment. [Figure 16] 13 shows examples of images acquired in an experiment according to the embodiment and examples of images generated. [Figure 17] 11 is a histogram showing experimental results according to an embodiment. [Figure 18] 13 shows examples of images acquired in an experiment according to another embodiment and examples of images generated. [Figure 19] 13 is an example of an image generated in an experiment according to another embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] <1. Overview> A rubber composition is a polymeric compound having elasticity, and is typically produced by kneading together a number of ingredients. Examples of ingredients include monomers (butadiene, styrene, etc.), fillers (silica, carbon, etc.), and crosslinking agents. In developing a rubber composition, the ingredients are selected from a wide variety of raw materials according to the purpose, the amount of ingredients is determined, and a prototype is produced, which is repeated, and thus the development costs tend to be high. For this reason, there is a growing need for technology to develop rubber compositions more efficiently.

[0019] Incidentally, features related to the structure of a rubber composition can be extracted from a microscopic image of a rubber composition taken by a microscope. FIG. 1 is a scanning electron microscope image of the inside of a rubber composition taken at a magnification of 20,000 times. In the image of FIG. 1, a specific compound can be identified by the light and dark represented in gray scale. For example, carbon aggregates appear in the image as relatively dark areas, while silica aggregates appear as relatively bright areas. That is, a microscopic image of a rubber composition contains information expressing the structure of the rubber composition, such as the size of filler aggregates, the distance between filler aggregates, and the distribution (partition) of the filler in the polymer phase.

[0020] As a result of intensive research, the present inventors have invented a technique for estimating the correlation between the structure of a rubber composition shown in an image and compounding data representing the compounding of materials constituting the rubber composition. More specifically, the present inventors have conceived of training a machine learning model, generating an image simulating a microscopic image of a rubber composition from the compounding data of the rubber composition, and estimating the compounding data of the rubber composition from the microscopic image of the rubber composition. Hereinafter, an image generating device, a rubber composition compounding estimation device, and a learning device according to one embodiment of the present invention will be described.

[0021] [A. First embodiment] <2-1. Learning device> The learning device 1 according to the first embodiment of the present disclosure is a device that trains AC-GAN (auxiliary classifier generative adversarial network), which is one of conditional image generation models. FIG. 2 is a conceptual diagram showing a machine learning model according to the present embodiment. AC-GAN includes two machine learning models, a first machine learning model 200 and a second machine learning model 300. The learning device 1 competitively trains these machine learning models to generate a trained model 200A that generates a simulated microscope image, which is a new image similar to a microscope image of a rubber composition, and a trained model 300A that estimates compounding data of a rubber composition from an input image. The generated trained model 200A is implemented in an image generation device 2 (hereinafter, sometimes simply referred to as "generation device 2") described later, and the generated trained model 300A is implemented in a rubber composition compounding estimation device 3 (hereinafter, sometimes simply referred to as "estimation device 3") described later.

[0022] 3 is a block diagram showing the electrical configuration of the learning device 1. The learning device 1 of this embodiment is a general-purpose computer in terms of hardware. The learning device 1 includes a storage unit 10, a control unit 11, a communication interface 12, an external interface 13, an input device 14, and an output device 15. These units 10 to 15 are connected to each other via a bus line and can communicate with each other.

[0023] The storage unit 10 is composed of a non-volatile storage device such as a hard disk or a flash memory, and stores training data 100 prepared for training AC-GAN. The storage unit 10 also stores a training program 101 for training AC-GAN. The training data 100 and the training program 101 can be acquired from a storage medium 7 such as a USB memory, a network 8 to which the training device 1 is connected, or the like. The storage unit 10 also stores parameters 102 of the first machine learning model 200 and the second machine learning model 300 that are updated in the process of training AC-GAN.

[0024] The control unit 11 is composed of a processor such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), a RAM (Random Access Memory), and a ROM (Read only memory). The learning program 101 may be stored in the ROM. The control unit 11 operates as a learning processing unit 111 and a storage processing unit 112 by reading and executing the learning program 101 from the storage unit 10. The operations of the learning processing unit 111 and the storage processing unit 112 will be described later.

[0025] The communication interface 12 is composed of, for example, a wired LAN module, a wireless LAN module, etc. The communication interface 12 is an interface for performing wired or wireless data communication with an external device via the network 8. The external interface 13 is composed of, for example, a USB (Universal Serial Bus) port or a dedicated port. The external interface 13 is an interface for establishing a connection that enables communication with an external device.

[0026] The input device 14 is composed of a mouse, a keyboard, an audio microphone, operation buttons, etc. The input device 14 accepts operations from a user to the control unit 11. The output device 15 is composed of a display, a speaker, etc., and outputs information. The input device 14 and the output device 15 may be integrally configured using a touch panel display, etc.

[0027] <2-2. Learning process> 4 and 5 are flowcharts showing the flow of the learning process performed by the learning device 1.

[0028] The training data 100 is data derived from an actual rubber composition as a sample. More specifically, the training data 100 is a large number of data sets that combine microscopic images of the rubber composition captured by a microscope with compounding data that indicates the compounding ratio of the materials in the rubber composition. The microscopic images are images of the rubber composition as a sample captured by a microscope, and the imaging conditions such as the magnification and the electron microscope mode are common to each sample. The i-th training sample (i=1, 2, ..., N image ;N image is the number of training samples)

number

number

[0029] In this embodiment, the microscope is a scanning electron microscope (SEM). However, the microscope is not limited thereto, and can be appropriately selected from other types of electron microscopes (TEM, STEM, etc.) and optical microscopes. In addition, an example of the resolution of the microscope is preferably in the range of 135 nm / pixel (length of an object imaged per pixel). Furthermore, an example of the magnification for imaging the rubber composition is preferably in the range of 2500 to 40000 times. The part of the rubber composition imaged by the microscope is not particularly limited, but it is preferably an inner part rather than a surface part that is a boundary with the external space.

[0030] There is one set of compounding data for each type of sample. Compounding data includes, for example, monomers, fillers, and other additives. com This is data that represents the ratio of each type of material. The composition vector, whose elements are the ratios of the composition data, is

number

[0031] Fig. 4 is a flowchart showing the flow of the learning process of the second machine learning model 300. The learning processing unit 111 of this embodiment first performs learning of the second machine learning model 300 while keeping the parameters of the first machine learning model 200 constant. When the learning program 101 is executed by a user's instruction and an instruction from the user to start learning is received via the input device 14, the learning process shown in Fig. 4 starts.

[0032] In step S1, the learning processing unit 111 calculates the combination vector y fake With uniform distribution p z The random vector z according to the formula (101) is input to the first machine learning model 200. The random vector z is data generated from random numbers, that is, a vector whose elements are noise. The random numbers may be generated by the learning processing unit 111 each time, or may be generated by a program other than the learning program 101 and acquired by the learning processing unit 111. fake is input to condition the first machine learning model 200. fake is the blend vector y i In step S1 of the present embodiment, the blend vector y fake As a blend vector y derived from training data 100 i is input, but vectors that are not derived from the training data 100 are also mixed vector y fake It can be entered as:

[0033] The first machine learning model 200 is a mixture vector y fakeand a random vector z are input, the model is configured to generate an image using these as feature quantities, and includes parameters that are updated by machine learning. In this embodiment, the first machine learning model 200 is a neural network, and the above-mentioned parameters include the weights of connections between neurons, the thresholds of each neuron, etc. The first machine learning model 200 generates an image using the input combination vector y fake and the random vector z is used to map to the data space.

number

[0034] In step S2, the learning processing unit 111

number

[0035] The second machine learning model 300 is a model configured to output, when an image is input, a discrimination result indicating whether the input image is derived from learning data (real) or not (fake), and includes parameters updated by machine learning. The discrimination result output by the second machine learning model 300 may be a probability indicating whether the image is "real" or "fake", or may be data indicating whether the image is "real" or "fake" determined based on this probability. In this embodiment, the second machine learning model 300 is a neural network, and the above-mentioned parameters include the weight of the connection between each neuron and the threshold value of each neuron. The second machine learning model 300 of this embodiment outputs a probability indicating whether each patch is "real" or "fake" for the input patch group. In addition, the second machine learning model 300 of this embodiment is configured to output data estimated as the combination data of the input image from the feature amount of the patch group at the same time as the discrimination result. Specifically, when an image is input, the second machine learning model 300:

number

number

[0036] In step S3, the learning processing unit 111 calculates the output classification result and y #pred(P) is compared with the correct answer as to whether the input is "real" or "fake" and with the combination data included in the training data 100, and the parameters of the second machine learning model 300 are adjusted. More specifically, the learning processing unit 111 adjusts the parameters of the second machine learning model 300 so that the probability of the correct answer being output by the second machine learning model 300 is increased, and at the same time, the difference between the data output by the second machine learning model 300 and the combination data included in the training data 100 is reduced. In this embodiment, the learning processing unit 111 adjusts the parameters of the second machine learning model 300 using a loss function L expressed by the following equations (1) and (2): C and the loss function L S Sum of (-L C +L S The parameters of the second machine learning model 300 are adjusted so that ) is maximized.

number

number

number

[0037] In step S4, the learning processing unit 111 determines whether or not the learning of the second machine learning model 300 is completed. The learning processing unit 111 may determine that the learning is completed when the repetition of steps S1 to S3 reaches a predetermined number of times, or may determine that the learning is completed when the repetition of steps S1 to S3 reaches a predetermined number of times. C +L S) reaches the target value, it may be determined that learning is complete. If the learning processing unit 111 determines that learning is not complete, it further repeats steps S1 to S3. In this way, the learning processing unit 111 optimizes the parameters of the second machine learning model 300. The saving processing unit 112 overwrites and saves the latest parameters as parameters 102 in the memory unit 10 each time one learning is performed and the parameters of the second machine learning model 300 are adjusted in step S3.

[0038] When the learning processing unit 111 determines that the learning of the second machine learning model 300 is completed, it ends the learning process of the second machine learning model 300. Next, the learning processing unit 111 starts learning of the first machine learning model 200 with the parameters of the second machine learning model 300 kept constant. FIG. 5 is a flowchart showing the flow of the learning process of the first machine learning model 200. As the parameters of the second machine learning model 300 at this time, parameters optimized in the most recent learning of the second machine learning model 300 can be used.

[0039] In step S21, the learning processing unit 111 calculates the combination vector y fake The random vector z is input to the first machine learning model 200 together with

number

[0040] In step S22, the learning processing unit 111 inputs the simulated microscope image generated in step S21 to the second machine learning model 300. The second machine learning model 300 calculates a probability distribution D(S|P) and a blending vector y pred Output (P).

[0041] In step S23, the learning processing unit 111 decreases the probability of the correct answer of the classification result of the second machine learning model 300 and at the same time, fake and the blend vector y pred (P) is reduced. In other words, the learning processing unit 111 trains the first machine learning model 200 so that the simulated microscope image is more similar to the microscope image and the second machine learning model 300 can more easily recognize the simulated microscope image as the original microscope image. In addition, the parameters of the first machine learning model 200 are adjusted so that the blending data estimated by the second machine learning model 300 from the simulated microscope image is closer to the original blending data. In this embodiment, the learning processing unit 111 adjusts the parameters of the first machine learning model 200 so that the difference between -(L C +L S The parameters of the first machine learning model 200 are adjusted so that ) is maximized.

[0042] In step S24, the learning processing unit 111 judges whether or not the learning of the first machine learning model 200 is completed. The learning processing unit 111 may judge that the learning is completed when the repetition of steps S21 to S23 reaches a predetermined number of times, or may judge that the learning is completed when -(L C +L S ) reaches a target value, it may be determined that learning is complete. If the learning processing unit 111 determines that learning is not complete, it further repeats steps S21 to S23. In this way, the learning processing unit 111 optimizes the parameters of the first machine learning model 200. The saving processing unit 112 overwrites and saves the latest parameters as parameters 102 in the storage unit 10 every time the parameters of the first machine learning model 200 are adjusted in step S23.

[0043] When the learning processing unit 111 determines in step S24 that the learning of the first machine learning model 200 is completed, the learning processing unit 111 ends the learning process of the first machine learning model 200. Then, the learning of the second machine learning model 300 described above is started again. In this way, the learning processing unit 111 alternately repeats the learning of the second machine learning model 300 and the learning of the first machine learning model 200. The number of times the learning is repeated may be a predetermined number of times. In this way, when the blending data is input, the trained model 200A that outputs a simulated microscope image that is paired with the blending data is obtained. In addition, when a microscope image or a simulated microscope image is input, the trained model 300A that outputs a discrimination result as to whether or not the image is a microscope image and outputs data that estimates the blending data that is paired with the input image is obtained.

[0044] <2-3. Image generation device> The configuration of the generation device 2 using the trained model 200A and the processing performed by the generation device 2 will be described below. FIG. 6 is a block diagram showing the electrical configuration of the image generation device 2. The generation device 2 of this embodiment is a general-purpose computer in terms of hardware. The generation device 2 includes a storage unit 20, a control unit 21, a communication interface 22, an external interface 23, an input device 24, and an output device 25. These units 20 to 25 are connected to each other via a bus line and can communicate with each other.

[0045] The storage unit 20 is composed of a non-volatile storage device such as a hard disk or a flash memory, and stores a program 201 for operating the generation device 2. The program 201 includes a trained model 200A including parameters trained by the learning device 1. The program 201 can be acquired from a storage medium 7 such as a USB memory, a network 8, or the like.

[0046] The control unit 21 is composed of a processor such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), a RAM (Random Access Memory), and a ROM (Read only memory). The program 201 may be stored in the ROM. The control unit 21 reads out the program 201 from the storage unit 20 and executes it to operate as a data acquisition unit 210 and an image generation unit 211. The operation of each unit will be described later.

[0047] The communication interface 22 is composed of, for example, a wired LAN module, a wireless LAN module, etc. The communication interface 22 is an interface for performing wired or wireless data communication with an external device via the network 8. The external interface 23 is composed of, for example, a USB (Universal Serial Bus) port or a dedicated port. The external interface 23 is an interface for establishing a connection that enables communication with an external device.

[0048] The input device 24 is composed of a mouse, a keyboard, an audio microphone, operation buttons, etc. The input device 24 accepts operations from a user to the control unit 21. The output device 25 is composed of a display, a speaker, etc., and outputs information. The input device 24 and the output device 25 may be integrally configured by a touch panel display, etc.

[0049] 7 is a flowchart showing the flow of the image generation process performed by the generation device 2. When the generation device 2 receives an instruction to start the image generation process from the user via the input device 24, the generation device 2 starts the image generation process.

[0050] In step S41, the data acquisition unit 210 acquires compounding data representing the compounding of materials constituting the rubber composition. The compounding data is data representing the compounding ratio of each of a predetermined type of material in the range of [0, 1], and can also be treated as a vector. The compounding data may be acquired via the storage medium 7, via the network 8, or directly from an external device connected so as to be able to communicate with the device.

[0051] In the next step S42, the image generating unit 211 inputs the acquired compounding data together with a random vector representing noise to the trained model 200A. This random vector that follows a uniform distribution may be generated by the image generating unit 211 each time, or may be generated by a program other than the program 201 and acquired by the image generating unit 211 each time. The trained model 200A regards the compounding data as a compounding vector and maps it together with the random vector into a data space. As a result, in step S43, a simulated microscope image that simulates a microscope image of a rubber composition captured by a microscope is output.

[0052] The output image can be used, for example, to predict the structure of a rubber composition whose compounding data is known in advance. The output image can also be used, for example, as data for training a machine learning model that estimates the properties of a rubber composition based on a microscopic image of the rubber composition. The output image can also be used, for example, to predict the aging of the structure of the rubber composition. For example, when microscopic images of a rubber composition for each elapsed time from production are available, this is used to prepare multiple learning data 100 for each different elapsed time, and each of these is trained by the learning device 1 to generate multiple trained models 200A. Each of the multiple trained models 200A thus created outputs a simulated microscopic image corresponding to a predetermined elapsed time.

[0053] <2-4. Estimation device> Hereinafter, the configuration of the rubber composition estimation device 3 using the trained model 300A and the processing performed by the estimation device 3 will be described. FIG. 8 is a block diagram showing the electrical configuration of the estimation device 3. The estimation device 3 of this embodiment is a general-purpose computer as hardware. The estimation device 3 includes a storage unit 30, a control unit 31, a communication interface 32, an external interface 33, an input device 34, and an output device 35. These units 30 to 35 are connected to each other via a bus line and can communicate with each other.

[0054] The storage unit 30 is composed of a non-volatile storage device such as a hard disk or a flash memory, and stores a program 301 for operating the estimation device 3. The program 301 includes a trained model 300A including parameters trained by the learning device 1. The program 301 can be acquired from a storage medium 7 such as a USB memory, a network 8, or the like.

[0055] The control unit 31 is composed of a processor such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), a RAM (Random Access Memory), and a ROM (Read only memory). The program 301 may be stored in the ROM. The control unit 31 reads out the program 301 from the storage unit 30 and executes it to operate as an image acquisition unit 310 and a blending estimation unit 311. The operation of each unit will be described later.

[0056] The communication interface 32 is composed of, for example, a wired LAN module, a wireless LAN module, etc. The communication interface 32 is an interface for performing wired or wireless data communication with an external device via the network 8. The external interface 33 is composed of, for example, a USB (Universal Serial Bus) port or a dedicated port. The external interface 33 is an interface for establishing a connection that enables communication with an external device.

[0057] The input device 34 is composed of a mouse, a keyboard, an audio microphone, operation buttons, etc. The input device 34 accepts operations from a user to the control unit 31. The output device 35 is composed of a display, a speaker, etc., and outputs information. The input device 34 and the output device 35 may be integrally configured using a touch panel display, etc.

[0058] 9 is a flowchart showing the flow of the blend estimation process performed by the estimation device 3. When the estimation device 3 receives an instruction to start the blend estimation process from the user via the input device 34, the estimation device 3 starts the blend estimation process.

[0059] In step S51, the image acquisition unit 310 acquires an input image to be input to the trained model 300A. The input image may be a microscopic image of a rubber composition captured by a microscope, or an image similar to a microscopic image, for example, a simulated microscopic image that simulates a microscopic image. The input may be acquired via the storage medium 7, via the network 8, or directly from an external device connected to the estimation device 3 so as to be able to communicate with the estimation device 3. If the input image is a microscopic image, the captured image may be imported into the estimation device 3 from a microscope connected to the estimation device 3 so as to be able to communicate data with the estimation device 3.

[0060] In the next step S52, the blending estimation unit 311 inputs the acquired input image to the trained model 300A. In step S53, the trained model 300A estimates the blending of materials constituting the rubber composition based on the input image. That is, the trained model 300A outputs blending data paired with the input image.

[0061] The formulation data output by the estimation device 3 can be used to estimate the materials and formulation of a rubber composition, for example, whose constituent materials and formulation are unknown.

[0062] [B. Second embodiment] <3-1. Learning device> Hereinafter, a learning device 1A according to a second embodiment of the present disclosure will be described with reference to FIGS. 10 to 14. FIG. 10 is a block diagram showing an electrical configuration of the learning device 1A. The learning device 1 according to the first embodiment is configured to learn a machine learning model by AC-GAN, but the learning device 1A according to this embodiment is configured to learn a machine learning model by "Conditional Style Generative Adversarial Network: StyleGAN", which is one of conditional image generation models. For this reason, the learning device 1A is a general-purpose personal computer having a similar configuration to the learning device 1 in terms of hardware, but the learning program 101A and parameters 102A stored in the storage unit 10 are different from those of the learning device 1. The learning program 101A incorporates the machine learning model (StyleGAN) according to this embodiment, and the parameters 102A are updated according to the learning result and stored in the storage unit 10. In the following description, the configurations common to the learning device 1 are denoted by common reference numerals and description is omitted, and the configuration of the learning device 1A different from that of the learning device 1, that is, the configuration of the machine learning model and its learning process, will be described.

[0063] <3-2. Learning process> FIG. 11 is a diagram for explaining an outline of the machine learning model according to the present embodiment. In this machine learning model, similar to the machine learning model according to the first embodiment, competitive learning is performed between the third machine learning model 400 that generates an image and the fourth machine learning model 500 that identifies the authenticity of the image. However, in learning of StyleGAN, the third machine learning model 400 and the fourth machine learning model 500 are learned so that the resolution of the generated image is gradually increased from low resolution to medium resolution and from medium resolution to high resolution. This allows learning to be performed by gradually focusing on detailed information such as position and pattern from rough information of the image such as color, and finally, a high-resolution image having a high similarity to the image of the learning data can be generated. The trained third machine learning model 400 (hereinafter sometimes referred to as the "trained model 400A") can be implemented in a general-purpose computer or the like, similar to the trained model 200A, to configure the image generating device 2A (hereinafter sometimes simply referred to as the "generating device 2A").

[0064] The configuration of StyleGAN and the flow of the learning process performed by the learning device 1A will be described below with reference to Fig. 11 to Fig. 14. Fig. 13 is a flowchart showing the flow of the learning process performed by the learning device 1.

[0065] The learning data 100A of this embodiment is a microscopic image X i is pre-divided into H×W patches

number

[0066] 12 is a diagram illustrating the configuration of a third machine learning model 400 according to this embodiment. As shown in FIG. 12, the third machine learning model 400 includes a projection network 401 and a generation network 402. The projection network 401 derives feature quantities from input noise and rubber compounding data.

number

[0067] In step ST1 of the learning process shown in FIG. 13, the learning processing unit 111 inputs the combination vector y fake and the random vector z~N(μ,σ 2) is input, and the feature value w, which is the output from the projection network 401, is derived.

[0068] In step ST2, the learning processing unit 111 converts the feature quantity w output from the projection network 401 into two parameters using an affine transformation.

number

[0069] In step ST3, the learning processing unit 111 calculates the parameter w s , w b and the random vector z map ~N(μ,σ 2 ) into the following equation, the feature quantity w Ada where "x (bold italics)" in the formula is the random vector z map are intermediate feature outputs obtained by inputting the above to the generative network 402 at each stage of learning, which will be described later.

number

[0070] Feature quantity calculated in step ST3

number

[0071] In step ST4, the learning processing unit 111 calculates a random vector zmap ~N(μ,σ 2 ) and feature quantity w Ada is input to the generation network 402, and the simulated microscope image P fake Here, the generator network 402 outputs the same random vector z map ~N(μ,σ 2 ) and feature quantity w Ada The generative network 402 is configured to generate an image of higher resolution, i.e., a larger size, from the input of . The generative network 402 is first configured as a generator 403 that outputs a small-sized image. When the generator 403 is able to output an image closer to the same-sized image of the training data 100A through training, a further layer of a machine learning model is added to the generator 403, resulting in the generator 403 outputting an image of one size larger. This generator 403 is trained again so that it can output an image closer to the same-sized image of the training data 100A. By repeating this procedure, a generative network 402 capable of generating images of high resolution is finally generated. Hereinafter, the layer added each time training is performed is referred to as a "generative layer."

[0072] The generator 403 is not particularly limited in structure as long as it is a machine learning model that generates and outputs an image having a predetermined size from input data. The generator 403 of this embodiment has one or more upsampling layers 4030. The upsampling layer 4030 generates data larger in size than the input data by performing transposition convolution. Parameters such as the transposition convolution are included in the parameters 102A and are adjusted by learning.

[0073] Random vector z map and feature quantity w Ada is input to the generator 403. More specifically, a random vector z map and feature quantity w Ada is input to the generator 403. That is, the simulated microscope image P fakeThe style information to be reflected in the image is captured during the generation process of the image at each resolution, and characterizes the different properties of the information contained in the image at each resolution. This allows the high-quality simulated microscope image P fake It is possible to generate a trained model 400A that outputs:

[0074] Hereinafter, an example will be described in which the generation network 402 is initially configured as a 4×4 generator 403. The 4×4 generator 403 is a machine learning model that outputs an image of 4×4 pixels. In step ST4, a random vector z map and feature quantity w Ada When input, a 4×4 simulated microscope image P fake is output from the 4×4 generator 403.

[0075] In step ST5, the learning processing unit 111 generates the 4×4 simulated microscope image P fake and the 4×4 patch image included in the learning data 100A are input to the fourth machine learning model 500. The fourth machine learning model 500 is configured as a classifier 501 paired with the generator 403. As with the generator 403, the classifier 501 becomes a classifier 501 that gradually recognizes images with higher resolution as learning progresses. That is, as shown in FIG. 14, a generation layer is added to the generator 403, and as the generator 403 generates images of larger sizes, a layer for identifying images of larger sizes is also added to the paired classifier 501, and the fourth machine learning model 500 finally becomes capable of identifying images of larger sizes. Therefore, here, the simulated microscope image P is input to the 4×4 classifier 501 paired with the 4×4 generator 403. fake and a 4×4 patch image is input. Hereinafter, a layer added to the classifier 501 in conjunction with the addition of a generation layer will be referred to as a “classification layer”.

[0076] The structure of the classifier 501 is not particularly limited as long as it is a machine learning model that outputs a result of determining whether an input image is an image derived from the training data 100A (true) or an image generated by the generative network 402 (false). The classifier 501 of this embodiment includes a probability output layer. The 4×4 classifier 501 extracts features for each input image and outputs the probability or likelihood that the image is an image (true) included in the training data 100A.

[0077] In step ST6, the learning processing unit 111 adjusts the parameters of either the third machine learning model 400 or the fourth machine learning model 500. Steps ST1 to ST6 are repeated a predetermined number of times, and the learning processing unit 111 alternately adjusts the parameters of the third machine learning model 400 or the parameters of the fourth machine learning model 500 each time the process reaches step ST6. First, a case in which the parameters of the third machine learning model 400 are adjusted while keeping the parameters of the fourth machine learning model 500 fixed will be described. In this case, in step ST6, the learning processing unit 111 calculates a loss function represented by the following formula, and optimizes the parameters of the generation network 402 including the projection network 401 and the generator 403 so that the loss function value is minimized. In the formula, f(·) represents the projection network 401, g(·) represents the generation network 402, d(·) represents the fourth machine learning model 500, and λ represents a regularization coefficient. In addition, p z is the uniform distribution that the random vector z follows.

number

[0078] The loss function is based on Wasserstein divergence. Wasserstein divergence is similar to the loss function in the first embodiment in that it focuses on the difference between the data output by the third machine learning model 400 and the corresponding learning data 100A. However, when the simulated microscope image P fake By moving the probability distribution that follows, patch P ji The difference is that it is based on the idea of ​​minimizing the cost of matching a probability distribution to a given set of ranks. The loss function used for training is not limited to this, but using a loss function based on Wasserstein divergence makes StyleGAN training more stable.

[0079] The learning processing unit 111 calculates the loss function so as to minimize the value of the loss function, that is, the simulated microscope image P fake The probability distribution that follows and the patch P of the training data 100A j i The parameters of the third machine learning model 400 are adjusted so as to increase the degree of agreement with the probability distribution followed by the fourth machine learning model 500. In other words, the parameters of the third machine learning model 400 are adjusted so as to decrease the probability of the correct answer of the classification result of the fourth machine learning model 500. The saving processing unit 112 updates the parameters of the third machine learning model 400 included in the parameters 102A to the adjusted parameters.

[0080] Furthermore, when the parameters of the fourth machine learning model 500 are adjusted while the parameters of the third machine learning model 400 are fixed, in step ST6, the learning processing unit 111 optimizes the parameters of the fourth machine learning model 500 so as to maximize the value of the above-mentioned loss function. That is, the parameters of the fourth machine learning model 500 are adjusted so as to increase the probability that the classification result of the fourth machine learning model 500 is correct. The saving processing unit 112 updates the parameters of the fourth machine learning model 500 included in the parameters 102A to the adjusted parameters.

[0081] In step ST7, the learning processing unit 111 judges whether or not the learning of the third machine learning model 400 and the fourth machine learning model 500 at the current point in time has been completed. The criteria for judging whether or not the learning has been completed can be determined appropriately. For example, it can be judged that the learning has been completed when steps ST1 to ST6 have been repeated a predetermined number of times since the generation layer and the discrimination layer paired with it were last added. Alternatively, it can be judged that the learning has been completed when the probability of correct answer of the discrimination result of the fourth machine learning model 500 converges to a low value and the simulated microscope image P output from the third machine learning model 400 becomes high. fake is the patch P of the training data 100A. j i When it is determined that the learning is not completed, the process returns to step ST1.

[0082] Returning to step ST1, the learning processing unit 111 again selects the combination vector y fake Then, steps ST1 to ST6 are repeated.

[0083] If it is determined in step ST7 that the learning is completed, the process proceeds to step ST8. In step ST8, the learning processing unit 111 determines whether or not to add a new generation layer to the generation network 402 and a new discrimination layer to the fourth machine learning model 500. The generation layer and the discrimination layer are generated by the generator 403, i.e., the simulated microscope image P fake is added until it reaches a predetermined size. That is, in step ST8, the learning processing unit 111 performs a process in which the generation network 402 generates a simulated microscope image P fake When the learning processing unit 111 determines to add a generation layer and a discrimination layer, the process proceeds to step ST9.

[0084] In step ST9, a generation layer corresponding to a resolution one step higher is added to the generator 403 for which learning has been completed, and a generator 403 with a resolution one step higher is generated. For example, after learning of the 4×4 generator 403 is completed, an 8×8 simulated microscope image P fake A generation layer is added to generate an 8 × 8 simulated microscope image P fake Similarly, in step ST9, a discrimination layer corresponding to a resolution one step higher is added to the fourth machine learning model 500, to the discriminator 501 for which learning has been completed. For example, after the learning of the 4×4 discriminator 501 is completed, an 8×8 discrimination layer for discriminating between the authenticity and falsehood of an 8×8 image is added to the fourth machine learning model 500, that is, the 4×4 discriminator 501. In this way, the generators 403 and discriminators 501 corresponding to each resolution are generated and trained in stages, so that finally, when the noise and blending vector are input, a simulated microscope image P of a predetermined resolution is generated. fake In this way, by gradually adding generative layers and discriminative layers and training the model, it becomes possible to generate a machine learning model that outputs high-resolution images.

[0085] On the other hand, if no further generation layers or discrimination layers are added in step ST8, the learning process ends. As a result, a trained model 400A including a projection network 401 and a generation network 402 is generated. The generation network 402 adds the feature quantity w based on AdaIN to each upsampling layer 4030. Ada In other words, the simulated microscope image P fake Since the style information to be reflected in the image generation process for each resolution exists, it is possible to manipulate a variety of styles using elements of the rubber compounding vector.

[0086] <3-3. Image generation device and image generation process> When the trained model 400A is regarded as one trained model, it can be said that it is a model that outputs a simulated microscope image when noise and blending data are input. Therefore, by implementing the trained model 400A in hardware, it is possible to manufacture a generation device 2A that generates a simulated microscope image when noise and blending data are input. FIG. 15 is a block diagram showing the electrical configuration of the generation device 2A. The generation device 2A is a general-purpose personal computer having a similar configuration to the generation device 2 and the learning devices 1 and 1A, etc., as hardware, and a program 201A incorporating the trained model 400A is stored in the storage unit 20. The other configurations of the generation device 2A are common to the generation device 2 and the learning devices 1 and 1A, etc., and therefore the same reference numerals are used and the description is omitted.

[0087] The type of data input to the generating device 2A is the same as the type of data input to the generating device 2. The type of data output from the generating device 2A is the same as the type of data output from the generating device 2. Therefore, the flow of the image generation process by the generating device 2A is the same as the flowchart shown in FIG. 7. However, the trained model 400A generates the simulated microscope image P for each resolution. fake Therefore, the generating device 2A can be said to include a trained generating layer that generates the simulated microscope image P fake By configuring the generating device 2A in this way, it is possible to consider how the compounding of the rubber composition affects the characteristics of the microscope image at each resolution. <4. Modifications> Although one embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and various modifications are possible without departing from the spirit of the present invention. The following modified examples can be appropriately combined.

[0088] <4-1> The learning device 1 may perform machine learning using another image generation model, for example, a variational autoencoder (VAE), instead of machine learning using AC-GAN. That is, the trained model 200A and the trained model 300A may be generated by machine learning using VAE or machine learning using another image generation model.

[0089] <4-2> In learning the first machine learning model 200 of the learning device 1, the Kullback-Leibler (KL) divergence described later may be used. The KL divergence is a measure of the distance between probability distributions and can take values ​​of [0, ∞]. It can be said that the smaller the value of the KL divergence, the higher the similarity of the data according to the respective probability distributions. Specifically, the second machine learning model 300 may be configured to output the KL divergence between the simulated microscope image and the microscope image generated by the first machine learning model 200, and the learning processing unit 111 may train the first machine learning model 200 so that the value of the KL divergence becomes small.

[0090] <4-3> The learning devices 1, 1A, the image generating devices 2, 2A, and the estimation device 3 may be configured to include an application specific integrated circuit (ASIC), a programmable logic device (PLD), or the like instead of or in addition to a processor, and to be used for calculations. Furthermore, the learning programs 101, 101A, the programs 201, 201A, and the program 301 cause the devices in which they are installed to execute steps S1 to S4, S21 to S24, S41 to S43, and S51 to S53, respectively, but these programs may have at least some steps distributed and executed by another computer or device, a service provided via the Internet, or the like.

[0091] <4-4> Steps S1 to S4 may be performed after steps S21 to S24. In other words, the learning device 1 may be configured to first train the first machine learning model 200, and then train the second machine learning model 300. EXAMPLES

[0092] Examples of the present invention will be described below. However, the following examples are merely illustrative of the present invention and the present invention is not limited thereto.

[0093] A simulated microscope image (patch) was generated using a trained model trained by a training device similar to those in the first and second embodiments. The similarity between the generated patch and the patch of the microscope image was compared.

[0094] [Experiment 1: Experiment using the image generation method according to the first embodiment] <Experimental conditions> 72 samples of rubber compositions with different compounding data were prepared. Each rubber composition was photographed with an electron microscope, and a total of 156 microscope images were obtained. That is, there were multiple microscope images for each sample. The microscope images were grayscale and had a resolution of 1536 x 1024 pixels. The magnification was 20,000 times.

[0095] Each microscope image was divided into patches of 128 × 128 pixels, generating 522 patches per microscope image. The slide width at the time of patch division was 50 pixels. com was set to 32. In other words, the blend vector was a 32-dimensional vector.

[0096] The batch size during learning was 64, and the number of epochs, which is the number of learning times, was 50. AC-GAN learning was performed using the deep learning framework TensorFlow. The validation method used was a 10-fold tolerance test. In other words, the rubber composition samples were divided into training samples and evaluation samples, and a data set of microscopic images and compounding data of the training samples was used as learning data, and a data set of microscopic images and compounding data of the evaluation samples was used for validation.

[0097] In this experiment, we created a patch of a simulated microscope image from the compounding data of the evaluation sample, and evaluated how similar the generated patch was to the patch of the microscope image of the evaluation sample. The KL divergence was used as the method to evaluate the similarity as follows. The smaller the KL divergence value, the more similar the images are. (1-1) A single microscopic image was divided using the method described above to generate 522 patches. (1-2) The blend data of the evaluation samples was input into the first machine learning model trained using a learning device, and 522 patches were generated. (2) For each patch generated in (1-1) and (1-2), a feature representing the image feature was calculated. The feature was calculated using Inception-v3 (Google), which has been trained using ImageNet, a well-known dataset. The feature was a 2048-dimensional vector output from the third pooling layer of Inception-v3. (3) Principal component analysis was performed on the calculated features, and dimensionality reduction was performed to obtain a 78-dimensional vector. (4) The KL divergence D between the probability distribution S of the feature vectors of the microscopic image patches and the probability distribution T of the feature vectors of the simulated microscopic image patches KL (S||T) was calculated based on the following formula:

number

[0098] In addition, when there were multiple microscopic images of the same sample, the average value of the KL divergence with each microscopic image was calculated. The evaluation was performed using the ranking of the sample to be generated (the sample having the blending data that was the basis of the patch of the simulated microscopic image) when the KL divergence with respect to all samples was arranged in ascending order.

[0099] <Experimental Results> Figure 16 shows an example of a microscope image of a sample, a patch (a) generated from the microscope image, and a patch (b) of a simulated microscope image generated from the formulation data of the sample. As shown in Figure 16, it was confirmed that the patch (a) and the patch (b) are qualitatively similar.

[0100] Figure 17 is a histogram of the ranks of samples to be generated when the KL divergence for all 72 samples is sorted in ascending order. As shown in Figure 17, the number of samples with the highest similarity is the largest, and then the number of corresponding samples decreases as the rank decreases. This confirmed that it is possible to create a simulated microscope image similar to the microscope image of the rubber composition from the compounding data. The average rank was 17.5th.

[0101] [Experiment 2: Experiment using the image generation method according to the second embodiment] <Experimental conditions> 72 samples of rubber compositions with different compounding data were prepared. Each rubber composition was photographed with an electron microscope, and a total of 156 microscopic images were obtained. In other words, there were multiple microscopic images for each sample. The microscopic images were grayscale and had a resolution of 1536 x 1024 pixels. The magnification was 20,000 times. The 156 sets of data sets, each pairing compounding data with its corresponding microscopic image, were divided, and 90% were used as training samples for learning StyleGAN, and the remaining 10% were used as evaluation samples for learning verification.

[0102] Each microscope image was divided into patches of 256 × 256 pixels, generating 416 patches per microscope image. The slide width for patch division was 50 pixels. com was set to 33. In other words, the blending vector was a 33-dimensional vector. The batch size during learning was 64, and the number of epochs, which is the number of learning times, was 100. Under these conditions, the third machine learning model and the fourth machine learning model according to the second embodiment were trained to generate trained models that generate simulated microscope images of 256 × 256 pixels (hereinafter referred to as "generated patches").

[0103] <Evaluation> We evaluated the similarity between the distribution of generated patches generated by the trained model and the distribution of patches generated (divided) from the microscopic image of the evaluation sample (hereinafter referred to as "real patches"). First, the real patch and generated patch corresponding to the mth (m = 1, 2, ..., M; M is the type of the evaluation sample's blend data) blend data were input to DenseNet121 trained with ImageNet, and the features of each patch were calculated. The mean and covariance of the features calculated for the real patch were respectively defined as μ r m and Σ r m Similarly, for the generated patches, the mean and covariance of the features are defined as μ g m and Σ g m Then, the Frechet Inception Distance (FID PM ) was calculated. PM is an index representing the similarity between a generated patch generated from specific combination data and an actual patch corresponding to that combination data, and the smaller the value, the higher the similarity between the distributions.

number

number

[0104] In addition, as a comparison index, FID is calculated based on the following formula: CM The FID was calculated. CM is an index that represents the similarity between a generated patch generated from a specific combination data and an actual patch corresponding to the combination data excluding the specific combination data, and a smaller value indicates a higher similarity between the distributions.

number

number

[0105] When each is calculated, FID PM =181.9, FID CM =233.6, FID PM FID CM The value was smaller than . This indicates that the similarity between a generated patch generated based on a specific combination data and an actual patch corresponding to the combination data is generally higher than the similarity between the generated patch and an actual patch corresponding to combination data other than the combination data. This confirms that the trained model according to the second embodiment can generate a patch that represents the combination data.

[0106] <Reference 1> For reference, FIG. 18 shows an image comparing a 256×256 generated patch with a microscope image corresponding to the original formulation data. Rubber composition A and rubber composition B are rubber compositions having different formulation data. When comparing the microscope image of rubber composition A with the microscope image of rubber composition B, the microscope image of rubber composition A is darker and the microscope image of rubber composition B is brighter. Similarly, when comparing the generated patch generated from the formulation data of rubber composition A with the generated patch generated from the formulation data of rubber composition B, the generated patch generated from the formulation data of rubber composition A is darker. In other words, it can be confirmed that the generated patch reflects the features appearing in the original microscope image.

[0107] <Reference 2> As another reference, we created compounding data in which the compounding ratio of two specific types of polymers in the rubber composition was gradually changed. These compounding data were input into the trained model, and 256 x 256 generated patches were output. Each generated patch is shown in Figure 19. As can be seen from Figure 19, the features appearing in the generated patches change depending on the polymer compounding ratio. [Explanation of symbols]

[0108] 1,1A Learning Device 2,2A Image generating device 3 Mixture estimation device 10 Storage section 100,100A learning data 111 Learning processing unit 200 First Machine Learning Model 210 Data Acquisition Section 211 Image Generation Unit 300 Second Machine Learning Model 310 Image Acquisition Unit 311 Mixture Estimation Department 400 3rd generation machine learning models 500 4th Machine Learning Model

Claims

1. a data acquisition unit that acquires blending data representing the blending of materials that constitute the rubber composition; an image generation unit that inputs the acquired blending data together with noise into a plurality of machine learning models and generates, as outputs of the plurality of machine learning models, a plurality of simulated microscope images that simulate microscope images of the rubber composition captured by a microscope; Equipped with The plurality of simulated microscope images generated as outputs of the plurality of machine learning models simulate microscope images obtained by capturing the rubber compositions with different elapsed times from production using a microscope. Image generating device.

2. The microscope is an electron microscope. The image generating device of claim 1 .

3. Each of the plurality of machine learning models includes a projection network that calculates and outputs a feature amount representing a property to be reflected in the plurality of simulated microscope images from the input formulation data and the noise.

3. The image generating device according to claim 1 or 2.

4. Each of the plurality of machine learning models further includes a generation network that upsamples input data to generate data larger in size than the input data and outputs the data as the plurality of simulated microscope images, and the feature amount is incorporated into the generation network each time upsampling is performed. The image generating device according to claim 3 .

5. obtaining formulation data representing a formulation of materials constituting a rubber composition; inputting the acquired blending data together with noise into a plurality of machine learning models, and generating, as outputs of the plurality of machine learning models, a plurality of simulated microscope images that simulate microscope images of the rubber composition captured by a microscope; Including, The plurality of simulated microscope images generated as outputs of the plurality of machine learning models simulate microscope images obtained by capturing the rubber compositions with different elapsed times from production using a microscope. Image generation method.

6. obtaining formulation data representing a formulation of materials constituting a rubber composition; inputting the acquired blending data together with noise into a plurality of machine learning models, and generating, as outputs of the plurality of machine learning models, a plurality of simulated microscope images that simulate microscope images of the rubber composition captured by a microscope; on the computer, The plurality of simulated microscope images generated as outputs of the plurality of machine learning models simulate microscope images obtained by capturing the rubber compositions with different elapsed times from production using a microscope. Image generation program.

7. A step of preparing first learning data that combines a first microscopic image of a rubber composition that has been produced for a first time with formulation data that represents the formulation of materials that make up the rubber composition; preparing second learning data in which a second microscope image of the rubber composition taken with a microscope after a second time period different from the first time period has elapsed since production is combined with the blending data; a step of training a first machine learning model that, when the blending data and noise are input, generates a first simulated microscope image that simulates the first microscope image based on the input blending data and noise, and a second machine learning model that, when the first microscope image or the first simulated microscope image is input, outputs a classification result indicating whether the input image is the first microscope image; a step of training a third machine learning model that, when the blending data and noise are input, generates a second simulated microscope image that simulates the second microscope image based on the input blending data and noise, and a fourth machine learning model that, when the second microscope image or the second simulated microscope image is input, outputs a classification result indicating whether the input image is the second microscope image; Equipped with The step of training the first machine learning model and the second machine learning model includes: training the second machine learning model so that, when the first microscope image or the first simulated microscope image is input to the second machine learning model, the accuracy rate of the classification result output by the second machine learning model is increased; training the first machine learning model so that, when the first simulated microscope image is input to the second machine learning model, the probability of the classification result output by the second machine learning model being correct is reduced; Repeat alternately, The step of training the third machine learning model and the fourth machine learning model includes: training the fourth machine learning model so that, when the second microscope image or the second simulated microscope image is input to the fourth machine learning model, the classification result output by the fourth machine learning model has a high probability of being correct; training the third machine learning model so that, when the second simulated microscope image is input to the fourth machine learning model, the probability of the classification result output by the fourth machine learning model being correct is reduced; Repeat alternately, How to generate a trained model.

8. the first machine learning model includes a generation layer that generates data larger in size than the input data by upsampling the input data and outputs the data as the first simulated microscope image of a predetermined size; the second machine learning model includes an identification layer that is paired with the generation layer, and that, when the first simulated microscope image output by the generation layer or the first microscope image having the same size as the first simulated microscope image is input, outputs an identification result indicating whether the input image is the first microscope image; After the step of training the first machine learning model and the second machine learning model, adding one new generation layer to the first machine learning model and adding one new identification layer to the second machine learning model, the new generation layer being paired with the added generation layer; After the adding step, further training the first machine learning model and the second machine learning model; Furthermore, repeating the adding step and the further learning step alternately; a size of the first simulated microscope image output by the added generation layer is increased stepwise each time the adding step is repeated; The method for generating a trained model according to claim 7.

9. The second machine learning model further outputs data estimating blending data corresponding to the input first microscope image or the input first simulated microscope image; the step of training the second machine learning model includes a step of training the second machine learning model so that, when the first microscopic image is input to the second machine learning model, a difference between the data output by the second machine learning model and the blending data becomes small; the step of training the first machine learning model includes a step of training the first machine learning model so that, when the first simulated microscope image is input to the second machine learning model, a difference between the data output by the second machine learning model and the formulation data input to the first machine learning model becomes small. The method for generating a trained model according to claim 7.

10. The microscope is an electron microscope. A method for generating a trained model according to any one of claims 7 to 9.

11. A step of preparing first learning data that combines a first microscopic image of a rubber composition that has been produced for a first time with formulation data that represents the formulation of materials that make up the rubber composition; preparing second learning data in which a second microscope image of the rubber composition taken with a microscope after a second time period different from the first time period has elapsed since production is combined with the blending data; a step of training a first machine learning model that, when the blending data and noise are input, outputs a first simulated microscope image that simulates the first microscope image based on the input blending data and noise, and a second machine learning model that, when the first microscope image or the first simulated microscope image is input, outputs a classification result indicating whether the input image is the first microscope image; a step of training a third machine learning model that, when the blending data and noise are input, generates a second simulated microscope image that simulates the second microscope image based on the input blending data and noise, and a fourth machine learning model that, when the second microscope image or the second simulated microscope image is input, outputs a classification result indicating whether the input image is the second microscope image; on the computer, The step of training the first machine learning model and the second machine learning model includes: training the second machine learning model so that, when the first microscope image or the first simulated microscope image is input to the second machine learning model, the accuracy rate of the classification result output by the second machine learning model is increased; training the first machine learning model so that, when the first simulated microscope image is input to the second machine learning model, the probability of the classification result output by the second machine learning model being correct is reduced; Repeat alternately, The step of training the third machine learning model and the fourth machine learning model includes: training the fourth machine learning model so that, when the second microscope image or the second simulated microscope image is input to the fourth machine learning model, the classification result output by the fourth machine learning model has a high probability of being correct; training the third machine learning model so that, when the second simulated microscope image is input to the fourth machine learning model, the probability of the classification result output by the fourth machine learning model being correct is reduced; Repeat alternately, A program for generating trained models.

12. A memory unit for storing first learning data which is a combination of a first microscopic image of a rubber composition taken by a microscope after a first time has passed since its production and formulation data which represents the formulation of materials constituting the rubber composition, and second learning data which is a combination of a second microscopic image of the rubber composition taken by a microscope after a second time has passed since its production which is different from the first time and is the formulation data; a learning processing unit that trains the following machine learning models: a first machine learning model that, when the blending data and noise are input, outputs a first simulated microscope image that simulates the first microscope image, which is generated based on the input blending data and noise; a second machine learning model that, when the first microscope image or the first simulated microscope image is input, outputs a classification result indicating whether the input image is the first microscope image; a third machine learning model that, when the blending data and noise are input, generates a second simulated microscope image that simulates the second microscope image, based on the input blending data and noise; and a fourth machine learning model that, when the second microscope image or the second simulated microscope image is input, outputs a classification result indicating whether the input image is the second microscope image. Equipped with The learning processing unit training the second machine learning model so that, when the first microscope image or the first simulated microscope image is input to the second machine learning model, the accuracy rate of the classification result output by the second machine learning model is increased; training the first machine learning model so that, when the first simulated microscope image is input to the second machine learning model, the probability of the classification result output by the second machine learning model being correct is reduced; Repeat alternately, training the fourth machine learning model so that, when the second microscope image or the second simulated microscope image is input to the fourth machine learning model, the classification result output by the fourth machine learning model has a high probability of being correct; training the third machine learning model so that, when the second simulated microscope image is input to the fourth machine learning model, the probability of the classification result output by the fourth machine learning model being correct is reduced; Repeat alternately, Learning device.

Images