Model generation method and model generation apparatus for microstructure material

Abstract

To generate a model reproducing a three dimensional structure of microstructure material whose structure changes according to a three dimensional position in AI for generating the model of the three dimensional structure from a two dimensional image of the microstructure material.SOLUTION: In a model generation method for microstructure material, learning of AI is performed by combining a two dimensional image of the microstructure material and position information of the two dimensional image in the microstructure material (step S10), and a model of a three dimensional structure is generated by using a learned AI (step S80).SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a method and apparatus for generating models of microstructured materials. [Background technology]

[0002] Conventionally, microstructured materials with various microstructures have been used depending on the application, such as porous materials with many pores arranged three-dimensionally within the material, crystalline materials with a specific crystal structure, etc. When designing such microstructured materials, it is necessary to determine the microstructure so that the appropriate properties can be obtained depending on the application.

[0003] Conventional design methods for microstructured materials include, for example, creating several prototypes based on the designer's experience and intuition, measuring the three-dimensional structure and properties of each of these prototypes, and then determining the final three-dimensional structure of the microstructured material. However, because this design method requires measuring the three-dimensional structure of the prototype, it has several drawbacks, including the fact that the measurement requires more time and effort than two-dimensional observation methods, the obtained measurement results have lower resolution, and the structural features may not be fully captured due to the limited field of view during measurement.

[0004] On the other hand, in recent years, as described in Non-Patent Document 1, for example, it has been proposed to use machine learning to train AI (artificial intelligence) to learn the relationship between two-dimensional images and three-dimensional structures of various microstructure materials, and to use this trained AI to generate a three-dimensional structure model from two-dimensional images of a specific microstructure. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Steve Kench and Samuel J. Cooper, "GENERATING 3D STRUCTURES FROM A 2D SLICE WITH GAN-BASED DIMENSIONALITY EXPANSION", arXiv:2102.07708 [cs.CV], 10 Feb 2021 Summary of the Invention [Problem to be solved by the invention]

[0006] The method described in Non-Patent Document 1 can generate a model that accurately reproduces the three-dimensional structure of a microstructured material in which the entire material has a constant structure regardless of position within the material. However, it is difficult to reproduce the three-dimensional structure of a microstructured material in which the structure changes depending on the position within the material, for example, in which the structure differs significantly at a certain boundary surface within the material.

[0007] In view of the above-mentioned problems, the present invention aims to provide a technology in AI that generates a three-dimensional structure model from a two-dimensional image of a microstructure material, capable of generating a model that reproduces the three-dimensional structure of a microstructure material whose structure changes depending on its three-dimensional position. [Means for solving the problem]

[0008] The method for generating a model of a microstructured material according to the present invention is a method for generating a three-dimensional structure model from a two-dimensional image of a microstructured material, in which the two-dimensional image is combined with positional information of the two-dimensional image in the microstructured material to train an AI, and the model is generated using the trained AI. The model generation device for microstructured materials according to the present invention is a device for generating a three-dimensional structure model from a two-dimensional image of a microstructured material, and comprises an AI learning unit that combines the two-dimensional image with positional information of the two-dimensional image in the microstructured material to train an AI, and an AI calculation unit that generates the model using the AI ​​that has been trained by the learning unit. [Effects of the Invention]

[0009] According to the present invention, in an AI that generates a three-dimensional structure model from a two-dimensional image of a microstructure material, a technology can be provided that can generate a model that reproduces the three-dimensional structure of a microstructure material whose structure changes depending on its three-dimensional position. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing the configuration of a model generation device according to an embodiment of the present invention. [Figure 2] 1 is a flowchart showing the flow of processing of a model generating device according to an embodiment of the present invention. [Figure 3] 10 is a flowchart showing details of a learning process. [Figure 4] 10 is a flowchart showing details of a learning process. [Figure 5] 10 is a flowchart showing details of a learning process. [Figure 6] 10 is a flowchart showing details of a generated structure evaluation process. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the following embodiment, a model generation device will be described that uses AI to generate a three-dimensional structure model from two-dimensional images of microstructure materials such as porous bodies and crystalline bodies having various microstructures and their position information.

[0012] Fig. 1 is a block diagram showing the configuration of a model generation device according to one embodiment of the present invention. The model generation device 100 shown in Fig. 1 is a computer including a control unit 1, a storage unit 2, a memory 3, an operation input device 4, a display device 5, and a network device 6, and is configured by connecting these devices to each other via a bus 7.

[0013] The control unit 1 is configured using, for example, a CPU (Central Processing Unit) and performs various processes and calculations to operate the model generation device 100. The control unit 1 executes programs stored in the storage unit 2 to realize the functional blocks of the AI ​​calculation unit 11 and the AI ​​learning unit 12. Details of these functional blocks will be described later. Note that some or all of the functions of the control unit 1 may be realized using devices other than a CPU, such as a GPU (Graphic Processing Unit), FPGA (Field Programmable Gate Array), or ASIC (Application Specific Integrated Circuit).

[0014] The storage unit 2 is configured using a large-capacity, non-volatile storage device, such as a magnetic storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and stores programs executed by the control unit 1 and various information used in the processing of the control unit 1. The information stored in the storage unit 2 includes generator data 21, classifier data 22, teacher image data 23a, actual position data 23b, generative structure data 24, hyperparameter data 25, and evaluation index data 26. Details of these data will be described later.

[0015] The memory 3 is configured using a high-speed and volatile storage device such as a DRAM (Dynamic Random Access Memory), and is used as a work area when the control unit 1 executes a program.

[0016] The operation input device 4 is a device for detecting operation input by the user, and is configured using, for example, a keyboard or a mouse. The display device 5 is a device for displaying the processing results of the model generation device 100 on a screen to present them to the user, and is configured using, for example, a liquid crystal display or an organic EL display. The network device 6 is a device for communicating with other computers via a network (not shown). By using the network device 6, the model generation device 100 can transmit various data stored in the storage unit 2 to other computers, and can receive various data transmitted from other computers and store it in the storage unit 2. Note that other computers that can communicate with the model generation device 100 may be used as the operation input device 4 or the display device 5.

[0017] 1 may be physically constructed on a single computer, or may be constructed in a distributed manner on multiple computers. Furthermore, the model generation device 100 may be realized by a cloud computer installed on a cloud, a virtual machine operating in a virtual environment, or the like.

[0018] Next, we will explain the functional blocks of the AI ​​calculation unit 11 and the AI ​​learning unit 12 in the control unit 1. In the model generation device 100, the control unit 1 operates as these functional blocks, allowing the AI ​​to learn and the trained AI to generate a three-dimensional structural model of a microstructure material having a desired microstructure.

[0019] The AI ​​calculation unit 11 has a generator 110 and a discriminator 111. The generator 110 is an AI that generates a model of a microstructure material having a desired three-dimensional structure according to input conditions, and performs AI calculations using a predetermined algorithm with a neural network based on generator data 21 stored in the memory unit 2. Information on the three-dimensional structure model of the microstructure material generated by the generator 110 is stored in the memory unit 2 as generated structure data 24. The discriminator 111 is an AI that determines whether an input cross-sectional image of the three-dimensional structure is a cross-sectional image of the actual microstructure material or a cross-sectional image of the three-dimensional structure model generated by the generator 110, and performs AI calculations using a predetermined algorithm with a neural network based on discriminator data 22 stored in the memory unit 2.

[0020] In this embodiment, the AI ​​calculation unit 11 performs the above-described AI calculations on a microstructured material whose structure varies depending on the position within the material, for example, when the structure varies significantly across a boundary surface within the material. This calculation takes into account positional information within the microstructured material. In the following description, a region within the microstructured material that has similar structural characteristics is referred to as a "structural region." That is, in the AI ​​calculation unit 11, the generator 110 generates a three-dimensional structural model of the microstructured material consisting of multiple structural regions, each with different structural characteristics. Meanwhile, the discriminator 111 determines whether a cross-sectional image acquired from the three-dimensional structural model of the microstructured material is a cross-sectional image of the actual object or a cross-sectional image of the three-dimensional structural model generated by the generator 110, taking into account the structural characteristics of the structural region corresponding to the positional information of the cross-sectional image.

[0021] In the AI ​​computing unit 11, the generator 110 and the discriminator 111 have opposing objectives, and AI learning is performed by the generator 110 and the discriminator 111 competing with each other to achieve their respective objectives. That is, the generator 110 is trained to generate a three-dimensional structural model of a microstructure material that is closer to the actual object. On the other hand, the discriminator 111 is trained to more accurately discriminate that a cross-sectional image of the three-dimensional structural model generated by the generator 110 is not a cross-sectional image of the actual object. Such a configuration of the AI ​​computing unit 11 is called a GAN (Generative Adversarial Network). The learning state of the generator 110 is reflected in generator data 21 as weight information of the neural network that constitutes the generator 110. Similarly, the learning state of the discriminator 111 is reflected in discriminator data 22 as weight information of the neural network that constitutes the discriminator 111.

[0022] The AI ​​learning unit 12 is a functional block for AI learning, and includes a learning processing unit 120, an evaluation index calculation unit 121, and a learning progress determination unit 122. The learning processing unit 120 performs learning processing of the generator 110 and the discriminator 111 by the learning method described above using multiple combinations of a cross-sectional image of the actual microstructure material represented by the teacher image data 23a, or an image obtained by cutting out a part of this cross-sectional image, and position information of the actual microstructure material represented by the actual position data 23b, and multiple combinations of a cross-sectional image of a three-dimensional structure model represented by the generated structure data 24, or an image obtained by cutting out a part of this cross-sectional image, and position information of the image in the three-dimensional structure model. At this time, the learning processing unit 120 sets conditions for the learning processing based on various hyperparameters represented by the hyperparameter data 25. The evaluation index calculation unit 121 calculates an evaluation index relating to the geometric shape difference between the actual structure of the microstructure material represented by the teacher image data 23a and actual position data 23b and the three-dimensional structure model represented by the generated structure data 24 for each of the aforementioned structure regions, and stores the calculation results as evaluation index data 26 in the storage unit 2. The learning progress determination unit 122 determines the progress of AI learning by the learning processing unit 120 based on the evaluation index calculated by the evaluation index calculation unit 121.

[0023] The generated structure data 24 may be transmitted from the model generating device 100 to a computer that designs the microstructured material, for example, by the network device 6. The computer can design the microstructured material based on the generated structure data 24 received from the model generating device 100.

[0024] As described above, when the generator 110 and the discriminator 111 are trained in the learning processing unit 120, multiple images of the actual cross-sectional image of the microstructure material represented by the teacher image data 23, or an image obtained by cutting out a portion of this cross-sectional image, and multiple images of the cross-sectional image of the three-dimensional structure model represented by the generated structure data 24, or an image obtained by cutting out a portion of this cross-sectional image, are used in combination with the positional information of each cross-section. In the following description, of the images used when training the generator 110 and the discriminator 111, the actual cross-sectional image of the microstructure material represented by the teacher image data 23, or an image obtained by cutting out a portion of this cross-sectional image, will be referred to as a "first learning image," and the cross-sectional image of the three-dimensional structure model represented by the generated structure data 24, or an image obtained by cutting out a portion of this cross-sectional image, will be referred to as a "second learning image." The image sizes of these first learning images and second learning images can be determined arbitrarily depending on the image size of the actual cross-sectional image of the microstructure material represented by the teacher image data 23 and the size of the three-dimensional structure model represented by the generated structure data 24.

[0025] Moreover, of the first training images, the number of first training images used for training the classifier 111 will be referred to as "number of images A" below. On the other hand, of the second training images, the number of second training images used for training the classifier 111 will be referred to as "number of images B" below. Furthermore, of the second training images, the number of second training images used for training the generator 110 will be referred to as "number of images C" below.

[0026] Next, details of the processing performed by the control unit 1 will be described with reference to Fig. 2. Fig. 2 is a flowchart showing the processing flow of a model generation device according to one embodiment of the present invention. In the model generation device 100, the control unit 1 executes the processing shown in the flowchart of Fig. 2 in response to a user's operational input, thereby training the generator 110 and the discriminator 111 and generating a three-dimensional structure model using the trained generator 110. In this way, a model is generated that reproduces the three-dimensional structure of a microstructure material whose structure changes depending on the three-dimensional position.

[0027] When the user instructs the model generation device 100 to start processing via the operation input device 4, in step S10, the control unit 1 performs a learning process on the generator 110 and the classifier 111 using the AI ​​calculation unit 11 and the AI ​​learning unit 12. Details of this learning process will be described later with reference to FIGS.

[0028] In step S40, the evaluation index calculation unit 121 performs a generated structure evaluation process to evaluate the three-dimensional structure model finally generated by the generator 110 in the learning process performed in step S10. Here, using cross-sectional images and their position information acquired from the three-dimensional structure model represented by the generated structure data 24, along with the teacher image data 23a and actual position data 23b, an evaluation index representing the magnitude of difference in feature quantities between these images is calculated for each structural region. This allows evaluation indexes relating to the geometric shape difference between the cross-sectional images of the actual structure of the microstructure material and the three-dimensional structure model generated by the generator 110 to be calculated for each of multiple positions in the microstructure material. The values ​​of the evaluation indexes calculated for each structural region in step S40 are stored in the storage unit 2 as evaluation index data 26. Details of the generated structure evaluation process performed in step S40 will be described later with reference to FIG. 6.

[0029] In step S50, the learning progress determination unit 122 calculates statistics of the evaluation index of each structural region obtained by the generated structure evaluation process of step S40. Here, values ​​such as the average and variance are calculated as statistics from the evaluation index for each structural region represented by the evaluation index data 26 stored in the storage unit 2. In this way, an evaluation index for the entire three-dimensional structural model consisting of multiple structural regions is obtained based on the evaluation index data 26.

[0030] In step S60, the learning progress determination unit 122 determines the progress of AI learning by the learning processing unit 120 based on the statistics of the evaluation index calculated in step S50. Here, for example, the average or variance value of the evaluation index is compared with a predetermined threshold, and based on the comparison result, it is determined whether the progress of AI learning is sufficient, i.e., whether the generator 110 and the discriminator 111 have been sufficiently trained. In this way, the learning progress of the AI ​​can be determined based on the evaluation index calculated by the evaluation index calculation unit 121 for each position in the microstructure material.

[0031] In step S70, the control unit 1 determines whether the AI ​​has sufficiently learned based on the determination result of the AI ​​learning progress made in step S60. If the AI ​​learning is insufficient, that is, if it is determined in step S60 that the AI ​​learning progress is not sufficient, the process returns to step S10 and repeats the above-mentioned process. If the AI ​​has sufficiently learned, that is, if it is determined in step S60 that the AI ​​learning progress is sufficient and that the generator 110 and the discriminator 111 have sufficiently learned, the process proceeds to step S80.

[0032] In step S80, structure generation is performed by the trained generator 110. Here, a three-dimensional structure model of the microstructure material according to predetermined conditions is generated by AI calculations performed by the generator 110, and information on the generated three-dimensional structure model is stored in the memory unit 2 as generated structure data 24. As a result, information on the three-dimensional structure model of the microstructure material generated by the trained generator 110 is recorded in the generated structure data 24.

[0033] Through the processing of steps S10 to S80, the generator 110 and the discriminator 111 are trained, and once a three-dimensional structure model of the microstructure material is generated by the trained generator 110, the control unit 1 ends the processing shown in the flowchart of Figure 2.

[0034] 3, 4 and 5 are flowcharts showing the details of the learning process executed in step S10 of FIG.

[0035] In step S100 of FIG. 3, various hyperparameters are set. Here, various hyperparameters represented by the hyperparameter data 25 are set to predetermined default values. For example, values ​​such as the learning rates of the generator 110 and the classifier 111 (the update width of the weights during learning), the numbers of images A, B, and C mentioned above, and the learning ratio of the generator 110 and the classifier 111 (the ratio of the number of times the classifier 111 has learned to the number of times the generator 110 has learned) are set as hyperparameters to be used in the learning process. Note that at this time, values ​​of the hyperparameters adjusted by a predetermined method may be set, or the user may set arbitrary values ​​for each hyperparameter.

[0036] In step S110, the teacher image data 23a and the actual position data 23b stored in the memory unit 2 are read into the learning processing unit 120. Here, a combination of each cross-sectional image of the actual microstructure material represented by the teacher image data 23a and the position information of each cross-section represented by the actual position data 23b is acquired as information to be used in subsequent processing.

[0037] In step S120, it is determined whether the teacher image data 23a contains sufficient cross-sectional images of the actual structure of the microstructure material based on the teacher image data 23a and the actual position data 23b read in step S110. Here, it is determined whether a predetermined number or more of cross-sectional images are recorded in the teacher image data 23a for each structural region of the microstructure material, for example, based on the position information of each cross-sectional image represented by the actual position data 23b. As a result, if the predetermined number or more of cross-sectional images are recorded in the teacher image data 23a for all structural regions, it is determined that there are sufficient cross-sectional images of the actual structure, and the process proceeds to step S140. On the other hand, if the number of cross-sectional images recorded in the teacher image data 23a for at least one structural region is less than the predetermined number, it is determined that there are insufficient cross-sectional images of the actual structure, and the process proceeds to step S130.

[0038] In step S130, based on the position information represented by the actual position data 23b read in step S110, new position information is generated by changing the acquisition position of the cross-sectional image represented by the teacher image data 23a, for which the number of images was determined to be insufficient in step S120, to another position within the same structure region. This virtually increases the number of cross-sectional images represented by the teacher image data 23a within the structure region, so that a sufficient number of cross-sectional images of the actual microstructure material can be obtained for the structure region.

[0039] In step S140, the number of times D the classifier 111 learns and the number of times G the generator 110 learns are determined based on the hyperparameters set in step S100. Here, the number of times D and G the learns are determined so as to satisfy the set learning ratio.

[0040] In step S150, the value of the variable d for counting the number of times the classifier 111 has learned is set to an initial value of 0.

[0041] In step S160, it is determined whether or not an important reproduction region has been designated. An important reproduction region is a structural region designated as a portion where the actual structure of the microstructure material should be prioritized for reproduction when the generator 110 generates a three-dimensional structural model. When the user causes the model generating device 100 to execute processing according to the flowchart in Fig. 2, the user can designate any structural region within the microstructure material as the important reproduction region. If an important reproduction region has been designated, the process proceeds to step S170; if not, the process proceeds to step S180.

[0042] In step S170, when the processing of the subsequent steps S180 and S190 is carried out, the cut-out ratio of the cross-sectional image and the ratio of the generated structure with respect to the designated important reproduction region are set to be increased. Once this setting has been carried out, the process proceeds to step S180.

[0043] In step S180, A cross-sectional images of a predetermined size are randomly cut out as the above-mentioned first learning images from the multiple teacher images represented by the teacher image data 23a read in in step S110, based on the value of the number of images A from the set hyperparameters. Also, position information corresponding to the cut-out position of each first learning image is obtained from the position information of each cross-sectional image represented by the actual position data 23b read in in step S110. Then, A combinations of the A cut-out first learning images and their position information are input to the classifier 111, and the classifier 111 performs AI calculations. At this time, if new position information was generated in step S130, the combination of that position information and cross-sectional image is used to cut out the first learning images as a different combination from the combination of the original position information and cross-sectional image. Also, if a setting was made in step S170 to increase the cut-out ratio of cross-sectional images for the important reproduction region, the A combinations are made to include more combinations of cross-sectional images and position information of the important reproduction region than combinations in other structural regions. In this case, it is preferable that the number A of first learning images is set to 50 or more.

[0044] In step S190, structure generation is performed by the generator 110. Here, a three-dimensional structural model of the microstructure material is generated in accordance with predetermined conditions by AI calculations performed by the generator 110, and information about the generated three-dimensional structural model is stored in the storage unit 2 as generated structure data 24. At this time, if a setting was made in step S170 to increase the ratio of the generated structure to the important reproduction region, the three-dimensional structural model is generated so that the structure of the important reproduction region is reproduced in greater detail. Once the processing of step S190 has been performed, the process proceeds to step S200 in Figure 4.

[0045] In step S200, based on the value of the number of images B from the set hyperparameters, B cross-sectional images of a predetermined size are randomly cut out as the above-mentioned second learning images from the three-dimensional structural model of the microstructure material generated in step S190 of Fig. 3, and positional information corresponding to each cut-out position is obtained. Then, B combinations of the B cut-out second learning images and their respective positional information are input to classifier 111, which performs AI calculations. At this time, if the setting in step S170 was to increase the cut-out ratio of cross-sectional images with respect to the important reproduction region, the B combinations are made to include more combinations of cross-sectional images and positional information of the important reproduction region than combinations in other structural regions. Note that in this case, the number B of second learning images is preferably set to 150 or more.

[0046] In step S210, the weights of the classifier 111 represented by the classifier data 22 are updated based on the value of the learning rate of the classifier 111, which is one of the set hyperparameters. Here, the classification results obtained in step S180 of FIG. 3 for the combinations of A first learning images and position information and the classification results obtained in step S200 for the combinations of B second learning images and position information are fed back to the weights of the classifier 111 according to the learning rate of the classifier 111 so as to increase the difference between these classification results. That is, the weights of the classifier 111 are adjusted and the values ​​of the classifier data 22 are updated so that the classification results in step S180 are more likely to be determined as cross-sectional images of the real object and the classification results in step S200 are less likely to be determined as cross-sectional images of the real object.

[0047] In step S210, the classifier 111 is trained as described above. That is, the position information used for training the classifier 111 is determined based on the acquisition position of the first training image or the second training image in the microstructure material, or on the position after the acquisition position has been changed to another position within the microstructure material. Then, the classifier 111 is trained by combining the first training image and the second training image with the determined position information. At this time, if the processing of step S170 has been performed, the number of data sets for the combination of the first training image and the second training image with the position information corresponding to the specific positional region designated as the emphasis reproduction region within the microstructure material is set relatively larger than the number of data sets for the combination of the first training image and the second training image with the position information corresponding to other positional regions, and the classifier 111 is trained.

[0048] In step S220, the value of the aforementioned variable d is incremented by 1, thereby counting up the variable d.

[0049] In step S230, it is determined whether the value of variable d counted up in step S220 exceeds the number of times D of learning of classifier 111 determined in step S140 of Fig. 3. If the value of variable d is equal to or less than the number of times D of learning of classifier 111, the process returns to step S160 of Fig. 3, and the processes of steps S160 to S220 are repeated using the classifier data 22 updated in step S210. On the other hand, if the value of variable d exceeds the number of times D of learning, the loop process of steps S160 to S220 is terminated and the process proceeds to step S240.

[0050] In step S240, the value of the variable g for counting the number of times the generator 110 has learned is set to an initial value of 0.

[0051] In step S250, a check is made to see if an important reproduction region has been designated, just like in step S160 in Fig. 3. If an important reproduction region has been designated, the process proceeds to step S260, and if not, the process proceeds to step S270.

[0052] In step S260, when the processing of the next step S270 is carried out, the model generation ratio for the designated important reproduction region is set to be increased. Once this setting has been carried out, the process proceeds to step S270.

[0053] In step S270, structure generation is performed by the generator 110. Here, as in step S190 in Fig. 3, a three-dimensional structural model of the microstructure material is generated in accordance with predetermined conditions by AI calculations performed by the generator 110, and information about the generated three-dimensional structural model is stored in the storage unit 2 as generated structure data 24. At this time, if a setting was made in step S260 to increase the ratio of the generated structure to the important reproduction region, the three-dimensional structural model is generated so that the structure of the important reproduction region is reproduced in greater detail. Once the processing of step S270 has been performed, the process proceeds to step S280 in Fig. 5.

[0054] In step S280, based on the value of the number of images C among the set hyperparameters, C cross-sectional images of a predetermined size are randomly cut out from the three-dimensional structure model of the microstructure material generated in step S270 of Figure 4 as the above-mentioned second learning images, and position information corresponding to each cut-out position is obtained.

[0055] In step S290, it is determined whether or not the capacity of memory 3 is sufficient in model generating device 100. If the capacity of memory 3 is equal to or greater than a predetermined value and is sufficient for the amount of memory required to process the C cross-sectional images acquired in step S280, the process proceeds to step S300. On the other hand, if the capacity of memory 3 is less than the predetermined value and is insufficient, the process proceeds to step S310.

[0056] In step S300, C combinations of the C second learning images extracted in step S280 and their respective position information are input to classifier 111, and AI calculations are performed by classifier 111. Note that the number C of second learning images at this time is preferably set to 150 or more, similar to the number B of images described above. After performing the processing of step S300, the process proceeds to step S330.

[0057] In step S310, the C cross-sectional images acquired as second learning images in step S280 are grouped by structural region based on their respective position information, thereby generating X (X≦C) groups each consisting of one or more second learning images belonging to the same structural region.

[0058] In step S320, for X groups obtained by grouping the C second learning images (cross-sectional images) by structural region in step S310, combinations of representative images and position information for each group are input to the discriminator 111, and AI calculations are performed by the discriminator 111. Here, one of the images for each group is designated as the representative image, and X combinations of the designated representative image and its position information are input to the discriminator 111. After performing the processing of step S320, the process proceeds to step S330.

[0059] By the processing of steps S310 and S320 described above, cross-sectional images in the same structural region that have similar structural features are grouped together, and the number of combinations of cross-sectional images and position information input to the classifier 111 is reduced from C to X. As a result, when the capacity of the memory 3 is insufficient, the number of AI calculations by the classifier 111 can be reduced, thereby reducing the processing load on the control unit 1.

[0060] In step S330, the weights of the generator 110 represented by the generator data 21 are updated based on the value of the learning rate of the generator 110, which is one of the set hyperparameters. Here, the discrimination results obtained in step S300 for the combinations of C second training images and position information, or the discrimination results obtained in step S320 for the combinations of X representative images and position information, are fed back to the weights of the generator 110 according to the learning rate of the generator 110 so that these discrimination results approach the discrimination result that the image is a cross-sectional image of the real object. In other words, the weights of the generator 110 are adjusted and the values ​​of the generator data 21 are updated so that the discrimination result in step S300 or step S320 is more likely to be determined to be a cross-sectional image of the real object.

[0061] In step S340, the value of the aforementioned variable g is incremented by 1, thereby counting up the variable g.

[0062] In step S350, it is determined whether the value of variable g counted up in step S340 exceeds the number of times G the generator 110 has learned, which was determined in step S140 of Fig. 3. If the value of variable g is equal to or less than the number of times G the generator 110 has learned, the process returns to step S250 of Fig. 4, and the processes of steps S250 to S340 are repeated using the generator data 21 updated in step S330. On the other hand, if the value of variable g exceeds the number of times G the generator 110 has learned, the process ends the loop of steps S250 to S340, and proceeds to step S360.

[0063] In step S360, it is determined whether learning has been performed the specified number of times. Here, the number of times the series of processes in steps S150 to S350 has been executed is compared with the predetermined number of learning times, and if the number of times the series of processes has been executed is less than the learning number, the process returns to step S150 in Fig. 3 and the above-mentioned process is repeated. On the other hand, if the number of times the series of processes has been executed reaches the learning number, the learning process shown in the flowcharts of Figs. 3 to 5 ends.

[0064] FIG. 6 is a flowchart showing the details of the generated structure evaluation process executed in step S40 of FIG.

[0065] In the generated structure evaluation process, the processes of steps S400 to S430 described below are performed for each structural region having its own unique structural feature.

[0066] In step S400, cross-sectional images of the generated structure obtained in the immediately preceding learning process are acquired. Here, the generated structure data 24 stored in the storage unit 2 is read, a large number of cross sections are set for the three-dimensional structure model of the microstructure material represented by the generated structure data 24, and images of a predetermined image size are randomly cut out from each cross section, thereby making it possible to acquire a large number of cross-sectional images from the generated structure obtained in the immediately preceding learning process.

[0067] In step S410, feature quantities of the geometric shape of the generated structure are calculated from the cross-sectional images acquired in step S400. Here, feature quantities of the geometric shape of the generated structure can be calculated by calculating a probability function, for example, a two-point probability function (tppf), for the generated structure based on a large number of cross-sectional images cut out from the generated structure. Note that the tppf is a probability function that represents the distribution of pores in the three-dimensional structure of a porous body, and is expressed as a function of distance r that represents the probability that two points separated by an arbitrary distance r within the three-dimensional structure of the porous body are both pores. This can be calculated using well-known calculation methods.

[0068] In step S420, the feature quantities of the geometric shape of the actual structure are calculated from the actual cross-sectional image of the microstructure material represented by the teacher image data 23. Here, similar to step S410, the feature quantities of the geometric shape of the actual structure can be calculated by calculating, for example, tppf for the actual structure.

[0069] In step S430, the difference between the feature amounts of the geometric shapes of the generated structure and the actual structure calculated in steps S410 and S420 is calculated. Here, for example, the sum of squares S of the errors of the first to third quartiles of the tppf of the actual structure and the generated structure, expressed by the following formula (1), can be calculated as the difference between the feature amounts of the geometric shapes of the actual structure and the generated structure.

number

[0070] In formula (1), N represents the image size of the cross-sectional image used when calculating the feature quantities of the geometric shapes of the generated structure and the real structure in steps S410 and S420, and is, for example, 64. q (r) and tppf_synthetic_2d_iso q (r) represents the q-th quartile (q=1 to 3) of the feature quantities of the geometric shapes of the generated structure and the real structure calculated in steps S410 and S420, respectively, and is defined, for example, by the following equations (2) and (3), respectively.

number

[0071] In equations (2) and (3), J original and J. synthetic represents a set of cross-sectional images of a predetermined size randomly cut out from the real structure and the generated structure. q [ ] represents the q-th quartile of the data group in the brackets, and tppf_2d(r;I,dir) represents the tppf in the dir (v: vertical, h: horizontal) direction of the two-dimensional cross-sectional image I.

[0072] By performing the loop process of steps S400 to S430 described above for each structure region, it is possible to calculate the difference in the feature quantities of the geometric shape of the generated structure and the actual structure for each part of the microstructure material that has different structural features. After completing the loop process of steps S400 to S430 for all structure regions, proceed to step S440.

[0073] In step S440, the difference in the feature quantities of the geometric shape of the generated structure and the actual structure for each structural region obtained by the loop processing of steps S400 to S430 is output as an evaluation index for each structural region regarding the geometric shape difference between the actual structure of the microstructure material and the structure of the model generated by the AI ​​calculation unit 11. The value of the evaluation index for each structural region output in step S440 is stored in the storage unit 2 as evaluation index data 26. After step S440 is performed, the generated structure evaluation process shown in the flowchart of FIG. 6 is terminated.

[0074] According to the embodiment of the present invention described above, the following advantageous effects are achieved.

[0075] (1) A model generation method for a microstructured material using a model generation device 100 generates a three-dimensional structure model from a two-dimensional image of the microstructured material. In this model generation method, the model generation device 100, which is a computer, combines the two-dimensional image of the microstructured material with positional information of the two-dimensional image in the microstructured material to train the AI ​​calculation unit 11 (step S10), and generates a three-dimensional structure model using the trained AI calculation unit 11 (step S80). In this way, the AI ​​that generates a three-dimensional structure model from the two-dimensional image of the microstructured material can generate a model that reproduces the three-dimensional structure of the microstructured material, whose structure changes depending on the three-dimensional position.

[0076] (2) The AI ​​calculation unit 11 is a GAN having a generator 110 that generates a three-dimensional structure model from a two-dimensional image and a discriminator 111 that determines whether the cross-sectional image of the model generated by the generator 110 is a cross-sectional image of the actual microstructure material (first two-dimensional image) or a cross-sectional image of the model generated by the AI ​​calculation unit 11 (second two-dimensional image). In the learning process of the AI ​​calculation unit 11, the generator 110 and the discriminator 111 are trained by combining a first learning image cut out from the first two-dimensional image, a second learning image cut out from the second two-dimensional image, and position information (steps S180 to S210, S270 to S330). In this way, an AI that can generate a three-dimensional structure model from a two-dimensional image of a microstructure material with high reproducibility can be realized by using a trained GAN.

[0077] (3) In the generated structure evaluation process executed by the evaluation index calculation unit 121, an evaluation index relating to the geometric shape difference between the actual structure of the microstructure material and the structure of the model generated by the AI ​​calculation unit 11 is calculated for each of a plurality of positions in the microstructure material (step S40). Based on the evaluation index calculated for each position in this way, the learning progress determination unit 122 determines the learning progress of the AI ​​(step S60). This allows the learning progress of the AI ​​to be accurately determined.

[0078] (4) In the learning process of the AI ​​calculation unit 11, position information to be used for learning of the generator 110 and the discriminator 111 is determined based on the acquisition position of the first learning image or the second learning image in the microstructure material, or on the position after changing the acquisition position of the first learning image to another position within the microstructure material (steps S110, S130). Then, the generator 110 and the discriminator 111 are trained by combining the acquired first learning image and second learning image with the determined position information (steps S180 to S210, S270 to S330). In this way, when the number of cross-sectional images of the actual structure that can be used as first learning images is insufficient, the number can be virtually increased and the learning process can be performed.

[0079] (5) Furthermore, in the learning process of the AI ​​calculation unit 11, the number of data sets of the combinations of two-dimensional images and positional information corresponding to a specific positional region designated as a priority reproduction region within the microstructure material may be set relatively larger than the number of data sets of two-dimensional images and positional information corresponding to other positional regions (steps S170 to S180, S260 to S270). In this way, AI learning can be focused on an arbitrary positional region, enabling a three-dimensional structure model to be generated with a high degree of reproduction.

[0080] In the above-described embodiment, the position information of the actual position data 23b combined with the cross-sectional image of the actual microstructure material represented by the teacher image data 23a or a partially cut-out image of this cross-sectional image, and the position information of the cross-sectional image of the three-dimensional structure model represented by the generated structure data 24 or a partially cut-out image of this cross-sectional image, may specify coordinate values ​​for all three coordinate axes (x-axis, y-axis, z-axis) that identify three-dimensional positions within the microstructure material. In this case, the coordinate values ​​of each axis may be defined in any format, such as a normalized value with the maximum value of each axis set to 1. Alternatively, the position information may be defined using only one or two of the three coordinate axes, with the other axes omitted. In the present invention, position information expressed in any format can be used, taking into account the distribution of structural features possessed by the microstructure material.

[0081] The present invention is not limited to the above-described embodiment, and can be implemented using any components without departing from the spirit of the present invention.

[0082] The above-described embodiments and modifications are merely examples, and the present invention is not limited to these details as long as the features of the invention are not impaired. Furthermore, although various embodiments and modifications have been described above, the present invention is not limited to these details. Other aspects that can be considered within the scope of the technical idea of ​​the present invention are also included within the scope of the present invention. [Explanation of symbols]

[0083] 1...control unit, 2...storage unit, 3...memory, 4...operation input device, 5...display device, 6...network device, 7...bus, 11...AI calculation unit, 110...generator, 111...discriminator, 12...AI learning unit, 120...learning processing unit, 121...evaluation index calculation unit, 122...learning progress determination unit, 21...generator data, 22...discriminator data, 23a...teacher image data, 23b...actual position data, 24...generated structure data, 25...hyperparameter data, 26...evaluation index data, 100...model generation device

Claims

1. 1. A method for generating a model of a three-dimensional structure from a two-dimensional image of a microstructured material, comprising: The two-dimensional image is combined with positional information of the two-dimensional image in the microstructure material to perform AI learning; A method for generating a model of a microstructured material, which generates the model using the trained AI.

2. 2. The method for generating a model of a microstructured material according to claim 1, The AI ​​is a GAN having a generator that generates the model from the two-dimensional image, and a discriminator that discriminates whether a cross-sectional image of the model generated by the generator is a first two-dimensional image that is a cross-sectional image of the actual microstructure material, or a second two-dimensional image that is a cross-sectional image of the model, A method for generating a model of a microstructure material, in which the generator and the discriminator are trained by combining a first training image cut out from the first two-dimensional image, a second training image cut out from the second two-dimensional image, and the position information.

3. 3. The method for generating a model of a microstructured material according to claim 1 or 2, Calculating evaluation indices relating to the geometric shape difference between the actual structure of the microstructure material and the structure of the model generated by the AI ​​for each of a plurality of positions in the microstructure material; A method for generating a model of a microstructure material, which determines the learning progress of the AI ​​based on the evaluation index calculated for each position.

4. 3. The method for generating a model of a microstructured material according to claim 2, determining the position information used for training the generator and the discriminator based on the acquisition position of the first training image or the second training image in the microstructure material, or a position after changing the acquisition position of the first training image to another position in the microstructure material; A method for generating a model of a microstructure material, the method comprising: training the generator and the discriminator by combining the acquired first training image and the second training image with the determined position information.

5. 3. The method for generating a model of a microstructured material according to claim 1 or 2, A method for generating a model of a microstructured material, in which the number of data sets of the two-dimensional image and the positional information corresponding to a specific positional area specified within the microstructured material is relatively larger than the number of data sets of the two-dimensional image and the positional information corresponding to other positional areas, and the AI ​​is trained using this data set.

6. 1. An apparatus for generating a three-dimensional structural model from a two-dimensional image of a microstructured material, comprising: an AI learning unit that combines the two-dimensional image and position information of the two-dimensional image in the microstructure material to perform AI learning; A model generation device for microstructured materials comprising: an AI calculation unit that generates the model using the AI ​​that has been learned by the learning unit.

7. 7. The apparatus for generating a model of a microstructured material according to claim 6, The AI ​​is a GAN having a generator that generates the model from the two-dimensional image, and a discriminator that discriminates whether a cross-sectional image of the model generated by the generator is a first two-dimensional image that is a cross-sectional image of the actual microstructure material, or a second two-dimensional image that is a cross-sectional image of the model, The AI ​​learning unit is a model generation device for microstructure materials that trains the generator and the discriminator by combining a first learning image cut out from the first two-dimensional image, a second learning image cut out from the second two-dimensional image, and the position information.

8. 8. The apparatus for generating a model of a microstructured material according to claim 6 or 7, an evaluation index calculation unit that calculates evaluation indexes relating to the geometric shape difference between the actual structure of the microstructure material and the structure of the model generated by the AI ​​for each of a plurality of positions in the microstructure material; A model generation device for microstructure materials comprising: a learning progress determination unit that determines the learning progress of the AI ​​by the AI ​​learning unit based on the evaluation index calculated for each position by the evaluation index calculation unit.

9. 9. The apparatus for generating a model of a microstructured material according to claim 8, The AI ​​learning unit determining the position information used for training the generator and the discriminator based on the acquisition position of the first training image or the second training image in the microstructure material, or a position after changing the acquisition position to another position within the microstructure material; A model generation device for microstructure materials that performs training of the generator and the discriminator by combining the acquired first training image and second training image with the determined position information.

10. 8. The apparatus for generating a model of a microstructured material according to claim 6 or 7, The AI ​​learning unit learns the AI ​​by making the number of data sets of the two-dimensional image and the position information corresponding to a specific position area specified within the microstructure material relatively larger than the number of data sets of the two-dimensional image and the position information corresponding to other position areas, in a model generation device for microstructure materials.

Images