A CLIP-based method for reverse engineering properties to microstructures

Abstract

The present invention discloses a method for reverse generation of properties to microstructures based on CLIP, including training and reasoning stages, specifically: using a graph representation to represent a three-dimensional microstructure, training a variational autoencoder to encode and decode the graph representation; using a trained variational autoencoder to extract potential features of the graph representation, using CLIP to extract property features of the properties, calculating the loss between the potential features and the property features, and training using an optimizer according to the loss to obtain trained model parameters. Input the target property into the trained CLIP model to obtain the encoded property features, then input the property features into the decoder of the trained variational autoencoder to decode to obtain the graph representation, and finally convert the graph representation into a three-dimensional structure modeling file. The present invention realizes the reverse generation of properties to microstructures, and makes judgments on the symmetry, connectivity, etc. of the microstructure, so that the quality of the generated microstructure is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of deep learning generation technology and microstructure reverse generation design, and in particular to a CLIP-based property-to-microstructure reverse generation method. Background Art

[0002] Microstructure property analysis is to evaluate the equivalent macroscopic material properties based on microstructure geometric information. The property-driven reverse generative design of microstructures is an inverse process, but the optimization design of geometric microstructures with specific property requirements in a specified space is an NP-hard problem. Microstructure reverse design methods based on deep learning and machine learning are gradually emerging, especially generative models such as variational autoencoders and generative adversarial networks. However, most of the reverse generative design methods based on such models are based on iterative optimization methods, not direct reverse design methods, and the generated microstructures have great defects in terms of connectivity quality. The CLIP (Contrastive Language-Image Pre-Training) model is a multimodal pre-trained neural network. The core idea of ​​the model is to use a large amount of paired data of images and texts for pre-training to learn the alignment relationship between images and texts. Based on this, the present invention is proposed. Summary of the invention

[0003] In view of the deficiencies of the prior art, the object of the present invention is to provide a method for reversely generating properties to microstructures based on CLIP.

[0004] The object of the present invention is achieved through the following technical solutions: a CLIP-based property-to-microstructure inverse generation method, including a training phase and a testing phase;

[0005] The training phase includes:

[0006] Acquire a truss structure data set, wherein the truss structure data set includes graph structure data of a three-dimensional structure and corresponding property information, and represent the truss structure data set using a graph representation;

[0007] Using the graph representation to train an encoder and a decoder of a variational autoencoder;

[0008] The encoder of the trained variational autoencoder is used to extract latent space features represented by all graphs, the property information in the truss structure data set is input into the property extractor in the CLIP model to extract the property features, the loss is calculated for the property features and the latent space features, the loss is optimized by the optimizer to adjust the CLIP model parameters, and the training of the CLIP model is realized;

[0009] The testing phase includes:

[0010] Input the target mechanical properties into the property extractor in the CLIP model to obtain the property characteristics;

[0011] Input the property features into the decoder of the trained variational autoencoder to obtain the decoded graph representation;

[0012] A three-dimensional structure is modeled in 3D modeling software according to the decoded graphic representation to obtain a STEP model file of the three-dimensional structure.

[0013] Furthermore, the training phase is specifically as follows:

[0014] Acquire a public truss structure data set, wherein the truss structure data set includes graph structure data of a three-dimensional structure and a mechanical property stiffness vector w corresponding to the three-dimensional structure;

[0015] The truss structure data set is represented by a graph, the connection nodes of the graph structure data in the data set are numbered in sequence, and the adjacency matrix of the graph is obtained according to whether the nodes are connected , where n is the number of nodes, a matrix element of 0 indicates that the nodes are not connected, a matrix element of 1 indicates that the nodes are connected, and the three-dimensional coordinates of the nodes are node feature representations recorded as ;

[0016] Representing data in graphs Input into the encoder E of the variational autoencoder to encode the graph representation and split the encoded features into mean and variance , the mean and variance are resampled to obtain the sampled latent space feature z, and the latent space feature z is input into the decoder D to obtain the decoded feature, i.e., the graph structure data ;

[0017] The resampling is performed in the following manner:

[0018] ;

[0019] Where u represents the standard normal distribution;

[0020] ;

[0021] in, It means that u satisfies the standard normal distribution with mean 0 and variance 1;

[0022] The loss is calculated for the variational autoencoder, and the loss formula is as follows:

[0023] ;

[0024] Among them, MSE is the mean square error loss function, KL is the Kullback-Leibler divergence loss function, N is the normal distribution, is the weight coefficient between the losses, and the SGD optimizer is used to train the losses to optimize the network parameters;

[0025] The encoder E of the trained variational autoencoder is used to extract the latent space features of the structure, and the mechanical properties w are input into the property extractor in the CLIP model to extract the property features and train the CLIP model.

[0026] Furthermore, the encoder E and the decoder D are both composed of five layers of graph convolution and activation function layers, and each layer of graph convolution is connected to the activation function layer.

[0027] Furthermore, the activation function adopts a ReLU activation function.

[0028] Furthermore, the training CLIP model is specifically:

[0029] The encoder E of the trained variational autoencoder is used to extract the latent space features of the structure, and the mechanical properties w are input into the property extractor in the CLIP model to extract the property features, where the property extractor uses a two-layer fully connected network and an activation function, and each layer of full connection is followed by an activation function;

[0030] Calculate the loss of the CLIP model. Assume that there are n sample pairs in a training batch, that is, n pairs of graph representation-property pairs, the graph representation is encoded as z, the property is encoded as f, and calculate the similarity matrix ,element Indicates The figure shows The similarity between properties; the loss formula is as follows:

[0031] ;

[0032] ;

[0033] Where label = 0, 1, 2, ... n-1, is the cross entropy loss, axis=0 means calculating the loss for all properties corresponding to each graph representation, and axis=1 means calculating the loss for all graph representations corresponding to each property;

[0034] AdamW optimizer is used to optimize the loss and adjust the CLIP model parameters.

[0035] Furthermore, the activation function adopts a ReLU activation function.

[0036] Furthermore, the testing phase is specifically as follows:

[0037] Input the target mechanical properties into the property extractor in the CLIP model to extract the property features;

[0038] Input the property features into the trained decoder D to obtain the decoded graph representation data ;

[0039] According to the graph, the data is represented Obtaining the STEP model file of the structure includes the following steps:

[0040] a. Representing data based on graphs Get the connection relationship of the nodes, set the initial radius of the truss rod to 0.1, and the connection relationship is in the form of the starting point coordinates and the end point coordinates of each rod;

[0041] b. Using the pythonOCC library, the entity structure is obtained according to the connection relationship and radius conversion, and the volume fraction is calculated using the functions in the pythonOCC library;

[0042] c. Use the binary search method to adjust the radius of the truss rod according to the calculated volume fraction, and repeat step b according to the new radius until the difference between the obtained volume fraction and the volume fraction of the public truss structure dataset is less than the set threshold, and the final STEP file of the solid structure is obtained.

[0043] Furthermore, the volume fraction of the public truss structure data set is 0.15, and the threshold is set to 0.01.

[0044] The present invention also provides an electronic device, comprising a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the CLIP-based property-to-microstructure inverse generation method.

[0045] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the CLIP-based property-to-microstructure inverse generation method is implemented.

[0046] The beneficial effects of the present invention are as follows: the present invention is based on a method combining the CLIP method and graph structure data, and utilizes the CLIP method to align the structure space and the property space, so as to realize direct reverse generation of microstructures. Meanwhile, the use of graph structure data to represent the microstructure can ensure the validity and quality of the generated structure when generating the structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 is a training flow chart of the CLIP-based property-to-microstructure reverse generation method according to an embodiment of the present application;

[0049] Figure 2 This is a test flow chart of the CLIP-based property-to-microstructure reverse generation method according to an embodiment of the present application;

[0050] Figure 3 It is a three-dimensional structure display diagram obtained by converting the figure representation in the embodiment of the present application;

[0051] Figure 4 It is a three-dimensional structure display diagram generated by the method of the present invention in the embodiment of the present application;

[0052] Figure 5 A schematic diagram of the structure of the CLIP-based property-to-microstructure reverse generation device of the present invention. DETAILED DESCRIPTION

[0053] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0054] The terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "the" and "the" used in the present invention and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0055] It should be understood that although the terms first, second, third, etc. may be used in the present invention to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0056] The present invention is described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the features of the following embodiments and implementations can be combined with each other.

[0057] The embodiment of the present invention provides a CLIP-based property-to-microstructure reverse generation method, which includes a training phase and a testing phase, as follows:

[0058] like Figure 1 As shown in the figure, the detailed steps of the training phase are:

[0059] Step 1.1, Dataset acquisition: Get the public truss structure dataset Truss Dataset (https: / / www.research-collection.ethz.ch / handle / 20.500.11850 / 618078). This dataset has 965736 samples, each of which includes the graph structure data of the truss three-dimensional structure: adjacency matrix and node coordinates and the mechanical property stiffness matrix corresponding to the structure. The structure volume fraction of this dataset is 0.15. According to the volume fraction, adjacency matrix and node coordinates, the STEP model file of the structure can be modeled in the 3D modeling software, such as Figure 3 The figure shows the three-dimensional structure obtained by converting the samples in the data set in this embodiment.

[0060] Step 1.2, Dataset representation: Use the graph structure dataset to number the connected nodes of the graph structure data in the truss structure dataset in step 1.1 in sequence, and obtain the adjacency matrix of the graph based on whether the nodes are connected , where n is the number of nodes. In this embodiment, n=27. A matrix element of 0 indicates that the nodes are not connected, and a matrix element of 1 indicates that the nodes are connected. The three-dimensional coordinates of the nodes are node feature representations and are recorded as .

[0061] Step 1.3: Figure 1 As shown, the graph represents the data Input into the encoder E of the variational autoencoder to encode the structure and split the encoded features into mean and variance The feature splitting method is to use the first half of the encoded feature as the mean and the second half as the variance, and then use the mean and variance to resample to obtain the sampled latent space feature z, and finally input it into the decoder D to obtain the decoded feature, that is, the graph representation data .

[0062] Among them, the encoder E and the decoder D are composed of five layers of graph convolution and activation function layers. Each graph convolution is followed by an activation function. The activation function adopts the ReLU activation function in this embodiment; the resampling is performed in the following way, sampling from a standard normal distribution, that is, a normal distribution with a mean of 0 and a variance of 1, and the sampled data is multiplied by the variance Plus the mean Get the resampled feature z:

[0063] ;

[0064] in It is a standard normal distribution, that is, it follows a normal distribution with a mean of 0 and a variance of 1.

[0065] ;

[0066] Step 1.4: Calculate the loss of the variational autoencoder described in step 1.3. The loss formula is as follows:

[0067] ;

[0068] Where MSE is the mean square error loss function, KL is the Kullback-Leibler divergence loss function, is the weight coefficient between losses, with values ​​of 0.3, 0.3, and 0.4 respectively. The SGD optimizer is used to train the loss to optimize the network parameters. The learning rate is set to 0.01, the number of iterations is 120, and the training batch size is 64.

[0069] Step 1.5: Represent the data in a graph The input is input into the encoder E of the variational autoencoder trained in step 1.4 to extract the latent space features, and the mechanical property w is input into the property extractor of the CLIP model to extract the property features. The property extractor in the CLIP model uses a two-layer fully connected network and activation function, and each layer of full connection is followed by an activation function ReLU.

[0070] Step 1.6. Calculate the loss of the CLIP model. The loss formula is as follows:

[0071] There are n sample pairs in a training batch, i.e. n pairs of graph representation-property pairs. In this embodiment, n=64. The graph representation is encoded as z and the property is encoded as f. The similarity matrix is ​​calculated. ,element Indicates The figure shows The similarity between the properties.

[0072]

[0073] ;

[0074] Where label=0,1,2,...,63, is the cross entropy loss, axis=0 means that the loss is calculated for all properties corresponding to each graph representation, and axis=1 means that the loss is calculated for all graph representations corresponding to each property. The AdamW optimizer training loss is used to optimize the network parameters, and the learning rate is set to 0.001, the number of iterations is 100, and the training batch size is 64.

[0075] like Figure 2 As shown below, the detailed steps of the testing phase are:

[0076] Step 2.1. Input the target mechanical properties into the property extractor in the CLIP model to extract the property features.

[0077] Step 2.2: Input the property features into the decoder D trained in step 1.3 of the training phase to obtain the decoded graph representation data .

[0078] Step 2.3: Represent data based on graphs And the volume fraction is 0.15, and the STEP model file of the structure is converted using the pythonocc library, as follows:

[0079] Step 2.3.1: First, according to the graph data The coordinates of the nodes and the connection relationship between the coordinate points are obtained, and the radius of the initial truss rod is set to 0.1. The connection relationship is in the form of the connection start point coordinates and end point coordinates of each rod.

[0080] Step 2.3.2: Use the pythonOCC library to obtain the solid structure according to the connection relationship and radius conversion, and use the function of calculating volume fraction in the pythonOCC library to calculate the volume fraction.

[0081] Step 2.3.3, adjust the radius using the binary search method according to the calculated volume fraction, and continue step 2.3.2 using the new radius until the difference between the volume fraction and 0.15 is less than the set threshold. In this embodiment, the threshold is 0.01, and the final STEP file of the solid structure is obtained.

[0082] like Figure 4As shown, it is a display of the three-dimensional structure reversely generated during the testing phase of the method of the present invention in this embodiment.

[0083] Corresponding to the aforementioned embodiment of a method for reversely generating properties from microstructures based on CLIP, the present invention also provides an embodiment of an apparatus for reversely generating properties from microstructures based on CLIP.

[0084] like Figure 5 As shown, an embodiment of the present invention provides a CLIP-based property-to-microstructure reverse generation device, which includes one or more processors for implementing a CLIP-based property-to-microstructure reverse generation method in the above embodiment.

[0085] An embodiment of the CLIP-based property-to-microstructure reverse generation device of the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. The device embodiment can be implemented through software, or through hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located reading the corresponding computer program instructions in the non-volatile memory into the internal memory for execution. From the hardware level, if Figure 4 As shown, it is a hardware structure diagram of any device with data processing capability where the device for reverse generation of microstructure based on CLIP is located, except Figure 4 In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiments is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0086] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0087] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0088] An embodiment of the present invention also provides an electronic device, including a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement a CLIP-based property-to-microstructure inverse generation method as described in any embodiment.

[0089] The embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, a CLIP-based property-to-microstructure reverse generation method described in any embodiment is implemented. The computer-readable storage medium can be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium can also be any device with data processing capabilities, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), an SD card, a flash card (Flash Card), etc. equipped on the device. Further, the computer-readable storage medium can also include both an internal storage unit of any device with data processing capabilities and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and can also be used to temporarily store data that has been output or is to be output.

[0090] Those skilled in the art will readily appreciate other embodiments of the present application after considering the description and practicing the contents disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application. The description and examples are intended to be exemplary only.

[0091] It should be understood that the present application is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof.

Claims

1. A CLIP-based method for reverse engineering properties to microstructures, characterized in that: It includes training phase and testing phase; The training phase includes: Acquire a truss structure data set, wherein the truss structure data set includes graph structure data of a three-dimensional structure and corresponding property information, and represent the truss structure data set using a graph representation; Using the graph representation to train an encoder and a decoder of a variational autoencoder; The encoder of the trained variational autoencoder is used to extract latent space features represented by all graphs, the property information in the truss structure data set is input into the property extractor in the CLIP model to extract the property features, the loss is calculated for the property features and the latent space features, the loss is optimized by the optimizer to adjust the CLIP model parameters, and the training of the CLIP model is realized; The testing phase includes: Input the target mechanical properties into the property extractor in the CLIP model to obtain the property characteristics; Input the property features into the decoder of the trained variational autoencoder to obtain the decoded graph representation; A three-dimensional structure is modeled in 3D modeling software according to the decoded graphic representation to obtain a STEP model file of the three-dimensional structure.

2. A CLIP-based property-to-microstructure inverse generation method according to claim 1, characterized in that: The training phase is specifically as follows: Acquire a public truss structure data set, wherein the truss structure data set includes graph structure data of a three-dimensional structure and a mechanical property stiffness vector w corresponding to the three-dimensional structure; The truss structure data set is represented by a graph, the connection nodes of the graph structure data in the truss structure data set are numbered in sequence, and the adjacency matrix of the graph is obtained according to whether the nodes are connected , where n is the number of nodes, a matrix element of 0 indicates that the nodes are not connected, a matrix element of 1 indicates that the nodes are connected, and the three-dimensional coordinates of the nodes are node feature representations recorded as ; Representing data in graphs Input into the encoder E of the variational autoencoder to encode the graph representation and split the encoded features into mean and variance , the mean and variance are resampled to obtain the sampled latent space feature z, and the latent space feature z is input into the decoder D to obtain the decoded feature, i.e., the graph structure data ; The resampling is performed in the following manner: ； Where u represents the standard normal distribution; ； in, It means that u satisfies the standard normal distribution with mean 0 and variance 1; The loss is calculated for the variational autoencoder, and the loss formula is as follows: ； Among them, MSE is the mean square error loss function, KL is the Kullback-Leibler divergence loss function, N is the normal distribution, is the weight coefficient between the losses, and the SGD optimizer is used to train the losses to optimize the network parameters; The encoder E of the trained variational autoencoder is used to extract the latent space features of the structure, and the mechanical properties w are input into the property extractor in the CLIP model to extract the property features and train the CLIP model.

3. The CLIP-based property-to-microstructure inverse generation method according to claim 2, characterized in that: The encoder E and decoder D are both composed of five layers of graph convolution and activation function layers, and each layer of graph convolution is connected to the activation function layer.

4. The CLIP-based property-to-microstructure inverse generation method according to claim 3, characterized in that: The activation function adopts the ReLU activation function.

5. The CLIP-based property-to-microstructure inverse generation method according to claim 2, characterized in that: The training CLIP model is specifically as follows: The encoder E of the trained variational autoencoder is used to extract the latent space features of the structure, and the mechanical property w is input into the property extractor in the CLIP model to extract the property features, wherein the property extractor adopts a two-layer fully connected network and an activation function, and each layer of full connection is followed by an activation function; Calculate the loss of the CLIP model. Assume that there are n sample pairs in a training batch, that is, n pairs of graph representation-property pairs, the graph representation is encoded as z, the property is encoded as f, and calculate the similarity matrix ,element Indicates The figure shows The similarity between properties; the loss formula is as follows: ； ； Where label = 0, 1, 2, ... n-1, is the cross entropy loss, axis=0 means calculating the loss for all properties corresponding to each graph representation, and axis=1 means calculating the loss for all graph representations corresponding to each property; AdamW optimizer is used to optimize the loss and adjust the CLIP model parameters.

6. The CLIP-based property-to-microstructure inverse generation method according to claim 5, characterized in that: The activation function adopts the ReLU activation function.

7. The CLIP-based property-to-microstructure inverse generation method according to claim 1, characterized in that: The testing phase is specifically as follows: Input the target mechanical properties into the property extractor in the CLIP model to extract the property features; Input the property features into the trained decoder D to obtain the decoded graph representation data ; According to the graph, the data is represented Obtaining the STEP model file of the structure includes the following steps: a. Representing data based on graphs Get the connection relationship of the nodes, set the initial radius of the truss rod to 0.1, and the connection relationship is in the form of the starting point coordinates and the end point coordinates of each rod; b. Using the pythonOCC library, the entity structure is obtained according to the connection relationship and radius conversion, and the volume fraction is calculated using the functions in the pythonOCC library; c. Use the binary search method to adjust the radius of the truss rod according to the calculated volume fraction, and repeat step b according to the new radius until the difference between the obtained volume fraction and the volume fraction of the public truss structure dataset is less than the set threshold, and the final STEP file of the solid structure is obtained.

8. The CLIP-based property-to-microstructure inverse generation method according to claim 7, characterized in that: The volume fraction of the public truss structure dataset is 0.15, and the threshold is set to 0.

01.

9. An electronic device, comprising a memory and a processor, characterized in that: The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement a CLIP-based property-to-microstructure inverse generation method as described in any one of claims 1-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a CLIP-based property-to-microstructure inverse generation method as described in any one of claims 1 to 8 is implemented.

Images