A material structure design method and system based on multi-modal contrast learning

Abstract

This invention discloses a material structure design method based on multimodal contrastive learning, comprising the following steps: constructing an image-performance pairing dataset; training an image encoder and a performance encoder through cross-modal contrastive learning to map microstructure images and effective elastic performance matrices to a shared latent space, such that the embedded features of mutually matching image-performance pairs in this latent space are close to each other, while those of mismatched image-performance pairs are far apart, thereby obtaining semantically aligned image embedding features and performance embedding features; and performing one or more downstream design tasks using the semantically aligned image embedding features and / or performance embedding features. This invention provides a material structure design method and system based on multimodal contrastive learning, which, by constructing a semantically aligned joint embedding space, provides enhanced feature representations for downstream surrogate modeling and generative design, thereby accelerating the design process and improving design quality.

Description

Technical Field

[0001] This invention relates to the fields of materials design, artificial intelligence and computer vision, and in particular to a material structure design method and system based on multimodal contrastive learning. Background Technology

[0002] Material structures are a class of artificial materials that acquire extraordinary macroscopic mechanical properties through their microstructure geometry, such as negative Poisson's ratio structures and compression-torsion coupled structures. Traditional material structure design relies on physical intuition, bionics, or computationally intensive topology optimization and high-throughput screening. When faced with complex and multi-objective design requirements, it is often difficult to simultaneously ensure extensive exploration of the design space, satisfaction of multiple constraints, and guarantee of manufacturing feasibility.

[0003] In recent years, data-driven design methods driven by artificial intelligence have brought new possibilities to this field. Existing technologies typically employ two parallel approaches: (1) Forward prediction: Training surrogate models (such as multilayer perceptrons and convolutional neural networks) to directly map microstructure images to their macroscopic mechanical properties, replacing expensive finite element simulations. (2) Inverse generation: Using generative models such as generative adversarial networks or variational autoencoders to generate new microstructures based on target performance requirements.

[0004] However, these existing methods typically train models directly from raw data inputs (such as image pixels or raw performance parameters), failing to effectively utilize the inherent structure-performance relationships implicit in the data. This results in slow model training convergence, limited prediction accuracy, and difficulty in guaranteeing the diversity and physical feasibility of generated structures during inverse generation. Summary of the Invention

[0005] In view of the aforementioned shortcomings of existing technologies, the technical problem to be solved by this invention is that existing material structure design methods fail to effectively utilize the inherent structure-performance relationships implicit in the data. This leads to slow model training convergence speed, limited prediction accuracy, and difficulty in ensuring the diversity and physical feasibility of generated structures during reverse generation. Therefore, this invention provides a material structure design method and system based on multimodal contrastive learning. By constructing a semantically aligned joint embedding space, it provides enhanced feature representations for downstream proxy modeling and generative design, thereby accelerating the design process and improving design quality.

[0006] To achieve the above objectives, this invention provides a material structure design method based on multimodal contrastive learning, comprising the following steps:

[0007] Step S1: Construct an image-performance pairing dataset, which contains multiple microstructure images and an effective elastic performance matrix corresponding to each microstructure image;

[0008] Step S2: Train an image encoder and a performance encoder through cross-modal contrastive learning to map the microstructure image and the effective elastic performance matrix to a shared latent space, such that the embedded features of mutually matching image-performance pairs in the latent space are close to each other, while the embedded features of mismatched image-performance pairs are far from each other, thereby obtaining semantically aligned image embedding features and performance embedding features.

[0009] Step S3: Perform one or more downstream design tasks using semantically aligned image embedding features and / or performance embedding features.

[0010] Furthermore, the downstream design tasks include forward prediction tasks and backward prediction tasks; among them,

[0011] The forward prediction task involves inputting image embedding features into a surrogate model, which is then trained to output a prediction resilience performance matrix corresponding to the input image embedding features.

[0012] The inverse generation task involves using the embedded features of the target performance as a conditional signal and inputting them into a generative model. The generative model is then trained to generate a microstructure image that conforms to the target performance based on the conditional signal.

[0013] Furthermore, the generative model is a conditional diffusion model, and its generation process includes: starting from random noise, and gradually restoring the microstructure image through iterative denoising under the guidance of the target performance embedded features.

[0014] Furthermore, the contrastive learning in step S2 employs a two-stage training strategy, specifically including:

[0015] The first stage is used to optimize only the cross-modal contrastive loss to establish a basic mapping between image embedding features and performance embedding features;

[0016] The second stage is used to enhance the robustness and discriminativeness of feature representations within the same modality by increasing intramodal contrast loss while optimizing cross-modal contrast loss.

[0017] Furthermore, constructing the image-performance pairing dataset involves generating microstructures with different target mechanical behaviors through topology optimization, and calculating their corresponding effective elastic performance matrices through numerical homogenization.

[0018] Furthermore, it also includes the following steps:

[0019] Construct an intelligent agent system based on a large language model, which integrates an image encoder, a performance encoder, an agent model, and a generative model;

[0020] Receive natural language or image queries from the user;

[0021] Parse the query, determine the user intent, and automatically call the corresponding model to execute the task in step S3 based on the intent;

[0022] The execution results will be fed back to the user in natural language and / or visual form.

[0023] Another preferred embodiment of the present invention provides a material structure design system based on multimodal contrastive learning, comprising the following modules:

[0024] The data construction module is used to build an image-performance paired dataset, which contains multiple microstructure images and an effective elastic performance matrix corresponding to each microstructure image;

[0025] The cross-modal alignment module is used to train an image encoder and a performance encoder through cross-modal contrastive learning to map the microstructure image and the effective elastic performance matrix to a shared latent space, such that the embedded features of mutually matching image-performance pairs in the latent space are close to each other, while the embedded features of mismatched image-performance pairs are far from each other, thereby obtaining semantically aligned image embedding features and performance embedding features.

[0026] Downstream task module, used to perform one or more downstream design tasks using semantically aligned image embedding features and / or performance embedding features.

[0027] Furthermore, it also includes an intelligent interaction agent module, which integrates a large language model and interacts with the cross-modal alignment module and downstream task modules to parse natural language or image instructions input by the user and coordinate the various modules to execute corresponding design tasks.

[0028] Another preferred embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the material structure design method based on multimodal contrastive learning as described above.

[0029] Another preferred embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the material structure design based on multimodal contrastive learning as described above.

[0030] Technical effect

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] 1. Improved Model Performance: The semantically aligned embedding space constructed through contrastive learning provides high-quality input features and conditional signals for downstream surrogate and generative models. Experiments demonstrate that the surrogate model based on this provides higher prediction accuracy (Ri).2 (Reaching 0.989), the diffusion model generates better quality data and converges faster.

[0033] 2. Accelerate training convergence: Pre-aligned embedding features contain the inherent relationship between structure and performance, so that downstream models do not need to learn these complex relationships from scratch, thereby significantly reducing the time and amount of data required for training.

[0034] 3. Achieving efficient bidirectional design: This invention constructs a closed-loop bidirectional design framework that can quickly predict performance from structural images and generate diverse, physically reasonable candidate structures from the target performance, greatly improving design efficiency.

[0035] 4. Promote intelligent human-computer interaction design: The integrated intelligent agent system enables materials scientists to design through natural language dialogue without having to master complex modeling and simulation software operations, thus lowering the design threshold and opening up new avenues for automated and intelligent materials discovery.

[0036] The following will further explain the concept, specific structure, and technical effects of the present invention in conjunction with the accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Attached Figure Description

[0037] Figure 1 This is a schematic diagram illustrating the microstructure design, optimization, and performance analysis of an embodiment of the present invention;

[0038] Figure 2 This is a diagram illustrating the cross-modal contrastive learning network architecture and training strategy according to an embodiment of the present invention.

[0039] Figure 3 This is a network architecture diagram of the proxy model and conditional diffusion model in an embodiment of the present invention;

[0040] Figure 4 This is a visualization of the dataset features in an embodiment of the present invention;

[0041] Figure 5 This is a schematic diagram of the cross-modal bidirectional retrieval results according to an embodiment of the present invention;

[0042] Figure 6 This is a schematic diagram illustrating the surrogate model prediction and diffusion model generation process in an embodiment of the present invention;

[0043] Figure 7 A diversity and attribute conformity verification diagram for the diffusion model generated in this embodiment of the invention;

[0044] Figure 8 This is a comparison chart of ablation experiment results from an embodiment of the present invention;

[0045] Figure 9This is a diagram showing the macroscopic mechanical verification results of an embodiment of the present invention;

[0046] Figure 10 This is a flowchart of the intelligent agent system according to an embodiment of the present invention. Detailed Implementation

[0047] To make the technical problems to be solved, the technical solutions, and the beneficial effects of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0048] In the following description, specific details, such as particular internal procedures and techniques, are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will appreciate that the invention may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of the invention with unnecessary detail.

[0049] This invention provides a material structure design method based on multimodal contrastive learning, comprising the following steps:

[0050] Step S1: Construct an image-performance pairing dataset. This dataset contains multiple microstructure images and the corresponding effective elastic performance matrix for each microstructure image. This step generates a large number of microstructure images with different target mechanical behaviors and their corresponding effective elastic performance matrices through topology optimization and numerical homogenization, establishing a structure-performance pairing database. The microstructure images are topological images of two-dimensional representative volumetric units, and the effective elastic performance matrix is ​​a two-dimensional elastic stiffness matrix. Using density-based topology optimization combined with numerical homogenization, a diverse library of microstructures is generated. These microstructures, after systematic design, can exhibit three different mechanical behaviors: negative Poisson's ratio (NPR), compression-torsional coupling (CTC), and a combination of both (e.g., ...). Figure 1 (As shown in b). The optimization framework is based on the Solid Isotropic Material Penalty Model (SIMP) method, under periodic boundary conditions and volume constraints. The design domain is a 2D representative volume element (RVE), which is discretized as... Each element is a plane stress element, and each element is assigned a relative density design variable. To enhance sample diversity, this embodiment of the invention started with 70 different initial material distributions and performed systematic data augmentation: solid-void inversion (×2), four-grid shuffling and rearrangement (×24), and variations of three optimization objectives (×3). This resulted in a total of 70×2×24×3=10,080 design topologies.

[0051] Optimize problem description

[0052]

[0053] in, It is the effective elastic tensor component obtained through numerical homogenization; , and These represent the global stiffness matrix, displacement vector, and force vector, respectively. and The weighting coefficients controlling the relative importance of negative Poisson's ratio (NPR) and compression-torsion coupling (CTC) behavior are shown in the table below.

[0054]

[0055] Promotes stretching behavior. The aim is to achieve tension-shear coupling because of the elastic constant. and Describes normal stress , With shear strain The coupling between them. When such a two-dimensional unit cell is stretched along the third dimension into a three-dimensional cylinder, this in-plane tension-shear coupling manifests as the desired compression-torsion effect. This helps to promote the formation of the topology in the early iterations, and its penalty term increases with the number of iterations. The increase in [something] decreases exponentially. Periodic boundary conditions ensure the periodicity of the unit cell (e.g., [something]). Figure 1 As shown in a), volume constraints limit the amount of material used to 50% of the design domain.

[0056] Numerical homogenization and sensitivity analysis. Effective elastic properties. The density distribution is calculated using a numerical homogenization method based on finite element analysis. For each two-dimensional periodic microstructure, the strain is measured in three independent unit tests. Next, we solve for the displacement field at the microscale. The components of the homogenized elastic tensor are:

[0057]

[0058] in It is the area of ​​a single cell. It is a periodic fluctuating strain field. It is a local elasticity tensor defined using SIMP interpolation:

[0059]

[0060] solid modulus Punishment factor , To avoid numerical singularities; Poisson's ratio of solid and void phases. Both are 0.45. It is the isotropic linear elastic matrix for plane stress problems under unit Young's modulus:

[0061]

[0062] The sensitivity of the objective function to the design variables is derived analytically:

[0063]

[0064] The sensitivity of the homogenization tensor is:

[0065]

[0066] Using a radius of A sensitivity filter (relative to the unit cell size) is used to suppress checkerboard patterns.

[0067] Optimization algorithm. Design variables are updated using the Optimization Criterion (OC) method. The Lagrangian function is:

[0068]

[0069] Density updates follow:

[0070]

[0071] in

[0072]

[0073] and Consistent with volume constraints. Movement limit is... The numerical damping coefficient is .

[0074] For each optimization objective, this embodiment of the invention performed topology optimization on 3,360 initial configurations, resulting in a total of 3,360 × 3 = 10,080 independent microstructures. All optimizations were run for a maximum of 50 iterations, which was sufficient to achieve stable convergence and obtain well-defined topologies.

[0075] All microstructures are stored in a uniform format: each sample contains a 100×100 density matrix. The corresponding 3×3 effective elastic tensor The dataset includes a classification label indicating its mechanical behavior (NPR / CTC / Mixed), and 3,360 NPR microstructures, 3,360 CTC microstructures, and 3,360 combined behavior microstructures. This database supports downstream cross-modal contrastive learning tasks, including image-attribute joint embedding, performance prediction, and topology generation.

[0076] Direction-dependent properties are represented through coordinate transformations. Flexibility matrix. By angle Rotate to obtain , where the rotation matrix By direction cosine and definition:

[0077]

[0078] The directional modulus is thus derived: Young's modulus Poisson's ratio and shear modulus (like Figure 1 (as shown in c).

[0079] Step S2: Through cross-modal contrastive learning, train an image encoder and a performance encoder to map the microstructure image and the effective elastic performance matrix to a shared latent space, such that the embedded features of mutually matching image-performance pairs in this latent space are close to each other, while those of mismatched image-performance pairs are far apart, thereby obtaining semantically aligned image embedding features and performance embedding features. This step specifically includes:

[0080] An image encoder and a performance encoder are constructed to extract feature vectors from the microstructure image and the elastic performance matrix, respectively.

[0081] A contrastive learning strategy is adopted, using paired structure-performance samples as positive samples and unpaired samples as negative samples to train the image encoder and performance encoder. This makes the feature vector distance of positive sample pairs in the shared latent space close, while the feature vector distance of negative sample pairs widens, thereby obtaining semantically aligned image embedding features and performance embedding features.

[0082] Preferably, the contrastive learning strategy employs a two-stage training method: the first stage optimizes only the cross-modal contrastive loss to establish a basic mapping between modalities; the second stage incorporates the intra-modal contrastive loss to enhance the robustness and discriminativeness of the feature representation.

[0083] Specifically, the following steps are included:

[0084] 1. Encoder Architecture and Feature Processing

[0085] To map the microstructure image and its elastic stiffness tensor to a shared latent space, embodiments of the present invention design a corresponding encoder architecture (such as...). Figure 2 (As shown in a and b). Both encoders project their respective inputs into a common 128-dimensional latent space, thereby achieving cross-modal alignment through contrastive learning.

[0086] Image Encoder: This embodiment of the invention uses ResNet18 as the backbone network because it achieves the best balance between computational efficiency and feature extraction capability for 100×100 pixel grayscale microstructure images. Through transfer learning, this embodiment adapts the pre-trained ResNet18 weights and modifies the first convolutional layer to accept single-channel (grayscale) input. The resulting 512-dimensional feature vector is then passed through two fully connected layers with ReLU activation functions (512→128) to generate an image embedding. .

[0087] Attribute Encoder: A 3×3 stiffness matrix is ​​flattened into a 9-dimensional vector and fed into a four-layer multilayer perceptron (MLP), with its dimensions progressively expanding (9→32→64→128→128). All hidden layers use the ReLU activation function, ultimately generating attribute embeddings aligned with the image feature space. .

[0088] For a containing The training batch of each sample, the embedding vector is processed... Normalization process:

[0089]

[0090] These normalized embedding vectors are arranged into a batch matrix. and ,in To perform intra-modal contrastive learning to obtain enhanced features. and In this embodiment of the invention, isotropic Gaussian noise is added to the normalized features. Then normalize again. Noise scale It was selected based on the verification performance.

[0091] 2 Training Strategy and Loss Function

[0092] The similarity between normalized feature pairs is calculated using cosine similarity scaled by the temperature parameter τ:

[0093]

[0094] in ( represent or ) represents the original feature, It is its noise-enhanced version. For each mode, and A positive sample pair is formed, while all other samples in the same batch are considered negative samples. This design aims to simultaneously promote consistency within a modality and discriminability between samples.

[0095] Each similarity matrix is ​​optimized using InfoNCE loss (e.g., ...). Figure 2 (as shown in c)

[0096] in This refers to the batch size. The total loss is a weighted combination of the three components:

[0097] .

[0098] To train the cross-modal encoder, this embodiment of the invention employs a course learning strategy, which is divided into two training phases ( Figure 2 d) Gradually introduce more complex objectives:

[0099] Phase 1 (Rounds 1-30): Cross-modal alignment is performed only. This embodiment of the invention optimizes only the contrast loss between the matched image and the stiffness matrix features (in formula (13)). For each microstructure, its image and corresponding stiffness matrix are treated as a positive sample pair, pulled closer to each other in the latent space, while mismatched sample pairs are pushed away. This stage establishes the basic cross-modal mapping.

[0100] Phase 2 (Rounds 31-100): Joint Alignment. Based on the cross-modal objective, this embodiment of the invention activates the intra-modal contrast loss, with weights set in formula (13). For each modality, the original feature and its noise-enhanced version ( These samples form positive sample pairs and are simultaneously compared with features from other samples. This dual objective enhances intramodal robustness and inter-sample discriminative power.

[0101] Referring to previous comparative learning studies, the embodiments of the present invention are set up To sharpen the similarity distribution while maintaining training stability. Two encoders were trained end-to-end for 100 epochs using the Adam optimizer, with a learning rate of... Exponential decay rate and Batch size .

[0102] 3. Cross-modal retrieval

[0103] After training, this embodiment of the invention directly evaluates the quality of the learned representation through bidirectional cross-modal retrieval: given a query (image or stiffness matrix) from one modality, the task is to retrieve the corresponding instance of another modality from the database. This capability supports two practical application flows: (1) image to attribute—retrieving stiffness matrix candidates based on microstructure image queries; (2) attribute to image—retrieving microstructure images that match the target stiffness specification.

[0104] In form, given a query feature (Image or stiffness matrix) and a pre-computed database , The retrieval process consists of three steps. First, the query is encoded and normalized: Secondly, calculation Compared with all pre-computed and normalized target modal features in the database The system calculates the cosine similarity between the two pairs of strings. Finally, it returns the top pairs in descending order of similarity. 100 entries. The search results reflect the semantic alignment between visual topology and mechanical properties achieved through contrastive learning.

[0105] Step S3: Utilize semantically aligned image embedding features and / or performance embedding features to perform one or more downstream design tasks. Downstream tasks include:

[0106] Forward prediction task: Input image embedding features into a lightweight surrogate model, which is trained to output a corresponding prediction elasticity performance matrix based on the input image embedding features.

[0107] Inverse generation task: The target performance embedded features are used as conditional signals and input into a conditional diffusion model. The conditional diffusion model is trained to iteratively denoise random noise based on the conditional signals to generate a microstructure image that conforms to the target performance.

[0108] Proxy Model and Diffusion Generator

[0109] Based on the unified latent representation obtained through contrastive learning, this invention develops two downstream tasks: a lightweight surrogate model for fast attribute prediction and a conditional diffusion model for generating diverse microstructures. The surrogate model performs a one-to-one mapping from image representation to attributes, while the diffusion model achieves one-to-many generation—that is, generating multiple different microstructures for a given attribute embedding under inherent uncertainty. These two complementary tasks jointly leverage aligned cross-modal embeddings, achieving high accuracy in forward prediction and output diversity in inverse design.

[0110] Proxy model for image-to-attribute mapping

[0111] To achieve rapid prediction of attributes from microstructure images, this invention constructs a lightweight multilayer perceptron (MLP) to predict attributes from pre-trained image features. The six independent components of the stiffness matrix are regressed. The network consists of an input layer (128 neurons), three hidden layers with 70, 40, and 20 neurons respectively, and an output layer (6 neurons). Each hidden layer is followed by a ReLU activation function. Figure 4 a). This progressive dimensionality reduction captures hierarchical feature abstractions while maintaining computational efficiency.

[0112] The model was trained for 100 epochs using the Adam optimizer, with a learning rate of [missing value]. attenuation rate and The loss function is mean squared error (MSE). This compact architecture directly benefits from the semantic alignment features obtained during contrast pre-training, eliminating the need to learn representations from scratch.

[0113] Conditional Diffusion Model for Attribute-to-Image Generation

[0114] To achieve "one-to-many" generation—that is, generating multiple different microstructures that satisfy the same target attribute under uncertainty—this invention implements a conditional denoising diffusion probability model (DDPM). This model uses attribute embedding... As a conditional signal, and through an iterative denoising process starting from Gaussian noise, a microstructure image is generated (e.g. Figure 4 (as shown in b).

[0115] Diffusion process: Forward diffusion process in Within a time step, the original image It gradually degrades into isotropic Gaussian noise. Let... For time step The noisy image at that location is defined as follows:

[0116]

[0117] in It is the signal retention rate. It is a cumulative product. This represents a standard normal distribution. Noise scheduling follows linear interpolation: ,in , .

[0118] Inverse process: The inverse process learns to reconstruct an image from noise. At each step, the denoised image... Obtained through the following formula:

[0119]

[0120] in It is variance scheduling. This is random noise injected during the denoising process. (Noise prediction network) With noisy images Time step and conditional embedding As input, predict the noise that needs to be removed. .

[0121] Network Architecture: Conditional U-Net employs Adaptive Group Normalization (AdaGN) to inject conditional information. Figure 3 c). The time step t is encoded using sinusoidal position embedding, following the Transformer architecture:

[0122]

[0123] in It is the embedded dimension. Index dimension. The resulting time embedding. With attribute embedding Connect them to form a joint condition vector. The vector is then passed through a linear layer to generate a tensor, which is then split in two along the channel dimension: a scaling factor. Translation factor The AdaGN operation is defined as follows:

[0124]

[0125] Where GN represents group normalization.

[0126] The encoder consists of four downsampling stages, each containing two residual blocks (convolution, AdaGN, and SiLU activation). SiLU stands for Sigmoid linear unit. Spatial resolution is halved by max pooling. The decoder uses transposed convolutions and skip connections symmetric to the encoder. All convolutional layers use cyclic padding to enforce periodic boundary conditions. Conditional information is injected hierarchically at each layer of the network.

[0127] Training objective: This model is optimized using a composite loss function.

[0128]

[0129] The main noise prediction loss is Reconstruction losses Measuring the original image and its reconstruction The mean square error between them. Adaptive weights. In the later stages of noise reduction ( (When larger) Emphasis is placed on reconstruction accuracy. The diffusion model was trained for 600 epochs on a single NVIDIA A6000 GPU using the AdamW optimizer, with a learning rate of [missing information]. The weight decays to Batch size .

[0130] To verify the manufacturability of our designed structure, this embodiment of the invention evaluated whether the elastic parameters derived from the 2D representative volume element (RVE) could predict the mechanical response of a macroscopic 3D cylindrical metamaterial. Representative designs with compression-torsional coupling (CTC) and negative Poisson's ratio (NPR) were selected from a diffusion-generated dataset, and cylindrical specimens with periodic microstructure arrangements were constructed. Using the Gent hyperelastic constitutive model, the responses of these specimens under axial tension and compression were simulated through finite element analysis.

[0131] The method provided by this invention also includes constructing an intelligent agent system based on a large language model. This system integrates an image encoder, a performance encoder, an agent model, a conditional diffusion model, and one or more computational tools (such as CAD modeling tools and finite element analysis tools) to receive natural language or image queries input by the user, parse the user's intent, automatically invoke the corresponding tools or models, perform analysis from image to performance or design from performance to structure, and provide feedback to the user in the form of natural language and / or visualization results.

[0132] To achieve conversational interaction with the material structure design framework, this embodiment of the invention developed SynthMorph—an intelligent agent built on LangGraph and based on a large language model. This agent integrates task planning, memory management, tool invocation, and workflow execution into a unified interface. This embodiment of the invention deployed an untuned Qwen-3-VL-Thinking-30B-A3B model locally as the base large language model.

[0133] 1. System Architecture. This agent consists of five core modules:

[0134] Task planner: From user queries The system identifies the input modality and task intent, and divides the request into two main paths—forward prediction (from microstructure to elastic properties) or inverse generation (from target elastic matrix to microstructure design).

[0135] Memory: Maintain the agent's global context across all graph nodes to ensure state consistency throughout the entire dialogue process.

[0136] Tools: Supports deterministic execution of domain-specific computational tasks, including image / attribute encoding, surrogate model prediction, diffusion generation, mechanical analysis, CAD modeling, and finite element analysis simulation.

[0137] Human-machine collaboration mechanism: Interruptible verification checkpoints are inserted before resource-intensive operations, allowing users to intervene in real time, reject erroneous models, and reduce computational waste.

[0138] Response synthesizer: Combines multimodal outputs (including microstructure images, polar plots of elastic properties, and deformation animations) with physical-based natural language descriptions to generate coherent, interpretable, and interactive feedback.

[0139] 2. Memory mechanism. Memory is encapsulated in a persistent state container. In this context, the container propagates among all graph nodes. The state comprises three parts: (1) the dialogue history. (1) Save multimodal interaction records (text / images); (2) Task context (3) Domain-specific variables It stores physical quantities (elasticity matrix, structural image, finite element analysis results, etc.).

[0140] Unlike retrieval-enhanced systems, the memory in this embodiment of the invention operates as a stateful execution context from which each node reads and writes. No external retrieval is required. This design reduces the risk of illusions while ensuring strict consistency between the computational output and the dialogue context through explicit state propagation, formalized as follows:

[0141]

[0142] 3. Tool Integration. Computational capabilities are embedded into the LangGraph state machine through two types of execution primitives: deterministic workflow nodes and dynamic tools. The system processes domain-specific computational tasks—including image / attribute encoding, surrogate model prediction, diffusion generation, mechanical analysis, CAD modeling, and finite element analysis—as deterministic workflow nodes. These nodes are executed sequentially according to the task direction (forward or backward) determined in the preprocessing stage, and data dependencies are enforced through the edge topology of the state graph.

[0143] In addition to these deterministic workflows, a dynamic tool (Update matrix elements) Function calls generated by a large language model support specifying six independent stiffness components in a dialog-like manner. :

[0144]

[0145] in This indicates a state update fragment. This hybrid design restricts dynamic tool calls to the interactive parameter collection phase, while delegating computationally intensive tasks to deterministic workflows—ensuring physical correctness and dialogue flexibility without sacrificing computational reproducibility.

[0146] 4. Hint Engineering. This system employs stage-specific hints to ensure the behavior of the large language model aligns with different computational stages of the workflow. Two main hint templates control the agent's behavior:

[0147] Matrix collection ( This hint guides the six independent stiffness components via an iterative protocol. Interactive retrieval—checking integrity, summarizing progress, and requesting missing values. Physical verification is delegated to tool patterns rather than natural language instructions.

[0148] Attribute analysis (P_analyze): Activated after matrix prediction or generation, this prompt guides the large language model to interpret direction-related mechanical properties—including Young's modulus, Poisson's ratio, shear modulus, and anisotropy indices—and combines this with a physics-based interpretation to present a polar graph. This phased design imposes constraints that meet phase requirements without overloading a single large prompt. By shifting domain-specific rules from natural language instructions to executable validation (tool patterns and state checks), embodiments of the invention minimize the risk of illusion while maintaining dialogue fluency.

[0149] The following specific embodiments will illustrate the material structure design method based on multimodal contrastive learning provided by the present invention.

[0150] This embodiment provides a material structure design method based on multimodal contrastive learning, which specifically includes the following steps:

[0151] Step S1: Construct the image-performance pairing dataset.

[0152] A density-based topology optimization method based on the solid isotropic material penalty (SIMP) approach, combined with numerical homogenization techniques, is employed to generate paired data of microstructure images and their corresponding effective elastic property matrices. The design domain is a 100×100 pixel two-dimensional representative volume unit. Different optimization objectives (e.g., negative Poisson's ratio, compression-torsional coupling, and combinations thereof) are set. Figure 1As shown in b), optimization begins with various initial material distributions to generate diverse microstructure images that satisfy periodic boundary conditions. A database containing 10,080 image-performance pairs is ultimately obtained. The image data consists of 100×100 pixel grayscale images, and the performance data is a 3×3 elastic stiffness matrix C. Based on the coordinate rotation matrix, the Young's modulus, Poisson's ratio, and shear modulus in various directions of the microstructure can be obtained. The polar coordinate distribution is shown below. Figure 2 As shown in c. Following this, each microstructure image undergoes CAE modeling and finite element analysis to analyze the deformation mechanism under tensile and compressive loads, including clockwise and counterclockwise torsion, expansion, or contraction, etc. (e.g., Figure 1 (As shown in d). This embodiment of the invention analyzed a topology optimization dataset containing 10,080 image-attribute pairs, corresponding to three mechanical objectives: negative Poisson's ratio (NPR), compression-torsion coupling (CTC), and mixed objectives. (See example d). Figure 4 As shown in Figure a, each optimization objective produces a unique topological configuration: the NPR structure exhibits concave chiral rotation characteristics, which promote lateral expansion under tension; the CTC structure includes symmetrically inclined struts that break symmetry, converting axial loads into shear and torsional effects; while the hybrid structure combines features of both prototypes—this phenomenon is referred to as "structural feature fusion" in this embodiment. The mechanical properties of each type of structure are clearly separable in the property space. When projected onto the principal stiffness components... and coupling terms When the subspace was defined, the NPR and CTC structures formed distinct clusters, confirming their unique mechanical properties. Figure 4 (b, c). The hybrid target structure is distributed in the transition region between the two clusters, functionally connecting the two behavioral patterns, thus confirming its hybrid characteristics from the perspective of attribute space.

[0153] Step S2: Cross-modal contrastive learning to construct a semantically aligned embedding space.

[0154] like Figure 2 As shown, two encoders are constructed and trained:

[0155] Image encoder: such as Figure 2 As shown in Figure a, using ResNet18 as the backbone network, a pooling layer is applied before the image to receive single-channel image data, followed by a fully connected layer for downsampling, mapping the 100×100 grayscale image into a 128-dimensional image embedding feature. .

[0156] Performance encoders: such as Figure 2As shown in b, a 4-layer multilayer perceptron upsamples the 9-dimensional stiffness matrix vector (C11, C12, C13, C22, C23, C33) and maps it to a 128-dimensional performance embedding feature. .

[0157] Images and matched elastic stiffness matrices form positive sample pairs, while other elastic stiffness matrices form negative sample pairs. A loss function is constructed based on the cosine similarity matrix (e.g., ...). Figure 2 (As shown in c). A two-stage training strategy is used to optimize the encoder parameters, as follows: Figure 2 As shown in diagram d, in the first stage (the first 30 epochs), only cross-modal contrastive loss (InfoNCE Loss) is used, aiming to bring the paired structure-performance features closer together in the embedding space. In the second stage (epochs 31-100), based on the cross-modal loss, intra-modal contrastive loss is added by changing the parameters of the loss function. This makes the features of the same sample and its enhanced sample within the same modality more similar, further optimizing the semantic structure of the feature space. After training, semantically aligned 128-dimensional image embeddings and performance embeddings are obtained.

[0158] To illustrate the bidirectional retrieval capability based on cosine similarity, in image-to-attribute retrieval (... Figure 5 For a given microstructure image, the first five stiffness matrices retrieved are all different, but mechanically consistent with the true values. Conversely, when the target stiffness matrix is ​​used as the query (ac), Figure 5 The top 5 retrieved images (df) are visually similar but not identical structural variants, and all highly approximate the mechanical properties of the query. This demonstrates the accuracy and inherent diversity of bidirectional cross-modal retrieval based on cosine similarity. Although retrieval accuracy is ultimately limited by the coverage of the dataset, the well-aligned and semantically organized representations built through contrastive learning provide a strong foundation for downstream tasks. These representations facilitate the construction of accurate surrogate models for attribute prediction and conditional diffusion models for diverse image generation, enabling the framework to move beyond similarity matching towards accurate generalization and creative synthesis.

[0159] Step S3-1: Downstream Task 1: Positive performance prediction based on the surrogate model.

[0160] like Figure 3As shown in (a), a lightweight surrogate model consisting of four fully connected layers (128→70→40→20→6) is constructed. This compact design leverages the strong semantic alignment between visual and mechanical representations, enabling efficient training and stable convergence within 100 training epochs. The dataset is divided into training and validation sets in a 9:1 ratio, and both sets maintain low losses during training without overfitting. The input to the model is the image embedding features obtained in step 2. The output consists of six independent stiffness matrix components (C11, C12, C13, C22, C23, C33). The model is trained using mean squared error loss and ultimately achieves R on the test set. 2 = 0.989 prediction accuracy ( Figure 6 a) By building upon robust, semantically aligned features obtained through upstream contrastive learning, the compact regression network of this embodiment achieves accurate prediction of mechanical properties. This result validates the effectiveness of the two-step modeling strategy: contrastive learning first extracts high-level cross-modal representations, and then a lightweight downstream regressor learns accurate function mappings from these features, thereby effectively decoupling representation learning from the regression task and ultimately constructing an efficient and accurate end-to-end performance prediction framework.

[0161] Step S3-2: Downstream Task 2: Reverse Structure Generation Based on Conditional Diffusion Model.

[0162] like Figure 3 As shown in (b,c), a conditional denoising diffusion probability model is constructed. The conditional signal of this model is the performance embedding feature obtained in step 2. and embedding features of time step The model includes a forward diffusion process, gradually moving towards the real image. The process involves adding noise and a learned inverse denoising process, the core of which is a conditional U-Net used to predict the noise to be removed at each step. The model is trained by optimizing the combination of noise prediction loss and reconstruction loss. After training, given a target performance matrix, the model can extract its corresponding embedded features. Starting with random Gaussian noise, the process iteratively denoises the image, ultimately generating a microstructure image that meets the target performance requirements (e.g., ...). Figure 3 (As shown in b and 6c). After demonstrating the generative capabilities, this embodiment of the invention further verifies the model's ability to meet preset performance constraints. This embodiment of the invention selects three representative target attributes from the test set—NPR, CTC, and Mixed—attributes that have not appeared during training. For each target, the model generates five microstructure images with different topologies ( Figure 7a, c, e). Although the visual patterns differ significantly, demonstrating diversity under fixed attribute constraints, the corresponding stiffness matrices closely match the specified target values. Figure 7 (b, d, f). For example, in Figure 8 a and Figure 8 In b, the target stiffness matrix is ​​C 11 =134.86、C 22 =134.31、C 33 =29.45、C 12 =100.91、C 13 =-53.77、C 23 =-53.45, and the matrices obtained from the five generated samples are as follows: C 11 = [134.86, 131.35, 127.85, 130.85, 130.57], C 22 = [133.62, 133.69, 136.34, 133.41, 136.00], C 33 = [32.96, 30.26, 30.25,33.68, 33.55], C 12 = [95.86, 100.05, 100.22, 97.12, 96.47], C 13 = [-52.57, -54.25, -53.40, -53.26, -52.82], C 23 = [-52.26, -53.85, -54.42, -52.93, -53.00]. This confirms that mechanical properties can serve as effective conditional signals throughout the diffusion process, guiding the synthesis of designs that are functionally consistent but morphologically diverse. Furthermore, embodiments of the invention note that, since all training images satisfy periodic boundary conditions, the generated output can correspond to different cropped regions of the same periodic unit cell.

[0163] Step S4: Ablation experiment to verify effectiveness

[0164] The core innovation of this invention's implementation framework lies in utilizing contrastive learning to achieve cross-modal semantic alignment and intra-modal feature clustering, and then using the learned latent representation as input to the surrogate model and conditional embedding of the diffusion model. To evaluate the efficiency improvement brought by this method, this invention's implementation conducted two ablation studies: (1) surrogate modeling without using contrastive learning image features—keeping the network architecture unchanged, but directly training the surrogate model on the original image and its corresponding mechanical property pairs; (2) using the original stiffness matrix as the conditional embedding diffusion model, instead of using semantically aligned attribute features.

[0165] Ablation study of surrogate model. When training with contrastive learning image features ( Figure 8 a) Both training and validation losses decrease rapidly and converge smoothly within 100 epochs, with minimal generalization difference. In contrast, training the same architecture directly on the original image ( Figure 8 (b) This leads to a significantly slower convergence speed and noticeable oscillations. Around the 20th epoch, the validation loss shows a sharp peak; by the end of training, the training loss is about an order of magnitude lower than the validation loss, indicating severe overfitting and a decline in generalization performance.

[0166] A diffusion model ablation study. This invention evaluated two conditional strategies over 600 training epochs, assessing image quality and mechanical fidelity every 50 epochs for three representative target attributes. Figure 8 c–h summarized the results. The model conditioned on contrastive learning representations produced structurally stable images after approximately 100 epochs, while the model conditioned directly on the stiffness matrix required approximately 400 epochs to achieve comparable visual clarity. In terms of mechanical accuracy, the feature-conditioned model achieved an R-value of 0.996 throughout the training process. 2 The R of the direct condition model 2 The value is 0.871, which indicates that generative design has better controllability.

[0167] These ablation experiments demonstrate that semantically aligned representations obtained from contrastive learning accelerate the convergence of the surrogate model and enhance its generalization ability. Furthermore, as efficient conditional embeddings, they expedite the training of the diffusion model and improve design controllability. These findings validate the efficiency gains achieved by using contrastive learning as an upstream representation learning stage to support downstream surrogate models and generation tasks.

[0168] To verify the functional characteristics of the generated structure, CTC microstructures ( Figure 9 a) Microscopic structural asymmetry induces significant macroscopic CTC behavior. Under axial tension, the cylinder twists clockwise; under compression, it twists counterclockwise. At +11.1% axial strain, the torsion angle reaches 14.51°; at –11.1% strain, the torsion angle reaches –22.02°. Figure 9 b), demonstrating reversible CTC behavior. NPR microstructure ( Figure 9 c) It retains its tensile properties in its three-dimensional form. Under axial tension, the cylinder expands radially; under compression, it contracts radially. At +11.1% axial strain, Poisson's ratio reaches -0.228; at -11.1% strain, Poisson's ratio reaches -0.306. Figure 9(d) This confirms that the tensile effect is enhanced in the cylindrical configuration, highlighting the successful transformation of the two-dimensional topology into a functional three-dimensional metamaterial.

[0169] Step S5: Integrate the SynthMorph intelligent agent system.

[0170] like Figure 10 As shown, all the trained models (image encoder, performance encoder, surrogate model, diffusion model) and tools such as CAD modeling and finite element analysis are integrated into an intelligent agent system based on LangGraph and locally deployed large language model. This system can understand user queries in natural language or image form, automatically plan task flows, invoke appropriate tools, and return design results and analysis reports in a combined text and image format. SynthMorph is an intelligent agent built on a large language model that coordinates the invocation of a set of pre-trained computational tools: a contrastive learning encoder for image and mechanical properties, a surrogate model and a diffusion generation model, a two-dimensional property analysis module, and interfaces for CAD modeling and finite element analysis (FEA). This agent facilitates a bidirectional interactive workflow, bridging image-based property analysis and performance-driven structure generation. Its two main operating modes are as follows: Figure 10 As shown.

[0171] In the structure-to-attribute analysis model ( Figure 10 (See the left image below) When a user uploads a microstructure image, the workflow is as follows: First, the image is encoded by a pre-trained image encoder; its extracted features are passed to a proxy model to predict the complete elastic stiffness matrix. Then, a 2D property analysis module evaluates the orientation-dependent mechanical properties and provides preliminary engineering suggestions. Next, the proxy invites the user to decide whether to continue with 3D modeling and simulation. Once the user confirms, the system automatically reconstructs a 3D CAD model from the 2D topology and performs finite element analysis to visualize the macroscopic deformation of the structure under tensile and compressive loads.

[0172] In the attribute-to-structure generation mode ( Figure 10 (See the right-hand image below) The user specifies a target elastic stiffness matrix. The 2D property analysis module parses the target properties and suggests potential application scenarios. If the user requests a matching structure, the agent invokes the conditional diffusion model to synthesize a microstructure image that meets the target mechanical properties, starting from random noise. Subsequently, the user can choose to verify the design through automated CAD reconstruction and finite element analysis, thus completing a closed loop from performance specifications to 3D mechanical verification.

[0173] By integrating material design, property prediction, structure generation, and simulation into a unified interactive platform, SynthMorph demonstrates the flexibility and practical application value of AI-enabled cross-modal reasoning in the field of advanced materials discovery.

[0174] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A material structure design method based on multimodal contrastive learning, characterized in that, Includes the following steps: Step S1: Construct an image-performance pairing dataset, which contains multiple microstructure images and an effective elastic performance matrix corresponding to each microstructure image; Step S2: Train an image encoder and a performance encoder through cross-modal contrastive learning to map the microstructure image and the effective elastic performance matrix to a shared latent space, such that the embedded features of mutually matching image-performance pairs in the latent space are close to each other, while the embedded features of mismatched image-performance pairs are far from each other, thereby obtaining semantically aligned image embedding features and performance embedding features. Step S3: Utilize the semantically aligned image embedding features and / or performance embedding features to perform one or more downstream design tasks.

2. The material structure design method based on multimodal contrastive learning as described in claim 1, characterized in that, The downstream design task includes forward prediction and backward prediction tasks; wherein, The forward prediction task involves inputting the image embedding features into a surrogate model, which is trained to output a prediction elasticity performance matrix corresponding to the input image embedding features. The reverse generation task involves using the embedded features of the target performance as a conditional signal and inputting them into a generation model. The generation model is trained to generate a microstructure image that conforms to the target performance based on the conditional signal.

3. The material structure design method based on multimodal contrastive learning as described in claim 2, characterized in that, The generation model is a conditional diffusion model, and its generation process includes: starting from random noise, and under the guidance of the target performance embedding features, gradually restoring the microstructure image through iterative denoising.

4. The material structure design method based on multimodal contrastive learning as described in claim 1, characterized in that, The contrastive learning described in step S2 employs a two-stage training strategy, specifically including: The first stage is used to optimize only the cross-modal contrast loss to establish a basic mapping between the image embedding features and the performance embedding features; The second stage is used to enhance the robustness and discriminativeness of feature representations within the same modality by increasing intramodal contrast loss while optimizing cross-modal contrast loss.

5. The material structure design method based on multimodal contrastive learning as described in claim 1, characterized in that, Constructing the image-performance paired dataset involves generating microstructures with different target mechanical behaviors through topology optimization, and calculating their corresponding effective elastic performance matrices through numerical homogenization.

6. The material structure design method based on multimodal contrastive learning as described in claim 1, characterized in that, It also includes the following steps: Construct an intelligent agent system based on a large language model, which integrates the image encoder, performance encoder, agent model, and generative model; Receive natural language or image queries from the user; The query is parsed to determine the user's intent, and the corresponding model is automatically invoked to execute the task in step S3 based on the intent; The execution results will be fed back to the user in natural language and / or visual form.

7. A material structure design system based on multimodal contrastive learning, characterized in that, Includes the following modules: A data construction module is used to construct an image-performance paired dataset, which contains multiple microstructure images and an effective elastic performance matrix corresponding to each microstructure image; A cross-modal alignment module is used to train an image encoder and a performance encoder through cross-modal contrastive learning to map the microstructure image and the effective elastic performance matrix to a shared latent space, such that the embedding features of mutually matching image-performance pairs in the latent space are close to each other, while the embedding features of mismatched image-performance pairs are far from each other, thereby obtaining semantically aligned image embedding features and performance embedding features. Downstream task module, used to perform one or more downstream design tasks using the semantically aligned image embedding features and / or performance embedding features.

8. The material structure design system based on multimodal contrastive learning as described in claim 7, characterized in that, It also includes an intelligent interaction agent module, which integrates a large language model and interacts with the cross-modal alignment module and downstream task module to parse natural language or image instructions input by the user and coordinate the various modules to execute corresponding design tasks.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the material structure design method based on multimodal contrastive learning as described in any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the material structure design method based on multimodal contrastive learning as described in any one of claims 1 to 6.

Images