Composite Material Performance Prediction Model and Method Based on Multimodal Fusion of Microstructure and Material Composition

Abstract

Since the properties of composite materials are usually comprehensively affected by the microstructure and material component information, most of the existing methods are based on the microstructure or material components for prediction, and the data used are relatively single, making it impossible to accurately predict the properties of composite materials. In view of this, the present invention discloses a composite material property prediction model and method based on multi-modal fusion of microstructure and material components, fully considering the influence of both on the final properties of composite materials, and constructing a multi-modal composite material property prediction model composed of a material microstructure feature vector extraction network, a component mass ratio feature vector extraction network, a crystal category feature vector extraction network, an adaptive feature vector fusion module, and a fully connected layer serving as a regressor. Compared with the existing prediction based on single-modal information, using multiple modal information such as material microstructure images, component mass ratios, and component crystal categories has higher prediction accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of computational materials science, and specifically relates to a composite material performance prediction model and method based on multimodal fusion of microstructure and material components. Background Art

[0002] The performance of composite materials has important guiding significance for design and research and development, as well as application scenario selection. The existing methods for obtaining the performance of composite materials are mainly physical experiments and finite element numerical simulations. The former consumes a large amount of raw materials and requires a lot of manpower, while the latter requires high-throughput numerical calculations, and it often takes a long time to calculate a finite element model. With the development of big data and materials informatics, a new paradigm for predicting material performance based on artificial intelligence methods has emerged, which can accelerate material research and development through efficient performance prediction and has good economy and effectiveness.

[0003] Composite materials are new materials composed of optimized combinations of materials with different properties, and there are two or more material components with different chemical and physical properties. The proportion distribution of these material components has a great influence on the final material performance. At the same time, during the processing of composite materials, the microstructure of the crystal grains bonded by different component materials also affects the material performance. Most of the existing material performance predictions based on artificial intelligence methods use either material components alone or microscopic images that can characterize the microstructure of materials alone to achieve the performance prediction of composite materials, resulting in insufficient utilization of data. Material performance is affected by many aspects, and using single-modal data for performance prediction has poor effects. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a composite material performance prediction model and method based on multimodal fusion of microstructure and material components, which fully utilizes material component information such as material component proportion and crystal type, and material microstructure information to achieve more accurate prediction of composite material performance.

[0005] To achieve the above purpose, the present invention provides the following technical solutions:

[0006] The present invention first proposes a composite material performance prediction model based on multimodal fusion of microstructure and material components, including a material microstructure feature vector extraction network, a component mass ratio feature vector extraction network, a crystal category feature vector extraction network, an adaptive feature vector fusion module, and a fully connected layer serving as a regressor;

[0007] The material microstructure feature vector extraction network includes a convolutional neural network, a 1×1 convolutional layer, and a global average pooling layer connected in sequence. The convolutional neural network is used to extract the feature map of the material microstructure. The feature map undergoes channel feature fusion through the 1×1 convolutional layer to obtain a high-level semantic feature map reflecting the material microstructure information. The high-level semantic feature map is processed by the global average pooling layer to obtain the material microstructure feature vector.

[0008] The component mass ratio feature vector extraction network includes an input layer and three fully connected layers I. The number of neurons in the input layer is equal to the number of consecutive attributes of similar mass ratios in the material components. ReLU activation functions are used between adjacent fully connected layers I to enhance the feature extraction and expression ability of the component mass ratio information. The last fully connected layer I outputs the material component mass ratio feature vector.

[0009] The crystal category feature vector extraction network includes an Embedding layer and two fully connected layers II. The Embedding layer is used to embed the encoded information of the crystal categories of the material components into dense vectors that can be used for distance measurement. The number of Embedding modules included in the Embedding layer is equal to the number of crystal categories of the components, and the input dimension of each Embedding module is equal to the number of corresponding crystal categories of the components. ReLU activation functions are used between the two fully connected layers II to enhance the non-linear expression ability of the component crystal category information. The last fully connected layer II outputs the material crystal sub-category feature vector.

[0010] The adaptive feature vector fusion module is used to fuse the material microstructure feature vector, the material component mass ratio feature vector, and the material crystal sub-category feature vector and then output the abstract semantic feature. The fully connected layer used as a regressor is used to perform regression prediction on the material properties, and the number of neurons output after the fully connected layer used as a regressor is equal to the number of mechanical property parameters to be predicted.

[0011] Furthermore, the convolutional neural network is improved for lightweight using ResNet convolutional neural network, VGG neural network, or Efficient-Net neural network.

[0012] Furthermore, the convolutional neural network is obtained by improving the ResNet convolutional neural network for lightweight. The lightweight improved ResNet convolutional neural network includes three stages, and the number of channels in the three stages are 16, 32, and 64 respectively. 64 feature maps are extracted by the lightweight improved ResNet convolutional neural network.

[0013] Furthermore, the principle of the global average pooling layer is:

[0014]

[0015] Among them, h and w are the height and width of a single feature map respectively; is the k-th element of the material microstructure feature vector v1 after global average pooling; represents the element at the i-th row and j-th column in the k-th feature map.

[0016] Furthermore, the number of neurons in the three fully connected layers I are 16, 20, and 32 respectively.

[0017] Furthermore, the encoding principle of the Embedding layer is:

[0018]

[0019] Among them, n is the number of component crystal categories; y i represents the encoding of the i-th component crystal category; Embedding i represents the Embedding module corresponding to the i-th component crystal category y i of.

[0020] Furthermore, the principle of the adaptive feature vector fusion module is:

[0021]

[0022] Among them, V represents the abstract semantic feature; AdaptiveFusion represents the adaptive feature vector fusion module; v1 represents the material microstructure feature vector; v2 represents the material component mass ratio feature vector; v3 represents the material crystal sub-category feature vector; and:

[0023] Let the dimension of the feature vector v i be 1×r, then the summation function sum is expressed as:

[0024]

[0025] The weight coefficient of the feature vector v i is expressed as:

[0026]

[0027] Among them, softmax represents the softmax function.

[0028] The present invention also proposes a composite material performance prediction method based on multi-modal fusion of microstructure and material components, including the following steps:

[0029] Step 1: Obtain a multi-modal dataset of composite materials. The multi-modal dataset includes microstructure images, material component information, and labels. The microstructure images are used to reflect the microscopic structure of the materials, and the material component information includes the mass percentage of material components and the crystal categories of material components; the label is the performance index of the composite material.

[0030] Step 2: Construct a performance prediction model for composite materials based on the multi-modal fusion of microstructure and material components as described above.

[0031] Step 3: Train the performance prediction model for composite materials by means of K-fold cross-validation.

[0032] Step 4: Use the trained performance prediction model for composite materials to predict the performance index of the composite material using the microstructure image and material component information.

[0033] Furthermore, in the above-mentioned Step 1, the multi-modal dataset is divided into k groups, k - 1 of which are taken to divide the training set and the validation set, and the remaining 1 group is used as the test set.

[0034] Furthermore, in the above-mentioned Step 3, the training set is used for model training, the validation set is used to select the model with better prediction effect during the training process, and the generalization performance evaluation of material performance prediction is carried out on the test set. The model with the best generalization performance is selected as the final performance prediction model for composite materials; the loss function during the training process adopts MSE, and the optimizer adopts Adam.

[0035] The beneficial effects of the present invention are as follows:

[0036] The performance of composite materials is usually comprehensively affected by factors such as microstructure and material component information. Most of the existing methods are based on microstructure or material components for prediction, and the data used is relatively single. The performance prediction method for composite materials based on the multi-modal fusion of microstructure and material components of the present invention fully considers the influence of both on the final performance of composite materials, and constructs a multi-modal composite material performance prediction model composed of a microstructure feature vector extraction network, a component mass percentage feature vector extraction network, a crystal category feature vector extraction network, an adaptive feature vector fusion module, and a fully connected layer used as a regressor. Compared with the existing prediction based on single-modal information, using multiple modal information such as microstructure images of materials, component mass percentages, and component crystal categories has higher prediction accuracy.

[0037] Moreover, during the multi-modal fusion learning process, the material component information includes the material component crystal category and the material component mass ratio, which are discrete category information and continuous numerical information respectively. They cannot interact directly, and the category information is usually input into the computer in the form of one-hot vector encoding. There are problems such as difficulty in measuring the similarity of material component crystal categories and high dimensions. The present invention extracts the material component mass ratio feature vector and the material crystal sub-category feature vector through two branch networks respectively, and then fuses them to solve the problem that the underlying data cannot interact and learn directly. And by means of Embedding vector embedding, the one-hot vector is transformed into a dense vector to solve the problems of difficulty in measuring the similarity of component crystal categories and high vector dimensions. Since the information of different modalities such as microstructure and material components has different effects on the final mechanical properties of the composite material, the present invention can automatically learn the weight of the influence of different modality information on the final mechanical properties during the model training process through the adaptive feature vector fusion module, and finally achieve more accurate prediction of the properties of the composite material. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to make the objectives, technical solutions and beneficial effects of the present invention clearer, the present invention provides the following drawings for description:

[0039] Figure 1 It is a flowchart of the method for predicting the properties of a composite material based on multi-modal fusion of microstructure and material components according to the present invention;

[0040] Figure 2 It is a structural diagram of a composite material property prediction model;

[0041] Figure 3 It is the structure of the Res-block module;

[0042] Figure 4 It is a structural diagram of the adaptive feature vector fusion module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0043] The following further describes the present invention with reference to the drawings and specific embodiments, so that those skilled in the art can better understand the present invention and implement it, but the embodiments given are not intended to limit the present invention.

[0044] As Figure 1 shown, the method for predicting the properties of a composite material based on multi-modal fusion of microstructure and material components in this embodiment includes the following steps.

[0045] Step 1: Obtain a multi-modal dataset of composite materials. The multi-modal dataset includes microscopic structure images, material component information and labels. The microscopic structure images are used to reflect the microstructure of the material. The material component information includes the material component mass ratio and the material component crystal category; the label is the performance index of the composite material.

[0046] Composite materials are usually processed from two or more materials through a composite process. The material component information and microstructure have a great impact on the final mechanical properties, and the constructed multimodal dataset of composite materials should fully contain this information. The component information can be characterized by the mass fraction of crystals and the crystal particle categories. The microstructure of the material can be characterized by SEM images obtained by scanning electron microscopy. Microstructural information such as the microscopic morphology of the material crystal particles and the coating situation of the binder can be obtained from the SEM images.

[0047] Several groups of experimental data are obtained according to the mass fractions of different components of the composite material and the variation of crystal types. Each group of data includes the corresponding component mass fractions, crystal types, SEM images of the polymer, and the corresponding material property data labels. Specifically, in this embodiment, a substitute composite material of an energetic material is taken as an example for illustration. It contains four components: Ba(NO3)2, PVC, CAB, and ESTANE. Among them, medium-sized Ba(NO3)2 and fine-sized Ba(NO3)2 together account for 65%, PVC (TS201, TS450) accounts for 30%, CAB accounts for 4.5%, and ESTANE accounts for 0.5%. Keeping the mass fractions of CAB and ESTANE unchanged, adjust the mass fractions of medium-sized Ba(NO3)2, fine-sized Ba(NO3)2, and the types of PVC, and combine different formulas to prepare sample particles and specimens. It can be analyzed that since the mass fractions of CAB and ESTANE remain unchanged, these parameters can be excluded from the dataset, mainly including the mass fractions of medium-sized Ba(NO3)2 particles and fine-sized Ba(NO3)2 particles, and the types of PVC particles. At the same time, SEM images are obtained through the sample polymer particles, and material property indexes such as compressive strength, tensile strength, and modulus are obtained by using MTS based on cylindrical specimens with a diameter of 20 mm and a height of 20 mm. Taking the mass fraction of medium-sized Ba(NO3)2, the mass fraction of fine-sized Ba(NO3)2, the type of PVC, and the corresponding SEM images as inputs, and tensile strength, compressive strength, modulus, etc. as outputs, a multimodal dataset is constructed.

[0048] Step 2: Construct a prediction model for the properties of composite materials based on the multimodal fusion of microstructure and material components.

[0049] As Figure 2 shown, the prediction model for the properties of composite materials based on the multimodal fusion of microstructure and material components in this embodiment includes a material microstructure feature vector extraction network, a component mass fraction feature vector extraction network, a crystal category feature vector extraction network, an adaptive feature vector fusion module, and a fully connected layer used as a regressor.

[0050] (1) The material microstructure feature vector extraction network includes a convolutional neural network, a 1×1 convolutional layer, and a global average pooling layer connected in sequence; the convolutional neural network is used to extract the feature map of the material microstructure; the feature map undergoes channel feature fusion through the 1×1 convolutional layer to obtain a high-level semantic feature map reflecting the material microstructure information; the high-level semantic feature map is processed by the global average pooling layer to obtain the material microstructure feature vector. The convolutional neural network is obtained by lightweight improvement of the ResNet convolutional neural network, the VGG neural network, or the Efficient-Net neural network. In this embodiment, the ResNet convolutional neural network is adopted.

[0051] Specifically, the ResNet convolutional neural network in this embodiment is further improved on the basis of the ResNet18 convolutional neural network. Since ResNet18 is used for large-scale datasets such as ImageNet and completes complex tasks with a large number of classification categories, the model depth and width are set relatively large. The ResNet18 convolutional neural network contains four stages in total, and the number of channels is 64, 128, 256, and 512. On a small-scale dataset, this will lead to slow training speed and overfitting phenomena. Therefore, in this embodiment, for the specific material property prediction task, the network structure and the number of channels are adjusted. In this embodiment, the original fourth stage is removed, and the channels of the first three stages are set to 16, 32, and 64. After being extracted by the ResNet convolutional neural network, 64 feature maps are obtained. In order to adapt to the material component mass ratio feature vector, channel feature fusion is performed through the 1×1 convolutional layer, and finally 32 high-level semantic feature maps that can reflect the material microstructure information are obtained. Then, the global average pooling layer GAP is used to obtain the material microstructure feature vector. Assume that the obtained feature map FeatureMap is h×w×c, where h and w are the height and width of a single feature map respectively, and c is the number of all channels; is the k-th element of the material microstructure feature vector v1 after GAP, represents the element in the i-th row and j-th column of the k-th feature map, then the principle of the global average pooling layer GAP is:

[0052]

[0053] where h and w are the height and width of a single feature map respectively; c is the number of all channels; is the k-th element of the material microstructure feature vector v1 after global average pooling; represents the element in the i-th row and j-th column of the k-th feature map.

[0054] (2) The component mass ratio feature vector extraction network includes an input layer and three fully connected layers I. The number of neurons in the input layer is equal to the number of consecutive attributes of similar mass ratios in the material components. Between two adjacent fully connected layers I, the ReLU activation function is used to enhance the feature extraction and expression ability of the component mass ratio information. The last fully connected layer I outputs the component mass ratio feature vector of the material components.

[0055] Specifically, the input layer of the component mass ratio feature vector extraction network in this embodiment has m neurons, and the setting of its number depends on the number of consecutive attributes of similar mass ratios in the material components. For example, in this embodiment, the consecutive attributes include two mass ratio parameters of medium-sized Ba(NO3)2 particle crystals and fine-sized Ba(NO3)2 particle crystals. Therefore, the number of neurons m in the input layer is set to 2. Subsequently, it will pass through three fully connected layers I, and the number of neurons in each fully connected layer I is 16, 20, and 32 respectively. Among them, after the data passes through the first two fully connected layers I, it will pass through the ReLU activation function to enhance the feature extraction and expression ability of the component mass ratio information. The third fully connected layer I is used to output the component mass ratio feature vector v2 of the material components.

[0056] (3) The crystal category feature vector extraction network includes an Embedding layer and two fully connected layers II; the Embedding layer is used to embed the encoded information of the crystal categories of the material components into a dense vector that can be used for distance measurement; the number of Embedding modules included in the Embedding layer is equal to the number of crystal categories of the components, and the input dimension of each Embedding module is equal to the number of corresponding crystal categories of the components; between the two fully connected layers II, the ReLU activation function is used to enhance the non-linear expression ability of the crystal category information of the components, and the last fully connected layer II outputs the detailed crystal category feature vector of the material.

[0057] The crystal category feature vector extraction network is mainly used to extract the high-level semantic feature vectors of the crystal category information of different component particles in the composite material. Since the traditional one-hot form of category encoding has problems such as high dimension and difficulty in measuring the similarity between categories, in this embodiment, the Embedding layer is used to embed the encoded information of the crystal categories of the material components into a dense vector that can be used for distance measurement. The Embedding module is essentially a two-dimensional matrix, which can encode the input sparse one-hot vector into a dense vector. This matrix can be autonomously learned during the model training process to obtain a dense vector that can best reflect the crystal category information of the components and is easy to measure the distance.

[0058] Suppose there are n component crystal category parameters, numbered y1,…,y n , and the number of each component crystal category is d i, that is, each number y i has d i values. Then n Embedding modules need to be set, and the input dimension of each Embedding module is d i , and the output dimension is 16. Let the dense vector of crystal category information finally obtained be Y, and its dimension is 1x16. Then the encoding principle of the Embedding layer is:

[0059]

[0060] Among them, n is the number of component crystal categories; y i represents the encoding of the i-th component crystal category; Embedding i represents the Embedding module corresponding to the i-th component crystal category y i .

[0061] In this embodiment, only the types of PVC component crystals are different, which are TS201 and TS450 respectively. Therefore, only one Embedding module is set, with an input dimension of 2 and an output dimension of 16.

[0062] Subsequently, the dense vector Y of crystal category information will pass through two fully connected layers II, and the number of neurons in the two fully connected layers II is 20 and 32 respectively. After the first fully connected layer II, the ReLU activation function needs to be passed through to increase the non-linear expression ability of the component crystal category information. The second fully connected layer II outputs the material crystal sub-category feature vector v3.

[0063] (4) The adaptive feature vector fusion module is used to fuse the material microstructure feature vector v1, the material component mass ratio feature vector v2, and the material crystal sub-category feature vector v3 and then output the abstract semantic features.

[0064] Specifically, through the above three branch networks, the material microstructure feature vector v1, the material component mass ratio feature vector v2, and the material crystal sub-category feature vector v3 are obtained. These feature vectors represent three different modalities of information, and the influence of each modality of information on the material properties is also different. Therefore, in this embodiment, an adaptive feature vector fusion module (AdaptiveFusion) is designed based on the attention mechanism, as Figure 4As shown. This module first sums the eigenvectors v1, v2, and v3 that reflect different information of the material to obtain eigenvalues, and then uses the softmax function to obtain the weight coefficients w1, w2, and w3. They determine the influence of the material microstructure information, the mass fraction information of the material components, and the material subdivision crystal category information on the final model prediction result. The weight coefficients can be adaptively learned and adjusted through model training. Let the output result of this module be the abstract semantic feature V that fuses three modal information. Then the principle of the adaptive feature vector fusion module is as follows:

[0065]

[0066] Among them, V represents the abstract semantic feature; AdaptiveFusion represents the adaptive feature vector fusion module; v1 represents the material microstructure feature vector; v2 represents the mass fraction feature vector of the material components; v3 represents the material crystal subcategory feature vector; and:

[0067] Let the dimension of the eigenvector v i be 1×r, then the summation function sum is expressed as:

[0068]

[0069] The weight coefficient of the eigenvector v i is expressed as:

[0070]

[0071] Among them, softmax represents the softmax function.

[0072] (5) The fully connected layer used as a regressor is used to perform regression prediction on the material properties, and the number of neurons output after the fully connected layer used as a regressor is the number of mechanical property parameters to be predicted.

[0073] Specifically, after obtaining the semantic vector V that fuses the material microstructure information, the mass fraction information of the material components, and the material crystal subcategory information, it is input into a fully connected layer with 32 neurons. The main function of this layer is to perform regression prediction on the material properties, and the final number of output neurons is the number of mechanical property parameters to be predicted. In this embodiment, the predicted mechanical indexes are compressive strength, tensile strength, and elastic modulus. Therefore, the number of output neurons is set to 3 to predict the above three mechanical property indexes respectively.

[0074] Step 3: Train the composite material performance prediction model by using the K-fold cross-validation method.

[0075] The multimodal dataset is divided into k groups. Take k - 1 of these groups to divide the training set and the validation set, and the remaining 1 group is used as the test set. During the model training stage, the component information such as the mass ratio of medium - sized Ba(NO3)2, the mass ratio of fine - sized Ba(NO3)2, and the types of PVC particles, as well as the SEM micrographs reflecting the material's microstructure, are mainly used as inputs, and the compressive strength, tensile strength, and elastic modulus of the composite material, etc., are used as output labels. The training set is used to train the model, and the validation set is used to select the model with better prediction performance during the training process. During the training process, the loss function uses the mean - square error (MSE), and the optimizer uses Adam. The generalization performance of the material property prediction is evaluated on the test set, and the model with the best generalization performance is selected as the final composite material property prediction network model.

[0076] Step 4: Use the trained composite material property prediction model to predict the performance indicators of the composite material using the microstructure image and the material component information.

[0077] Based on the composite material property prediction model trained in Step 3, using the microstructure image and the material component information of the material, such as the SEM image, the mass ratio of medium - sized Ba(NO3)2, the mass ratio of fine - sized Ba(NO3)2, and the types of PVC particles, and inputting this information into the trained composite material property prediction model, the prediction of performance indicators such as the tensile strength, compressive strength, and elastic modulus of the composite material can be realized.

[0078] The above - described embodiments are only preferred embodiments given to fully illustrate the present invention, and the protection scope of the present invention is not limited thereto. Equivalent substitutions or transformations made by those skilled in the art on the basis of the present invention are all within the protection scope of the present invention. The protection scope of the present invention is subject to the claims.

Claims

1. A composite material property prediction model based on the multimodal fusion of microstructure and material components, characterized in that: It includes a material microstructure feature vector extraction network, a component mass ratio feature vector extraction network, a crystal category feature vector extraction network, an adaptive feature vector fusion module, and a fully connected layer serving as a regressor; The material microstructure feature vector extraction network includes a convolutional neural network, a 1×1 convolutional layer, and a global average pooling layer connected in sequence; the convolutional neural network is used to extract the feature map of the material microstructure; the feature map undergoes channel feature fusion through the 1×1 convolutional layer to obtain a high-level semantic feature map reflecting the material microstructure information; the high-level semantic feature map is processed by the global average pooling layer to obtain the material microstructure feature vector; The component mass ratio feature vector extraction network includes an input layer and three fully connected layers I. The number of neurons in the input layer is equal to the number of consecutive attributes of similar mass ratios in the material components. The ReLU activation function is used between adjacent fully connected layers I to enhance the feature expression ability of the component mass ratio information. The last fully connected layer I outputs the material component mass ratio feature vector; The crystal category feature vector extraction network includes an Embedding layer and two fully connected layers II; the Embedding layer is used to embed the encoded information of the crystal categories of the material components into a dense vector that can be used for distance measurement; the number of Embedding modules included in the Embedding layer is equal to the number of component crystal categories, and the input dimension of each Embedding module is equal to the number of corresponding component crystal categories; the ReLU activation function is used between the two fully connected layers II to enhance the non-linear expression ability of the component crystal category information. The last fully connected layer II outputs the material crystal sub-category feature vector; The adaptive feature vector fusion module is used to fuse the material microstructure feature vector, the material component mass ratio feature vector, and the material crystal sub-category feature vector and then output the abstract semantic feature; the fully connected layer serving as a regressor is used to perform regression prediction on the material properties, and the number of neurons output after the fully connected layer serving as a regressor is the number of mechanical property parameters to be predicted.

2. The composite material property prediction model based on multimodal fusion of microstructure and material components according to claim 1, wherein: The convolutional neural network is obtained by lightweight improvement of the ResNet convolutional neural network, VGG neural network, or Efficient-Net neural network.

3. The composite material property prediction model based on the multi-modal fusion of microstructure and material components according to claim 2, characterized in that: The convolutional neural network is obtained by lightweight improvement of the ResNet convolutional neural network. The lightweight improved ResNet convolutional neural network includes three stages, and the number of channels in the three stages is 16, 32, and 64 respectively. 64 feature maps are extracted by the lightweight improved ResNet convolutional neural network.

4. The composite material property prediction model based on the multimodal fusion of microstructure and material components according to claim 1, characterized in that: The principle of the global average pooling layer is: where h and w are the height and width of a single feature map respectively; is the k-th element of the material microstructure feature vector v1 after global average pooling; represents the element at the i-th row and j-th column in the k-th feature map.

5. The composite material property prediction model based on the multimodal fusion of microstructure and material components according to claim 1, wherein: The number of neurons in the three fully connected layers I is 16, 20, and 32 respectively.

6. The composite material property prediction model based on the multimodal fusion of microstructure and material components according to claim 1, characterized in that: The encoding principle of the Embedding layer is: where n is the number of component crystal categories; y i represents the serial number of the i-th component crystal category; Embedding i represents the Embedding module corresponding to the i-th component crystal category y i module.

7. The composite material property prediction model based on the multimodal fusion of microstructure and material components according to claim 1, characterized in that: The principle of the adaptive feature vector fusion module is: Among them, V represents the abstract semantic feature; AdaptiveFusion represents the adaptive feature vector fusion module; v1 represents the material microstructure feature vector; v2 represents the material component mass ratio feature vector; v3 represents the material crystal sub-category feature vector; and: Let the eigenvector be v i with a dimension of 1×r. Then the summation function sum is expressed as: The eigenvector v i The weight coefficient is expressed as: Among them, softmax represents the softmax function.

8. A method for predicting the properties of a composite material based on the multimodal fusion of microstructure and material components, characterized in that: It includes the following steps: Step 1: Obtain a multi-modal dataset of composite materials. The multi-modal dataset includes microstructure images, material component information, and labels. The microstructure images are used to reflect the material microstructure. The material component information includes the mass ratio of material components and the crystal category of material components; the label is the performance index of the composite material; Step 2: Construct a composite material performance prediction model based on multi-modal fusion of microstructure and material components as described in any one of claims 1-7; Step 3: Train the composite material performance prediction model in the way of K-fold cross-validation; Step 4: Use the trained composite material performance prediction model to predict the performance index of the composite material using the microstructure image and the material component information.

9. The method for predicting the properties of a composite material based on the multimodal fusion of microstructure and material components according to claim 8, characterized in that: In the said Step 1, the multi-modal dataset is divided into k groups, k-1 groups are taken to divide the training set and the validation set, and the remaining 1 group is used as the test set.

10. The method for predicting the properties of a composite material based on the multimodal fusion of microstructure and material components according to claim 8, wherein: In the said Step 3, the training set is used for model training, the validation set is used to select the model with better prediction effect during the training process, and the generalization performance evaluation of material performance prediction is carried out on the test set. The model with the best generalization performance is selected as the final composite material performance prediction model; the loss function during the training process adopts MSE, and the optimizer adopts Adam.

Images