Method and system for designing new materials with the aid of a large language model

Abstract

The application relates to the field of material performance prediction, and provides a method and system for quantitatively predicting the design of new materials with the assistance of a large language model, the method comprising the following steps: S1, collecting literature and preprocessing to obtain preprocessed corpus; S2, training a word segmenter; S3, training a large language model; S4, obtaining process feature encoding; S5, obtaining component feature encoding; S6, constructing and training a neural network prediction model; and S7, predicting the performance of new materials. The system comprises a corpus preprocessing module, a word segmenter training module, a large language model training module, a process feature encoding module, a component feature encoding module, a neural network prediction model training module and a material performance prediction module. The application breaks through the problems, such as structured alignment and high-dimensional sparsity, that are faced by traditional machine learning methods in the representation of material preparation process routes, ensures the quantitative and accurate prediction of material performance while giving consideration to the flexible representation of process routes, and provides reliable tools and approaches for the design of new materials and process optimization.

Description

Technical Field

[0001] This invention relates to the field of material property prediction, and in particular to a method and system for quantitative prediction and design of new materials assisted by a large language model. Background Technology

[0002] Material properties primarily depend on their composition and manufacturing process. Taking steel as an example, steel production involves a series of processes, such as smelting, solidification, multi-pass deformation, and heat treatment. The sequence and parameters of these processes have a decisive impact on the final product's performance. In recent years, in machine learning-based material design and performance prediction, before feature selection and model training, the composition and manufacturing process routes are typically converted into tabular data to construct a performance prediction model. However, developing new manufacturing processes using tabular process feature representations results in high-dimensional sparse feature matrices, making it difficult to implement.

[0003] This invention utilizes natural language descriptions of different processing route sequences and component representations incorporating text features to retrain a large language model based on materials science corpus, and combines materials design with the language model to achieve quantitative prediction of materials properties based on natural language descriptions. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a method and system for quantitative prediction and design of new materials using a large language model. This method solves problems such as structured data extraction, sequence alignment, and high-dimensional sparsity in process routes, and provides a feasible technical route for material performance prediction and process design.

[0005] The present invention adopts the following technical solution:

[0006] On the one hand, this invention provides a method for quantitative prediction and design of new materials using a large language model, comprising:

[0007] S1. Collect relevant literature in the field of materials, preprocess the corpus of the literature to obtain preprocessed corpus;

[0008] S2. Based on the preprocessed corpus obtained in step S1, train the word segmenter;

[0009] S3. Based on the word segmenter trained in step S2 and the preprocessed corpus obtained in step S1, train a large language model.

[0010] S4. Using the trained word segmenter and the large language model, obtain the process feature encoding;

[0011] S5. Using the trained word segmenter and the large language model, obtain the component feature encoding;

[0012] S6. Construct a neural network prediction model. For the preprocessed corpus, after steps S4 and S5, the process feature codes and composition feature codes of the existing materials are obtained respectively, and used as inputs to train the neural network prediction model.

[0013] S7. For the new material to be tested, after steps S4 and S5, the process feature code and composition feature code of the new material to be tested are obtained respectively. These are used as inputs to the trained neural network prediction model to obtain the prediction performance of the new material to be tested.

[0014] Steps S4 and S5 have no specific order.

[0015] In addition to any of the possible implementations described above, another implementation is provided, wherein step S1 specifically includes:

[0016] S1.1 Collect relevant literature in the field of materials, including abstracts and full texts;

[0017] S1.2 Construct a parser for a specific document database to parse and collect the information text of the document;

[0018] S1.3 Preprocesses the information text obtained in step S1.2. The preprocessing includes: replacing the Unicode character set, uniformly converting it to lowercase, and removing special characters to obtain preprocessed corpus.

[0019] In addition to any of the possible implementations described above, another implementation is provided in which, in step S1.2, a document parser for a specific database is constructed to extract relevant information text, and to parse relevant information such as article title, abstract, author, publication date, source database, and digital object identifier (DOI).

[0020] In addition to any of the possible implementations described above, another implementation is provided in which, in step S1.3, the parsed text information is preprocessed, and the replacement characters of the Unicode character set are constructed using the Python third-party modules of NLTK and Transformer. At the same time, the text is uniformly converted to lowercase and special characters that appear multiple times are removed.

[0021] In addition to any of the possible implementations described above, another implementation is provided, wherein step S2 specifically comprises:

[0022] The preprocessed corpus is used to train a word segmenter using the BPE (Byte-Pair Encoding) algorithm. First, a dictionary consisting of individual characters is constructed based on the corpus. Then, all corpora are divided into individual characters, and the frequency of each character is counted. Next, the most frequent strings from the corpus are added to the dictionary sequentially until the loop ends, at which point the word segmenter training is complete. Finally, the text is input into the word segmenter to obtain the corresponding sub-word indexes, token_ids.

[0023] In addition to any of the possible implementations described above, another implementation is provided in which, in step S3, the large language model is a DeBERTa model architecture based on the Transformer network architecture. The encoder part of this model adopts a decoupled attention mechanism and an enhanced decoder to more accurately capture text semantics. The encoder consists of 12 Transformer modules and 12 self-attention heads, which are used to receive input of no more than 512 words and obtain a sequence representation. The sequence representation is a vector sequence encoding representation containing contextual text features, thereby grasping the internal structural details of the sentence.

[0024] In addition to any of the possible implementations described above, another implementation is provided in which, in step S3, each word is represented by two different vectors: one vector captures its semantic content, and the other vector encodes its word order position. The attention mechanism is calculated as shown in formula (1), where Q is the query matrix, K is the key matrix, and V represents the value matrix, d... k Let Q and K be the matrix dimensions, which are typically used to eliminate the influence of dimension on the self-attention mechanism.

[0025]

[0026] In addition to any of the possible implementations described above, another implementation is provided, wherein step S3 specifically includes:

[0027] S3.1 The preprocessed corpus is divided into a training set and a validation set;

[0028] S3.2 The large language model is pre-trained using a masked language model (MLM) task. A set proportion (e.g., 15%) of the words in the preprocessed corpus are masked as the words to be predicted. The masked words are replaced with [MASK] tags. Then, the corresponding word index token_ids are output by the word segmenter to form the input of the large language model. Using the initialization parameters of the large language model, the input is transformed multiple times to obtain a text vector based on context representation. After forward propagation of the neural network, the vector representation of the [MASK] tags is obtained. Then, cross-entropy is used as the loss function to calculate the training loss. After that, backpropagation is used to calculate the gradient of the loss function to update the overall parameters of the large language model. After repeated iterations until the cross-entropy loss on the training set and validation set tends to converge, the optimal parameters of the large language model are obtained, and the training is completed.

[0029] In addition to any of the possible implementations described above, another implementation is provided in which, in step S3.1, 5% of the corpus is used to construct the validation set.

[0030] In addition to any of the possible implementations described above, a further implementation is provided. In step S3.2, the specific method is as follows: During the pre-training process of the material-based large language model, a masked language model (MLM) is used for training, masking 15% of the subwords in the corpus as subwords to be predicted. The initial input text is encoded and embedded, then transformed through multiple network layers. Specifically, the text is first segmented into subwords by a word segmenter, and [CLS] and [SEP] markers are added to the beginning and end of the text. Some subwords are marked with [MASK] to indicate masked words. The input to the material-based large language model consists of subword sequences and position indices, including... Token embedding, position embedding, and type embedding are first passed to the encoding layer after being input into the large language model. The encoder consists of 12 Transformer modules and 12 self-attention heads, and can receive input of no more than 512 words and generate a sequence representation. The generated sequence is a vector sequence encoding representation containing contextual text features, thus grasping the internal structural details of the sentence. The attention mechanism uses different linear projections to transform the text encoding multiple times to obtain text vectors based on contextual representation. Finally, cross-entropy is used as the loss function in the MLM task. The loss function is:

[0031]

[0032] Where H(p,q) is the training loss, p i Let q be the true distribution labeled [MASK]. i The predicted distribution labeled [MASK].

[0033] In addition to any of the possible implementations described above, another implementation is provided, wherein step S4 specifically includes:

[0034] S4.1 uses a word segmenter to divide the process features into sub-word sequences, and adds [CLS] and [SEP] markers to the beginning and end of the sub-word sequences, respectively;

[0035] S4.2 Based on the word segmenter, the sub-word sequence is converted into the input word index token_ids and attention mask sequence;

[0036] S4.3 Input the word index token_ids and attention mask sequence input in step S4.2 into the large language model to obtain the sequence encoding representation of the process features. Finally, the vector representation of [CLS] is used as the process feature encoding.

[0037] In addition to any of the possible implementations described above, another implementation is provided, wherein step S5 specifically comprises:

[0038] For the input of material composition features, textual features of chemical symbols are introduced and fused with weight percentage features to construct a hybrid feature of the composition. Using a trained word segmenter and a large language model, the chemical symbols of elements from the periodic table are encoded and stored sequentially. At the same time, combined with the element weight percentage, a special [CLS] marker of length 768 is used as the element symbol feature code e. The material composition feature encoding is as follows:

[0039]

[0040] Where E represents the material composition characteristic code, e i w represents the feature code corresponding to the i-th chemical element symbol of the material. i This represents the weight percentage of the i-th chemical element, and I represents the number of element types contained in the material.

[0041] This composition feature not only encodes the weight percentage of the chemical composition of the material, but also introduces text encoding of the chemical element symbols, making the information expressed by the composition feature more comprehensive. At the same time, it solves the data sparsity problem caused by composition differences when facing various material design needs.

[0042] In addition to any of the possible implementations described above, another implementation is provided in which, in step S6, the neural network prediction model includes a shared-independent feature extraction network layer and a hybrid feature extraction network layer; step S6 specifically includes:

[0043] S6.1 constructs a 16*768 hybrid feature encoding matrix based on component feature encoding and text feature encoding of length 768. That is, component feature encoding and text feature encoding are sequentially filled along the row direction to construct a hybrid feature encoding containing component feature encoding and text feature encoding. This three-channel parallel feature input method allows the model to obtain single and independent component and process feature encoding while retaining the original hybrid features. Therefore, more information is passed to the model and it also prevents the model from overfitting due to continuous learning of a single feature. Compared with single feature or hybrid feature input methods, the performance on the dataset is improved by 20% to 30%.

[0044] S6.2 For the processing of single feature inputs of component feature encoding and text feature encoding, a multilayer perceptron network architecture with 512, 256, 128, and 64 neurons is used. The ReLU function is used as the activation function, and dropout is used to discard the output of some network nodes to prevent overfitting.

[0045] S6.3 Based on the initial hybrid feature encoding, process feature encoding, and component feature encoding, a feature extraction process based on three parallel channels is constructed, which includes pooling layers of Max and Avg, with a convolution kernel size of 3×3. The output is constructed from the initial single channel to multi-channel and then back to single channel. Finally, the intermediate output of the two-dimensional matrix is ​​expanded into a one-dimensional vector output to obtain three vector representations with a length of 64.

[0046] In the hybrid feature extraction network layer part S6.4, the three vectors of length 64 output from the shared-independent extraction network layer are further constructed using independent MLP network layers to extract features. The neural network layers with 64, 32, and 16 neurons are used, the ReLU function is used as the activation function, and dropout is used to prevent overfitting.

[0047] S6.5 concatenates the reconstructed feature representations output from step 6.4 to obtain mixed features of material processing and composition. Then, it constructs an MLP network layer to output predicted material properties, serving the prediction of the mechanical properties of steel materials. The predicted results and the loss value of the true labels are calculated, using the mean squared error (MSE) loss function, as shown in the following formula:

[0048]

[0049] Where n represents the number of samples. Y represents the predicted output value of the neural network for the i-th sample. i This represents the true label value of the i-th sample;

[0050] Calculate the training loss of the current neural network prediction model parameters, iterate and update the weights of the neural network prediction model multiple times; after training a batch of samples, use the validation set to evaluate the model performance and output the results, and save the optimal model parameters; if the model loss on the validation set is close to convergence, stop training.

[0051] On the other hand, the present invention also provides a system for large language model-assisted quantitative prediction and design of new materials, the system being used to implement the above-mentioned method, the system comprising:

[0052] The corpus preprocessing module is used to preprocess the corpus of the documents to obtain preprocessed corpus;

[0053] The word segmenter training module trains the word segmenter based on the preprocessed corpus;

[0054] The large language model training module trains the large language model based on the trained word segmenter and the preprocessed corpus.

[0055] A process feature encoding module is used to obtain the process feature encoding of the preprocessed corpus;

[0056] A component feature encoding module is used to obtain the component feature encoding of the preprocessed corpus;

[0057] The neural network prediction model training module, for the preprocessed corpus, obtains the process feature code and composition feature code of the existing material through the process feature coding module and the composition feature coding module, respectively, and uses them as input to train the neural network prediction model;

[0058] The material performance prediction module, for the new material to be tested, obtains the encoding of the process characteristics and the encoding of the composition characteristics of the new material to be tested through the process feature encoding module and the composition feature encoding module, respectively. These are used as inputs to the trained neural network prediction model to predict the performance of the new material to be tested.

[0059] The beneficial effects of this invention are as follows:

[0060] By collecting material-related corpora, a large-scale language model for materials, based on a Transformer-encoder architecture, was constructed for encoding textual features. The decoupled attention mechanism and enhanced decoder architecture of the large-scale language model were utilized to better extract global semantic information. Through multimodal hybrid feature encoding of natural language text and components, a convolutional network-based mechanical performance prediction network was constructed, considering both individual features and the output of two types of hybrid features. This invention exhibits good convergence and stability, providing a feasible approach for material design and significantly improving the accuracy of performance prediction. Attached Figure Description

[0061] Figure 1 The diagram shown is a flowchart illustrating a method for quantitative prediction and design of new materials assisted by a large language model, according to an embodiment of the present invention.

[0062] Figure 2 The diagram shows the network structure for predicting the mechanical properties of steel materials in this embodiment. Detailed Implementation

[0063] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that the technical features or combinations of technical features described in the following embodiments should not be considered in isolation, but can be combined with each other to achieve better technical effects.

[0064] like Figure 1 As shown in the figure, an embodiment of the present invention provides a method for quantitative prediction and design of new materials using a large language model, comprising:

[0065] S1. Collect relevant literature in the field of materials, preprocess the corpus of the literature to obtain preprocessed corpus;

[0066] S2. Based on the preprocessed corpus obtained in step S1, train the word segmenter;

[0067] S3. Based on the word segmenter trained in step S2 and the preprocessed corpus obtained in step S1, train a large language model.

[0068] S4. Using the trained word segmenter and the large language model, obtain the process feature encoding;

[0069] S5. Using the trained word segmenter and the large language model, obtain the component feature encoding;

[0070] S6. Construct a neural network prediction model. For the preprocessed corpus, after steps S4 and S5, the process feature codes and composition feature codes of the existing materials are obtained respectively, and used as inputs to train the neural network prediction model.

[0071] S7. For the new material to be tested, after steps S4 and S5, the process feature code and composition feature code of the new material to be tested are obtained respectively. These are used as inputs to the trained neural network prediction model to obtain the prediction performance of the new material to be tested.

[0072] Steps S4 and S5 have no specific order.

[0073] In one specific embodiment, step S1 involves collecting full-text articles on steel materials, related conference papers, patents, and abstracts from sources up to 2021. A document parser is built for the specific database to parse relevant key information text, including article titles, abstracts, authors, publication dates, source databases, and DOIs. Further, the parsed text information undergoes preprocessing. Using Python third-party modules of NLTK and Transformer, replacement characters for the Unicode character set are constructed, and the text is uniformly converted to lowercase, while frequently occurring special characters are removed. Based on the parsed corpus of each type of text, a corpus containing full-text steel material documents and other material abstracts is constructed. This corpus includes approximately 4.2 million material abstracts and 55,000 full-text steel articles (approximately 960 million characters) from sources up to 2021. The collected corpus is divided into a training set and a validation set, with 5% of the corpus used to construct the validation set.

[0074] In one specific embodiment, step S2 involves training a tokenizer for the materials domain. Training the tokenizer based on the collected corpus specifically includes using the Byte-Pair Encoding (BPE) algorithm to segment sentences from preprocessed text data. BPE first divides words into individual characters, then sequentially replaces the most frequent pair of characters with another character until the loop ends. This effectively balances the dictionary size and the number of encoding steps, ultimately constructing a corpus dictionary containing approximately 128,100 sub-words. For example, the sentence "The ingots were homogenized at 1250℃ for 2h, then…" is segmented by the tokenizer into the following sub-word sequence: "[CLS]" "the" "ingots" "were" "homogen" "##ized" "at" "1250" "℃" "for" "2" "h" "," "then" "…".

[0075] "[SEP]".

[0076] In one specific embodiment, in step S3, the large-scale language model SteelBERT is trained. SteelBERT, based on DeBERTa, provides a separate attention mechanism and an enhanced decoder for handling long-range dependencies that are crucial for understanding complex knowledge. The attention mechanism calculation formula is shown in (1), where Q is the query matrix, K is the key matrix, and V represents the value matrix, d k Let Q and K be the matrix dimensions, which are typically used to eliminate the influence of dimension on the self-attention mechanism.

[0077]

[0078] SteelBERT has 188 million parameters, and the model consists of 12 stacked Transformer-encoders, with each hidden layer containing 12 attention heads. We trained SteelBERT on a corpus using the original DeBERTa code. We set the maximum sentence length to 512 words and trained the model until the training loss gradually decreased. During the pre-training of the SteelBERT language model, a masked language model was used, masking 15% of the words in the corpus as the words to be predicted. The pre-training process of SteelBERT used 8 NVIDIA A100 40GB GPUs, took 840 hours, and had a batch size of 576. We used the AdamW optimizer, where β1 = 0.9, β... 2＝ 0.98, ε=1e ―6 The weight decay is 0.01, the learning rate uses linear decay, and the peak learning rate is 0.0001. The model pre-training code is mainly written using PyTorch and Transformers libraries.

[0079] In one specific embodiment, steps S4 and S5 extract the process text and component feature vectors. For text features, a tokenizer is used to convert them into word sequences, and [CLS] and [SEP] tokens are added to the beginning and end of the text. The word sequences are then converted into input word indices. For word sequences longer than 512, the portion exceeding 512 is truncated, and the portion shorter than 512 is padded with [PAD], which does not represent any semantics. After being input into SteelBERT, the data is first passed to the SteelBERT encoding layer. The encoder consists of 12 Transformer modules and 12 self-attention heads, and can receive input of no more than 512 tokens and generate sequence representations. The generated sequence is a vector sequence encoding representation containing contextual text features, thereby grasping the internal structural details of the sentence. Finally, the vector representation of [CLS] is used as the semantic feature representation of the entire sentence context. For the input of material composition features, textual encoding features of chemical symbols are introduced and weight percentage features are fused to construct hybrid features. The chemical symbols of elements from the periodic table are encoded and stored sequentially. A vector of length 768 using the [CLS] special label is used as the textual feature encoding representation of the elements. At the same time, the multimodal feature E constructed by combining the element weight percentage is represented as shown in formula (3):

[0080]

[0081] Where E represents the material composition characteristic code, e i w represents the feature code corresponding to the i-th chemical element symbol of the material. iThis represents the weight percentage of the i-th chemical element, and I represents the number of element types contained in the material.

[0082] In one specific embodiment, such as Figure 2 As shown, in step S6, a CNN-based performance prediction model is constructed. Based on the obtained component feature vectors and text feature vectors of length 768, a 16×768 hybrid feature matrix is ​​constructed. This involves cross-fusing component features and text features and filling them sequentially along the first dimension of the matrix to construct a hybrid element feature matrix containing both text and components. For individual feature inputs of components and text, a multilayer perceptron network architecture with 512, 256, 128, and 64 neurons is used. The ReLU activation function is employed, and dropout is used to prevent overfitting by discarding a portion of the network node outputs with a certain probability. Based on the initial fused features and the independent features of the components, a feature extraction process based on three-channel parallel features is constructed, where the main... Pooling layers containing Max and Avg, with a convolution kernel size of 3×3, construct outputs from single-channel to multi-channel and back to single-channel. Finally, the intermediate outputs of the two-dimensional matrix are expanded into one-dimensional vectors, resulting in three vector representations of length 64. For each of the three feature vector representations, independent MLP network layers are used to further construct feature extraction representations, primarily employing neural network layers with 64, 32, and 16 neurons, using the ReLU activation function and dropout to prevent overfitting. Based on the reconstructed feature representations of the three parts, a hybrid feature of multimodal steel material processing and composition is obtained. Then, an MLP network layer is constructed for predicting the mechanical properties of steel materials.

[0083] In one specific embodiment, based on existing text data and a steel material mechanical property prediction model, the process text features and composition features of the steel material sample are input, and the loss value between the prediction result and the true label is calculated. The loss function used is mean square error (MSE), as shown in formula (4):

[0084]

[0085] Where n represents the number of samples. Y represents the predicted output value of the neural network for the i-th sample. iThis represents the true label value of the i-th sample; finally, backpropagation is performed on the loss value, and the network model weights are updated multiple times. Each training iteration uses a training set of 128 samples, and the validation set is used to evaluate the model's performance and output the results, saving the optimal model state during this process. If the model's performance does not improve after 1000 training iterations, training is stopped. After the model training is complete, the test text data sequence and component features are fed into the model, and finally, the neural network predicts the output to achieve accurate prediction of the mechanical properties of steel materials.

[0086] This invention provides a system for quantitative prediction and design of new materials using a large language model. The system is used to implement the above-described method and includes:

[0087] The corpus preprocessing module is used to preprocess the corpus of the documents to obtain preprocessed corpus;

[0088] The word segmenter training module trains the word segmenter based on the preprocessed corpus;

[0089] The large language model training module trains the large language model based on the trained word segmenter and the preprocessed corpus.

[0090] A process feature encoding module is used to obtain the process feature encoding of the preprocessed corpus;

[0091] A component feature encoding module is used to obtain the component feature encoding of the preprocessed corpus;

[0092] The neural network prediction model training module, for the preprocessed corpus, obtains the process feature code and composition feature code of the existing material through the process feature coding module and the composition feature coding module, respectively, and uses them as input to train the neural network prediction model;

[0093] The material performance prediction module, for the new material to be tested, obtains the encoding of the process characteristics and the encoding of the composition characteristics of the new material to be tested through the process feature encoding module and the composition feature encoding module, respectively. These are used as inputs to the trained neural network prediction model to predict the performance of the new material to be tested.

[0094] This invention overcomes the challenges of structured alignment and high-dimensional sparsity faced by traditional machine learning methods in representing material preparation process routes. While taking into account the flexible representation of process routes, it ensures the quantitative and accurate prediction of material properties, providing a reliable tool and approach for the design of new materials and process optimization.

[0095] While several embodiments of the present invention have been provided herein, those skilled in the art should understand that modifications can be made to these embodiments without departing from the spirit of the invention. The above embodiments are merely exemplary and should not be construed as limiting the scope of the invention.

Claims

1. A method for quantitative prediction and design of new materials assisted by a large language model, characterized in that, The method includes: S1. Collect relevant literature in the field of materials, preprocess the corpus of the literature to obtain preprocessed corpus; S2. Based on the preprocessed corpus obtained in step S1, train the word segmenter; S3. Based on the word segmenter trained in step S2 and the preprocessed corpus obtained in step S1, train a large language model. S4. Using the trained word segmenter and the large language model, obtain the process feature encoding; S5. Using the trained word segmenter and the large language model, obtain the component feature encoding; S6. Construct a neural network prediction model. For the preprocessed corpus, after steps S4 and S5, the process feature codes and composition feature codes of the existing materials are obtained respectively, and used as inputs to train the neural network prediction model. S7. For the new material to be tested, after steps S4 and S5, the process feature code and composition feature code of the new material to be tested are obtained respectively. These are used as inputs to the trained neural network prediction model to obtain the prediction performance of the new material to be tested. Steps S4 and S5 have no specific order; In step S6, the neural network prediction model includes a shared-independent feature extraction network layer and a hybrid feature extraction network layer; step S6 specifically includes: S6.1 Based on the component feature code and text feature code of length 768, construct a 16*768 hybrid feature code matrix, and fill the component feature code and text feature code with crosses along the row direction to construct a hybrid feature code containing component feature code and text feature code; S6.2 For the processing of single feature inputs in component feature encoding and text feature encoding, a multilayer perceptron network architecture with 512, 256, 128, and 64 neurons is used. The ReLU function is used as the activation function, and dropout is used to discard the output of some network nodes to prevent overfitting. S6.3 Based on the initial hybrid feature encoding, process feature encoding, and component feature encoding, a feature extraction process based on three parallel channels is constructed, which includes pooling layers of Max and Avg, with a convolution kernel size of 3×3. The output is constructed from the initial single channel to multi-channel and then back to single channel. Finally, the intermediate output of the two-dimensional matrix is ​​expanded into a one-dimensional vector output to obtain three vector representations with a length of 64. In the hybrid feature extraction network layer part S6.4, the three vectors of length 64 output from the shared-independent extraction network layer are further constructed using independent MLP network layers to extract features. The neural network layers with 64, 32, and 16 neurons are used, the ReLU function is used as the activation function, and dropout is used to prevent overfitting. S6.5 Based on the feature representation output from step 6.4, the features are concatenated to obtain mixed features of material processing and composition. Then, the MLP network layer is constructed to output the predicted output value of material properties. The loss value between the predicted output value and the true label is calculated. The loss function uses the mean squared error (MSE), as shown in the following formula: ； Where n represents the number of samples. This represents the predicted output value of the neural network for the i-th sample. This represents the true label value of the i-th sample; Calculate the training loss of the current neural network prediction model parameters, iterate and update the weights of the neural network prediction model multiple times; after training a batch of samples, use the validation set to evaluate the model performance and output the results, and save the optimal model parameters; if the model loss on the validation set is close to convergence, stop training.

2. The method for quantitative prediction and design of new materials assisted by a large language model as described in claim 1, characterized in that, Step S1 specifically includes: S1.1 Collect relevant literature in the field of materials, including abstracts and full texts; S1.2 Construct a parser for a specific document database to parse and collect the information text of the document; S1.3 Preprocess the information text obtained in step S1.

2. The preprocessing includes: replacing the Unicode character set, uniformly converting it to lowercase, and removing special characters to obtain preprocessed corpus.

3. The method for quantitative prediction and design of new materials assisted by a large language model as described in claim 1, characterized in that, Step S2 is as follows: The BPE algorithm is used to train the word segmenter on the preprocessed corpus. First, a dictionary composed of single characters is constructed based on the corpus. Then, all corpora are divided into single characters, and the frequency of each character is counted. Then, the most frequent string in the corpus is added to the dictionary in turn until the loop ends and the word segmenter is trained. Finally, the text is input into the word segmenter to obtain the corresponding sub-word index token_ids.

4. The method for quantitative prediction and design of new materials assisted by a large language model as described in claim 1, characterized in that, In step S3, the large language model is a DeBERTa model architecture based on the Transformer network architecture. The encoder part of this model adopts a decoupled attention mechanism and an enhanced decoder to capture text semantics more accurately. The encoder consists of 12 Transformer modules and 12 self-attention heads, which are used to receive input of no more than 512 words and obtain a sequence representation. The sequence representation is a vector sequence encoding containing contextual text features, thereby grasping the internal structural details of the sentence.

5. The method for quantitative prediction and design of new materials assisted by a large language model as described in claim 3, characterized in that, Step S3 specifically includes: S3.1 The preprocessed corpus is divided into a training set and a validation set; S3.2 The large language model is pre-trained using the Masked Language Modeling (MLM) task. A set proportion of words in the preprocessed corpus are masked as words to be predicted. The masked words are replaced with [MASK] tags. Then, the corresponding word index token_ids are output by the word segmenter to form the input of the large language model. Using the initialization parameters of the large language model, the input is transformed multiple times to obtain a text vector based on context representation. After forward propagation of the neural network, the vector representation of the [MASK] tags is obtained. Then, cross-entropy is used as the loss function to calculate the training loss. After that, backpropagation is used to calculate the gradient of the loss function to update the overall parameters of the large language model. After repeated iterations until the cross-entropy loss on the training set and validation set tends to converge, the optimal parameters of the large language model are obtained, and the training is completed.

6. The method for quantitative prediction and design of new materials assisted by a large language model as described in claim 5, characterized in that, In step S3.2, the loss function is: in, For training loss, The true distribution of [MASK] labels. The predicted distribution labeled [MASK].

7. The method for quantitative prediction and design of new materials assisted by a large language model as described in claim 1, characterized in that, Step S4 specifically includes: S4.1 Based on the word segmenter, the process features are divided into sub-word sequences, and [CLS] and [SEP] markers are added to the beginning and end of the sub-word sequences respectively; S4.2 Based on the word segmenter, the sub-word sequence is converted into the input word index token_ids and attention mask sequence; S4.3 Input the word index token_ids and attention mask sequence input in step S4.2 into the large language model to obtain the sequence encoding representation of the process features. Finally, the vector representation of [CLS] is used as the process feature encoding.

8. The method for quantitative prediction and design of new materials assisted by a large language model as described in claim 1, characterized in that, Step S5 is as follows: Using the trained word segmenter and large language model, the chemical symbols of elements from the periodic table are sequentially encoded and stored. A special [CLS] marker of length 768 is used as the element symbol feature code e. The material composition feature code is as follows: ； wherein E represents the composition feature code of the material, e i represents the feature code corresponding to the i-th chemical element symbol of the material, w i represents the weight percentage corresponding to the i-th chemical element, and I represents the number of element types contained in the material.

9. A system for quantitative prediction and design of new materials assisted by a large language model, characterized in that, The system is used to implement the method as described in any one of claims 1-8, the system comprising: The corpus preprocessing module is used to preprocess the corpus of the documents to obtain preprocessed corpus; The word segmenter training module trains the word segmenter based on the preprocessed corpus; The large language model training module trains the large language model based on the trained word segmenter and the preprocessed corpus. A process feature encoding module is used to obtain the process feature encoding of the preprocessed corpus; A component feature encoding module is used to obtain the component feature encoding of the preprocessed corpus; The neural network prediction model training module, for the preprocessed corpus, obtains the process feature code and composition feature code of the existing material through the process feature coding module and the composition feature coding module, respectively, and uses them as input to train the neural network prediction model; The material performance prediction module, for the new material to be tested, obtains the encoding of the process characteristics and the encoding of the composition characteristics of the new material to be tested through the process feature encoding module and the composition feature encoding module, respectively. These are used as inputs to the trained neural network prediction model to predict the performance of the new material to be tested.

Images