System and method for predicting physical properties of material

Abstract

A system for predicting the physical properties of a material, of the present invention: extracts a graph embedding by inputting material information into a first AI model; extracts a text embedding by inputting a text description of a crystal structure of the material into a second AI model; partitions the text embedding into a plurality of structural information embeddings; and combines the graph embedding with at least one of the plurality of structural information embeddings, wherein the structural information embeddings may be partitioned by including global information, semi-global information, and local information of the crystal structure.

Description

Material property prediction system and prediction method

[0001] The present invention relates to a material property prediction system and method, and more particularly, to a material property prediction system and method capable of predicting material properties using a graph-based model and a text-based model.

[0002] Artificial intelligence (AI) technology has recently been garnering attention across society as it demonstrates cutting-edge developments. AI encompasses "a computer brain capable of executing tasks previously reserved for human intelligence," "the engineering and science of creating intelligent machines," and "a set of algorithmic systems designed to think, perceive, and act like humans," enabling computers to perform highly advanced, human-like intellectual abilities.

[0003] AI, combined with augmented reality, the Internet of Things, edge computing, and digital twins, is being touted as a key new technology that will lead the Fourth Industrial Revolution, promising highly integrated smart spaces. Furthermore, AI is gaining attention as a next-generation growth engine capable of evolving industrial ecosystems beyond simply solving standardized problems. It is being actively applied across knowledge service industries, including IT, healthcare, agriculture, energy, automobiles, and robotics, as well as distribution, finance, law, education, real estate, advertising, and communications. In other words, AI is integrating with all existing systems, from those that seek to improve the convenience and quality of daily life to the broader cultural and artistic landscape of our society, preparing for a new era.

[0004] With the recent diversification of product development methods, the development of new materials for use in product manufacturing is actively underway. These materials significantly impact product characteristics, and their physical properties can even determine the characteristics of the final product. Therefore, predicting and analyzing the properties of these materials is essential for more efficient development and mass production.

[0005] Traditionally, to determine the properties of a product based on the material's characteristics, multiple materials were developed, each with its own characteristics verified, and then tested on the final product to confirm the properties. However, this conventional method required significant costs and time for material and material characteristic development, and it was difficult to identify the material with optimal properties.

[0006] To improve upon these traditional methods, active research is underway on methods for predicting material properties using artificial intelligence. A related technology is Republic of Korea Patent Publication No. 10-2024-0011349 (January 26, 2024).

[0007] An embodiment of the present invention aims to provide a material property prediction system and prediction method capable of predicting material properties by combining a material property-related graph-based structural embedding and a material property-related text embedding derived from a language model.

[0008] One embodiment of the present invention aims to provide a material property prediction system and prediction method capable of predicting material properties by cross-attentioning a material property-related graph-based structural embedding and a material property-related text embedding derived from a language model.

[0009] A system for predicting the properties of a material of one embodiment of the present invention comprises: at least one processor; and at least one memory for storing commands or information executed by the at least one processor; wherein operations performed by the commands or information executed by the at least one processor include: inputting material information into a first AI model to extract a graph embedding; inputting a text description of a crystal structure of the material into a second AI model to extract a text embedding; dividing the text embedding into a plurality of structural information embeddings; and combining the graph embedding with at least one of the plurality of structural information embeddings, wherein the structural information embeddings can be divided into global information, semi-global information, and local information of the crystal structure.

[0010] Here, the first AI model may include an embedding layer that encodes graph nodes for atoms of the crystal structure of the material information and connection information between atoms; and an interaction layer that iteratively refines the representation of the material based on the graph nodes and connection information.

[0011] In addition, the global information may include comprehensive arrangement information of the crystal structure including mineral type, space group, dimensionality, and symmetry property, the semi-global information may include atomic arrangement information within the crystal structure including geometry and connectivity within the crystal structure, and the local information may include atomic-level detailed information within the crystal structure including types of atoms and interatomic bond lengths within the crystal structure.

[0012] Additionally, the graph embedding projected into a 128-dimensional vector through the first projection head of the first AI model and the text embedding projected into a 128-dimensional vector through the second projection head of the second AI model can be combined in a combination layer to generate a multimodal embedding.

[0013] Additionally, the above multimodal embedding can be input into a fully connected layer to predict the properties of the target material.

[0014] In one embodiment, the graph embedding may be combined with a text embedding that includes at least the semi-global information.

[0015] In one embodiment, the graph embedding may be combined with a text embedding including the global information and the semi-global information.

[0016] A computerized method for predicting material properties of one embodiment of the present invention comprises the steps of: inputting material information into a first AI model executed by a processor to extract a graph embedding; inputting a text description of a crystal structure of the material into a second AI model executed by the processor to extract a text embedding; segmenting the text embedding into a plurality of structural information embeddings through the processor; and combining the graph embedding with at least one of the plurality of structural information embeddings through the processor, wherein the structural information embeddings can be segmented to include global information, semi-global information, and local information of the crystal structure.

[0017] Here, in the step of extracting the graph embedding, the material information input to the first AI model is encoded as graph nodes for atoms of the crystal structure of the material information and connection information between atoms in the embedding layer, and the representation of the material can be iteratively refined based on the graph nodes and connection information in the interaction layer.

[0018] In addition, the global information may include comprehensive arrangement information of the crystal structure including mineral type, space group, dimensionality, and symmetry property, the semi-global information may include atomic arrangement information within the crystal structure including geometry and connectivity within the crystal structure, and the local information may include atomic-level detailed information within the crystal structure including types of atoms and interatomic bond lengths within the crystal structure.

[0019] In addition, prior to the combining step, the graph embedding is projected into a 128-dimensional vector through a first projection head of the first AI model, and the text embedding is projected into a 128-dimensional vector through a second projection head of the second AI model, and in the combining step, the projected graph embedding and the text embedding can be combined in a combining layer to generate a multimodal embedding.

[0020] In addition, one embodiment further includes a step of predicting a property of a target material; wherein the step of predicting the property can predict the property of the target material by inputting the multimodal embedding into a fully connected layer.

[0021] Additionally, in one embodiment, in the combining step, the graph embedding may be combined with a text embedding including at least the semi-global information.

[0022] Additionally, in one embodiment, in the combining step, the graph embedding may be combined with a text embedding including the global information and the semi-global information.

[0023] In an application-specific integrated circuit comprising a memory storing information and instructions to which one embodiment of the present invention is applied, and a functional block including at least one processor requesting access to the memory, the memory stores instructions or information that input material information into a first AI model to extract a graph embedding, input a text description of a crystal structure of the material into a second AI model to extract a text embedding, divide the text embedding into a plurality of structural information embeddings, and combine the graph embedding with at least one of the plurality of structural information embeddings, wherein the structural information embeddings can be divided into global information, semi-global information, and local information of the crystal structure.

[0024] A system for predicting the properties of a material of one embodiment of the present invention comprises: at least one processor; and at least one memory for storing commands or information executed by the at least one processor; wherein operations performed by the commands or information executed by the at least one processor may include an operation of inputting material information into a first AI model to extract a graph embedding, an operation of inputting a text description of the crystal structure of the material into a second AI model to extract a text embedding, and an operation of cross-attentioning the graph embedding and the text embedding.

[0025] Here, the first AI model may include an embedding layer that encodes graph nodes for atoms of the crystal structure of the material information and connection information between atoms; and an interaction layer that iteratively refines the representation of the material based on the graph nodes and connection information.

[0026] In addition, the text embedding includes comprehensive arrangement information of the material crystal structure, geometric information of atomic arrangement within the crystal structure, and atomic-level detailed information within the crystal structure, wherein the comprehensive arrangement information includes mineral type, space group, dimensionality, and symmetry property, the atomic arrangement geometric information includes geometry and connectivity within the crystal structure, and the atomic-level detailed information may include types of atoms within the crystal structure and interatomic bond lengths.

[0027] Additionally, the cross-attention can predict the query by setting the graph embedding as a query and the text embedding as a key and value.

[0028] Additionally, the operation performed by the command or information may further include an operation of fine-tuning the second AI model, and an operation of inputting target material information into the first AI model to predict the properties of the target material.

[0029] Additionally, the fine tuning can be performed using a LoRA (Low-Rank Adaptation) method that freezes the second AI model weights and reduces the number of learnable parameters.

[0030] A computerized method for predicting material properties of one embodiment of the present invention may include the steps of: inputting material information into a first AI model executed by a processor to extract a graph embedding; inputting a text description of the crystal structure of the material into a second AI model executed by the processor to extract a text embedding; and cross-attentioning the graph embedding and the text embedding through the processor.

[0031] Here, in the step of extracting the graph embedding, the material information input to the first AI model is encoded as graph nodes for atoms of the crystal structure of the material information and connection information between atoms in the embedding layer, and the representation of the material can be iteratively refined based on the graph nodes and connection information in the interaction layer.

[0032] In addition, the text embedding includes comprehensive arrangement information of the material crystal structure, geometric information of atomic arrangement within the crystal structure, and atomic-level detailed information within the crystal structure, wherein the comprehensive arrangement information includes mineral type, space group, dimensionality, and symmetry property, the atomic arrangement geometric information includes geometry and connectivity within the crystal structure, and the atomic-level detailed information may include types of atoms within the crystal structure and interatomic bond lengths.

[0033] Additionally, in the cross-attention step, the graph embedding is set as a query and the text embedding is set as a key and a value, so that the query can be predicted by referring to the key and value.

[0034] In addition, the material property prediction method of one embodiment may further include a step of fine-tuning the second AI model; and a step of inputting target material information into the first AI model to predict the properties of the target material.

[0035] Additionally, in the fine-tuning step, the fine-tuning may be performed in a LoRA (Low-Rank Adaptation) manner in which the weights of the second AI model are frozen and the number of learnable parameters is reduced.

[0036] In an application-specific integrated circuit comprising a memory storing information and instructions of one embodiment of the present invention and a functional block including at least one processor requesting access to the memory, the memory may store instructions or information including operations of inputting material information into a first AI model to extract a graph embedding, inputting a text description of a crystal structure of the material into a second AI model to extract a text embedding, cross-attentioning the graph embedding and the text embedding, fine-tuning the second AI model, and inputting target material information into the first AI model to predict a property of the target material.

[0037] According to one embodiment of the present invention, the prediction performance of the properties of a new material can be improved by combining a graph-based structural embedding and a text embedding for the properties of the material to predict the properties of the new material.

[0038] According to one embodiment of the present invention, the prediction performance of the properties of a new material can be improved by cross-attentioning a graph-based structural embedding and a text embedding for the properties of the material to predict the properties of the new material.

[0039] Figure 1 is a schematic diagram of an electronic device according to one embodiment of the present invention.

[0040] Figure 2 is a schematic diagram showing a method for predicting material properties according to one embodiment of the present invention.

[0041] Figure 3 is a schematic diagram showing a text description of the material crystal structure of one embodiment of the present invention.

[0042] Figure 4 is a schematic diagram showing a learning model of one embodiment of the present invention.

[0043] Figure 5 is a schematic diagram showing a method for predicting material properties according to another embodiment of the present invention.

[0044] Figure 6 is a schematic diagram showing a pre-learning model of another embodiment of the present invention.

[0045] Figure 7 is a schematic diagram showing a property prediction model of another embodiment of the present invention.

[0046]

[0047] In order to clarify the technical idea of ​​the present disclosure, embodiments of the present disclosure will be described in detail with reference to the attached drawings. In describing the present disclosure, if a detailed description of a related known function or component is determined to unnecessarily obscure the gist of the present disclosure, the detailed description will be omitted. In the drawings, components having substantially the same functional configuration are given the same reference numbers and symbols as possible even if they are shown in different drawings. For convenience of explanation, devices and methods are described together when necessary. Each operation of the present disclosure does not necessarily have to be performed in the described order and may be performed in parallel, selectively, or individually.

[0048] The terms used in the embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of the present disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, the applicant may arbitrarily select terms, and in such cases, their meanings will be described in detail in the description of the relevant embodiments. Therefore, the terms used in this specification should not be defined simply as names of terms, but rather based on their meanings and the overall content of the present disclosure.

[0049] Throughout this disclosure, singular expressions may include plural expressions unless the context clearly dictates otherwise. Terms such as "comprise" or "have" should be understood to indicate the presence of a feature, number, step, operation, component, part, or combination thereof, but do not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof. In other words, when it is said throughout this disclosure that a part "comprises" a certain component, unless specifically stated otherwise, this does not mean that other components may be included, but rather that other components may be excluded.

[0050] Expressions such as "at least one" modify the entire list of elements, not individual elements of the list. For example, "at least one of A, B, and C" and "at least one of A, B, or C" refer to only A, only B, only C, both A and B, both B and C, both A and C, all of A, B, and C, or any combination thereof.

[0051] In addition, terms such as “...unit”, “...module”, etc. described in the present disclosure mean a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software.

[0052] Throughout this disclosure, when a part is said to be "connected" to another part, this includes not only cases where the parts are "directly connected," but also cases where the parts are "electrically connected" with other elements intervening. Furthermore, when a part is said to "include" a component, this does not exclude other components, but rather includes other components, unless otherwise specifically stated.

[0053] The expression “configured to” as used throughout this disclosure can be used interchangeably with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” does not necessarily mean something that is “specifically designed to” in terms of hardware. Instead, in some contexts, the expression “a system configured to” can mean that the system, together with other devices or components, is “capable of.” For example, the phrase “a processor configured (or set) to perform A, B, and C” may mean a dedicated processor (e.g., an embedded processor) for performing those operations, or a generic-purpose processor (e.g., a CPU or application processor) that can perform those operations by executing one or more software programs stored in memory.

[0054] Artificial intelligence (AI) is a field of computer engineering and information technology that studies how to enable computers to perform human-like tasks, such as thinking, learning, and self-improvement. It aims to enable computers to mimic human intelligent behavior. Furthermore, AI does not exist in isolation; rather, it is closely linked, both directly and indirectly, to other fields of computer science. In particular, efforts are actively underway to incorporate AI elements into various fields of information technology and utilize them to solve problems in those fields.

[0055] Machine learning is a branch of artificial intelligence that empowers computers to learn without explicit programming. Specifically, machine learning is the study and development of algorithms and systems that learn from empirical data, make predictions, and improve their own performance. Rather than executing strictly defined, static program instructions, machine learning algorithms build specific models based on input data to derive predictions or decisions. The term "machine learning" can be used interchangeably with "machine learning."

[0056] Many machine learning algorithms have been developed to classify data. Representative examples include decision trees, Bayesian networks, support vector machines (SVMs), and artificial neural networks (ANNs). Decision trees are an analytical method that performs classification and prediction by diagramming decision rules in a tree-like structure. Bayesian networks are models that represent probabilistic relationships (conditional independence) between multiple variables in a graph structure. Bayesian networks are suitable for data mining through unsupervised learning. Support vector machines are supervised learning models for pattern recognition and data analysis, primarily used for classification and regression analysis. Artificial neural networks model the operating principles and interconnected relationships of biological neurons. They are information processing systems in which numerous neurons, called nodes or processing elements, are connected in layers.

[0057] An artificial neural network (ANN) is a model used in machine learning. It is a statistical learning algorithm inspired by biological neural networks (especially the brain, the central nervous system of animals) in machine learning and cognitive science. Specifically, an ANN can refer to a general model in which artificial neurons (nodes) form a network through the connection of synapses, which change the strength of the synaptic connections through learning, thereby achieving problem-solving capabilities. The term "ANN" can be used interchangeably with the term "neural network."

[0058] An artificial neural network can include multiple layers, each of which can include multiple neurons. Furthermore, an artificial neural network can include synapses, which connect neurons. An artificial neural network can generally be defined by three factors: a) the connection pattern between neurons in different layers, b) a learning process that updates the weights of the connections, and c) an activation function that generates an output value from a weighted sum of the inputs received from the previous layer.

[0059] Artificial neural networks may include, but are not limited to, network models such as Deep Neural Networks (DNNs), Recurrent Neural Networks (RNNs), Bidirectional Recurrent Deep Neural Networks (BRDNNs), Multilayer Perceptrons (MLPs), and Convolutional Neural Networks (CNNs). In this specification, the term "layer" may be used interchangeably with the term "layer."

[0060] Artificial neural networks are categorized into single-layer neural networks and multi-layer neural networks based on the number of layers. A typical single-layer neural network consists of an input layer and an output layer. A typical multi-layer neural network consists of an input layer, one or more hidden layers, and an output layer.

[0061] The input layer is the layer that receives external data, and the number of neurons in the input layer is the same as the number of input variables, and the hidden layer is located between the input layer and the output layer. It receives signals from the input layer, extracts characteristics, and transmits them to the output layer. The output layer receives signals from the hidden layer and outputs output values ​​based on the received signals. The input signals between neurons are multiplied by each connection strength (weight) and then added, and if this sum is greater than the threshold of the neuron, the neuron is activated and outputs the output value obtained through the activation function.

[0062] Meanwhile, deep neural networks, which include multiple hidden layers between the input and output layers, are representative artificial neural networks that implement deep learning, a type of machine learning technique. The term "deep learning" can be used interchangeably with "deep learning," and the term "learning" can be used interchangeably with "training."

[0063] The workflow of machine learning consists of a series of steps: collecting data for learning and validation, modeling, and then training the model. This can include the processes of collecting training data, inspecting and exploring the data, preprocessing and cleaning the data, modeling, and training.

[0064] 1. Collect training data

[0065] The training data applied to the learning model of this specification can be generated using data collected from multiple samples. In this specification, at least one or more different types of training data sets can be used to train the learning model, and each training data set can further include one or more experimental results used as feature labels. At least a portion of the training data set can be used to train the learning model, and another portion can be used to validate the learned learning model.

[0066] The data used in the graph model of one embodiment of the present invention may have the Simplified Molecular-Input Line-Entry System (SMILES) format, which is generally used to represent the chemical formula of a molecule as a string. SMILES is a notation that expresses molecular structures in the form of strings, so that the molecular structures can be applied to various machine learning and deep learning algorithms. Generally, SMILES can be composed of atoms, bonds, rings, aromaticity, and branches. In the SMILES notation, each atom is represented by its corresponding element symbol. For example, carbon can be represented as C, nitrogen as N, oxygen as O, chlorine as Cl, and hydrogen atom H can be omitted. Bonds are represented by eight symbols: ".", "-", "=", "#", "$", ":", " / ", and " / ". For example, a double bond can be represented as "=", a triple bond as "#", and a quadruple bond as "$". A ring is represented by breaking a bond at any arbitrary point in the molecular structure and numbering the two atoms at that broken point. Aromaticity refers to the presence of an aromatic ring, which is a stable structure formed by bonding carbon compounds in a planar ring form. This aromatic ring is represented in the same way as the ring described above, but the B, C, N, O, P, and S atoms contained within it are represented in lowercase letters. Branches in the molecule are indicated by parentheses. The first atom included within the parentheses and the first atom following the parentheses can be connected to the same atom. This can cause ambiguity because the same molecular structure can have different SMILES representations written in different ways. Tools such as the RDKit library can be utilized to resolve this ambiguity.By removing duplicate compounds and unclear structural forms within the collected data set, a final dataset for molecular graphing can be secured. This SMILES-represented dataset can be represented as an adjacency matrix or adjacency list, representing the graph in the form of nodes and edges. The adjacency matrix and adjacency list represent the graph's connectivity as a two-dimensional array and list, respectively.

[0067] The data used for text embedding in one embodiment of the present invention may include a text description of a crystal structure. In one embodiment, the text description data may be obtained through the Robocrystallographer package, which generates text descriptions similar to the way a real crystallographer analyzes a structure. The Robocrystallographer package indicates symmetry, local environment, and extended connectivity when generating text descriptions of crystal structures, and the package may include utilities for identifying molecule names, component orientations, heterostructure information, etc. For example, the Robocrystallographer used in one embodiment, given SnO2 as input, reports "SnO2 is Rutile structured and crystallizes in the tetragonal P4_2 / mnm space group. The structure is three-dimensional. Sn(1) is bonded to six equivalent O(1) atoms to form a mixture of edge and corner-sharing SnO6 octahedra. The corner-sharing octahedral tilt angles are 51°. All Sn(1)-O(1) bond lengths are 2.09 . O(1) is bonded in a trigonal planar geometry to three equivalent Sn(1) atoms." Such text descriptions can contain a wide range of information, including global properties (e.g., space group and crystal type), local details (e.g., bond lengths and coordination environments), and semi-global properties (e.g., connectivity and structural arrangement).

[0068] 2. Checking and exploring data

[0069] Once training data for learning a learning model is collected, the collected training data can be inspected and explored for data structure, noisy data, and data cleaning methods for applying machine learning.

[0070] This data review and exploration phase is called Exploratory Data Analysis (EDA), and EDA can be defined as the process of observing and understanding collected data from various perspectives. Before data learning, visualizations such as graphs and statistical tests are used to examine independent and dependent variables, variable types, and their data types, allowing for preliminary identification of data characteristics and inherent structural relationships. Through EDA, data distributions and values ​​can be examined to better understand the phenomena expressed by the data and identify potential problems. Furthermore, through the process of examining data from various perspectives, various patterns that might not have been detected during the problem definition phase can be discovered, allowing for modification of existing hypotheses or the development of new ones. Exploratory data analysis can broadly include the process of searching for outliers and analyzing the relationships between data attributes.

[0071] The process of detecting outliers involves determining whether data contains outliers. This process can involve sampling, statistical methods, and visualization methods. Sampling methods extract a random sample from the data to identify overall trends and anomalies in the data values. Statistical methods can utilize summary statistics such as the mean, median, and mode to determine the center of the data, or the range and variance to determine the distribution of the data. Visualization methods can utilize probability density functions, histograms, dotplots, word clouds, time series charts, and maps to determine which statistical indicators are appropriate for each attribute of the collected data. However, when using statistical indicators, it is important to note that the mean reflects all data values ​​in the set, so outliers can affect the value, whereas the median uses the single value in the middle, so it can produce representative results even with outliers.

[0072] The process of analyzing the relationship between data attributes is to find combinations of attributes that have meaningful correlations within the data. The relationship analysis can be performed differently depending on the combination of attributes between qualitative attributes (Categorical Variable; Qualitative) that cannot be expressed numerically but can be arbitrarily quantified and quantitative attributes (Numeric Variable; Quantitative). The qualitative-qualitative relationship (Categorical - Categorical) can be displayed by using cross tables and mosaic plots to count the number of values ​​corresponding to each pair of attribute values. The quantitative-qualitative relationship (Numeric-Categorical) can be visually expressed by observing statistical values ​​(mean, median, etc.) by category or using box plots. The quantitative-quantitative relationship (Numeric-Numeric) can be analyzed for the association between two attributes using correlation coefficients. A correlation coefficient of -1 indicates a negative correlation where the two attributes change in opposite directions, 0 indicates no correlation, and 1 indicates a positive correlation where the two attributes always change in the same direction. The relationship between two attributes with a correlation coefficient can take many forms, and this can be visually represented using a scatter plot.

[0073] 3. Data preprocessing and cleaning

[0074] Once the data has been inspected and explored, data preprocessing is performed to transform it into a format suitable for machine learning training models. Data preprocessing involves refining data and transforming it into a form understandable by the model. Data preprocessing typically includes handling missing data, removing outliers, scaling, categorical data encoding, feature selection and extraction, and data transformation. The detailed data preprocessing steps can be performed in whole or in part, and a separate machine learning model may be used for data preprocessing.

[0075] Handling missing data involves handling missing values ​​in data. Missing values ​​can be displayed as NaN (Not a Number) or blank, or they can be deleted. Filling in or deleting missing values ​​within the data improves data completeness. When filling in missing values, values ​​such as the mean, median, or mode can be used.

[0076] Outlier removal is the process of removing outliers, values ​​that deviate from the normal data pattern. Outliers can degrade model performance and should therefore be removed or replaced. Identifying outliers can be accomplished by deleting the corresponding rows or columns or replacing them with different values.

[0077] Data scaling is the process of adjusting the size of data. Through data scaling, the range of data can be adjusted, and the performance of the model or the convergence speed can be improved. Through data scaling, the characteristics of the data can be adjusted to a similar range, and data scaling can generally be applied with standardization and normalization. Standardization is a method of converting data into a distribution with a mean of 0 and a standard deviation of 1, and is mainly converted using the mean and standard deviation, and the standardized value z is It can be expressed as (x is the original value, μ is the mean, σ is the standard deviation). Normalization is a method to convert the range of data to [0,1] or [-1,1], and mainly converts data using the minimum and maximum values, and the normalized value x norm silver can be expressed as (x is the original value, x min is the minimum, x max is the maximum value).

[0078] Categorical data encoding is the process of converting categorical variables, represented as strings or integers that cannot be directly input into a model, into numerical data types that can be input into the model. Typically, one-hot encoding or label encoding is used to convert categorical variables into numerical data types.

[0079] Feature selection and extraction is a process to improve model performance by selecting the most useful features for model learning or extracting new features. This process can reduce model complexity and prevent overfitting.

[0080] Data transformation is the process of transforming data to extract new information or to improve model understanding. This can include tokenizing text data or preprocessing image data. Data transformation can extract useful features from source data or transform data into an appropriate format, improving model performance.

[0081] Through data preprocessing as described above, the performance of machine learning models can be improved and stability can be secured.

[0082] Meanwhile, when training a learning model according to one embodiment of the present invention, a process of preprocessing information expressed in natural language and a process of learning a language model based on the preprocessed data may be performed.

[0083] 3-1. Text Preprocessing for Large-Scale Language Models

[0084] If the collected data has not been preprocessed to suit your needs, tokenization, cleaning, and normalization can be performed to suit the intended use of the data.

[0085] Tokenization refers to the process of dividing given data into units called tokens, which can be broadly defined as meaningful units. Tokenization can broadly include word tokenization and sentence tokenization.

[0086] Tokenization refers to the process of dividing given data into units called tokens, which can be broadly defined as meaningful units. Tokenization can broadly include word tokenization and sentence tokenization.

[0087] Word tokenization refers to cases where tokens are based on words, and in this case, words can include not only individual words but also word phrases and meaningful strings. Word tokenization separates words based on spaces or punctuation marks, such as periods, commas, question marks, semicolons, and exclamation marks. However, removing all punctuation or special characters during tokenization can sometimes result in tokens losing their meaning, necessitating a more precise tokenization algorithm. For example, if a word itself contains punctuation or uses special characters with meaning, simply removing them may not be enough. Therefore, tokenization rules such as the Penn Treebank Tokenization Rules can be applied during tokenization.

[0088] Sentence tokenization refers to the process of dividing text into sentences. Typically, if the data is unrefined, the corpus is not segmented into sentences, and thus sentence tokenization may be necessary to meet the intended use. Various rules for sentence tokenization can be defined depending on the language being used and how special characters are used within the corpus.

[0089] Tokenization is the process of classifying tokens according to their intended use. Before and after tokenization, cleaning and normalization are performed on text data to suit the intended use. Cleaning removes noise, while normalization integrates words with different representations and transforms them into a single, consistent word.

[0090] Refinement can occur before tokenization to eliminate any interference and facilitate tokenization. However, it can also be performed continuously and iteratively after tokenization to remove any remaining noise. The noise data removed during refinement are meaningless characters. Methods for removing unnecessary words include removing stopwords, low-frequency words, and short words.

[0091] Normalization work includes unifying words with different spellings based on rules, unifying uppercase and lowercase letters, etc. Unifying uppercase and lowercase letters is a normalization method that can reduce the number of words in English-speaking languages. In English-speaking languages, uppercase letters are only used in certain situations such as the beginning of a sentence, and most texts are written in lowercase letters, so unifying uppercase and lowercase letters can mostly be done by converting uppercase letters to lowercase letters.

[0092] Processing natural language in computing systems requires preprocessing, which involves digitizing text. This involves mapping each word in the text to a unique integer. This mapping process can utilize techniques such as integer encoding, padding, and one-hot encoding.

[0093] Integer encoding is a method of assigning integers to words. It creates a vocabulary by sorting words in order of frequency, and assigns integers in order of frequency, starting with the lowest number. Integer encoding performs sentence tokenization on text data containing multiple sentences, and performs word tokenization through parallel refinement and normalization. During this process, words are lowercase to unify the number of words, and stopwords and word length can be deleted. Through this, words can be recorded as keys and the frequency of each word as values. Integer encoding can be performed by sorting the text in order of frequency and assigning integers to words with high frequencies.

[0094] Padding is the process of randomly adjusting the length of sentences of different lengths within a text to the same length. Computing systems can perform parallel computations by grouping sentences of the same length into a single matrix. Specifically, to perform parallel computations, the lengths of sentences of different lengths within a text can be randomly padded with "0" to equalize the integer encoding results. Specifically, the longest sentence in a set of integer-encoded words can be identified, and a "0" can be added to the integer matrix corresponding to the length of the longest sentence. The computing system can then process sentences of the same length as a single matrix, allowing it to perform parallel processing. At this point, the computing system can ignore the "0" word, which is perceived as meaningless. This process of adjusting the size (shape) of data by filling it with a specific value is called padding. Using the number "0" to adjust the length is called zero padding.

[0095] One-hot encoding is a vector representation of words that uses the size of the vector as the dimension of the word set, assigning a value of 1 to the index of the word to be expressed, and 0 to all other indices. The vector expressed in this way is called a one-hot vector. One-hot encoding consists of integer encoding and an index assignment process. After integer encoding is performed and a unique integer is assigned to each word, the unique integer of the word to be expressed is regarded as an index, and a "1" is assigned to the corresponding position, and a "0" is assigned to the index positions of other words. However, one-hot encoding has the disadvantage that the space required to store the vector increases as the number of words increases (the dimensionality of the vector increases), and the similarity between words cannot be identified. To address these shortcomings, techniques that reflect the latent meaning of words and vectorize them into a multidimensional space include LSA (Latent Semantic Analysis), a count-based vectorization method; NNLM, RNNLM, Word2Vec, and FastText, which vectorize based on prediction; and the GloVe method, which uses both count-based and prediction-based methods.

[0096] Meanwhile, for computers to understand and process text, it must be appropriately converted into numbers. Because the performance of natural language processing can vary significantly depending on how words are represented, numerous techniques have been proposed to quantify words. Currently, the most widely used method is word embedding, which vectorizes each word through artificial neural network training.

[0097] Word embedding is a method of representing words as vectors, converting them into dense representations. The resulting word embedding is called a dense vector, or embedding vector. Word embedding methods include LSA, Word2Vec, FastText, and Glove.

[0098] 4. Modeling and Training

[0099] Artificial neural networks can be trained using training data. Here, "training" refers to the process of determining the parameters of an artificial neural network using training data to achieve objectives such as classification, regression analysis, or clustering of input data. Representative examples of artificial neural network parameters include the weights assigned to synapses and the biases applied to neurons.

[0100] An artificial neural network trained using training data can classify or cluster input data based on its patterns. Meanwhile, an artificial neural network trained using training data is referred to herein as a "trained model."

[0101] The following explains the learning methods of artificial neural networks. Learning methods of artificial neural networks can be broadly categorized into supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning.

[0102] Supervised learning is a machine learning method that infers a function from training data. Among these inferred functions, regression analysis is the process of outputting continuous values, while classification is the process of predicting and outputting the class of an input vector.

[0103] In supervised learning, an artificial neural network is trained with labels for training data. Here, the label can mean the correct answer (or result value) that the artificial neural network should infer when training data is input to the artificial neural network. In this specification, the correct answer (or result value) that the artificial neural network should infer when training data is input is called a label or labeling data. In addition, in this specification, setting a label on training data for learning of the artificial neural network is called labeling the training data. In this case, the training data and the label corresponding to the training data constitute a single training set, and can be input to the artificial neural network in the form of a training set.

[0104] Meanwhile, training data represents multiple features, and labeling the training data can mean that the features represented by the training data are labeled. In this case, the training data can represent the features of the input object in vector form. An artificial neural network can use the training data and labeled data to infer a function regarding the relationship between the training data and the labeled data. Furthermore, the parameters of the artificial neural network can be determined (optimized) by evaluating the function inferred by the artificial neural network.

[0105] Unsupervised learning is a type of machine learning in which training data is not labeled. Specifically, unsupervised learning can be a learning method that trains an artificial neural network to find and classify patterns in the training data itself, rather than the relationship between the training data and the corresponding labels. Examples of unsupervised learning include clustering and independent component analysis (ICA). In this specification, the term "clustering" may be used interchangeably with the term "clustering."

[0106] Examples of artificial neural networks that utilize unsupervised learning include generative adversarial networks (GANs) and autoencoders (AEs).

[0107] Generative adversarial networks (GANs) are a machine learning method in which two different AI components, a generator and a discriminator, compete to improve performance. In this case, the generator is a model that creates new data, capable of generating new data based on original data. The discriminator, a model that recognizes data patterns, can determine whether the input data is original or new data generated by the generator. The generator learns from data that fails to fool the discriminator, while the discriminator learns from data that the generator deceives. Accordingly, the generator can evolve to fool the discriminator as effectively as possible, while the discriminator can evolve to effectively distinguish between original data and data generated by the generator.

[0108] An autoencoder is a neural network that aims to reproduce the input itself as an output. An autoencoder comprises an input layer, at least one hidden layer, and an output layer. In this case, since the number of nodes in the hidden layer is smaller than that in the input layer, the data dimensionality is reduced, leading to compression or encoding. Furthermore, data output from the hidden layer is fed into the output layer. In this case, since the number of nodes in the output layer is larger than that in the hidden layer, the data dimensionality increases, leading to decompression or decoding.

[0109] Meanwhile, autoencoders express input data as hidden layer data by adjusting the connection strengths of neurons through learning. The hidden layer expresses information with a smaller number of neurons than the input layer. The ability to reproduce input data as output implies that the hidden layer has discovered and expressed hidden patterns in the input data.

[0110] Semi-supervised learning is a type of machine learning that utilizes both labeled and unlabeled training data. One technique for semi-supervised learning is to infer labels for unlabeled training data and then use these inferred labels to perform training. This technique can be useful in situations where labeling is expensive.

[0111] Reinforcement learning is the theory that, if an agent is given an environment in which it can determine the optimal action at any given moment, it can find the optimal path through experience without data. Reinforcement learning is primarily implemented using a Markov Decision Process (MDP). A Markov Decision Process is described as follows: first, an environment containing the information necessary for the agent to take the next action is provided; second, how the agent will act in that environment is defined; third, what rewards the agent will receive for performing well and what penalties will be imposed for performing poorly is defined; and fourth, the optimal policy is derived through repeated experience until the future reward reaches its maximum.

[0112] The structure of an artificial neural network is specified by the model configuration, activation function, loss function or cost function, learning algorithm, optimization algorithm, etc., and the hyperparameters are set in advance before learning, and the model parameters are set through learning afterwards, so that the content can be specified.

[0113] For example, factors that determine the structure of an artificial neural network may include the number of hidden layers, the number of hidden nodes included in each hidden layer, the input feature vector, and the target feature vector.

[0114] Hyperparameters include various parameters that must be initially set for learning, such as initial values ​​for model parameters. Furthermore, model parameters include various parameters to be determined through learning. For example, hyperparameters may include initial values ​​for inter-node weights, initial values ​​for inter-node biases, mini-batch size, number of learning iterations, and learning rates. Furthermore, model parameters may include inter-node weights, inter-node biases, and more.

[0115] The loss function can be used as an indicator (standard) to determine the optimal model parameters during the learning process of an artificial neural network. In an artificial neural network, learning refers to the process of manipulating model parameters to reduce the loss function, and the purpose of learning can be seen as determining the model parameters that minimize the loss function. The loss function can mainly use the mean squared error (MSE) or the cross entropy error (CEE), but the present invention is not limited thereto. The cross entropy error can be used when the correct answer label is one-hot encoded. One-hot encoding is an encoding method that sets the correct answer label value to 1 only for neurons corresponding to the correct answer, and sets the correct answer label value to 0 for neurons that are not the correct answer.

[0116] In machine learning or deep learning, learning optimization algorithms can be used to minimize the loss function. Learning optimization algorithms include gradient descent (GD), stochastic gradient descent (SGD), momentum, Nesterov Accelerate Gradient (NAG), Adagrad, AdaDelta, RMSProp, Adam, and Nadam.

[0117] Gradient descent is a technique that adjusts model parameters in a direction that reduces the loss function value by considering the gradient of the loss function at the current state. The direction of model parameter adjustment is called the step direction, and the size of the adjustment is called the step size. Here, the step size can represent the learning rate. Gradient descent obtains the gradient by partially differentiating the loss function with respect to each model parameter, and updates the model parameters by changing the learning rate in the direction of the obtained gradient.

[0118] Stochastic gradient descent is a technique that divides learning data into mini-batches and performs gradient descent on each mini-batch to increase the frequency of gradient descent.

[0119] Adagrad, AdaDelta, and RMSProp are techniques for improving optimization accuracy by adjusting the step size in SGD. In SGD, momentum and NAG are techniques for improving optimization accuracy by adjusting the step direction. Adam combines momentum and RMSProp to improve optimization accuracy by adjusting the step size and step direction. Nadam combines NAG and RMSProp to improve optimization accuracy by adjusting the step size and step direction.

[0120] The learning speed and accuracy of artificial neural networks are significantly influenced by not only the network structure and the type of learning optimization algorithm, but also hyperparameters. Therefore, to obtain a good learning model, it is crucial not only to determine an appropriate artificial neural network structure and learning algorithm, but also to set appropriate hyperparameters.

[0121] Typically, hyperparameters are experimentally set to various values ​​while training an artificial neural network, and the learning results are set to the optimal values ​​that provide stable learning speed and accuracy.

[0122] Embodiments of the material property prediction system and method of the present invention can be applied to the field of development of new materials based on graphs and text descriptions related to material properties. For example, embodiments of the present invention can be applied to predicting battery characteristics such as shear modulus, bulk modulus, and bandgap of battery materials. In particular, the present embodiment can be applied to the development of a manganese-rich cathode material that has a different structure from existing cathode materials applied to batteries but can utilize manganese (Mn), a relatively inexpensive material, by more than 60%. In addition, embodiments of the present invention can be applied to the field of predicting the structure and properties of other structures such as crystal structures, molecular structures, protein structures, catalyst structures, and metal-organic frameworks (MOFs).

[0123] Figure 1 is a schematic diagram of an electronic device according to one embodiment of the present invention.

[0124] As illustrated in FIG. 1, an electronic device (100) according to an embodiment of the present invention may include at least one processor (110), a memory (120), and a communication unit (130). The electronic device (100) is a basic configuration for performing a computing environment, and in other embodiments, the electronic device (100) may be implemented by additionally or alternatively including some other components, may be implemented as a single or multiple entities, or may be implemented as only some of the disclosed configurations. Components or at least some of the components inside or outside the electronic device (100) may be connected to each other through a BUS, a GPIO (General Purpose Input / Output), an SPI (Serial Peripheral Interface), or a MIPI (Mobile Industry Processor Interface), thereby transmitting and receiving data or signals.

[0125] Unless the context clearly indicates otherwise, the processor (110) may refer to a set of one or more processors, and may control components of the processor (110) and the electronic device (100) by executing software (e.g., commands, programs, etc.) stored in at least the memory (120). In addition, the processor (110) may perform various operations such as calculations, processing, data generation or processing, and may read data from or store data in the memory (120). The processor (110) may be composed of at least one core and may include a processor for data analysis, machine learning (ML), or deep learning (DL), such as a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), or a tensor processing unit (TPU). The processor (110) may read software stored in the memory (120) to perform data processing for machine learning (or deep learning) of the present invention. According to one embodiment of the present disclosure, the processor (110) can perform operations for learning a neural network. The processor (110) can perform calculations for learning a neural network, such as processing input data for learning in deep learning, extracting features from the input data, calculating errors, and updating weights of the neural network using backpropagation. At least one of the CPU, GPGPU, and TPU of the processor (110) can process learning of a neural network model. For example, the CPU and GPGPU can together process learning of a neural network model and data classification using a neural network model. In addition, in one embodiment of the present disclosure, at least one processor (110) of the electronic device (100) can be used together to process learning of a neural network model and data classification using a neural network model.

[0126] The memory (120) is for storing various data, and the data is data acquired, processed, or used by at least one component of the electronic device (100), and may include software (e.g., commands, programs, etc.). Unless explicitly expressed otherwise in the context, the memory (120) may refer to a set of one or more memories, and may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a RAM (Random Access Memory), a SRAM (Static Random Access Memory), a ROM (Read Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, an optical disk, and a web storage that performs a storage function on the Internet. The instructions or programs or software stored in the memory (120) may be used to refer to an operating system, an application for controlling components of the electronic device (100), or middleware that provides various functions to the application so that the application can utilize the components of the electronic device (100). In one embodiment, when the processor (110) performs a specific operation, the memory (120) may store instructions that are performed by the processor (110) and correspond to the specific operation.

[0127] The communication unit (130) performs wireless or wired communication between the electronic device (100) and another device (e.g., a user terminal or another server), and the communication unit (130) can use wireless communication systems according to methods such as eMBB, URLLC, MMTC, LTE, LTE-A, NR, UMTS, GSM, CDMA, WCDMA, TDMA, FDMA, OFDMA, SCFDMA, WiBro, WiFi, Bluetooth, NFC, GPS, or GNSS. In addition, the communication unit (130) can use various wired communication systems such as USB, HDMI, RS-232 (Recommended Standard-232), POTS (Plain Old Telephone Service), PSTN (Public Switched Telephone Network), xDSL (x Digital Subscriber Line), RADSL (Rate Adaptive DSL), MDSL (Multi Rate DSL), VDSL (Very High Speed ​​DSL), UADSL (Universal Asymmetric DSL), HDSL (High Bit Rate DSL), and local area network (LAN). In one embodiment of the present invention, the communication unit (130) can be configured regardless of the communication mode such as wired or wireless, and can be configured with various communication networks such as a personal area network (PAN), a wide area network (WAN), etc. Additionally, the communication network may be the well-known World Wide Web (WWW), and may also utilize a wireless transmission technology used for short-distance communication, such as infrared (IrDA: Infrared Data Association) or Bluetooth.

[0128] An electronic device (100) according to an embodiment of the present invention can execute software that configures a material property prediction system or a material property prediction method.

[0129] As illustrated in FIG. 2, a method for predicting material properties of one embodiment of the present invention may include a step of inputting material information into a first AI model to extract a graph embedding (S110), a step of inputting a text description of the crystal structure of the material into a second AI model to extract a text embedding (S120), a step of dividing the text embedding into a plurality of structural information embeddings (S130), and a step of combining at least one of the graph embedding and the plurality of structural information embeddings (S140).

[0130] In the step (S110) of inputting material information into the first AI model to extract a graph embedding, a graph embedding including connectivity and interaction patterns within the crystal structure can be generated based on material information about the crystal structure of the material. The first AI model of one embodiment of the present invention can generate a graph embedding, which is a graph representation of the crystal structure of the material, by iteratively refining graph nodes corresponding to atoms of the crystal structure and connection information between each atom. The graph-based structural embedding (graph embedding) of one embodiment can be generated by a graph-based model that derives an embedding through a graph input for the crystal structure of the material. For example, one embodiment can use the coGN (Connectivity Optimized Graph Networks) model, which exhibits excellent performance in capturing the connectivity of the crystal structure among various graph-based models. In one embodiment, the coGN model receives graph data as input, encodes it, and generates a vector, and specifically, encodes the local atomic environment in the crystal structure of the material to predict properties (characteristics, physical properties) that depend on short-range interactions. These graph-based models can excel at capturing local information around atoms, namely short-range interactions between atoms and their surroundings. However, these graph models also have the disadvantage of being unable to account for non-local information, which plays a crucial role in determining the properties of materials, where long-range interactions between atoms or global structural characteristics are crucial. To address this issue, one embodiment of the present invention combines a language model-based text embedding with the graph (structural) embedding of the graph model to train the model, thereby improving the performance of material property prediction.

[0131] Referring to FIG. 4, the first AI model may include an embedding layer (Embedding) that encodes graph nodes for atoms of a crystal structure of material information and connection information between atoms, an interaction layer (Interaction) that iteratively refines the representation of the material based on the graph nodes and connection information, and a first projection head (Projection head) that projects the refined structural embedding in the interaction layer into a 128-dimensional vector. The embedding layer may then encode the characteristics of the initial atoms and connection information between atoms as a starting point of the interaction layer and transmit them to the interaction layer. The interaction layer may iteratively refine the graph representation of the material through message transmission between graph nodes through multiple iterations, for example, five iterations (here, h in FIG. 4 represents a node and m represents an edge embedding). The refined structural embedding may be projected into a 128-dimensional vector by the first projection head.

[0132] In the step (S120) of extracting text embeddings by inputting a text description of the crystal structure of a material into a second AI model, the collected text descriptions related to the crystal structure of the material may be input into the second AI model (LM; Language Model). In one embodiment of the present invention, the text descriptions related to the material may be generated from a text generator (e.g., Robocrystallographer package) that generates text descriptions of the crystal structure of the material in a manner similar to how an actual crystallographer analyzes the structure.

[0133] In one embodiment, a text description of a crystal structure generated by a text generator may include symmetry, local environment, and extended connectivity of the crystal structure. In addition, the text description may include information that can identify molecule names, component orientations, heterostructure information, etc. For example, in one embodiment, when the material information "SnO2" is given as input to the text generator, "SnO2 is Rutile structured and crystallizes in the tetragonal P4_2 / mnm space group. The structure is three-dimensional. Sn(1) is bonded to six equivalent O(1) atoms to form a mixture of edge and corner-sharing SnO6 octahedra. The corner-sharing octahedral tilt angles are 51°. All Sn(1)-O(1) bond lengths are 2.09 A text description such as ". O(1) is bonded in a trigonal planar geometry to three equivalent Sn(1) atoms." may be output. Such a text description may include a wide range of information, including global properties (e.g., space group and crystal type), local information (e.g., bond length and coordination environment), and semiglobal properties (e.g., geometry, connectivity, and structural arrangement). The text description generated and collected by the text generator may be input into a second AI model based on a language model to extract a text embedding. The text description for the material obtained in this way may be embedded based on a language model trained on material science literature. In one embodiment of the present invention, MatSciBERT, which is a language model trained on material science literature, may be used as the second AI model, and the second AI model may receive a text description and generate a text embedding.

[0134] As illustrated in FIG. 3, the text description generated for a material includes global information, semi-global information, and local information about the crystal structure of the material. That is, in a text description composed of sentences about the crystal structure of a material, each sentence can be distinguished as describing global information, semi-global information, and local information about the crystal structure of the material based on words, verbs, adjectives, etc. included in the sentence. For example, in the text description of the material "DyPd3" in FIG. 3, the sentence "DyPd3 is Uranium Silicide Structured and crystallizes in the cubic Pm-3m space group." contains words such as "Uranium Silicide," "cubic," and "space group," as well as words such as "structured" and "crystallizes," and thus this sentence can be distinguished as representing global information including the space group, crystal type, and dimensionality of the crystal structure of the material. Also, the sentence "Dy(1) is bonded to twelve equivalent Pd(1) atoms to form a mixture of face- and corner-sharing DyPd12 cuboctahedra." contains words such as "twelve equivalent Pd(1) atoms," "face and corner-sharing," and "cuboctahedra," so this sentence can be distinguished as representing semiglobal information including connectivity, structural arrangement, and geometry of the material's crystal structure. Similarly, the sentence "Pd(1) is bonded in a distorted square co-planar geometry to four equivalent Dy(1) atoms." contains words such as "distorted square co-planar geometry" and "four equivalent Dy(1) atoms", so this sentence can also be distinguished as representing semiglobal information. In addition, the sentence "All Dy(1)-Pd(1) bond lengths are 2.92. "bond lengths", "2.92 in the sentence ", this sentence can be distinguished as representing local information including interatomic bond lengths or coordination environments of the material crystal structure.

[0135] That is, in the step (S120) of extracting text embedding by inputting a text description of the crystal structure of the material into the second AI model, the second AI model can generate multiple structural information embeddings based on words included in the sentences when encoding each sentence of the text description.

[0136] Referring to FIG. 4, the second AI model may include a language model (LM) that receives a text description of the crystal structure of a material and generates a text embedding, and a second projection head that projects the text embedding into a 128-dimensional vector. The language model (LM) may generate and distinguish three types of structural information embeddings (global information embedding, semi-global information embedding, and local information embedding) for each sentence based on words included in the text description of the input material crystal structure. The generated text embedding may be projected into a 128-dimensional vector by the second projection head.

[0137] In the step (S130) of dividing the text embedding into multiple structural information embeddings, the structural information embeddings generated from multiple sentences included in a single text description through the second AI model may include a global information embedding, a semi-global information embedding, and a local information embedding, and these global information embeddings, semi-global information embeddings, and local information embeddings may be mutually distinguishable. In one embodiment, the second AI model may include an identifier for identifying each structural information embedding in each embedding.

[0138] In the step (S140) of combining at least one of a graph embedding and a plurality of structural information embeddings, a graph embedding (structural embedding) generated through a first AI model and a structural information embedding (text embedding) generated through a second AI model can be combined in a combining layer.

[0139] In the first AI model, the refined graph representation (structural embedding) through the interaction layer can be combined with the text embedding in the concatenation layer, passing through the first projection head, which projects the learned features into a 128-dimensional vector. Furthermore, in one embodiment, the structural embedding can be combined with at least one of the three structural information embeddings of the text embedding. The text embedding, including the structural information embedding, can be combined with the structural embedding (graph embedding) in the concatenation layer, passing through the second projection head. That is, in one embodiment of the present invention, the graph embedding can be combined with a global information embedding, in another embodiment, the graph embedding can be combined with a semi-global information embedding, in another embodiment, the graph embedding can be combined with a local information embedding, in another embodiment, the graph embedding can be combined with a global information embedding and a semi-global information embedding, and in another embodiment, the graph embedding can be combined with a text embedding including all structural information embeddings.

[0140] One embodiment of the present invention includes a step (S150) of predicting the properties of a target material, wherein the properties of the target material can be predicted by inputting a combined multimodal embedding (a combination of a graph embedding and a text embedding) in a combined layer into a fully connected layer.

[0141] The material property prediction method of the embodiments of the present invention and the material property prediction system that executes the method can exhibit improved prediction performance compared to models that predict material properties. As shown in [Table 1] below, the performance of the embodiments of the present invention was evaluated by comparing them with a graph model (Structure only) that uses only structural information of the material crystal structure to predict the existing material crystal structure and a language model (Text only) that uses only textual descriptions of the material crystal structure.

[0142] MAE(R2) Shear modulus Bulk modulus Structure only0.0883±0.0061 (0.884)0.0554±0.0088 (0.932)Text only (①+②+③)0.0825±0.0032 (0.895)0.0455±0.0005 (0.949)Structure + Text(①)0.0886±0.0009 (0.875)0.0546±0.0002 (0.919)Structure + Text(②)0.0749±0.0035 (0.908)0.0399±0.0018 (0.960)Structure + Text(③)0.0872±0.0021 (0.882)0.0516±0.0008 (0.917)Structure + Text(①+②)0.0759±0.0013 (0.906)0.0385±0.0005 (0.962)Structure + Text(①+②+③)0.0756±0.0008 (0.907)0.0406±0.0028 (0.960)

[0143] The learning models of the embodiments compared and evaluated with existing graph models and language models can be composed of a first model (Structure + Text(①)) learned with a multimodal embedding that combines a graph embedding and a global information embedding, a second model (Structure + Text(②)) learned with a multimodal embedding that combines a graph embedding and a semi-global information embedding, a third model (Structure + Text(③)) learned with a multimodal embedding that combines a graph embedding and a local information embedding, a fourth model (Structure + Text(①+②)) learned with a multimodal embedding that combines a graph embedding and a global information embedding and a semi-global information embedding, and a fifth model (Structure + Text(①+②+③)) learned with a multimodal embedding that combines a graph embedding and all structural information embeddings.

[0144] The performance of the model was evaluated using the Mean Absolute Error (MAE) for the main properties of the material, namely the shear modulus and bulk modulus of the material. A lower MAE value indicates higher prediction accuracy (prediction performance) (R2 is the coefficient of determination; a higher R2 indicates higher prediction performance).

[0145] As shown in [Table 1], the graph model (Structure only), which is a reference model that uses only the structural information of the material, shows a shear modulus of 0.0883 and a volume modulus of 0.0554, and the language model (Text only) predicted based on the text description of the material crystal structure shows a shear modulus of 0.0825 and a volume modulus of 0.0455, showing that the language model based on text description (Text only) showed improved performance compared to the graph model.

[0146] Among the embodiments of the present invention, the first model (Structure + Text (①)) showed a shear modulus of 0.0886 and a volume modulus of 0.0546. Compared to the reference graph model and language model, the shear modulus prediction performance was lower, but the volume modulus prediction performance showed improved prediction performance compared to the graph model. The second model (Structure + Text (②)) showed a shear modulus of 0.0749 and a volume modulus of 0.0399. Compared to the reference graph model and language model, the prediction performance showed improved results. In particular, the second model showed significantly improved shear modulus prediction performance compared to the reference models. The third model (Structure + Text (③)) showed a shear modulus of 0.0872 and a volume modulus of 0.0516. Compared to the reference graph model, the prediction performance was improved, but compared to the language model that used all text descriptions, the prediction performance was lower. The fourth model (Structure + Text(①+②)) showed a shear modulus of 0.0759 and a volume modulus of 0.0385, demonstrating improved prediction performance compared to the reference graph model and language model. In particular, the fourth model showed significantly improved prediction performance for volume modulus compared to the reference models. The fifth model (Structure + Text(①+②+③)) showed a shear modulus of 0.0756 and a volume modulus of 0.0406, demonstrating improved prediction performance compared to the reference models.

[0147] As described above, in the prediction performance evaluation of the reference models and the embodiments of the present invention, the second model, which was trained by combining the semi-global information embedding among the text embeddings generated from the text description-based language model with the graph embedding (structural embedding) of the graph model, showed the highest performance improvement in shear modulus prediction. This confirms the importance of semi-global information, such as the geometry and connectivity of the crystal structure, in predicting the mechanical properties of materials, such as the shear modulus. In addition, the fourth model, which was trained by combining the global information embedding and the semi-global information embedding with the graph embedding, showed the highest performance improvement in bulk modulus prediction. This confirms the importance of global characteristics, such as the crystal type and space group of the crystal structure, and semi-global characteristics, such as the geometry and connectivity of the crystal structure, in predicting the mechanical properties of materials, such as the bulk modulus.

[0148] This demonstrates that textual descriptions of crystal structures can complement graph models, filling gaps related to global and semi-global properties that are difficult to identify in graph models and improving the overall model performance.

[0149]

[0150] As illustrated in FIG. 5, a method for predicting material properties of another embodiment of the present invention may include a step of inputting material information into a first AI model executed by a processor to extract a graph embedding (S210), a step of inputting a text description of the crystal structure of the material into a second AI model executed by the processor to extract a text embedding (S220), a step of cross-attentioning the graph embedding and the text embedding through the processor (S230), a step of fine-tuning the second AI model for which pre-learning has been completed (S240), and a step of inputting target material information into the first AI model to predict the properties of the target material (S250).

[0151] In the step (S210) of extracting a graph embedding by inputting material information into the first AI model, a graph embedding including connectivity and interaction patterns within the crystal structure can be generated based on material information about the crystal structure of the material. The first AI model of another embodiment of the present invention can generate a graph embedding, which is a graph representation of the crystal structure of the material, by iteratively refining graph nodes corresponding to atoms of the crystal structure and connection information between each atom. The graph-based structural embedding (graph embedding) of another embodiment can be generated by a graph-based model that derives an embedding through a graph input for the crystal structure of the material. For example, another embodiment can use the coGN (Connectivity Optimized Graph Networks) model, which shows excellent performance in capturing the connectivity of the crystal structure among various graph-based models. In another embodiment, the coGN model receives graph data as input, encodes it, and generates a vector, and specifically, encodes the local atomic environment in the crystal structure of the material to predict properties (characteristics, physical properties) that depend on short-range interactions. These graph-based models can excel at capturing local information around atoms, namely short-range interactions between atoms and their surroundings. However, these graph models also have the disadvantage of being unable to account for non-local information, which plays a crucial role in determining the properties of materials, such as long-range interactions between atoms or global structural characteristics. To address this issue, another embodiment of the present invention trains a model by cross-attentioning the graph (structural) embedding of the graph model and the text embedding based on a language model, thereby improving the performance of material property prediction.

[0152] Referring to FIG. 6, the first AI model may include an embedding layer (Embedding) that encodes graph nodes for atoms of the crystal structure of material information and connection information between atoms, and an interaction layer (Interaction) that iteratively refines the representation of the material based on the graph nodes and connection information. The embedding layer may then encode the characteristics of the initial atoms and connection information between atoms as a starting point of the interaction layer and pass them to the interaction layer. The interaction layer may iteratively refine the graph representation of the material through message passing between graph nodes through multiple iterations, for example, three iterations (h in FIGS. 6 and 7 represents a node, and m represents an edge embedding). The refined structural embedding may be passed to a cross-attention layer (Xattn) to perform text embedding and cross-attention. Here, the first AI model serves as a decoder of a transformer, and the graph embedding generated from the first AI model is set as a query in the cross-attention layer. In model pretraining, queries can predict masked tokens by referencing the text embeddings of a second AI model, the encoder. In other words, model pretraining involves masking graph nodes in the material information to generate masked tokens, and then having the cross-attention layer predict the original element of the masked token.

[0153] In the step (S220) of extracting text embeddings by inputting a text description of the crystal structure of a material into a second AI model, the collected text descriptions related to the crystal structure of the material may be input into the second AI model (LM; Language Model). In another embodiment of the present invention, the text descriptions related to the material may be generated from a text generator (e.g., Robocrystallographer package) that generates text descriptions of the crystal structure of the material in a manner similar to how an actual crystallographer analyzes the structure.

[0154] In another embodiment, a text description of a crystal structure generated by a text generator may include symmetry, local environment, and extended connectivity of the crystal structure. In addition, the text description may include information that can identify molecule names, component orientations, heterostructure information, etc. For example, in one embodiment, when the material information "SnO2" is given as input to the text generator, "SnO2 is Rutile structured and crystallizes in the tetragonal P4_2 / mnm space group. The structure is three-dimensional. Sn(1) is bonded to six equivalent O(1) atoms to form a mixture of edge and corner-sharing SnO6 octahedra. The corner-sharing octahedral tilt angles are 51°. All Sn(1)-O(1) bond lengths are 2.09 A text description such as ". O(1) is bonded in a trigonal planar geometry to three equivalent Sn(1) atoms." may be output. Such a text description may include a wide range of information, including global properties (e.g., space group and crystal type), local information (e.g., bond length and coordination environment), and semiglobal properties (e.g., geometry, connectivity, and structural arrangement). The text description generated and collected by the text generator may be input into a second AI model based on a language model to extract a text embedding. The text description for the material obtained in this way may be embedded based on a language model trained on material science literature. In one embodiment of the present invention, MatSciBERT, which is a language model trained on material science literature, may be used as the second AI model, and the second AI model may receive a text description and generate a text embedding.

[0155] Referring to FIG. 3, the text description generated for a material includes global information, semi-global information, and local information about the crystal structure of the material. That is, in a text description composed of sentences about the crystal structure of a material, each sentence can be distinguished as describing global information, semi-global information, and local information about the crystal structure of the material based on words, verbs, adjectives, etc. included in the sentence. For example, in the text description of the material "DyPd3" in FIG. 3, the sentence "DyPd3 is Uranium Silicide Structured and crystallizes in the cubic Pm-3m space group." contains words such as "Uranium Silicide," "cubic," and "space group," as well as words such as "structured" and "crystallizes," and thus this sentence can be distinguished as representing global information including the space group, crystal type, and dimensionality of the crystal structure of the material. Also, the sentence "Dy(1) is bonded to twelve equivalent Pd(1) atoms to form a mixture of face- and corner-sharing DyPd12 cuboctahedra." contains words such as "twelve equivalent Pd(1) atoms," "face and corner-sharing," and "cuboctahedra," so this sentence can be distinguished as representing semiglobal information including connectivity, structural arrangement, and geometry of the material's crystal structure. Similarly, the sentence "Pd(1) is bonded in a distorted square co-planar geometry to four equivalent Dy(1) atoms." contains words such as "distorted square co-planar geometry" and "four equivalent Dy(1) atoms", so this sentence can also be distinguished as representing semiglobal information. In addition, the sentence "All Dy(1)-Pd(1) bond lengths are 2.92. "bond lengths", "2.92 in the sentence ", this sentence can be distinguished as representing local information including interatomic bond lengths or coordination environments of the material crystal structure.

[0156] That is, in the step (S220) of extracting text embedding by inputting a text description of the crystal structure of the material into the second AI model, the second AI model can generate multiple structural information embeddings based on words included in the sentences when encoding each sentence of the text description.

[0157] Referring to Fig. 6, the second AI model may include a language model (LM) that receives a text description of the crystal structure of a material and generates a text embedding, and a second projection head that projects the text embedding into a 128-dimensional vector. The language model (LM) may generate and distinguish three types of structure information embeddings (global information embedding, semi-global information embedding, and local information embedding) for each sentence based on words included in the sentences included in the text description of the input material crystal structure. Referring to Fig. 6, the second AI model, which is the language model (LM), may function as an encoder of a transformer, and the text embedding generated by the second AI model may function as a key and value referenced in the query graph embedding. That is, in the cross-attention layer, the query graph embedding may predict a masking token by referencing the text embedding.

[0158] In the step (S230) of cross-attentioning the graph embedding and the text embedding of the first AI model, which is a decoder of the transformer, and the text embedding of the second AI model, which is an encoder, may perform cross-attention in the cross-attention layer (Xattn). That is, the graph embedding of the first AI model may be set as a query, and the text embedding of the second AI model may be set as a key and value referenced by the query. In the pre-training of the model, a masking token generated by masking any one of the graph nodes may be predicted to be the original element of the masking token by referring to the text embedding. In this way, by repeating the process of generating a masking token from graph information about the crystal structure of a material and predicting it, a learning model of another embodiment of the present invention may be pre-trained.

[0159] In the step (S240) of fine-tuning the second AI model for which pre-training has been completed, the second AI model, which is large in size, can be fine-tuned using the LoRA (Low-Rank Adaptation) method. The LoRa method was developed to improve the shortcomings of the existing Full FineTuning. When fine-tuning using the LoRa method, it has the advantage of indirectly training some layers of the neural network by optimizing the rank decomposition matrices for changes in the layers during adaptation while keeping the pre-trained weights fixed. In another embodiment, the second AI model can significantly reduce the number of trainable parameters by freezing the weights of the pre-trained model and inserting low-rank (rank decomposition) matrices into each layer. This reduces computational and memory requirements while maintaining performance similar to that of full fine-tuning.

[0160] The step (S250) of inputting target material information into the first AI model to predict the properties of the target material can predict the properties of the target material by adding an output block to the rear of the learning model for which fine tuning has been completed, with reference to FIG. 7.

[0161] The property prediction model of the present invention as described above may include, as another embodiment, a prediction model fine-tuned using the LoRA method based on a pre-learned prediction model implemented from step (S210) to step (S230), and a prediction model learned from scratch from step (S210) to step (S250) without pre-learning.

[0162] In addition, the prediction model of another embodiment of the present invention can combine the graph embedding generated from the first AI model and the text embedding generated from the second AI model in a concatenate layer. The graph representation (structural embedding) refined through the interaction layer in the first AI model can be combined with the text embedding in the concatenate layer through a first projection head that projects the learned features into a 128-dimensional vector. Furthermore, in another embodiment, the structural embedding can be combined with at least one of three structural information embeddings of the text embedding. The text embedding including the structural information embedding can be combined with the structural embedding (graph embedding) in the concatenate layer through a second projection head. That is, in another embodiment of the present invention, the graph embedding can be combined with a global information embedding, in another embodiment, the graph embedding can be combined with a semi-global information embedding, in another embodiment, the graph embedding can be combined with a local information embedding, in another embodiment, the graph embedding can be combined with a global information embedding and a semi-global information embedding, and in another embodiment, the graph embedding can be combined with a text embedding including all structural information embeddings.

[0163] The material property prediction method of the embodiments of the present invention and the material property prediction system executing the method exhibit improved prediction performance compared to existing models for predicting material properties. As shown in [Table 2] below, the performance of the embodiments of the present invention was evaluated by comparing them with a graph model (coGN) that uses only structural information of the material crystal structure to predict the existing material crystal structure and a language model (MatSciBert) that uses only textual descriptions of the material crystal structure.

[0164] Graph model (coGN) Language model (MatSciBert) Concatenation model (Concat) First learning model (Xattn) Second learning model (Xattn(ft)) Total Energy 0.67 3 0.39 0 0.35 3 0.27 7 0.25 6 Bandgap 0.38 10.42 0 0.43 2 0.39 4 0.35 4 Shear Modulus 0.09 1 1 0.08 9 4 0.07 5 10.07 1 8 0.06 9 4 Bulk Modulus 0.04 9 8 0.05 2 7 0.04 0 6 0.04 1 5 0.039 2

[0165] The models that serve as the basis for performance evaluation are a graph model and a language model. The coGN (Connectivity Optimized Graph Networks) model, which shows excellent performance in capturing the connectivity of crystal structures, was used as the graph model, and MatSciBERT, a language model trained on material science literature, was used as the language model.

[0166] Embodiments of the present invention in which the performance of predicting material properties is compared with that of a reference model may be composed of a combined model that concatenates graph embedding and text embedding, a first learning model (Xattn) in which graph embedding and text embedding are cross-attentioned without prior learning, and a second learning model (Xattn(ft)) that fine-tunes the first learning model for which prior learning has been completed.

[0167] The performance of the model was evaluated using the Mean Absolute Error (MAE) for the main properties of the material, namely the bandgap, shear modulus, and bulk modulus of the material, with a lower MAE value indicating higher prediction accuracy (prediction performance).

[0168] As shown in [Table 2], the graph model (coGn), which is a reference model using only the structural information of the material, shows a band gap of 0.381, a shear modulus of 0.0911, and a bulk modulus of 0.0498, and the language model (MatSciBert) predicted based on the text description of the material crystal structure shows a band gap of 0.420, a shear modulus of 0.0894, and a bulk modulus of 0.0527.

[0169] Among the embodiments of the present invention, the coupled model (Concat) was shown to have improved performance compared to the graph model and the language model in shear modulus and bulk modulus. The first learning model (Xattn), which was trained without prior learning, showed a band gap of 0.394, a shear modulus of 0.0718, and a bulk modulus of 0.0415, showing improved prediction performance for shear modulus and bulk modulus compared to the reference graph model, and improved performance in all of the band gap, shear modulus, and bulk modulus compared to the reference language model. The first learning model (Xattn) shows improved prediction performance for band gap and shear modulus compared to the coupled model. The second learning model (Xattn(ft)) shows significantly improved prediction performance in all of the band gap, shear modulus, and bulk modulus compared to the reference graph model, language model, coupled model, and the first learning model.

[0170] As described above, in the prediction performance evaluation of the reference models and other embodiments of the present invention, the second learning model, which was fine-tuned on a pre-trained model by cross-attentioning graph embedding and text embedding, showed the highest performance improvement in the material property prediction performance evaluation.

[0171] Meanwhile, one embodiment of the present invention may be implemented as an application-specific integrated circuit (ASIC) manufactured to suit the special functions of a specific application field and device.

[0172] An application-specific integrated circuit is also called an application-specific semiconductor. Unlike standard semiconductors that have set specifications and can be applied to any electronic product or application as long as certain requirements are met, an application-specific semiconductor is an integrated circuit that a semiconductor manufacturer manufactures according to a specific order for a specific product or function. In other words, an application-specific semiconductor is designed and manufactured to perform only the functions required for a specific device or specific function. Depending on the design method, application-specific semiconductors are largely divided into full custom ICs, which design and manufacture the circuit from scratch according to the user's needs, and semi-custom ICs, which design and manufacture the circuit using some standardized designs.

[0173] Application-specific semiconductors are primarily used in communications systems, high-performance computing systems, consumer electronics, automobiles, industrial automation, medical devices, military, and aerospace industries. Recently, they are being applied to AI semiconductors that perform large-scale calculations required for AI implementation with high performance and power efficiency.

[0174] Application-specific integrated circuits (ASICs) are core components of network routers, switches, and modems in communication systems, performing data packet processing, protocol conversion, and signal processing to deliver high throughput and low latency. In high-performance computing systems, ASICs are key components for high-speed and parallel processing. In consumer electronics such as digital cameras, smartphones, tablets, and game consoles, ASICs provide high-performance and low-power solutions required to perform specific functions. In the automotive industry, ASICs control various electronic systems within vehicles, and in industrial automation systems, ASICs provide solutions for high-precision control and high-performance processing.

[0175] An application-specific integrated circuit to which embodiments of the present invention are applied includes a memory in which an individual memory interface (I / F) is implemented, and may include a plurality of functional blocks that request memory access. Each functional block may be a direct memory access (DMA) functional block, a processor, a video processor, a cache controller, a decompression block, or a data path block. The basic configuration of the application-specific integrated circuit may include a transistor that amplifies or switches an electrical signal, a logic gate that is a circuit that performs a logical function by combining transistors, a memory cell that stores data, an analog circuit that is a circuit that processes a continuous voltage or current by combining transistors, and an IP core (Intellectual Property Core) such as a microprocessor, DSP, or graphic core that is pre-designed to perform a specific function.

[0176] The ASIC may also include a separate memory I / F interfacing with individual memories and an embedded memory I / F interfacing with embedded memories. The separate memory I / F is connected to each functional block, receives memory access signals (e.g., control signals, address signals, and data signals), and generates signals for controlling the individual memories based on these input signals. The embedded memory I / F is connected to each functional block, receives memory access signals (e.g., control signals, address signals, and data signals), and generates modified memory access signals for controlling the embedded memories based on these input signals. The separate memory I / F and the embedded memory I / F may be designed within the memory control block of the ASIC to provide a memory control structure that can be flexibly applied to both the individual memories and the embedded memories.

[0177] Additionally, an application-specific integrated circuit (ASIC) for an artificial neural network (ANN) may be configured to include a plurality of neurons arranged in an array and a plurality of synaptic circuits, each neuron including a register, a microprocessor, and at least one input, and each synaptic circuit including a memory for storing synaptic weights. Each neuron of the ASIC may be connected to at least one other neuron through one of the plurality of synaptic circuits.

[0178] Although the present disclosure has been described above as being generally implemented by a computing device, those skilled in the art will appreciate that the present disclosure may also be implemented in combination with computer-executable instructions and / or other program modules that may be executed on one or more computers and / or as a combination of hardware and software.

[0179] Those skilled in the art will appreciate that information and signals may be represented using any of a variety of different technologies and techniques. For example, the data, instructions, commands, information, signals, bits, symbols, and chips referenced in the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0180] Those skilled in the art will appreciate that the various illustrative logical blocks, modules, processors, means, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, various forms of programs or design code (referred to herein, for convenience, as software), or a combination of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

[0181] The various embodiments presented herein can be implemented as a method, apparatus, or article of manufacture using standard programming and / or engineering techniques. The term article of manufacture includes a computer program, carrier, or media accessible from any computer-readable storage device. For example, computer-readable storage media include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips, etc.), optical disks (e.g., CDs, DVDs, etc.), smart cards, and flash memory devices (e.g., EEPROMs, cards, sticks, key drives, etc.). Furthermore, various storage media presented herein include one or more devices and / or other machine-readable media for storing information.

[0182] It should be understood that the specific order or hierarchy of steps in the presented processes is merely an example of exemplary approaches. It should be understood that the specific order or hierarchy of steps in the processes may be rearranged within the scope of the present disclosure based on design priorities. The appended method claims provide elements of various steps in a sample order, but are not intended to be limited to the specific order or hierarchy presented.

[0183] The description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the embodiments disclosed herein, but is to be construed in the broadest scope consistent with the principles and novel features disclosed herein.

Claims

1. As a system for predicting the properties of materials, at least one processor; and At least one memory for storing instructions or information executed by at least one processor; The operation performed by the above command or information executed by at least one processor is: An operation of inputting material information into a first AI model to extract a graph embedding, an operation of inputting a text description of the crystal structure of the material into a second AI model to extract a text embedding, an operation of dividing the text embedding into a plurality of structural information embeddings, and an operation of combining the graph embedding with at least one of the plurality of structural information embeddings, wherein the structural information embedding is divided into global information, semi-global information, and local information of the crystal structure. Material properties prediction system.

2. In claim 1, The first AI model comprises an embedding layer encoding graph nodes for atoms of the crystal structure of the material information and connection information between atoms; and An interactive layer that iteratively refines the representation of the material based on the above graph node and connection information; Material properties prediction system.

3. In claim 1, The above global information includes comprehensive arrangement information of the crystal structure, including mineral type, space group, dimensionality, and symmetry property. The above semi-global information includes information on the arrangement of atoms within the crystal structure, including geometry and connectivity within the crystal structure. The above local information includes atomic level details within the crystal structure, including the types of atoms within the crystal structure and the interatomic bond lengths. Material properties prediction system.

4. In claim 1, The graph embedding projected into a 128-dimensional vector through the first projection head of the first AI model and the text embedding projected into a 128-dimensional vector through the second projection head of the second AI model are combined in a combination layer to generate a multimodal embedding. Material properties prediction system.

5. In claim 4, The above multimodal embedding is input to a fully connected layer to predict the properties of the target material. Material properties prediction system.

6. In claim 3, The above graph embedding is combined with a text embedding including at least the semi-global information. Material properties prediction system.

7. In claim 3, The above graph embedding is combined with a text embedding including the global information and the semi-global information. Material properties prediction system.

8. A computerized method for predicting material properties, A step of extracting a graph embedding by inputting material information into a first AI model executed by a processor; A step of extracting text embeddings by inputting a text description of the crystal structure of the material into a second AI model executed by the processor; A step of dividing the text embedding into a plurality of structural information embeddings through a processor; A step of combining the graph embedding and at least one of the plurality of structural information embeddings through a processor; The above structural information embedding is distinguished by including global information, semi-global information, and local information of the above crystal structure. Method for predicting material properties.

9. In claim 8, In the step of extracting the above graph embedding, The material information input to the first AI model is encoded as graph nodes for atoms of the crystal structure of the material information and connection information between atoms in the embedding layer, and the representation of the material is iteratively refined based on the graph nodes and connection information in the interaction layer. Method for predicting material properties.

10. In claim 8, The above global information includes comprehensive arrangement information of the crystal structure, including mineral type, space group, dimensionality, and symmetry property. The above semi-global information includes information on the arrangement of atoms within the crystal structure, including geometry and connectivity within the crystal structure. The above local information includes atomic level details within the crystal structure, including the types of atoms within the crystal structure and the interatomic bond lengths. Method for predicting material properties.

11. In claim 8, Before the above combining step, The above graph embedding is projected into a 128-dimensional vector through the first projection head of the first AI model, and the above text embedding is projected into a 128-dimensional vector through the second projection head of the second AI model. In the above combining step, the projected graph embedding and the text embedding are combined in a combining layer to generate a multimodal embedding. Method for predicting material properties.

12. In claim 11, It further includes a step of predicting the properties of the target material; The step of predicting the above properties is to input the multimodal embedding into a fully connected layer to predict the properties of the target material. Method for predicting material properties.

13. In claim 10, In the above combining step, The above graph embedding is combined with a text embedding including at least the semi-global information. Method for predicting material properties.

14. In claim 10, In the above combining step, The above graph embedding is combined with a text embedding including the global information and the semi-global information. Method for predicting material properties.

15. In an application-specific integrated circuit comprising a memory storing information and instructions and a functional block including at least one processor requesting access to the memory, The above memory stores instructions or information for inputting material information into a first AI model to extract a graph embedding, inputting a text description of the crystal structure of the material into a second AI model to extract a text embedding, dividing the text embedding into a plurality of structural information embeddings, and combining the graph embedding with at least one of the plurality of structural information embeddings. The above structural information embedding is distinguished by including global information, semi-global information, and local information of the above crystal structure. Custom integrated circuits for material property prediction.

16. As a system for predicting the properties of materials, at least one processor; and At least one memory for storing instructions or information executed by at least one processor; The operation performed by the above command or information executed by at least one processor is: An operation including an operation of inputting material information into a first AI model to extract a graph embedding, an operation of inputting a text description of the crystal structure of the material into a second AI model to extract a text embedding, and an operation of cross-attentioning the graph embedding and the text embedding. Material properties prediction system.

17. In claim 16, The first AI model comprises an embedding layer encoding graph nodes for atoms of the crystal structure of the material information and connection information between atoms; and An interactive layer that iteratively refines the representation of the material based on the above graph node and connection information; Material properties prediction system.

18. In claim 16, The above text embedding includes comprehensive arrangement information of the material crystal structure, geometric information of atomic arrangement within the crystal structure, and atomic-level details within the crystal structure. The comprehensive arrangement information includes mineral type, space group, dimensionality, and symmetry property, the atomic arrangement geometric information includes geometry and connectivity within the crystal structure, and the atomic level detailed information includes types of atoms within the crystal structure and interatomic bond lengths. Material properties prediction system.

19. In claim 16, The above cross-attention predicts the query by setting the graph embedding as a query and the text embedding as a key and value. Material properties prediction system.

20. In claim 16, The actions performed by the above commands or information are: Further comprising an operation of fine-tuning the second AI model, and an operation of inputting target material information into the first AI model to predict the properties of the target material. Material properties prediction system.

21. In claim 20, The above fine tuning is performed using the LoRA (Low-Rank Adaptation) method, which freezes the second AI model weights and reduces the number of learnable parameters. Material properties prediction system.

22. A computerized method for predicting material properties, A step of extracting a graph embedding by inputting material information into a first AI model executed by a processor; A step of extracting text embeddings by inputting a text description of the crystal structure of the material into a second AI model executed by the processor; A step of cross-attentioning the graph embedding and the text embedding through a processor; Method for predicting material properties.

23. In claim 22, In the step of extracting the above graph embedding, The material information input to the first AI model is encoded as graph nodes for atoms of the crystal structure of the material information and connection information between atoms in the embedding layer, and the representation of the material is iteratively refined based on the graph nodes and connection information in the interaction layer. Method for predicting material properties.

24. In claim 22, The above text embedding includes comprehensive arrangement information of the material crystal structure, geometric information of atomic arrangement within the crystal structure, and atomic-level details within the crystal structure. The comprehensive arrangement information includes mineral type, space group, dimensionality, and symmetry property, the atomic arrangement geometric information includes geometry and connectivity within the crystal structure, and the atomic level detailed information includes types of atoms within the crystal structure and interatomic bond lengths. Method for predicting material properties.

25. In claim 22, In the above cross-attention step, The above graph embedding is set as a query and the above text embedding is set as a key and a value, so that the query is predicted by referring to the key and value. Method for predicting material properties.

26. In claim 25, A step of fine-tuning the second AI model; and A step of inputting target material information into the first AI model to predict the properties of the target material; Method for predicting material properties.

27. In claim 25, In the above fine tuning step, The above fine tuning is performed in the LoRA (Low-Rank Adaptation) method in which the weights of the second AI model are frozen and the number of learnable parameters is reduced. Method for predicting material properties.

28. In an application-specific integrated circuit comprising a memory storing information and instructions and a functional block including at least one processor requesting access to the memory, The above memory is, A command or information is stored that includes an operation of inputting material information into a first AI model to extract a graph embedding, inputting a text description of the crystal structure of the material into a second AI model to extract a text embedding, cross-attentioning the graph embedding and the text embedding, fine-tuning the second AI model, and inputting target material information into the first AI model to predict the properties of the target material. Custom integrated circuits for material property prediction.

Images