Spinel ceramic microwave dielectric constant prediction method of attention convolutional network based on component limitation

Abstract

The invention provides a spinel ceramic microwave dielectric constant prediction method based on a component-limited attention convolutional network, and the method comprises the following specific steps: firstly, collecting the chemical formula and dielectric constant of spinel type microwave dielectric ceramic in literatures, and converting the chemical formula into a basic element vector; and performing data embedding in combination with composition coding and logarithmic coding. Secondly, performing hyper-parameter optimization by adopting an Adam optimization algorithm; the embedded data is subjected to feature extraction through a Transform encoder, the convolutional layer further processes the features, and the pooling layer retains important features. Then, training the data by using a residual network; after training is completed, a to-be-predicted spinel type microwave dielectric ceramic chemical formula is input into the model, and the dielectric constant can be output. According to the method, more accurate performance prediction can be realized through chemical formula input, the material development efficiency is remarkably improved, and the experiment trial and error cost is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of prediction of dielectric properties of spinel microwave dielectric ceramics, and specifically provides a method for predicting the microwave dielectric constant of spinel ceramics based on a component-restricted attention convolutional network. Background Art

[0002] Microwave dielectric ceramics play a vital role in high-frequency applications. With their excellent dielectric constant, low dielectric loss and high thermal stability, they are widely used in devices such as microwave filters, antennas, resonators and oscillators. With the advancement of science and technology and the continuous improvement of communication needs, especially the development of 5G / 6G networks and advanced wireless communication systems, the demand for microwave dielectric ceramics is also growing rapidly. As one of the key indicators of the performance of microwave dielectric ceramics, the dielectric constant determines the efficiency of the material in signal transmission and the volume of the equipment. Microwave dielectric ceramics with high dielectric constants are suitable for medium and low frequency applications, which can improve signal transmission efficiency and make the equipment more compact; medium dielectric constant materials provide a balance between performance and loss, and are widely used in high-frequency filters and resonators. Microwave dielectric ceramics with low dielectric constants effectively reduce signal propagation delay and loss, and are suitable for high-frequency millimeter waves and long-distance communications.

[0003] In recent years, spinel microwave dielectric ceramics have played an important role in wireless communication fields such as communication technology, radar, sensors and microwave integrated circuits due to their excellent dielectric properties. The chemical formula of spinel is AB2O4, where the A site is occupied by a larger cation and the B site is occupied by a smaller cation, which are in the octahedral and tetrahedral coordination positions respectively. This unique coordination environment and ion arrangement give the spinel structure excellent dielectric properties. Therefore, in order to meet the requirements of high-frequency wireless communication systems for stability and low loss, it is particularly important to further optimize the design of spinel microwave dielectric ceramics.

[0004] With the rapid development of data-driven methods, we are able to predict material properties and accelerate the discovery and design of new materials. As one of the core technologies of data-driven methods, machine learning builds models by analyzing material data (such as chemical composition, structure, properties, etc.), which significantly improves the efficiency of material screening. Although traditional machine learning can effectively predict the performance of existing spinel data sets, it usually relies on complex feature engineering. In contrast, deep learning performs well in large-scale data and has the ability to automatically learn data features without relying on manually designed features. It can more efficiently accelerate the discovery and screening of high-quality materials.

[0005] Deep learning has significant advantages in material prediction. It extracts high-level features from data through multi-layer neural networks, reduces reliance on artificial feature design, and thus improves prediction accuracy. In addition, deep learning performs well in processing high-dimensional and nonlinear data, and can better capture complex material performance relationships. Its powerful generalization ability and ability to process complex data make deep learning have great application potential in materials science, which can more accurately predict material properties and accelerate the development and optimization of new materials. Therefore, it is of great significance to build a model for predicting microwave dielectric constant with simple input and excellent prediction performance. Summary of the invention

[0006] A method for predicting microwave dielectric constant of spinel ceramics based on component-restricted attention convolutional network, comprising the following steps:

[0007] 1) The chemical formula and dielectric constant of spinel microwave dielectric ceramics were collected from previous literature as data set samples.

[0008] 2) For the chemical formula in step 1), use Mat2vec encoding to convert it into a basic element vector representation.

[0009] 3) Based on the basis element vector in step 2), data embedding is performed by combining composition coding and logarithmic coding.

[0010] 4) Use the Adam optimization algorithm to adjust the hyperparameters in the model and configure a suitable hyperparameter combination, including the learning rate, the number of encoder layers, the number of attention mechanism heads and input dimensions, the weight drop rate, and the number of samples.

[0011] 5) For the data in step 3), feature extraction is performed using the Transformer encoder as the main body. Each encoding layer consists of a multi-head self-attention mechanism and a feedforward neural network, which effectively captures the complex relationships and high-order features of the data.

[0012] 6) The features extracted in step 5) are processed using a convolutional layer, and the processed features are reduced in dimension using a pooling layer to retain the most important feature information.

[0013] 7) The data obtained in step 6) is trained using a residual network. The skip connection in the residual network can effectively alleviate the gradient vanishing problem in the deep network, allowing the network to learn complex features at a deeper level.

[0014] 8) Use the trained dielectric constant prediction model to predict the dielectric constant of spinel microwave dielectric ceramics.

[0015] Compared with the prior art, the present invention has the following advantages, which are of practical value in the field of spinel microwave dielectric ceramics:

[0016] The present invention realizes the prediction of the dielectric constant of spinel microwave dielectric ceramics. Compared with traditional machine learning methods, deep learning does not need to rely on complex feature engineering, and can automatically learn the complex relationship between features and target values ​​in the data, thereby improving the accuracy and generalization ability of prediction. This provides a new idea for the research and development of spinel microwave dielectric ceramics, which helps to accelerate the process of material design and optimization.

[0017] Another major advantage of the present invention is its high prediction speed. Compared with traditional machine learning methods, it can quickly complete the prediction task of dielectric constant with higher computational efficiency. This not only speeds up the research and development process, but also promotes the innovation and development of spinel microwave dielectric ceramics in the application field.

[0018] The present invention is of great significance in the application of wireless communication technology. Spinel microwave dielectric ceramics play a key role in wireless communication equipment (such as antennas, filters, etc.). The present invention provides a more accurate model for predicting microwave dielectric constants, which helps to promote the design and performance development of wireless communication equipment and improve the efficiency and performance of wireless communication systems. Therefore, the present invention has broad application potential in the research and application of the field of spinel microwave dielectric ceramics, especially in promoting the further development of wireless communication technology.

[0019] The invention plays a key role in wireless communication fields such as communication technology, radar, sensor and microwave integrated circuit. The invention achieves high-accuracy prediction accuracy by simplifying input parameters, thus promoting the development of wireless communication technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Flow chart for predicting microwave dielectric constant of spinel ceramics provided in the embodiment of the present invention

[0021] Figure 2 The RMSE, MAE and R of the prediction model provided by the embodiment of the present invention on the test set 2 picture

[0022] Figure 3 Prediction effect diagram of microwave dielectric constant of spinel ceramics provided by the embodiment of the present invention DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0024] Example: Modeling results of the dielectric constant prediction model of spinel microwave dielectric ceramics established based on 385 samples. The modeling process is as follows: Figure 1 As shown, the following steps are included:

[0025] Step 1: Collect the chemical formula and dielectric constant of spinel microwave dielectric ceramics from previous literature as data set samples.

[0026] In this embodiment, the samples are 385 spinel microwave dielectric ceramics, and the division ratio of the training set, the validation set and the test set is 72 / 8 / 20.

[0027] Step 2: Take the chemical formula and dielectric constant of spinel microwave dielectric ceramics as a data set sample.

[0028] Step 3: For the chemical formulas collected in step 2, use Mat2vec encoding to convert them into basic element vector representation.

[0029] Step 4: Based on the basic element vector representation in step 3 as the element matrix, data embedding is performed by combining logarithmic encoding and composition encoding.

[0030] Step 5: Use the Adam optimization algorithm to adjust the hyperparameters in the model and configure a hyperparameter combination suitable for this task. The hyperparameters include learning rate, number of encoder layers, number of attention heads and input dimension, weight drop rate, and number of samples. The specific parameters are shown in Table 1.

[0031] surface :Model optimization parameters

[0032]

[0033] Step 6: For the data in step 4, use the Transformer encoder as the main body to extract features. Each encoding layer consists of a multi-head self-attention mechanism and a feedforward neural network, which can effectively capture the complex relationships and high-order features of the data.

[0034] Step 7: The features in step 6 are further processed using a convolutional layer, and the feature map is reduced in dimension using a pooling layer to retain the most important feature information.

[0035] Step 8: Use the residual network to train the data set in step 7. In this example, we use three evaluation indicators to measure the model performance, namely, mean absolute error (MAE), root mean square error (RMSE) and determination coefficient (R 2 ).

[0036] In step 8, three evaluation indicators MAE, RMSE and R 2 The calculation is as follows:

[0037]

[0038]

[0039]

[0040] Where N is the number of samples of microwave dielectric ceramics, is the true relative permittivity value obtained from experimental measurement, is the relative permittivity value predicted by the machine learning model, is the true mean relative permittivity of all samples.

[0041] Step 9: Use the test set to evaluate the performance of the prediction model trained in step 8. The results are as follows: Figure 2 As shown in the figure, the evaluation indicators of the model are: RMSE is 1.68, MAE is 0.938 and R 2 The predicted effect of microwave dielectric constant is as follows Figure 3 shown.

[0042] It can be seen from this embodiment that this method can accurately predict the dielectric constant of ceramic materials without synthesizing spinel microwave dielectric ceramics. This not only saves the cost of experimental trial and error, but also provides a more convenient and efficient solution for the research and development and application of microwave dielectric ceramics.

Claims

1. A method for predicting microwave dielectric constant of spinel ceramics based on component-restricted attention convolutional network, comprising the following steps: Step 1: Collect the chemical formula and dielectric constant measurement values ​​of spinel microwave dielectric ceramics from existing literature as data set samples; Step 2: Use Mat2vec to convert the chemical formulas collected in step 1 into basic element vector representations; these vectors can capture the inherent characteristics and relationships of the elements, providing rich initial information for subsequent data embedding and feature extraction; Step 3: Combine the basic element vector representation obtained in step 2 with composition coding and logarithmic coding to embed data; composition coding provides information about the number of elements, while logarithmic coding alleviates the difference in the number of elements at different orders of magnitude; these codes are concatenated with the basic element vector representation to form a comprehensive high-dimensional embedding vector; Step 4: Use the Adam optimization algorithm to optimize hyperparameters. It combines momentum and adaptive learning rate methods to find hyperparameters suitable for the model. The hyperparameters of the model are the input dimension and number of heads of the attention mechanism, the weight drop rate, the number of encoder layers, and the learning rate. Step 5: For the data obtained in step 3, use the Transformer encoder to extract features. Through the multi-head self-attention mechanism and feedforward neural network in the multi-layer encoder structure, the complex relationship and global dependency between the elements in the input sequence are captured to generate a set of high-level feature representations. Step 6: The features obtained in step 5 are further processed through the convolution layer, which can effectively capture local features and patterns. Then, the pooling layer reduces the dimension of the processed features, reduces the dimension of the data, and retains the most important feature information, thereby reducing the amount of calculation and preventing overfitting, providing optimized feature input for subsequent residual network processing. Step 7: Train the data obtained in step 6 through the residual network. The skip connection in the residual network can effectively alleviate the gradient vanishing problem in the deep network, allowing the network to learn complex features at a deeper level. Step 8: Use the trained dielectric constant prediction model to predict the dielectric constant of spinel microwave dielectric ceramics.

2. According to the method of claim 1, the deep learning model used in the method is CRANCNN (Compositionally restricted attention-based convolutional neural network, CRANCNN for short), and the workflow of the model can be divided into the following steps: [1] Data conversion: First, the chemical formula needs to be converted into a basic element vector representation using Mat2vec, and the data is embedded by combining composition encoding and logarithmic encoding; [2] Feature extraction: The Transformer encoder is the main body; the encoder consists of multiple stacked encoding layers; each encoding layer contains two core components: a multi-head self-attention mechanism and a feedforward neural network; the multi-head self-attention mechanism allows the encoder to consider all other elements in the input sequence when processing each element, thereby capturing the relationship between each element; The formula is as follows: （1） in, is the dot product of the query and the key, which is used to measure the similarity between each position, and the scaling factor This is to prevent the dot product result from being too large, and to prevent the softmax function from entering a saturated state, which would affect the gradient propagation. The dot product result is converted into a probability distribution through the softmax function, indicating the contribution of each position to the output of the current position. The final output is obtained through weighting, and the weight is determined by The softmax calculation result of is determined and then combined with the value vector V; [3] Convolutional residual network module: Using the convolutional residual network, the features are further processed through multiple residual blocks; each residual block consists of two parts: a deep separation convolution layer and a skip connection; the deep separation convolution layer is responsible for processing the different elements in each chemical formula separately, treating them as different channels and performing channel-by-channel convolution operations; this can effectively calculate the contribution of each element to the final prediction value; then, the pooling layer reduces the dimension of the feature map, reducing the dimension of the data while retaining the most important feature information; the skip connection allows part of the input to be directly passed to the output of the residual block, helping the network to analyze and extract features more deeply; the formula of the residual block is as follows: （2） Among them, x is the input feature map; is a nonlinear transformation function composed of operations such as convolutional layers and activation functions; +x is a skip connection, which adds the input directly to the output result to form a residual structure, thereby helping to alleviate the gradient disappearance problem in deep networks.

Images