InP HEMT small signal modeling method based on multilayer Transform neural network

Abstract

The invention discloses an InP HEMT small signal modeling method based on a multilayer Transform neural network, and the method comprises the following steps: obtaining an S parameter value of an InP HEMT device and an input data set comprising preprocessed small signal characteristic data based on a plurality of static bias voltages in a preset frequency range; based on a Transform model, constructing a neural network model; and training the neural network model, and optimizing model parameters of the neural network model. According to the invention, the deep learning model based on Transformer can accurately capture complex dynamic behaviors and nonlinear characteristics of the device in a high-frequency and wide-frequency range. The neural network model can process device characteristics under different working conditions, is suitable for various bias voltage, temperature and frequency ranges, and does not need to perform independent modeling on each working point.

Description

Technical Field

[0001] The present application belongs to the field of radio frequency semiconductor technology, and specifically relates to an InP HEMT small signal modeling method based on a multi-layer Transformer neural network. Background Art

[0002] In recent years, with the rapid development of high-speed communication and radio frequency microwave technology, high electron mobility transistor (HEMT) devices have been increasingly used in microwave and millimeter wave circuits. Among them, indium phosphide (InP) HEMT devices have become an ideal choice for the next generation of high-speed and high-frequency electronic devices due to their excellent electron mobility and frequency response characteristics. In order to give full play to the performance of InPHEMT devices, accurate device modeling is particularly important.

[0003] However, existing small-signal modeling methods usually use equivalent circuit models or S-parameter fitting methods based on empirical formulas, which are insufficient in complex frequency responses and device nonlinear characteristics, especially at high frequencies, where the accuracy is poor. Existing small-signal modeling methods rely on linear fitting of S parameters, but cannot fully capture the complex dynamic characteristics of InP HEMT devices in high-frequency and broadband applications.

[0004] Therefore, a more versatile and accurate modeling method is needed that can effectively capture the multi-frequency response and RF characteristics of the device. Summary of the invention

[0005] The present application provides an InP HEMT small signal modeling method based on a multi-layer Transformer neural network to solve the above-mentioned technical problems.

[0006] In order to solve the above technical problems, a technical solution adopted in the present application is: a method for small signal modeling of InP HEMT based on a multi-layer Transformer neural network, comprising the following steps:

[0007] Obtaining S parameter values ​​of the InP HEMT device and an input data set including preprocessed small signal characteristic data based on a plurality of static bias voltages within a preset frequency range;

[0008] Build a neural network model based on the Transformer model;

[0009] Train the neural network model and optimize the model parameters of the neural network model.

[0010] Further, based on a plurality of static bias voltages within a preset frequency range, a method for obtaining S parameter values ​​of an InP HEMT device and an input data set includes:

[0011] Obtaining S parameter values ​​based on InP HEMT devices with different bias voltages and frequencies;

[0012] Noise filtering and normalization are performed on the S parameter value parameters to obtain an input data set.

[0013] Furthermore, the S parameter values ​​include input reflection coefficient, gain or transmission coefficient, isolation coefficient, output reflection coefficient and DC bias voltage.

[0014] Furthermore, based on the Transformer model, a method for constructing a neural network model includes:

[0015] Based on formula (1), a neural network model is constructed; wherein formula (1) is:

[0016] S=f Transformer (I) (1);

[0017] Among them, S is an n×8 matrix and I is an n×3 matrix.

[0018] Furthermore, based on the Transformer model, a method for constructing a neural network model includes:

[0019] Based on formula (1), a neural network model is constructed; wherein formula (1) is:

[0020] S=f Transformer (I) (1);

[0021] Among them, S is an n×8 matrix and I is an n×3 matrix.

[0022] Further, the neural network model includes:

[0023] The first Linear layer is used to preprocess the input data set and convert it into a third-order matrix data of n×8×128;

[0024] An encoder layer, comprising eight encoders connected in series, wherein the encoder is used to encode the third-order matrix data, and the dimension of the encoder output data remains unchanged;

[0025] The second Linear layer is used to convert the encoder output data into a second-order matrix of n×8 and pass it to the output layer.

[0026] Furthermore, the encoder consists of two sub-layers, namely the multi-head attention layer and the feed-forward neural network layer.

[0027] Furthermore, residual connections and layer normalization are added to each sub-layer.

[0028] Furthermore, the method for training the neural network model and optimizing the model parameters of the neural network model includes:

[0029] Based on the preprocessed small signal characteristic data and the supervised learning algorithm, the neural network model is trained and the model parameters are optimized to accurately predict the actual small signal response of the InP HEMT device;

[0030] The output values ​​of the trained neural network model are compared with the real test data to verify the accuracy of the model, and the model parameters are further optimized through the back-propagation algorithm.

[0031] The beneficial effect of the present application is that compared with the traditional equivalent circuit model, the Transformer-based deep learning model of the present application can accurately capture the complex dynamic behavior and nonlinear characteristics of the device in a high frequency and wide frequency range. The neural network model of the present application can handle the device characteristics under different working conditions, and is suitable for a variety of bias voltages, temperatures and frequency ranges, without the need to model each operating point separately. Through model training and prediction, the present application greatly reduces the time for manual debugging of the equivalent circuit model, and can automatically adapt to a variety of application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a flow chart of an embodiment of InP HEMT small signal modeling based on a multi-layer Transformer neural network of the present application;

[0033] Figure 2 yes Figure 1 A schematic flow chart of an embodiment of step S1 in FIG.

[0034] Figure 3 It is a structural schematic diagram of a neural network model of an embodiment of InP HEMT small signal modeling based on a multi-layer Transformer neural network of the present application;

[0035] Figure 4 yes Figure 3 Schematic diagram of the structure of the Transformer encoder layer in ;

[0036] Figure 5 yes Figure 1 A schematic flow chart of an embodiment of step S3 in FIG.

[0037] Figure 6 In one embodiment of the InP HEMT small signal modeling based on the multi-layer Transformer neural network of the present application, the bias is at V gs =0V,V ds =Comparison of the model simulation results and the test S parameter data of the InP HEMT device under the condition of 1.0V;

[0038] Figure 7 In one embodiment of the InP HEMT small signal modeling based on the multi-layer Transformer neural network of the present application, the bias is at V gs =0V,V ds =Comparison of the model simulation results and the test S parameter data of the InP HEMT device under the condition of 1.2V;

[0039] Figure 8 In one embodiment of the InP HEMT small signal modeling based on the multi-layer Transformer neural network of the present application, the bias is at V gs =-0.05V,V ds =Comparison of the model simulation results and the tested S parameter data of the InP HEMT device under the condition of 1.2V. DETAILED DESCRIPTION

[0040] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below in conjunction with specific embodiments.

[0041] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways than those described herein. Therefore, the present invention is not limited to the specific embodiments of the following disclosure.

[0042] See also Figure 1 , Figure 1 1 is a flow chart of an embodiment of the InP HEMT small signal modeling based on a multi-layer Transformer neural network of the present application. The method comprises:

[0043] Step S1. Based on a plurality of static bias voltages within a preset frequency range, an S parameter value of an InP HEMT device and an input data set including preprocessed small signal characteristic data are obtained.

[0044] For details, see Figure 2 , Figure 2 yes Figure 1 Schematic diagram of the process of step S1 in an embodiment. Step S1 includes:

[0045] Step S11. Obtaining S parameter values ​​based on InP HEMT devices with different bias voltages and frequencies.

[0046] Specifically, collect and measure data, and measure the S parameter values ​​of InP HEMT devices at different bias voltages and frequencies from the laboratory. The S parameters include S 11 (input reflection coefficient), S 21 (gain or transmission coefficient), S12 (isolation coefficient), S 22 (output reflection coefficient), DC bias voltage includes V ds (drain-source voltage) and V gs (gate-source voltage).

[0047] Step S12: Perform noise filtering and normalization processing on the S parameter value parameters to obtain an input data set.

[0048] Specifically, the data is preprocessed, and the collected S parameters are subjected to noise filtering, normalization and other processing to eliminate possible measurement errors to meet the input requirements of the Transformer model. Among them, noise filtering: The Fourier transform method is used to filter the S parameter data for noise to eliminate random noise in the measurement. Normalization processing: To ensure that the data has a consistent magnitude when the model is input, the measured S parameter data is normalized so that its value is distributed in the [0,1] interval. Interpolation processing: For measurement points with large frequency intervals, the missing data points are filled by linear interpolation method to ensure the continuity and integrity of the data.

[0049] Step S2. Build a neural network model based on the Transformer model.

[0050] Specifically, based on the Transformer model, a neural network model for small signal modeling is constructed. The neural network model includes several layers of self-attention layers and feedforward network layers, which can effectively capture the small signal response of InP HEMT devices under different frequencies and bias voltages.

[0051] See also Figure 3 , the output layer of the neural network model contains 8 sample features, namely S 11 , S 21 , S 12 , S 22 The amplitude Ma and phase φ.

[0052] The input layer includes three sample features, namely frequency freq, gate-source voltage V gs and the drain-source voltage V ds ;

[0053] Based on formula (1), a neural network model is constructed; wherein formula (1) is:

[0054] S=f Transformer (I) (1);

[0055] Among them, S is an n×8 matrix, expressed as:

[0056]

[0057] I is an n×3 matrix, expressed as:

[0058]

[0059] Among them, the hidden layer of the neural network model consists of a Transformer encoder layer and a linear layer.

[0060] First, the data is preprocessed through the first Linear layer and the input data is converted into a third-order matrix of n×8×128.

[0061] Then it passes through 8 identical layers of Transformer encoders connected in series. In this process, the output dimension remains unchanged. Among them, the 8 layers of Transformer encoders in the neural network model are connected in series.

[0062] Finally, through the second Linear layer, the data is converted into a second-order matrix of n×8 and passed to the output layer.

[0063] See also Figure 4 , each Transformer encoder layer contains two sub-layers, namely the multi-head attention layer (Multi-head Attention) and the feedforward neural network layer (FeedForwardNeural Network).

[0064] Residual connections and layer normalization are added after each sublayer. The main purpose of residual connections is to alleviate the gradient vanishing problem in deep neural networks and promote the flow of information. By introducing residual connections, the input can bypass one or more layers and be directly added to the output, making the network easier to train.

[0065] The main content of the self-attention mechanism is to calculate the attention weight Attention(Q, K, V) by calculating the query matrix, key matrix and value matrix of the input matrix X. The formula is as follows:

[0066] Q=XW Q ;

[0067] K=XW K ;

[0068] V=XW V ;

[0069] Among them, W Q , W K , W V is the weight matrix;

[0070] Based on formula (2), the attention weight is obtained; formula (2) is:

[0071]

[0072] in, is the scaling factor. The self-attention layer of the model uses a multi-head attention mechanism. The input is computed in parallel through h independent heads, each with its own query, key, and value matrices.

[0073] The output for each header is as follows:

[0074]

[0075] Then, concatenate the outputs of all heads and pass them through a linear transformation:

[0076] MultiHead(Q,K,V)=Concat(head 1 …, head h )W O ;

[0077] Among them, W O is the output weight matrix.

[0078] The feedforward network layer formula of the neural network model is as follows:

[0079] FFN(x)=max(0,xW 1 +b 1 )W 2 +b 2 ;

[0080] Where x is the input variable, W 1 , W 2 is the weight matrix of linear transformation, b 1 , b 2 is the bias vector.

[0081] The formula for the residual connection of the neural network model is:

[0082] y=F(x)+x;

[0083] Among them, the input is x, and the output after passing through a certain sublayer (such as a multi-head self-attention layer or a feedforward neural network) is F(x).

[0084] The main purpose of layer normalization of the above neural network model is to normalize the features of each sample to speed up the training process and improve the stability of the model. Assuming the input is h, the layer normalization operation is as follows:

[0085] (1) Calculate the mean and variance:

[0086]

[0087] (2) Standardization:

[0088]

[0089] (3) Linear transformation:

[0090]

[0091] Here, γ and β are trainable parameters, and ε is a small constant used to prevent division by zero.

[0092] Step S3: Train the neural network model and optimize the model parameters of the neural network model.

[0093] For details, see Figure 5 , Figure 5 yes Figure 1 Schematic diagram of the process of step S3 of an embodiment. Step S3 includes:

[0094] Step S31. Based on the preprocessed small signal characteristic data and the supervised learning algorithm, the neural network model is trained and the model parameters are optimized to accurately predict the actual small signal response of the InP HEMT device.

[0095] Specifically, a supervised learning algorithm is used during training, with the normalized S parameter data as the input data set and the actually measured S parameters as labels.

[0096] When training the neural network model, the mean square error loss (MSE) is selected as the loss function of the model, and the model parameters are optimized by minimizing the error between the predicted S parameters and the actual measured data. The formula of the mean square error loss is as follows:

[0097]

[0098] in, is the predicted value of the model, y i is the true value of the model, and n is the number of samples.

[0099] In this embodiment, the Adam optimizer is used in the training process, and the cross-validation method is used to evaluate the generalization performance of the model. The input data set is divided into a training set (80%) and a validation set (20%), and the model is prevented from overfitting by alternating training and validation.

[0100] Step S32: Compare the output value of the trained neural network model with the real test data to verify the accuracy of the model, and further optimize the model parameters through the back propagation algorithm.

[0101] Specifically, the trained model is compared with the real S-parameter test data under different working conditions to verify the prediction accuracy of the model.

[0102] refer to Figure 6-8 , Figure 6 , Figure 7 and Figure 8 , respectively give the model simulation results and test results of InP HEMT devices with different biases in the 0.5-40GHz frequency band 11 , S 21 , S 12 and S 22 Comparison chart of parameter data.

[0103] The performance of the model is evaluated by comparing the difference between the model prediction value and the real data. In some embodiments, the prediction accuracy of the above-trained model is not high in certain frequency ranges. The number of layers and the number of attention heads of the neural network model can be adjusted to optimize the structure of the model. The prediction accuracy of the model can be gradually improved by adjusting the model structure through multiple iterations.

[0104] The neural network model after training and verification can be used in the small signal simulation and RF circuit design of InP HEMT devices. The model can quickly predict the small signal characteristics of the device under different frequencies and bias voltage conditions, providing an effective reference for RF circuit designers. Through this model, designers can complete the design and optimization of complex RF systems in a short time.

[0105] The above descriptions are merely embodiments of the present application and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for small signal modeling of InP HEMT based on a multi-layer Transformer neural network, characterized in that: The following steps are involved: Obtaining S parameter values ​​of the InP HEMT device and an input data set including preprocessed small signal characteristic data based on a plurality of static bias voltages within a preset frequency range; Build a neural network model based on the Transformer model; The neural network model is trained to optimize the model parameters of the neural network model.

2. The method according to claim 1, characterized in that: The method for obtaining S parameter values ​​of an InP HEMT device and inputting a data set based on multiple static bias voltages within a preset frequency range includes: Obtaining the S parameter value based on the InP HEMT device with different bias voltages and frequencies set; Noise filtering and normalization processing are performed on the S parameter value parameters to obtain the input data set.

3. The method according to claim 2, characterized in that The S parameter values ​​include input reflection coefficient, gain or transmission coefficient, isolation coefficient, output reflection coefficient and DC bias voltage.

4. The method according to claim 1, characterized in that: The method for constructing a neural network model based on the Transformer model includes: Based on formula (1), the neural network model is constructed; wherein the formula (1) is: S=f Transformer (I) (1); Among them, S is an n×8 matrix and I is an n×3 matrix.

5. The method according to claim 5, characterized in that: The neural network model includes: The first Linear layer is used to preprocess the input data set and convert it into a third-order matrix data of n×8×128; An encoder layer, comprising eight encoders connected in series, wherein the encoders are used to encode third-order matrix data, and the dimensions of the encoder output data remain unchanged; The second Linear layer is used to convert the encoder output data into an n×8 second-order matrix and pass it to the output layer.

6. The method according to claim 5, characterized in that The encoder includes two sub-layers, namely a multi-head attention layer and a feed-forward neural network layer.

7. The method according to claim 6, characterized in that Residual connections and layer normalization are added to each of the sub-layers.

8. The method according to claim 1, characterized in that The method for training the neural network model and optimizing the model parameters of the neural network model comprises: based on the preprocessed small signal characteristic data and the supervised learning algorithm, training the neural network model and optimizing the model parameters to accurately predict the actual small signal response of the InP HEMT device; The output value of the trained neural network model is compared with the actual test data to verify the accuracy of the model, and the model parameters are further optimized through the back propagation algorithm.

Images