A microwave device modeling method based on multi-layer neural network model structure adaptation

Abstract

The present invention relates to a microwave device modeling method based on a multi-layer neural network model structure adaptation, which belongs to the field of microwave device modeling and is used to solve the problem of determining the optimal model structure in microwave device modeling based on artificial neural networks. The present invention proposes a microwave device modeling method based on a multi-layer neural network model structure adaptation, which adopts a new multi-layer neural network model structure and combines an error function based on l1 regularization to train the neural network model, thereby achieving simultaneous adjustment of the number of hidden layers of the model and the number of neurons in each hidden layer, and finally obtaining the optimal model structure. The method of the present invention can adaptively obtain different optimal model structures for different microwave modeling problems, which can save time in model development in actual use and has the advantages of flexibility and efficiency.

Description

Technical Field

[0001] The present invention relates to the field of microwave device modeling, and in particular to the application of artificial neural network technology in microwave device modeling. Background Art

[0002] With the rapid development of wireless communication technology, new microwave devices are constantly emerging. Users have higher requirements for the accuracy of microwave device models and the efficiency of modeling. Artificial neural network technology, as a flexible modeling method, has been widely used in the field of microwave device modeling. [1]-[5] The trained neural network model can accurately characterize the nonlinear relationship between the input-output response of the device within the modeling range, and the calculation speed from input to output is extremely fast. Using the microwave device model based on artificial neural network to design microwave circuits or systems can greatly shorten the design time.

[0003] For modeling problems of different microwave devices, the required neural network model structures are often completely different, which requires the neural network modeling algorithm to be able to adaptively adjust the model structure for different modeling problems. [6][7] In the past, two methods were mostly used to adjust the model structure: one is to adjust the number of neurons in each hidden layer under the premise of fixing the number of hidden layers of the neural network; the other is to adjust the number of hidden layers under the premise of fixing the number of neurons in each hidden layer. When faced with the problem of modeling microwave devices with unknown difficulty, the existing model structure adjustment methods are not flexible enough, and it usually takes many attempts to obtain the optimal model structure while meeting the model accuracy. This is often a time-consuming process. To address this problem, it is necessary to develop a new multi-layer neural network model structure adaptive method for microwave devices to further improve modeling efficiency.

[0004] The present invention combines the beneficial properties of l1 regularization [8][9] , a novel multi-layer neural network model structure adaptive method for microwave devices is proposed. By proposing a novel multi-layer neural network structure and a novel two-stage training scheme combined with l1 regularization, the present invention can adaptively adjust the number of hidden layers and the number of neurons in each hidden layer of the multi-layer neural network model for different microwave modeling problems, thereby obtaining the optimal model structure, effectively shortening the model development time and improving the modeling efficiency.

[0005] References:

[0006] [1]F.Feng,W.Na,J.Jin,W.Zhang,and Q.-J.Zhang.ANNs for fastparameterized EM modeling:The state of the art in machine learning for designautomation of passive microwave structures[J]IEEE Microwave Magazine,vol.22,no.10,pp.37-50,Oct.2021.

[0007] [2]Zhang Q J,Gupta K C.Neural Networks for RF and Microwave Design / Q.J.Zhang,K.C.Gupta.

[0008] [3]H.M.Torun,A.C.Durgun,K.Aygün,and M.Swaminathan.Causal and passiveparameterization of S-parameters using neural networks[J].IEEE Transactionson Microwave Theory and Techniques,vol.68,no.10,pp.4290-4304,Oct.2020.

[0009] [4]S.Li,Y.Wang,M.Yu,and A.Panariello.Efficient modeling of kubandhigh power dielectric resonator filter with applications of neural networks[J].IEEE Transactions on Microwave Theory and Techniques,vol.67,no.8,pp.3427-3435,Aug.2019.

[0010] [5]Z.Zhang,Q.S.Cheng,H.Chen,and F.Jiang.An efficient hybrid samplingmethod for neural network-based microwave component modeling and optimization[J].IEEE Microwave and Wireless Components Letters,vol.30,no.7,pp.625-628,July.2020.

[0011] [6]W.Na and Q.Zhang.Automated parametric modeling of microwavecomponents using combined neural network and interpolation techniques[J].IEEEMTT-S International Microwave Symposium Digest,Jun.2013,pp.1-3.

[0012] [7]Q.-J.Zhang,E.Gad,B.Nouri,W.Na,and M.Nakhla.Simulation andautomated modeling of microwave circuits:State-of-the-art and emerging trends[J].IEEE Journal of Microwaves,vol.1,no.1,pp.494-507,Jan.2021.

[0013] [8]J.Hald and K.Madsen.Combined LP and quasi-newton methods fornonlinearl1optimization[J].Siam Journal on Numerical Analysis,vol.22,no.1,pp.68-80,1985.

[0014] [9]W. Na, F. Feng, C. Zhang, and Q.-J. Zhang. A unified automated parametric modeling algorithm using knowledge-based neural network and l1 optimization[J]. IEEE Transactions on Microwave Theory and Techniques, vol. 65, no. 3, pp. 729-745, Mar. 2017.

[0015]

[10] J. Ros, P. Pacheco, H. Gonzalez, V. Esbert, C. Martin, M. Calduch, S. Borras, and B. Martinez. Fast automated design of waveguide filters using aggressive space mapping with a new segmentation strategy and a hybrid optimization algorithm[J]. IEEE Transactions on Microwave Theory and Techniques, vol. 53, no. 4, pp. 1130-1142, Apr. 2005.

[0016]

[11] J. Jin, F. Feng, W. Na, J. Zhang, W. Zhang, Z. Zhao, and Q.-J. Zhang. Advanced cognition-driven EM optimization incorporating transfer function-based feature surrogate for microwave filters[J]. IEEE Transactions on Microwave Theory and Techniques, vol. 69, no. 1, pp. 15-28, Jan. 2021 Summary of the Invention

[0017] The present invention proposes a microwave device modeling method based on a multi-layer neural network model structure that is adaptive. The method can automatically determine the optimal number of hidden layers of the neural network and the optimal number of neurons in each hidden layer during the modeling process, and is applicable to various microwave device modeling problems. The present invention specifically proposes a novel multi-layer neural network model structure, and a novel two-stage training scheme suitable for training the neural network model structure. The present invention can effectively solve the following problems existing in the existing neural network modeling methods in the field of microwave device modeling: (1) The existing microwave device modeling methods based on artificial neural networks usually cannot adjust the number of hidden layers of the model and the number of neurons in each hidden layer at the same time, that is, the number of hidden layers can only be adjusted when the number of neurons in each layer remains unchanged, or the number of neurons in each hidden layer can only be adjusted when the number of hidden layers remains unchanged; (2) The existing neural network modeling methods may not be able to obtain a model that meets the target accuracy and has the optimal structure at one time during the actual modeling process, so repeated attempts are required, resulting in a relatively long modeling time. The neural network model structure adaptive method proposed in the present invention has the advantages of flexibility and high efficiency, and realizes the simultaneous adjustment of the number of hidden layers of the model and the number of neurons in each hidden layer, which can save time for model development in actual use, thereby improving the efficiency of microwave device modeling.

[0018] The present invention proposes a microwave device modeling method based on multi-layer neural network model structure adaptation, which mainly includes the following steps:

[0019] Step 1: Obtain multiple sets of microwave device design parameters and their corresponding electromagnetic responses. The design parameters include size parameters, electrical parameters, and frequency parameters, where the design parameters are inputs and the electromagnetic responses are outputs. Select some groups as test data for establishing a neural network model, and the remaining groups as training data. The user sets the initial number of hidden layers L of the neural network and the number of neurons H1 in the first hidden layer as needed, and calculates the number of neurons H in each hidden layer. i , i=2,3,…,L.

[0020] Step 2: Create a new multi-layer neural network structure, including an input layer, L hidden layers, and an output layer. The input layer has M neurons and each hidden layer has H neurons. i , the output layer has N neurons; the first stage of training is performed using the training error function based on l1 regularization, and the training error E of the model is obtained Tr and the test error E Te .

[0021] Step 3: Determine E Tr With βE d The size of E dis the model target accuracy set by the user, β is a user-defined constant greater than 1; if E Tr ≤βE d , then go directly to step 4; if E Tr >βE d , then increase the number of neurons in each hidden layer, that is, increase H1 and update H through formula (1) i , then skip to step 2.

[0022] Step 4: Determine E Te With βE d The size of E Te ≤βE d , then go directly to step 5; if E Te >βE d , then reduce the number of neurons in each hidden layer, that is, reduce H1 and update H through formula (1) i , then skip to step 2.

[0023] Step 5: Calculate J (L) , J (L) is the bias function of the Lth hidden layer, indicating the impact of deleting the Lth hidden layer on the model accuracy, and judging J (L) and γ, γ is a user-defined constant greater than 0. If J (L) >γ, then jump to step 7; if J (L) ≤γ, it is considered that the Lth hidden layer does not affect the model accuracy, and the Lth hidden layer and the related connection weights u are deleted. (L) and v (L) , then go to step 6.

[0024] Step 6: Let L=L-1. If L>0, jump to step 2; if L≤0, go directly to step 7.

[0025] Step 7: Perform the second phase of training on the current new multi-layer neural network structure to further improve the accuracy of the model, and finally quickly establish a multi-layer neural network model that is most suitable for the microwave device. At the same time, the accuracy of the model reaches the accuracy required by the user. Use this model to replace electromagnetic simulation to quickly obtain the electromagnetic response of the microwave device under different design parameters.

[0026] The present invention is different from the prior art in the following aspects:

[0027] 1. In step 2, the present invention proposes a new type of multi-layer neural network model structure, which is first proposed by the present invention and applied to the field of microwave device modeling. The two adjacent layers of the traditional multi-layer perceptron (MLP) structure are fully connected, which does not support the use of l1 regularization in the present invention to achieve adaptive adjustment of the neural network model structure. The present invention changes the connection mode of neurons on the basis of traditional MLP. The neurons in adjacent hidden layers are partially connected, and the input layer and all hidden layers are directly fully connected to the output layer. Figure 1 The novel multi-layer neural network structure proposed in this invention is shown, where x is the input of the neural network, y is the output of the neural network, and u (i) is the total connection weight between the i-th hidden layer and the i-1-th hidden layer, u (1) is the total connection weight between the first hidden layer and the input layer, v (i) The total connection weights between the i-th hidden layer and the output layer, v (0) It is the total connection weight between the input layer and the output layer. The user sets the initial number of hidden layers of the neural network L and the number of neurons in the first hidden layer H1, the number of neurons in other hidden layers H i (i=2,3,…,L) is

[0028]

[0029] The number of neurons in the output layer is N, the number of neurons in the input layer is M, when i=2, H′=M, and when i>2, H′=H i-1 Thus, the nth output y of the novel multi-layer neural network model structure proposed by the present invention is n The relationship between the model input and

[0030]

[0031] in is the connection weight between the pth neuron in the i-th hidden layer and the nth neuron in the output layer, is the connection weight between the pth neuron in the input layer and the nth neuron in the output layer, x p is the output of the pth neuron in the input layer, is the output of the pth neuron in the i-th hidden layer and is calculated as

[0032]

[0033] in is the connection weight between the pth neuron in the i-th hidden layer and the qth neuron in the i-1th hidden layer, is the connection weight between the p-th neuron in the input layer and the q-th neuron in the input layer, and σ is the Sigmoid function.

[0034] 2. In step 2 of the present invention, a new training error function based on l1 regularization is proposed, namely

[0035]

[0036] Where ω=[(u (1) ) T ,…,(u (L) ) T ,(v (0) ) T ,(v (1) ) T ,…,(v (L) ) T ] T is the total connection weight of the entire neural network model, N is the number of neurons in the output layer, N d is the total number of training data, x (j) and d (j) is the input and output part of the jth group of training data, λ is a non-negative constant, when λ = 0, formula (4) is the traditional error function, y(x (j) ,ω) is the neural network model input x (j) Output when .

[0037] This error function is first proposed by the present invention by combining the principle of artificial neural network with the l1 regularization method. As part of the training scheme newly proposed by the present invention, this error function is used in the first stage of training. The traditional error function is only used to evaluate the error between the model output result and the training data, and cannot be used to adaptively adjust the neural network model structure. The error function proposed by the present invention is based on the traditional error function and adds the connection weight v between the Lth hidden layer and the output layer of the current neural network. (L) Based on the beneficial properties of l1 regularization, the model training process using this error function can make v (L) As many weights as possible are zero. After training, non-zero weights are considered valid weights, and zero weights are considered invalid weights. The present invention determines whether the Lth hidden layer is helpful for modeling (i.e., whether it affects the model accuracy) by analyzing whether there are valid weights in the Lth layer, thereby providing clues for adjusting the number of hidden layers, and finally realizing the self-adaptation of the multi-layer neural network model structure.

[0038] 3. In step 5 of the present invention, a deviation function J is proposed for determining whether the Lth hidden layer is retained. (L) The calculation method is

[0039]

[0040] in, represents the test error under normal conditions, It represents the test error when all the connection weights between the Lth hidden layer and the output layer are cleared. This deviation function is part of the training scheme proposed by the present invention, and is closely combined with the error function based on l1 regularization in step 2 to determine whether the Lth hidden layer plays a key role in modeling after the first stage of training and whether it needs to be retained.

[0041] In summary, the present invention proposes a microwave device modeling method based on a multi-layer neural network model structure that is adaptive. The invention proposes a new multi-layer neural network model structure, which provides a structural basis for adjusting the number of hidden layers, and uses a new error function based on l1 regularization to train the neural network model and obtain the optimal model structure (i.e., the optimal number of hidden layers and the optimal number of hidden layer neurons), making the modeling process more flexible and efficient, and shortening the time of model development. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a structural schematic diagram of a novel multi-layer neural network model proposed in the present invention;

[0043] Figure 2 It is a flow chart of a microwave device modeling method based on multi-layer neural network model structure adaptation proposed by the present invention;

[0044] Figure 3 It is a device structure of an embodiment of the present invention (microwave cavity filter);

[0045] Figure 4 It is the verification data and model output characteristic curve of the embodiment of the present invention (microwave cavity filter);

[0046] Figure 5 It is a comparison between the modeling results of the embodiment of the present invention (microwave cavity filter) and the existing modeling method. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present invention more clear, an embodiment of the present invention (microwave cavity filter) will be described in detail below with reference to the accompanying drawings.

[0048] This embodiment adopts Figure 2 The neural network modeling method proposed by the present invention is shown as follows Figure 3 A microwave cavity filter shown

[10]

[11] Among all the size parameters of this filter, such as cavity length and width, resonant column height, etc., we select S1, S2, S3, Sc1 ,S c2 As the input variable of the model, that is, x = [S1, S2, S3, S c1 ,S c2 ] T , where S1, S2, S3 are the heights of the middle column of the coupling window resonance, S c1 ,S c2 are the heights of the resonant columns in the resonant cavity, respectively. The modeling range is S1 = 3.16 ~ 3.87 mm, S2 = 3.80 ~ 4.64 mm, S3 = 3.35 ~ 4.09 mm, S c1 =2.93~3.58mm, S c2 =2.67~3.26mm. In addition to the above-mentioned size parameters, the design parameters can also be electrical parameters and frequency parameters, where the electrical parameters include resistance, inductance, and capacitance, and the frequency parameters include frequency range and frequency points. The output variable of the model is the output reflection coefficient S in the electromagnetic response. 21 The amplitude of y = |S 21 |. The accuracy requirement for this modeling problem is E d ≤1%. The number of hidden layers L=3, the number of neurons in the first hidden layer H1=30, β=4, γ=1%. 600 sets of training data and 94 sets of test data were obtained. The initial H2=11, H3=6 were calculated using formula (1). The first stage of neural network training was performed using the l1 regularized error function proposed in the present invention. (3) =0.1%, J (2) = 24.4%, and finally determined that the optimal model structure contains two hidden layers, with the first hidden layer containing 73 neurons and the second hidden layer containing 20 neurons. Then after the second stage of training, a neural network model with a test error of 0.86% was quickly obtained, which met the target accuracy required by the user. Figure 4 To use the output characteristics of the model established by the present invention (|S 21 |) curve and the verification data comparison chart, it can be seen that the output curve of the neural network model has a good fit with the verification data. For comparison, we also use the existing modeling method to model the filter. The modeling results are compared with Figure 5 Compared with the existing modeling methods, the microwave device modeling method based on the adaptive multi-layer neural network model structure proposed in the present invention can not only adjust the number of hidden layers and the number of neurons in each hidden layer of the model at the same time to establish an accurate neural network model, but also saves about 45% of the time in the actual model development process compared with the existing modeling methods, and the model development efficiency is further improved.

Claims

1. A microwave device modeling method based on multi-layer neural network model structure adaptation, characterized by: The following steps are involved: Step 1: Obtain multiple sets of microwave device design parameters and their corresponding electromagnetic responses, where the design parameters are inputs and the electromagnetic responses are outputs. Select some sets as test data for establishing a neural network model, and the remaining sets as training data. The user sets the initial number of hidden layers L of the neural network and the number of neurons H1 in the first hidden layer as needed, and calculates the number of neurons H in each hidden layer. i , i = 2, 3, ..., L; Step 2: Create a new multi-layer neural network structure, including an input layer, L hidden layers, and an output layer. The input layer has M neurons and each hidden layer has H neurons. i , the output layer has N neurons; the first stage of training is performed using the training error function based on l1 regularization, and the training error E of the model is obtained Tr and the test error E Te ; Step 3: Determine E Tr βE d The size of E d is the model target accuracy set by the user, β is a user-defined constant greater than 1; if E Tr ≤βE d , then go directly to step 4; if E Tr >βE d , then increase the number of neurons in each hidden layer, that is, increase H1 and update H through formula (1) i , then jump to step 2; Step 4: Determine E Te βE d The size of E Te ≤βE d , then go directly to step 5; if E Te >βE d , then reduce the number of neurons in each hidden layer, that is, reduce H1 and update H through formula (1) i , then jump to step 2; Step 5: Calculate J (L) , J (L) is the bias function of the Lth hidden layer, indicating the impact of deleting the Lth hidden layer on the model accuracy, and judging J (L) and γ, γ is a user-defined constant greater than 0. If J (L) >γ, then jump to step 7; if J (L) ≤γ, it is considered that the Lth hidden layer does not affect the model accuracy, and the Lth hidden layer and the related connection weights u are deleted. (L) and v (L) , then go to step 6; Step 6: Let L = L-1. If L>0, jump to step 2; if L≤0, go directly to step 7; Step 7: Conduct the second stage training of the current new multi-layer neural network structure to further improve the accuracy of the model, and finally quickly establish a multi-layer neural network model that best matches the microwave device. At the same time, the accuracy of the model reaches the accuracy required by the user. Use this model to replace electromagnetic simulation and quickly obtain the electromagnetic response of the microwave device under different design parameters; The number of neurons in the i-th hidden layer in step 1 is H i , i = 2, 3, ..., L, the calculation method is Where N is the number of neurons in the output layer, when i = 2, H′ is equal to the number of neurons in the input layer, and when i>2, H′=H i-1 ; The output y of the nth neuron in the output layer described in step 2 n , calculated as in is the connection weight between the pth neuron in the i-th hidden layer and the nth neuron in the output layer, is the connection weight between the pth neuron in the input layer and the nth neuron in the output layer, x p is the output of the pth neuron in the input layer, is the output of the pth neuron in the i-th hidden layer and is calculated as in is the connection weight between the pth neuron in the i-th hidden layer and the qth neuron in the i-1th hidden layer, is the connection weight between the p-th neuron in the input layer and the q-th neuron in the input layer, and σ is the Sigmoid function; The training error function based on l1 regularization described in step 2 is calculated as follows: Where ω=[(u (1) ) T ,…u (i) …,(u (L) ) T ,(v (0) ) T ,(v (1) ) T ,…v (i) …,(v (L) ) T ] T are all the connection weights of the new multi-layer neural network structure, u (i) is the total connection weight between the i-th hidden layer and the i-1-th hidden layer, u (1) is the total connection weight between the first hidden layer and the input layer, v (i) is the total connection weight between the i-th hidden layer and the output layer, v (0) is the total connection weight between the input layer and the output layer, N is the number of neurons in the output layer, and N d is the total number of training data, x (j) and d (j) is the input and output part of the jth group of training data, λ is a non-negative constant, when λ=0, formula (4) degenerates into a conventional error function, y(x (j) , ω) is the input design parameter x of the new multi-layer neural network structure (j) The electromagnetic response of the output; The bias function J of the Lth hidden layer described in step 5 (L) , calculated as in, represents the test error without changing the connection weights between the Lth hidden layer and the output layer, It represents the test error when all the connection weights between the Lth hidden layer and the output layer are reset to zero.

2. The microwave device modeling method based on multi-layer neural network model structure adaptation according to claim 1 is characterized in that: The design parameters include size parameters, electrical parameters and frequency parameters.

Images