A joint simulation optimization method and system for microwave sensor resonant structure
Through the combined simulation optimization method of microwave sensor resonant structure combined with current neural network and full-wave electromagnetic simulation software, the problem of cumbersome resonant structure optimization process in the existing technology is solved, and the excellent resonant structure is quickly and reliably obtained, which promotes microwave sensors industrialization.
Patent Information
- Application Number
- CN202110878769.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-08-02
- Publication Date
- 2024-12-17
- Estimated Expiration
- 2041-08-02
AI Technical Summary
The prior art has heavy, cumbersome and time-consuming tasks in the optimization of the resonant structure of microwave sensors, and it is difficult to quickly and reliably obtain a resonant structure layout with excellent performance, which has affected the industrialization process of microwave sensors.
A microwave sensor resonant structure joint simulation optimization method is adopted, and the current neural network and target neural network are combined with full-wave electromagnetic simulation software, and the resonant structure is automatically adjusted to simplify the design process through pixelated structure optimization, random noise guidance and strategic gradient methods. Rapidly optimize the resonant structure.
It realizes the rapid and reliable acquisition of the resonant structure of microwave sensors with excellent performance, simplifies the manual design process, promotes the industrialization of microwave sensors, and improves sensitivity and efficiency.
Smart Images
Figure CN113627113B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of microwave sensors, and in particular to a joint simulation optimization method and system for a microwave sensor resonant structure. Background Art
[0002] Microwave sensors are attractive and developing rapidly in various industrial sensing applications, such as material identification, displacement detection, and biomedical sensing, because they can provide high measurement accuracy, fast detection speed, low cost, and non-invasiveness. In particular, resonant microwave microfluidic sensors have extremely high sensitivity and the interaction between liquid samples and electric fields in microchannels.
[0003] In recent years, complementary split ring resonators excited by microstrip lines have been widely used in sensor applications due to their excellent performance. The resonator has many shapes, such as square, circular, etc., and these complementary split ring resonant structures of different shapes show different sensitivities when used for the characterization of liquid material properties. In other words, the geometric structure of the resonator plays an important role in the propagation of microwave signals. The affected signal quality leads to different sensor sensitivities, so the optimal design of the resonator structure is particularly important.
[0004] Therefore, it is necessary to provide a joint simulation optimization method and system for the resonant structure of a microwave sensor. For the nonlinear high-dimensional optimization problem, the system can use historical samples to fully learn the useful features of high-dimensional data, and then obtain an optimization strategy for the initial resonant structure to solve the problem that the existing technology is heavy, cumbersome and time-consuming for the optimization process of the resonant structure of microwave sensors. On the other hand, it also plays a very positive role in promoting the industrialization process of microwave sensors. Summary of the invention
[0005] The purpose of the present invention is to address the defects of the prior art and provide a microwave sensor resonant structure joint simulation optimization method and system, which can automatically and quickly obtain a microwave sensor resonant structure layout with excellent performance.
[0006] In order to achieve the above purpose, the present invention adopts the following technical solutions:
[0007] A microwave sensor resonant structure joint simulation optimization method, comprising:
[0008] S1. The optimized area of the original resonant structure of the pixelated microwave microfluidic sensor is a square pixel unit;
[0009] S2. Initialize the state set of the resonant structure and the action set for adjusting the resonant structure;
[0010] S3. Initialize the parameters of the current neural network and the parameters of the target neural network; wherein the current neural network includes the current policy network and the current evaluation network, and the target neural network includes the target policy network and the target evaluation network;
[0011] S4. The current strategy network outputs an action set according to the current state set, and adds random noise to the output action set; the current state set generates a microwave sensor model script after adjusting the resonance structure according to the action set;
[0012] S5. Read the microwave sensor model script based on the full-wave electromagnetic simulation software and automatically construct the microwave sensor model, simulate the constructed microwave sensor model, and obtain simulation data;
[0013] S6. Calculate the reward value obtained after the current action set is executed based on the current state set according to the obtained simulation data;
[0014] S7. storing the required data into the experience replay pool; wherein the required data includes the current state set, the current action set, the next state set, the next action set, and the reward value;
[0015] S8. The current evaluation network calculates the mathematical expectation of the first reward value based on the current action in the current state set and the current strategy action in the current action set in the experience replay pool, and the target evaluation network calculates the mathematical expectation of the second reward value based on the next action in the next state set and the next strategy action in the next action set;
[0016] S9. Calculate the difference between the mathematical expectation of the first reward value and the mathematical expectation of the second reward value, and update the parameters of the current policy network and the current evaluation network using the policy gradient method based on the calculated difference;
[0017] S10. Update the parameters of the target strategy network and the target evaluation network according to the update method of the full-wave electromagnetic simulation software;
[0018] S11. Loop through steps S4-S10 until the parameters of the current neural network and the target neural network converge to obtain an intelligent set model, wherein the intelligent machine model outputs an optimization strategy for the resonant structure, and adjusts the original resonant structure based on the output optimization strategy to obtain an optimized resonant structure.
[0019] Furthermore, the state set of the resonant structure in step S2 is expressed as:
[0020] S t ={z1, z2, ..., z M·W-1 , z M·W}
[0021] Among them, S trepresents the state set of the resonant structure; M and W represent the number of rows and columns; z M·W Indicates the status of M rows and W columns;
[0022] The action set for adjusting the resonant structure in step S2 is expressed as:
[0023]
[0024] Among them, A t The set of actions representing the adjustment of the resonant structure; Indicates the adjustment point; Indicates adjustment of length; Indicates the moving distance.
[0025] Furthermore, in step S4, the current policy network outputs a policy-related action set according to the current state set, and adds random noise to the output action set, which is expressed as:
[0026] A t =β(S t )=μ(S t ,θ μ )+N t
[0027] Among them, A t represents the action set; μ represents the current policy network; θ μ Represents the network parameters of the current policy network; N t represents random noise; β represents the action set based on the current state.
[0028] Furthermore, in step S6, the reward value obtained after the current action set is executed based on the current state set is calculated according to the obtained simulation data, which is expressed as:
[0029]
[0030] Among them, V RR Represents the reward value of the correct action; V RW represents the penalty value for wrong action; r1 represents the proportional factor; Δf represents the relative frequency shift caused by the resonant structure; V F Represents a fixed parameter.
[0031] Furthermore, the reward value V for the correct action is RR The sensitivity T of the microwave sensor is also included and is expressed as:
[0032]
[0033] Among them, ε r represents the relative dielectric constant of the injected liquid sample; f unloaded and They respectively represent the resonant frequency point in the simulation data when the liquid to be tested is not injected and the resonant frequency point in the simulation data when the liquid to be tested is injected.
[0034] Furthermore, the step S8 also includes using the calculated mathematical expectation of the first reward value to evaluate the current strategy network Q μ , expressed as:
[0035] Q μ (S t , A t )=E[R(S t , A t )+γQ μ (S t+1 ,μ(S t+1 ,θ Q ))]
[0036] Where Q is the mathematical expectation of the first reward value; γ is the reward discount factor in the Bellman equation; S t+1 represents the state set of the resonant structure after adjustment by the action set; θ Q Represents the network parameters of the current evaluation network.
[0037] Furthermore, the step S9 also includes constructing a corresponding loss function L(θ Q ), expressed as:
[0038]
[0039] Where N is the number of samples sampled in the experience replay pool; y t Represents a label variable, expressed as:
[0040] y t =R t +γQ′(S t+1 ,μ′(S t+1 ,θ μ′ ), θ Q′ )
[0041] Where Q′ represents the mathematical expectation of the second reward value; θ Q′ represents the parameters of the target evaluation network; μ′ represents the target strategy network; θ μ′ represents the parameters of the target policy network; γ represents the reward factor.
[0042] Furthermore, the policy gradient method in step S9 is expressed as:
[0043]
[0044] in, represents the policy gradient function; N represents the number of samples sampled by the experience replay pool; represents the gradient of the action policy with respect to the network parameters; Represents the gradient of the reward value relative to the action policy, A represents the action policy set, and S represents the state space.
[0045] Furthermore, in step S10, the parameters of the target strategy network and the target evaluation network are updated according to the updating method of the full-wave electromagnetic simulation software, which is expressed as:
[0046]
[0047] Where τ represents the update coefficient of the full-wave electromagnetic simulation software.
[0048] Accordingly, a microwave sensor resonant structure joint simulation optimization system is also provided, including:
[0049] A pixelation module, wherein the original resonant structure optimization area of the pixelated microwave microfluidic sensor is a square pixel unit;
[0050] A first initialization module is used to initialize a state set of the resonant structure and an action set for adjusting the resonant structure;
[0051] A second initialization module is used to initialize the parameters of the current neural network and the parameters of the target neural network; wherein the current neural network includes the current policy network and the current evaluation network, and the target neural network includes the target policy network and the target evaluation network;
[0052] The output module is used for the current strategy network to output the action set according to the current state set and add random noise to the output action set; the current state set generates a microwave sensor model script after adjusting the resonant structure according to the action set;
[0053] A construction module is used to read the microwave sensor model script based on the full-wave electromagnetic simulation software and automatically construct the microwave sensor model, simulate the constructed microwave sensor model, and obtain simulation data;
[0054] A first calculation module, used for calculating the reward value obtained after the current action set is executed based on the current state set according to the obtained simulation data;
[0055] A storage module is used to store the required data into the experience replay pool; wherein the required data includes the current state set, the current action set, the next state set, the next action set, and the reward value;
[0056] A second calculation module is used for the current evaluation network to calculate the mathematical expectation of the first reward value based on the current action in the current state set and the current strategy action in the current action set in the experience replay pool, and the target evaluation network to calculate the mathematical expectation of the second reward value based on the next action in the next state set and the next strategy action in the next action set;
[0057] A first updating module is used to calculate the difference between the mathematical expectation of the first reward value and the mathematical expectation of the second reward value, and update the parameters of the current policy network and the current evaluation network using a policy gradient method based on the calculated difference;
[0058] The second updating module is used to update the parameters of the target strategy network and the target evaluation network according to the updating method of the full-wave electromagnetic simulation software;
[0059] The optimization module is used to obtain an intelligent set model when the parameters of the current neural network and the target neural network converge. The intelligent machine model outputs an optimization strategy for the resonant structure, and adjusts the original resonant structure based on the output optimization strategy to obtain an optimized resonant structure.
[0060] Compared with the prior art, the combined simulation and optimization method and system of the microwave sensor resonant structure based on complementary open resonant rings of the present invention can simplify the manual design process of the microwave sensor resonant structure based on the principle that the microwave resonant sensor can restore the dielectric constant of the liquid to be measured by measuring the resonant frequency. It can automatically and quickly obtain a microwave sensor resonant structure with excellent performance, which plays a very positive role in promoting the industrialization process of microwave sensors. BRIEF DESCRIPTION OF THE DRAWINGS
[0061] Figure 1 This is a flow chart of a microwave sensor resonant structure joint simulation optimization method provided in Example 1;
[0062] Figure 2 is a schematic diagram of the initial resonance structure of the microwave microfluidic sensor provided in Example 1;
[0063] Figure 3 is a schematic diagram of adjusting the sensor resonant structure by the action set provided in the first embodiment;
[0064] Figure 4 is a diagram of the system neural network optimization convergence process provided in Example 1;
[0065] Figure 5 Schematic diagram of the optimized resonance structure of the microwave sensor provided in Example 1
[0066] Figure 6 This is a comparison diagram of the no-load and loaded results of the initial structure provided in Example 1;
[0067] Figure 7 Comparison diagram of initial no-load structure simulation and experimental results provided in Example 1
[0068] Figure 8 This is a diagram of the optimized structure experimental data provided in Example 1. DETAILED DESCRIPTION
[0069] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.
[0070] The purpose of the present invention is to provide a microwave sensor resonant structure joint simulation optimization method and system in view of the defects of the prior art.
[0071] Embodiment 1
[0072] This embodiment provides a microwave sensor resonant structure joint simulation optimization method, such as Figure 1 As shown, including:
[0073] S1. The optimized area of the original resonant structure of the pixelated microwave microfluidic sensor is a square pixel unit;
[0074] S2. Initialize the state set of the resonant structure and the action set for adjusting the resonant structure;
[0075] S3. Initialize the parameters of the current neural network and the parameters of the target neural network; wherein the current neural network includes the current policy network and the current evaluation network, and the target neural network includes the target policy network and the target evaluation network;
[0076] S4. The current strategy network outputs an action set according to the current state set, and adds random noise to the output action set; the current state set generates a microwave sensor model script after adjusting the resonance structure according to the action set;
[0077] S5. Read the microwave sensor model script based on the full-wave electromagnetic simulation software and automatically construct the microwave sensor model, simulate the constructed microwave sensor model, and obtain simulation data;
[0078] S6. Calculate the reward value obtained after the current action set is executed based on the current state set according to the obtained simulation data;
[0079] S7. storing the required data into the experience replay pool; wherein the required data includes the current state set, the current action set, the next state set, the next action set, and the reward value;
[0080] S8. The current evaluation network calculates the mathematical expectation of the first reward value based on the current action in the current state set and the current strategy action in the current action set in the experience replay pool, and the target evaluation network calculates the mathematical expectation of the second reward value based on the next action in the next state set and the next strategy action in the next action set;
[0081] S9. Calculate the difference between the mathematical expectation of the first reward value and the mathematical expectation of the second reward value, and update the parameters of the current policy network and the current evaluation network using the policy gradient method based on the calculated difference;
[0082] S10. Update the parameters of the target strategy network and the target evaluation network according to the update method of the full-wave electromagnetic simulation software;
[0083] S11. Loop through steps S4-S10 until the parameters of the current neural network and the target neural network converge to obtain an intelligent set model, wherein the intelligent machine model outputs an optimization strategy for the resonant structure, and adjusts the original resonant structure based on the output optimization strategy to obtain an optimized resonant structure.
[0084] like Figure 2 The figure shows the schematic diagram of the microwave microfluidic sensor resonant structure. The numerical joint simulation optimization system of the microwave sensor resonant structure is only for Figure 2 In the resonance optimization area, the other microwave sensor structure variable parameter values are Figure 2 The substrate material was determined to be Rogers 4350.
[0085] The length and width of the optimization zone of this embodiment are set to 12 mm and 6 mm respectively, and half of it is defined as the adjustment area, and the other half is adjusted symmetrically in a mirror manner. The width and height of the microfluidic channel are set to 0.2 mm and 0.2 mm. The layout of the microwave sensor resonant structure will be determined by the joint simulation optimization system. During the design stage of the microwave resonant sensor, if multiple resonant layouts are designed manually, the workload will be extremely large and complicated.
[0086] In step S1, the original resonant structure optimization area of the pixelated microwave microfluidic sensor is a square pixel unit.
[0087] In this embodiment, the resonant structure optimization area is set and pixelated into a matrix unit of size 30x30; the resonant area is converted into a matrix, and the matrix value is 0 or 1, wherein 0 represents that the matrix unit is not selected, and in the model it represents that the part is covered with metal, and 1 represents that the matrix unit is selected as a unit on the structure, and in the model it represents that the part is etched into a groove and has no metal coverage.
[0088] In step S2, a state set of the resonant structure is initialized, and an action set for adjusting the resonant structure is initialized.
[0089] The state set S of the resonant structure t Reflecting the entire resonator structure, S t It is expressed as:
[0090] S t ={z1, z2, ..., z M·W-1 , z M·W}
[0091] Among them, S t represents the state set of the resonant structure; M and W represent the number of rows and columns; z M·W represents the state of M rows and W columns; S represents the state space.
[0092] Action Set A t Reflecting the action of the resonator structure, A t It is expressed as:
[0093]
[0094] Among them, A t represents the action set for adjusting the resonant structure; A represents the action space; Indicates the adjustment point, Indicates adjustment of length. Indicates the moving distance, n represents the maximum number of adjustments for a single action strategy.
[0095] like Figure 3 The schematic diagram of the action set adjusting the sensor resonant structure is shown in The composed action set determines the adjustment strategy of the sensor resonant structure.
[0096] In this embodiment, the amount of liquid used in the optimization process can be considered as a variable to explore the most cost-effective amount of liquid, that is, compared with the original resonant structure, the optimization can use the least amount of liquid to increase the sensitivity the most. Therefore, the initial action set is adjusted to
[0097]
[0098] Among them, V LUTrepresents the volume of the liquid, which is determined by the action set in the embodiment of the present invention;
[0099] In step S4, the current strategy network outputs a strategy action set according to the current state set, and adds random noise to the output action set; the current state set generates a microwave sensor model script after adjusting the resonant structure according to the action set.
[0100] The current policy network outputs a policy-related action set based on the current state set, and adds random noise to the output action set, expressed as:
[0101] A t =β(S t )=μ(S t ,θ μ )+N t
[0102] Among them, A t represents the action set; μ represents the current policy network; θ μ Represents the network parameters of the current policy network; N t represents random noise; β represents the action strategy set based on the current state.
[0103] In step S5, the microwave sensor model script is read based on the full-wave electromagnetic simulation software and the microwave sensor model is automatically constructed, and the constructed microwave sensor model is simulated to obtain simulation data.
[0104] After the current state set is structurally adjusted according to the action set, a sensor model script is generated and passed into the simulation environment. The full-wave electromagnetic simulation software reads the script file to automatically build the model and simulate it. The obtained simulation data is used by the full-wave electromagnetic simulation software program to calculate the reward value obtained after the action set is executed based on the current state set.
[0105] In step S6, the reward value obtained after the current action set is executed based on the current state set is calculated according to the obtained simulation data, which is expressed as:
[0106]
[0107] Among them, V RR Represents the reward value of the correct action; V RW represents the penalty value for wrong actions; r1 represents the proportional factor; V F represents a fixed parameter; Δf represents the relative frequency shift caused by the resonant structure, and the parameters are as shown in Table 1:
[0108] Table 1
[0109]
[0110] In this embodiment, the reward value of the correct action includes the relative frequency shift Δf. The relative frequency shift reflects the sensitivity of the sensor, which is the optimization target of the microwave sensor. The sensitivity T of the microwave sensor is defined as the ratio of the relative resonant frequency shift Δf to the change in the dielectric constant, which is expressed as:
[0111]
[0112] Among them, ε r represents the relative dielectric constant of the injected liquid sample; f unloaded and They respectively represent the resonant frequency point in the simulation data when the liquid to be tested is not injected and the resonant frequency point in the simulation data when the liquid to be tested is injected.
[0113] According to the sensitivity T calculation formula, when the liquid to be measured is the same and only the resonant structure is changed, the larger the sensitivity function value obtained, the better the performance of the microwave sensor with the resonant structure.
[0114] In step S7, the required data is stored in the experience replay pool; wherein the required data includes the current state set, the current action set, the next state set, the next action set, and the reward value.
[0115] In this embodiment, the data stored in the experience replay pool is used for training the current strategy and evaluation network after subsequent sampling.
[0116] In step S8, the current evaluation network calculates the mathematical expectation of the first reward value based on the current action in the current state set and the current strategy action in the current action set in the experience replay pool, and the target evaluation network calculates the mathematical expectation of the second reward value based on the next action in the next state set and the next strategy action in the next action set.
[0117] This embodiment also includes using the calculated mathematical expectation of the first reward value to evaluate the current strategy network Q μ , according to the Bellman equation, Q μ It is expressed as:
[0118] Q μ (S t , A t )=E[R(S t , A t )+γQ μ (S t+1 ,μ(S t+1 ,θ Q ))]
[0119] Where Q is the mathematical expectation of the first reward value; γ represents the reward discount factor in the Bellman equation. By simulating the Bellman equation, the Q value can be recursively solved; St+1 represents the state set of the resonant structure after adjustment by the action set; θ Q Represents the network parameters of the current evaluation network.
[0120] In step S9, the difference between the mathematical expectation of the first reward value and the mathematical expectation of the second reward value is calculated, and based on the calculated difference, the policy gradient method is used to update the parameters of the current policy network and the current evaluation network.
[0121] The target evaluation network will calculate the corresponding Q value to update the parameters of the current evaluation network, and the update will involve a label variable y t ,y t It is expressed as:
[0122] y t =R t +γQ′(S t+1 ,μ′(S t+1 ,θ μ′ ), θ Q′ )
[0123] Where Q′ represents the mathematical expectation of the second reward value; θ Q′ represents the parameters of the target evaluation network; μ′ represents the target strategy network; θ μ′ represents the parameters of the target policy network; γ represents the reward factor.
[0124] In this embodiment, the target evaluation network also transmits yt to the current evaluation network, and the current evaluation network constructs the corresponding loss function L(θ Q ), expressed as:
[0125]
[0126] Where N is the number of samples sampled by the experience replay pool.
[0127] According to the current evaluation network and the [current state set, current action set; next state set; next action set; reward value] sampled in the experience replay pool, the current policy network is trained using policy gradient, where the policy gradient method is expressed as:
[0128]
[0129] Among them, ρ β (S) represents the probability distribution function of state S based on the behavior strategy β,
[0130] According to the Monte-Carlo method, by substituting small batches of data into the expression of the policy gradient, an unbiased estimate of the expected value can be obtained, and the policy gradient method can be further expressed as
[0131]
[0132] in, represents the policy gradient function; N represents the number of samples sampled by the experience replay pool; represents the gradient of the action policy with respect to the network parameters; Represents the gradient of the reward value relative to the action policy, A represents the action policy set, and S represents the state space.
[0133] In step S10, the parameters of the target strategy network and the target evaluation network are updated according to the update method of the full-wave electromagnetic simulation software, which is expressed as:
[0134]
[0135] Wherein, τ represents the update coefficient of the full-wave electromagnetic simulation software, and in this embodiment, τ=0.001.
[0136] like Figure 4 The figure shows the convergence process of the system neural network optimization, where Episode Q0 is the predicted reward value, Episode reward is the actual reward value, and Avg reward is the average reward value. In the 124th round, the average reward value reaches the set threshold. At the same time, when the sliding window length is 5, the error between the average reward and the predicted reward value drops from 5% to 0.366%, and the optimization system converges to output the optimal resonant structure.
[0137] like Figure 5 Shown is a schematic diagram of the optimized resonant structure of a microwave sensor. The entire groove pattern is a resonant structure, and the dark part represents the resonant structure covered by the microfluidic channel.
[0138] like Figure 6 The figure shows the comparison of the no-load and loaded results of the initial structure. The comparison between the no-load and loaded results shows that the initial microwave sensor resonant structure has the ability to characterize the dielectric properties of the liquid, which provides a premise for optimization.
[0139] like Figure 7 The figure shows the comparison of the initial no-load structure simulation and experimental results. The comparison of the simulation and experimental results under no-load conditions after processing shows the reliability of the optimized system.
[0140] like Figure 8 The figure shows the experimental results of the optimized structure. The simulation curves of the structure under different concentrations can be obtained. The curves of different concentrations can clearly show the degree of differentiation, which can achieve the ability to characterize the dielectric properties of the liquid. Especially in the case of water injection, the relative frequency offset of the optimized structure reaches 0.523, which is 27.56% higher than the 0.41 before optimization.
[0141] This embodiment optimizes the microwave microfluidic sensor Figure 4 As shown in FIG. 1 , the specific test process is as follows: the microfluidic channel is located at the bottom of polydimethylsiloxane (PDMS). In order to cover the entire CSRR structure, the size of the PDMS microfluidic channel substrate is set to 10 mm × 16 mm × 5 mm, the channel height is 0.2 mm, and there are cylinders with a diameter of 0.35 mm on both sides connected to the top of the PDMS as the inlet and outlet of the liquid to be tested. The bottom is covered on the surface of the microwave microfluidic sensor, and the etching groove part of the channel optimization area completely overlaps with the microfluidic channel part of the PDMS; the liquid to be tested is injected into the top inlet of the PDMS using a needle, and the liquid flows through the etching groove part to cause the change of the dielectric constant in the microfluidic channel, and the resonant frequency point measured by the microwave microfluidic sensor will change; pure water and ethanol-water mixtures with a concentration of 20%-100% (interval 20%) are injected respectively, and the S parameter experimental results obtained are shown in FIG. Figure 8 As shown, as the ethanol concentration decreases, the resonance curve shifts to the right, thereby realizing the measurement of the dielectric constant of ethanol solutions of different concentrations; the resonance frequency points of the experimental S parameter curves loaded with pure water and not loaded with pure water in the experimental measurement are basically consistent, indicating that this method is reliable and practical.
[0142] The microwave sensor resonant structure joint simulation optimization method of this embodiment is based on the principle that the complementary open resonant ring resonant sensor can restore the dielectric constant of the liquid to be measured by measuring the resonant frequency. It can simplify and speed up the manual design process of the microwave sensor resonant structure, and can automatically and quickly obtain a microwave sensor resonant structure with excellent performance, which plays a very positive role in promoting the industrialization process of microwave sensors.
[0143] Embodiment 2
[0144] This embodiment provides a microwave sensor resonant structure joint simulation optimization system, including:
[0145] A pixelation module, wherein the original resonant structure optimization area of the pixelated microwave microfluidic sensor is a square pixel unit;
[0146] A first initialization module is used to initialize a state set of the resonant structure and an action set for adjusting the resonant structure;
[0147] A second initialization module is used to initialize the parameters of the current neural network and the parameters of the target neural network; wherein the current neural network includes the current policy network and the current evaluation network, and the target neural network includes the target policy network and the target evaluation network;
[0148] The output module is used for the current strategy network to output the action set according to the current state set and add random noise to the output action set; the current state set generates a microwave sensor model script after adjusting the resonant structure according to the action set;
[0149] A construction module is used to read the microwave sensor model script based on the full-wave electromagnetic simulation software and automatically construct the microwave sensor model, simulate the constructed microwave sensor model, and obtain simulation data;
[0150] A first calculation module, used for calculating the reward value obtained after the current action set is executed based on the current state set according to the obtained simulation data;
[0151] A storage module is used to store the required data into the experience replay pool; wherein the required data includes the current state set, the current action set, the next state set, the next action set, and the reward value;
[0152] A second calculation module is used for the current evaluation network to calculate the mathematical expectation of the first reward value based on the current action in the current state set and the current strategy action in the current action set in the experience replay pool, and the target evaluation network to calculate the mathematical expectation of the second reward value based on the next action in the next state set and the next strategy action in the next action set;
[0153] A first updating module is used to calculate the difference between the mathematical expectation of the first reward value and the mathematical expectation of the second reward value, and update the parameters of the current policy network and the current evaluation network using a policy gradient method based on the calculated difference;
[0154] The second updating module is used to update the parameters of the target strategy network and the target evaluation network according to the updating method of the full-wave electromagnetic simulation software;
[0155] The optimization module is used to obtain an intelligent set model when the parameters of the current neural network and the target neural network converge. The intelligent machine model outputs an optimization strategy for the resonant structure, and adjusts the original resonant structure based on the output optimization strategy to obtain an optimized resonant structure.
[0156] It should be noted that the microwave sensor resonant structure joint simulation optimization system provided in this embodiment is similar to that in the first embodiment, and will not be described in detail here.
[0157] Compared with the prior art, the microwave sensor resonant structure joint simulation optimization method and system based on complementary open resonant rings in this embodiment can simplify the manual design process of the microwave sensor resonant structure based on the principle that the microwave resonant sensor can restore the dielectric constant of the liquid to be measured by measuring the resonant frequency. It can automatically and quickly obtain a microwave sensor resonant structure with excellent performance, which plays a very positive role in promoting the industrialization process of microwave sensors.
[0158] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims
1. A microwave sensor resonant structure joint simulation optimization method, characterized in that: The steps include: S1. The optimized area of the original resonant structure of the pixelated microwave microfluidic sensor is a square pixel unit; S2. Initialize the state set of the resonant structure and the action set for adjusting the resonant structure; S3. Initialize the parameters of the current neural network and the parameters of the target neural network; wherein the current neural network includes the current policy network and the current evaluation network, and the target neural network includes the target policy network and the target evaluation network; S4. The current strategy network outputs an action set according to the current state set, and adds random noise to the output action set; the current state set generates a microwave sensor model script after adjusting the resonance structure according to the action set; S5. Read the microwave sensor model script based on the full-wave electromagnetic simulation software and automatically construct the microwave sensor model, simulate the constructed microwave sensor model, and obtain simulation data; S6. Calculate the reward value obtained after the current action set is executed based on the current state set according to the obtained simulation data; S7. storing the required data into the experience replay pool; wherein the required data includes the current state set, the current action set, the next state set, the next action set, and the reward value; S8. The current evaluation network calculates the mathematical expectation of the first reward value based on the current action in the current state set and the current strategy action in the current action set in the experience replay pool, and the target evaluation network calculates the mathematical expectation of the second reward value based on the next action in the next state set and the next strategy action in the next action set; S9. Calculate the difference between the mathematical expectation of the first reward value and the mathematical expectation of the second reward value, and update the parameters of the current policy network and the current evaluation network using the policy gradient method based on the calculated difference; S10. Update the parameters of the target strategy network and the target evaluation network according to the update method of the full-wave electromagnetic simulation software; S11. Loop through steps S4-S10 until the parameters of the current neural network and the target neural network converge to obtain an intelligent machine model, wherein the intelligent machine model outputs an optimization strategy for the resonant structure, and adjusts the original resonant structure based on the output optimization strategy to obtain an optimized resonant structure.
2. A microwave sensor resonant structure joint simulation optimization method according to claim 1, characterized in that: The state set of the resonant structure in step S2 is expressed as: S t ={z1,z2,…,z M·W-1 ,z M·W } Among them, S t represents the state set of the resonant structure; M and W represent the number of rows and columns; z M·W Indicates the status of M rows and W columns; The action set for adjusting the resonant structure in step S2 is expressed as: Among them, A t The set of actions representing the adjustment of the resonant structure; Indicates the adjustment point; Indicates adjustment of length; Indicates the moving distance.
3. A microwave sensor resonant structure joint simulation optimization method according to claim 2, characterized in that: In step S4, the current policy network outputs a policy-related action set based on the current state set, and adds random noise to the output action set, which is expressed as: A t =β(S t )=μ(S t ,i μ )+N t Among them, A t represents the action set; μ represents the current policy network; θ μ Represents the network parameters of the current policy network; N t represents random noise; β represents the action set based on the current state.
4. A microwave sensor resonant structure joint simulation optimization method according to claim 3, characterized in that: In step S6, the reward value obtained after the current action set is executed based on the current state set is calculated according to the obtained simulation data, which is expressed as: Among them, V RR Represents the reward value of the correct action; V RW represents the penalty value for wrong action; r1 represents the proportional factor; Δf represents the relative frequency shift caused by the resonant structure; V F Represents a fixed parameter.
5. A microwave sensor resonant structure joint simulation optimization method according to claim 4, characterized in that: The reward value V for the correct action RR The sensitivity T of the microwave sensor is also included and is expressed as: Among them, ε r represents the relative dielectric constant of the injected liquid sample; f unloaded and They respectively represent the resonant frequency point in the simulation data when the liquid to be tested is not injected and the resonant frequency point in the simulation data when the liquid to be tested is injected.
6. A microwave sensor resonant structure joint simulation optimization method according to claim 5, characterized in that: Step S8 also includes using the calculated mathematical expectation of the first reward value to evaluate the current strategy network Q μ , expressed as: Q μ (S t ,A t )=E[R t (S t ,A t )+γQ μ (S t+1 ,μ(S t+1 ,θ Q ))] Where Q is the mathematical expectation of the first reward value; γ is the reward discount factor in the Bellman equation; S t+1 represents the state set of the resonant structure after adjustment by the action set; θ Q Represents the network parameters of the current evaluation network.
7. A microwave sensor resonant structure joint simulation optimization method according to claim 6, characterized in that: Step S9 also includes constructing the corresponding loss function L(θ Q ), expressed as: Where N is the number of samples sampled in the experience replay pool; y t Represents a label variable, expressed as: y t =R t +γQ′(S t+1 ,μ′(S t+1 ,i μ′ ),θ Q′ ) Where Q′ represents the mathematical expectation of the second reward value; θ Q′ represents the parameters of the target evaluation network; μ′ represents the target strategy network; θ μ′ represents the parameters of the target policy network; γ represents the reward factor.
8. A microwave sensor resonant structure joint simulation optimization method according to claim 7, characterized in that: The policy gradient method in step S9 is expressed as: in, represents the policy gradient function; N represents the number of samples sampled by the experience replay pool; represents the gradient of the action policy with respect to the network parameters; Represents the gradient of the reward value relative to the action policy, A represents the action policy set, and S represents the state space.
9. A microwave sensor resonant structure joint simulation optimization method according to claim 8, characterized in that: In step S10, the parameters of the target strategy network and the target evaluation network are updated according to the updating method of the full-wave electromagnetic simulation software, which is expressed as: Where τ represents the update coefficient of the full-wave electromagnetic simulation software.
10. A microwave sensor resonant structure joint simulation optimization system, characterized in that: Includes the following modules: A pixelation module, wherein the original resonant structure optimization area of the pixelated microwave microfluidic sensor is a square pixel unit; A first initialization module is used to initialize a state set of the resonant structure and an action set for adjusting the resonant structure; A second initialization module is used to initialize the parameters of the current neural network and the parameters of the target neural network; wherein the current neural network includes the current policy network and the current evaluation network, and the target neural network includes the target policy network and the target evaluation network; The output module is used for the current strategy network to output the action set according to the current state set and add random noise to the output action set; the current state set generates a microwave sensor model script after adjusting the resonant structure according to the action set; A construction module is used to read the microwave sensor model script based on the full-wave electromagnetic simulation software and automatically construct the microwave sensor model, simulate the constructed microwave sensor model, and obtain simulation data; A first calculation module, used for calculating the reward value obtained after the current action set is executed based on the current state set according to the obtained simulation data; A storage module is used to store the required data into the experience replay pool; wherein the required data includes the current state set, the current action set, the next state set, the next action set, and the reward value; A second calculation module is used for the current evaluation network to calculate the mathematical expectation of the first reward value based on the current action in the current state set and the current strategy action in the current action set in the experience replay pool, and the target evaluation network to calculate the mathematical expectation of the second reward value based on the next action in the next state set and the next strategy action in the next action set; A first updating module is used to calculate the difference between the mathematical expectation of the first reward value and the mathematical expectation of the second reward value, and update the parameters of the current policy network and the current evaluation network using a policy gradient method based on the calculated difference; The second updating module is used to update the parameters of the target strategy network and the target evaluation network according to the updating method of the full-wave electromagnetic simulation software; The optimization module is used for the convergence of the parameters of the current neural network and the target neural network to obtain an intelligent machine model. The intelligent machine model outputs an optimization strategy for the resonant structure, and adjusts the original resonant structure based on the output optimization strategy to obtain an optimized resonant structure.