Bandgap optimization method for elastic wave metamaterials based on Kriging surrogate model and Q-learning algorithm

Abstract

The present invention discloses an elastic wave metamaterial bandgap optimization method based on a Kriging proxy model and a Q-learning algorithm, which belongs to the technical field of acoustic metamaterial bandgap optimization, including S1, constructing an elastic wave metamaterial structural parameter optimization model; S2, using an LHS experimental design method to obtain sampling points and generate an I / O data set; S3, constructing a Kriging proxy model; S4, defining the state, action and reward function of the Q-learning algorithm, and generating an initial Q value; S5, initializing state parameters, interacting with the environment and obtaining a reward value, using a greedy strategy to select actions, entering a new state, interacting with the environment again and obtaining a new reward value; S6, updating the Q value table according to the formula of the Q value function; S7, repeating S5 and S6 operations until all optimization plots are completed. The present invention adopts the above-mentioned elastic wave metamaterial bandgap optimization method based on a Kriging proxy model and a Q-learning algorithm, and adopts a Q-learning algorithm combined with a Kriging proxy model to improve the efficiency of the optimal solution search process in the optimization problem and reduce the calculation cost.

Description

Technical Field

[0001] The present invention relates to the technical field of acoustic metamaterial bandgap optimization, and in particular to an elastic wave metamaterial bandgap optimization method based on a Kriging proxy model and a Q-learning algorithm. Background Art

[0002] Acoustic metamaterials (AM) refer to artificial composite structures / materials with extraordinary physical properties. Through their microstructure or arrangement, they can control, adjust or change the way and properties (i.e., bandgap characteristics) of sound or elastic waves propagating in them. Elastic metamaterials (EMs) are a subcategory of acoustic metamaterials that can open sound insulation band gaps in the low-frequency range, which makes them have broad application prospects in the field of low-frequency vibration reduction / noise reduction. From a practical point of view, it is hoped that the band gap produced by the designed EMs is in the low-frequency range and the wider the better. The bandgap range of elastic waves depends largely on the structural parameters of the EMs. It is a very challenging task to select different combinations of design parameters to ensure that the EMs exhibit a bandgap response that meets the expected operating frequency range.

[0003] Since the concept of EMs emerged, several methods have been developed to effectively optimize structural parameters. Initially, an empirical design method was adopted, which used empirical analysis and numerical tools to design the structural parameters of AM in an iterative, trial-and-error manner. This method depends largely on the designer's experience and trial-and-error process. When it comes to complex, multi-parameter EMs, this method will fail, and the resulting structural band gap range is narrow or even no band gap, which seriously restricts the application of EMs. In order to solve the above problems, a method based on structural optimization has been developed, combining topology optimization with traditional optimizers. There are two main types of such optimization methods: one is a gradient-based optimization method, such as the moving asymptote algorithm (MMA) and the bidirectional evolutionary structural optimization algorithm (BESO). This type of algorithm processes the sensitivity information of the objective function and the constraints, so that the feasible solution moves and updates along the gradient direction of the problem. It can be combined with the finite element method to solve the problem, and has a high solution efficiency, but it involves complex sensitivity analysis and is prone to fall into the local optimal solution. The second type is non-gradient optimization methods, such as genetic algorithms (GA), which have higher global search capabilities. However, this type of algorithm has the problems of high computational cost and low iteration efficiency. The main reason is that when the dimension of the design variable increases or the feasible domain of the problem to be solved becomes larger, the number of iterations of the population increases exponentially, and each iteration requires a high-density search in the design space and calculation of the fitness value of each individual, which seriously restricts the optimization ability of EMs structural parameters. As the structure of EMs becomes more and more complex, the development of an efficient and robust parameter optimization algorithm is an urgent problem to be solved.

[0004] When solving complex, multi-parameter optimization problems, simulating high-fidelity physical models is a computationally expensive process. In order to reduce the computational cost, the surrogate model method has emerged and has become a key technology in the field of structural design optimization. Surrogate models refer to approximate mathematical models that replace complex calculations in the analysis and optimization process. The use of surrogate model methods in the field of structural optimization can not only greatly improve the efficiency of optimization design, but also reduce the difficulty of optimization. Recently, the rapid development of artificial intelligence and machine learning (ML) algorithms has significantly affected many research fields. In the EMs design framework, an increasing number of studies have demonstrated the potential of ML-based methods. ML algorithms can promote the design of metamaterials by mining the mapping relationship between data target responses and structural parameters. Compared with structure-based optimization methods, data-driven design methods bring greater flexibility to the structural parameter design of EMs. However, with the increase in problem complexity, especially for the structural optimization design of EMs, it is usually difficult to obtain high-precision models in actual practice. The reason is that the structural parameter information of EMs unit cells needs to be represented by high-dimensional data, which makes the network model directly obtained by data fitting insufficient generalization ability. Reinforcement learning (RL) is another type of ML technology. Different from the above ML algorithms, RL trains the agent by interacting with the environment and giving feedback reward values ​​to determine the best search strategy. In the training phase of the algorithm, the agent receives rewards from the environment immediately after selecting the action in the current state, and chooses greedy behavior or exploratory behavior according to the strategy to evaluate more possibilities in the state space in order to make better decisions. In the field of structural optimization of EMs, traditional experience-based or structure-based optimization methods may fall into local optimal solutions or have computational bottlenecks due to the increase in design parameter space. Due to the high efficiency of the agent model in structural optimization problems and the reward mechanism and exploration ability in the reinforcement learning algorithm, the elastic wave metamaterial design framework that combines the agent model with reinforcement learning can greatly reduce the computational workload. Summary of the invention

[0005] The purpose of the present invention is to provide a bandgap optimization method for elastic wave metamaterials based on a Kriging proxy model and a Q-learning algorithm, so as to solve the problems of the shortcomings of the existing elastic wave metamaterial structural parameter optimization methods in the above-mentioned background technology.

[0006] To achieve the above-mentioned purpose, the present invention provides an elastic wave metamaterial bandgap optimization method based on the Kriging proxy model and the Q-learning algorithm. Q-learning is a popular and representative reinforcement learning algorithm. Similarly, the Kriging proxy model has good approximation capabilities and unique error estimation functions for nonlinear functions, and is also one of the most representative proxy model methods currently. A low-cost Kriging proxy model is introduced to replace the time-consuming finite element analysis process to evaluate the structural scheme; a parameter optimization method based on the Q-learning algorithm is used to solve the optimization scheme. The low-cost Kriging proxy model combined with the Q-learning algorithm can quickly and effectively search for the approximate optimal structural parameters of the elastic wave metamaterial. The following steps are included:

[0007] S1. Construct an elastic wave metamaterial structural parameter optimization model, including design variables, objective functions and constraints;

[0008] S2, using the Latin Hypercube Sampling (LHS) experimental design method to obtain sampling points, and performing finite element analysis on these sampling points to generate I / O data sets;

[0009] S3, construct a Kriging proxy model based on the data set generated by S2, and embed it into the Q-learning optimization algorithm framework to search for optimal structural parameters;

[0010] S4. Define the state, action, and reward function of the Q-learning algorithm and generate the initial Q value;

[0011] S5, initialize state parameters, interact with the environment (i.e., bring in the Kriging agent model for calculation) and obtain the reward value (i.e., the objective function), use the greedy (ε-greed) strategy to select actions, enter a new state, interact with the environment again and obtain a new reward value;

[0012] S6, select the state corresponding to the larger reward value of the two reward values ​​(new / old reward value) as the current state to continue optimization, and update the Q value table according to the formula of the Q value function;

[0013] S7. Repeat S5 and S6 until all optimization scenarios are completed.

[0014] Preferably, the design variables are the main structural parameters that affect the bandgap performance of the elastic wave metamaterial:

[0015] x i =[x1,x2,…,x α ],x i ∈[x min ,x max],i=1,2,…,α(1)

[0016] Among them, x i is the design variable, α is the number of design variables, x min and x max The design variables x are i The lower and upper limits of the value are set, and the upper limit is set to constrain the structural quality of the optimized solution. The total structural quality of the optimized solution is expressed as:

[0017]

[0018] Where m(x) is the total mass of the optimized structure, g is the number of material components in a unit cell, and ρ g and V g are the density and volume of each material component in the unit cell, respectively, and m0 is a predefined value to prevent the structural quality of the optimization solution; according to the optimization goal, the objective function is constructed using the bandgap structure of the elastic wave metamaterial, which is set to maximize the relative bandgap width between the first n+1 energy bands:

[0019]

[0020] Among them, Δω n is the band gap width between the nth band gap and the n+1th band, is the center frequency of the corresponding nth band gap, n=1,2,…,n max is the serial number of the energy band; the elastic wave metamaterial structural parameter optimization model is expressed as:

[0021]

[0022] Preferably, the experimental design of LHS adopts Iman's method: the distribution with the smallest correlation between variables is taken as the final result to improve the situation of unsatisfactory spatial distribution of sampling points and effectively avoid the sampling instability of LHS. This experimental design method provides a set of feasible initial design schemes for the construction of the surrogate model:

[0023]

[0024] Among them, X s is the input data set of sampling points, N dv is the design variable dimension, N sp is the number of sampling points. Its implementation process can be described as follows: dv N is obtained by sampling in the dimensional design space sp sampling points.

[0025] Preferably, when performing finite element analysis on these sampling points in S2, the solid mechanics module in the multi-physics field coupling software COMSOL is used to perform secondary development of its script language according to the requirements of the problem, and the program code in the MATLAB environment is obtained, and the joint simulation of COMSOL and MATLAB is started to perform the input data set X s Perform finite element numerical simulation to obtain the target response value The two constitute the data set X s ,Y s .

[0026] Preferably, in S3, the Kriging surrogate model includes model construction, model training, and model evaluation. The Kriging surrogate model is an interpolation Bayesian metamodel technology that combines a global model and a local deviation:

[0027] yx=YX=μx+Zx(5)

[0028] Among them, μx is a polynomial global approximation to the function yx, which represents the mathematical expectation of yx; Zx is a random process with zero mean and non-zero covariance; the non-zero covariance Zx is given by the following formula:

[0029] cov[Z(x (i) ), Z(x (j) )]=σ 2 R(x (i) , x (j) ) (6)

[0030] Where R is the correlation matrix, R(x (i) , x (j) ) are any two sampling points x (i) and x (j) The correlation function between them; using Gaussian correlation function, R(x (i) , x (j) ) is expressed as:

[0031]

[0032] Among them, θ k are the position-related parameters to be determined, i.e., the hyperparameters of the Kriging surrogate model;

[0033] For any unknown sampling point x in the design space, a fast prediction of the predicted value is achieved based on the correlation function between known sampling points in the space:

[0034]

[0035] in, is the maximum likelihood value of parameter μ, is a unit vector, r T (x) is the correlation vector between the unknown position sampling point and the data set, expressed as:

[0036]

[0037] In formula (8) The following equation is used to estimate:

[0038]

[0039] The estimated variance of the output model is calculated using the following formula:

[0040]

[0041] The hyperparameter θ is estimated using maximum likelihood as:

[0042]

[0043] in, and |R| are functions of θ;

[0044] After the Kriging proxy model is built, the model training is completed by training its model hyperparameters. The hyperparameter optimization problem is:

[0045]

[0046] in, and are the parameters μ and σ 2 The maximum likelihood estimate of θ k (k=1,2,…,N dv ) is the hyperparameter of Kriging;

[0047] During the training process, the Kriging surrogate model uses error evaluation indicators: root mean square error RMSE and complex correlation function R 2 The prediction accuracy of the model is evaluated by using the value, and the expressions of the two are:

[0048]

[0049] Among them, N te To test the number of sampling points, is the target value of the test sample, is the predicted value of the test sample, is the target mean of the test sample. R 2 The closer the value is to 1, the better the overall prediction accuracy of the proxy model is, and the closer the RMSE is to 0, the smaller the overall prediction error of the proxy model is.

[0050] The RL model is divided into three parts: environment, agent, and policy. In the current state, the agent performs an action and obtains a reward from the environment. Based on the feedback, the policy is updated to force the agent to take the correct action next time in the same state, thereby obtaining a greater reward. Q-learning is a popular and representative RL algorithm that records the search learning process of the RL algorithm in the form of a Q-value table, where the Q-value function in the Q-value table regards the state and action as variables. Before the optimization begins, the Q-value table is initialized, and then the Q-value function in the table is updated according to the reward value in the training step until the reward is maximized. The Q-learning algorithm is used in the EMs structural parameter optimization framework: the band gap (i.e., the objective function) is obtained by finite element analysis; the design variable of the optimization problem is the agent; and the ε-greed strategy is used as the action selection strategy.

[0051] Preferably, in S4, states, actions, and reward functions are all important components of the Q-learning algorithm. 1. The number of state divisions should be appropriate and the expression should be reasonable. Too many states will make the algorithm converge too slowly and difficult to reach the optimal solution; too few states will miss the optimal solution. i =[x1,x2,…,x α ] as a set of states, the upper and lower limits of the design variables [x min ,x max ] is the boundary of the state space, and the state space composed of all states is the range of all feasible solutions; 2. In the parameter optimization problem, action refers to changing a certain design variable, thereby changing the state of the intelligent agent. The increase and decrease values ​​of the design variable are all d, and each state contains 2i actions to choose from, that is: Ai=x i ±d; when a variable exceeds the boundary after a certain action changes, the variable will not change; 3. In the parameter optimization problem of elastic wave metamaterials, the reward function is directly defined as the size of the objective function. Therefore, the agent's learning always tends to find a state with a larger reward value. In the Q-learning algorithm, all state-action pairs and the Q values ​​of the corresponding positions are combined into a Q value table, and the initial Q value table is set to zero.

[0052] Preferably, the initialization state parameters input in S5 are the initial structural parameters of the elastic wave metamaterial, the environment is the Kriging agent model constructed in S3, and the reward value is the objective function value predicted by the agent model. The Q-learning algorithm uses the ε-greed strategy to select actions, that is, when selecting actions, a random action is selected with a probability of ε, and the best action is selected with a probability of 1-ε.

[0053] Preferably, at the beginning of the optimization, all Q values ​​in the Q value table are zero, and thereafter in each training step, the Q value of the corresponding position in the Q value table is updated according to the current "state-action" pair and the obtained reward value. The Q value update formula is as follows:

[0054] Q(S t , A t )=Q(S t , A t )+α[Rt +1 +γmaxQ(S t+1 , A)-Q(S t , A t )] (15)

[0055] Among them, Q(S t , A t ) indicates that it is in state S t And take action A t The updated Q value after the equal sign; Q (S t , A t ) represents the Q value before update; α is the learning rate, which has a value range of [0, 1] and determines the degree of update of the function value; γ is the discount factor; R t+1 Reward for feedback.

[0056] Preferably, in S7, the maximum number of exploration episodes of the Q-leaming algorithm is set to Max-episode.

[0057] Therefore, the present invention adopts the elastic wave metamaterial bandgap optimization method based on the Kriging proxy model and the Q-learning algorithm, constructs a low-cost Kriging proxy model and integrates it into the Q-learning optimization algorithm framework, which replaces the complex and time-consuming finite element analysis process on the one hand, and guides the search direction of the optimization algorithm on the other hand. While ensuring the elastic wave metamaterial bandwidth optimization requirements, the optimization efficiency of the entire optimization process is improved, and the calculation cost is greatly reduced.

[0058] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A schematic diagram of the unit cell structure of the elastic wave metamaterial in the present invention;

[0060] Figure 2 A schematic diagram of a periodically configured elastic wave metamaterial vibration reduction structure in the present invention;

[0061] Figure 3A flowchart of the structural parameter optimization process of elastic wave metamaterials based on the Kriging-(Q-learning) method in the present invention;

[0062] Figure 4 It is a historical convergence curve diagram of the objective function based on the Kriging-(Q-learning) method of an embodiment of the present invention;

[0063] Figure 5 The energy band diagram of the optimal elastic wave metamaterial structure based on the Kriging-(Q-learning) method according to an embodiment of the present invention;

[0064] Figure 6 4 is an energy band diagram of the initial configuration of the elastic wave metamaterial according to an embodiment of the present invention. DETAILED DESCRIPTION

[0065] Example

[0066] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0067] Figure 1 It is a schematic diagram of the unit cell structure of elastic wave metamaterial; Figure 2 Schematic diagram of elastic wave metamaterial structure; This embodiment studies a two-dimensional regular hexagonal lattice composed of three materials: aluminum, lead and silicone rubber, and a local resonance elastic wave metamaterial containing multiple resonators. Its lattice constant is a = 50mm. Aluminum is used as the matrix; silicone rubber is used as the elastic medium; lead is used as the local oscillator counterweight between the scatterer and the interlayer, and its material properties are shown in Table 1 below:

[0068] Table 1 Material properties

[0069]

[0070]

[0071] The elastic wave metamaterial structural parameters that need to be optimized are: the diameter of the inclusion scatterer d0 = 6mm, d1 = 4mm, d2 = 4mm and the thickness of the multilayer coating t1 = 6mm, t2 = 6mm. The corresponding positions of the specific parameters are as follows Figure 1 shown.

[0072] Figure 3This is the process framework for optimizing the structural parameters of elastic wave metamaterials based on the Kriging-(Q-learning) method in this embodiment. This embodiment includes the following steps:

[0073] Step 1: The initial design variables of the optimization model are x5 = [6, 4, 4, 6, 6], and the upper and lower limits of the design variables are x min =[10,8,8,7.5,7.5], x max =[6, 4, 4, 3.5, 3.5]; m0 takes the middle value of the design variable range, that is, m0 = 10.8706 kg; the objective function is to maximize the relative band gap width between the first 12 energy bands. Substituting it into formula (4), the objective function value is: f(x) = 0.5428.

[0074] Step 2: The implementation process of LHS is: dv = N is sampled in the 5-dimensional design space dv = 175 sampling points, these 175 initial sampling points are in the range of [0, 1], and then the generated initial sampling points are converted to the design variable parameter space range. The 175 sampling points X obtained from LHS are 175 Bring it into the finite element software for analysis and obtain the objective function response value Y corresponding to 175 sampling points 175 All sampling points and corresponding function response values ​​constitute the I / O data set (X 175 , Y 175 ).

[0075] Step 3: First, according to the data set (X 175 , Y 175 ) constructs a Kriging surrogate model for this optimization problem, replaces the expensive finite element analysis process with a low-cost Kriging surrogate model, and constructs a new environment space by obtaining an approximate objective function in the Q-learning parameter optimization algorithm.

[0076] Step 4: In this embodiment, a set of design variable values, such as x5 = [6, 4, 4, 6, 6], is taken as a state, and the upper and lower limits of the state are x min =[10,8,8,7.5,7.5], x max =[6, 4, 4, 3.5, 3.5]; the increase or decrease value of each design variable is d = 0.5, and each state has A 10 =x5±0.5=10 actions can be selected; the reward function is the approximate objective function value calculated by the Kriging agent model; all state-action pairs and corresponding Q values ​​form a Q value table, and all Q values ​​are initialized to "0" before the optimization starts.

[0077] Steps 5 and 6, substitute the initial state Initial-State = [6, 4, 4, 6, 6] into the environment built by the Kriging agent model for interaction, and obtain a reward value Reward. Among them, the ε of the ε-greed strategy is set to 0.8, that is, a random action is selected with a probability of 0.8, and the best action is selected with a probability of 0.2. According to the selected action Action, the design variables are changed and the new state New-State is entered. Bring it into the environment again for interaction to obtain a new reward value New-Reward. According to formula (6), update the Q value corresponding to the state-action pair (Initial-state)-Action in the Q value table, and compare the size of Reward and New-Reward. Keep the state corresponding to the larger value to replace Initial-state to continue searching.

[0078] Step 7: Repeat steps 5 and 6 until the termination condition of the Q-learning algorithm is met, that is, the maximum number of episodes Max-episode = 200 is reached.

[0079] The Kriging-(Q-learning) optimization method is used to solve the mathematical model described in equation (4). The optimization converges after 200 episode iterations, and the historical convergence curve of the objective function value is obtained as follows: Figure 4 shown.

[0080] Table 2 Kriging-(Q-learning) optimization algorithm optimization results

[0081]

[0082] Table 2 shows the optimization results of elastic wave metamaterials by Kriging-(Q-learning) method. It can be seen that the optimized design variables are: d0 = 10mm, d1 = 4.5mm, d2 = 6.5mm, t1 = 5.5mm, t2 = 4mm. Compared with the initial structure, the circular scatterer diameter d0 and the inner and outer layer inclusion diameters d1 and d2 of the optimized structure increase, and the thickness t1 and t2 of the elastic coating layer decrease. The optimized objective function value is 0.8719, which is 60.63% higher than the initial design target value of 0.5428.

[0083] The optimized elastic wave metamaterial band structure is shown in Figure 2. Figure 5 As shown in the figure, the frequency ranges of the three complete band gaps are [314.95~418.67] Hz, [495.22~692.08] Hz and [933.26~1130.63] Hz. Figure 6), the first relative band gap width decreases slightly, while the second and third band gaps increase significantly, which ultimately leads to an increase in the function target value.

[0084] In order to illustrate the effect of the algorithm of this embodiment, Table 3 shows the comparison of the results of using the Kriging-(Q-learning) method and the FEA-GA method in this embodiment. It can be seen from the table that the objective function values ​​optimized by the Kriging-(Q-learning) method and the FEA-GA method are 0.8719 and 0.8814, respectively, which are 60.63% and 62.38% higher than the initial structural parameter configuration, respectively, and the difference between the two is very small. However, under the same computer configuration (same optimization environment), the optimal solution calculation time of the Kriging-(Q-learning) method and the FEA-GA method is 18 and 186 hours, respectively.

[0085] Table 3 Comparison of optimization results between Kriging-Q-learning and FEA-GA

[0086]

[0087] The results show that the Kriging-(Q-learning) method is used to optimize the structural parameters of elastic wave metamaterials, which significantly improves the total band gap value of the first 12 orders of its energy band, achieving the optimization goal. At the same time, the optimization method based on Kriging-(Q-learning) makes up for the complex and time-consuming calculation process of the FEA-GA optimization method, greatly improving the optimization efficiency.

[0088] Therefore, the present invention adopts the above-mentioned elastic wave metamaterial band gap optimization method based on the Kriging proxy model and the Q-learning algorithm. The Q-learning algorithm is used to improve the efficiency of the optimal solution search process in the optimization problem. At the same time, the low-cost Kriging proxy model is combined to greatly reduce the number of complex and time-consuming finite element analyses, thereby reducing the computational cost.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A bandgap optimization method for elastic wave metamaterials based on Kriging surrogate model and Q-learning algorithm, characterized in that: The following steps are involved: S1. Construct an elastic wave metamaterial structural parameter optimization model, including design variables, objective functions and constraints; The design variables are the main structural parameters that affect the bandgap performance of elastic wave metamaterials: Among them, x i is the design variable, α is the number of design variables, x min and x max The design variables x are i The lower and upper limits of the value are set, and the upper limit is set to constrain the structural quality of the optimized solution. The total structural quality of the optimized solution is expressed as: Where m(x) is the total mass of the optimized structure, g is the number of material components in a unit cell, and ρ g and V g are the density and volume of each material component in the unit cell, respectively, and m0 is a predefined value. According to the optimization goal, the objective function is constructed using the bandgap structure of the elastic wave metamaterial, which is set to maximize the relative bandgap width between the first n+1 energy bands: Among them, Δω n is the band gap width between the nth band gap and the n+1th band, is the center frequency of the corresponding nth band gap, n=1,2,…,n max is the serial number of the energy band; the elastic wave metamaterial structural parameter optimization model is expressed as: Find:x={x r ,x t }=[x r1 ,x r2 ,…,x rs ,x t1 ,x t1 ,…,x tl ] S2, using the Latin hypercube sampling experimental design method to obtain sampling points, and performing finite element analysis on these sampling points to generate I / O data sets; S3, constructing a Kriging proxy model based on the data set generated by S2, and embedding it into the Q-learning optimization algorithm framework to search for optimal structural parameters; The Kriging surrogate model includes model construction, model training, and model evaluation: y(x)=Y(X)=μ(x)+Z(x) (5) Where μ(x) is a polynomial global approximation to the function y(x), representing the mathematical expectation of y(x); Z(x) is a random process with zero mean and non-zero covariance; the non-zero covariance Z(x) is given by: cov[Z(x (i) ),Z(x (j) )]=σ 2 R(x (i) ,x (j) ) (6) Where R is the correlation matrix, R(x (i) ,x (j) ) are any two sampling points x (i) and x (j) The correlation function between them; using Gaussian correlation function, R(x (i) ,x (j) ) is expressed as: Among them, θ k is the position-related parameter to be determined; For any unknown sampling point x in the design space, a fast prediction of the predicted value is achieved based on the correlation function between known sampling points in the space: in, is the maximum likelihood value of parameter μ, is a unit vector, where N sp is the number of sampling points, Y s is the target response value, r T (x) is the correlation vector between the unknown position sampling point and the data set, expressed as: In formula (8) The following equation is used to estimate: The estimated variance of the output model is calculated using the following formula: The hyperparameter θ is estimated using maximum likelihood as: in, and |R| are functions of θ; After the Kriging proxy model is built, the model training is completed by training its model hyperparameters. The hyperparameter optimization problem is: in, and are the parameters μ and σ 2 The maximum likelihood estimate of θ k (k=1,2,…,N dv ) is the hyperparameter of Kriging, N dv is the design variable dimension; During the training process, the Kriging surrogate model uses error evaluation indicators: root mean square error RMSE and complex correlation function R 2 The prediction accuracy of the model is evaluated by using the value, and the expressions of the two are: Among them, N te To test the number of sampling points, is the target value of the test sample, is the predicted value of the test sample, is the target mean of the test sample; S4. Define the state, action, and reward function of the Q-learning algorithm and generate the initial Q value; S5, initialize state parameters, interact with the environment and obtain reward values, use greedy strategies to select actions, enter a new state, interact with the environment again and obtain new reward values; S6, select the state corresponding to the larger reward value of the two reward values ​​as the current state to continue optimization, and update the Q value table according to the formula of the Q value function; S7. Repeat S5 and S6 until all optimization scenarios are completed.

2. The elastic wave metamaterial band gap optimization method based on the Kriging proxy model and the Q-learning algorithm according to claim 1, characterized in that: The experimental design of Latin hypercube sampling adopts Iman's method, which provides a set of feasible initial design schemes for the construction of surrogate models: Among them, X s is the input data set of sampling points, N dv is the design variable dimension, N sp is the number of sampling points.

3. The elastic wave metamaterial bandgap optimization method based on Kriging proxy model and Q-learning algorithm according to claim 1, characterized in that: When performing finite element analysis on these sampling points in S2, the solid mechanics module in the multi-physics coupling software COMSOL is used to perform secondary development on its script language, obtain the program code in the MATLAB environment, start the joint simulation of COMSOL and MATLAB, and perform the input data set X s Perform finite element numerical simulation to obtain the target response value The two constitute the data set (X s ,Y s ).

4. The elastic wave metamaterial bandgap optimization method based on the Kriging proxy model and the Q-learning algorithm according to claim 1, characterized in that: In S4, the value of the design variable x in equation (1) is i =[x1,x2,…,x α ] as a set of states, the upper and lower limits of the design variables [x min ,x max ] is the boundary of the state space, and the state space composed of all states is the range of all feasible solutions; the increase and decrease values ​​of the design variables are d, and each state contains 2i actions to choose from; in the Q-learning algorithm, all state-action pairs and the Q values ​​of the corresponding positions are combined into a Q value table, and the initial Q value table is set to zero.

5. The elastic wave metamaterial bandgap optimization method based on Kriging proxy model and Q-learning algorithm according to claim 1, characterized in that: The initialization state parameters input in S5 are the initial structural parameters of the elastic wave metamaterial, and the environment is the Kriging proxy model constructed in S3.

6. The elastic wave metamaterial bandgap optimization method based on Kriging proxy model and Q-learning algorithm according to claim 1, characterized in that: The update formula of Q value is as follows: Q(S t ,A t )=Q(S t ,A t )+α[R t+1 +γmaxQ(S t+1 ,A)-Q(S t ,A t )] (15) Among them, Q(S t ,A t ) indicates that it is in state S t And take action A t The updated Q value after the equal sign; Q (S t ,A t ) represents the Q value before update; α is the learning rate, which has a value range of [0,1] and determines the degree of update of the function value; γ is the discount factor; R t+1 Reward for feedback.

7. The elastic wave metamaterial bandgap optimization method based on Kriging proxy model and Q-learning algorithm according to claim 1, characterized in that: In S7, the maximum number of exploration episodes of the Q-learning algorithm is set to Max-episode.

Images