A FBG sensor layout optimization method for shield tunnels

Abstract

The present invention discloses a FBG sensor layout optimization method for a shield tunnel, and relates to the technical field of shield tunnel structure monitoring optimization methods, S1: the intelligent agent interacts with the environment to generate state experience and stores it in a playback experience pool; S2: batch samples are extracted from the playback experience pool for training; S3: after several steps of training, the WOA algorithm is used to iteratively update the strategy network; S4: the WOA population is initialized, and the individuals are random parameters generated based on the intelligent agent strategy network parameters; S5: each individual in the population interacts with the environment to obtain the corresponding fitness function value and select the optimal strategy network parameters; S6: the parameters of other individuals in the population are iterated with the optimal individual network parameters; S7: the final optimal strategy network is updated to the intelligent agent; S8: the termination condition is met, and the training is terminated; if the termination condition is not met, steps S2-S7 are repeated. The present invention adopts the above-mentioned FBG sensor layout optimization method for a shield tunnel to improve the accuracy of shield tunnel health monitoring.

Description

Technical Field

[0001] The invention relates to the technical field of shield tunnel structure monitoring optimization methods, and in particular to an FBG sensor layout optimization method for a shield tunnel. Background Art

[0002] At present, some scholars have used convergence meter measurement, ground three-dimensional laser scanning (LiDAR), video images and total station electronic distance meter to reconstruct the tunnel morphology in three dimensions, and obtained a good three-dimensional reconstruction model. However, these methods have more or less certain problems in actual engineering applications. For example, when using convergence meters for monitoring, the measurement results will be inaccurate due to the nature of the equipment and inappropriate human operation; the equipment of ground three-dimensional laser scanning and total station electronic distance meter is expensive and relies on manual on-site data collection, which cannot meet the needs of real-time tunnel morphology monitoring; using video images to reconstruct tunnel morphology can quickly obtain tunnel reconstruction models in real time, but its monitoring accuracy cannot meet the needs. Traditional monitoring equipment and monitoring methods cannot meet the structural morphology perception accuracy and perception speed required by modern tunnel maintenance work due to their poor timeliness, few monitoring points, low monitoring accuracy, difficulty in deployment, and susceptibility to environmental factors.

[0003] When choosing the FBG sensor layout method, most of them rely on subjective experience, which is not very interpretable. It is a sub-optimal method for solving the sensor layout optimization problem. In actual engineering applications, the sensor layout schemes obtained by these optimization methods do not achieve the best information acquisition accuracy for the monitoring target, resulting in insufficient accuracy in the structural morphology reconstruction of the shield tunnel.

[0004] Therefore, it is necessary to provide a FBG sensor layout optimization method for shield tunnels to solve the above problems. Summary of the invention

[0005] The purpose of the present invention is to provide a FBG sensor layout optimization method for a shield tunnel, which solves the effectiveness of the optimal layout problem of FBG sensor elements in the shield tunnel section, provides theoretical support for the sensor layout of structural health monitoring, and further improves the reconstruction accuracy by an average of 1.73%, thereby further improving the accuracy of shield tunnel health monitoring.

[0006] To achieve the above object, the present invention provides a method for optimizing the layout of FBG sensors in a shield tunnel, introduces a swarm intelligence algorithm, takes the strategy network of a single agent as a population individual, uses the accumulated reward value in the DQN algorithm as the objective function of the WOA algorithm, and performs a population iteration of the WOA after the DQN algorithm is iterated several times, which specifically includes the following steps:

[0007] S1: Execute the DQN algorithm to store the state experience generated by the interaction between the agent and the environment into the replay experience pool;

[0008] S2: Extract batch samples from the replay experience pool to train the agent policy network;

[0009] S3: After several steps of training, the policy network is iteratively updated using the WOA algorithm;

[0010] S4: Initialize the WOA population, where individuals are random parameters generated based on the agent strategy network parameters;

[0011] S5: Each individual in the population interacts with the environment to obtain the corresponding fitness function value and select the optimal strategy network parameters;

[0012] S6: According to the optimal individual network parameters, perform parameter iteration on other individuals in the population respectively;

[0013] S7: Update the final optimal strategy network to the agent;

[0014] S8: If the termination condition is met, terminate the training; if the termination condition is not met, repeat steps S2-S7. Preferably, in S1, the DQN algorithm includes the following steps:

[0015] Step 1: Define each selected point as an independent state, define the action of selecting interpolation points and the process of changing interpolation points as a Markov decision process, and propose a control algorithm based on WOA-DQN to learn the strategy of selecting and changing interpolation points;

[0016] Step 2: The agent learns the strategy, which is specifically manifested in controlling the change of the interpolation point value;

[0017] Step 3: Compare the total error of the fitting curve of the initial state s0 with the current state fitting s i Compare the overall error of the curve to get the reward value;

[0018] Step 4: Experience playback, storing the experience of each state into the cache;

[0019] Step 5: The action-value function represents the expected cumulative reward of the agent's optional behavior in the current state in the future steps. Preferably, in step 1, the Markov decision process specifically includes, for each state, using the currently selected interpolation point sequence to represent:

[0020] s i =(P1, P2, ..., P k )

[0021] where i = 0, 1, 2, ..., N a , N a is the maximum number of actions per round, k = 1, 2, ..., n, n is the maximum number of interpolation points, P kThe value of is related to the angle, and the interpolation points are evenly distributed on the shield tunnel section curve. The initial state s0 is expressed as:

[0022]

[0023] Where j = 1, 2, ..., n-1, starting from the initial state s0, the interpolation point sequence of the next state is generated by controlling the change of the interpolation point through the agent decision, until the optimal state is found or the constraint condition is triggered;

[0024] Given the current state s i , the agent chooses the next action a i , get the next state s i+1 and reward r i+1 , the state update formula is:

[0025]

[0026] where i = 0, 1, 2, ..., N a -1, is the state transfer function, a i is the agent's decision-making behavior. The Markov decision process of this model is:

[0027]

[0028] Preferably, in step 2, the strategy is a probability function of output Q relative to the function selection of each possible action, expressed as:

[0029]

[0030] Where Π(·) is the mapping strategy, S and A represent the state space and action space in state s respectively, the strategy for selecting actions π={π1,π2}, π1 is the interpolation point for selecting the agent to change in the current state, and π2 is the action after the agent selects π1 in the current state.

[0031] π1(a|s)=pp=0,1,…,n-1

[0032]

[0033] Where p is the position number of the interpolation point selected for change. The NoisyNet method is introduced in the DQN network to add decomposed Gaussian noise to the neural network, allowing the agent to automatically adjust the noise parameters during training and define the noise formula:

[0034] ω=μ+∑⊙ε

[0035] Where ω is the noise weight parameter, μ, ∑ are the adaptive noises of the Noisy layer, and ε is the zero-mean Gaussian noise.

[0036] Preferably, in step 3, the final reward value formula of the algorithm is:

[0037]

[0038] where e i is the overall fitting error of the interpolation point fitting curve selected in the current round, e0 is the overall fitting error of the initial interpolation point fitting curve, r i The reward obtained in the current step, the value range is (-1, 1];

[0039] The reward is set to a decaying reward of K steps, and the reward formula is:

[0040]

[0041] Where γ is the reward decay rate, R i is the K-step decay reward, r i+k is the reward for k steps after the current step. Preferably, in step 4, the priority weight update formula is as follows:

[0042] δ i =|R i +γQ target (s i , arg max Q(s i , a))-Q(s i-1 , A i-1 )|

[0043] Where Q target (·) and Q(·) represent the target Q function and the Q function to be trained in the algorithm, respectively.

[0044] Preferably, in step 5, the agent selects the action in a way that maximizes the total future reward, and the expected reward The optimal Q function is as follows:

[0045]

[0046] Q * (s, a) obeys the Bellman equation, and the update formula is as follows:

[0047]

[0048] Among them, s′ and a′ represent the next state and the possible action of the next step respectively;

[0049] In the DQN algorithm, a Q neural network consisting of multiple hidden layers is used to gradually approximate the Q function. After the Q network receives the state at a certain moment, it calculates and finally outputs the Q value of each possible action. When training the neural network, the loss function is defined as:

[0050]

[0051] Where ω′ is the target network ω is the network weight of the estimated network Q, and the weight is updated using the stochastic gradient descent method:

[0052]

[0053] Where λ is the neural network learning rate.

[0054] Therefore, the present invention adopts the above-mentioned FBG sensor layout optimization method for a shield tunnel, which has the following beneficial effects:

[0055] (1) The present invention is effective in solving the optimal layout problem of FBG sensor elements in shield tunnel sections, providing theoretical support for sensor layout for structural health monitoring, and further improving the reconstruction accuracy by an average of 1.73%, further improving the accuracy of shield tunnel health monitoring.

[0056] (2) In the study of hyperparameter sensitivity, the present invention uses the control variable method to analyze multiple influencing parameters such as the learning rate of the neural network, the initial noise parameters of Noisy-Net, the hidden layer parameters of the neural network, and the corresponding update frequency of the neural network, and obtains the hyperparameter value range with the fastest convergence speed of the intelligent agent and the best stability after convergence.

[0057] (3) After several independent experiments, the present invention obtained the optimal number and optimal layout positions of FBG sensor elements on the shield tunnel section under various working conditions. Taking various working conditions into consideration, the FBG sensor element layout scheme was finally fixed to 10 and the specific layout positions of the sensor elements were obtained.

[0058] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is a flow chart of a method for optimizing the layout of FBG sensors in a shield tunnel according to the present invention; DETAILED DESCRIPTION

[0060] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0061] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.

[0062] The words "include" or "comprises" and the like used in the present invention mean that the elements before the word include the elements listed after the word, and do not exclude the possibility of also including other elements. The orientation or position relationship indicated by the terms "inside", "outside", "upper", "lower", etc. is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation of the present invention. When the absolute position of the described object changes, the relative position relationship may also change accordingly. In the present invention, unless otherwise clearly specified and limited, the terms "attachment" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral body; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0063] like Figure 1 As shown, the present invention provides a shield tunnel FBG sensor layout optimization method, introduces a swarm intelligence algorithm, takes the strategy network of a single agent as a population individual, uses the cumulative reward value in the DQN algorithm as the objective function of the WOA algorithm, and performs a WOA population iteration after the DQN algorithm iterates several times. In the WOA-DQN algorithm, the single-step state quantity is no longer concerned, but the current population fitness is calculated through the overall reward of the entire round and the population iteration is performed. Specifically, the following steps are included:

[0064] S1: Execute the DQN algorithm to store the state experience generated by the interaction between the agent and the environment into the replay experience pool;

[0065] S2: Extract batch samples from the replay experience pool to train the agent policy network;

[0066] S3: After several steps of training, the policy network is iteratively updated using the WOA algorithm;

[0067] S4: Initialize the WOA population, where individuals are random parameters generated based on the agent strategy network parameters;

[0068] S5: Each individual in the population interacts with the environment to obtain the corresponding fitness function value and select the optimal strategy network parameters;

[0069] S6: According to the optimal individual network parameters, perform parameter iteration on other individuals in the population respectively;

[0070] S7: Update the final optimal strategy network to the agent;

[0071] S8: If the termination condition is met, the training is terminated; if the termination condition is not met, steps S2-S7 are repeated.

[0072] In step S1, the DQN algorithm includes the following steps:

[0073] Step 1: Define each selected point as an independent state, define the action of selecting interpolation points and the process of changing interpolation points as a Markov decision process, and propose a control algorithm based on WOA-DQN to learn the strategy of selecting and changing interpolation points;

[0074] Step 2: The agent learns the strategy, which is specifically manifested in controlling the change of the interpolation point value;

[0075] Step 3: Compare the total error of the fitting curve of the initial state s0 with the current state fitting s i Compare the overall error of the curve to get the reward value;

[0076] Step 4: Experience playback, storing the experience of each state into the cache;

[0077] Step 5: The action-value function represents the expected cumulative reward in future steps for the agent's optional actions in the current state.

[0078] In step 1, the Markov decision process specifically includes, for each state, representing the currently selected interpolation point sequence:

[0079] s i =(P1, P2, ..., P k )

[0080] where i = 0, 1, 2, ..., N a , N a is the maximum number of actions per round, k = 1, 2, ..., n, n is the maximum number of interpolation points, P k The value of is related to the angle, and the interpolation points are evenly distributed on the shield tunnel section curve. The initial state s0 is expressed as:

[0081]

[0082] Where j = 1, 2, ..., n-1, starting from the initial state s0, the interpolation point sequence of the next state is generated by controlling the change of the interpolation point through the agent decision, until the optimal state is found or the constraint condition is triggered;

[0083] Given the current state s i , the agent chooses the next action a i , get the next state s i+1and reward r i+1 , the state update formula is:

[0084]

[0085] where i = 0, 1, 2, ..., N a -1, is the state transfer function, a i is the agent's decision-making behavior. The Markov decision process of this model is:

[0086]

[0087] Step 2: The agent learns the strategy, which is specifically manifested in the way the interpolation point value changes. In step 2, the strategy is a probability function of the output Q relative to the function to select each possible action, expressed as:

[0088]

[0089] Where Π(·) is the mapping strategy, S and A represent the state space and action space in state s respectively, the strategy for selecting actions π={π1,π2}, π1 is the interpolation point for selecting the agent to change in the current state, and π2 is the action after the agent selects π1 in the current state.

[0090] π1(a|s)=pp=0,1,…,n-1

[0091]

[0092] Where p is the position number of the interpolation point selected for change. In order to allow the agent to explore unknown states as much as possible and effectively, the NoisyNet method is introduced in the DQN network to add decomposed Gaussian noise to the neural network, so that the agent can automatically adjust the noise parameters during training to achieve effective exploration. The noise formula is defined as:

[0093] ω=μ+∑⊙ε

[0094] Where ω is the noise weight parameter, μ, ∑ are the adaptive noises of the Noisy layer, and ε is the zero-mean Gaussian noise.

[0095] Step 3: Compare the total error of the fitting curve of the initial state s0 with the current state fitting s i The final reward value is obtained by comparing the overall error of the curve; in step three, the final reward value formula of the algorithm is:

[0096]

[0097] where e iis the overall fitting error of the interpolation point fitting curve selected in the current round, e0 is the overall fitting error of the initial interpolation point fitting curve, r i The reward obtained in the current step, the value range is (-1, 1];

[0098] In order to enable the agent to predict the impact of the current action on future actions, the reward is set as a decaying reward of K steps. The reward formula is:

[0099]

[0100] Where γ is the reward decay rate, R i is the K-step decay reward, r i+k is the reward for k steps after the current step.

[0101] Step 4: Experience replay, store the experience of each state into the cache; in step 4, give priority to repeatedly letting the agent learn the experience where the current agent performs poorly, and use TD-error to represent the value learned by the agent under the current experience. The priority weight update formula is as follows:

[0102] δ i =|R i +γQ target (s i , arg max Q(s i , a))-Q(s i-1 , A i-1 )|

[0103] Where Q larget (·) and Q(·) represent the target Q function and the Q function to be trained in the algorithm, respectively.

[0104] Step 5: The action-value function represents the expected cumulative reward of the agent's optional actions in the current state in the future steps. In step 5, the agent chooses actions in a way that maximizes the total future reward. The expected reward is The optimal Q function is as follows:

[0105]

[0106] Obeying the Bellman equation, the update formula is as follows:

[0107]

[0108] Among them, s′ and a′ represent the next state and the possible action of the next step respectively;

[0109] In the DQN algorithm, a Q neural network consisting of multiple hidden layers is used to gradually approximate the Q function. After the Q network receives the state at a certain moment, it calculates and finally outputs the Q value of each possible action. When training the neural network, the loss function is defined as:

[0110]

[0111] Where ω′ is the target network ω is the network weight of the estimated network Q, and the weight is updated using the stochastic gradient descent method:

[0112]

[0113] Where λ is the neural network learning rate. :

[0114] Embodiment 1

[0115] After several independent experiments, the optimal number and optimal layout positions of FBG sensor elements on the shield tunnel section under various working conditions were obtained. Taking various working conditions into consideration, the FBG sensor element layout scheme was finally fixed to 10 and the specific layout positions of the sensor elements were obtained.

[0116] Therefore, the present invention adopts the above-mentioned FBG sensor layout optimization method for shield tunnels, which solves the effectiveness problem of the optimal layout of FBG sensor elements in the shield tunnel section, provides theoretical support for the sensor layout of structural health monitoring, and further improves the reconstruction accuracy by an average of 1.73%, thereby further improving the accuracy of shield tunnel health monitoring.

[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for optimizing the layout of FBG sensors in a shield tunnel, characterized in that: The swarm intelligence algorithm is introduced, and the strategy network of a single agent is used as a population individual. The accumulated reward value in the DQN algorithm is used as the objective function of the WOA algorithm. After the DQN algorithm is iterated several times, a population iteration of the WOA is performed. Specifically, the following steps are included: S1: Execute the DQN algorithm to store the state experience generated by the interaction between the agent and the environment into the replay experience pool; In S1, the DQN algorithm includes the following steps: Step 1: Define each selected point as an independent state, define the interpolation point selection action and interpolation point change action process as a Markov decision process, and propose a control algorithm based on WOA-DQN to learn the interpolation point selection and change strategy; In step 1, the Markov decision process specifically includes, for each state, using the currently selected interpolation point sequence to represent: ； in , is the maximum number of actions per round, , is the maximum number of interpolation points, The value of is related to the angle, and the interpolation points are evenly distributed on the shield tunnel section curve. It is expressed as: ； in , from the initial state At the beginning, the interpolation point sequence of the next state is generated by controlling the change of the interpolation point through the decision-making of the agent until the optimal state is found or the constraint condition is triggered; Step 2: The agent learns the strategy, which is specifically manifested in controlling the change of the interpolation point value; Step 3: Initial state The overall error of the fitting curve is consistent with the current state Compare the overall error of the curve to get the reward value; Step 4: Experience playback, storing the experience of each state into the cache; Step 5: The action-value function represents the expected cumulative reward of the agent's optional actions in the current state in future steps; S2: Extract batch samples from the replay experience pool to train the agent policy network; S3: After several steps of training, the policy network is iteratively updated using the WOA algorithm; S4: Initialize the WOA population, where individuals are random parameters generated based on the agent strategy network parameters; S5: Each individual in the population interacts with the environment to obtain the corresponding fitness function value and select the optimal strategy network parameters; S6: According to the optimal individual network parameters, perform parameter iteration on other individuals in the population respectively; S7: Update the final optimal strategy network to the agent; S8: If the termination condition is met, the training is terminated; if the termination condition is not met, steps S2-S7 are repeated; After using the WOA-DQN algorithm for layout optimization, the optimal number and optimal layout positions of FBG sensor elements on the shield tunnel section under various working conditions were obtained, where the number of FBG sensor elements is 10.

2. The FBG sensor layout optimization method for a shield tunnel according to claim 1, characterized in that: In step 1, given the current state , the agent chooses the next action , get the next state and rewards , the state update formula is: ； in , is the state transfer function, is the agent's decision-making behavior. The Markov decision process of this model is: 。 3. The FBG sensor layout optimization method for a shield tunnel according to claim 2, characterized in that: In step 2, the strategy is an output The probability function of choosing each possible action relative to the function is expressed as: ； in is the mapping strategy, and Respectively indicate the status The state space and action space in , and the strategy for selecting actions , To select the interpolation point that the agent chooses to change in the current state, To select the agent in the current state Action after action ； ； in In order to select the position number of the interpolation point to be changed, the NoisyNet method is introduced into the DQN network, Gaussian noise is added to the neural network, and the agent automatically adjusts the noise parameters during the training process. The noise formula is defined as: ； in is the noise weight parameter, The adaptive noise of the Noisy layer, is zero-mean Gaussian noise.

4. The FBG sensor layout optimization method for a shield tunnel according to claim 3 is characterized in that: In step 3, the final reward value formula of the algorithm is: ； in is the overall fitting error of the fitting curve of the interpolation point selected in the current round, is the overall fitting error of the fitting curve of the initial interpolation points, is the reward obtained in the current step, the value range is ; Set the reward to The decay reward of the step, the reward formula is: ； in is the reward decay rate, for Step decay reward, After the current step Step reward.

5. The FBG sensor layout optimization method for a shield tunnel according to claim 4, characterized in that: In step 4, the priority weight update formula is as follows: ； in and Represent the goals of the algorithm Function and to be trained function.

6. The FBG sensor layout optimization method for a shield tunnel according to claim 5, characterized in that: In step 5, the agent chooses actions in a way that maximizes the total future reward, and the expected return , Optimal The function is as follows: ； Obeying the Bellman equation, the update formula is as follows: ； in Respectively represent the next state and the next possible action; In the DQN algorithm, a Q neural network consisting of multiple hidden layers is used to gradually approximate the Q function. After the Q network receives the state at a certain moment, it calculates and finally outputs the Q value of each possible action. When training the neural network, the loss function is defined as: ； in The target network The network weights, It is an estimated network The network weights are updated using the stochastic gradient descent method: ； in is the neural network learning rate.

Images