Structural damage detection method based on theory-assisted multi-agent

Abstract

The invention relates to a structural damage detection method based on theory-assisted multi-agent, which comprises the following steps: establishing a tutor guidance action generator based on a damage detection theoretical model of sensitivity correction; filling an experience playback pool under a plurality of virtual damage distributions; a self-adaptive experience playback mechanism is adopted in the experience playback pool for sampling, and a student action network is trained to enable the output of the student action network to approach a tutor guidance action generator; introducing real sensor data, generating a corresponding virtual damage task according to the theoretical model, and training an action and comment network at the same time; gradually reducing the participation probability of a tutor guiding an action generator according to a probability model, wherein the gradual network is dominated by a student action network; model indexes are evaluated in the training process, and an updating strategy of the model is adopted according to the indexes. Compared with the prior art, a theoretical auxiliary mechanism and a self-adaptive experience playback mechanism are introduced, so that the convergence rate is high, the calculation efficiency is high, and the robustness is high.

Description

Technical Field

[0001] The present invention relates to the field of civil engineering, and in particular to a real-time damage detection method based on multi-agent reinforcement learning for architectural heritage. Background Art

[0002] The identification of structural damage is one of the most critical research issues in the field of civil engineering. With the continuous expansion of the scale of civil engineering in modern society and the increasing complexity of building structures, the need for accurate and efficient identification of structural damage has become more urgent. In actual engineering, building structures may be damaged by various factors such as long-term natural environmental erosion (such as the impact of natural disasters such as weathering, earthquakes, and floods), material aging, and overload during use. If these damages are not discovered and evaluated in time, they may lead to serious safety accidents, causing huge casualties and property losses. Therefore, the development of a more advantageous structural damage detection method has become one of the current research focuses in the field of civil engineering, and this is also the core problem that current researchers are committed to solving, aiming to provide reliable technical support for ensuring the safety of various building structures, especially architectural heritage with important cultural value.

[0003] At present, the method based on finite element model correction to achieve structural damage identification is a more practical and scientific method. There are two general methods for finite element model correction, matrix correction method and element correction method. The matrix correction method takes the overall matrix as the correction object. In comparison, the element correction method has more physical significance in the finite element model after correction. Among the element correction methods, the sensitivity-based correction method is the most widely used in practical engineering, and the more typical one is the correction based on Newton's method. Although the theoretical research on the sensitivity-based model correction method is relatively mature, its high computational cost and easy to fall into the ill-conditioned solution are two major limitations. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a structural damage detection method based on theory-assisted multi-agent, which introduces a theory-assisted mechanism and an adaptive experience playback mechanism, so that the convergence rate is high, the calculation efficiency is high, and the robustness is strong; the present invention can detect the damage of architectural heritage and provide a theoretical basis for scientifically evaluating the safety status of architectural heritage.

[0005] The core concept logic of the present invention is:

[0006] The present invention proposes an adaptive experience replay and theory-assisted multi-agent damage detection method (TA2-MATD3) based on an improved multi-agent double-delayed deep deterministic policy gradient (MATD3) algorithm. An adaptive experience replay mechanism is adopted to ensure that new samples are stably sampled in subsequent learning, while avoiding equal probability sampling that wastes too many training steps in the initial stage on actions with low absolute feedback values. The use of a theory-assisted mechanism alleviates the problems of high learning target complexity and low learning sample richness faced by traditional MATD3 when used for damage detection. The present invention can provide a theoretical basis for real-time assessment of structural damage status and provide a theoretical basis for structural safety assessment and reinforcement and repair design of architectural heritage.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] The present invention provides a structural damage detection method based on theory-assisted multi-agent, comprising the following steps:

[0009] S1: Based on the sensitivity-corrected damage detection theoretical model, a tutor-guided action generator based on sensitivity calculation is constructed to initialize relevant parameters and provide a benchmark and initial parameter setting for the agent's actions;

[0010] S2: Generate random targets and initial damage distribution. Each agent outputs actions based on the current strategy and forms an experience set. At the same time, the actions output by the action generator guided by the instructor are included in the experience replay pool. By repeating the S2 process many times, sufficient data is accumulated.

[0011] S3: Randomly select multiple moments of experience sets from the experience replay pool to update the action network and the critic network. The action network update goal is to approximate the output of the mentor-guided action generator, and the critic network is updated based on the long-term value evaluation of the action.

[0012] S4: After completing the target number of virtual damage distribution learning tasks, introduce the time parameter into the real sensor data, repeat steps S2 to S3, and gradually guide the network output to approach the real damage situation by modifying the target damage distribution;

[0013] S5: Gradually reduce the probability of the instructor guiding the action generator to participate according to the probability model, and gradually the network is dominated by the student action network;

[0014] S6: During the training process, the indicators of the damage detection theoretical model are evaluated, and an update strategy of the damage detection theoretical model is adopted according to the indicators. Finally, the trained damage detection theoretical model is used for structural damage detection.

[0015] Furthermore, in S1, the multi-agent architecture adopted includes:

[0016] Multiple intelligent agents are set up for different areas of the structure. Each intelligent agent is responsible for detecting component damage information in an area. The intelligent agent observes the error vector of relevant modal information and the damage distribution vector as its own state matrix.

[0017] The concept of virtual time step is introduced. In each time step, the agent takes action according to the observed state. The action will affect the damage distribution, thereby changing the environmental state and obtaining incentive feedback, forming an experience set stored in the experience replay pool to provide data for network training.

[0018] Furthermore, in the multi-agent architecture, agent α i It can be observed that the modal information error vector W related to it at this time T,ti (γ T,ti ), and its associated damage distribution vector γ T,ti , note S T,ti =[W T,ti (γ T,ti ),γ T,ti ] represents the state matrix of agent i at this time, where γ T,ti W is the material damage reduction factor of the components in the area that the agent is responsible for. T,ti (γ T,ti ) selects the modal information on the overall degrees of freedom;

[0019] In the multi-agent architecture, define a T,ti represents the agent α at time t i The action taken has the same dimension as γ T,ti Consistent, a T,ti =Δγ T,ti , using the deterministic policy gradient algorithm, π T,ti (S T,ti ,θ T,ti ) is the agent α i At time t, based on the parameter θ T,ti Given a strategy, mapping the input to the observed state S T,ti , the output is pending action a T,ti ;

[0020] In a multi-agent architecture, strategy π T,ti (S T,ti ,θ T,ti ) is the deep neural network of the local performer network for fitting, θ T,ti Represents the parameters to be trained of the network, and introduces the target performer network π′ T,ti (S T,(t+1)i ,θ T ' ,ti ), which means the state S at time t is input at time t+1 T,(t+1)i , to predict the action at time t+1;

[0021] In the multi-agent architecture, a noise distribution ζ is introduced. At time t, the noise distribution of agent α i According to the strategy T,ti (S T,ti ,θ T,ti ) output action superimposed with a low-amplitude noise, a T,ti =π T,ti (S T,ti ,θ T,ti )+ζ T,ti , and there is a target action a′ T,(t+1)i =π′ T,ti (S T,(t+1)i ,θ T ' ,ti )+ζ T ' ,(t+1)i ;

[0022] In the multi-agent architecture, based on γ T,t Perform finite element modal analysis to output the calculated modal information, and solve the modal error W based on the test modal information at that moment. T,t (γ T,t ), the above process is defined as an environment that interacts with the agent. In the interactive environment, k agents jointly make an action a T,t After that, according to the state S T,ti γ T,t , γ T,t+1 =γ T,t +a T,t ;

[0023] In the multi-agent architecture, define r T,ti =lg(W T,ti (γ T,ti ) / W T,(t+1)i (γ T,(t+1)i )) is the agent α i Take action a at time t T,ti The incentives obtained after interacting with the environment. After interacting with the environment, k agents obtain the experience set Z t =(S T,ti ,a T,ti ,r T,ti ,S T,(t+1)i ), which includes the state, action and incentive at time t, and the set of states at time t+1;

[0024] In the multi-agent architecture, the value Q is defined to measure the long-term benefits of an action. Each critic network is divided into a local critic network and a target critic network, respectively denoted as Q 1T,ti (S T,ti ,a T,t ,ψ 1T,ti ) and Q2T,ti (S T,ti ,a T,t ,ψ 2T,ti ), and Q1′ T,(t+1)i (S T,(t+1)i ,a′ T,t+1 ,ψ1′ T,ti ) and Q2′ T,(t+1)i (S T,(t+1)i ,a′ T,t+1 ,ψ2′ T,ti ), represents the agent α i According to the S it observes at that moment T,ti , the action a taken by all agents together T,t , based on the parameters of the neural network at this time, make a long-term value assessment of the action, using Q1′ T,(t+1)i and Q2′ T,(t+1)i The smaller value of is shown in formula (1):

[0025] Q T,ti =r T,ti +λmin(Q1′ T,(t+1)i ,Q2′ T,(t+1)i ) (1)

[0026] Among them, λ is the discount rate, which means action a at time t T,ti The attenuation effect on subsequent actions in this scene.

[0027] Furthermore, in S1, the process of constructing a tutor-guided action generator based on sensitivity calculation includes:

[0028] The sensitivity-based calculation process is performed using equations (2) and (3):

[0029] Δγ p =(S T (γ p )S(γ p )) -1 S T (γ p )W(γ p ) (2)

[0030] Where S(γ p ) represents the p-1th iteration step, W(γ p ) is about γ p-1 The sensitivity matrix of

[0031] The current damage sensitivity matrix S(γ p ), and the difference method shown in formula (2) is used to solve it:

[0032]

[0033] The matrix S(γ p ) is the number of parameters involved in nl components;

[0034] The process of initializing relevant parameters includes: initializing parameters θ 0,0i ,ψ 10,0i ,ψ 20,0i ,θ0′ ,0i , ψ1′ 0,0i , ψ2′ 0,0i and θ.

[0035] Furthermore, in S2, the specific process includes:

[0036] Generate several sets of random target damage distributions and initial damage distribution At this time, T represents different goals Each T is divided into different scenes with different

[0037] At time t of a certain scene at time T, each agent outputs action a T,ti =π T,ti (S T,ti ,θ T,ti ), and form the experience set Z t =(S T,ti ,a T,ti ,r T,ti ,S T,(t+1)i );

[0038] At the same time, according to the instructor's guidance action generator, output action a T,ti,P , a T,ti,P Incorporating the experience set, we have Z t ″=(S T,ti ,a T,ti ,r T,ti ,S T,(t+1)i ,a T,ti,P ), and store it in the experience replay pool, repeat step S2 to a preset number of times, so as to enrich the capacity of the experience replay pool Θ.

[0039] Furthermore, in S3, the specific process includes:

[0040] After completing the process of S2 once, randomly select the experience set Z″ of M moments from the experience replay pool Θ j =(S T,ji ,a T,ji ,r T,ji ,S T,(j+1)i ,a T,ji,P );

[0041] According to equations (4) and (5), the action network and critic network are updated. At this time, the update goal of the action network is to train a student action network so that its actions are close to the instructor-guided action generator:

[0042]

[0043]

[0044] It represents the gradient attenuation coefficient of the teacher network, that is, when the time exceeds t0, the effect of the teacher is slowly weakened over time.

[0045] Furthermore, in S4, the specific process includes:

[0046] Complete several virtual damage distributions After the learning task, a T is introduced into the real sensor data, and each T is divided into different scenes with different Repeat steps S2 to S3 to identify the damage distribution at this time

[0047] Then, by modifying the target damage distribution, the target damage distribution Modified to Complete the training again. At this time, the output is close to the student behavior generator θ of Newton's method. pre-train , and a critic network that can evaluate its output.

[0048] Furthermore, in S5, the specific process includes:

[0049] The probability function of formula (6) is used as the selection function of the action generator, and the student behavior action generator is gradually used to replace the tutor guidance action generator. Modified to If the teacher-guided action is not used at a certain moment, the experience set is Z j =(S T,ji ,a T,ji ,r T,ji ,S T,(j+1)i ), if the teacher guides the action, the experience set is Z″ j =(S T,ji ,a T,ji ,r T,ji ,S T,(j+1)i ,a T,ji,P ), the network is updated using equations (5), (7), and (8) for the experience set Zj sampled at M moments, Z″ j Use equations (4), (5), and (7) to update the network:

[0050]

[0051]

[0052]

[0053] Where P min represents the minimum probability of the appearance of the tutor-guided action generator, epoch represents the number of training rounds, χ is the control coefficient, which is used to control the decay rate of the probability of the appearance of the tutor-guided action generator, and τ is the soft update coefficient.

[0054] Furthermore, in S6, the specific process includes:

[0055] In the learning process, an evaluation module is introduced, that is, a number of moments are randomly selected, and real data and virtual data are required to be sampled respectively, and one scene of the task is performed respectively;

[0056] The current local performer network π is determined by using four indicators: the mean value of the incentives of each agent at each moment, the convergence rate within L actions, the convergence rate of the damage distribution of L actions, and the error value between the identified damage distribution and the preset damage distribution. T,ti (S T,ti ,θ T,ti ) Output the quality of the action, according to Q 1T,ti and Q 2T,ti The convergence of determines the quality of the local critic network;

[0057] If the damage recognition success rate decreases by less than 15% during the evaluation process, the current action network parameter θ T,ti To update θ pre-train If the success rate decreases by more than 15%, use formula (9) to update the current network θ T,ti :

[0058] θ T,ti =τθ T,ti +(1-τ)θ pre-train (9)

[0059] When the quality of the actions output by the performer network and the value of the critic network output meet the established requirements during the evaluation, the training is completed.

[0060] Furthermore, in S1 to S3, the experience replay adopts the following adaptive experience replay mechanism:

[0061] Introduce the probability array P, which is a one-dimensional vector, the dimension is consistent with the number of samples in the experience replay pool Θ, and assume that the capacity of the experience replay pool is N rb , which is a variable value before reaching the maximum value, and the selected batch size is N bs is a fixed value;

[0062] The sampling is divided into two stages:

[0063] In the first stage, the network just started to enter the training phase and met the N bs Sampling N pick After times, the probability that each sample is taken at least once is greater than 95%, N bs , N pick and N rb The approximate estimated relationship between the three is shown in the following formula:

[0064]

[0065] In the second stage, equation (10) is used for calculation.

[0066]

[0067] in represents the average reward of multiple agents at time t in the experience set, It means to normalize the average reward, set the control coefficient 0<κ1<1 to control the proportion of negative incentives selected, Tanh represents the hyperbolic tangent function, P T,ti represents the probability of selecting other experience sets from the experience pool except the new experience set, κ2∈N + Represents the rate at which samples become equally probable.

[0068] Compared with the prior art, the present invention has the following beneficial effects:

[0069] 1. The detection method of the present invention introduces a theoretical auxiliary mechanism to alleviate the main problems of traditional reinforcement learning for damage detection: high complexity of learning objectives and low richness of learning samples;

[0070] 2. The detection method of the present invention introduces an adaptive experience replay mechanism, which alleviates the problem of poor experience replay efficiency faced by traditional reinforcement learning when used for damage detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 This is a schematic diagram of the data flow during the training phase of the present invention;

[0072] Figure 2 It is a schematic diagram of the algorithm flow adopted by the present invention. DETAILED DESCRIPTION

[0073] In general, the present invention discloses a damage detection method based on theory-assisted multi-agent. The calculation method includes the following steps: S1: Establishing a tutor-guided action generator based on a sensitivity-corrected damage detection theoretical model; S2: Filling an experience replay pool under several virtual damage distributions; S3: Using an adaptive experience replay mechanism to sample from the experience replay pool, training the student action network so that its output is close to the tutor-guided action generator;

[0074] S4: Introduce real sensor data, generate corresponding virtual damage tasks according to the theoretical model, and train the action and comment network at the same time; S5: According to the probability model, gradually reduce the participation probability of the instructor-guided action generator, and gradually the network is dominated by the student action network; S6: Evaluate the model indicators during the training process, and adopt the model update strategy according to the indicators. The present invention introduces a theoretical auxiliary mechanism and an adaptive experience playback mechanism, so it has a high convergence rate, high computational efficiency, and strong robustness; the present invention can detect the damage of architectural heritage and provide a theoretical basis for scientifically evaluating the safety status of architectural heritage.

[0075] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. The features such as the structural module name, control mode, algorithm, process or composition ratio not clearly described in this technical solution are regarded as common technical features disclosed in the prior art.

[0076] Example 1

[0077] This embodiment discloses a structural damage detection method based on theory-assisted multi-agent, comprising the following steps:

[0078] For each sensor, the actual time T at which the sensor collects data, and the sensor collection time is ΔT, the modal error vector within this time can be recorded as W T,0 (γ T,0 ). In the virtual time step t, it is assumed that there are k agents, each of which is responsible for component damage detection in a region of the structure. Then agent α i It can be observed that the modal information error vector W associated with it at this time T,ti (γ T,ti ), and its associated damage distribution vector γ T,ti , note S T,ti =[W T,ti (γ T,ti ),γ T,ti ] represents the state matrix of agent i at this time. T,ti W is the material damage reduction factor of the components in the area that the agent is responsible for. T,ti (γ T,ti ) selects the modal information on the overall degrees of freedom.

[0079] DefinitionT,ti represents the agent α at time t i The action taken has the same dimension as γ T,ti Consistent, can be understood as a T,ti =Δγ T,ti , represents the agent α i The array Δγ that adjusts the damage of the component it is responsible for observing T,ti . Using the deterministic policy gradient algorithm, π T,ti (S T,ti ,θ T,ti ) is defined as agent α i At time t, based on the parameter θ T,ti Given a strategy, mapping the input to the observed state S T,ti , the output is pending action a T,ti In MATD3, strategy π T,ti (S T,ti ,θ T,ti ) can be fitted using a deep neural network called a local performer network, θ T,ti Represents the parameters to be trained of the network. Introducing the target performer network π′ T,ti (S T,(t+1)i ,θ T ' ,ti ), which means the state S at time t is input at time t+1 T,(t+1)i , to predict the action at time t+1.

[0080] Introducing the noise distribution ζ, at time t, for the agent α i According to the strategy T,ti (S T,ti ,θ T,ti ) output action superimposed with a low-amplitude noise, then there is a T,ti =π T,ti (S T,ti ,θ T,ti )+ζ T,ti , and there is a target action a′ T,(t+1)i =π′ T,ti (S T,(t+1)i ,θ T ' ,ti )+ζ T ' ,(t+1)i .

[0081] Based on γ T,t Perform finite element modal analysis to output the calculated modal information, and then solve the modal error W based on the test modal information at that moment T,t (γ T,t ) is defined as an environment that interacts with an agent. For this environment, k agents jointly make an action a T,tAfter that, according to the state S T,ti γ T,t , then γ T,t+1 =γ T,t +a T,t . T,t+1 Enter the environment and solve W at this time T,t+1 (γ T,t+1 ), and according to W T,t+1 (γ T,t+1 ) and γ T,t+1 , determine the state S at time t+1 T,(t+1)i =[W T,(t+1)i (γ T,(t+1)i ),γ T,(t+1)i ].

[0082] Definition T,ti =lg(W T,ti (γ T,ti ) / W T,(t+1)i (γ T,(t+1)i )) represents agent α i Take action a at time t T,ti Interact with the environment to change the global state space from S T,ti Change to S T,(t+1)i After that, the environment gives back the incentive to the agent. Thus, after interacting with the environment, the k agents obtain an experience set Z t =(S T,ti ,a T,ti ,r T,ti ,S T,(t+1)i ), which includes the state, action and incentive at time t, and the state at time t+1. The experience set Z is stored in the experience replay pool Θ, such as Figure 1 .

[0083] The value Q is defined to measure the long-term benefit of an action. To avoid overfitting the Q value, a pair of critic networks is introduced. For each critic network, it is divided into a local critic network and a target critic network, denoted as Q 1T,ti (S T,ti ,a T,t ,ψ 1T,ti ), Q 2T,ti (S T,ti ,a T,t ,ψ 2T,ti ), Q1′ T,(t+1)i (S T,(t+1)i ,a′ T,t+1 ,ψ1′ T,ti ), Q2′ T,(t+1)i (S T,(t+1)i ,a′ T,t+1 ,ψ1′ T,ti ). It represents the agent α iAccording to the S it observes at that moment T,ti , the action a made by all agents together T,t , based on the parameters of the neural network at this time, make a long-term value assessment of the action. At this time, use Q1′ T,(t+1)i and Q2′ T,(t+1)i The smaller value of is obtained by using the Bellman equation as shown in formula (1).

[0084] Q T,ti =r T,ti +λmin(Q1′ T,(t+1)i ,Q2′ T,(t+1)i ) (1)

[0085] Among them, λ is the discount rate, which means action a at time t T,ti The attenuation effect on subsequent actions in this scene.

[0086] The steps of the structural damage detection method based on theory-assisted multi-agent are as follows: Figure 2 As shown:

[0087] Step 1, the sensitivity-based calculation process of equations (2) and (3) is adopted, and it is named as the tutor-guided action generator. At this time, the original performer model can be distinguished as the student action performer model.

[0088] Δγ p =(S T (γ p )S(γ p )) -1 S T (γ p )W(γ p ) (2)

[0089] Where S(γ p ) represents the p-1th iteration step, W(γ p )About γ p-1 In formula (1), the most important thing is to calculate the current damage sensitivity matrix S(γ p ). Generally, the difference method shown in formula (2) is used to solve it.

[0090]

[0091] The matrix S(γ p ) is the number of parameters involved in nl components. If each component contains two parameters to be modified, density and elastic modulus, then the matrix S(γ p ) has 2nl columns.

[0092] Initialization parameter θ 0,0i ,ψ 10,0i ,ψ20,0i ,θ0′ ,0i , ψ1′ 0,0i , ψ2′ 0,0i and θ.

[0093] Step 2: Generate several groups of random target damage distributions and initial damage distribution At this time, T represents different goals Each T is divided into different scenes with different At time t of a certain scene at time T, each agent outputs action a T,ti =π T,ti (S T,ti ,θ T,ti ), and form the experience set Z t =(S T,ti ,a T,ti ,r T,ti ,S T,(t+1)i ). At the same time, according to the instructor's guidance action generator, the action a can be output T,ti,P . T,ti,P Incorporating the experience set, we have Z″ t =(S T,ti ,a T,ti ,r T,ti ,S T,(t+1)i ,a T,ti,P ), and store it in the experience replay pool. After repeating Step 2 enough times, the experience replay pool Θ has a certain capacity.

[0094] Step 3: After completing Step 2, randomly select an experience set Z″ of M moments from the experience replay pool Θ j =(S T,ji ,a T,ji ,r T,ji ,S T,(j+1)i ,a T,ji,P ). Update the action network and critic network according to equations (4) and (5). At this time, the update target of the action network is no longer to maximize the action output of Q value, but to train a student action network so that its action is close to the instructor-guided action generator.

[0095]

[0096]

[0097] It represents the gradient attenuation coefficient of the teacher network, that is, when the time exceeds t0, the effect of the teacher is slowly weakened over time.

[0098] Step 4: Complete several virtual damage distributions After the learning task, a T is introduced into the real sensor data, and each T has different Repeat Step 2-Step 3 to identify the damage distribution at this time The above process is repeated several times, and only the target damage distribution in Step 2 needs to be Modified to That is, from the random target damage distribution, a target damage distribution that randomly oscillates near the real damage distribution is modified, and the output is close to the student behavior action generator θ of the Newton method. pre-train , and a critic network that can evaluate its output.

[0099] Step 5: Based on this network, the probability function of formula (6) is used as the selection function of the action generator, and the student behavior action generator is gradually used to replace the tutor guidance action generator. Modified to If the teacher-guided action is not used at a certain moment, the experience set is Z j =(S T,ji ,a T,ji ,r T,ji ,S T,(j+1)i ), if the teacher guides the action, the experience set is Z″ j =(S T,ji ,a T,ji ,r T,ji ,S T,(j+1)i ,a T,ji,P ). For the experience set Z sampled at this time M moments j Use equations (5), (7), and (8) to update the network, Z″ j Use equations (4), (5), and (7) to update the network.

[0100]

[0101]

[0102]

[0103] Where P min represents the minimum probability of the appearance of the tutor-guided action generator, epoch represents the number of training rounds, χ is the control coefficient, which is used to control the decay rate of the probability of the appearance of the tutor-guided action generator, and τ is the soft update coefficient.

[0104] Step 6, after performing the above process several times, the training of the real data and virtual damage data of the previous T moments can be completed. In the learning process, an evaluation module is introduced, that is, several moments are randomly selected, and real data and virtual data are required to be sampled separately, and one scene task is performed separately. In addition to the two conventional indicators of the incentive mean of each moment and each agent, and the ratio of convergence within L actions, the ratio of convergence of the damage distribution of L actions can also be defined under the virtual damage data task, as well as the error value between the identified damage distribution and the preset damage distribution. Use the above four indicators to determine the current local performer network π T,ti (S T,ti ,θ T,ti ) Output the quality of the action. 1T,ti and Q 2T,ti In addition, if the damage recognition success rate decreases by less than 15% during the evaluation process, the current action network parameter θ T,ti To update θ pre-train If the success rate decreases by more than 15%, use formula (9) to update the current network θ T,ti When the quality of the actions output by the performer network and the value of the critic network output meet the established requirements during the evaluation, the training is completed.

[0105] θ T,ti =τθ T,ti +(1-τ)θ pre-train (9)

[0106] Furthermore, the experience replay adopts the following adaptive experience replay mechanism.

[0107] Introduce the probability array P, which is a one-dimensional vector, and its dimension is consistent with the number of samples in the experience replay pool Θ. Suppose the capacity of the experience replay pool is N rb , which is a variable value before reaching the maximum value, and the selected batch size is N bs is a fixed value. The sampling is divided into two stages. In the first stage, the network just starts to enter the training phase and meets the requirements of N bs Sampling N pick After 100 times, the probability that each sample is taken at least once is greater than 95%. To meet the above requirements, N bs , N pick and N rb The approximate estimated relationship between the three is shown in the following formula: After a large number of numerical tests, the present invention recommends N pick Take 100, and N rb =2N bs As a dividing line. At this stage, an equal probability mode can be used for sampling to allow the agent to explore the entire space as much as possible.

[0108]

[0109] In the second stage, the number of samples in the experience replay pool increases. When equal probability sampling is still used, the expectation of new samples being sampled gradually decreases. Therefore, at this time, the last element of P can be set to 1, which means that the latest experience set will definitely be selected in the next sampling. On the other hand, in this stage, the remaining elements in P are calculated using formula (8).

[0110]

[0111] in Represents the average reward of multiple agents at time t in the experience set. It means to normalize the average reward and set the control coefficient 0<κ1<1 to control the proportion of negative incentives selected. Tanh means the hyperbolic tangent function, P T,ti represents the probability of selecting other experience sets from the experience pool except the new experience set, κ2∈N + Represents the rate at which samples become equally probable.

[0112] In order to evaluate the accuracy of the present invention for structural damage detection, damage detection was carried out on a residential building. A residual neural network was constructed using a similar structure for the action network and the critic network. The only difference between the action network and the critic network is that no tanh and action coefficient scaling are required for the output layer of the critic network. There are 5 layers of residual blocks for the action network and 8 layers of residual blocks for the critic network. Except for the residual blocks near the input and output, which have 256 hidden layer neurons, the number of hidden layer neurons in the residual blocks is 128. The number of agents is 3. The exploration noise is 3×10 -4 ~1×10 -3 , using exponential sine decay noise. The L2 regularization parameter decay coefficient is 1×10 -3 The discount rate is 0.95, the soft update rate is 0.9, and the action network learning rate is 3×10 -4 , the critic network learning rate is 1×10 -4 , the gradient clipping threshold is 100. The LN (LayerNormalization) layer is used for regularization in the feature dimension. A warm up mechanism is introduced to exponentially decay the learning rate. A gradient clipping mechanism is introduced.

[0113] In order to compare the performance of the models, the following models will be compared: (1) damage localization based on Newton's method, hereinafter referred to as Newton's method; (2) damage localization based on Newton's method and modified by L-curve method, hereinafter referred to as L-curve; (3) the model after training of the present invention; (4) regression model based on Model-Agnostic Meta-Learning (MAML).

[0114] For 1500 virtual damage tasks, the following indicators are used to test MSE (Mean Squared Error), MSLE (Mean Squared Log Error), MDAE (Median Absolute Error), MAE (Mean Absolute Error), Explained Variance Score (EVS), and R2 (R-squared coefficient). The calculation results are shown in Table 1, which shows that the damage detection method proposed in the present invention has the best comprehensive performance.

[0115]

[0116] Where n is the sample size, D is the pre-set damage, It is to predict damage.

[0117] Table 1 Evaluation indicators for measuring calculation errors

[0118]

[0119] Note: The smaller the MSE, MSLE, MDAE, and MAE, the better the results. 2 The closer to 1, the better the result.

[0120] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0121] The above description of the embodiments is to facilitate the understanding and use of the invention by those skilled in the art. It is obvious that those skilled in the art can easily make various modifications to these embodiments and apply the general principles described herein to other embodiments without creative work. Therefore, the present invention is not limited to the above embodiments, and improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the present invention should be within the scope of protection of the present invention.

Claims

1. A structural damage detection method based on theory-assisted multi-agent, characterized in that: The following steps are involved: S1: Based on the sensitivity-corrected damage detection theoretical model, a tutor-guided action generator based on sensitivity calculation is constructed to initialize relevant parameters and provide a benchmark and initial parameter setting for the agent's actions; S2: Generate random targets and initial damage distribution. Each agent outputs actions based on the current strategy and forms an experience set. At the same time, the actions output by the action generator guided by the instructor are included in the experience replay pool. By repeating the S2 process many times, sufficient data is accumulated. S3: Randomly select multiple moments of experience sets from the experience replay pool to update the action network and the critic network. The action network update goal is to approximate the output of the mentor-guided action generator, and the critic network is updated based on the long-term value evaluation of the action. S4: After completing the target number of virtual damage distribution learning tasks, introduce the time parameter into the real sensor data, repeat steps S2 to S3, and gradually guide the network output to approach the real damage situation by modifying the target damage distribution; S5: Gradually reduce the probability of the instructor guiding the action generator to participate according to the probability model, and gradually the network is dominated by the student action network; S6: During the training process, the indicators of the damage detection theoretical model are evaluated, and an updating strategy of the damage detection theoretical model is adopted according to the indicators. Finally, the trained damage detection theoretical model is used for structural damage detection.

2. According to claim 1, a theory-assisted multi-agent based structural damage detection method is characterized in that: In S1, the multi-agent architecture used includes: Multiple intelligent agents are set up for different areas of the structure. Each intelligent agent is responsible for detecting component damage information in an area. The intelligent agent observes the error vector of relevant modal information and the damage distribution vector as its own state matrix. The concept of virtual time step is introduced. In each time step, the agent takes action according to the observed state. The action will affect the damage distribution, thereby changing the environmental state and obtaining incentive feedback, forming an experience set stored in the experience replay pool to provide data for network training.

3. The method for structural damage detection based on theory-assisted multi-agent according to claim 2, characterized in that: In the multi-agent architecture, agent α i It can be observed that the modal information error vector W related to it at this time T,ti (γ T,ti ), and its associated damage distribution vector γ T,ti , note S T,ti =[W T,ti (γ T,ti ),γ T,ti ] represents the state matrix of agent i at this time, where γ T,ti W is the material damage reduction factor of the components in the area that the agent is responsible for. T,ti (γ T,ti ) selects the modal information on the overall degrees of freedom; In the multi-agent architecture, define a T,ti represents the agent α at time t i The action taken has the same dimension as γ T,ti Consistent, a T,ti =Δγ T,ti ; Using the deterministic policy gradient algorithm, π T,ti (S T,ti ,θ T,ti ) is the agent α i At time t, based on the parameter θ T,ti Given a strategy, mapping the input to the observed state S T,ti , the output is pending action a T,ti ; In a multi-agent architecture, strategy π T,ti (S T,ti ,θ T,ti ) is the deep neural network of the local performer network for fitting, θ T,ti Represents the parameters to be trained of the network, and introduces the target performer network π′ T,ti (S T,(t+1)i ,θ T ' ,ti ), which means that the state S at time t is input at time t+1 T,(t+1)i , to predict the action at time t+1; In the multi-agent architecture, a noise distribution ζ is introduced. At time t, the noise distribution of agent α i According to the strategy T,ti (S T,ti ,θ T,ti ) output action superimposed with a low-amplitude noise, a T,ti =π T,ti (S T,ti ,θ T,ti )+ζ T,ti , and there is a target action a′ T,(t+1)i =π′ T,ti (S T,(t+1)i ,θ T ' ,ti )+ζ T ' ,(t+1)i ; In the multi-agent architecture, based on γ T,t Perform finite element modal analysis to output the calculated modal information, and solve the modal error W based on the test modal information at that moment. T,t (γ T,t ), defining the above process as an environment that interacts with the agent; In the interactive environment, k agents jointly perform an action a T,t After that, according to the state S T,ti γ T,t , we get γ T,t+1 =γ T,t +a T,t ; In the multi-agent architecture, define r T,ti =lg(W T,ti (γ T,ti ) / W T,(t+1)i (γ T,(t+1)i )) is the agent α i Take action a at time t T,ti The incentives gained from interacting with the environment; After k agents interact with the environment, they obtain the experience set Z t =(S T,ti ,a T,ti ,r T,ti ,S T,(t+1)i ), which includes the state, action and incentive at time t, and the set of states at time t+1; In the multi-agent architecture, the value Q is defined to measure the long-term benefits of an action. Each critic network is divided into a local critic network and a target critic network, respectively denoted as Q 1T,ti (S T,ti ,a T,t ,ψ 1T,ti ) and Q 2T,ti (S T,ti ,a T,t ,ψ 2T,ti ), and Q1′ T,(t+1)i (S T,(t+1)i ,a′ T,t+1 ,ψ1′ T,ti ) and Q2′ T,(t+1)i (S T,(t+1)i ,a′ T,t+1 ,ψ2′ T,ti ), represents the agent α i According to the S it observes at that moment T,ti , the action a taken by all agents together T,t , based on the parameters of the neural network at this time, make a long-term value assessment of the action, using Q1′ T,(t+1)i and Q2′ T,(t+1)i The smaller value of is shown in formula (1): Q T,ti =r T,ti +λmin(Q1′ T,(t+1)i ,Q2′ T,(t+1)i ) (1) Among them, λ is the discount rate, which means action a at time t T,ti The attenuation effect on subsequent actions in this scene.

4. The method for structural damage detection based on theory-assisted multi-agent according to claim 3 is characterized in that: In S1, the process of building a tutor-guided action generator based on sensitivity calculation includes: The sensitivity-based calculation process is performed using equations (2) and (3): Dg p =(S T (c p )S(γ p )) -1 S T (c p )W(γ p ) (2) Where S(γ p ) represents the p-1th iteration step, W(γ p ) is about γ p-1 The sensitivity matrix of The current damage sensitivity matrix S(γ p ), and the difference method shown in formula (2) is used to solve it: The matrix S(γ p ) is the number of parameters involved in nl components; The process of initializing relevant parameters includes: initializing parameters θ 0,0i ,ψ 10,0i ,ψ 20,0i ,θ0′ ,0i , ψ1′ 0,0i , ψ2′ 0,0i and θ.

5. The method for structural damage detection based on theory-assisted multi-agent according to claim 4, characterized in that: In S2, the specific process includes: Generate several sets of random target damage distributions and initial damage distribution At this time, T represents different goals Each T is divided into different scenes with different At time t of a certain scene at time T, each agent outputs action a T,ti =π T,ti (S T,ti ,θ T,ti ), and form the experience set Z t =(S T,ti ,a T,ti ,r T,ti ,S T,(t+1)i ); At the same time, according to the instructor's guidance action generator, output action a T,ti,P , a T,ti,P Incorporating the experience set, we have Z t ″=(S T,ti ,a T,ti ,r T,ti ,S T,(t+1)i ,a T,ti,P ), and store it in the experience replay pool, repeat step S2 to a preset number of times, so as to enrich the capacity of the experience replay pool Θ.

6. The method for structural damage detection based on theory-assisted multi-agent according to claim 5, characterized in that: In S3, the specific process includes: After completing the process of S2 once, randomly select the experience set Z of M moments from the experience replay pool Θ j ″=(S T,ji ,a T,ji ,r T,ji ,S T,(j+1)i ,a T,ji,P ); According to equations (4) and (5), the action network and critic network are updated. At this time, the update goal of the action network is to train a student action network so that its actions are close to the instructor-guided action generator: It represents the gradient attenuation coefficient of the teacher network, that is, when the time exceeds t0, the effect of the teacher is slowly weakened over time.

7. The method for structural damage detection based on theory-assisted multi-agent according to claim 6, characterized in that: In S4, the specific process includes: Complete several virtual damage distributions After the learning task, a T is introduced into the real sensor data, and each T is divided into different scenes with different Repeat steps S2 to S3 to identify the damage distribution at this time Then, by modifying the target damage distribution, the target damage distribution Modified to Complete one more training session; At this time, the output is close to the student behavior action generator θ of Newton's method pre-train , and a critic network that can evaluate its output.

8. The method for structural damage detection based on theory-assisted multi-agent according to claim 7, characterized in that: In S5, the specific process includes: The probability function of formula (6) is used as the selection function of the action generator, and the student behavior action generator is gradually used to replace the tutor guidance action generator. Modified to If the teacher-guided action is not used at a certain moment, the experience set is Z j =(S T,ji ,a T,ji ,r T,ji ,S T,(j+1)i ), if the teacher guides the action, the experience set is Z″ j =(S T,ji ,a T,ji ,r T,ji ,S T,(j+1)i ,a T,ji,P ), for the experience set Z sampled at this time j Use equations (5), (7), and (8) to update the network, Z j ″Use equations (4), (5), and (7) to update the network: Where P min represents the minimum probability of the appearance of the tutor-guided action generator, epoch represents the number of training rounds, χ is the control coefficient, which is used to control the decay rate of the probability of the appearance of the tutor-guided action generator, and τ is the soft update coefficient.

9. The method for structural damage detection based on theory-assisted multi-agent according to claim 8, characterized in that: In S6, the specific process includes: In the learning process, an evaluation module is introduced, that is, a number of moments are randomly selected, and real data and virtual data are required to be sampled respectively, and one scene of the task is performed respectively; The current local performer network π is determined by using four indicators: the mean value of the incentives of each agent at each moment, the convergence rate within L actions, the convergence rate of the damage distribution of L actions, and the error value between the identified damage distribution and the preset damage distribution. T,ti (S T,ti ,θ T,ti ) Output the quality of the action, according to Q 1T,ti and Q 2T,ti The convergence of determines the quality of the local critic network; If the damage recognition success rate decreases by less than 15% during the evaluation process, the current action network parameter θ T,ti To update θ pre-train If the success rate decreases by more than 15%, use formula (9) to update the current network θ T,ti : i T,ti =tθ T,ti +(1-τ)θ pre-train (9) When the quality of the actions output by the performer network and the value of the critic network output meet the established requirements during the evaluation, the training is completed.

10. The method for structural damage detection based on theory-assisted multi-agent according to claim 1, characterized in that: In S1 to S3, experience replay uses the following adaptive experience replay mechanism: Introduce the probability array P, which is a one-dimensional vector, the dimension is consistent with the number of samples in the experience replay pool Θ, and assume that the capacity of the experience replay pool is N rb , which is a variable value before reaching the maximum value, and the selected batch size is N bs is a fixed value; The sampling is divided into two stages: In the first stage, the network just started to enter the training phase and met the N bs Sampling N pick After times, the probability that each sample is taken at least once is greater than 95%, N bs , N pick and N rb The approximate estimated relationship between the three is shown in the following formula: In the second stage, formula (10) is used for calculation: in represents the average reward of multiple agents at time t in the experience set, It means to normalize the average reward, set the control coefficient 0<κ1<1 to control the proportion of negative incentives selected, Tanh represents the hyperbolic tangent function, P T,ti represents the probability of selecting other experience sets from the experience pool except the new experience set, κ2∈N + Represents the rate at which samples become equally probable.

Images