A structural damage detection method based on theory-assisted multi-agent

Abstract

The present invention relates to a theory-assisted multi-agent structural damage detection method, comprising the following steps: establishing a mentor-guided action generator based on a sensitivity-corrected damage detection theoretical model; populating an experience replay pool under several virtual damage distributions; sampling from the experience replay pool using an adaptive experience replay mechanism to train a student action network so that its output approximates the mentor-guided action generator; introducing real sensor data and generating corresponding virtual damage tasks based on the theoretical model, while simultaneously training the action and comment networks; gradually reducing the participation probability of the mentor-guided action generator based on a probabilistic model, so that the network is gradually dominated by the student action network; and evaluating model indicators during training and adopting a model update strategy based on the indicators. Compared with existing technologies, the present invention introduces a theory-assisted mechanism and an adaptive experience replay mechanism, resulting in a high convergence rate, high computational efficiency, and strong robustness.

Description

Technical Field

[0001] The present invention relates to the field of civil engineering, and in particular to a real-time damage detection method based on multi-agent reinforcement learning for architectural heritage. Background Art

[0002] The identification of structural damage is one of the most critical research issues in the field of civil engineering. With the continuous expansion of the scale of civil engineering in modern society and the increasing complexity of building structures, the need for accurate and efficient identification of structural damage has become increasingly urgent. In actual engineering, building structures may be damaged by various factors such as long-term natural environmental erosion (such as the impact of natural disasters such as weathering, earthquakes, and floods), material aging, and overload during use. If these damages are not discovered and evaluated in a timely manner, they may lead to serious safety accidents, causing huge casualties and property losses. Therefore, the development of a more advantageous structural damage detection method has become one of the current research focuses in the field of civil engineering, and this is also the core problem that current researchers are committed to solving, aiming to provide reliable technical support for ensuring the safety of various building structures, especially architectural heritage with important cultural value.

[0003] At present, the method based on finite element model modification to achieve structural damage identification is a relatively practical and scientific method. There are two general methods for finite element model modification: matrix modification method and element modification method. The matrix modification method uses the overall matrix as the modification object. In comparison, the element modification method has more physical significance for the modified finite element model. Among the element modification methods, the sensitivity-based modification method is the most widely used in actual engineering, and the more typical one is the modification based on Newton's method. Although the theoretical research on the sensitivity-based model modification method is relatively mature, its high computational cost and the susceptibility to ill-conditioned solutions are two major limitations. Summary of the Invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned existing technologies and provide a structural damage detection method based on theory-assisted multi-agent, which introduces a theory-assisted mechanism and an adaptive experience replay mechanism, so that the convergence rate is high, the computational efficiency is high, and the robustness is strong; the present invention can detect damage to architectural heritage and provide a theoretical basis for scientifically evaluating the safety status of architectural heritage.

[0005] The core concept logic of the present invention is:

[0006] The present invention proposes an adaptive experience replay, theory-assisted multi-agent damage detection method (TA2-MATD3) based on an improved multi-agent double-delay deep deterministic policy gradient (MATD3) algorithm. The adaptive experience replay mechanism is adopted to ensure that new samples are stably sampled in subsequent learning, while avoiding equal probability sampling that wastes too many training steps in the early stage on actions with low absolute feedback values. The use of a theory-assisted mechanism alleviates the problems of high learning target complexity and low learning sample richness faced by traditional MATD3 when used for damage detection. The present invention can provide a theoretical basis for real-time assessment of structural damage status and provide a theoretical basis for structural safety assessment and reinforcement and repair design of architectural heritage.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] The present invention provides a structural damage detection method based on theory-assisted multi-agent, comprising the following steps:

[0009] S1: Based on the sensitivity-corrected damage detection theoretical model, a tutor-guided action generator based on sensitivity calculation is constructed to initialize relevant parameters and provide a benchmark and initial parameter setting for the agent's actions;

[0010] S2: Generates random targets and initial damage distributions. Each agent outputs actions based on the current strategy and forms an experience set. At the same time, the actions output by the mentor-guided action generator are incorporated into the experience replay pool. By repeating the S2 process multiple times, sufficient data is accumulated.

[0011] S3: Randomly select multiple experience sets at different moments from the experience replay pool and update the action network and critic network. The action network is updated to approximate the output of the mentor-guided action generator, and the critic network is updated based on the long-term value evaluation of the action.

[0012] S4: After completing the target number of virtual damage distribution learning tasks, introduce the time parameter into the real sensor data and repeat steps S2 to S3. By modifying the target damage distribution, the network output is gradually guided to approach the real damage situation.

[0013] S5: Gradually reduce the probability of the instructor guiding the action generator to participate according to the probability model, and gradually the network is dominated by the student action network;

[0014] S6: During the training process, the damage detection theoretical model indicators are evaluated and an update strategy for the damage detection theoretical model is adopted based on the indicators. Finally, the trained damage detection theoretical model is used for structural damage detection.

[0015] Furthermore, in S1, the multi-agent architecture adopted includes:

[0016] Multiple intelligent agents are set up for different areas of the structure. Each intelligent agent is responsible for detecting component damage information in a certain area. The intelligent agent observes the relevant modal information error vector and damage distribution vector as its own state matrix.

[0017] The concept of virtual time step is introduced. In each time step, the agent takes action according to the observed state. The action affects the damage distribution, thereby changing the environmental state and obtaining incentive feedback, forming an experience set stored in the experience replay pool to provide data for network training.

[0018] Furthermore, in the multi-agent architecture, agent α i It is possible to observe the modal information error vector W associated with it at this time T,ti (γ T,ti ), and its associated damage distribution vector γ T,ti , note S T,ti =[W T,ti (γ T,ti ),γ T,ti ] represents the state matrix of agent i at this time, where γ T,ti W is the material damage reduction factor of the components in the area that the agent is responsible for, T,ti (γ T,ti ) selects the modal information on the overall degree of freedom;

[0019] In the multi-agent architecture, define a T,ti Represents agent α at time t i The action taken has the same dimension as γ T,ti Consistent, a T,ti =Δγ T,ti , using the deterministic policy gradient algorithm, π T,ti (S T,ti ,θ T,ti ) is the agent α i At time t, based on the parameter θ T,ti Given a strategy, mapping the input to the observed state S T,ti , the output is pending action a T,ti ;

[0020] In a multi-agent architecture, strategy π T,ti (S T,ti ,θ T,ti ) is the deep neural network fitting of the local performer network, θ T,ti Represents the parameters to be trained of the network, and introduces the target performer network π′ T,ti (S T,(t+1)i ,θ T ' ,ti ), which means the state S at time t is input at time t+1 T,(t+1)i , to predict the action at time t+1;

[0021] In the multi-agent architecture, the noise distribution ζ is introduced, and at time t, the agent α i According to the strategy π T,ti (S T,ti ,θ T,ti ) output action superimposed with a low-amplitude noise, a T,ti =π T,ti (S T,ti ,θ T,ti )+ζ T,ti , and there is a target action a′ T,(t+1)i =π′ T,ti (S T,(t+1)i ,θ T ' ,ti )+ζ T ' ,(t+1)i ;

[0022] In the multi-agent architecture, based on γ T,t Perform finite element modal analysis to output the calculated modal information, and solve the modal error W based on the test modal information at that moment T,t (γ T,t ), the above process is defined as an environment that interacts with the agent. In the interactive environment, k agents jointly make an action a T,t After that, according to the state S T,ti γ in T,t , γ T,t+1 =γ T,t +a T,t ;

[0023] In the multi-agent architecture, define r T,ti =lg(W T,ti (γ T,ti ) / W T,(t+1)i (γ T,(t+1)i )) is the agent α i Perform action a at time t T,ti The incentives obtained after interacting with the environment. After interacting with the environment, k agents obtain the experience set Z t =(S T,ti ,a T,ti ,r T,ti ,S T,(t+1)i ), which includes the state, action and incentive at time t, and the state at time t+1;

[0024] In the multi-agent architecture, the value Q is defined to measure the long-term benefits of the action. Each critic network is divided into a local critic network and a target critic network, respectively denoted as Q 1T,ti (S T,ti ,a T,t ,ψ 1T,ti ) and Q2T,ti (S T,ti ,a T,t ,ψ 2T,ti ), and Q1′ T,(t+1)i (S T,(t+1)i ,a′ T,t+1 ,ψ1′ T,ti ) and Q2′ T,(t+1)i (S T,(t+1)i ,a′ T,t+1 ,ψ2′ T,ti ), represents the agent α i According to the S it observes at that moment T,ti , the action a taken by all agents together T,t , based on the parameters of the neural network at this time, make a long-term value assessment of the action, using Q1′ T,(t+1)i and Q2′ T,(t+1)i The smaller value in is shown in formula (1):

[0025] Q T,ti =r T,ti +λmin(Q1′ T,(t+1)i ,Q2′ T,(t+1)i ) (1)

[0026] Where λ is the discount rate, which means action a at time t T,ti The attenuation effect on subsequent actions in the scene.

[0027] Furthermore, in S1, the process of constructing a tutor-guided action generator based on sensitivity calculation includes:

[0028] The sensitivity-based calculation process is performed using equations (2) and (3):

[0029] Δγ p =(S T (γ p )S(γ p )) -1 S T (γ p )W(γ p ) (2)

[0030] Where S(γ p ) represents the p-1th iteration step, W(γ p ) is about γ p-1 The sensitivity matrix of

[0031] Current damage sensitivity matrix S(γ p ), the difference method shown in formula (2) is used to solve it:

[0032]

[0033] Matrix S(γ p ) is the number of parameters involved in nl components;

[0034] The process of initializing relevant parameters includes: initializing parameters θ 0,0i ,ψ 10,0i ,ψ 20,0i ,θ0′ ,0i , ψ1′ 0,0i , ψ2′ 0,0i and Θ.

[0035] Furthermore, in S2, the specific process includes:

[0036] Generate several sets of random target damage distributions and initial damage distribution At this time, T represents different goals Each T is divided into different scenes with different

[0037] At time t of a certain scene at time T, each agent outputs action a T,ti =π T,ti (S T,ti ,θ T,ti ), and form the experience set Z t =(S T,ti ,a T,ti ,r T,ti ,S T,(t+1)i );

[0038] At the same time, according to the instructor's guidance action generator, output action a T,ti,P , a T,ti,P Incorporating the experience set, we have Z t ″=(S T,ti ,a T,ti ,r T,ti ,S T,(t+1)i ,a T,ti,P ), and store it in the experience replay pool, repeat step S2 to a preset number of times, so as to enrich the capacity of the experience replay pool Θ.

[0039] Furthermore, in S3, the specific process includes:

[0040] After completing the process of S2, randomly select the experience set Z″ of M moments from the experience replay pool Θ j =(S T,ji ,a T,ji ,r T,ji ,S T,(j+1)i ,a T,ji,P );

[0041] According to equations (4) and (5), the action network and the critic network are updated. At this time, the goal of updating the action network is to train a student action network so that its actions are close to the mentor-guided action generator:

[0042]

[0043]

[0044] It represents the gradient attenuation coefficient of the teacher network, that is, when the time exceeds t0, the effect of the teacher is slowly weakened over time.

[0045] Furthermore, in S4, the specific process includes:

[0046] Complete several virtual damage distributions After the learning task, a T is introduced into the real sensor data, and each T is divided into different scenes with different Repeat steps S2 to S3 to identify the damage distribution at this time

[0047] Then, by modifying the target damage distribution, the target damage distribution Modified to Complete the training again. At this time, the output is close to the student behavior generator θ of the Newton method pre-train , and a critic network that can evaluate its output.

[0048] Furthermore, in S5, the specific process includes:

[0049] Using the probability function of formula (6) as the selection function of the action generator, the student behavior action generator is gradually used to replace the instructor guidance action generator. Modified to If the teacher-guided action is not used at a certain moment, the experience set is Z j =(S T,ji ,a T,ji ,r T,ji ,S T,(j+1)i ), if the teacher guides the action, the experience set is Z″ j =(S T,ji ,a T,ji ,r T,ji ,S T,(j+1)i ,a T,ji,P ), the network is updated using formulas (5), (7), and (8) for the experience set Zj sampled at M moments, Z″ j Update the network using equations (4), (5), and (7):

[0050]

[0051]

[0052]

[0053] Among them, P min represents the minimum probability of the tutor-guided action generator appearing, epoch represents the number of training rounds, χ is the control coefficient used to control the decay rate of the tutor-guided action generator appearance probability, and τ is the soft update coefficient.

[0054] Furthermore, in S6, the specific process includes:

[0055] In the learning process, an evaluation module is introduced, that is, several moments are randomly selected, and real data and virtual data are sampled respectively, and one scene of the task is performed separately;

[0056] The current local performer network π is determined using four indicators: the mean value of the incentives of each agent at each moment, the convergence rate within L actions, the convergence rate of the damage distribution of L actions, and the error value between the identified damage distribution and the preset damage distribution. T,ti (S T,ti ,θ T,ti ) Output the quality of the action, according to Q 1T,ti and Q 2T,ti The convergence of is used to judge the quality of the local critic network;

[0057] If the damage identification success rate decreases by less than 15% during the evaluation process, the current action network parameter θ T,ti To update θ pre-train If the success rate decreases by more than 15%, use formula (9) to update the current network θ T,ti :

[0058] θ T,ti =τθ T,ti +(1-τ)θ pre-train (9)

[0059] When the quality of the actions output by the performer network and the value of the critic network output meet the established requirements during the evaluation, the training is completed.

[0060] Furthermore, in S1 to S3, experience replay adopts the following adaptive experience replay mechanism:

[0061] Introduce the probability array P, which is a one-dimensional vector. The dimension is consistent with the number of samples in the experience replay pool Θ. Suppose the capacity of the experience replay pool is N rb , which is a changing value before reaching the maximum value, and the selected batch size is N bs is a fixed value;

[0062] Sampling is divided into two stages:

[0063] In the first stage, the network just started to enter the training phase and met the requirements of N bs Sampling N pick After times, the probability that each sample is taken at least once is greater than 95%, N bs , N pick and N rb The approximate estimated relationship between the three is shown in the following formula:

[0064]

[0065] In the second stage, equation (10) is used for calculation.

[0066]

[0067] in represents the average reward of multiple agents at time t in the experience set, Indicates normalizing the average reward, setting the control coefficient 0<κ1<1 to control the proportion of negative incentives selected, Tanh represents the hyperbolic tangent function, P T,ti Indicates the probability that other experience sets are selected from the experience pool except the new experience set, κ2∈N + Indicates the rate at which samples approach equal probability.

[0068] Compared with the prior art, the present invention has the following beneficial effects:

[0069] 1. The detection method of the present invention introduces a theoretical auxiliary mechanism, which alleviates the main problems of traditional reinforcement learning for damage detection: high complexity of learning objectives and low richness of learning samples;

[0070] 2. The detection method of the present invention introduces an adaptive experience replay mechanism, which alleviates the problem of poor experience replay efficiency faced by traditional reinforcement learning when used for damage detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 This is a schematic diagram of data flow during the training phase of the present invention;

[0072] Figure 2 It is a schematic diagram of the algorithm flow adopted by the present invention. DETAILED DESCRIPTION

[0073] Overall, the present invention discloses a damage detection method based on theory-assisted multi-agent learning. The computational method includes the following steps: S1: Building a mentor-guided action generator based on a sensitivity-corrected damage detection theoretical model; S2: Populating an experience replay pool under several virtual damage distributions; S3: Using an adaptive experience replay mechanism to sample from the experience replay pool, training the student action network so that its output approximates the mentor-guided action generator;

[0074] S4: Introducing real sensor data and generating corresponding virtual damage tasks based on the theoretical model, while simultaneously training the action and comment networks. S5: Based on the probabilistic model, the instructor's participation in the action generator is gradually reduced, gradually becoming dominated by the student action network. S6: Evaluating model metrics during training and adopting a model update strategy based on these metrics. This invention incorporates a theoretical assistance mechanism and an adaptive experience replay mechanism, resulting in high convergence rate, high computational efficiency, and strong robustness. This invention can detect damage to architectural heritage and provide a theoretical basis for scientifically assessing the safety status of architectural heritage.

[0075] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. Any features such as the structural module names, control modes, algorithms, process steps, or composition ratios not explicitly described in this technical solution are considered to be common technical features disclosed in the prior art.

[0076] Example 1

[0077] This embodiment discloses a structural damage detection method based on theory-assisted multi-agent, including the following steps:

[0078] For each sensor, the actual time T at which the sensor collects data is ΔT, and the modal error vector within this time can be recorded as W T,0 (γ T,0 ). In the virtual time step t, it is assumed that there are k agents, each of which is responsible for component damage detection in a region of the structure. Then agent α i It can be observed that the modal information error vector W associated with it at this time T,ti (γ T,ti ), and its associated damage distribution vector γ T,ti , note S T,ti =[W T,ti (γ T,ti ),γ T,ti ] represents the state matrix of agent i at this time. T,ti W is the material damage reduction factor for components in the area that the agent is responsible for. T,ti (γ T,ti ) selects the modal information on the overall degrees of freedom.

[0079] Definition aT,ti represents the agent α at time t i The action taken has the same dimension as γ T,ti Consistent, can be understood as a T,ti =Δγ T,ti , represents the agent α i The array Δγ that adjusts the damage of the component it is responsible for observing T,ti . Using the deterministic policy gradient algorithm, π T,ti (S T,ti ,θ T,ti ) is defined as agent α i At time t, based on the parameter θ T,ti Given a strategy, mapping the input to the observed state S T,ti , the output is pending action a T,ti In MATD3, strategy π T,ti (S T,ti ,θ T,ti ) can be fitted using a deep neural network called a local performer network, θ T,ti Indicates the parameters to be trained of the network. Introducing the target performer network π′ T,ti (S T,(t+1)i ,θ T ' ,ti ), which means the state S at time t is input at time t+1 T,(t+1)i , to predict the action at time t+1.

[0080] Introducing the noise distribution ζ, at time t, for the agent α i According to the strategy π T,ti (S T,ti ,θ T,ti ) output action superimposed with a low-amplitude noise, then there is a T,ti =π T,ti (S T,ti ,θ T,ti )+ζ T,ti , and there is a target action a′ T,(t+1)i =π′ T,ti (S T,(t+1)i ,θ T ' ,ti )+ζ T ' ,(t+1)i .

[0081] Based on γ T,t Perform finite element modal analysis to output the calculated modal information, and then solve the modal error W based on the test modal information at that moment T,t (γ T,t ) is defined as an environment that interacts with an agent. For this environment, k agents jointly make an action a T,tAfter that, according to the state S T,ti γ in T,t , then γ T,t+1 =γ T,t +a T,t . T,t+1 In the input environment, solve W at this time T,t+1 (γ T,t+1 ), and according to W T,t+1 (γ T,t+1 ) and γ T,t+1 , determine the state S at time t+1 T,(t+1)i =[W T,(t+1)i (γ T,(t+1)i ),γ T,(t+1)i ].

[0082] Define r T,ti =lg(W T,ti (γ T,ti ) / W T,(t+1)i (γ T,(t+1)i )) represents agent α i Perform action a at time t T,ti Interact with the environment so that the global state space changes from S T,ti Change to S T,(t+1)i After that, the environment feeds back the incentives of the agent. Thus, after interacting with the environment, k agents obtain an experience set Z t =(S T,ti ,a T,ti ,r T,ti ,S T,(t+1)i ), which includes the state, action and incentive at time t, and the state at time t+1. The experience set Z is stored in the experience replay pool Θ, as Figure 1 .

[0083] The value Q is defined to measure the long-term benefit of an action. To avoid overfitting the Q value, a pair of critic networks is introduced. For each critic network, it is divided into a local critic network and a target critic network, respectively denoted as Q 1T,ti (S T,ti ,a T,t ,ψ 1T,ti ), Q 2T,ti (S T,ti ,a T,t ,ψ 2T,ti ), Q1′ T,(t+1)i (S T,(t+1)i ,a′ T,t+1 ,ψ1′ T,ti ), Q2′ T,(t+1)i (S T,(t+1)i ,a′ T,t+1 ,ψ1′ T,ti ). It represents the agent α iAccording to the S it observes at that moment T,ti , the action a made by all agents together T,t , based on the parameters of the neural network at this time, make a long-term value assessment of the action. At this time, use Q1′ T,(t+1)i and Q2′ T,(t+1)i The smaller value of is obtained by using the Bellman equation as shown in formula (1).

[0084] Q T,ti =r T,ti +λmin(Q1′ T,(t+1)i ,Q2′ T,(t+1)i ) (1)

[0085] Where λ is the discount rate, which means action a at time t T,ti The attenuation effect on subsequent actions in the scene.

[0086] The steps of the structural damage detection method based on theory-assisted multi-agent are as follows: Figure 2 As shown:

[0087] Step 1, adopt the sensitivity-based calculation process of formula (2) and formula (3), which is named as the tutor-guided action generator. At this time, the original performer model can be distinguished as the student action performer model.

[0088] Δγ p =(S T (γ p )S(γ p )) -1 S T (γ p )W(γ p ) (2)

[0089] Where S(γ p ) represents the p-1th iteration step, W(γ p )About γ p-1 In formula (1), the most important thing is to calculate the current damage sensitivity matrix S(γ p ). Generally, the difference method shown in formula (2) is used to solve it.

[0090]

[0091] Matrix S(γ p ) is the number of parameters involved in nl components. If each component contains two parameters to be modified, density and elastic modulus, then the matrix S(γ p ) has a column number of 2nl.

[0092] Initialization parameter θ 0,0i ,ψ 10,0i ,ψ20,0i ,θ0′ ,0i , ψ1′ 0,0i , ψ2′ 0,0i and Θ.

[0093] Step 2: Generate several sets of random target damage distributions and initial damage distribution At this time, T represents different goals Each T is divided into different scenes with different At time t of a certain scene at time T, each agent outputs action a T,ti =π T,ti (S T,ti ,θ T,ti ), and form the experience set Z t =(S T,ti ,a T,ti ,r T,ti ,S T,(t+1)i ). At the same time, according to the instructor’s guidance, the action generator can output action a T,ti,P . T,ti,P Incorporating the experience set, we have Z″ t =(S T,ti ,a T,ti ,r T,ti ,S T,(t+1)i ,a T,ti,P ), and store it in the experience replay pool. After repeating Step 2 enough times, the experience replay pool Θ has a certain capacity.

[0094] Step 3: After completing Step 2, randomly select M time experience sets Z″ from the experience replay pool Θ j =(S T,ji ,a T,ji ,r T,ji ,S T,(j+1)i ,a T,ji,P ). Update the action network and critic network according to equations (4) and (5). At this time, the update target of the action network is no longer to maximize the Q value of the action output, but to train a student action network so that its actions are close to the mentor-guided action generator.

[0095]

[0096]

[0097] It represents the gradient attenuation coefficient of the teacher network, that is, when the time exceeds t0, the effect of the teacher is slowly weakened over time.

[0098] Step 4: Complete several virtual damage distributions After the learning task, a T is introduced into the real sensor data, and each T is divided into different scenes with different Repeat Step 2-Step 3 to identify the damage distribution at this time The above process is repeated several times, and only the target damage distribution in Step 2 is required. Modified to That is, from the random target damage distribution, a target damage distribution that randomly oscillates near the real damage distribution is modified, and the output is obtained as the student behavior action generator θ close to the Newton method pre-train , and a critic network that can evaluate its output.

[0099] Step 5: Based on this network, the probability function of formula (6) is used as the selection function of the action generator, and the student behavior action generator is gradually used to replace the tutor guidance action generator. During this period, similar to Step 2-Step 4, Modified to If the teacher-guided action is not used at a certain moment, the experience set is Z j =(S T,ji ,a T,ji ,r T,ji ,S T,(j+1)i ), if the teacher guides the action, the experience set is Z″ j =(S T,ji ,a T,ji ,r T,ji ,S T,(j+1)i ,a T,ji,P ). For the Z of the experience set sampled at M moments at this time j Update the network using equations (5), (7), and (8), Z″ j Use equations (4), (5), and (7) to update the network.

[0100]

[0101]

[0102]

[0103] Among them, P min represents the minimum probability of the appearance of the tutor-guided action generator, epoch represents the number of training rounds, χ is the control coefficient used to control the decay rate of the tutor-guided action generator's appearance probability, and τ is the soft update coefficient.

[0104] Step 6, after performing the above process several times, the training of the real data and virtual damage data of the previous T moments can be completed. In the learning process, an evaluation module is introduced, that is, several moments are randomly selected, and real data and virtual data are required to be sampled separately, and one scene task is performed separately. In addition to the two conventional indicators of the incentive mean of each moment and each agent, and the ratio of convergence within L actions, the ratio of convergence of the damage distribution of L actions can also be defined under the virtual damage data task, as well as the error value between the identified damage distribution and the preset damage distribution. The above four indicators are used to determine the current local performer network π T,ti (S T,ti ,θ T,ti ) Output the quality of the action. According to Q 1T,ti and Q 2T,ti The convergence of θ is used to judge the quality of the local critic network. In addition, if the damage recognition success rate decreases by less than 15% during the evaluation process, the current action network parameter θ T,ti To update θ pre-train If the success rate decreases by more than 15%, use formula (9) to update the current network θ T,ti When the performance quality of the performer network and the value of the critic network output meet the established requirements during the evaluation, the training is completed.

[0105] θ T,ti =τθ T,ti +(1-τ)θ pre-train (9)

[0106] Furthermore, the experience replay adopts the following adaptive experience replay mechanism.

[0107] Introduce the probability array P, which is a one-dimensional vector with the same dimension as the number of samples in the experience replay pool Θ. Suppose the capacity of the experience replay pool is N rb , which is a changing value before reaching the maximum value, and the selected batch size is N bs is a fixed value. The sampling is divided into two stages. In the first stage, the network just enters the training phase and meets the requirements of N bs Sampling N pick After 100 times, the probability that each sample is taken at least once is greater than 95%. To meet the above requirements, N bs , N pick and N rb The approximate relationship between the three is shown in the following formula: After a large number of numerical tests, the present invention recommends N pick Take 100, and N rb =2N bs As a dividing line. At this stage, an equal probability pattern can be used for sampling to allow the agent to explore the entire space as much as possible.

[0108]

[0109] In the second phase, the number of samples in the experience replay pool increases. When equal probability sampling is still used, the expectation of new samples being sampled gradually decreases. Therefore, at this time, the last element of P can be set to 1, which means that the most recent experience set will definitely be selected in the next sampling. On the other hand, in this phase, the remaining elements in P are calculated using Equation (8).

[0110]

[0111] in represents the average reward of multiple agents at time t in the experience set. Indicates normalizing the average reward and setting the control coefficient 0<κ1<1 to control the proportion of negative incentives selected. Tanh represents the hyperbolic tangent function, P T,ti Indicates the probability that other experience sets are selected from the experience pool except the new experience set, κ2∈N + Indicates the rate at which samples approach equal probability.

[0112] In order to evaluate the accuracy of the present invention for structural damage detection, damage detection was performed on a residential building. A similar structure was used for the action network and the critic network to construct a residual neural network. The only difference between the action network and the critic network is that there is no need to apply tanh and action coefficient scaling to the output layer of the critic network. There are 5 layers of residual blocks for the action network and 8 layers of residual blocks for the critic network. Except for the residual blocks near the input and output, which have 256 hidden layer neurons, the number of hidden layer neurons in the residual blocks is 128. The number of agents is 3. The exploration noise is 3×10 -4 ~1×10 -3 , using exponential sine decay noise. The L2 regularization parameter decay coefficient is 1×10 -3 The discount rate is 0.95, the soft update rate is 0.9, and the action network learning rate is 3×10 -4 , the critic network learning rate is 1×10 -4 , with a gradient clipping threshold of 100. A Layer Normalization (LN) layer is used for regularization in the feature dimension. A warm-up mechanism is introduced to exponentially decay the learning rate. A gradient clipping mechanism is also introduced.

[0113] To compare the performance of the models, the following models will be compared: (1) damage localization based on Newton's method, hereinafter referred to as Newton's method; (2) damage localization based on Newton's method and modified by the L-curve method, hereinafter referred to as L-curve; (3) the model after training of the present invention; and (4) a regression model based on Model-Agnostic Meta-Learning (MAML).

[0114] For 1500 virtual damage tasks, the following indicators are used to test MSE (Mean Squared Error), MSLE (Mean Squared Log Error), MDAE (Median Absolute Error), MAE (Mean Absolute Error), Explained Variance Score (EVS), and R2 (R-squared coefficient). The calculation results are shown in Table 1, indicating that the damage detection method proposed in this invention has the best comprehensive performance.

[0115]

[0116] Where n is the sample size, D is the pre-set damage, It is to predict damage.

[0117] Table 1 Evaluation indicators for measuring calculation errors

[0118]

[0119] Note: The smaller the MSE, MSLE, MDAE, and MAE, the better the results. 2 The closer to 1, the better the result.

[0120] Throughout this specification, references to terms such as "one embodiment," "example," or "specific example" indicate that the specific features, structures, materials, or characteristics described in conjunction with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0121] The above description of the embodiments is intended to facilitate understanding and use of the invention by those skilled in the art. It will be apparent that those skilled in the art can readily make various modifications to these embodiments and apply the general principles described herein to other embodiments without requiring inventive effort. Therefore, the present invention is not limited to the above-described embodiments. Improvements and modifications made by those skilled in the art based on the disclosure of the present invention, without departing from the scope of the present invention, should be within the scope of protection of the present invention.

Claims

1. A structural damage detection method based on theory-assisted multi-agent, characterized in that: The following steps are involved: S1: Based on the sensitivity-corrected damage detection theoretical model, a tutor-guided action generator based on sensitivity calculation is constructed to initialize relevant parameters and provide a benchmark and initial parameter setting for the agent's actions; S2: Generates random targets and initial damage distributions. Each agent outputs actions based on the current strategy and forms an experience set. At the same time, the actions output by the mentor-guided action generator are incorporated into the experience replay pool. By repeating the S2 process multiple times, sufficient data is accumulated. S3: Randomly select multiple experience sets at different moments from the experience replay pool and update the action network and critic network. The action network is updated to approximate the output of the mentor-guided action generator, and the critic network is updated based on the long-term value evaluation of the action. S4: After completing the target number of virtual damage distribution learning tasks, introduce the time parameter into the real sensor data and repeat steps S2 to S3. By modifying the target damage distribution, the network output is gradually guided to approach the real damage situation. S5: Gradually reduce the probability of the instructor guiding the action generator to participate according to the probability model, and gradually the network is dominated by the student action network; S6: Evaluate the indicators of the damage detection theoretical model during the training process, adopt an update strategy for the damage detection theoretical model based on the indicators, and finally use the trained damage detection theoretical model to perform structural damage detection; In S2, the specific process includes: Generate several sets of random target damage distributions and initial damage distribution , in this case T represents different targets Each T is divided into different scenes with different ; At time t of a certain scene at time T, each agent outputs an action , and form an experience set ; At the same time, according to the instructor's guidance, the action generator outputs the action ,Will Incorporating the experience set, we have , and store it in the experience replay pool, repeat step S2 to the preset number of times to enrich the experience replay pool capacity; In S3, the specific process includes: After completing the S2 process, randomly select the experience replay pool Select the experience set of M moments ; According to equations (4) and (5), the action network and the critic network are updated. At this time, the goal of updating the action network is to train a student action network so that its actions are close to the mentor-guided action generator: (4) (5) Represents the gradient decay coefficient of the teacher network, that is, when time passes t 0 Then, the role of teachers is gradually weakened over time.

2. The method for structural damage detection based on theory-assisted multi-agent according to claim 1, characterized in that: In S1, the multi-agent architecture used includes: Multiple intelligent agents are set up for different areas of the structure. Each intelligent agent is responsible for detecting component damage information in a certain area. The intelligent agent observes the relevant modal information error vector and damage distribution vector as its own state matrix. The concept of virtual time step is introduced. In each time step, the agent takes action according to the observed state. The action affects the damage distribution, thereby changing the environmental state and obtaining incentive feedback, forming an experience set stored in the experience replay pool to provide data for network training.

3. The method for structural damage detection based on theory-assisted multi-agent according to claim 2, characterized in that: In a multi-agent architecture, agents It is possible to observe the modal information error vector associated with it at this time , and its associated damage distribution vector ,remember Represents the state matrix of agent i at this time, where The material damage reduction coefficient of the components in the area that the agent is responsible for, Then select the modal information on the overall degree of freedom; In the multi-agent architecture, define represents the agent at time t The action taken has the same dimensions as Consistent, ; Using a deterministic policy gradient algorithm, For intelligent agents At time t based on the parameters Given a strategy, mapping input to observed state , the output is the pending action ; In a multi-agent architecture, strategies Fitting a deep neural network to a local performer network, Represents the parameters to be trained of the network, introducing the target performer network , which means the state at time t+1 is input at time t , to predict the action at time t+1; Introducing noise distribution in multi-agent architecture , at time t, for the agent According to the strategy The output action superimposes a low-amplitude noise. , with target action ; In the multi-agent architecture, Perform finite element modal analysis to output and calculate modal information, and solve the modal error based on the test modal information at that moment ,define the above process as an environment that interacts with the agent; In the interactive environment, k agents take actions together. After that, according to the status in ,get ; In the multi-agent architecture, define For intelligent agents Take action at time t The incentives gained from interacting with the environment; After interacting with the environment, k agents obtain the experience set , including the state, action and incentive at time t, and the state at time t+1; In the multi-agent architecture, the value Q is defined to measure the long-term benefits of the action. Each critic network is divided into a local critic network and a target critic network, which are respectively denoted as and ,as well as and , representing the agent According to what it observes at that moment , the actions taken by all agents together , based on the parameters of the neural network at this time, make a long-term value assessment of the action, using and The smaller value in is shown in formula (1): (1) in, is the discount rate, indicating the action at time t The attenuation effect on subsequent actions in the scene.

4. The method for structural damage detection based on theory-assisted multi-agent according to claim 3, characterized in that: In S1, the process of building a tutor-guided action generator based on sensitivity calculation includes: The sensitivity-based calculation process is performed using equations (2) and (3): (2) in represents the p-1th iteration step, It's about The sensitivity matrix of Current damage sensitivity matrix , and solve it using the difference method shown in formula (2): (3) matrix The number of columns is the number of parameters involved in nl components; The process of initializing related parameters includes: initializing parameters , , , , , and .

5. The method for structural damage detection based on theory-assisted multi-agent according to claim 1, characterized in that: In S4, the specific process includes: Complete several virtual damage distributions After the learning task, a T is introduced into the real sensor data, and each T is divided into different scenes with different Repeat steps S2 to S3 to identify the damage distribution at this time ; Then, by modifying the target damage distribution, the target damage distribution Modified to , complete the training again; At this time, the output is close to the student behavior generator of Newton's method , and a critic network that can evaluate its output.

6. The method for structural damage detection based on theory-assisted multi-agent according to claim 5, characterized in that: In S5, the specific process includes: Using the probability function of formula (6) as the selection function of the action generator, the student behavior action generator is gradually used to replace the instructor guidance action generator. Modified to , if the teacher-guided action is not used at a certain moment, the experience set is , if the teacher guides the action, the experience set is , the experience concentration of the M moments sampled at this time Use equations (5), (7), and (8) to update the network. Update the network using equations (4), (5), and (7): (6) （7） (8) in represents the minimum probability of the tutor guiding the action generator to appear, represents the number of training rounds, is the control coefficient, which is used to control the decay rate of the probability of the tutor-guided action generator appearing. is the soft update coefficient.

7. The method for structural damage detection based on theory-assisted multi-agent according to claim 6, characterized in that: In S6, the specific process includes: In the learning process, an evaluation module is introduced, that is, several moments are randomly selected, and real data and virtual data are sampled respectively, and one scene of the task is performed separately; The current local performer network is determined using four indicators: the mean value of the incentives of each agent at each moment, the convergence rate within L actions, the convergence rate of the damage distribution of L actions, and the error value between the identified damage distribution and the preset damage distribution. The quality of the output action is determined by and The convergence of is used to judge the quality of the local critic network; If the damage identification success rate decreases by less than 15% during the evaluation process, the current action network parameters To update If the success rate decreases by more than 15%, use formula (9) to update the current network : (9) When the quality of the actions output by the performer network and the value of the critic network output meet the established requirements during the evaluation, the training is completed.

8. The method for structural damage detection based on theory-assisted multi-agent according to claim 1, characterized in that: In S1 to S3, experience replay uses the following adaptive experience replay mechanism: Introducing probability arrays , which is a one-dimensional vector with the same dimension as the experience replay pool The number of samples in is the same, and the capacity of the experience replay pool is , which is a changing value before reaching the maximum value, and the selected batch size is is a fixed value; Sampling is divided into two stages: In the first phase, the network has just entered the training phase and meets the requirements of sampling After 300 times, the probability that each sample is taken at least once is greater than 95%. , and The approximate estimated relationship between the three is shown in the following formula: In the second stage, the calculation is performed using formula (10): （10） in represents the average reward of multiple agents at time t in the experience set, Indicates normalizing the average reward and setting the control coefficient To control the proportion of negative incentives selected, Tanh represents the hyperbolic tangent function, Indicates the probability of other experience sets being selected from the experience pool except the new experience set, Indicates the rate at which samples approach equal probability.

Images