Bridge health detection method and bridge health monitoring system based on reinforcement learning

Abstract

The invention discloses a bridge health detection method and a bridge health monitoring system based on reinforcement learning, and relates to the technical field of bridge structure health monitoring, an optimization strategy is designed through a reinforcement learning algorithm, then a state-action-award model is established, and the detection strategy is dynamically adjusted in the bridge health detection process. According to the bridge health detection method based on reinforcement learning and the bridge health monitoring system provided by the invention, intelligence and optimization of bridge health detection are realized by introducing a reinforcement learning algorithm, the efficiency and accuracy of the monitoring system are improved, and the safety and durability of a bridge can be better guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of bridge structure health monitoring, and more specifically, to a bridge health detection method and a bridge health system based on reinforcement learning. Background Art

[0002] Bridges are an important part of modern transportation infrastructure. They not only bear huge loads but also face various environmental impacts, which makes them vulnerable to structural damage. Therefore, real-time monitoring of their health status is of great significance for ensuring traffic safety, reducing accident risks and carrying out maintenance and repair in a timely manner.

[0003] In the existing bridge monitoring process, sensors are usually installed at various important locations to monitor various types of data at each location during the operation of the bridge structure in real time. When analyzing the health status of all locations in the bridge structure, each location in the bridge structure is mainly used as each node in the graph attention network model. The health status of each location in the bridge structure is classified and analyzed to determine the health status of the bridge. However, the above technology still has the following shortcomings: 1. Lack of intelligence in detection strategies Existing bridge health monitoring systems usually adopt fixed detection strategies and fail to consider real-time data and environmental changes, resulting in waste of resources and reduced detection effectiveness.

[0004] 2. Model update is not timely Traditional models rely on static data during training and lack self-learning and optimization mechanisms, which makes it difficult for the models to reflect the latest health status of the bridge.

[0005] 3. Insufficient decision support Existing systems often lack effective decision-making support tools and are unable to provide intelligent inspection plans and maintenance recommendations based on monitoring data, which reduces the scientificity and effectiveness of bridge management. Summary of the invention

[0006] An object of the present invention is to solve at least the above problems and / or disadvantages and to provide at least the advantages which will be described hereinafter.

[0007] In order to achieve these objectives and other advantages of the present invention, a bridge health detection method based on reinforcement learning is provided, which designs an optimization strategy through a reinforcement learning algorithm, and then establishes a state-action-reward model to dynamically adjust the detection strategy during the bridge health detection process.

[0008] Preferably, the process of establishing the state-action-reward model includes: S1. For the data set pre-stored or currently measured in the data set X ,based on p —The greedy strategy selects the corresponding action, where p To explore probability; S2. After executing the selected action, observe the next state S and reward R; S3, based on the observation results of S2, use the following trigger function to update the state: In the above formula, α is the learning rate, c is the discount factor, is the behavior value estimate for the current state, Estimate the value of the action for the next state.

[0009] Preferably, in S1, the action is to randomly select an action with probability p a and choose the behavior value estimate of the initial state with probability 1−p The corresponding minimum behavior a ′.

[0010] Preferably, in S1, the behavior value estimate The acquisition process includes: S10. Based on the bridge health detection parameters and the corresponding detection standard requirements, the following decision target table is formed: D : In the above formula, oh Represents the weight corresponding to the detection parameter, S Indicates the standard value of the detection parameter, n Indicates the number of detection parameters; S11, through the following formula To initialize: In the above formula, oh i , x i , S i Respectively represent i The weight, measured value, and standard value of each parameter; S12. Determine threshold value through training M , and adjust the weights by continuous iteration. Stop learning.

[0011] Preferably, in S3, the action value estimate is calculated based on the observed reward and the maximum value estimate of the next state. The corresponding status is updated to evaluate the health status of the bridge structure based on the minimum M.

[0012] Preferably, it also includes, after each detection is completed, collecting the detection results and feedback information for the training of the reinforcement learning model, and continuously updating the learning rate α and discount factor c , enabling it to adapt to changing bridge conditions and environmental factors, and gradually improve its decision-making optimization capabilities.

[0013] A bridge health monitoring system includes: a data acquisition module, a state assessment module, a decision optimization module and a feedback learning module The present invention includes at least the following beneficial effects: the present invention realizes the intelligence and optimization of bridge health detection by introducing a reinforcement learning algorithm, improves the efficiency and accuracy of the monitoring system, and helps to better ensure the safety and durability of the bridge.

[0014] Other advantages, objectives and features of the present invention will be embodied in part through the following description, and in part will be understood by those skilled in the art through study and practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a technical flow chart of the bridge health detection optimization algorithm based on reinforcement learning of the present invention. DETAILED DESCRIPTION

[0016] The present invention is further described in detail below in conjunction with the accompanying drawings so that those skilled in the art can implement the invention with reference to the description.

[0017] A bridge health monitoring system is mainly used in a bridge health detection optimization algorithm based on reinforcement learning. The bridge health monitoring system mainly includes: a data acquisition module, a state evaluation module, a decision optimization module and a feedback learning module. The data acquisition module obtains dynamic monitoring data of the bridge through a variety of sensors; the state evaluation module analyzes the current health state and outputs the characteristic state; the decision optimization module dynamically adjusts the detection strategy based on the reinforcement learning algorithm to obtain the best detection effect with the minimum resource consumption; the feedback learning module optimizes the reinforcement learning model by continuously accumulating and updating data. Through this invention, the intelligence level and response speed of bridge health detection can be effectively improved to ensure the safety and durability of the bridge.

[0018] like Figure 1As shown, the present invention relates to a bridge health detection optimization algorithm based on reinforcement learning, which aims to improve the efficiency and accuracy of bridge health detection. The algorithm combines the bridge health monitoring system and realizes real-time monitoring and evaluation of the bridge structure through adaptive optimization detection strategy. Specifically, a bridge health detection optimization algorithm based on reinforcement learning includes the following steps: 1. Data Collection Deploy a variety of sensors (such as strain, displacement, acceleration and temperature sensors) at key parts of the bridge to collect real-time bridge operation data. Transmit the sensor data to the central processing system via wireless networks, such as Figure 1 As shown, a certain measurement data during data collection is .

[0019] 2. Status Assessment The state assessment module of the central processing system uses machine learning technology to pre-process the collected data, including data denoising (filtering method) and feature extraction (translation-range transformation method). Based on the extracted feature data, computer-recognizable data is formed. At the same time, the state assessment is used to convert the measured data into To assign the corresponding weight I , the measured data and the corresponding weights are processed to obtain the weighted sum. The present invention uses the weighted distance estimation method mentioned above, that is, , substitute into the trigger function, the present invention adopts the threshold trigger principle, sets the threshold M ,when It then enters the feedback learning phase and conducts detection using different detection strategies in the simulated environment.

[0020] 3. Feedback Learning The feedback learning module of the central processing system collects the test results and feedback information after each test is completed, and obtains the evaluation parameters based on the state evaluation. h i (i.e. i The evaluation of the parameters can be considered as the data after the actual measurement is standardized) is used for the subsequent reinforcement learning model training, and a new weight II is assigned based on the actual situation (in most cases, it is the same as the weight I, when h i When obvious anomalies occur, new weights are assigned. It should be noted that obvious anomalies here refer to those based on statistics. s principle, σ is the standard deviation of the normal distribution, when the processed data deviates from the mean by more than 3 s By continuously updating the learning rate α and the discount factor c , enabling it to adapt to changing bridge conditions and environmental factors, gradually improving decision-making 4. Decision optimization The decision optimization module of the central processing system uses reinforcement learning algorithms to design optimization strategies and dynamically adjusts the detection strategy by establishing a state-behavior-reward model. By simulating different detection strategies and results in the environment, the optimal detection solution is selected to improve efficiency. The implementation method of this step is as follows: S1. Determine the test parameters and corresponding standard requirements according to the specifications and procedures to form a decision target table ,in, oh Represents the weight corresponding to the detection parameter, S Indicates the standard value of the detection parameter, n Indicates the number of detection parameters; S2, suppose a certain measurement data is , the initial data for iteration is ; S3. According to p - The greedy strategy selects actions where p represents the exploration probability.

[0021] S30. Initialize behavior value estimation based on actual needs and exploration probability p ,in The weighted distance estimation is adopted, namely: In the formula, oh i , x i , S i Respectively represent i The weight, measured value, and standard value of each parameter; Use existing data to train and determine critical value estimates M , iterate continuously and adjust the weights Stop learning when the value is low enough, which means that there is no need for in-depth learning, thus saving corresponding computing resources.

[0022] S31, or randomly select a behavior a (i.e., with probability p Continue to explore), or select Minimal behavior a′ (i.e., retaining behavior with probability 1-p a′ ), and execute the behavior ( a or a′ ).

[0023] S32. Update behavior value estimate ,Based on the observed reward and the maximum value estimate of the next state, the state update trigger formula is: In the formula, α is the learning rate, c is a discount factor that determines the importance of future rewards relative to current rewards, is the behavior value estimate for the current state, Estimate the value of the action for the next state.

[0024] S4. Based on M Evaluate the health of bridge structures and Figure 1 The corresponding output Real-time monitoring and early warning are carried out, that is, when the output does not meet the requirements of the corresponding regulations, an early warning is given. M It is an estimate of the critical value determined based on actual conditions. For example, if the full score is 10, it can be assumed that when the score is lower than the predetermined score, there is no need for further in-depth study. At this time, the state value is considered to be the current state of the bridge.

[0025] In the decision optimization of this step, based on the reinforcement learning algorithm, the collection strategy can be automatically optimized according to the existing data requirements (such as current specifications, etc.), the optimal path can be selected, unnecessary path waste can be avoided, and the detection effect can be optimized. The algorithm involved in this step has high parameter sensitivity. Combined with the existing dynamic sensors, it can be timely and continuously self-iterated and updated, and the updated results can be fed back to the bridge structure. In addition, the reinforcement learning algorithm involved in this step can be visualized and connected to existing statistical analysis tools (such as Excel, etc.), and various statistical data can be given based on statistical principles. Based on the high sensitivity of the algorithm, the results can be fed back to the structure to provide decision parameter support in a timely manner.

[0026] 5. Real-time monitoring and early warning The central processing system applies the optimized detection strategy to actual bridge monitoring and analyzes the health status of the bridge in real time. Based on the detection results, it generates health status reports and early warning information, and provides maintenance recommendations and decision support.

[0027] The above solution is only an illustration of a preferred embodiment, but is not limited thereto. When implementing the present invention, appropriate replacement and / or modification can be performed according to user needs.

[0028] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and the embodiments. It can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily realized. Therefore, without departing from the general concept defined by the claims and equivalent scope, the present invention is not limited to the specific details and the illustrations shown and described here.

Claims

1. A bridge health detection method based on reinforcement learning, characterized in that: An optimization strategy is designed through a reinforcement learning algorithm, and then a state-action-reward model is established to dynamically adjust the detection strategy during the bridge health detection process.

2. The bridge health detection method based on reinforcement learning as claimed in claim 1, characterized in that: The process of establishing the state-action-reward model includes: S1. For the data set pre-stored or currently measured in the data set X ,based on p —The greedy strategy selects the corresponding action, where p To explore probability; S2. After executing the selected action, observe the next state S and reward R; S3, based on the observation results of S2, use the following trigger function to update the state: In the above formula, α is the learning rate, γ is the discount factor, is the behavior value estimate for the current state, Estimate the value of the action for the next state.

3. The bridge health detection method based on reinforcement learning as claimed in claim 2, characterized in that: In S1, the actions are based on the probability p Randomly select a behavior a and choose the behavior value estimate of the initial state with probability 1−p The corresponding minimum behavior a ′.

4. The bridge health detection method based on reinforcement learning as claimed in claim 3, characterized in that: In S1, the behavior value is estimated The acquisition process includes: S10. Based on the bridge health detection parameters and the corresponding detection standard requirements, the following decision target table is formed: D : In the above formula, ω Represents the weight corresponding to the detection parameter, S Indicates the standard value of the detection parameter, n Indicates the number of detection parameters; S11, through the following formula To initialize: In the above formula, ω i , x i , S i Respectively represent i The weight, measured value, and standard value of each parameter; S12. Determine threshold value through training M , and adjust the weights by continuous iteration. Stop learning.

5. The bridge health detection method based on reinforcement learning as claimed in claim 4, characterized in that: In S3, the action value is estimated based on the observed reward and the maximum value estimate of the next state The corresponding status is updated to evaluate the health status of the bridge structure based on the minimum M.

6. The bridge health detection method based on reinforcement learning as claimed in claim 2, characterized in that: It also includes collecting test results and feedback information after each test is completed, which is used to train the reinforcement learning model and continuously update the learning rate. α and discount factor γ , enabling it to adapt to changing bridge conditions and environmental factors, and gradually improve its decision-making optimization capabilities.

7. A bridge health monitoring system, applied to the bridge health detection method based on reinforcement learning as claimed in any one of claims 1 to 6, characterized in that: The bridge health monitoring system includes: a data acquisition module, a state evaluation module, a decision optimization module and a feedback learning module.

Images