Bridge structure detection, evaluation and prediction method based on air-wall amphibious unmanned aerial vehicle carried machine vision and intelligent algorithm

Abstract

The invention discloses a bridge structure detection, evaluation and prediction method based on air-wall amphibious unmanned aerial vehicle carried machine vision and an intelligent algorithm, and the method comprises the steps: 1, putting an air-wall amphibious unmanned aerial vehicle into a to-be-detected structure region, and carrying out the corresponding image and signal collection in a flight mode; contact type vibration detection and disease detection are carried out in the adherence mode; 2, extracting a displacement time travel curve and an acceleration time travel curve of the bridge in an operation state, and extracting frequency, vibration mode, damping and other characteristic values of the obtained signal by adopting a signal decomposition method optimized by an intelligent algorithm; 3, acquiring an abnormal image of the surface of the bridge structure, identifying crack and spalling diseases in real time by using a deep learning target detection algorithm, and measuring the crack width through a pixel-level segmentation algorithm U-Net; synchronous non-contact detection is realized; 4, the data processing terminal receives the image data and the signal data collected by the unmanned aerial vehicle in real time, and information fusion of bridge modal parameter decomposition and local disease type recognition is achieved at the terminal.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of bridge structure detection, and particularly relates to a bridge structure detection, evaluation and prediction method based on an air-wall amphibious unmanned aerial vehicle carrying machine vision and intelligent algorithm, which is suitable for efficient and accurate detection of overall performance evaluation and local structure disease of a bridge. BACKGROUND

[0002] As a key node of the traffic network, especially a large-span complex bridge structure, the service reliability thereof is not only related to the system stability of the traffic network, but also directly related to the safety of people's life and property and the efficiency of social and economic development, so the importance of maintenance or inspection thereof is greatly increased.

[0003] In the field of bridge structure detection, many patents have been devoted to improving the efficiency and accuracy of detection. For example, the "Bridge Main Girder Exposed Steel Bar Cause Analysis Method Based on Unmanned Aerial Vehicle Image Collection" of Shenzhen City Traffic Planning and Design Research Center Co., Ltd. collects images with the help of unmanned aerial vehicles, and identifies exposed steel bar problems in combination with AI algorithms, but this patent only focuses on the single disease of exposed steel bar, lacks comprehensive detection capability for other common diseases of bridges such as cracks and peeling, and has a blank in the detection of structural vibration and other mechanical properties. The "Bridge Crack Detection Method and System Based on Unmanned Aerial Vehicle" applied by China Railway Group Limited and others can identify cracks to a certain extent, but is limited by the connectivity of binary image segmentation. When there are holes or fractures in segmentation, the crack width measurement error is large, it is difficult to accurately evaluate the crack risk, and it does not involve analysis of the overall structural parameters of the bridge and the dynamic evolution of the disease. The "Unmanned Aerial Vehicle Bridge Slope Disease Detection Method and System Based on AI Image Recognition" applied by the company Prilidate and others integrates multi-modal data acquisition technology, but still lacks in the fine detection of complex bridge structures, such as contact detection of key parts of the bridge and adaptability detection in special environments.

[0004] Therefore, there is an urgent need for a structure detection method that integrates "local disease identification-global modal analysis" to improve detection efficiency and accuracy. SUMMARY

[0005] To overcome the above problems, the application provides a bridge structure detection, evaluation and prediction method based on an air-wall amphibious unmanned aerial vehicle carrying machine vision and intelligent algorithm, which can realize structure disease identification, modal parameter acquisition and finite element model correction, significantly improve the efficiency and accuracy of static / dynamic response prediction and bearing performance evaluation, and provide an innovative solution for intelligent infrastructure operation and maintenance.

[0006] The specific technical scheme adopted by the application is as follows:

[0007] A bridge structure detection, evaluation and prediction method based on air-wall amphibious unmanned aerial vehicle carrying machine vision and intelligent algorithm, characterized in that the method comprises the following steps:

[0008] Step 1: The air-wall amphibious unmanned aerial vehicle is launched to the structure area to be detected, and corresponding image and signal collection is carried out in the flight mode; contact vibration detection and disease detection are carried out in the wall-adhesion mode; the two modes are automatically triggered to switch through sensor fusion decision according to the actual needs of the detection site, or manually switched by the remote control console, realizing multi-source information acquisition, including bridge dynamic response signal collection, bridge modal parameter detection and local disease defect close-range detection;

[0009] Step 2: The displacement time history curve of the bridge in operation state is extracted by using the visual algorithm; the unmanned aerial vehicle carries an acceleration sensor, and the acceleration sensor is temporarily fixed on the bridge surface by the adsorption device to obtain the acceleration time history curve; the obtained acceleration time history curve is extracted by using the signal decomposition method optimized by the intelligent algorithm to obtain the characteristic values such as frequency, mode shape and damping;

[0010] Step 3: When the unmanned aerial vehicle detects the structure surface anomaly through the visual camera, the path to the target position is automatically planned; the abnormal image of the bridge structure surface is collected by the camera equipment carried on the unmanned aerial vehicle, the crack and spalling disease are identified in real time by using the deep learning target detection algorithm, and the crack width is measured by using the pixel-level segmentation algorithm U-Net; the infrared thermal imager and millimeter wave radar are used to synchronously detect the hidden defects inside the bridge in a non-contact manner;

[0011] Step 4: The data processing terminal receives the image data and signal data collected by the unmanned aerial vehicle in real time, realizes the information fusion of bridge modal parameter decomposition and local disease type identification in the terminal, inputs the bridge historical detection data, environmental parameters, material deterioration mechanism and real-time collected multi-source information into the LSTM intelligent algorithm, constructs a prediction model, and completes the structure detection, evaluation and prediction by using the air-wall amphibious unmanned aerial vehicle carrying machine vision and intelligent algorithm.

[0012] Further, in the step 1, the air-wall amphibious unmanned aerial vehicle is launched at a specified location and a preset inspection route, and the specified location and the inspection route are pre-planned according to the characteristics of the bridge structure and the detection task requirements.

[0013] Furthermore, in flight mode, the drone's onboard LiDAR system collects real-time position information, dynamically monitoring the distance, angle, and flight attitude between the drone and the bridge structure surface. Based on the collected position information, the drone's flight control system records the drone's flight attitude and acceleration in real time via IMU, dynamically adjusting the flight attitude to ensure that the onboard machine vision equipment maintains the optimal shooting distance and angle with the target structure surface, and performs long-distance target image acquisition. In wall-hugging mode, the drone first uses a positioning system that integrates LiDAR and visual sensors to perform high-precision positioning of the bridge structure surface and captures contact signals. The two modes are automatically switched based on the actual needs of the inspection site through sensor fusion decision-making (such as visual odometry + IMU inertial navigation). Subsequently, the drone's flight control system issues a mode switching command, driving the robotic arm to quickly extend. The suction device at the end of the robotic arm is activated simultaneously, establishing sufficient suction force within 3 seconds to firmly attach the drone to the bridge structure surface. The drone achieves climbing functionality through the coordinated joint movements of the robotic arm, which adopts a multi-degree-of-freedom design, with each joint having a range of motion of at least ±180°, enabling flexible adaptation to the complex surface morphology of the bridge structure. During mode switching, the robotic arm utilizes lightweight, high-strength carbon fiber materials and a rapid-action mechanism design to ensure that the extension or retraction time of the robotic arm does not exceed 10 seconds. Simultaneously, the drone integrates an inertial measurement unit and a visual positioning system to perform real-time pose correction during mode switching, ensuring that the drone's wall-hugging positioning accuracy error is less than 0.2 meters, thereby ensuring the continuity and accuracy of the inspection work.

[0014] Furthermore, in step 2, when acquiring image data in flight mode, the UAV's flight control system, based on the pre-planned route, uses the coordinated positioning of lidar and visual positioning system to keep the UAV stably hovering within a safe detection range of 0.5 to 2 meters vertically and 0.3 to 1.5 meters horizontally from the bridge structure.

[0015] Furthermore, in step 2, the displacement time history curve acquisition adopts a feature matching image displacement measurement algorithm to accurately capture the dynamic response of the bridge structure under environmental excitation and vehicle load; the displacement response is converted into acceleration response by signal decomposition to obtain the structural modal parameters.

[0016] The aforementioned accelerometer employs an integrated circuit design, such as a silicon-based piezoresistive sensing structure. The sensor maintains close contact with the surface via an elastic coupling agent (such as silicone) and features a high sampling frequency of at least 100Hz to ensure efficient vibration signal transmission. This sensor incorporates a digital temperature sensor and an adaptive compensation algorithm, automatically correcting measurement errors caused by temperature variations (operating environment -10℃ to 50℃), ensuring an acceleration signal measurement accuracy better than ±0.05m / s². 2 .

[0017] In the acceleration signal decomposition process, various signal decomposition methods such as FMD (fast multivariate decomposition), EMD (empirical mode decomposition), VMD (variational mode decomposition) can be used. For the parameter optimization requirements of the above different signal decomposition methods, intelligent algorithms such as whale optimization algorithm (WOA), grey wolf optimization algorithm (GWO), sparrow search algorithm (SSA) are used for parameter optimization, so that the cross-correlation coefficient of each order modal component after decomposition is less than 0.1, and the frequency feature extraction accuracy is ensured.

[0018] Further, the basic idea of the VMD characteristic modal decomposition method combined with the sparrow search algorithm SSA is as follows:

[0019] ① Define the optimization variable and search space, and determine the variable as the number of modal components K and the penalty factor α.

[0020] ② Design fitness function:

[0021] The fitness function index selects the modal aliasing index MD and the signal reconstruction error RE,

[0022]

[0023] In the formula, K is the modal component, IMF is the intrinsic modal function, the correlation coefficient of adjacent IMF is calculated by corr, and the aliasing possibility of adjacent IMF pairs is checked.

[0024]

[0025] In the formula, N is the number of sampling points of the signal, u(t) is the original input signal, is the IMF signal reconstructed by VMD decomposition.

[0026] ③ Initialize the population size N and initial position (K, α) of the sparrow population.

[0027] ④ Perform the iterative optimization process:

[0028] 1. Calculate the fitness: perform VMD decomposition on the parameters (K, α) of each sparrow individual. The VMD decomposition process is as follows:

[0029] (1) Determine the number of modal components K, the penalty factor α, and initialize each modal component, and initialize the center frequency ω k (0) of each modal component.

[0030] (2) Decompose the original signal x(t) into the sum of K modal components u k (t). Solve the analytic signal by Hilbert transform, and then multiply it by the corresponding exponential term and modulate it to the center frequency. The objective function is:

[0031]

[0032] The constraint condition is:

[0033]

[0034] (3) Introducing a quadratic penalty term and a Lagrange multiplier, the constrained problem is converted into an unconstrained problem, and the alternating direction multiplier method (ADMM) is used to iteratively update each modal component u k (t) and the Lagrange multiplier λ(t). The iterative formula is:

[0035]

[0036] In the formula, K represents the number of modal components, α represents the penalty factor, u k (t) represents the kth intrinsic modal function (IMF) component, ω k represents the center frequency of the kth IMF, δ(t) represents the Dirac function, and x(t) represents the original input signal.

[0037] (4) Continuously iterate until the convergence condition is met, obtain K modal components u k (t), and calculate the fitness value.

[0038] 2. Update the position of the discoverer: judge whether to perform global or local search according to the warning value.

[0039] 3. Update the position of the follower: approach the high-quality solution or perform random search to maintain diversity.

[0040] 4. Disturb the position of the guard: randomly mutate part of the individuals to avoid falling into local optimum.

[0041] 5. Preserve the optimal solution: record the optimal parameters and corresponding fitness of each generation until the termination condition (such as the maximum number of iterations, fitness convergence) is met.

[0042] Further, the specific content of step 3 is:

[0043] When acquiring the disease image dataset in the adhesion mode, the target area is divided into a rectangular detection grid (such as 50 cm x 50 cm), and the key components of the bridge (such as the root of the bridge pier, expansion joint, and cable anchorage area) are set with an encrypted collection grid to ensure that the collected images cover the key areas of the bridge structure surface. When the unmanned aerial vehicle detects that the target area (such as the bridge tower) is not convenient to approach and climb, it switches to the flight mode to remotely collect structural disease images. The visual algorithm training part is based on a deep learning architecture to build a target structure surface disease detection model. During the visual training process, a multi-scale training strategy is adopted to train disease images of different resolutions, improving the model's detection accuracy for different sizes of diseases. The model's detection accuracy (mAP) for cracks is not less than 90%, and the minimum identifiable crack width is 0.1 mm. The mounted infrared thermal imager can analyze the temperature field anomalies on the structure surface and locate the local heating caused by the corrosion of the internal reinforcement of the bridge. The mounted millimeter wave radar can non-contact detect hidden defects such as voids under the bridge deck pavement and water accumulation inside the box girder.

[0044] Further, the specific content of step 4 is:

[0045] The intelligent algorithm-driven bridge structure evaluation and full-life-cycle prediction system is constructed by using intelligent classification algorithms to output disease types through learning a large amount of labeled data. In the model training process, the bridge disease dataset is trained, combined with the static and dynamic responses identified by the unmanned aerial vehicle, and the results of load testing, theoretical analysis, and reliability methods are fused to construct a three-level evaluation index system of local indicators of crack propagation rate, global indicators of structure fundamental frequency change rate, and comprehensive indicators of bearing capacity degradation index. Through evidence theory, multi-source data is fused to establish an evaluation confidence model. At the same time, for structure response prediction, the long short-term memory network (LSTM) combined with the attention mechanism is used to input the bridge historical detection data, environmental parameters, material degradation mechanism, and real-time collected sensor data, etc. to predict the response of the structure in the future period of time, realize the comprehensive coverage from static evaluation to dynamic prediction of bridge bearing capacity, and from deterministic analysis to probabilistic evaluation. Finally, an innovative structure bearing capacity and service performance detection, evaluation, and prediction method is constructed to realize the evaluation of the structure's full-life-cycle reliability and build an intelligent agent model-driven prediction system.

[0046] The technical concept of the present application is that the flight mode completes the overall appearance detection of the bridge, and periodically collects the macro data of the bridge, while the wall climbing mode is used for detailed review of suspected disease areas or key parts, and the collaborative application of the flight mode and the wall climbing mode is constructed; finally, an efficient proxy model is constructed by using an intelligent algorithm, and the boundary conditions, material nonlinear parameters and contact relationship between components of the structure finite element model and other related parameters are iteratively corrected by using the proxy model; based on the corrected high-precision finite element model, the static response and dynamic response of the accurate structure under the action of the load are used to detect and evaluate the bearing capacity; the intelligent algorithm with time series analysis capability such as LSTM is used to construct a prediction model according to multi-source information such as historical diseases and material deterioration mechanism, so as to complete the structure detection evaluation and prediction by using the air-wall amphibious unmanned vehicle carrying machine vision and intelligent algorithm.

[0047] The beneficial effects of the present application are:

[0048] (1) The air-wall amphibious unmanned vehicle detection in the present application can complete "overall modal analysis + local disease identification" through one operation, without manual climbing of high-rise structures for detection, greatly shortening the detection period, saving time and cost, and improving the detection efficiency by more than 3 times, and the correlation of the collected image and signal data is enhanced.

[0049] (2) The present application adopts multi-source data fusion technology, which can fully utilize the complementary information of modal parameters and visual disease images, and obtain a high-precision finite element model through intelligent algorithm optimization of real-time data, thereby improving the reliability of the evaluation result. BRIEF DESCRIPTION OF DRAWINGS

[0050] Fig. 1 The structure detection evaluation method flow chart of the air-wall amphibious unmanned vehicle is shown.

[0051] Fig. 2 The overall structure diagram of the air-wall amphibious unmanned vehicle for bridge structure detection and evaluation is shown.

[0052] Fig. 3 The VMD bridge signal decomposition flow chart combined with the sparrow search algorithm is shown. DETAILED DESCRIPTION

[0053] The specific implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the embodiments of the present application, and is not used to limit the embodiments of the present application.

[0054] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0055] The present application will be described in detail below with reference to the accompanying drawings and in combination with exemplary embodiments.

[0056] Reference Figs. 1 to 3 A bridge structure detection, evaluation and prediction method based on air-wall amphibious unmanned aerial vehicle carrying machine vision and intelligent algorithm, characterized in that the method comprises the following steps:

[0057] Step 1: The air-wall amphibious unmanned aerial vehicle is launched into the structure area to be detected, and corresponding image and signal collection is carried out in the flight mode; contact vibration detection and disease detection are carried out in the wall-adhesion mode; the two modes are automatically triggered to switch through sensor fusion decision according to the actual needs of the detection site, or manually switched by the remote control console, realizing multi-source information acquisition, including bridge dynamic response signal collection, bridge modal parameter detection and local disease defect close-range detection;

[0058] Specifically, in step 1, the air-wall amphibious unmanned aerial vehicle is launched at a designated location and a preset inspection route is set. The designated launch point and the inspection route are pre-planned according to the characteristics of the bridge structure and the detection task requirements, to ensure that the unmanned aerial vehicle can safely and efficiently carry out the detection work.

[0059] In the flight mode, the laser radar system carried by the unmanned aerial vehicle is used to collect position information in real time, and the distance, angle and flight attitude of the unmanned aerial vehicle and the bridge structure surface are dynamically monitored. Based on the collected position information, the unmanned aerial vehicle flight control system records the flight attitude and acceleration of the unmanned aerial vehicle in real time through the IMU, dynamically adjusts the flight attitude, ensures that the machine vision equipment carried always maintains the best shooting distance and angle with the target structure surface, and carries out long-distance target image collection. In the wall-adhesion mode, the unmanned aerial vehicle first carries out high-precision positioning of the bridge structure surface through the positioning system of the laser radar and the vision sensor fusion, and carries out contact signal capture.

[0060] The above two modes are automatically triggered to switch through sensor fusion decision (such as visual odometry + IMU inertial navigation) according to the actual needs of the detection site. Subsequently, the unmanned aerial vehicle flight control system issues a mode switching instruction to drive the mechanical arm to quickly expand, and the suction device at the end of the mechanical arm is started synchronously, establishing sufficient suction force within 3 seconds to make the unmanned aerial vehicle firmly adsorbed on the bridge structure surface.

[0061] More specifically, the unmanned aerial vehicle realizes the climbing function through the joint cooperation of the mechanical arm, the mechanical arm adopts a multi-degree-of-freedom design, each joint has an activity range of not less than ±180°, and can flexibly adapt to the complex surface morphology of the bridge structure. In the mode switching process, the mechanical arm is designed with lightweight high-strength carbon fiber material and fast linkage mechanism to ensure that the expansion or contraction time of the mechanical arm is not more than 10 seconds. At the same time, the unmanned aerial vehicle integrates an inertial measurement unit and a visual positioning system to perform real-time pose correction during mode switching, ensuring that the wall positioning accuracy error of the unmanned aerial vehicle is less than 0.2 meters, thereby ensuring the continuity and accuracy of the detection work.

[0062] Step 2: Extract the displacement time history curve of the bridge in the operating state by using a visual algorithm; the unmanned aerial vehicle carries an acceleration sensor, and the acceleration sensor is temporarily fixed to the bridge surface by the adsorption device to obtain an acceleration time history curve; the obtained acceleration time history curve is extracted by a signal decomposition method optimized by an intelligent algorithm to obtain characteristic values such as frequency, mode shape, and damping;

[0063] Specifically, in step 2, when acquiring image data in the flight mode, the unmanned aerial vehicle flight control system stabilizes the unmanned aerial vehicle in the air within a safe detection range of 0.5-2 meters in the vertical direction and 0.3-1.5 meters in the horizontal direction from the bridge structure according to the pre-planned route through the cooperative positioning of the laser radar and the visual positioning system. The displacement time history curve is collected by using a feature matching image displacement measurement algorithm to accurately capture the dynamic response of the bridge structure under environmental excitation and vehicle load, and the displacement response is converted into an acceleration response by signal decomposition to obtain structural modal parameters.

[0064] More specifically, the acceleration sensor adopts an integrated circuit design such as a silicon-based piezoresistive sensing structure, the sensor is in close contact with the structure surface through an elastic coupling agent (such as silicone), has a high sampling frequency of not less than 100Hz, and ensures the transmission efficiency of the vibration signal. The built-in digital temperature sensor and self-adaptive compensation algorithm of the sensor can automatically correct the measurement error caused by temperature changes (-10℃-50℃ working environment), and ensure that the acceleration signal measurement accuracy is better than ±0.05m / s 2 .

[0065] Specifically, in the acceleration signal decomposition process, various signal decomposition methods such as FMD (fast multi-element decomposition), EMD (empirical mode decomposition), and VMD (variational mode decomposition) can be used, intelligent algorithms such as whale optimization algorithm (WOA), grey wolf optimization algorithm (GWO), and sparrow search algorithm (SSA) are used for parameter optimization according to the parameter optimization requirements of different decomposition methods, so that the correlation coefficient of each modal component after decomposition is less than 0.1, and the frequency characteristic extraction accuracy is ensured.

[0066] More specifically, the basic idea of the VMD feature modal decomposition method combined with the sparrow search algorithm SSA is as follows:

[0067] ① Define the optimization variable and search space, and determine the variable as the number of modal components K and the penalty factor α.

[0068] ② Design the fitness function:

[0069] The fitness function index selects the modal aliasing index MD and the signal reconstruction error RE,

[0070]

[0071] In the formula, K is the modal component, IMF is the intrinsic modal function, corr is used to calculate the correlation coefficient of adjacent IMFs, and the aliasing possibility of adjacent IMF pairs is checked.

[0072]

[0073] In the formula, N is the number of signal sampling points, u(t) is the original input signal, is the signal reconstructed by the IMF decomposed by VMD.

[0074] ④ Perform the iterative optimization process:

[0075] 1. Calculate the fitness: perform VMD decomposition on the parameters (K, α) of each sparrow individual. The VMD decomposition process is as follows:

[0076] (1) Determine the number of modal components K, the penalty factor α, etc., and initialize each modal component, and initialize the center frequency ω k (0) of each modal component.

[0077] (2) Decompose the original signal x(t) into the sum of K modal components u k (t). Solve the analytic signal by Hilbert transform, and then multiply it by the corresponding exponential term and modulate it to the center frequency. The objective function is:

[0078]

[0079] The constraint condition is:

[0080]

[0081] (3) Introduce a quadratic penalty term and a Lagrange multiplier, convert the constrained problem into an unconstrained problem, and use the alternating direction multiplier method (ADMM) to iteratively update each modal component u k (t) and the Lagrange multiplier λ(t). The iteration formula is:

[0082]

[0083] where K denotes the number of modal components, a denotes the penalty factor, u k (t) denotes the kth intrinsic modal function (IMF) component, ω k denotes the center frequency of the kth IMF, δ(t) denotes the Dirac function, and x(t) denotes the original input signal.

[0084] (4) iterates continuously until the convergence condition is met, obtaining K modal components u k (t) and calculating the fitness value.

[0085] 2. Update the position of the discoverer: determine whether to perform global or local search based on the warning value.

[0086] 3. Update the position of the follower: move closer to the high-quality solution or perform random search to maintain diversity.

[0087] 4. Disturb the position of the scout: randomly mutate part of the individuals to avoid falling into local optimum.

[0088] 5. Preserve the optimal solution: record the optimal parameters and corresponding fitness of each generation until the termination condition (such as maximum number of iterations or fitness convergence) is met.

[0089] Step 3: When the unmanned aerial vehicle detects structural surface abnormalities through visual cameras, it automatically plans a path to the target location; the bridge structure surface anomaly images are collected by the camera equipment carried on the unmanned aerial vehicle, and the crack and spalling diseases are identified in real time using a deep learning target detection algorithm, and the crack width is measured by the pixel-level segmentation algorithm U-Net; the infrared thermal imager and millimeter wave radar are used to synchronously detect hidden defects inside the bridge in a non-contact manner;

[0090] Specifically, in step 3, when acquiring the disease image dataset in the wall-climbing mode, the target area is divided into rectangular detection grids (such as 50cm×50cm), and the key components of the bridge (such as the bridge pier root, expansion joint, and cable anchorage area) are set with an encrypted collection grid to ensure that the collected images cover the key areas of the bridge structure surface. When the unmanned aerial vehicle detects that the target area (such as the bridge tower) is not convenient to approach and climb, it switches to the flight mode to remotely collect structural disease images. The visual algorithm training part constructs a target structural surface disease detection model based on a deep learning architecture. During the visual training process, a multi-scale training strategy is adopted to train disease images of different resolutions, improving the detection accuracy of the model for different sizes of diseases, so that the detection accuracy (mAP) of the model for cracks is not less than 90%, and the minimum recognizable crack width is 0.1mm. The infrared thermal imager carried can analyze the temperature field anomalies on the structure surface and locate the local heating caused by bridge internal steel corrosion. The millimeter wave radar carried can non-contact detect hidden defects such as voids under the bridge deck pavement and water accumulation inside the box girder.

[0091] Step 4: The data processing terminal receives the image data and signal data collected by the unmanned aerial vehicle in real time, and realizes information fusion of bridge modal parameter decomposition and local disease type identification at the terminal; an LSTM intelligent algorithm is used to input historical detection data of the bridge, environmental parameters, material deterioration mechanism and real-time collected multi-source information, a prediction model is constructed, and structure detection evaluation and prediction using the air-wall amphibious unmanned aerial vehicle carrying machine vision and intelligent algorithm are completed.

[0092] Specifically, in step 4, the bridge structure evaluation and full life cycle prediction system driven by intelligent algorithm is constructed, an intelligent classification algorithm is used to output the disease type through learning a large number of labeled data. In the model training process, the bridge disease data set is used for training. Combined with the static and dynamic response identified by the unmanned aerial vehicle, the results of load test, theoretical analysis and reliability method are fused to construct a three-level evaluation index system of local index of crack propagation rate, global index of structure fundamental frequency change rate and comprehensive index of bearing capacity degradation index. Through evidence theory, multi-source data is fused to establish an evaluation confidence model. At the same time, for structure response prediction, long short-term memory network (LSTM) combined with attention mechanism is used to input historical detection data of the bridge, environmental parameters, material deterioration mechanism and real-time collected sensor data, etc. to predict the response of the structure in the future period of time. From static evaluation to dynamic prediction of bridge bearing capacity, from deterministic analysis to probabilistic evaluation, a comprehensive coverage is realized. Finally, an innovative structure bearing capacity and service performance detection, evaluation and prediction method is constructed, the reliability of the structure in the whole life cycle is evaluated, and an intelligent agent model driven prediction system is built.

[0093] The present application "Bridge structure detection, evaluation and prediction method based on air-wall amphibious unmanned aerial vehicle carrying machine vision and intelligent algorithm" effectively solves the above problems. Through the unique design of the air-wall amphibious unmanned aerial vehicle, the image and signal can be comprehensively collected in the flight mode, and the contact vibration detection and disease detection can be carried out in the wall mode, which makes up for the one-sidedness of the traditional method of relying only on flight data collection, realizes the all-around and multi-dimensional detection of the bridge. In data processing and analysis, the signal decomposition method optimized by intelligent algorithm is used to extract frequency and other characteristic values, and the deep learning target detection algorithm and pixel-level segmentation algorithm U-Net can accurately identify various diseases such as cracks and spalling and measure the crack width. Compared with the traditional algorithm, the accuracy and comprehensiveness of disease identification are greatly improved. In addition, information fusion is realized through the construction of a data processing terminal, the bridge modal parameters and local diseases can be comprehensively analyzed, which provides a reliable basis for the long-term performance evaluation and prediction of the bridge structure, and solves the problem that the traditional method cannot systematically evaluate the overall state of the bridge.

[0094] The detection system used in the above method comprises a fuselage and a power module, a mechanical arm and an adsorption module, an image and signal acquisition module, and a control system and a data processing module, wherein:

[0095] The fuselage and the power module: the multi-rotor unmanned aerial vehicle platform is equipped with a plurality of brushless motors and propellers, the maximum flight speed reaches 10 m / s, the endurance time is not less than 30 minutes, and the unmanned aerial vehicle can fly according to a preset flight route and adjust the flight path according to the real-time environment;

[0096] The mechanical arm and the adsorption module: four foldable mechanical arms are arranged at the lower part, and the unmanned aerial vehicle is stably adsorbed on the surface of the structure through vacuum negative pressure or magnetic adsorption, the adsorption force of a single mechanical arm is not less than 50 N, and the device can resist a wind speed of 3 m / s; the device and the mechanical arm are connected through a force feedback joint, and can adapt to the curvature of the surface of the structure;

[0097] The image and signal acquisition module: the high-definition camera equipment carried by the unmanned aerial vehicle platform can adjust the shooting angle within ±180°, and the resolution is not less than 2K, supports automatic focusing and low-light illumination functions, and ensures the image acquisition quality; the non-cooled microbolometer is used in the infrared thermal imager, the thermal sensitivity is ≤0.05℃, the temperature resolution is 0.1℃, and the detection temperature range is-20℃-150℃; the laser radar has a scanning rate of ≥200,000 points per second, and the ranging accuracy is within ±5 mm; the acceleration sensor adopts a design such as a silicon-based piezoresistive sensitive structure, and has a high sampling frequency of not less than 100 Hz; the penetration depth of the millimeter wave radar reaches 5-10 cm, and the resolution is not greater than 10 cm 3 .

[0098] The control system and the data processing module: the flight control system receives the control instructions of the terminal, completes the switching of the flight mode and the wall-climbing mode, realizes the remote control of the unmanned aerial vehicle, the high-performance processor is built in the data processing module, the visual algorithm and the signal decomposition processing can be run in real time, the obtained image and signal data are identified and analyzed, the processed multi-source data are transmitted to the remote data processing terminal, the agent model is constructed to correct the finite element model with high precision, and the structure bearing capacity prediction model is obtained in combination with deep learning.

[0099] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A method for bridge structure detection, evaluation, and prediction based on an air-to-wall amphibious unmanned aerial vehicle (UAV) equipped with machine vision and intelligent algorithms, characterized in that, The method includes the following steps: Step 1: Deploy the air-wall amphibious UAV to the area of ​​the structure to be tested, and collect relevant images and signals in flight mode; conduct contact vibration detection and defect detection in wall-hugging mode; the two modes are automatically triggered to switch according to the actual needs of the test site through sensor fusion decision, or can be manually switched by the remote control console to achieve multi-source information acquisition, including bridge dynamic response signal acquisition, bridge modal parameter detection and close-range detection of local defects; Step 2: Extract the displacement time history curve of the bridge under operational conditions using a visual algorithm; the UAV carries an acceleration sensor and temporarily fixes the acceleration sensor to the bridge surface using an adsorption device to obtain the acceleration time history curve; the obtained acceleration time history curve is used to extract characteristic values ​​such as frequency, mode shape, and damping using a signal decomposition method optimized by an intelligent algorithm. Step 3: When the drone detects an anomaly on the structural surface using its visual camera, it automatically plans a path to the target location; The drone collects abnormal images of the bridge structure surface using camera equipment, uses deep learning target detection algorithms to identify cracks and spalling defects in real time, and measures the crack width using the pixel-level segmentation algorithm U-Net. The onboard infrared thermal imager and millimeter-wave radar simultaneously perform non-contact detection of hidden defects inside the bridge. Step 4: The data processing terminal receives image and signal data collected by the UAV in real time, and realizes information fusion of bridge modal parameter decomposition and local defect type identification at the terminal; using LSTM intelligent algorithm, inputting bridge historical inspection data, environmental parameters, material deterioration mechanism and real-time multi-source information, constructing a prediction model, and completing the structural inspection, evaluation and prediction using an air-wall amphibious UAV equipped with machine vision and intelligent algorithms.

2. The method for bridge structure detection, evaluation, and prediction based on an air-to-wall amphibious unmanned aerial vehicle equipped with machine vision and intelligent algorithms as described in claim 1, characterized in that, During the acceleration signal decomposition process, fast multivariate decomposition, empirical mode decomposition, or variational mode decomposition methods are adopted. At the same time, for the parameter optimization requirements of different signal decomposition methods, the Whale Algorithm (WOA), Grey Wolf Algorithm (GWO), or Sparrow Search Algorithm (SSA) are used to optimize the parameters so that the cross-correlation coefficient of each order mode component after decomposition is less than 0.1, thus ensuring the accuracy of frequency feature extraction.

3. The method for bridge structure detection, evaluation, and prediction based on an air-to-wall amphibious unmanned aerial vehicle equipped with machine vision and intelligent algorithms as described in claim 1, characterized in that, The VMD (Feature Mode Decomposition) method, combined with the Sparrow Search Algorithm (SSA), is as follows: ① Define the optimization variables and search space, and determine the variables as the number of modal components K and the penalty factor α; ② Design the fitness function: The fitness function indices used are the mode aliasing exponent (MD) and the signal reconstruction error (RE). In the formula, K is the modal component, IMF is the intrinsic mode function, and the correlation coefficient of adjacent IMFs is calculated by corr to check the possibility of aliasing of adjacent IMF pairs; In the formula, N is the number of sampling points of the signal, and u(t) is the original input signal. The signal is reconstructed from the IMF after VMD decomposition; ③ Initialize the population size N and initial position (K, α) of the sparrow population; ④ Perform iterative optimization.

4. The method for bridge structure detection, evaluation, and prediction based on an air-to-wall amphibious unmanned aerial vehicle equipped with machine vision and intelligent algorithms as described in claim 3, characterized in that, The iterative optimization is as follows: a1. Calculate fitness: Perform VMD decomposition on the parameters (K, α) of each sparrow individual; a2. Update the location of the discoverer: Determine whether to perform a global or local search based on the alert value; a3. Update follower positions: move closer to high-quality solutions or conduct random searches to maintain diversity; a4. Vigilant position perturbation: Some individuals undergo random mutation to avoid getting trapped in local optima; a5. Preserve the optimal solution: Record the optimal parameters and corresponding fitness for each generation until the termination condition is met.

5. The method for bridge structure detection, evaluation, and prediction based on an air-to-wall amphibious unmanned aerial vehicle equipped with machine vision and intelligent algorithms as described in claim 1, characterized in that, In step 2, when acquiring image data in flight mode, the UAV flight control system uses a pre-planned route and a coordinated positioning system of lidar and visual positioning system to keep the UAV stably hovering within a safe detection range of 0.5 to 2 meters vertically and 0.3 to 1.5 meters horizontally from the bridge structure.

6. The method for bridge structure detection, evaluation, and prediction based on an air-to-wall amphibious unmanned aerial vehicle equipped with machine vision and intelligent algorithms as described in claim 1, characterized in that, In step 2, the displacement time history curve acquisition adopts the feature matching image displacement measurement algorithm to accurately capture the dynamic response of the bridge structure under environmental excitation and vehicle load; the displacement response is converted into acceleration response by signal decomposition to obtain the structural modal parameters.

7. The method for bridge structure detection, evaluation, and prediction based on an air-to-wall amphibious unmanned aerial vehicle equipped with machine vision and intelligent algorithms as described in claim 1, characterized in that, In step 3, when acquiring the disease image dataset in wall-hugging mode, the target area is divided into rectangular detection grids, and a dense acquisition grid is set for key bridge components to ensure that the acquired images cover the key areas of the bridge structure surface.

8. A method for bridge structure detection, evaluation, and prediction based on an air-to-wall amphibious unmanned aerial vehicle equipped with machine vision and intelligent algorithms, as described in claim 1, is characterized in that... In step S4, during the model training process, the bridge defect dataset captured by photography is used for training. Combined with the static and dynamic responses obtained by UAV identification, the results of load tests, theoretical analysis and reliability methods are integrated to construct a three-level evaluation index system, which includes a local index of crack propagation rate, a global index of structural fundamental frequency change rate, and a comprehensive index of bearing capacity degradation index. An evaluation confidence model is established by integrating multi-source data through evidence theory.

Images