Bridge internal disease detection method and system based on unmanned aerial vehicle

Abstract

The embodiment of the invention discloses a bridge internal disease detection method and system based on an unmanned aerial vehicle, and the method comprises the steps: deploying local positioning beacons around a to-be-detected bridge, building a local positioning system, and calculating the coordinates of the unmanned aerial vehicle; a mechanical arm is installed on the unmanned aerial vehicle, the unmanned aerial vehicle is controlled to conduct flight inspection according to the calculated coordinates so as to conduct non-contact visual inspection on the bridge, and visual inspection image data are obtained; constructing a disease detection model based on deep learning, performing typical bridge disease identification on the image obtained by visual detection by using the disease detection model, and positioning the position of each disease on the bridge according to an identification result; and the unmanned aerial vehicle flies to the identified bridge disease position, is attached to the bridge disease position through a mechanical arm on the unmanned aerial vehicle, and performs contact type internal flaw detection on the bridge by means of the mechanical arm so as to detect the disease condition in the bridge. According to the invention, apparent diseases and internal information of in-service structures such as bridges and the like can be rapidly detected.

Description

Technical Field

[0001] The present invention belongs to the technical field of UAV applications, and particularly relates to a method and system for detecting internal diseases of bridges based on UAVs. Background Art

[0002] The detection of bridge diseases is one of the important means to maintain the healthy service of bridges. Among them, the detection of the apparent diseases of bridges is the current research focus in the relevant fields. However, since typical diseases are not only reflected on the surface of the bridge, internal diseases such as internal cavities in concrete and corrosion of internal steel bars, or internal information of diseases such as the depth of cracks and the thickness of the anti-rust coating of steel structure bridges are also important objects in disease detection.

[0003] In view of the special detection requirements for the detection of the above internal information of bridges, how to develop a suitable and convenient detection system and method is a major problem in the industry. Taking the coating detection of steel structure bridges as an example, steel corrosion is one of the main diseases of steel structure bridges, and the surface coating of steel structures is the most important part for preventing structural corrosion. Once the coating deteriorates such as peeling and fading, corrosion will occur rapidly at the weak parts of the coating, reducing the durability of the structure. More seriously, if the corrosion location is not found or the corrosion is not treated for a long time, it may cause a reduction in the safety performance of the structure. In the safety maintenance work of bridges, it is generally required to regularly detect the integrity of the coating and randomly check the coating thickness to prevent the corrosion of steel structure bridges. However, different from apparent diseases such as cracks, coating detection is not only an apparent deterioration detection, but also requires quantitative measurement of the coating thickness. Therefore, it is required that the UAV can not only take long-distance pictures of the coating surface conditions, but also perform contactless non-destructive detection on specified positions. Therefore, ordinary UAVs are unable to perform this task.

[0004] To solve the above problems, some special drones with contact detection functions have been studied in recent years. For example, abroad, the Skygauge drone developed by Skygauge robotics consists of a drone with an extended coating gauge and four independently tiltable motors. This drone can move forward and backward without changing its attitude, facilitating the parallel alignment of the drone with the measurement point for contact detection. González et al. designed a contact detection payload for drones that includes two ranging sensors, and determined the control strategy for the drone in the approaching and contact states by calculating the distance and angle between the drone and the target. Santos et al. proposed a route planning method for contact detection of drones in indoor environments. Kocer et al. proposed an optimization algorithm composed of nonlinear moving horizon estimation to solve the control problem of drones in the contact state. Fumagalli et al. studied the modeling and control problems of drone contact detection. Sanchez et al. designed a drone with plastic frames wrapped around the propellers for use in conjunction with a total station for measurement. Takahiro et al. designed a three-degree-of-freedom robotic arm mounted on the top of a drone for contact measurement at the bottom of a bridge. Rashad et al. designed a method for projecting the desired trajectory of the end effector of a robot on the surface of a general shape to be detected during contact detection, and verified it in Gazebo simulation and laboratory experiments. The above research has laid the foundation for drone contact detection. However, the drawback is that most of the research has been verified based on simple laboratory environments and has not considered many difficult problems in actual bridge detection. In fact, problems such as lack of satellite positioning signals, strong crosswinds, and magnetic interference will also be encountered in bridge detection. In addition to the need for contact detection, when detecting near a bridge, the drone also needs to have a certain collision tolerance so that it will not be damaged or fall during a minor collision.

[0005] Generally speaking, for the detection of internal diseases of structures such as bridges, especially for the actual application environment of complex in-service bridges, some exploratory work has been done in the existing research. However, there is still a large room for improvement and perfection in aspects such as the collision tolerance of drones in narrow environments around bridges, the positioning of drones under satellite signal occlusion and magnetic interference, and the establishment of a complete set of high-efficiency detection methods for internal information using visual detection results. Summary of the Invention

[0006] An object of the present invention is to provide a method for detecting internal diseases of bridges based on drones in view of the deficiencies of the prior art. This method can quickly detect the apparent diseases and internal information of in-service structures such as bridges to serve the daily inspection and regular inspection of structures such as bridges.

[0007] To solve the above technical problems, the present invention adopts the following technical solutions: A method for detecting internal diseases of bridges based on unmanned aerial vehicles, comprising the following steps: Step 1: Deploy local positioning beacons around the bridge to be detected, establish a local positioning system, and calculate the coordinates of the unmanned aerial vehicle; Step 2: Install a robotic arm on the unmanned aerial vehicle, control the unmanned aerial vehicle to fly and inspect according to the calculated coordinates, and perform non-contact visual inspection on the bridge during the flight to obtain visual inspection image data; Step 3: Construct a disease detection model for identifying surface diseases of bridges based on deep learning, use the disease detection model to identify typical bridge diseases in the images obtained by visual inspection, and then locate the position of each disease on the bridge according to the identification results; Step 4: The unmanned aerial vehicle flies to the identified bridge disease, attaches to the position of the bridge disease through the robotic arm, and uses its robotic arm to perform contact internal flaw detection on the bridge to detect the internal disease condition of the bridge.

[0008] Further, the specific implementation manner of Step 1 is: Arrange multiple beacons around the bridge as anchor points, and the positions of the beacons cover the area to be detected of the bridge. After arranging the beacons, calibrate the coordinates of the beacons by self-calibration between the beacons; then the anchor point coordinates of multiple known beacons in space are (x 1 , y 1 , z 1 ),(x 2 , y 2 , z 2 ),…，(x n , y n , z n )，n is the total number of beacons, and the coordinates of the unmanned aerial vehicle to be calculated are (x t , y t , z t ), , and the straight-line distance between the beacon and the unmanned aerial vehicle is R i , i = 1 to n , where the method for obtaining the coordinates of the unmanned aerial vehicle by beacon positioning is: .

[0009] Further, the deep learning model adopted in Step 3 is a network model without an anchor mechanism.

[0010] Further, the disease detection model includes a classification branch, a regression branch, and a centrality branch; Among them, the classification branch includes 3×3 convolution, 1×1 convolution, and Sigmoid function activation, and is used to output the number of categories corresponding to the number of channels; The regression branch includes 3×3 and 1×1 convolutions and a linear activation function, and is used to regress and predict the 4D coordinates of the bounding box. The 4D coordinates include the center point offset of the bounding box and the width and height; The centrality branch includes 1×1 convolution and is used to perform centrality prediction to suppress low-quality edge predictions.

[0011] Further, before predicting the image, the disease detection model is first trained. The training method includes: obtaining images related to the bridge steel structure to be detected from an existing database, adjusting all the images to a size similar to that of the drone camera, and marking them to frame the location of each disease; after marking, performing data augmentation and amplification on the images to obtain a dataset; Dividing the dataset into a training set, a validation set, and a test set, training the disease detection model using the training set and validating it using the validation set. Preferably, testing the validated model using the test set to obtain an optimized disease detection model.

[0012] Further, in step 2, the robotic arm includes a base fixed on the drone, a plurality of cantilever legs hinged to the base at equal intervals in the circumferential direction, and a tactile feedback electromagnetic induction non-destructive measurement module fixed at the center position of the base. The cantilever leg includes an upper swing arm, a connecting rod, a lower swing arm, and a moving end support leg attached to the bridge surface during detection. Among them, the upper swing arm is also hinged to the base, and the upper swing arm, the connecting rod, the lower swing arm, and the moving end support leg are connected to form a quadrilateral.

[0013] Further, a damping spring is also provided on the cantilever leg. The damping spring is arranged on the diagonal of the cantilever leg. One end of the damping spring is fixed at the hinge point of the upper swing arm and the connecting rod, and the other end is fixed at the hinge point of the lower swing arm and the moving end support leg.

[0014] Further, in step 4, the robotic arm adheres to the bridge disease area through a self-stabilizing control method. The self-stabilizing control method includes: The tactile feedback electromagnetic induction non-destructive measurement module continuously obtains the normal pressure of the contact surface between the robotic arm and the bridge, and then uses the Kalman filter algorithm to eliminate vibration noise. The elimination process formula is: ; Among them, is the pressure after noise elimination at time k, is the state transition matrix, is the external input The control input matrix below is the process noise; Decompose the pressure moment after noise elimination into the lift component perpendicular to the bottom surface of the bridge and the interference component in the translation direction. When the detected pressure fluctuation exceeds the threshold, trigger a control response to adjust the lift; during the lift adjustment, the tactile feedback electromagnetic induction non-destructive measurement module outputs a lift control signal according to the pressure deviation. After receiving the lift control signal, the drone adjusts the lift of the lift motor on it to achieve the stable attachment of the robotic arm.

[0015] Furthermore, the method for adjusting the lift of the drone according to the pressure deviation includes: The tactile feedback electromagnetic induction non-destructive measurement module collects the current pressure in real time, compares it with the set value to obtain an error signal, combines the proportional, integral, and differential links to generate a control quantity, and adjusts the lift in the reverse direction according to the positive and negative characteristics of the control quantity; when the current pressure is greater than the set value, output a negative control quantity to control the lift motor to reduce the lift, and when the current pressure is less than the set value, output a positive control quantity to control the lift motor to increase the lift, thus forming a closed-loop feedback; The calculation method of the above lift adjustment process is: ; wherein, is the lift motor control signal, is the time, is the proportional gain, is the integral gain, is the derivative gain, is the error at time is the error at time is a temporary integral variable used to traverse the time points from 0 to ; The calculated above is the control signal input to the lift motor controller of the drone. The lift motor controller of the drone adjusts the lift of the lift motor according to the control signal to ensure the stability of the position, attitude, and contact force of the drone during contact detection.

[0016] The present invention also provides a system for implementing the above-mentioned method for detecting internal diseases of bridges based on drones, including: A route planning module for calculating the coordinates of the drone according to the local positioning system established by local positioning beacons deployed around the bridge to be detected; An image acquisition module for obtaining visual detection image data when the drone performs non-contact visual detection of the bridge during the flight inspection controlled by the calculated coordinates; The disease identification module is used to construct a disease detection model for identifying bridge surface diseases based on deep learning. The disease detection model is used to identify bridge diseases from the images obtained by visual detection, and then according to the identification results, the position of each disease on the bridge is located; The internal disease detection module is used to obtain the internal disease condition of the bridge through contact internal flaw detection by the robotic arm when the unmanned aerial vehicle (UAV) flies to the identified bridge disease and attaches to the position of the bridge disease through the robotic arm on it.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. Currently, most UAVs are used for apparent disease detection. Since non-destructive detection of internal information requires contact with the structural surface, it is difficult for existing UAVs to perform this operation. The UAV proposed in the present invention not only has a camera to perform rapid surface disease detection during flight, but its collision-resistant design, local positioning method, non-destructive detection robotic arm, and automatic attitude control of the contact state enable it to still achieve stable and accurate measurement during contact non-destructive detection; 2. Conventional UAVs use positioning signals such as GPS for positioning. In environments such as the bottom and periphery of bridges, due to the obstruction of the bridge, it is difficult to receive GPS and other positioning signals. In response to this problem, the UAV involved in the present invention adopts a positioning method based on ultra-wideband wireless communication beacons, and replaces GPS and other satellite positioning signals by self-building a local positioning network, enabling the UAV to be positioned in environments such as the bottom of the bridge; 3. In order to realize the real-time analysis of the camera video of the UAV during flight to obtain the video frames of diseases and the position coordinates of diseases in real time, the present invention proposes a lightweight anchor-free object detection network, which avoids the prior clustering analysis of disease bounding boxes by designing an anchor-free feature extraction mechanism, and also avoids the problem that the preset bounding box size does not match the size of the actually collected disease data; 4. The contact detection function is the core advantage of the system and method proposed in the present invention. Different from simply fixing a non-destructive measurement module on a UAV in the existing method, the present invention designs an elastic measurement robotic arm and a pressure feedback control method. Since the UAV is affected by its own flight control attitude correction and environmental wind when it touches the bridge surface, it is difficult for a conventional UAV to maintain stable contact hovering, resulting in the problem that the non-destructive detection system cannot accurately complete the measurement during contact. The elastic robotic arm of the present invention automatically corrects the attitude when the UAV touches the bridge surface through four cantilever legs with damped spring-back in a quadrilateral structure. The tactile feedback electromagnetic induction non-destructive measurement module in the middle of the robotic arm is a flexible contact pressure sensing film. During contact measurement, the pressure sensing film senses the pressure in real time and adjusts the lift of the UAV in real time through the pressure, enabling the UAV to stably adsorb on the bridge surface and providing conditions for accurately and stably measuring internal information; 5. Non-destructive testing is generally carried out in the form of spot checks. Existing methods determine the spot check points according to the design drawings. However, the positions where internal damage usually occurs are mostly the positions where apparent diseases already exist. Therefore, the present invention proposes a two-stage measurement method. In the first stage, the drone performs a fast flight-type apparent disease detection and records the positions of the identified apparent diseases in real-time analysis. In the second stage, fixed-point internal information detection is performed according to these positions. Compared with the existing spot check methods, the proposed method is more likely to discover potential problems of the structure. Description of the Drawings

[0018] Figure 1 Schematic diagram of the method for detecting internal diseases of a bridge based on a drone according to an embodiment of the present invention; Figure 2 Schematic diagram of the local positioning system according to an embodiment of the present invention; Figure 3 Schematic diagram of the structure affected by the drone disease detection according to an embodiment of the present invention; Figure 4 Schematic diagram of the structure of the robotic arm according to an embodiment of the present invention. Detailed Embodiment The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0019] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0020] The present invention will be further described below in conjunction with specific embodiments, but it is not limited to the present invention.

[0021] As Figure 1 shown, an embodiment of the present invention discloses a method for detecting internal diseases of a bridge based on a drone, including the following steps: Step 1: Deploy local positioning beacons around the bridge to be detected, establish a local positioning system, and calculate the coordinates of the drone; Since conventional commercial drones use signals such as GPS / Beidou for positioning, when detecting positions such as the bottom and periphery of the bridge, the drone cannot receive positioning signals due to the occlusion of the bridge. Therefore, in this embodiment, an ultra-wideband wireless communication beacon is designed for the drone as a positioning method. Although a positioning network needs to be pre-arranged, the weight of the beacon is less than 20 grams and the effective distance exceeds 500 meters, which is very suitable for the designed small and collision-resistant drone. See Figure 2, the positioning method of the ultra-wideband wireless communication beacon is specifically as follows: Multiple fixed beacons are arranged around the bridge as anchor points. In this embodiment, the number of beacons is 4, and the 4 beacons form a rectangular area. In other embodiments, other numbers can be selected according to actual needs. The positions of the beacons cover the area to be detected of the bridge and are at least 1 meter away from surrounding obstacles. After arranging the beacons, the coordinates of the beacons are located through self-calibration between the beacons. At this time, the anchor point coordinates of the four beacons in space are (x 1 , y 1 , z 1 ),(x 2 , y 2 , z 2 ), (x 3 , y 3 , z 3 ), (x 4 , y 4 , z 4 ), The coordinates of the drone to be calculated are (x t , y t , z t ), , the straight-line distances between the four beacons and the drone are R1, R2, R3, and R4. The specific calculation method for obtaining the position by four-point positioning in space is:

[0022] After the coordinates of the drone are calculated, they are directly output to the flight control of the drone, and the flight control controls the drone to fly according to these coordinates. After the drone obtains the positioning coordinates as described above, it can stably position and fly even under conditions where there is no satellite positioning signal such as GPS / Beidou at the bottom of the bridge, and the drone will not drift due to the action of wind or the like.

[0023] Step 2: Install a robotic arm on the drone, control the drone to fly for inspection according to the calculated coordinates, and perform non-contact visual inspection on the bridge during the flight to obtain visual inspection image data; In this embodiment, the design of the drone takes into account the following three requirements: ① The structure of the drone should be as simple and stable as possible. Therefore, the most commonly used and least component four-rotor layout of the multi-rotor type drone is adopted; ② To facilitate passing through narrow positions, the volume of the drone should be small enough and as light as possible. Therefore, the designed drone has an outer diameter of 46 cm and a weight of about 1.8 kg; ③ The drone should have the function of stably adsorbing on the bridge surface to facilitate non-destructive testing, considering that the most important and difficult-to-measure position of the bridge is the bottom of the bridge. Therefore, as Figure 3As shown in the figure, a robotic arm 2 is installed on the UAV 1, which is convenient for non-contact visual inspection. A two-axis gimbal and a camera 3 are mounted on the front of the UAV for non-contact visual inspection of the bridge to obtain visual inspection image data, and the visual inspection image data of the camera 3 is processed in real time by a microprocessor inside the UAV.

[0024] As Figure 4 shown, the robotic arm 2 includes a base 21 fixed on the UAV, a plurality of cantilever legs 22 hinged to the base 21 at equal intervals in the circumferential direction, and a tactile feedback electromagnetic induction non-destructive measurement module 23 fixed at the central position of the base 21. The cantilever leg 22 includes an upper swing arm 221, a connecting rod 222, a lower swing arm 223, and a moving end support leg 224 that is attached to the bridge surface during detection. Among them, the upper swing arm 221 is also hinged to the base 21, and the upper swing arm 221, the connecting rod 222, the lower swing arm 223, and the moving end support leg 224 are connected to form a quadrilateral. In order to enable the cantilever leg 22 to reduce the vibration during the contact detection, a damping spring 225 is also provided on each cantilever leg 22. The damping spring 225 is arranged on the diagonal of the cantilever leg 22, that is, one end of the damping spring 225 is fixed at the hinge point of the upper swing arm 221 and the connecting rod 222, and the other end is fixed at the hinge point of the lower swing arm 223 and the moving end support leg 224. Since the four cantilever legs 22 all have the same elastic force, even if the UAV 1 and the bridge are inclined at the beginning of contact, the UAV 1 can gradually return to a posture parallel to the bridge surface through the elastic force balance. The tactile feedback non-destructive measurement module 23 in the middle of the robotic arm 2 is in the central area, and an electromagnetic induction probe 231 is installed therein. The robotic arm 2 is made by 3D printing.

[0025] Step 3: Build a disease detection model for identifying bridge surface diseases based on deep learning, use the disease detection model to identify bridge diseases in the images obtained by visual inspection, and then locate the position of each disease on the bridge according to the identification results; The combination of visual detection and deep learning technology is a very promising automatic detection method. This method has been extensively studied and is increasingly widely applied in bridge engineering. In the engineering field, the indicators that are most concerned about the application of deep learning methods are not only the detection accuracy, but also the inference speed and whether it can be transplanted into industrial equipment. Therefore, the disease detection model designed in this embodiment has the characteristic of being lightweight. In addition, traditional object detection networks based on prior anchor boxes need to obtain several groups of optimal anchors in advance through clustering on the training set, and the optimal anchors obtained by this clustering method generally cannot cover all image detection boxes. For bridge detection, different bridge types and construction forms may make the specific characteristics of diseases vary greatly. The optimal anchors calculated on the training set are very likely to lack universality in application, resulting in poor detection accuracy of the trained model in application. Secondly, the anchoring mechanism increases the complexity of the detection head and the number of predictions for each image. Therefore, the disease detection model designed in this embodiment is a network model without an anchoring mechanism. The detection head of this network model without an anchoring mechanism adopts a per-pixel prediction method, directly outputting the classification probability and bounding box coordinates of the target.

[0026] The disease detection model includes a classification branch, a regression branch, and a centrality branch; among them, the classification branch includes 3×3 convolution, 1×1 convolution, and Sigmoid function activation, and is used to output the number of categories corresponding to the number of channels; the regression branch includes 3×3 and 1×1 convolution and a linear activation function, and is used to regress and predict the 4D coordinates (center point offset + width and height) of the bounding box; the centrality branch includes 1×1 convolution, and is used to perform centrality prediction to suppress low-quality edge predictions. After obtaining the visual detection image data collected by the drone, the classification branch in the disease detection model outputs the category of the disease, the regression branch outputs the center coordinates of the disease corresponding to the category and the position of the marked box boundary, and the centrality branch screens the results to eliminate overlapping marked boxes. After the above processing, a picture with disease marking results can be obtained.

[0027] After the disease detection model is established, a dataset is established for training the disease detection model. Taking the detection of coatings as an example, the pictures in the dataset of coating deterioration all come from in-service steel structures, including steel box girder bridges, steel truss bridges, and steel structure building venues. The number of images for each project is not less than 500, and the total number of images in the dataset is not less than 2,000. All images should be adjusted to a size similar to that of the drone camera, that is, 1920×1080 pixels, and manually marked to frame the location of each disease. After marking, the data is enhanced by adding random noise, random illumination, and random shadows to the images, and the dataset is extended to more than 8,000 images. Then the dataset is divided into a training set, a validation set, and a test set. The training set is used to train the network model without an anchor mechanism, and the validation set is used for validation. It is best to use the test set to test the validated model to obtain an optimized disease detection model.

[0028] Finally, the optimized disease detection model is transplanted into the on-board computer of the drone through an embedded program to directly process the video of the camera in real time to identify the apparent diseases of the bridge in real time.

[0029] Step 4: Control the drone to fly to the identified bridge disease, and attach it to the bridge disease location through the robotic arm on it, and use the robotic arm to perform contact internal flaw detection on the bridge to detect the internal disease conditions of the bridge; The stability of the contact state is the key to determining whether the internal flaw detection can be accurate. When performing internal flaw detection, the coating thickness at the bridge disease is mainly measured. Therefore, a detection probe is also set in the tactile feedback electromagnetic induction non-destructive measurement module. The detection probe is one of an ultrasonic probe, a magnetostrictive magnetization probe, a magnetic eddy current probe, and an electromagnetic ultrasonic probe. Since the detection probe is very small, once the robotic arm deviates or vibrates, it may lead to missed detection of diseases or large measurement errors. Therefore, the robotic arm is designed with a self-stabilizing control method for stable attachment, and the vertical lift of the drone is automatically controlled according to the pressure value collected by the tactile feedback electromagnetic induction non-destructive measurement module, so as to guide the drone to stably contact the bottom of the bridge. Specifically, the normal pressure of the contact surface between the robotic arm and the bridge is obtained in real time through the tactile sensor on the tactile feedback electromagnetic induction non-destructive measurement module at a sampling rate of 200 Hz. Since the vibration of the motor of the drone during operation will interfere with the pressure data collected by the tactile sensor and noisy pressure data will be collected. Therefore, in this embodiment, the Kalman filter algorithm is used to eliminate the vibration noise, and the elimination formula is: ; In the formula, is the pressure at time k, is the state transition matrix, is the external input under the control input matrix, is the process noise.

[0030] This operation can eliminate the noise caused by the vibration of the UAV itself during pressure acquisition. At the same time, the contact detection is mainly related to the pressure perpendicular to the bottom surface of the bridge. The pressure example is decomposed into the main lift component perpendicular to the bottom surface of the bridge and the interference component in the translation direction through moment decomposition. When the detected pressure fluctuation exceeds the threshold (in this embodiment, it is ±2N), the lift adjustment control response is triggered. In the lift adjustment link, the control response performs lift adjustment; during lift adjustment, the tactile feedback electromagnetic induction non-destructive measurement module outputs a lift control signal according to the pressure deviation. After receiving the lift control signal, the UAV adjusts the lift of its lift motor to achieve the stable attachment of the manipulator on the UAV.

[0031] The pressure adaptive control of the manipulator during the contact process realizes the stable maintenance of the UAV pressure by dynamically adjusting the lift through PID. Specifically, the tactile sensor continuously collects the current pressure and compares it with the set value (2N in this embodiment) to obtain an error signal, and combines the proportional, integral, and differential links to generate a control quantity. Among them, the proportional term quickly responds to the change of the error, the integral term eliminates the steady-state deviation and limits the integral accumulation amplitude to prevent overshoot, and the differential term predicts the pressure trend and suppresses the noise interference through filtering. The control quantity adjusts the lift in the reverse direction according to the positive and negative characteristics (that is, comparing the current pressure with the set value). When the pressure is greater than the set value, a negative control quantity is output, and the controller controls the lift motor to reduce the lift. When the pressure is less than the set value, a positive control quantity is output, and the controller controls the lift motor to increase the lift, thus forming a closed-loop feedback.

[0032] The calculation method of the above adaptive adjustment process is as follows: ; Wherein, is the lift motor control signal at time is time, is the proportional gain, is the integral gain, is the differential gain, is the error at time is the error at time is a temporary integral variable used to traverse the time points from 0 to .

[0033] The calculated by the above adaptive pressure control method is the control signal input to the lift motor controller of the UAV. The lift motor controller of the UAV receives this control signal and adjusts the lift of the lift motor to ensure the stability of the position, attitude and contact force of the UAV during contact detection, and avoid the deviation and detachment of the non-destructive testing probe.

[0034] When performing the internal information detection of fixed points, it is first necessary to determine the positions of the measurement points. The conventional detection method determines the positions of the samples to be inspected according to the drawings. However, internal damages generally occur at the positions where the apparent diseases occur. Therefore, in this embodiment, a phased detection method is adopted. In the first stage, the drone quickly flies and uses the on-board camera to detect the apparent diseases. The apparent diseases are automatically identified by the aforementioned disease detection model, and when a disease is identified, the positioning coordinates of the ultra-wideband wireless communication beacon positioning system of the drone at that moment are recorded. In the second stage, the drone flies to the identified bridge disease for fixed-point internal information measurement according to the coordinates recorded by aligning the timestamps of the aforementioned local positioning system. After the drone flies to one of the coordinate positions, the drone is manually controlled to rise and contact the bottom of the bridge. At this time, the tactile feedback electromagnetic induction non-destructive measurement module of the mechanical upper arm starts to work, enabling the drone to adaptively adjust the lift force and adsorb at the disease measurement position. At this time, the detection probe at the center of the robotic arm collects the coating thickness data to complete the internal information detection. After the detection is completed, the drone is manually controlled to descend and fly to the next coordinate position, repeating this operation.

[0035] An embodiment of the present invention also discloses a system for implementing the above-mentioned method for detecting internal diseases of bridges based on drones, including: A flight path planning module, configured to calculate the coordinates of the drone according to the local positioning system established by the local positioning beacons deployed around the bridge to be inspected; this module calculates the coordinates of the drone by self-calibrating the positioning beacon coordinates among the beacons deployed around the bridge to control the flight path of the drone; An image acquisition module, configured to obtain visual detection image data when the drone performs non-contact visual detection on the bridge during the flight inspection process controlled by the calculated coordinates; A disease recognition module, configured to construct a disease detection model for identifying bridge surface diseases based on deep learning, use the disease detection model to identify bridge diseases in the images obtained by visual detection, and then locate the positions of each disease on the bridge according to the recognition results; An internal disease detection module, configured to obtain the internal disease conditions of the bridge through contact internal flaw detection by the robotic arm when the drone flies to the identified bridge disease and attaches to the bridge disease position through its robotic arm.

[0036] The above are only the preferred embodiments of the present invention, and do not limit the implementation manners and protection scope of the present invention. For those skilled in the art, it should be realized that all the equivalent replacements and obvious changes made by using the content of the specification of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting internal diseases of bridges based on drones, characterized in that, It includes the following steps: Step 1: Deploy local positioning beacons around the bridge to be detected, establish a local positioning system, and calculate the coordinates of the drone; Step 2: Install a robotic arm on the drone, control the drone to fly for inspection according to the calculated coordinates, and perform non-contact visual inspection on the bridge during flight to obtain visual inspection image data; Step 3: Build a disease detection model for identifying bridge surface diseases based on deep learning, use the disease detection model to identify bridge diseases in the images obtained by visual inspection, and then locate the position of each disease on the bridge according to the identification results; Step 4: Control the drone to fly to the identified bridge disease, attach it to the bridge disease position through the robotic arm, and use the robotic arm to perform contact internal flaw detection on the bridge to detect the internal disease condition of the bridge.

2. The method for detecting internal diseases of a bridge based on an unmanned aerial vehicle according to claim 1, wherein, The specific implementation method of Step 1 is: Arrange multiple beacons around the bridge as anchor points. The positions of the beacons cover the area to be detected on the bridge. After arranging the beacons, self-calibrate between the beacons to locate the beacon coordinates; then the anchor coordinates of multiple beacons in space are known as (x 1 , y 1 , z 1 ),(x 2 , y 2 , z 2 ),…，(x n , y n , z n )，n is the total number of beacons, and the coordinates of the UAV to be calculated are (x t , y t , z t ), , and the straight-line distance between the beacon and the UAV is R i , i = 1~n , where the method of obtaining the UAV coordinates by beacon positioning is: 。 3. The method for detecting internal diseases of a bridge based on an unmanned aerial vehicle according to claim 1, wherein The deep learning model used in Step 3 is a network model without an anchor mechanism.

4. The method for detecting internal diseases of a bridge based on an unmanned aerial vehicle according to claim 3, wherein The disease detection model includes a classification branch, a regression branch, and a centerness branch; Among them, the classification branch includes 3×3 convolution, 1×1 convolution, and Sigmoid function activation, and is used to output the number of categories corresponding to the number of channels; The regression branch includes 3×3 and 1×1 convolution and a linear activation function, and is used to regress and predict the 4D coordinates of the bounding box. The 4D coordinates include the center point offset of the bounding box and the width and height; The centerness branch includes 1×1 convolution and is used for centerness prediction to suppress low-quality edge predictions.

5. The method for detecting internal diseases of a bridge based on an unmanned aerial vehicle according to claim 3, wherein Before predicting the images, first train the disease detection model; the training method includes: obtaining images related to the steel structure of the bridge to be detected from the existing database, adjusting all the images to a size similar to that of the drone camera, and performing marking to frame the position of each disease; after marking, perform data augmentation on the images and perform amplification to obtain a dataset; Divide the dataset into a training set, a validation set, and a test set, use the training set to train the disease detection model and use the validation set for validation, and preferably use the test set to test the validated model to obtain an optimized disease detection model.

6. The method for detecting internal diseases of a bridge based on an unmanned aerial vehicle according to claim 1, wherein, In Step 2, the robotic arm includes a base fixed on the drone, a plurality of cantilever legs hinged on the base at equal intervals in the circumferential direction, and a tactile feedback electromagnetic induction non-destructive measurement module fixed at the center position of the base. The cantilever leg includes an upper swing arm, a connecting rod, a lower swing arm, and a moving end support leg that is attached to the bridge surface during detection. Among them, the upper swing arm is also hinged on the base, and the upper swing arm, the connecting rod, the lower swing arm, and the moving end support leg are connected to form a quadrilateral.

7. The method for detecting internal diseases of a bridge based on an unmanned aerial vehicle according to claim 6, characterized in that, A damping spring is also provided on the cantilever leg. The damping spring is arranged on the diagonal of the cantilever leg. One end of the damping spring is fixed at the hinge point of the upper swing arm and the connecting rod, and the other end is fixed at the hinge point of the lower swing arm and the moving end support leg.

8. The method for detecting internal diseases of a bridge based on an unmanned aerial vehicle according to claim 6, characterized in that, In Step 4, the robotic arm is attached to the bridge disease through a self-stabilizing control method. The self-stabilizing control method includes: The tactile feedback electromagnetic induction non-destructive measurement module real-time obtains the normal pressure of the contact surface between the robotic arm and the bridge, and then uses the Kalman filter algorithm to eliminate vibration noise. The elimination process formula is: ； Among them, is the pressure after noise cancellation at time k, is the state transition matrix, is the external input under the control input matrix, is the process noise; The pressure moment after noise elimination is decomposed into a lift component perpendicular to the bottom surface of the bridge and a disturbance component in the translation direction. When the detected pressure fluctuation exceeds the threshold, a control response is triggered to adjust the lift. During lift adjustment, the tactile feedback electromagnetic induction non-destructive measurement module outputs a lift control signal based on the pressure deviation. After receiving the lift control signal, the drone adjusts the lift of the lift motor on it to achieve stable attachment of the robotic arm.

9. The method for detecting internal diseases of a bridge based on an unmanned aerial vehicle according to claim 8, characterized in that The method for adjusting the lift of the drone according to the pressure deviation includes: The tactile feedback electromagnetic induction non-destructive measurement module collects the current pressure in real time, compares it with the set value to obtain an error signal, combines the proportional, integral, and differential links to generate a control quantity, and adjusts the lift in the reverse direction according to the positive and negative characteristics of the control quantity. When the current pressure is greater than the set value, a negative control quantity is output to control the lift motor to reduce the lift. When the current pressure is less than the set value, a positive control quantity is output to control the lift motor to increase the lift, thus forming a closed-loop feedback; The calculation method for the above lift adjustment process is: ； Among them, is the lift motor control signal, is the time, is the proportional gain, is the integral gain, is the derivative gain, is the error at time is the error at time is a temporary integral variable used to traverse time points from 0 to ; The above calculated is the control signal input to the lift motor controller of the UAV. The lift motor controller of the UAV adjusts the lift of the lift motor according to the control signal to ensure the stability of the position, attitude and contact force of the UAV during contact detection.

10. A system for implementing the drone-based internal disease detection method for bridges described in any one of claims 1-9, characterized in that, Including: The route planning module is used to calculate the drone coordinates based on the local positioning system established by the local positioning beacons deployed around the bridge to be detected; The image acquisition module is used to obtain visual inspection image data during the non-contact visual inspection of the bridge when the drone controls the drone to fly for inspection according to the calculated coordinates; The disease identification module is used to construct a disease detection model for identifying bridge surface diseases based on deep learning, use the disease detection model to identify bridge diseases from the images obtained by visual inspection, and then locate the position of each disease on the bridge according to the identification results; The internal disease detection module is used to obtain the internal disease conditions of the bridge during the contact internal flaw detection of the bridge by the robotic arm when the drone flies to the identified bridge disease and attaches to the bridge disease position through the robotic arm on it.

Images