Flight hammering detection system for bridge steel structure bolt looseness and use method thereof

Abstract

The invention discloses a flight hammering detection system for bridge steel structure bolt looseness and a use method thereof, the flight hammering detection system at least comprises a computer device, an unmanned aerial vehicle remote controller, a flight mechanical arm device and a hammering detection device, the flight mechanical arm device at least comprises an unmanned aerial vehicle, a mechanical arm, a depth camera, a wireless transmission module and a flight mechanical arm controller; the hammering detection device is located on flying mechanical arm equipment and at least comprises a knocking hammer and an audio collector. The unmanned aerial vehicle remote controller is wirelessly connected with the wireless transmission module on the flying mechanical arm equipment, performs data interaction with the wireless transmission module and is used for commanding the flying work of the unmanned aerial vehicle; and the computer equipment performs data transmission with the wireless transmission module and is used for displaying the depth camera image. According to the invention, the flying mechanical arm can quickly fly against and stably contact with a difficult-to-reach area of a bridge steel structure, and the hammering detection procedure of bolt loosening is automatically completed.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of bridge steel structure bolt loosening safety detection, and mainly relates to a flying hammering detection system for bridge steel structure bolt loosening and a use method thereof. BACKGROUND

[0002] Modern bridges use a large number of steel structure bolts for connection, and the stability thereof directly affects the safety of the bridge. Long-term influence of factors such as wind load, traffic vibration and temperature change causes the bolts to be prone to loosening or even falling off, thereby causing structural hazards or even safety accidents. Bridge steel structure bolt loosening detection currently mainly relies on manual climbing for hammering detection. Due to the difficulty in reaching the detection area and the large number of bolts, there are problems such as high operation risk, low detection efficiency and heavy workload. SUMMARY

[0003] The present application is aimed at the deficiencies in the prior art, and provides a flying hammering detection system for bridge steel structure bolt loosening and a use method thereof, which at least comprises a computer device, a UAV remote controller, a flying mechanical arm device and a hammering detection device. The flying mechanical arm device at least comprises a UAV, a mechanical arm, a depth camera, a wireless transmission module and a flying mechanical arm controller. The hammering detection device is located on the flying mechanical arm device and at least comprises a knocking hammer and an audio collector. The UAV remote controller is wirelessly connected with the wireless transmission module on the flying mechanical arm device and performs data interaction with the wireless transmission module, and is used for commanding the flight work of the UAV. The computer device performs data transmission with the wireless transmission module and is used for displaying the depth camera image. The present application can realize fast flying and stable contact of the flying mechanical arm to the hard-to-reach area of the bridge steel structure, and automatically complete the hammering detection procedure of bolt loosening.

[0004] In order to achieve the above-mentioned purpose, the technical scheme adopted by the present application is as follows: a flying hammering detection system for bridge steel structure bolt loosening, at least comprising a computer device, a UAV remote controller, a flying mechanical arm device and a hammering detection device,

[0005] The flying mechanical arm device at least comprises a UAV, a mechanical arm, a depth camera, a wireless transmission module and a flying mechanical arm controller, wherein,

[0006] The unmanned aerial vehicle front end is provided with a magnetic adsorption type protection frame, and a depth camera is located below the magnetic adsorption type protection frame and is used to acquire flight trajectory images and bridge steel structure images of the unmanned aerial vehicle; the wireless transmission module is located on the top of the unmanned aerial vehicle and is used to transmit flight information of the unmanned aerial vehicle, image information of the depth camera and hammering information of the hammering detection device; the flight mechanical arm controller is located at the core part of the center of the unmanned aerial vehicle and is used to control and adjust the movement of the unmanned aerial vehicle during flight; the mechanical arms are two and are located at the front end and the rear end of the unmanned aerial vehicle respectively, the front mechanical arm located at the front end of the unmanned aerial vehicle is used to carry and move an audio collector, and the rear mechanical arm located at the rear end of the unmanned aerial vehicle is used to carry and drive a hammering device.

[0007] The hammering detection device is located on the flight mechanical arm equipment and at least includes a knocking hammer and an audio collector, the knocking hammer is connected with the mechanical arm through a fixing piece and is fixed at the tail end of the rear mechanical arm of the unmanned aerial vehicle and is used to hammer the bolt to be detected, and the audio collector is installed at the tail end of the front mechanical arm of the unmanned aerial vehicle through a fixed jaw, audio signals generated by hammering are collected through the audio collector, and the audio signals are sent to the wireless transmission module.

[0008] The unmanned aerial vehicle remote controller is wirelessly connected with the wireless transmission module on the flight mechanical arm equipment and performs data interaction with the wireless transmission module and is used to command the flight work of the unmanned aerial vehicle.

[0009] The computer equipment performs data transmission with the wireless transmission module and is used to display the depth camera images.

[0010] As an improvement of the present application, the hammering detection device further includes a driving motor and a pressure sensor, the driving motor is connected with the knocking hammer in the form of a mortise and tenon structure and serves as an energy source of hammering movement, and the pressure sensor is built in the tail end of the driving motor and is used to output the stress and load conditions of the hammering device.

[0011] As an improvement of the present application, the flight mechanical arm equipment further includes a laser radar, the laser radar utilizes the characteristic of constant light speed, obtains distance information by emitting laser pulses and accurately measuring the flight time of laser to and fro the target, obtains three-dimensional space coordinate points of the target surface in combination with the scanning direction of the laser beam, and a large number of point coordinates are combined to form a three-dimensional point cloud map.

[0012] The system adopts double mechanical arms for operation, in a single arm system, the next action must wait before the completion of one action, the double arm design can realize parallel operation, the double arms can simultaneously perform pose adjustment, the work efficiency is greatly improved, the hammering and signal collection are divided into two independent parts for processing, the error is reduced and the stability is improved.

[0013] In order to achieve the above object, the technical scheme adopted by the present application is: the use method of the flying hammer detection system for the loosening of bridge steel structure bolts, at least comprising the following steps:

[0014] S1: the flying mechanical arm device provided with a unmanned aerial vehicle is controlled by a unmanned aerial vehicle remote controller to fly to the lower side of the bridge steel structure;

[0015] S2: the depth camera in the flying mechanical arm device acquires the image of the lower side of the bridge steel structure and transmits the image to the computer device, and the operator controls the movement of the unmanned aerial vehicle according to the image information, when the magnetic adsorption type protection frame at the front end of the unmanned aerial vehicle is in parallel contact with the bridge steel structure, the power supply system of the magnetic adsorption type protection frame is started, so that the unmanned aerial vehicle and the bridge steel structure are magnetically adsorbed and adhered, and the unmanned aerial vehicle is relatively stationary to the bridge steel structure;

[0016] S3: after the unmanned aerial vehicle is completely stabilized, the depth camera carried by the fuselage starts to work, the generated left and right images of the same target in the camera are similar to human eyes, and the position difference of the same target in the generated left and right images is inversely proportional to the distance of the target; the depth camera directly learns pixel-level disparity mapping through a network, and utilizes a disparity-depth formula:

[0017]

[0018] The relative distance between the mechanical arm and the detection target is obtained; in the formula, Z is the distance from the target to the camera, f is the focal length of the camera, which can be obtained through camera calibration; B is the baseline length of the binocular camera, which is equivalent to the horizontal distance between the left and right optical centers of the camera; d is the parallax, that is, the horizontal coordinate difference of the target in the left and right images; it is assumed that the pixel coordinates of the target in the left image are (u, v), and the corresponding depth is Z, and the camera perspective projection formula is utilized:

[0019]

[0020] Through the coordinates (u, v) of the target in the image and the corresponding depth Z, combined with the camera intrinsic matrix K, the two-dimensional pixel coordinates are converted into the coordinates (X, Y, Z) in the three-dimensional camera coordinate system, and in the formula, f x ,f y are the focal lengths of the camera in the x and y axes, reflecting the zooming capability of the camera to the image; c x , c y are the pixel coordinates of the optical center of the camera in the image, that is, the offset of the image center point. The intrinsic matrix can be directly obtained in camera calibration;

[0021] The front mechanical arm with an audio acquisition device carries the audio acquisition device at the tail to a place with a horizontal distance of 5CM from the bolt to be detected according to the relative coordinates of the bolt to be detected transmitted by the depth camera;

[0022] S4: The rear mechanical arm with a hammering device carries the end of the knocking device to the bolt to be detected according to the relative coordinates of the bolt to be detected transmitted by the depth camera, and based on the piezoelectric effect, when stressed, the positive and negative charge centers in the force sensor shift, and an equal amount of opposite charges Q are generated on the surface

[0023] Q = d ij × F

[0024] In the above formula, Q is the amount of charge generated, d ij is the piezoelectric constant, and F is the contact force; the charge is converted into a voltage signal through an amplification circuit, and the voltage is proportional to the force; the charge amplifier formula is:

[0025]

[0026] V is the output voltage, and C is the equivalent capacitance, including the sensor capacitance, cable capacitance, and amplifier input capacitance; when the force sensor output voltage meets the set initial value, the system determines that the hammering device has reached the detection position, and the two mechanical arms are locked and remain relatively stationary with the bolt to be detected. After the hammering device is calibrated, the hammering detection begins;

[0027] S5: The audio acquisition device receives the audio signal generated by the knocking hammer, performs time domain analysis on the vibration decay time, and performs frequency domain analysis on the bolt vibration frequency. When the bolt is tightened, it is equivalent to a "rigid-elastic" system, and the vibration frequency is high and decays quickly after knocking. When it is loose, the connection stiffness decreases, the vibration frequency decreases, and the decay slows down. Its simplified vibration equation is:

[0028]

[0029] In the above formula, m is the equivalent mass of the bolt-connector, c is the damping coefficient, which increases due to increased contact friction when loose, k is the system stiffness, which decreases when loose, x is the vibration displacement, and F(t) is the knocking impact force (N). The main peak frequency of the tightened bolt is high, and when it is loose, the main peak frequency shifts to low frequency. After filtering the collected signals, they are imported into the set deep learning system, the system extracts features, uses neural networks for comparison and calculation, and compares the features of the measured signal with the features of the pre-set non-loose bolt signal. The system outputs the loosening detection data of the bolt to be detected. Each bolt is detected by knocking 5 times with intervals, and after the detection is completed, the system feeds back the final result of whether the bolt to be detected is loose to the terminal.

[0030] S6: After obtaining the bolt detection data, the detection is completed, the operator turns off the power supply system of the magnetic attraction type protective shell, and operates the flying mechanical arm device to return.

[0031] Compared with the prior art, the present application has the beneficial effects: the present application provides a flying hammer detection system for bolt loosening of a bridge steel structure and a method for using the same, the flying mechanical arm is remotely controlled on the ground to quickly cover the high-altitude difficult-to-reach area of the bridge steel structure, and the bolt loosening is comprehensively detected based on the hammering method, without the need for workers to climb up to manually hammer and detect, thereby reducing the risk and workload of high-altitude operation of personnel and improving the operation efficiency of bolt loosening detection. The present application fully combines binocular vision with unmanned aerial vehicle bolt detection, significantly improving the rate and accuracy of bolt detection. The present application adopts double mechanical arms instead of a single mechanical arm for operation, the double-arm design can realize parallel operation, the two mechanical arms can simultaneously adjust the pose, greatly improving the work efficiency, and the hammering and signal acquisition are divided into two independent parts for processing, avoiding mutual interference between them. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 It is a structural framework schematic diagram of the bolt loosening detection system of the present application.

[0033] Figure 2 It is a working scene schematic diagram of the flying mechanical arm equipment of the present application.

[0034] Figure 3 It is a schematic diagram of the end effector mechanical structure of the working platform in the flying mechanical arm equipment of the present application.

[0035] Figure 4 It is a processing and treatment schematic diagram of the flying mechanical arm equipment of the present application after collecting audio signals.

[0036] Figure 5 It is a bolt loosening detection step flow schematic diagram of the present application.

[0037] In the figure: 1, computer equipment; 2, unmanned aerial vehicle remote controller; 101, unmanned aerial vehicle protection frame; 102, magnetic adsorption device; 103, depth camera; 104, audio collector; 105, hammering device; 106, driving motor; 107, laser radar; 108, unmanned aerial vehicle landing gear; 109, unmanned aerial vehicle power supply system; 110, wireless transmission module; 111, flying mechanical arm controller; 121, front mechanical arm; 122, rear mechanical arm; 201, bridge steel structure; 202, bolt to be detected; 203, flying mechanical arm equipment; 204, unmanned aerial vehicle; 303, hammering detection device; 304, knocking hammer; 305, pressure sensor. DETAILED DESCRIPTION

[0038] The present application will be further illustrated in conjunction with the drawings and specific embodiments, and it should be understood that the following specific embodiments are only used to illustrate the present application and not to limit the scope of the present application.

[0039] Example 1

[0040] A flying hammer detection system for bolt loosening of bridge steel structure, as shown in Figure 1 The whole system includes a computer device 1, a UAV remote controller 2, a flying mechanical arm device 203, and a hammer detection device 303. The flying mechanical arm device 203 at least includes the following parts: a UAV protection frame 101, a magnetic adsorption device 102, a depth camera 103, an audio collector 104, a hammering device 105, a driving motor 106, a laser radar 107, a UAV landing gear 108, a UAV power supply system 109, a wireless transmission module 110, and a flying mechanical arm controller 111. The flying mechanical arm 203 uses the UAV 204 as a mobile platform, and uses the audio collector 104 and the hammering device 105 to complete the bolt loosening detection of the bridge steel structure 201.

[0041] The UAV 204 is provided with a UAV protection frame 101 and a magnetic adsorption device 102 at the front end, and the depth camera 103 is located below the UAV protection frame 101, which is used to obtain the flight trajectory image of the UAV 204 and the image of the bridge steel structure 201. The wireless transmission module 110 is located at the top of the UAV 204, which is used to transmit the flight information of the UAV 203, the image information of the depth camera 103, and the hammering information of the hammering detection device 303. The flying mechanical arm controller 111 is located at the center of the UAV 204, which is used to control the moving trajectory of the UAV 204 during flight. The mechanical arm is provided with two, which are respectively located at the front end and the rear end of the UAV 204. The front mechanical arm 121 located at the front end of the UAV 204 is used to carry and move the audio collector 104, and the rear mechanical arm 122 located at the rear end of the UAV 204 is used to carry and drive the hammering detection device 303.

[0042] The hammering detection device 303 is located on the flying mechanical arm device 203, which includes a knocking hammer 304 and an audio collector 104. The knocking hammer 304 is connected with the rear mechanical arm 122 through a fixing part, which is used to hammer the bolt 202 to be detected. The audio collector 104 is installed at the end of the front mechanical arm 121 through a fixed jaw, which collects the audio signal generated by hammering and sends the audio signal to the wireless transmission module 110.

[0043] The UAV remote controller 2 is wirelessly connected with the wireless transmission module 110 on the flying mechanical arm device 203, and performs data interaction with the wireless transmission module 110, which is used to command the flight work of the UAV 204.

[0044] The hammering device 105 serves as the core execution unit of the bolt detection system, and its design requirements include high-precision excitation, controllable energy output, stable contact feedback, and other stringent conditions. The device adopts a motor-driven scheme, consisting of a driving module, a control module, and a sensing module. The subsystems are interconnected through a real-time data bus, forming a closed-loop control system. The overall initialization process of the hammering device 105 is as follows: During the system warm-up phase, the capacitor bank charges the system for hammering motion in constant current mode. Voltage ripple control is used, and multi-stage LC filtering is adopted to strictly control the energy size of a single hammering. After the initialization of the above power supply, the system enters the self-checking phase. The pressure sensor 305 performs zero-point calibration and performs multiple self-checks under no-load conditions to ensure hammering accuracy and efficiency. After that, the electromagnetic coil impedance test is performed to measure the inductance and resistance values when the system transmits energy to evaluate the energy output efficiency. Finally, the joint encoder verification is performed, and each axis is reset to zero.

[0045] Embodiment 2

[0046] The flying hammering detection system for bolt loosening of bridge steel structure as described in embodiment 1 uses the method as shown in the working scene diagram of the flying mechanical arm device 203 Figure 2 , Figure 3 is a schematic diagram of the end effector mechanical structure of the working platform in the flying mechanical arm device 203, showing the pose state of the actuator during actual detection. Figure 4 is a processing and treatment diagram of the audio signal collected by the flying mechanical arm device 203 of the present application, and Figure 5 is a schematic diagram of the bolt loosening detection step flow, which specifically includes the following steps:

[0047] S1, the operator sets up a workbench within 30 meters of the bridge steel structure, performs equipment safety detection, and tests the data transmission of the wireless transmission module 110 of the unmanned aerial vehicle 204. After confirming the stability and real-time performance of the image data and audio data transmission, the motion planning test of the front mechanical arm 121 and the rear mechanical arm 122 is performed to evaluate and adjust their motion space, check whether the motion is smooth and whether there is stuttering and emergency stop, and ensure that the operation space of the front mechanical arm 121 and the rear mechanical arm 122 is the lower cone area of the unmanned aerial vehicle protection frame 101.

[0048] S2, after confirming that all instruments enter standby state, the operator controls the flight mechanical arm device 203 to take off through the unmanned aerial vehicle remote controller 2, after normal ascending, the laser radar 107 starts to emit short pulse laser, carries out ranging, and adjusts the laser direction in real time to generate sufficient mapping data. The front end of the laser radar 107 carries out real-time data preprocessing and point cloud generation, converts the measured data polar coordinates into 3D Cartesian coordinates, and carries out basic noise reduction and filtering, finally uses the SLAM algorithm to locate the flight mechanical arm device 203 position in real time locally, and composes the point cloud data into an online map to assist the operator to operate the flight mechanical arm device 203, so that it moves to the corresponding bridge steel structure 201. Collect the image data corresponding to the position, the wireless transmission module 110 returns the information, the operator selects the appropriate steel structure contact surface according to the image, and operates the unmanned aerial vehicle 204 close to the contact surface. When the unmanned aerial vehicle protection frame 101 in front of the unmanned aerial vehicle 204 is 10 cm away from the contact surface, the magnetic adsorption device 102 is started, and the magnetic adsorption device 102 starts to work, and the unmanned aerial vehicle 204 is adsorbed on the steel structure, providing a stable working environment for hammering detection. The depth camera 103 directly learns pixel-level disparity mapping through the network, and uses the disparity-depth formula:

[0049]

[0050] The relative distance between the mechanical arm and the detection target is obtained. In the formula, Z is the distance from the target to the camera, f is the focal length of the camera, which can be obtained by camera calibration. B is the baseline length of the binocular camera, which is equivalent to the horizontal distance between the left and right optical centers of the camera. D is the parallax, that is, the horizontal coordinate difference of the target in the left and right images. It is assumed that the pixel coordinates of the target in the left image are (u, v), and the corresponding depth is Z, and the camera perspective projection formula is used:

[0051]

[0052] Through the coordinates (u, v) of the target in the image and the corresponding depth Z, combined with the camera intrinsic matrix K, the two-dimensional pixel coordinates are converted into three-dimensional camera coordinates (X, Y, Z). In the formula, f x ,f y is the focal length of the camera in the x and y axes, reflecting the zooming capability of the camera to the image. c x , c y is the pixel coordinate of the camera optical center in the image, that is, the image center point offset. The intrinsic matrix can be directly obtained in camera calibration.

[0053] The mechanical arm with the audio collector 104 carries the end audio collector 104 to a place with a horizontal distance of 5 cm from the bolt to be detected according to the relative coordinates of the bolt to be detected transmitted by the depth camera 103.

[0054] The coordinate positioning in this step uses a binocular depth camera to collect position data. The binocular camera directly calculates the depth map in real time through parallax, uses the contrast between left and right images, reduces the feature loss problem caused by sudden changes in light (such as shadows and strong light), and also has good performance in low-texture scenes. The combination of binocular vision and unmanned aerial vehicle bolt detection can significantly improve the bolt detection rate and accuracy.

[0055] S3, the mechanical arm with the hammering device 105 transports the knocking hammer 304 at the end to the bolt 202 to be detected according to the relative coordinates of the bolt to be detected transmitted by the depth camera 103, and based on the piezoelectric effect, when stressed, the positive and negative charge centers in the force sensor shift, and an equal amount of opposite charges Q

[0056] Q=d ij ×F

[0057] In the above formula, Q is the amount of charge generated, d ij is the piezoelectric constant, and F is the contact force. The charge is converted into a voltage signal through an amplification circuit, and the voltage is proportional to the force. The charge amplifier formula is:

[0058]

[0059] V is the output voltage, and C is the equivalent capacitance, including the sensor capacitance, cable capacitance, and amplifier input capacitance. When the force sensor output voltage meets the set initial value, the system determines that the hammering device 105 has reached the position to be detected, and the two mechanical arms are locked and remain relatively stationary with the bolt 202 to be detected. After the hammering device is calibrated, the system starts the hammering detection with an output force of 5N.

[0060] In this step, there may be multiple bolts to be detected in the image area fed back by the depth camera 103. The unmanned aerial vehicle control system can automatically identify all the bolts 202 to be detected in the area and mark them one by one according to the input image, and provide the optimal detection path according to the difference between the coordinates of each bolt.

[0061] S4, the audio collector 104 receives the audio signal generated by the hammering device 105, performs time domain analysis on the vibration decay time, and performs frequency domain analysis on the bolt vibration frequency. When the bolt is tightened, it is equivalent to a "rigid-elastic" system, and the vibration frequency is high and the decay is fast after knocking. When loose, the connection stiffness decreases, the vibration frequency decreases, and the decay slows down. The simplified vibration equation is:

[0062]

[0063] In the formula, m is the equivalent mass of the bolt-connection, c is the damping coefficient which increases due to increased contact friction when loose, k is the system stiffness which decreases when loose, x is the vibration displacement, and F(t) is the knocking impact force (N). The frequency spectrum of the fastened bolt has a high main peak frequency, and the main peak frequency shifts to a low frequency when loose. The collected signals are filtered and input into the set deep learning system, the system extracts features, and the neural network is used to compare and calculate the time domain features and the frequency domain features respectively.

[0064] In the process of time domain feature extraction, the "zero-crossing rate (ZCR)" is used as the extraction index.

[0065]

[0066] In the formula, x i is the amplitude of the i-th sampling point, N is the number of sampling points, sign(x) is the sign function (1 when x>0 and -1 when x<0). ZCR is the number of times the signal crosses zero in a unit of time. Since the zero-crossing rate of high-frequency signals is often higher, the signal with a high zero-crossing rate is given a weight of 0.05 and the signal with a low zero-crossing rate is given a weight of 0.95 in the deep learning process, so that the system can quickly and accurately determine whether the bolt 202 under test has a main peak frequency shift to a low frequency due to loosening.

[0067] The frequency domain features are converted from the time domain signal to a "time-frequency" two-dimensional spectrum by short-time Fourier transform:

[0068]

[0069] w(n-t) is the Hanning window function, which divides the long signal into short-time frames to avoid spectral blurring, and X(t,f) is the complex amplitude of frequency f at time t. Then, the Mel scale is used:

[0070]

[0071] The original spectrum is converted to a feature that conforms to human auditory characteristics, reducing the dimension and retaining key information. In the CNN-LSTM-based neural network model, the obtained Mel(f) extracts its local frequency domain features through a convolution kernel, and the converted spectrum is optimized. The CNN layer of the neural network captures abnormal frequency patterns in the spectrum. The LSTM layer processes time series features, memorizes long-term dependencies through a gating mechanism, and captures abnormal rhythm mutations in the time domain.

[0072] In the process of model training of deep learning, a plurality of specifications of bolt knock audio are used as a data set to depict specific differences between signals, a frequency domain interval of 2kHz-8kHz is selected as a normal interval, a frequency of 0.8 is given to the interval, a low weight of 0.2 is given to the frequency outside the interval. The frequency spectrum generated by the unloosen bolt knock has a higher inherent frequency, and the frequency distribution is relatively concentrated, the bandwidth in the frequency spectrum is relatively narrow, in addition to the fundamental frequency, clear high-order harmonics will appear in the frequency spectrum, and the harmonic amplitude will decrease regularly with the increase of frequency. The energy proportion of the low frequency band (usually <1kHz) in the frequency spectrum is low, there is no messy low frequency peak or wide frequency noise. The neural network learns the mode of these characteristics, realizes the binary classification of "normal / abnormal" at the output end, and finally completes the output of the bolt looseness result through the above set index.

[0073] Each bolt is detected by 5 hammering intervals, and after the detection is completely finished, the system feedbacks the terminal with the final result of whether the bolt to be detected 202 is loose.

[0074] Compared with the traditional manual detection relying on workers to make subjective judgments, the bolt looseness detection method adopts a deep learning method, uses a computer to make objective and accurate system analysis, reduces the unstable factors generated by manual work, unifies the detection standard of bolt looseness, ensures that the detection standard is unchanged in multiple detections, improves the detection accuracy, and reduces the detection error.

[0075] In order to ensure the completeness and accuracy of audio acquisition, the audio collector 104 carried by the mechanical arm starts to collect audio several seconds before the hammering starts. In the later data processing, a plurality of filtering methods are used to reduce the influence of external environment on the obtained data.

[0076] S5, after the hammering detection is completely finished, the operator controls the unmanned aerial vehicle to return, and the operation is finished.

[0077] It should be noted that the above content only illustrates the technical idea of the present application, and cannot limit the protection scope of the present application. For ordinary skilled persons in the technical field, without departing from the principle of the present application, a plurality of improvements and refinements can be made, which fall within the protection scope of the claims of the present application.

Claims

1. A flying hammer detection system for bolt loosening in a bridge steel structure, characterized by: At least comprising a computer device, a UAV remote controller, a flying mechanical arm device and a hammering detection device, The flying mechanical arm device at least comprises a UAV, a mechanical arm, a depth camera, a wireless transmission module and a flying mechanical arm controller, wherein, The front end of the UAV is provided with a UAV protection frame and a magnetic adsorption device, and the depth camera is located below the UAV protection frame and used to acquire flight trajectory images and bridge steel structure images of the UAV; the wireless transmission module is located on the top of the UAV and used to transmit flight information of the UAV, image information of the depth camera and hammering information of the hammering detection device; the flying mechanical arm controller is located in the center of the UAV and used to control the moving trajectory of the UAV when flying; the mechanical arm is provided with two, respectively located at the front end and the rear end of the UAV, the front mechanical arm at the front end of the UAV is used to carry and move an audio collector, and the rear mechanical arm at the rear end of the UAV is used to carry and drive the hammering detection device; The hammering detection device is located on the flying mechanical arm device and at least comprises a hammering device and an audio collector; the hammering device is provided with a knocking hammer, the knocking hammer is connected with the mechanical arm through a fixing piece and fixed at the tail end of the rear mechanical arm of the UAV and used to hammer the bolt to be detected; the audio collector is installed at the tail end of the front mechanical arm of the UAV through a fixing jaw, collects audio signals generated by hammering and sends the audio signals to the wireless transmission module; The UAV remote controller is wirelessly connected with the wireless transmission module on the flying mechanical arm device and performs data interaction with the wireless transmission module, and is used to command the flight work of the UAV; The computer device performs data transmission with the wireless transmission module and is used to display the depth camera images.

2. The flying hammer strike detection system for bolt loosening in bridge steel structures as claimed in claim 1 wherein: The hammering detection device further comprises a driving motor and a pressure sensor, the driving motor is connected with the knocking hammer in a mortise and tenon structure; the pressure sensor is built-in at the front end of the driving motor and used to output the stress and load of the hammering device.

3. The flying hammer strike detection system for bolt loosening in bridge steel structures as claimed in claim 1 wherein: The flying mechanical arm device further comprises a laser radar, the laser radar obtains distance information by emitting laser pulses and measuring the flight time of laser returning to the target, obtains three-dimensional space coordinate points of the target surface in combination with the scanning direction of the laser beam, and a large number of point coordinate sets constitute a three-dimensional point cloud map.

4. The method of claim 1, wherein the method further comprises: At least comprising the following steps: S1: controlling the flying mechanical arm device provided with the UAV through the UAV remote controller, flying the flying mechanical arm device to below the bridge steel structure; S2: the depth camera in the flying mechanical arm device acquires images below the bridge steel structure and transmits the images to the computer device, the operator controls the UAV movement according to the image information, when the magnetic adsorption type protection frame at the front end of the UAV is in parallel contact with the bridge steel structure, the power supply system of the magnetic adsorption type protection frame is started, so that the UAV and the bridge steel structure are in magnetic adsorption type adhesion, and the UAV is relatively stationary to the bridge steel structure; S3: After the UAV is stabilized, the depth camera carried by the fuselage learns pixel-level disparity mapping to obtain the relative distance between the mechanical arm and the detected target; through the coordinates of the detected bolt in the image and the corresponding depth, combined with the camera intrinsic matrix, the two-dimensional pixel coordinates are converted into three-dimensional camera coordinates; the front mechanical arm with an audio acquisition device carries the audio acquisition device at the end to a place 5 cm horizontally away from the detected bolt according to the relative coordinates of the detected bolt transmitted by the depth camera; S4: The rear mechanical arm with a hammering device carries the knocking hammer at the end to the detected bolt according to the relative coordinates of the detected bolt transmitted by the depth camera in step S3; when the output voltage of the force sensor meets the set initial value, the system determines that the hammering device has reached the detected position, and the two mechanical arms are locked and kept relatively stationary with the detected bolt, and the hammering detection starts; S5: The audio acquisition device receives the audio signal generated by the knocking hammer, performs time domain analysis and feature extraction on the vibration attenuation time, and performs frequency domain analysis and feature extraction on the bolt vibration frequency; the vibration equation is: In the above formula, m is the equivalent mass of the bolt-connector, c is the damping coefficient, k is the system stiffness, x is the vibration displacement, and F(t) is the knocking impact force (N); after the collected signal is filtered, it is imported into the set deep learning system, the system performs feature extraction, and the neural network is used for comparison calculation; the features of the detected signal and the features of the pre-set non-loose bolt signal are compared, and the loosening detection data of the detected bolt is output; the system feeds back the final result of whether the detected bolt is loose to the terminal; S6: After the bolt detection is completed, the operator turns off the power supply system of the magnetic attraction type protective shell and controls the flight mechanical arm device to return.

5. The method of using a flying hammer strike detection system for bolt loosening in a bridge steel structure as claimed in claim 4 wherein: In step S3, pixel-level disparity mapping is learned by using a disparity-depth formula, and the formula is specifically: In the formula, Z is the distance from the target to the camera, f is the focal length of the camera, B is the baseline length of the binocular camera, and d is the horizontal coordinate difference of the target in the left and right images; In step S5, in the process of time domain feature extraction, the "zero-crossing rate ZCR" is used as an extraction index: Wherein, (u, v) is the pixel coordinate of the bolt to be detected in the image, Z is the corresponding depth, f x ,f y The focal length of the camera in the x, y axis, c x , c y The pixel coordinate of the camera optical center in the image.

6. The method of using a flying hammer strike detection system for bolt loosening in a bridge steel structure as claimed in claim 4 wherein: The frequency domain feature converts the time domain signal into a "time-frequency” two-dimensional spectrum by short-time Fourier transform: wherein x i is the amplitude of the i-th sample point, N is the number of sample points, sign(x) is a sign function, and ZCR is the number of times the signal crosses zero in a unit of time. In the formula, w(n-t) is the Hanning window function, indicating the system weight under the condition that the window length is n at time t, X(t,f) is the complex amplitude of frequency f at time t, and the original spectrum is converted by using the Mel scale, and the Mel scale is specifically: In the formula, f is the frequency at time t. ​

Images