Automatic patrol device for detecting underwater pile quality based on unmanned ship carrying stereo camera

Abstract

This invention discloses an automatic inspection device for detecting underwater pile quality based on a stereo camera mounted on an unmanned surface vessel (USV). The automatic inspection device includes an underwater platform, an environmental perception module, an attitude control module, an image acquisition module, a digital image processing module, and a physical model image restoration module. This invention utilizes a stereo camera mounted on an USV to detect typical defects in underwater piles, such as concrete cracking, concrete spalling, and steel pipe corrosion. It is mainly used in various fields such as underwater pile foundation quality inspection during service and construction. It overcomes the shortcomings of existing underwater pile foundation concrete cracking, spalling, and steel pipe corrosion detection technologies, which rely too heavily on manual labor, have low detection efficiency, and low levels of intelligence, effectively improving the efficiency of underwater pile foundation quality inspection.

Description

Technical Field

[0001] This invention relates to a structure quality inspection device, specifically an automatic inspection device based on an unmanned surface vessel equipped with stereoscopic photography to detect the quality of underwater piles. Background Technology

[0002] Since most pile foundation structures, such as those at wharves, are located underwater, structural inspections are currently mostly conducted by divers. However, the safety of the inspection environment is affected by many factors, such as tidal and wave conditions, mooring facilities, and operational conditions. Based on this, research has been conducted on using unmanned surface vessels (USVs) for underwater pile foundation inspections, but challenges remain regarding USV operations in turbid waters. Therefore, there is an urgent need to develop an automated inspection device based on USVs equipped with stereo cameras to detect the quality of underwater piles. Summary of the Invention

[0003] To address the problems of low intelligence, poor detection efficiency, and low accuracy in the quality inspection of underwater pile foundations in turbid water, this invention provides an automated inspection device based on an unmanned surface vessel equipped with a stereo camera for detecting the quality of underwater piles. This device boasts high safety and intelligence, enabling all-weather inspections and significantly improving the efficiency of identifying potential quality hazards in underwater piles.

[0004] The objective of this invention is achieved through the following technical solution:

[0005] An automated inspection device for detecting the quality of underwater piles based on an unmanned surface vessel (USV) equipped with a stereo camera includes an underwater platform, an environmental perception module, an attitude control module, an image acquisition module, a digital image processing module, and a physical model image restoration module, wherein:

[0006] The underwater platform is used to carry an environmental perception module, an attitude control module, an image acquisition module, a digital image processing module, and a physical model image restoration module.

[0007] The environmental perception module is used to construct point cloud data of the space, obtain the three-dimensional scene and the three-dimensional spatial position of the unmanned vessel in a single state, and realize real-time three-dimensional reconstruction of the underwater scene.

[0008] The attitude control module is used to receive point cloud data from the environmental perception module, integrate the perceived attitude of itself, form a three-dimensional holographic motion environment digital map, realize attitude control during the unmanned surface vessel's movement, and ensure the unmanned surface vessel's stable movement.

[0009] The image acquisition module is used to capture images of pile foundations in turbid water, and to construct a degraded and diverse dataset based on the acquired images and the expanded data volume.

[0010] The digital image processing module is used to improve the image quality captured by the image acquisition module and restore underwater turbid environment images;

[0011] The physical model image restoration module is used to identify typical defects such as concrete cracks, concrete spalling, and steel pipe corrosion.

[0012] An automated inspection method for detecting the quality of underwater piles using an unmanned surface vessel equipped with a stereo camera, utilizing the aforementioned automated inspection device, includes the following steps:

[0013] Step S1: Design an inspection route based on the orientation of the wharf structure, and design an appropriate number of survey lines on the transverse side of the wharf structure according to the actual number of pile foundations.

[0014] Step S2: Activate the environmental perception module to recover the three-dimensional scene in a single state based on lidar, obtain the three-dimensional spatial position of the unmanned surface vessel, and provide basic data for underwater attitude control.

[0015] Step S3: Simultaneously activate the attitude control module, receive point cloud data obtained from the environmental perception module, fuse the perceived attitude of the unmanned surface vessel to form a three-dimensional holographic motion environment digital map, and adjust the propeller position and speed through the motion controller to achieve stable movement of the unmanned surface vessel.

[0016] Step S4: When the unmanned surface vessel reaches the designated survey line position, the image acquisition module is activated. First, the LED lighting system is turned on, and the shooting position and angle are adjusted by the robotic arm. As the unmanned surface vessel moves, the underwater high-definition stereo camera captures high-resolution images in real time and stores them in real time.

[0017] Step S5: After a set of images is acquired, the digital image processing module is started. First, the angle of the captured images is corrected. Then, the deep learning algorithm is improved by high computational precision to restore the highly degraded underwater images.

[0018] Step S6: Activate the physical model image restoration module to realize the detection of typical defect targets.

[0019] Compared with the prior art, the present invention has the following advantages:

[0020] This invention utilizes a stereo camera mounted on an unmanned surface vessel to detect typical defects in underwater pile concrete, such as cracking, spalling, and corrosion of steel pipes. It is mainly used in various fields such as underwater pile foundation quality inspection during service and construction. It solves the shortcomings of existing underwater pile foundation concrete cracking, spalling, and steel pipe corrosion detection technologies, which rely too heavily on manpower, have low detection efficiency, and low level of intelligence. It can effectively improve the efficiency of underwater pile foundation quality inspection. Attached Figure Description

[0021] Figure 1This is a schematic diagram of an automatic inspection device for detecting the quality of underwater piles based on an unmanned surface vessel (USV) equipped with a stereo camera. In the diagram, 1-USV platform, 2-robotic arm, 3-robotic arm extension rod, 4-lifting mechanism, 5-environmental perception module, 6-attitude control module, 7-LED lighting system, 8-underwater high-definition stereo camera, 9-digital image processing module, 10-physical model image restoration module, 11-wifi module antenna, 12-pluggable 5G card module, and 13-power module.

[0022] Figure 2 This is an example of environmental point cloud data collected in this embodiment.

[0023] Figure 3 The image shows a defect captured in the example.

[0024] Figure 4 The image shown is the restored image in the example.

[0025] Figure 5 The defective target body identified in the embodiment. Detailed Implementation

[0026] The technical solution of the present invention will be further described below with reference to the accompanying drawings, but it is not limited thereto. Any modifications or equivalent substitutions to the technical solution of the present invention that do not depart from the spirit and scope of the technical solution of the present invention should be covered within the protection scope of the present invention.

[0027] This invention provides an automatic inspection device for detecting the quality of underwater piles based on an unmanned surface vessel equipped with a stereo camera. This device is a quality inspection equipment assembled according to a specific design scheme, such as... Figure 1 As shown, it includes an underwater platform, an environmental perception module 5, an attitude control module 6, an image acquisition module, a digital image processing module 9, and a physical model image restoration module 10, among which:

[0028] The underwater platform is used to carry an environmental perception module 5, an attitude control module 6, an image acquisition module, a digital image processing module 9, and a physical model image restoration module 10.

[0029] Both the environmental perception module 5 and the attitude control module 6 provide services for the smooth movement of the unmanned surface vessel.

[0030] The environmental perception module 5 is used to construct point cloud data of the space, obtain the three-dimensional scene and the three-dimensional spatial position of the unmanned vessel in a single state, and realize real-time three-dimensional reconstruction of the underwater scene.

[0031] The attitude control module 6 is used to receive point cloud data from the environment perception module 5, and combine it with the perceived attitude of the unmanned surface vessel to form a three-dimensional holographic motion environment digital map, so as to realize attitude control during the movement of the unmanned surface vessel and ensure the smooth movement of the unmanned surface vessel.

[0032] The image acquisition module is used to capture images of pile foundations in turbid water, and based on the acquired images, it expands the data volume by means of rotation, flipping, random scaling and other methods to construct a degraded and diverse dataset;

[0033] The digital image processing module 9 is used to improve the image quality of captured images and restore underwater turbid environment images;

[0034] The physical model image restoration module 10 is used to improve the detection efficiency of typical defect targets and efficiently identify typical defects such as concrete cracks, concrete spalling, and steel pipe corrosion.

[0035] The Wi-Fi module antenna 11 is used to connect to the on-site Wi-Fi network to realize data transmission from the external hard drive;

[0036] The pluggable 5G card module 12 is used to obtain network signals through a 5G SIM card to achieve data transmission when there is no Wi-Fi available on site.

[0037] The power module 12 is used to display the power information of the unmanned surface vessel.

[0038] In this invention, the underwater platform mainly consists of an unmanned surface vessel platform 1, a robotic arm 2, a power and propeller system, an electrical control system, a Wi-Fi module antenna, a pluggable 5G card module, a power module, etc., and operates at a water depth of not less than 50 meters and has corrosion resistance (pH 5~9).

[0039] In this invention, the environment perception module 5 includes a lidar and a processor. The environment perception module 5 uses a lidar-based real-time 3D scene mapping algorithm to convert spatial 3D coordinates to pixel coordinates in the processor, thereby generating a depth information map in real time. It then combines the RGB color image and the depth map to construct a dense 3D point cloud, enabling 3D scene recovery in a single state and providing basic data for underwater attitude control.

[0040] In this invention, the attitude control module 6 includes a motion controller and a propeller. The attitude control module 6 is based on the particle filter positioning principle and works with the mounted motion controller to sense its own attitude in order to form a virtual digital twin model. It also incorporates point cloud data to form a three-dimensional holographic motion environment digital map to improve the unmanned surface vessel's adaptability to complex environments and its motion stability.

[0041] In this invention, the image acquisition module includes an LED lighting system 7 and an underwater high-definition stereo camera 8. The LED lighting system 7 is mounted on an unmanned surface vessel platform 1. The underwater high-definition stereo camera 8 is connected to the unmanned surface vessel platform 1 via a robotic arm 2 and can rotate. Its shooting range is 360 degrees. Under the condition of water turbidity ≤ 500 NTU, the underwater imaging distance is ≥ 30 cm, the imaging line of sight is ≥ 8.5 times the attenuation length, and the imaging frame rate is ≥ 20 frames, which can realize high-resolution image capture of target objects.

[0042] In this invention, both the LED lighting system 7 and the underwater high-definition stereo camera 8 serve image acquisition. The LED lighting system 7 consists of a light source and a reflector, and includes multiple optical lenses. The light is first focused into a beam by the reflector (curved mirror or spherical mirror), and then the projection is controlled by different positions and combinations of the optical lenses.

[0043] In this invention, the robotic arm 2 consists of a lifting mechanism 3 and a robotic arm extension rod 4. The lifting mechanism 3 is mounted on the unmanned surface vessel platform 1. One end of the robotic arm extension rod 4 is connected to the lifting mechanism 3, and the other end is connected to the underwater high-definition stereo camera 8. The lifting mechanism 3 controls the lifting and lowering of the base of the robotic arm 2, and the robotic arm extension rod 4 controls the underwater high-definition stereo camera 8 at the end, so that it maintains a suitable distance and angle with the plane to be measured.

[0044] In this invention, the digital image processing module 9 serves the restoration of underwater turbid environment images, improving the image quality of degraded images through deep learning methods. The digital image processing module 9 includes a built-in high-precision image quality improvement deep learning algorithm and chip. The method for improving the image quality of captured images is as follows: First, the tilt correction function is used to correct the angle of the captured turbid underwater pile foundation image, obtaining a frontal view of the turbid underwater pile foundation structure; second, based on the constructed degraded diverse dataset, quantitative indicators such as PSNR and SSIM representing degradation are obtained, and Monte Carlo integration is used for re-formulation. A basis residual network is used in the edge calculator to control the number of feature images. Subsequently, multiple feature images are fused using pixel rearrangement, removing chaotic interference noise during the process to achieve magnification effects of different magnifications, gradually restoring to the target resolution size. This high-precision image quality improvement deep learning algorithm achieves the restoration of highly degraded underwater images.

[0045] In this invention, the physical model image restoration module 10 serves to detect typical defect targets. Through repeated training on artificial datasets, it can efficiently identify typical defects such as concrete cracks, concrete spalling, and steel pipe corrosion. The traditional convolution method of the YOLO algorithm structure is improved to deformable convolution, making the sampling position more closely match the defect morphology, thereby improving the feature extraction capability of the network. After training, the camera system can use the above method to identify typical defect targets and can accurately identify typical defect targets without mistakenly detecting dirt or seams.

[0046] An automated inspection method for detecting the quality of underwater pile foundations using an unmanned surface vessel equipped with a stereo camera, comprising the following steps:

[0047] Step S1: Design an inspection route for the structure such as the wharf, and design an appropriate number of survey lines on the transverse side of the structure such as the wharf, based on the actual number of pile foundations and the layout.

[0048] Step S2: Activate the environmental perception module to recover the three-dimensional scene in a single state based on lidar, obtain the three-dimensional spatial position of the unmanned surface vessel, and provide basic data for underwater attitude control.

[0049] Step S3: Simultaneously activate the attitude control module, receive point cloud data acquired from the environmental perception module, and adjust the propeller position and speed via the motion controller to achieve stable movement of the unmanned surface vessel.

[0050] Step S4: When the unmanned surface vessel reaches the designated survey line position, the image acquisition module is activated. First, the LED lighting system is turned on, and the shooting position and angle are adjusted by the robotic arm. As the unmanned surface vessel moves, the underwater high-definition stereo camera captures high-resolution images in real time and stores them in real time.

[0051] Step S5: After acquiring a set of images, start the digital image processing module. First, correct the angle of the captured images. Then, use a high-precision image quality improvement deep learning algorithm to restore the highly degraded underwater images.

[0052] Step S6: Activate the physical model image restoration module to realize the detection of typical defect targets.

[0053] Example:

[0054] Taking a dock inspection task as an example, the environmental point cloud data map constructed by the environmental perception module is as follows: Figure 2 As shown; the defect image acquired by the image acquisition module is as follows. Figure 3 As shown; the image processed by the physical model image restoration module is as follows. Figure 4 As shown; the identified defective target body is as follows Figure 5 As shown.

Claims

1. An automatic inspection device for detecting the quality of underwater piles based on an unmanned surface vessel equipped with a stereo camera, characterized in that... The automatic patrol device includes an underwater platform, an environmental perception module, an attitude control module, an image acquisition module, a digital image processing module, and a physical model image restoration module, wherein: The underwater platform is used to carry an environmental perception module, an attitude control module, an image acquisition module, a digital image processing module, and a physical model image restoration module. The environmental perception module is used to construct point cloud data of the space, obtain the three-dimensional scene and the three-dimensional spatial position of the unmanned vessel in a single state, and realize real-time three-dimensional reconstruction of the underwater scene. The attitude control module is used to receive point cloud data from the environmental perception module, and combine it with the perceived attitude of the unmanned surface vessel to form a three-dimensional holographic motion environment digital map, so as to realize attitude control during the movement of the unmanned surface vessel and ensure the stable movement of the unmanned surface vessel. The image acquisition module is used to capture images of pile foundations in turbid water, and to construct a degraded and diverse dataset based on the acquired images and the expanded data volume. The digital image processing module is used to improve the image quality captured by the image acquisition module and restore underwater turbid environment images; The physical model image restoration module is used to identify typical defects such as concrete cracks, concrete spalling, and steel pipe corrosion. The digital image processing module includes a built-in high-precision image quality improvement deep learning algorithm and chip. The method for improving the image quality of the captured image is as follows: First, the tilt correction function is used to correct the angle of the captured underwater pile foundation image and obtain a front view of the underwater pile foundation structure. Second, based on the constructed deteriorated diverse dataset, PSNR and SSIM are obtained as quantitative indicators of deterioration. Monte Carlo integration is used for re-formulation, and a basis residual network is used in the edge calculator to control the number of feature images. Then, multiple feature images are fused by pixel rearrangement. During the process, chaotic interference noise is removed to achieve magnification effects of different magnifications, gradually restoring to the target resolution size. This high-precision image quality improvement deep learning algorithm achieves the restoration of highly deteriorated underwater images.

2. The automatic inspection device for detecting the quality of underwater piles based on an unmanned surface vessel equipped with a stereo camera, as described in claim 1, is characterized in that... The underwater platform is designed to operate at a depth of no less than 50 meters and is corrosion-resistant.

3. The automatic inspection device for detecting the quality of underwater piles based on an unmanned surface vessel equipped with a stereo camera, as described in claim 1, is characterized in that... The environment perception module includes a lidar and a processor. The environment perception module uses a lidar-based real-time 3D scene mapping algorithm to convert spatial 3D coordinates to pixel coordinates in the processor, thereby generating a depth information map in real time. It combines RGB color images and depth maps to construct a dense 3D point cloud, enabling 3D scene recovery in a single state and providing basic data for underwater attitude control.

4. The automatic inspection device for detecting the quality of underwater piles based on an unmanned surface vessel equipped with a stereo camera, as described in claim 1, is characterized in that... The attitude control module includes a motion controller and a propeller. Based on the particle filter positioning principle, the attitude control module, together with the onboard motion controller, senses its own attitude to form a virtual digital twin model; it also incorporates point cloud data to form a three-dimensional holographic motion environment digital map to improve the unmanned surface vessel's adaptability to complex environments and its motion stability.

5. The automatic inspection device for detecting the quality of underwater piles based on an unmanned surface vessel equipped with a stereo camera, as described in claim 1, is characterized in that... The image acquisition module includes an LED lighting system and an underwater high-definition stereo camera. The LED lighting system is mounted on the unmanned surface vessel (USV) platform, and the underwater high-definition stereo camera is connected to the USV platform via a robotic arm and can rotate to achieve high-resolution image capture of the target object.

6. The automatic inspection device for detecting the quality of underwater piles based on an unmanned surface vessel equipped with a stereo camera, as described in claim 5, is characterized in that... The underwater high-definition stereo camera has a 360-degree shooting range, and under the condition that the water turbidity is ≤500 NTU, the underwater imaging distance is ≥30cm, the imaging line of sight is ≥8.5 times the attenuation length, and the imaging frame rate is ≥20 frames.

7. The automatic inspection device for detecting the quality of underwater piles based on an unmanned surface vessel equipped with a stereo camera, as described in claim 5, is characterized in that... The robotic arm consists of a lifting mechanism and a robotic arm extension rod. The lifting mechanism is mounted on an unmanned surface vessel platform. One end of the robotic arm extension rod is connected to the lifting mechanism, and the other end is connected to an underwater high-definition stereo camera. The lifting mechanism controls the lifting and lowering of the robotic arm base, and the robotic arm extension rod controls the distance and angle between the underwater high-definition stereo camera and the plane to be measured.

8. An automatic inspection method for detecting the quality of underwater piles using the automatic inspection device according to any one of claims 1-7, characterized in that... The method includes the following steps: Step S1: Design an inspection route based on the orientation of the wharf structure, and lay out survey lines on the transverse side of the wharf structure according to the actual number of piles. Step S2: Activate the environmental perception module to recover the three-dimensional scene in a single state based on lidar, obtain the three-dimensional spatial position of the unmanned surface vessel, and provide basic data for underwater attitude control. Step S3: Simultaneously activate the attitude control module, receive point cloud data obtained from the environmental perception module, fuse the perceived attitude of the unmanned surface vessel to form a three-dimensional holographic motion environment digital map, and adjust the propeller position and speed through the motion controller to achieve stable movement of the unmanned surface vessel. Step S4: When the unmanned surface vessel reaches the designated survey line position, the image acquisition module is activated. First, the LED lighting system is turned on, and the shooting position and angle are adjusted by the robotic arm. As the unmanned surface vessel moves, the underwater high-definition stereo camera captures high-resolution images in real time and stores them in real time. Step S5: After a set of images is acquired, the digital image processing module is started. First, the angle of the captured images is corrected. Then, the deep learning algorithm with high computational precision image quality improvement is used to restore the highly degraded underwater images. Step S6: Activate the physical model image restoration module to realize the detection of typical defect targets.

Images