Offshore wind power pile scouring monitoring and repairing system and method based on underwater robot

Abstract

The application discloses a kind of offshore wind power pile scouring monitoring repair system and method based on underwater robot, the system includes tracked underwater robot and shore base end, tracked underwater robot and shore base end are connected by zero buoyancy cable;Tracked underwater robot includes waterproof electronic cabin, power module, sensor module, operation module and illuminating lamp;Shore base end includes shore base end power carrier communication module and display screen, and shore base end power carrier communication module and display screen are connected by network cable;Waterproof electronic cabin includes control module, underwater end power carrier communication module, binocular camera, host computer, two degrees of freedom camera holder, battery and power management module;Power module includes propeller and tracked rudder engine;Sensor module includes depth sensor and inertial measurement unit;Operation module includes rudder engine, shovel and shovel arm.The application can timely and effectively prevent further deterioration of scour pit;Improve the accuracy of underwater binocular vision measurement.

Description

Technical Field

[0001] This invention relates to the fields of offshore wind power operation and maintenance and computer vision technology, and in particular discloses an offshore wind turbine pile scour monitoring and repair system and method based on an underwater robot. Background Technology

[0002] Wind power, as a clean and renewable energy source, is receiving increasing attention worldwide. Compared to onshore wind power, offshore wind power offers numerous advantages, including better wind resources, no land use restrictions, no visual impact, and no noise pollution, making it a growing trend in the domestic and international wind power industry in recent years. Currently, over 70% of offshore wind turbines use relatively mature large-diameter monopile foundations as their support structure. However, the monopile foundation is susceptible to scour by the continuous action of downwater, horseshoe vortices, lateral acceleration flows, and tail eddies, which can seriously threaten the safety of the wind turbine. Therefore, monitoring and repair of offshore wind turbine foundation scour are necessary.

[0003] Traditional fixed scour monitoring devices all require underwater installation, resulting in high construction and operation costs, poor maintainability, and unsuitability for complex marine environments. Mobile scour monitoring methods include shipborne multibeam echo sounders for regular sea measurements, and remotely operated vehicles (ROVs) equipped with sonar to monitor scour activity. However, these methods suffer from high equipment costs and limited operational conditions. The primary purpose of scour monitoring is to prevent scour pits from expanding and endangering wind turbine safety. Therefore, timely measures to repair scour pits are necessary during monitoring. However, current research lacks a technology that integrates scour pit monitoring and repair, often leading to delayed repairs. Therefore, a technology that integrates scour pit monitoring and repair is urgently needed.

[0004] With the development of computer vision technology, a low-cost technique for monitoring scour depth has become possible. Passive underwater binocular vision measurement technology is mainly used in the fisheries field. For example, patent CN111862048B invented an automatic fish posture and length analysis method based on key point detection and deep convolutional neural networks. It uses deep learning technology to detect key points on the fish body and uses the binocular principle to measure the fish size. This type of method requires repeated detection of key points in two perspectives of the binocular camera using deep learning technology, which increases the computational cost and has a large measurement error. Patent CN111887853B invented a fish length measurement device and method based on binocular vision, which uses traditional image processing methods to obtain key points on the fish body. This type of method is only applicable when the measured object is perpendicular to the optical axis of the binocular camera. Introducing passive underwater binocular vision measurement technology combined with deep learning technology into the field of offshore wind turbine pile scour monitoring urgently requires solving the problems of high computational cost, large measurement error, and strict limitations on binocular camera placement in passive underwater binocular vision measurement technology. Summary of the Invention

[0005] This invention provides a system and method for monitoring and repairing scour pits in offshore wind turbine piles based on an underwater robot. The purpose is to provide a technical solution that integrates monitoring and repairing scour pits. It introduces passive underwater binocular vision measurement technology combined with deep learning technology into the field of offshore wind turbine pile scour monitoring, and proposes an improved image enhancement method and vanishing point detection method to achieve accurate measurement of the depth of scour pits.

[0006] One aspect of the present invention relates to an offshore wind turbine pile scour monitoring and repair system based on an underwater robot, comprising a tracked underwater robot and a shore-based terminal, wherein the tracked underwater robot and the shore-based terminal are connected by a zero-buoyancy cable; the tracked underwater robot includes a waterproof electronics compartment, a power module, a sensor module, an operation module, and a lighting device; the shore-based terminal includes a shore-based power line carrier communication module and a display screen, wherein the shore-based power line carrier communication module and the display screen are connected by a network cable; The waterproof electronic compartment includes a control module, an underwater power line carrier communication module, a binocular camera, a host computer, a two-degree-of-freedom camera gimbal, a battery, and a power management module. The control module, acting as a lower-level machine, sends control commands to the power module, operation module, and lighting system, and receives data from the sensor modules and control commands from the host computer. The underwater power line carrier communication module transmits real-time images and scour monitoring data to the shore-based system. The binocular camera acquires image information. The host computer runs a scour depth recognition algorithm and sends commands to the control module. The two-degree-of-freedom camera gimbal adjusts the binocular camera's monitoring angle. The battery powers the various modules of the underwater robot. The power management module distributes power to each module and provides overvoltage, overcurrent, and temperature protection. The power module includes a thruster and tracked servo motors; the thruster is used to enable the underwater robot to surface, dive, move forward, move backward, turn, and pitch in floating mode; the tracked servo motors are used to drive the track chains to rotate to enable forward, backward, and turn in crawling mode. The sensor module includes a depth sensor and an inertial measurement unit; the depth sensor is used to send depth information to the control module to achieve constant depth closed-loop control; the inertial measurement unit is used to send yaw and pitch angle information to the control module to achieve orientation closed-loop control. The working module includes a servo motor, a bucket, and a boom; the servo motor includes a first servo motor and a second servo motor. The first servo motor is connected to the bucket and is used to drive the bucket to shovel sand and fill pits; the second servo motor is connected to the boom and is used to drive the boom to swing and assist the bucket in adjusting its position to shovel sand and fill pits more efficiently. The lighting lamp receives a PWM signal from the control module to adjust the brightness of the light to provide illumination for the binocular camera to acquire images; The shore-based power line carrier communication module is used to receive real-time images and scour monitoring data transmitted from the underwater power line carrier communication module; the display screen is used to display real-time images and scour monitoring data.

[0007] Furthermore, the control module is an STM32 control module.

[0008] Furthermore, the control module includes a main control chip, which is an STM32F103ZET6.

[0009] Furthermore, the thruster consists of six brushless DC motors with built-in ESCs. Two of these motors form a vertical thruster, responsible for the underwater robot's surfacing and diving in floating mode, and working with the depth sensor in the sensor module to achieve depth stabilization. The other four motors form a horizontal thruster, responsible for the underwater robot's forward and backward movement in floating mode, and working with the inertial measurement unit in the sensor module to achieve bow turning and pitching.

[0010] Another aspect of the present invention relates to a method for monitoring and repairing scour of offshore wind turbine piles based on an underwater robot, applied to the aforementioned system for monitoring and repairing scour of offshore wind turbine piles based on an underwater robot. The method for monitoring and repairing scour of offshore wind turbine piles based on an underwater robot includes the following steps: Mark the original seabed line around the offshore wind turbine pile foundation before it is eroded to obtain the original seabed marking zone; A binocular camera was used for calibration and correction to acquire images of scour pits with different viewpoints, lighting conditions, scour shapes and depths. The images were preprocessed, the scour pit detection boxes were marked, and the lowest point of the water-sand boundary line of the scour pit was marked. The training set and the validation set were then divided. A target detection network model was built to detect targets in the morphology of scour pits; a key point detection network model was built to detect key points at the lowest point of the water-sand boundary in the scour pits; the target detection network model and the key point detection network model were used to train the training set images respectively, and the validation set images were used for validation. The trained target detection network and key point detection network are used to detect scour pits in real time from the calibrated binocular camera image. The target detection box and key points are output in the video frame of the left view of the binocular camera, and the key points are within the target detection box. The lower edge line of the original seabed marker zone is obtained. The probabilistic Hough transform is used to detect straight lines. The obtained line segments are expanded and merged. The line segments are sorted and candidate vanishing points are obtained. The candidate vanishing points are voted on and clustered. The candidate vanishing points are updated using the vote weighting. The cluster center with the highest vote is selected as the vanishing point output. The vanishing points of the edge lines on both sides of the pile foundation are obtained and the relevant line segments are removed. Obtain the pixel coordinates Pl1(x1, y1) of the lowest point of the water-sand boundary line of the scour pit obtained from the detection and the vanishing points of the edge lines on both sides of the pile foundation. Connect the lowest point of the water-sand boundary line of the scour pit and the vanishing points of the edge lines on both sides of the pile foundation to obtain a straight line parallel to the edge lines on both sides of the pile foundation in the real physical three-dimensional space. Intersect the straight line with the lower edge line of the original seabed marker zone at an intersection point. The intersection point is the pixel coordinates Pl2(x2, y2) of the corresponding point of the lowest point of the water-sand boundary line of the scour pit on the original seabed marker zone. A window of a set size is constructed centered on the pixel coordinates Pl1(x1, y1) of the lowest point of the water-sand boundary line of the scour pit detected in the left view of the binocular camera. The optimal area corresponding to the size of the window is matched in the right view of the binocular camera. The key points are precisely matched within the area to obtain the matching point Pr1(x1', y1') in the right view. The vanishing points of the edge lines on both sides of the pile foundation in the right view and the corresponding pixel coordinates Pr2(x2', y2') of the lowest point of the water-sand boundary line of the scour pit in the right view on the original seabed marker zone are obtained. The parallax of the lowest point of the water-sand boundary line of the scour pit and its corresponding point on the original seabed marker zone are obtained respectively. The three-dimensional coordinates of the lowest point of the water-sand boundary line of the scour pit and its corresponding point on the original seabed marker zone are recovered according to the binocular triangulation principle. The Euclidean distance between the two points is calculated to obtain the maximum scour pit depth. The host computer obtains the maximum depth data of the scour pit and executes the pit filling operation judgment command, autonomously driving the underwater robot to carry out the pit filling operation until the conditions for the scour pit repair are met.

[0011] Furthermore, a binocular camera was used for calibration and correction to acquire images of scour pits with different viewpoints, lighting conditions, and scour shapes and depths. Image preprocessing was performed, including labeling the scour pit detection boxes and the lowest point of the water-sand boundary line of the scour pit. In the steps of dividing the training and validation sets, Zhang Zhengyou's calibration method was used for underwater binocular calibration. Zhang Zhengyou's calibration method calibrates the binocular camera by capturing a set of 30 checkerboard images from different viewpoints and distances, obtaining the intrinsic and extrinsic parameters and distortion coefficients of the binocular camera. Correction was performed using a stereo correction algorithm for binocular cameras to correct epipolar lines. The scour pit images included pile foundation-scour pit images when scour occurred and pile foundation images when no scour occurred. Image preprocessing included image augmentation and image enhancement. Image augmentation involved horizontally mirroring the image. Image enhancement employed an improved multi-scale color-preserving retinal enhancement algorithm, which is described as follows:

[0012]

[0013] in, For multi-scale color-preserving retina enhancement algorithms, N is the number of scales. Weights for each scale, Take 1 / N, This represents the output image of a single-scale retina. This represents the output image of MSR. This represents the image after color equalization. R represents the red channel; G represents the green channel; Indicates the blue channel; Indicates the intensity channel. The definition is as follows:

[0014] in, The value represents the intensity channel, and S represents the number of image channels. This represents the image after three-channel color equalization. Two-dimensional convolution is achieved through continuous vertical and horizontal filtering, restoring the image from the logarithmic domain while performing dynamic compensation. Finally, the inverse gray-world algorithm is used to alternately preserve the colors of the original image. The inverse gray-world algorithm is described as follows:

[0015] in, This is a reverse grayscale world algorithm. , This represents the average value of the original image. Indicates an enhanced image. This represents the average value of the intensity channel in the improved multi-scale retinal output image. This is the color retention factor.

[0016] Furthermore, the trained target detection network and keypoint detection network are used to perform real-time detection on the scour pit images acquired by the calibrated binocular camera. Target detection boxes and keypoints are output in the video frames of the left view of the binocular camera, with keypoints located within the target detection boxes. The lower edge line of the original seabed marker zone is obtained. Line detection is performed using probabilistic Hough transform, and the obtained line segments are expanded and merged. The line segments are sorted and candidate vanishing points are obtained. Voting and clustering are performed on the candidate vanishing points, and the candidate vanishing points are updated using a weighted average of votes. The cluster center with the highest number of votes is selected as the vanishing point output. In the step of obtaining the vanishing points of the edge lines on both sides of the pile foundation and removing related line segments, the RGB three-channel color image is converted to a grayscale image.

[0017] in, It is a grayscale image. , , These represent the R channel values, B channel values, and G channel values ​​of an RGB three-channel image, respectively. The adaptive median filtering algorithm first sets the initial median filtering window size to N×N, where N is a positive odd number. During the filtering process, the filtering window size is dynamically adjusted in real time. The specific process includes processes A and B: Process A:

[0018] Process B:

[0019] Among them, Z xy Z represents the pixel value of the current pixel (x, y). min Z med Z max These represent the minimum, median, and maximum values ​​of the pixels contained in the filter window, respectively. It is the difference between the median and minimum values ​​of the pixels contained in the filter window; It is the difference between the median and the maximum value of the pixels contained in the filter window; It is the difference between the current pixel value and the minimum value of the pixels contained in the filter window; Let S be the difference between the current pixel value and the maximum value of the pixels contained in the filter window; max This represents the maximum allowed value of the filter window. During filtering, if m1 > 0 and m2 < 0 in process A, execution jumps to process B; otherwise, the filter window size N is increased. If the current filter window size N <= S... max If the result is positive, process A will be executed repeatedly; otherwise, output Z. medIf n1 > 0 and n2 < 0 in process B, then output the current pixel value Z. xy Otherwise, output Z. med .

[0020] Furthermore, the trained target detection network and keypoint detection network are used to perform real-time detection on the scour pit images acquired by the calibrated binocular camera. Target detection boxes and keypoints are output in the video frames of the left view of the binocular camera, with keypoints located within the target detection boxes. The lower edge line of the original seabed marker zone is obtained. Line detection is performed using probabilistic Hough transform, and the obtained line segments are expanded and merged. The line segments are sorted and candidate vanishing points are obtained. Voting and clustering are performed on the candidate vanishing points, and the candidate vanishing points are updated using a weighted average of votes. The cluster center with the highest number of votes is selected as the output vanishing point. In the steps of obtaining the vanishing points of the edge lines on both sides of the pile foundation and removing related line segments, the Sobel operator is used to calculate the gradient intensity and direction of each pixel in the image. Non-maximum suppression is applied to eliminate stray responses caused by edge detection. The Sobel operator uses operator templates in four directions and adaptive median filtering image convolution to obtain gradient information, as described in detail below:

[0021]

[0022] in, , , , Representing pixels Gray-scale gradient values ​​in the horizontal, vertical, 45°, and 135° directions. For adaptive median filtering images, For pixels Total grayscale gradient value.

[0023] Furthermore, the trained target detection network and keypoint detection network are used to perform real-time detection on the scour pit images acquired by the calibrated binocular camera. Target detection boxes and keypoints are output in the video frames of the left view of the binocular camera, with keypoints located within the target detection boxes. The lower edge line of the original seabed marker zone is obtained. Line detection is performed using probabilistic Hough transform, and the obtained line segments are expanded and merged. The line segments are sorted and candidate vanishing points are obtained. Voting and clustering are performed on the candidate vanishing points, and the candidate vanishing points are updated using a weighted average of votes. The cluster center with the highest number of votes is selected as the output vanishing point. In the step of obtaining the vanishing points of the edge lines on both sides of the pile foundation and removing related line segments, voting on candidate vanishing points is performed. Specifically, the angle between the midpoint of the adjacent line segment I of candidate point c and the line connecting candidate point c to the original line segment I is calculated. The voting value is calculated as follows:

[0024] in, The voting value. Let be the Euclidean distance between the midpoint of the adjacent line segment I of candidate point c and candidate point c. Let be the angle between the midpoint of the adjacent line segment I of candidate point c, the line connecting candidate point c, and the original line segment; Let be the base of the natural logarithm, taken as 2.718; u is the robustness parameter; the clustering method adopts the K-Means algorithm, and the clustering termination condition is that the minimum distance between candidate points is greater than m pixels. The candidate extinction points are clustered, and the candidate extinction points are updated using vote weighting. The cluster center with the highest number of votes is selected as the extinction point.

[0025] Furthermore, in the step where the host computer acquires the maximum scour pit depth data value and executes the pit filling operation judgment command, autonomously driving the underwater robot to carry out the pit filling operation until the conditions for scour pit repair are met, the specific execution of the pit filling operation judgment command is as follows: when the maximum scour pit depth data value is greater than the set safety threshold, the pit filling operation command is triggered; when the maximum scour pit depth data value is less than the set safety threshold, the next offshore wind turbine is inspected according to the preset path. The autonomous underwater robot performs pit-filling operations as follows: The host computer sends a pit-filling command to the control module via a serial port. Based on the binocular ranging principle, the distance between the underwater robot and the pile is obtained. It is then determined whether the safe sand-shoveling distance condition is met. If the distance is too close, the control module controls the tracked servo motor to make the underwater robot retreat a preset step length. The main control servo motor drives the bucket and shovel arm to perform sand-shoveling operations. The control module controls the tracked servo motor to make the underwater robot advance a preset step length. It is then determined whether the safe pit-filling distance condition is met. If the condition is met, the control module controls the servo motor to drive the bucket and shovel arm to perform pit-filling operations. The condition for completing the repair of a single scour pit is that the maximum scour pit depth is less than the set safety threshold; the condition for the number of repairs is whether all four points around the wind turbine pile have been repaired.

[0026] The beneficial effects achieved by this invention are as follows: This invention provides a system and method for monitoring and repairing scour of offshore wind turbine piles based on an underwater robot. The system includes a tracked underwater robot and a shore-based terminal, which are connected by a zero-buoyancy cable. The tracked underwater robot includes a waterproof electronics compartment, a power module, a sensor module, an operation module, and a lighting device. The shore-based terminal includes a shore-based power line carrier communication module and a display screen, which are connected by a network cable. The waterproof electronics compartment includes a control module, an underwater power line carrier communication module, a binocular camera, a host computer, and a two-degree-of-freedom camera gimbal. The underwater robot comprises a battery and a power management module. The control module, acting as a lower-level machine, sends control commands to the power module, operation module, and lighting system, and receives data from the sensor modules and control commands from the upper-level machine. The underwater power line carrier communication module transmits real-time images and scour monitoring data to the shore-based system. A binocular camera acquires image information. The upper-level machine runs a scour depth recognition algorithm and sends commands to the control module. A two-degree-of-freedom camera gimbal adjusts the binocular camera's monitoring angle. The battery powers all modules of the underwater robot. The power management module distributes power to each module. It provides overvoltage, overcurrent, and temperature protection; the power module includes a thruster and tracked servo motors; the thruster is used to achieve surfacing, diving, forward, backward, bow turning, and pitching in the underwater robot's floating mode; the tracked servo motors drive the track chains to achieve forward, backward, and bow turning in crawling mode; the sensor module includes a depth sensor and an inertial measurement unit; the depth sensor sends depth information to the control module to achieve depth-controlled closed-loop control; the inertial measurement unit sends bow and pitch angle information to the control module to achieve orientation-controlled closed-loop control; the operation module includes servo motors, a bucket, and a shovel. The system comprises an arm; wherein the servo motors include a first servo motor and a second servo motor. The first servo motor is connected to the bucket and is used to drive the bucket to scoop sand and fill pits; the second servo motor is connected to the arm and is used to drive the arm to swing and assist the bucket in adjusting its position for more efficient sand scooping and pit filling; the lighting receives PWM signals from the control module to adjust the brightness of the lights to provide illumination for the binocular camera to acquire images; the shore-based power line carrier communication module is used to receive real-time images and scour monitoring data transmitted from the underwater power line carrier communication module; and the display screen is used to display real-time images and scour monitoring data. The beneficial effects achieved by the offshore wind turbine pile scour monitoring and repair system and method based on an underwater robot provided by this invention are as follows: 1) By introducing passive underwater binocular vision measurement technology into the field of offshore wind turbine pile scour monitoring, and combining it with deep learning technology, a low-cost technology for monitoring scour depth is provided.

[0027] 2) A technical solution integrating monitoring and repair of scour pits is provided, which can prevent the scour pits from deteriorating in a timely and effective manner and solve the problem of untimely repair of scour pits.

[0028] 3) An improved image enhancement method is provided, which effectively solves the problems of underwater image degradation and blurring, and improves the accuracy of underwater binocular vision measurement.

[0029] 4) An automatic vanishing point detection method is provided, which can reduce the computational cost of key point detection and enable the measurement of scour pit depth from any angle. A key point-based matching method is integrated, which can effectively reduce the matching computational cost and improve the matching speed. Attached Figure Description

[0030] Figure 1 This is a structural block diagram of the offshore wind turbine pile scour monitoring and repair system based on an underwater robot, according to the present invention. Figure 2 This is a top view of the test setup and the scour pit repair points; Figure 3 This is a flowchart of the offshore wind turbine pile scour monitoring and repair method based on an underwater robot according to the present invention; Figure 4 This is a flowchart of the automatic vanishing point detection method of the present invention; Figure 5 It is the viewing angle when the optical axis of the binocular camera is perpendicular to the object being measured; Figure 6 It is the viewing angle when the optical axis of the binocular camera is not perpendicular to the object being measured. Figure 7 This is a flowchart of the underwater robot's pit-filling operation judgment subroutine of the present invention.

[0031] Explanation of icon numbers: 1. Lowest point of the scour pit water-sand boundary line; 2. Original seabed marker line; 3. Corresponding point of the lowest point of the scour pit water-sand boundary line on the original seabed marker line; 4. Extinction point. Detailed Implementation

[0032] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0033] like Figure 1As shown, one embodiment of the present invention proposes a monitoring and repair system for offshore wind turbine pile scour based on an underwater robot, comprising a tracked underwater robot and a shore-based terminal, which are connected by a zero-buoyancy cable; the tracked underwater robot includes a waterproof electronics compartment, a power module, a sensor module, an operation module, and a lighting device; the shore-based terminal includes a shore-based power line carrier communication module and a display screen, which are connected by a network cable; the waterproof electronics compartment includes a control module, an underwater power line carrier communication module, a binocular camera, a host computer, and two free-floating components. The system comprises a two-degree-of-freedom camera gimbal, battery, and power management module. The control module, acting as a lower-level machine, sends control commands to the power module, operation module, and lighting system, and receives data from the sensor modules and control commands from the upper-level machine. The underwater power line carrier communication module transmits real-time images and scour monitoring data to the shore-based terminal. A binocular camera acquires image information. The upper-level machine runs a scour depth recognition algorithm and sends commands to the control module. The two-degree-of-freedom camera gimbal adjusts the binocular camera's monitoring angle. The battery powers all modules of the underwater robot. The power management module distributes power to each module. The system is equipped with a power supply and provides overvoltage, overcurrent, and temperature protection. The power module includes a thruster and tracked servo motors. The thruster enables the underwater robot to ascend, descend, move forward, backward, turn, and pitch in floating mode. The tracked servo motors drive the track chains to achieve forward, backward, and turn movements in crawling mode. The sensor module includes a depth sensor and an inertial measurement unit (IMU). The depth sensor sends depth information to the control module for depth-controlled closed-loop control. The IMU sends bow and pitch angle information to the control module for orientation-controlled closed-loop control. The operations module includes servo motors and a bucket. The system includes a shovel arm; the servo mechanism comprises a first servo and a second servo. The first servo is connected to the bucket and is used to drive the bucket to shovel sand and fill pits; the second servo is connected to the shovel arm and is used to drive the shovel arm to swing and assist the bucket in adjusting its position for more efficient sand shoveling and pit filling; the lighting receives PWM signals from the control module to adjust the brightness of the lights to provide illumination for the binocular camera to acquire images; the shore-based power line carrier communication module receives real-time images and scour monitoring data transmitted from the underwater power line carrier communication module; and the display screen displays real-time images and scour monitoring data.

[0034] In the above structure, please see Figure 1The offshore wind turbine pile scour monitoring and repair system based on an underwater robot provided in this embodiment uses an STM32 control module (STM32 main controller). The control module includes a main control chip, model STM32F103ZET6. The thrusters consist of six brushless DC motors with built-in ESCs. Two motors form a vertical thruster, responsible for the underwater robot's surfacing and diving in floating mode, working with the depth sensor in the sensor module to achieve depth stability. The other four motors form a horizontal thruster, responsible for the underwater robot's forward and backward movement in floating mode, working with the inertial measurement unit in the sensor module to achieve bow turning and pitching. The host computer is an Nvidia Jetson TX2. To counteract rotational inertia, the thrusters use a paired propeller configuration. The track servos consist of two waterproof servos controlled by the RS485 communication protocol, used to drive the track chain to achieve forward, backward, and bow turning in crawling mode. Specifically, the offshore wind turbine pile scour monitoring and repair system provided in this embodiment for an underwater robot consists of two waterproof servo motors controlled by the RS485 communication protocol in the operation module. The first servo motor drives the bucket to scour sand and fill pits, while the second servo motor drives the boom to swing and assist the bucket in adjusting its position for more efficient sand scour and pit filling. During the verification phase, the power line carrier communication module at the shore-based end transmits data to the display screen via the Ethernet protocol. The display screen displays scour monitoring data and image information in real time, facilitating timely algorithm adjustments and improving measurement accuracy. The power line carrier communication module at the shore-based end exchanges data with the power line carrier communication module at the underwater robot end via a zero-buoyancy cable. The power line carrier communication module at the underwater robot end exchanges data with the Nvidia Jetson TX2 via the Ethernet protocol.

[0035] This invention relates to a method for monitoring and repairing scour of offshore wind turbine piles based on an underwater robot, applicable to the aforementioned system for monitoring and repairing scour of offshore wind turbine piles based on an underwater robot, such as... Figures 1 to 7 As shown, the method for monitoring and repairing scour of offshore wind turbine piles based on underwater robots includes the following steps: Step S100: Mark the original seabed line around the offshore wind turbine pile foundation before scouring occurs to obtain the original seabed marking strip.

[0036] Epoxy coatings were used to mark the original seabed line around the offshore wind turbine pile foundation before erosion occurred, thus obtaining the original seabed marking zone.

[0037] Step S200: Use a binocular camera to calibrate and correct, acquire images of scour pits with different viewpoints, lighting conditions, scour shapes and depths, and perform image preprocessing. Mark the scour pit detection boxes and mark the lowest point of the water-sand boundary line of the scour pits, and divide the training set and validation set.

[0038] The underwater binocular camera was calibrated and corrected, and images of scour pits with different angles, lighting conditions, scour shapes and depths were captured. The images were preprocessed, and LabelMe (deep learning image labeling software) was used to annotate the scour pit detection boxes and the lowest point of the water-sand boundary of the scour pit. The training set and validation set were then divided.

[0039] Step S300: Build a target detection network model to detect targets in the erosion pit morphology; build a key point detection network model to detect key points in the lowest point of the water-sand boundary of the erosion pit; train the obtained training set images using the target detection network model and the key point detection network model respectively, and validate the validation set images.

[0040] A Real-Time Models for Object Detection (RTMDet-tiny) object detection network model was constructed. The backbone adopted CSPDarkNet and introduced 5*5 depthwise convolutions to increase the effective receptive field. The number of parameters and computational cost between the backbone and the neck were adjusted to achieve optimal performance in scour pit object detection. An Asymptotic Feature Pyramid Network (AFPN) was introduced to improve the feature pyramid network. The AdamW (AdamWeightDecayOptimizer) was selected as the optimizer to perform object detection on the scour pit morphology. A RTMPose-tiny keypoint detection network model was constructed to perform keypoint detection on the lowest point of the water-sand boundary of the scour pit. The object detection network model and the keypoint detection network model were used to train the training set images obtained in step S200, and the validation set images were used for validation.

[0041] Step S400: Real-time detection of scour pit images acquired by the calibrated binocular camera is performed using the trained target detection network and key point detection network. Target detection boxes and key points are output in the video frames of the left view of the binocular camera, with key points within the target detection boxes. The lower edge line of the original seabed marker zone is obtained. Line detection is performed using probabilistic Hough transform. The obtained line segments are expanded and merged, sorted, and candidate vanishing points are obtained. Candidate vanishing points are voted on and clustered. Candidate vanishing points are updated using a weighted average of votes. The cluster center with the highest number of votes is selected as the vanishing point output. The vanishing points of the edge lines on both sides of the pile foundation are obtained, and related line segments are removed.

[0042] The trained RTMDet-tiny target detection network and RTMPose-tiny keypoint detection network are used to detect scour pits in real time from the calibrated binocular camera. Target detection boxes and keypoints are output in the video frames of the left view of the binocular camera, with keypoints located within the target detection boxes. Image information from the target detection boxes in the left view of the binocular camera formed by the target detection network is extracted frame by frame. The RGB three-channel color image is converted to a grayscale image to unify the gradient. Adaptive median filtering is used to smooth the image and remove noise. The Sobel operator is used to calculate the gradient intensity and direction of each pixel in the image, and non-maximum suppression is applied to eliminate stray responses caused by edge detection. Dual threshold detection is applied to determine real and potential edges. Edge detection is finally completed by suppressing isolated weak edges, and the lower edge line of the original seabed marker zone described in S1 is obtained. Probabilistic Hough transform is used for line detection. The obtained line segments are expanded and merged. The line segments are sorted and candidate vanishing points are obtained. The candidate vanishing points are voted on and clustered. The candidate vanishing points are updated using vote weighting. The cluster center with the highest number of votes is selected as the vanishing point output. The vanishing points of the edge lines on both sides of the pile foundation are obtained, and the relevant line segments are removed.

[0043] Step S500: Obtain the pixel coordinates Pl1(x1, y1) of the lowest point of the water-sand boundary line of the scour pit obtained from the detection and the vanishing points of the edge lines on both sides of the pile foundation. Connect the lowest point of the water-sand boundary line of the scour pit and the vanishing points of the edge lines on both sides of the pile foundation to obtain a straight line parallel to the edge lines on both sides of the pile foundation in the real physical three-dimensional space. Intersect the straight line with the lower edge line of the original seabed marker zone at an intersection point. The intersection point is the pixel coordinates Pl2(x2, y2) of the corresponding point of the lowest point of the water-sand boundary line of the scour pit on the original seabed marker zone.

[0044] Obtain the pixel coordinates Pl1(x1, y1) of the lowest point of the water-sand boundary line of the scour pit detected in step S400, obtain the vanishing points of the edge lines on both sides of the pile foundation detected in step S400, connect the lowest point of the water-sand boundary line of the scour pit and the vanishing points of the edge lines on both sides of the pile foundation to obtain a straight line parallel to the edge lines on both sides of the pile foundation in the real physical three-dimensional space, the straight line intersects the lower edge line of the original seabed marker strip obtained in step S400 at an intersection point, the intersection point is the pixel coordinates Pl2(x2, y2) of the corresponding point of the lowest point of the water-sand boundary line of the scour pit on the original seabed marker strip.

[0045] Step S600: Construct a window of a set size centered on the pixel coordinates Pl1(x1, y1) of the lowest point of the water-sand boundary line of the scour pit detected in the left view of the binocular camera. Match the optimal area of ​​the same size as the window in the right view of the binocular camera. Perform precise matching of key points within the area to obtain the matching point Pr1(x1', y1') in the right view. Obtain the vanishing points of the edge lines on both sides of the pile foundation in the right view and the corresponding pixel coordinates Pr2(x2', y2') of the lowest point of the water-sand boundary line of the scour pit on the original seabed marker zone. Obtain the parallax of the lowest point of the water-sand boundary line of the scour pit and its corresponding point on the original seabed marker zone. Recover the three-dimensional coordinates of the lowest point of the water-sand boundary line of the scour pit and its corresponding point on the original seabed marker zone according to the binocular triangulation principle. Calculate the Euclidean distance between the two points to obtain the maximum scour pit depth.

[0046] A 10*10 window is constructed centered on the point Pl1 detected in the left view of the binocular camera in step S500. The optimal 10*10 region is then matched in the right view of the binocular camera using the Sum of Squared Differences (SSD) matching algorithm. Within this region, the threshold deviation absolute value method is used... Absolute Difference (TAD) uses the three-channel information of the image to accurately match key points, obtaining the matching point Pr1(x1', y1') in the right view. The vanishing points of the edge lines on both sides of the pile foundation in the right view are obtained by the detection method in step S400. The pixel coordinates Pr2(x2', y2') of the corresponding point on the original seabed marker zone of the lowest point of the water-sand boundary of the scour pit in the right view are obtained by the detection method in step S500. The parallax of the lowest point of the water-sand boundary of the scour pit and its corresponding point on the original seabed marker zone are obtained respectively. The three-dimensional coordinates of the lowest point of the water-sand boundary of the scour pit and its corresponding point on the original seabed marker zone are recovered according to the principle of binocular triangulation. The Euclidean distance between the two points is calculated to obtain the maximum scour pit depth.

[0047] In step S700, the host computer obtains the maximum depth data value of the scour pit and executes the pit filling operation judgment command, autonomously driving the underwater robot to carry out the pit filling operation until the conditions for the scour pit repair are met.

[0048] The algorithms in steps S400-S600 are deployed on the host computer Nvidia Jetson TX2. Nvidia Jetson TX2 obtains the maximum scour pit depth data value obtained in step S600 and executes the pit filling operation judgment command, autonomously driving the underwater robot to carry out the pit filling operation until the conditions for scour pit repair are met.

[0049] Furthermore, this embodiment provides a method for monitoring and repairing scour of offshore wind turbine piles based on underwater robots. In step S100, a waterproof epoxy coating is evenly applied around the pile foundation at the original seabed line, i.e., at the water-sand junction where scour has not occurred, to form a yellow marking band. The yellow marking band is 2cm wide and surrounds the pile foundation. The lower edge of the yellow marking band is the original seabed line.

[0050] Step S200 involves underwater binocular camera calibration using the Zhang Zhengyou calibration method. Calibration is performed by capturing a set of 30 checkerboard images from different angles and distances to obtain the intrinsic and extrinsic parameters and distortion coefficients of the binocular camera. Distortion correction is then performed using the calibration parameters. It is worth noting that in this embodiment, the offshore wind turbine pile is based on a 5MW offshore wind turbine monopile foundation. A 1:60 scale monopile foundation model is used as the research object. The monopile model is a transparent hollow organic glass tube, 2m long, 0.1m outer diameter, and 0.08m inner diameter, placed in a groove along with a fixing device. The groove is a sedimentation tank 3.8m long, 1.3m wide, and 0.3m deep. The experimental setup is described in [reference needed]. Figure 2 Non-cohesive standard river sand was placed in the sedimentation tank, and the sand bed was leveled. The river sand in the tank had been processed, dried, screened, and washed before being placed in the sedimentation tank. The diameter of the sand particles ranged from 0.3mm to 0.8mm, with a medium particle diameter (d50) of 0.55mm. The water channel flow generation system was activated, and scour pits gradually formed around the monopile. 400 pairs (800 images) of scour pits of various shapes and depths under different lighting conditions were captured from different angles using an underwater robot equipped with a binocular camera. The image dataset was expanded from the original 800 images to 1600 images by horizontal mirroring. LabelMe was used to annotate the scour pit detection boxes and the lowest point of the water-sand boundary line of the scour pits, dividing them into training and validation sets at an 8:2 ratio.

[0051] The specific process of step S300 involves building an RTMPose-tiny target detection network model. The image training set uses 1280 images randomly divided in step S200, while the validation set uses the remaining 320 images from step S200. An improved multi-scale color-preserving retina enhancement algorithm (IMSRCP) is used to enhance the images. The optimal performance of scour pit target detection is achieved by adjusting the number of parameters and computational cost between the backbone and the neck. When the number of parameters in the backbone is adjusted to 50% and the number of parameters in the neck is adjusted to 41%, the inference speed of the detector is the fastest. Target detection training is performed on the scour pit morphology, and the validation set images are used for verification. An RTMPose-tiny keypoint detection network model is also built to perform keypoint detection training on the lowest point of the water-sand boundary line of the scour pit, and the validation set images are used for verification.

[0052] The specific process of step S400 involves performing target detection and keypoint detection on the lowest point of the scour pit and the water-sand boundary by obtaining the weights of the target detection network and keypoint detection network from step S3, and outputting the target detection bounding box and keypoints respectively in the video frame of the left view of the stereo camera. See [link to documentation]. Figure 4 The document provides a flowchart of an automatic vanishing point detection method. It extracts image information from the target detection box in the left view of a binocular camera formed by a target detection network frame-by-frame, converting the RGB three-channel color image to a grayscale image to unify the gradient. Adaptive median filtering is used to smooth the image and remove noise. The Sobel operator is used to calculate the gradient intensity and direction of each pixel in the image, and non-maximum suppression is applied to eliminate stray responses from edge detection. Dual threshold detection is applied to determine real and potential edges, ultimately completing edge detection by suppressing isolated weak edges, and obtaining the lower edge line of the original seabed marker zone in step S100. Probabilistic Hough transform is used for line detection, expanding and merging the obtained line segments. The line segments are sorted and candidate vanishing points are obtained. These candidate vanishing points are voted on and clustered, and updated using a weighted average of votes. The cluster center with the highest number of votes is selected as the vanishing point output, obtaining the vanishing points of the edge lines on both sides of the pile foundation, and removing related line segments.

[0053] See Figure 5The angle of view is when the optical axis of the binocular camera mounted on the underwater robot is perpendicular to the object being measured. The specific process of step S500 is to obtain the pixel coordinates of the lowest point of the water-sand boundary line of the scour pit detected in step S400, as shown in the figure, the lowest point 1 of the water-sand boundary line of the scour pit is denoted as Pl1(x1, y1). A perpendicular line is drawn from the lowest point 1 of the water-sand boundary line of the scour pit, intersecting the original seabed marker line 2 at the corresponding point 3 on the original seabed marker line where the lowest point of the water-sand boundary line of the scour pit is located, denoted as Pl2(x2, y2). See also... Figure 6 The viewpoint of the binocular camera mounted on the underwater robot when the optical axis is not perpendicular to the object being measured is obtained. The vanishing points of the edge lines on both sides of the pile foundation detected in step S400 are obtained. The lowest point of the water-sand boundary line of the scour pit and the vanishing points of the edge lines on both sides of the pile foundation are connected to obtain a straight line parallel to the edge lines on both sides of the pile foundation in the real physical three-dimensional space. This straight line intersects the original seabed marker line 2 at an intersection point. This intersection point is the corresponding point 3 of the lowest point of the water-sand boundary line of the scour pit on the original seabed marker line.

[0054] The specific process of step S600 is as follows: A 10*10 window is constructed centered on the point Pl1 detected in the left view of the binocular camera in step S500. The optimal 10*10 region is matched in the right view of the binocular camera using the Sum of Squared Differences (SSD) matching algorithm. Within this region, the threshold absolute deviation method is used... The Difference (TAD) method uses the three-channel information of the image to accurately match key points, obtaining the matching point Pr1(x1', y1') in the right view. The vanishing point detection method in step S400 obtains the vanishing points of the edge lines on both sides of the pile foundation in the right view. The detection method in step S500 obtains the pixel coordinates Pr2(x2', y2') of the corresponding point on the original seabed marker zone of the lowest point of the water-sand boundary of the scour pit in the right view. The parallax of the lowest point of the water-sand boundary of the scour pit and its corresponding point on the original seabed marker zone are obtained respectively. The three-dimensional coordinates of the lowest point of the water-sand boundary of the scour pit and its corresponding point on the original seabed marker zone are recovered according to the principle of binocular triangulation. The Euclidean distance between the two points is calculated to obtain the maximum scour pit depth.

[0055] The specific process of step S700 is to obtain the average value of the scour pit depth data obtained by real-time detection within 5 seconds. Since the detection speed of the algorithm in this embodiment is 4 FPS (Frames Per Second), a total of 20 frames of image information are obtained within 5 seconds. The average value of the 20 sets of scour pit depth data values ​​is used as the input for the pit filling operation judgment command. See [link to relevant documentation]. Figure 7This is a flowchart of the underwater robot's pit-filling operation judgment subroutine. The specific execution of the pit-filling operation judgment instruction is as follows: when the maximum scour pit depth data value is greater than the set safety threshold, a pit-filling operation instruction is triggered; when the maximum scour pit depth data value is less than the set safety threshold, the repair proceeds to the next point according to the preset path. The autonomous driving of the underwater robot to perform the pit-filling operation is specifically handled by the host computer Nvidia Jetson. TX2 sends a hole-filling command to the STM32 main controller via serial port. Based on binocular ranging, it obtains the distance between the underwater robot and the pile, determining if the safe sand-shoveling distance condition is met. If the distance is too close, the STM32 main controller controls the tracked servo motors to make the underwater robot retreat a preset step length. The main controller then controls the servo motors to drive the bucket and arm to shovel sand. The main controller also controls the tracked servo motors to make the underwater robot advance a preset step length, again determining if the safe hole-filling distance condition is met. If the condition is met, the main controller controls the servo motors to drive the bucket and arm to fill the hole. The condition for completing the repair of a single scour pit is that the maximum scour pit depth is less than a set safety threshold. The condition for the number of repairs is whether all four points around the wind turbine pile have been repaired. The specific layout of the four points around the wind turbine pile is as follows: Figure 2 The underwater robot starts monitoring and repairing the scour pit depth from Point 1. Once Point 1 is repaired, it moves to the next point according to the preset path until Point 4 is repaired, which means that the repair work of a single offshore wind turbine pile is completed.

[0056] Preferably, in step S200, underwater binocular calibration is performed using the Zhang Zhengyou calibration method. The Zhang Zhengyou calibration method involves taking a set of 30 checkerboard images from different angles and distances to calibrate the binocular camera, obtaining its intrinsic and extrinsic parameters and distortion coefficients. Correction is performed using the binocular camera stereo correction (Bouguet) algorithm for epipolar correction. The scour pit images include pile foundation-scour pit images during scour and pile foundation images without scour. Image preprocessing includes image augmentation and image enhancement. Image augmentation involves horizontally mirroring the image. Image enhancement employs an improved multi-scale color-preserving retinal enhancement algorithm (IMSRCP), which is described below:

[0057] (1) In formula (1), For multi-scale color-preserving retina enhancement algorithms, N is the number of scales. Weights for each scale, Take 1 / N, This represents the output image of a single-scale retina. This represents the output image of MSR. This represents the image after color equalization. R represents the red channel; G represents the green channel; Indicates the blue channel; Indicates the intensity channel. The definition is as follows: (2) In formula (2), The value represents the intensity channel, and S represents the number of image channels. This represents the image after three-channel color equalization.

[0058] Two-dimensional convolution is achieved through continuous vertical and horizontal filtering, restoring the image from the logarithmic domain while performing dynamic compensation. Finally, the inverse gray-world algorithm is used to alternately preserve the colors of the original image. The inverse gray-world algorithm is described as follows: (3) In formula (3), This is a reverse grayscale world algorithm. , This represents the average value of the original image. Indicates an enhanced image. This represents the average value of the intensity channel in the improved multi-scale retinal output image. This is the color retention factor.

[0059] Preferably, in step S400, the RGB three-channel color image is converted into a grayscale image: (4) In formula (4), It is a grayscale image. , , These represent the R channel values, B channel values, and G channel values ​​of an RGB three-channel image, respectively. The adaptive median filtering algorithm first sets the initial median filtering window size to N×N, where N is a positive odd number. During the filtering process, the filtering window size is dynamically adjusted in real time. The specific process includes processes A and B: Process A: (5) Process B: (6) In formulas (5) to (6), Z xy Z represents the pixel value of the current pixel (x, y). min Z med Z max These represent the minimum, median, and maximum values ​​of the pixels contained in the filter window, respectively. It is the difference between the median and minimum values ​​of the pixels contained in the filter window; It is the difference between the median and the maximum value of the pixels contained in the filter window; It is the difference between the current pixel value and the minimum value of the pixels contained in the filter window; Let S be the difference between the current pixel value and the maximum value of the pixels contained in the filter window; max This represents the maximum allowed value of the filter window. During filtering, if m1 > 0 and m2 < 0 in process A, execution jumps to process B; otherwise, the filter window size N is increased. If the current filter window size N <= S... max If the result is positive, process A will be executed repeatedly; otherwise, output Z. med If n1 > 0 and n2 < 0 in process B, then output the current pixel value Z. xy Otherwise, output Z. med .

[0060] Further, in step S400, the Sobel operator is used to calculate the gradient intensity and direction of each pixel in the image, and non-maximum suppression is applied to eliminate stray responses caused by edge detection. The Sobel operator uses operator templates in four directions and adaptive median filtering image convolution to obtain gradient information, as described in detail below: (7)

[0061] In formula (7), , , , Representing pixels Gray-scale gradient values ​​in the horizontal, vertical, 45°, and 135° directions. For adaptive median filtering images, For pixels Total grayscale gradient value.

[0062] Further, in step S400, voting is performed on the candidate elimination points. Specifically, this involves calculating the angle between the midpoint of the adjacent line segment I of candidate point c and the line connecting candidate point c to the original line segment I. The voting value is calculated as follows: (8) In formula (8), The voting value. Let be the Euclidean distance between the midpoint of the adjacent line segment I of candidate point c and candidate point c. Let be the angle between the line connecting the midpoint of the adjacent line segment I of candidate point c and candidate point c and the original line segment; is the base of the natural logarithm, taken as 2.718; u is the robustness parameter; the clustering method adopts the K-Means algorithm, and the clustering termination condition is that the minimum distance between candidate points is greater than m pixels. The candidate vanishing points are clustered, and the candidate vanishing points are updated using vote weighting. The cluster center with the highest number of votes is selected as the vanishing point.

[0063] The offshore wind turbine pile scour monitoring and repair system and method based on an underwater robot provided in this embodiment, compared with the prior art, includes a tracked underwater robot and a shore-based terminal, which are connected by a zero-buoyancy cable. The tracked underwater robot includes a waterproof electronics compartment, a power module, a sensor module, an operation module, and a lighting device. The shore-based terminal includes a shore-based power line carrier communication module and a display screen, which are connected by a network cable. The waterproof electronics compartment includes a control module, an underwater power line carrier communication module, a binocular camera, a host computer, and two freewheeling devices. The system comprises a two-degree-of-freedom camera gimbal, battery, and power management module. The control module, acting as a lower-level machine, sends control commands to the power module, operation module, and lighting system, and receives data from the sensor modules and control commands from the upper-level machine. The underwater power line carrier communication module transmits real-time images and scour monitoring data to the shore-based terminal. A binocular camera acquires image information. The upper-level machine runs a scour depth recognition algorithm and sends commands to the control module. The two-degree-of-freedom camera gimbal adjusts the binocular camera's monitoring angle. The battery powers all modules of the underwater robot. The power management module distributes power to each module. The system is equipped with a power supply and provides overvoltage, overcurrent, and temperature protection. The power module includes a thruster and tracked servo motors. The thruster enables the underwater robot to ascend, descend, move forward, backward, turn, and pitch in floating mode. The tracked servo motors drive the track chains to achieve forward, backward, and turn movements in crawling mode. The sensor module includes a depth sensor and an inertial measurement unit (IMU). The depth sensor sends depth information to the control module for depth-controlled closed-loop control. The IMU sends bow and pitch angle information to the control module for orientation-controlled closed-loop control. The operations module includes servo motors and a bucket. The system includes a shovel arm; the servo mechanism comprises a first servo and a second servo, the first servo being connected to the bucket and used to drive the bucket to shovel sand and fill pits; the second servo being connected to the shovel arm and used to drive the shovel arm to swing and assist the bucket in adjusting its position for more efficient sand shoveling and pit filling; a lighting system receives PWM signals from the control module to adjust the brightness of the lights to provide illumination for the binocular camera to acquire images; a shore-based power line carrier communication module receives real-time images and scour monitoring data transmitted from the underwater power line carrier communication module; and a display screen displays the real-time images and scour monitoring data. The beneficial effects achieved by the offshore wind turbine pile scour monitoring and repair system and method based on an underwater robot provided in this embodiment are as follows: 1) By introducing passive underwater binocular vision measurement technology into the field of offshore wind turbine pile scour monitoring, and combining it with deep learning technology, a low-cost technology for monitoring scour depth is provided.

[0064] 2) A technical solution integrating monitoring and repair of scour pits is provided, which can prevent the scour pits from deteriorating in a timely and effective manner and solve the problem of untimely repair of scour pits.

[0065] 3) An improved image enhancement method is provided, which effectively solves the problems of underwater image degradation and blurring, and improves the accuracy of underwater binocular vision measurement.

[0066] 4) An automatic vanishing point detection method is provided, which can reduce the computational cost of key point detection and enable the measurement of scour pit depth from any angle. A key point-based matching method is integrated, which can effectively reduce the matching computational cost and improve the matching speed.

[0067] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.

Claims

1. A method for monitoring and repairing scour of offshore wind turbine piles based on an underwater robot, characterized in that, This system is applied to an underwater robot-based offshore wind turbine pile scour monitoring and repair system, which includes: The system comprises a tracked underwater robot and a shore-based terminal, which are connected by a zero-buoyancy cable. The tracked underwater robot includes a waterproof electronics compartment, a power module, a sensor module, an operation module, and a lighting device. The shore-based terminal includes a shore-based power line carrier communication module and a display screen, which are connected by a network cable. The waterproof electronic cabin includes a control module, an underwater power line carrier communication module, a binocular camera, a host computer, a two-degree-of-freedom camera gimbal, a battery, and a power management module. The control module, acting as a lower-level machine, sends control commands to the power module, operation module, and lighting system, and receives data from the sensor modules and control commands from the host computer. The underwater power line carrier communication module transmits real-time images and scour monitoring data to the shore-based system. The binocular camera acquires image information. The host computer runs a scour depth recognition algorithm and sends commands to the control module. The two-degree-of-freedom camera gimbal adjusts the binocular camera's monitoring angle. The battery powers the various modules of the underwater robot. The power management module distributes power to each module and provides overvoltage, overcurrent, and temperature protection. The power module includes a thruster and a tracked servo motor; wherein, the thruster is used to realize the underwater robot's surfacing, diving, forward, backward, bow turning and pitching in floating mode; the tracked servo motor is used to drive the track chain to rotate to realize forward, backward and bow turning in crawling mode; The sensor module includes a depth sensor and an inertial measurement unit; wherein, the depth sensor is used to send depth information to the control module to achieve constant depth closed-loop control; the inertial measurement unit is used to send yaw and pitch angle information to the control module to achieve orientation closed-loop control; The operating module includes a servo motor, a bucket, and a shovel arm; wherein, the servo motor includes a first servo motor and a second servo motor, the first servo motor being connected to the bucket and used to drive the bucket to shovel sand and fill pits; the second servo motor being connected to the shovel arm and used to drive the shovel arm to swing and assist the bucket in adjusting its position to shovel sand and fill pits more efficiently. The lighting lamp receives a PWM signal from the control module to adjust the brightness of the light, which is used to provide illumination for the binocular camera to acquire images; The shore-based power line carrier communication module is used to receive real-time images and scour monitoring data transmitted from the underwater power line carrier communication module; the display screen is used to display real-time images and scour monitoring data. The method for monitoring and repairing scour of offshore wind turbine piles based on underwater robots includes the following steps: Mark the original seabed line around the offshore wind turbine pile foundation before it is eroded to obtain the original seabed marking zone; A binocular camera was used for calibration and correction to acquire images of scour pits with different viewpoints, lighting conditions, scour shapes and depths. The images were preprocessed, the scour pit detection boxes were marked, and the lowest point of the water-sand boundary line of the scour pit was marked. The training set and the validation set were then divided. A target detection network model is built to detect targets in the morphology of scour pits; a key point detection network model is built to detect key points at the lowest point of the water-sand boundary of the scour pit; the target detection network model and the key point detection network model are used to train the training set images respectively, and the validation set images are used for validation. The trained target detection network and key point detection network are used to detect scour pit images acquired by the calibrated binocular camera in real time. Target detection boxes and key points are output in the video frames of the left view of the binocular camera, with key points within the target detection boxes. The lower edge line of the original seabed marker zone is obtained. Line detection is performed using probabilistic Hough transform. The obtained line segments are expanded and merged, sorted, and candidate vanishing points are obtained. The candidate vanishing points are voted on and clustered. The candidate vanishing points are updated using a weighted average of votes. The cluster center with the highest number of votes is selected as the vanishing point output. The vanishing points of the edge lines on both sides of the pile foundation are obtained, and the relevant line segments are removed. Obtain the pixel coordinates Pl1(x1, y1) of the lowest point of the water-sand boundary line of the scour pit obtained from the detection and the vanishing points of the edge lines on both sides of the pile foundation. Connect the lowest point of the water-sand boundary line of the scour pit and the vanishing points of the edge lines on both sides of the pile foundation to obtain a straight line parallel to the edge lines on both sides of the pile foundation in the real physical three-dimensional space. Intersect the straight line with the lower edge line of the original seabed marking zone at an intersection point. The intersection point is the pixel coordinates Pl2(x2, y2) of the corresponding point of the lowest point of the water-sand boundary line of the scour pit on the original seabed marking zone. A window of a predetermined size is constructed centered on the pixel coordinates Pl1(x1, y1) of the lowest point of the water-sand boundary line of the scour pit detected in the left view of the binocular camera. The optimal area corresponding to the size of the window is matched in the right view of the binocular camera. Precise matching of key points is performed within this area to obtain the matching point Pr1(x1', y1') in the right view. The vanishing points of the edge lines on both sides of the pile foundation in the right view and the corresponding pixel coordinates Pr2(x2', y2') of the lowest point of the water-sand boundary line of the scour pit on the original seabed marker strip are obtained. The parallax of the lowest point of the water-sand boundary line of the scour pit and its corresponding point on the original seabed marker strip are obtained. Based on the binocular triangulation principle, the three-dimensional coordinates of the lowest point of the water-sand boundary line of the scour pit and its corresponding point on the original seabed marker strip are recovered. The Euclidean distance between the two points is calculated to obtain the maximum scour pit depth. The host computer obtains the maximum depth data of the scour pit and executes the pit filling operation judgment command, autonomously driving the underwater robot to carry out the pit filling operation until the conditions for the scour pit repair are met.

2. The method for monitoring and repairing scour of offshore wind turbine piles based on an underwater robot as described in claim 1, characterized in that, The control module is an STM32 control module.

3. The method for monitoring and repairing scour of offshore wind turbine piles based on an underwater robot as described in claim 1, characterized in that, The control module includes a main control chip, which is an STM32F103ZET6.

4. The method for monitoring and repairing scour of offshore wind turbine piles based on an underwater robot as described in claim 1, characterized in that, The thruster consists of six brushless DC motors with built-in ESCs. Two of these motors form a vertical thruster, responsible for the underwater robot's surfacing and diving in floating mode, and working with the depth sensor in the sensor module to achieve depth control. The other four motors form a horizontal thruster, responsible for the underwater robot's forward and backward movement in floating mode, and working with the inertial measurement unit in the sensor module to achieve bow turning and pitching.

5. The method for monitoring and repairing scour of offshore wind turbine piles based on an underwater robot as described in claim 1, characterized in that, The process involves calibrating and correcting a binocular camera to acquire images of scour pits from different angles, under different lighting conditions, and with different scour shapes and depths. Image preprocessing is performed, including labeling the scour pit detection boxes and the lowest point of the water-sand boundary line. The training and validation sets are then divided. Underwater binocular calibration is performed using the Zhang Zhengyou calibration method, which involves capturing a set of 30 checkerboard images from different angles and distances to obtain the intrinsic and extrinsic parameters and distortion coefficients of the binocular camera. Correction utilizes a stereo correction algorithm for epipolar correction. The scour pit images include pile-scour pit images during scour and pile images without scour. Image preprocessing includes image augmentation and image enhancement. Image augmentation involves horizontally mirroring the image. Image enhancement employs an improved multi-scale color-preserving retinal enhancement algorithm, described below: in, For multi-scale color-preserving retina enhancement algorithms, N is the number of scales. Weights for each scale, Take 1 / N, This represents the output image of a single-scale retina. This represents the image after color equalization. R represents the red channel; G represents the green channel; Indicates the blue channel; Indicates the intensity channel. This represents the function term used for image enhancement calculations in the multi-scale color-preserving retina enhancement algorithm, where... Represents the pixel coordinates in the image plane. Indicates the first Each processing scale corresponds to a parameter variable, which is used to distinguish the calculation process at different scales; The definition is as follows: in, The value represents the intensity channel, and S represents the number of image channels. This represents the image after three-channel color equalization. Two-dimensional convolution is achieved through continuous vertical and horizontal filtering to recover the image from the logarithmic domain while performing dynamic compensation. Finally, the inverse gray-world algorithm is used to alternately preserve the colors of the original image. The inverse gray-world algorithm is described as follows: in, This is a reverse grayscale world algorithm. , Represents the original image. This represents the average value of the original image. Indicates an enhanced image. This represents the average value of the intensity channel in the improved multi-scale retinal output image. This is the color retention factor.

6. The method for monitoring and repairing scour of offshore wind turbine piles based on an underwater robot as described in claim 1, characterized in that, The process involves real-time detection of scour pit images acquired by a calibrated binocular camera using a trained target detection network and a keypoint detection network. Target detection boxes and keypoints are output in the video frame of the left view of the binocular camera, with keypoints located within the target detection boxes. The lower edge line of the original seabed marker zone is acquired. Line detection is performed using probabilistic Hough transform, and the resulting line segments are expanded and merged. The line segments are sorted, and candidate vanishing points are obtained. These candidate vanishing points are then voted on and clustered. The candidate vanishing points are updated using a weighted average of votes, and the cluster center with the highest number of votes is selected as the vanishing point output. In the step of acquiring the vanishing points of the edge lines on both sides of the pile foundation and removing relevant line segments, the RGB three-channel color image is converted to a grayscale image. in, It is a grayscale image. , , These represent the R channel values, B channel values, and G channel values ​​of an RGB three-channel image, respectively. The adaptive median filtering algorithm first sets the initial median filtering window size to N×N, where N is a positive odd number. During the filtering process, the size of the filtering window is dynamically adjusted in real time. This process includes processes A and B: Process A: Process B: Among them, Z xy Z represents the pixel value of the current pixel (x, y). min Z med Z max These represent the minimum, median, and maximum values ​​of the pixels contained in the filter window, respectively. It is the difference between the median and minimum values ​​of the pixels contained in the filter window; It is the difference between the median and the maximum value of the pixels contained in the filter window; It is the difference between the current pixel value and the minimum value of the pixels contained in the filter window; Let S be the difference between the current pixel value and the maximum value of the pixels contained in the filter window; max This represents the maximum value that the filter window can allow; during filtering, if m1>0 and m2<0 in process A, execution jumps to process B; otherwise, the filter window size N is increased; if the current filter window size N<=S max If the result is positive, process A will be executed repeatedly; otherwise, output Z. med If n1 > 0 and n2 < 0 in process B, then output the current pixel value Z. xy Otherwise, output Z. med .

7. The method for monitoring and repairing scour of offshore wind turbine piles based on an underwater robot as described in claim 1, characterized in that, The process involves real-time detection of scour pit images acquired by a calibrated binocular camera using a trained target detection network and a keypoint detection network. Target detection boxes and keypoints are output in the video frame of the left view of the binocular camera, with keypoints located within the target detection boxes. The lower edge line of the original seabed marker strip is acquired. Line detection is performed using probabilistic Hough transform, and the resulting line segments are expanded and merged. The line segments are sorted, and candidate vanishing points are obtained. These candidate vanishing points are then voted on and clustered. The candidate vanishing points are updated using a weighted average of votes, and the cluster center with the highest number of votes is selected as the output vanishing point. In the steps of acquiring the vanishing points of the edge lines on both sides of the pile foundation and removing related line segments, the Sobel operator is used to calculate the gradient intensity and direction of each pixel in the image. Non-maximum suppression is applied to eliminate stray responses caused by edge detection. The Sobel operator uses four-directional operator templates and adaptive median filtering image convolution to obtain gradient information, as described in detail below: in, , , , Representing pixels Gray-scale gradient values ​​in the horizontal, vertical, 45°, and 135° directions. For adaptive median filtering images, For pixels Total grayscale gradient value.

8. The method for monitoring and repairing scour of offshore wind turbine piles based on an underwater robot as described in claim 1, characterized in that, The process involves real-time detection of scour pit images acquired by a calibrated binocular camera using a trained target detection network and a keypoint detection network. Target detection boxes and keypoints are output in the video frame of the left view of the binocular camera, with keypoints located within the target detection boxes. The lower edge line of the original seabed marker zone is acquired. Line detection is performed using a probabilistic Hough transform, and the resulting line segments are expanded and merged. The line segments are sorted, and candidate vanishing points are obtained. These candidate vanishing points are then voted on and clustered. The candidate vanishing points are updated using a weighted average of votes, and the cluster center with the highest number of votes is selected as the output vanishing point. In the step of acquiring the vanishing points of the edge lines on both sides of the pile foundation and removing related line segments, the voting process for candidate vanishing points involves calculating the angle between the midpoint of the adjacent line segment I of candidate point c and the line connecting candidate point c to the original line segment. The voting value is calculated as follows: in, The voting value. Let be the Euclidean distance between the midpoint of the adjacent line segment I of candidate point c and candidate point c. Let be the angle between the midpoint of the adjacent line segment I of candidate point c, the line connecting candidate point c, and the original line segment; is the base of the natural logarithm, taken as 2.718; u is the robustness parameter; the clustering method adopts the K-Means algorithm, and the clustering termination condition is that the minimum distance between candidate points is greater than m pixels. The candidate extinction points are clustered, and the candidate extinction points are updated using vote weighting. The cluster center with the highest number of votes is selected as the extinction point.

9. The method for monitoring and repairing scour of offshore wind turbine piles based on an underwater robot as described in claim 1, characterized in that, In the step where the host computer acquires the maximum scour pit depth data value and executes the pit filling operation judgment command, and autonomously drives the underwater robot to perform pit filling operation until the conditions for scour pit repair are met, the execution of the pit filling operation judgment command specifically means: when the maximum scour pit depth data value is greater than the set safety threshold, the pit filling operation command is triggered; when the maximum scour pit depth data value is less than the set safety threshold, the next offshore wind turbine is inspected according to the preset path. The autonomous underwater robot performs the pit-filling operation as follows: The host computer sends a pit-filling command to the control module via a serial port. Based on the binocular ranging principle, the distance between the underwater robot and the pile is obtained. It is determined whether the safe sand-shoveling distance condition is met. When the distance is too close, the control module controls the tracked servo motor to make the underwater robot retreat a preset step length. The main control controls the servo motor to drive the bucket and shovel arm to perform sand-shoveling operations. The control module controls the tracked servo motor to make the underwater robot advance a preset step length. It is determined whether the safe pit-filling distance condition is met. If the condition is met, the control module controls the servo motor to drive the bucket and shovel arm to perform the pit-filling operation. The condition for completing the single-point repair of the scour pit is that the maximum scour pit depth data value is less than the set safety threshold. The repair quantity condition is whether all four points around the wind turbine pile have been repaired.

Images