A wind power pile foundation cable overhang height detection method based on binocular vision

Abstract

This invention discloses a method for detecting the suspended height of wind turbine pile foundation cables based on binocular vision, comprising: acquiring and correcting left and right binocular images of the cable area to be measured; performing edge detection and local contrast enhancement on the corrected images to optimize image quality; calculating a disparity map by optimizing the SGM algorithm and removing invalid disparities by combining connected component filtering and edge protection strategies; performing 3D reconstruction based on camera calibration parameters and effective disparities to generate an initial point cloud, and obtaining a real-scale 3D point cloud model by adaptive depth scaling factor and underwater refractive index correction; filtering spatial regions using y-axis constraints and extracting local point clouds of the target cable using dual-point positioning; and calculating the suspended height of the cable using a top and bottom plane fitting method. This invention realizes the entire process from image enhancement and robust disparity calculation to 3D point cloud reconstruction and height measurement, and has the advantages of high detection accuracy and strong applicability.

Description

Technical Field

[0001] This invention belongs to the field of marine engineering technology, specifically relating to a method for detecting the suspension height of wind power pile cables based on binocular vision. Background Technology

[0002] As the scale of offshore wind turbine foundations continues to expand, the safety of the foundation cables, as a critical infrastructure component, directly impacts the stability and reliability of the offshore wind power system. Therefore, accurately detecting the suspension height of the foundation cables is crucial for precise operation and maintenance of these cables, and is of great significance for the overall operation and maintenance of offshore wind turbine foundations.

[0003] Studies have shown that in recent years, ROVs / AUVs equipped with sonar have become the main technical means for the operation and maintenance inspection of wind turbine pile foundations. However, relative to the diameter of the pile foundation, sonar detection can only identify small-diameter cables and cannot accurately detect their suspension height. In contrast, binocular vision technology has been widely used in height detection because it can directly acquire the three-dimensional information of the target, but most of these applications are based on the object being stationary and undisturbed. In actual underwater operations, robot vision systems face problems such as uncertain pose and complex environmental factors such as water disturbance, which make binocular vision detection under dynamic conditions face technical challenges such as image matching difficulties and unstable depth estimation.

[0004] Chinese invention patent application number "202310529046.1" discloses a multi-vision method for measuring the size of small workpieces based on a binocular vision system, achieving accurate measurement of workpiece dimensions. However, this method relies on the stable and controllable relative pose between the binocular camera and the target workpiece, and the measurement environment is undisturbed air, making it unsuitable for underwater environments. Chinese invention patent application number "202511547095.3" discloses a method and device for measuring aircraft altitude based on binocular vision, achieving accurate measurement of aircraft altitude. However, this method also relies on the fixed position between the binocular camera and the aircraft, and the measurement is performed in an air environment, making it unsuitable for underwater environments.

[0005] Therefore, how to apply binocular vision to the actual underwater environment and achieve accurate detection of the suspended height of pile foundation cables is a technical problem that the offshore wind power industry urgently needs to solve. Summary of the Invention

[0006] Purpose of the Invention: To address the challenge of consistently and accurately obtaining the suspended height of cables in complex marine environments, this invention proposes a binocular vision-based method for detecting the suspended height of cables in pile foundations. This method provides precise data support for pile foundation cable condition assessment and maintenance decisions. The invention realizes the entire process from image enhancement and robust parallax calculation to 3D point cloud reconstruction and height measurement. It boasts advantages such as high detection accuracy and strong applicability, providing reliable technical support for pile foundation cable condition monitoring and maintenance.

[0007] Technical solution: A method for detecting the suspension height of wind turbine pile foundation cables based on binocular vision, comprising the following steps:

[0008] Acquire left and right eye images of the cable area of ​​the pile foundation to be tested, and correct the left and right eye images so that the corresponding points in the corrected left and right eye images are at the same horizontal height;

[0009] Edge detection and local contrast enhancement are performed on the corrected left and right eye images to obtain the final left and right output images;

[0010] Based on the final output images on the left and right, a matching cost function based on absolute difference is used to calculate the pixel matching cost. According to the pixel matching cost, the cost is aggregated in multiple directions using the SGM algorithm to obtain the aggregated cost value. For each pixel, the disparity value that minimizes the aggregated cost value is selected and used as the disparity value of that pixel. A disparity map is constructed based on the disparity values ​​of all pixels.

[0011] The actual depth is calculated based on the disparity value of each pixel in the disparity map. The three-dimensional coordinates of each pixel are then reconstructed based on the coordinates of each pixel in the disparity map and the actual depth, forming a complete three-dimensional point cloud data. The final output image is converted into an RGB image, and the colors are mapped onto the three-dimensional point cloud data to obtain colored three-dimensional point cloud data. After denoising and scaling the colored three-dimensional point cloud data, a point cloud set is obtained. The local point cloud set constituting the target cable is extracted from the point cloud set.

[0012] The voxel side length is adaptively determined based on the spatial range of the local point cloud set. The point cloud data space is divided into a cubic voxel grid with a side length equal to the voxel side length. For all points in each cubic voxel grid, the geometric centroid is calculated as the representative point of that cubic voxel grid. All representative points form a downsampled point cloud set. The 95th and 5th percentiles of the z-axis coordinates of all points in the downsampled point cloud set are calculated and used as the top and bottom thresholds, respectively. Based on the top and bottom thresholds, the top and bottom point cloud sets are selected. The center point cloud of the top and bottom point cloud sets is calculated. Based on the center point cloud of the top and bottom point cloud sets, the suspension height of the cable under test is calculated.

[0013] Furthermore, the step of performing edge detection and local contrast enhancement on the corrected left and right eye images to obtain the final left and right output images includes:

[0014] The corrected left and right eye images are converted from RGB format to grayscale images using a grayscale conversion operation.

[0015] The grayscale image is divided into multiple sub-regions, and the grayscale histogram of each sub-region is calculated. The grayscale histogram of each sub-region is then cropped and normalized in sequence to obtain the normalized cumulative histogram.

[0016] Based on the normalized cumulative histogram, the enhanced gray values ​​are obtained. The gray values ​​in the sub-regions are mapped to the enhanced gray values, and bilinear interpolation is used to fuse adjacent sub-regions to obtain the left and right enhanced images.

[0017] The Canny edge detection algorithm is used to extract the edge contours of the left and right enhanced images to obtain the edge output image;

[0018] Based on the edge output image, image enhancement is performed on the left and right enhanced images to obtain the final left and right output images.

[0019] Furthermore, the Canny edge detection algorithm is used to extract the edge contours of the left and right enhanced images to obtain the edge output image, including:

[0020] A two-dimensional Gaussian filter is used to blur each pixel in the left and right enhanced images;

[0021] A smoothed image is obtained by convolving a grayscale image with a Gaussian kernel.

[0022] The Sobel algorithm is used to calculate the first-order gradient of a smoothed image in the x and y directions.

[0023] Calculate the gradient magnitude and direction angle based on the first-order gradients in the x and y directions;

[0024] Perform non-maximum suppression operation on the gradient magnitude to obtain edge points;

[0025] A dual-threshold hysteresis connection strategy is adopted to determine the edge points, obtain the final edge points, and form the edge output image from the final edge points;

[0026] The dual-threshold hysteresis connection strategy is expressed as follows:

[0027]

[0028] In the formula, T h For a high threshold, T l The threshold is set to low, and M(x, y) represents the gradient magnitude.

[0029] Furthermore, the step of performing image enhancement on the left and right enhanced images based on the edge output image to obtain the final left and right output images includes:

[0030] A dilation operation is applied to the edge output image to obtain a dilated image;

[0031] The dilated image is smoothed using a Gaussian filter to obtain a Gaussian-smoothed image;

[0032] The Gaussian-filtered smoothed image is normalized to obtain the first weight map;

[0033] The first weight map and the second weight map obtained based on the gradient direction cosine are weighted to obtain the fused weight map.

[0034] Guided by the fusion weight map, the left and right enhanced images are weighted to obtain the final left and right output images.

[0035] Furthermore, the construction of the disparity map based on the disparity values ​​of all pixels includes:

[0036] An edge mask is constructed using the edge output image, and the disparity map is smoothed in non-edge regions;

[0037] Based on the analysis of connected components, a minimum area threshold is set, and regions with areas smaller than the minimum area threshold are removed to obtain the final disparity map.

[0038] Furthermore, after denoising and scaling the colored 3D point cloud data, a point cloud set is obtained, including:

[0039] In the z-direction, the colored 3D point cloud data is denoised to obtain a denoised point cloud set;

[0040] Based on the actual depth of the denoised point cloud, determine the depth range; based on the depth range, determine the distance between the binocular camera and the pile cable; define the adjustment coefficient corresponding to the distance.

[0041] Define the noise standard deviation for quantizing point cloud noise levels;

[0042] The adaptive scaling factor is determined based on the adjustment coefficients corresponding to near and far states and the noise standard deviation.

[0043] The adaptive scaling factor is used to scale the denoised point cloud set to obtain the final point cloud set.

[0044] Furthermore, the extraction of the local point cloud set constituting the target cable from the point cloud set includes:

[0045] In the left image of binocular vision, the highest and lowest pixels of the target cable are selected sequentially through manual interaction, and the validity of the selected pixels is checked to ensure that the corresponding area has a valid disparity value.

[0046] The selected pixels are converted into three-dimensional spatial coordinates through parallax mapping, thereby obtaining the upper and lower boundary positions of the target cable in three-dimensional space;

[0047] Using the upper and lower boundary positions of the target cable in three-dimensional space as constraints, the point cloud set within the constraints is retained, and the local point cloud set constituting the target cable is extracted.

[0048] Furthermore, the adaptive determination of voxel side lengths based on the spatial extent of the local point cloud includes:

[0049] Calculate the extent of the local point cloud on the three coordinate axes;

[0050] Calculate the diagonal length within the spatial range of the local point cloud based on the range on the three coordinate axes;

[0051] The voxel side length is defined based on the diagonal length.

[0052] Furthermore, the calculation of the center point clouds of the top and bottom point clouds, and the calculation of the suspension height of the cable under test based on the center point clouds of the top and bottom point clouds, includes:

[0053] Calculate the center point cloud of the top and bottom point clouds;

[0054] Calculate the average distance from each point cloud in the top and bottom point clouds to its respective center point cloud, and use it as the adaptive distance threshold for RANSAC plane fitting; use the RANSAC plane fitting algorithm to fit the plane model of the top and bottom point clouds, obtain the normal vector, and normalize the normal vector.

[0055] Based on the normalized normal vector, calculate the projection distance between the two center point clouds in the direction of the normal vector, which is used as the suspension height of the pile foundation cable.

[0056] Beneficial Effects: This invention reads and corrects left and right binocular images of the cable area of ​​the pile foundation under test; performs edge detection and local contrast enhancement on the corrected images to optimize image quality; calculates the disparity map by optimizing the SGM algorithm and eliminates invalid disparities by combining connected component filtering and edge protection strategies; performs 3D reconstruction based on camera calibration parameters and effective disparities to generate an initial point cloud, and obtains a real-scale 3D point cloud model through adaptive depth scaling factor correction; filters the spatial region using y-axis constraints and extracts the local point cloud of the target cable using dual-point positioning; and calculates the cable suspension height using a top and bottom plane fitting method. Through the above technical means, this invention has the following advantages:

[0057] (1) The binocular vision image correction method designed in this invention corrects distortion and epipolar lines of the binocular image based on the parameters obtained by camera calibration using a chessboard image, forming the left and right two-dimensional images required for stereo matching, thus eliminating the influence of lens distortion and stereo epipolar geometric constraints; the stereo matching method designed for binocular vision images generates a fused weight map based on the weight map obtained by Canny edge detection and dilation and the weight map obtained by gradient direction cosine; and then generates a high-precision disparity map through the SGM algorithm and disparity post-processing strategy. This innovative design effectively overcomes the problem of decreased binocular vision matching accuracy under underwater dynamic conditions and significantly improves the accuracy of binocular vision in detecting the suspended height of pile foundation cables;

[0058] (2) The three-dimensional reconstruction method of cable image based on binocular vision designed in this invention reconstructs the point cloud data of pile foundation cable in three dimensions based on the binocular camera calibration parameters and pixel parallax. The reconstructed point cloud is filtered by using an effective parallax mask, retaining only the effective three-dimensional points with positive parallax. The point cloud is denoised by limiting the height range in the z direction. At the same time, an adaptive scaling factor is introduced to adjust the scale of the point cloud, thereby obtaining stable and accurate three-dimensional point cloud data of the cable. The designed cable point cloud extraction method adopts a target point cloud extraction strategy that combines manual prior constraints and axial interval filtering. The highest and lowest pixel points of the target cable are selected as the upper and lower boundary positions of the cable, and the upper and lower boundary positions are used as the filtering conditions. This solves the problem that the target cable is difficult to effectively distinguish from other structures such as pile foundation and marine organisms in complex underwater backgrounds, and realizes the accurate extraction of the target cable point cloud.

[0059] (3) The method for calculating the suspended height of pile foundation cable designed in this invention calculates the suspended height of cable by using a dual-plane height estimation method based on the downsampled cable point cloud obtained by adaptive voxel sampling. By fitting the dual-plane model of the upper and lower surfaces of the cable, the measurement deviation caused by local noise or deformation interference is effectively reduced, thereby significantly improving the calculation accuracy of the suspended height of pile foundation cable in dynamic underwater environment by binocular vision.

[0060] In summary, the method of this invention enables accurate measurement of the suspension height of pile foundation cables, providing an advanced and effective technical means for the management and maintenance of pile foundation cables. Attached Figure Description

[0061] Figure 1 This is a flowchart of the overall process for a method for detecting the suspension height of wind power pile cables based on binocular vision proposed in this invention.

[0062] Figure 2 Image before distortion correction. Figure 2 In the image, (a) represents the left image before distortion correction. Figure 2 (b) in the image represents the right image before distortion correction;

[0063] Figure 3 This is the image after distortion correction. Figure 3 In the image, (a) represents the left image after distortion correction. Figure 3 (b) in the image represents the right image after distortion correction;

[0064] Figure 4 Image corrected for epipolarity;

[0065] Figure 5 For normalized cumulative histogram;

[0066] Figure 6 The smoothed image after convolution. Figure 6 In the image, (a) represents the left-smoothed image. Figure 6 In the image, (b) represents the right-smoothed image;

[0067] Figure 7 For dilated images, Figure 7 In the image, (a) represents the left dilated image. Figure 7 (b) in the image represents the right-dilated image;

[0068] Figure 8 Parallax diagram of cable for pile foundation;

[0069] Figure 9 It is a 3D point cloud model;

[0070] Figure 10 The filtered point cloud;

[0071] Figure 11 These are left and right images taken in three different scenarios. Figure 11 In the image, (a) represents the left image in a daytime scenario. Figure 11 (b) in the image represents the right image in a daytime scenario. Figure 11 (c) in the image represents the left image in a nighttime setting; Figure 11 (d) in the image represents the right image in a nighttime setting; Figure 11 (e) in the image represents the left image in a nighttime scenario with the lights on. Figure 11 (f) in the image represents the right image in the scenario of lights being on at night;

[0072] Figure 12 These are disparity maps for three different scenarios. Figure 12 In the diagram, (a) represents the parallax map under daytime conditions. Figure 12 (b) in the diagram represents the parallax map for a nighttime scene. Figure 12 (c) in the diagram represents the parallax diagram under the condition of lights being on at night;

[0073] Figure 13 The disparity map results for the three contrast methods are shown in a daytime scenario. Figure 13 In the diagram, (a) represents the disparity map of the traditional SGM algorithm. Figure 13 In the diagram, (b) represents the disparity map of the BM algorithm. Figure 13 In the diagram, (c) represents the disparity map of the SGBM algorithm;

[0074] Figure 14 The disparity map results for the three contrast methods are shown in the scenario of lights being on at night. Figure 14 In the diagram, (a) represents the disparity map of the traditional SGM algorithm. Figure 14 In the diagram, (b) represents the disparity map of the BM algorithm. Figure 14 In the diagram, (c) represents the disparity map of the SGBM algorithm;

[0075] Figure 15 The disparity map results for the three contrast methods are shown in a nighttime scene. Figure 15 In the diagram, (a) represents the disparity map of the traditional SGM algorithm. Figure 15 In the diagram, (b) represents the disparity map of the BM algorithm. Figure 15 In the diagram, (c) represents the disparity map of the SGBM algorithm;

[0076] Figure 16 For each scenario, there are three possible selection outcomes. Figure 16 In the text, (A) represents three selection options under the daytime scenario, which include options (a), (b), and (c). Figure 16 In the text, (B) represents three selection results in the nighttime scenario, which include three selection results: (a), (b), and (c). Figure 16 (C) in the text represents three selection results under the scenario of lights being on at night, which include three selection results: (a), (b), and (c).

[0077] Figure 17 These are left and right images of an outdoor lake scene. Figure 17 (a) in the image represents the left view of an outdoor lake scene. Figure 17 (b) in the image represents the right view of an outdoor lake scene;

[0078] Figure 18A parallax diagram of an outdoor lake scene;

[0079] Figure 19 The three selection results are shown in the outdoor lake scenario, including (a), (b), and (c). Detailed Implementation

[0080] The technical solution of the present invention will now be further described in conjunction with the accompanying drawings and embodiments.

[0081] like Figure 1 As shown in the figure, this invention proposes a method for detecting the suspension height of wind power pile cables based on binocular vision, in order to obtain a highly accurate cable suspension height. The specific steps are as follows:

[0082] Step 1: Acquire left and right binocular images of the cable area of ​​the pile foundation to be tested, such as... Figure 2 As shown, corrections are performed. Specifically, the following operations are included:

[0083] Step 1-1: Normalize the coordinates (u, v) in the left and right binocular images I to obtain normalized coordinates (x, y):

[0084]

[0085] In the formula, K is the camera intrinsic parameter matrix K corresponding to the processing of the left and right binocular images respectively. L K R .

[0086] Step 1-2: Based on the radial and tangential distortion coefficients of the camera, perform distortion correction calculations on the normalized coordinates to obtain the distortion-free coordinates (x... u , y v ), generating as Figure 3 Image after distortion correction:

[0087]

[0088] In the formula, r is the normalized radius, r 2 = x 2 + y 2 k1 is the second-order coefficient of the radial distortion coefficient; k2 is the fourth-order coefficient of the radial distortion coefficient; k3 is the sixth-order coefficient of the radial distortion coefficient; p1 is the tangential distortion coefficient in the x-direction, and p2 is the tangential distortion coefficient in the y-direction.

[0089] Steps 1-3: Perform the following steps on the distortion-corrected image: Figure 4 The epipolar correction shown yields the homogeneous image coordinates [x] after epipolar correction. r , y r , w r ]:

[0090]

[0091] In the formula, H is the epipolar correction homography matrix calculated from the camera's intrinsic and extrinsic parameters.

[0092] Steps 1-4: Align the secondary image coordinates [x] r , y r , w r The normalization process is performed to convert it into two-dimensional image coordinates (x, y). s , y s ):

[0093] .

[0094] Step 2: Perform edge detection and local contrast enhancement on the corrected image to optimize image quality, specifically including the following operations:

[0095] Step 2-1: Convert the corrected image from RGB format to grayscale using the following grayscale conversion operation:

[0096]

[0097] In the formula, I R (x s , y s ), I G (x s , y s ), I B (x s , y s I(x) represents the red, green, and blue pixel values ​​of each pixel; s , y s () is the grayscale value after conversion for each pixel.

[0098] Step 2-2: Perform adaptive histogram equalization (CLAHE) on the grayscale image; firstly, divide the grayscale image into multiple adjacent small blocks, that is, divide the entire image into m × n sub-regions T. ij Where i∈[1, m], j∈[1, n]; then for each subregion T ij Calculate the grayscale histogram h of each. ij (k), which represents the total number of pixels with a grayscale value of k.

[0099]

[0100] In the formula, k∈{0, 1, 2, …, 255}; I ij (x, y) is the subregion T ij The grayscale value of an inner pixel, δ(I) ijThe definition of (x,y)-k) is as follows:

[0101]

[0102] The grayscale histogram is cropped to obtain the cropped histogram. Then, the normalized cumulative histogram C is calculated from the clipped histogram. ij (k), such as Figure 5 As shown, the range is [0, 1].

[0103] Step 2-3: Based on the normalized cumulative histogram C ij (k) Map the grayscale values ​​within the sub-region to the enhanced output grayscale value I. e (x, y) and bilinear interpolation is used to fuse adjacent sub-regions to enhance the result, resulting in left and right enhanced images I. L I R .

[0104]

[0105] In the formula, ⌊⋅⌋ represents floor function, ensuring that the output is an integer grayscale value.

[0106] Step 2-4: Extract image edge contours using the Canny edge detection algorithm for grayscale images: First, use a two-dimensional Gaussian filter to enhance the left and right images I. L I R Each pixel (x) s , y s Blur processing is performed:

[0107]

[0108] In the formula, G(x) represents the standard deviation of the Gaussian kernel, which defaults to 1.4; s , y s ) is a Gaussian kernel.

[0109] Then, the grayscale image is convolved with a Gaussian kernel to obtain a smoothed image I. s ,like Figure 6 As shown:

[0110]

[0111] Next, the Sobel algorithm is used to calculate the first-order gradient G of the image in the x and y directions. x and G y The x-axis gradient is used to characterize the gray-level changes of an image in the horizontal direction, and the y-axis gradient is used to characterize the gray-level changes of an image in the vertical direction.

[0112]

[0113] Then, the gradient magnitude M(x, y) and direction angle θ(x, y) are calculated based on the gradients in the x and y directions:

[0114]

[0115] Finally, non-maximum suppression is performed on the gradient magnitude, that is, the gradient magnitude of each pixel is compared with its neighboring pixels along the gradient direction, and edge thinning is achieved by retaining only local maxima. Based on this, a dual-threshold hysteresis connection strategy is designed to determine edges:

[0116]

[0117] In the formula, T h The high threshold is set to 0.2max(M); T l The threshold is set to 0.1max(M); max(M) is the maximum gradient magnitude of all pixels.

[0118] This yields the final edge output image E(x, y) composed of the edge points.

[0119] Steps 2-5: Apply image enhancement techniques to the edge output image to improve local contrast and detail discernibility, obtaining the left and right enhanced images I. L I R .

[0120] First, to expand the edge regions of the image, a dilation operation is performed on the edge output image, resulting in the following: Figure 7 The dilated image I shown e .

[0121] Then, Gaussian filtering is applied to smooth the dilated image:

[0122]

[0123] In the formula, the Gaussian kernel G(x, y) The value is 2; W′(x, y) is the image after Gaussian filtering and smoothing.

[0124] Finally, the smoothed image is normalized to obtain the fused weight map W. c (x, y):

[0125]

[0126] In the formula, min(W′(x, y)) and max(W′(x, y)) are the minimum and maximum values ​​of all pixels in the W′(x, y) image.

[0127] Finally, a dual-weight fusion-based guidance enhancement strategy is introduced. This strategy uses a weight map W obtained based on Canny edge detection and dilation. c And the weight map W obtained based on gradient direction cosine g This enables joint guidance of the image enhancement process. The weight map W... g The calculation is as follows:

[0128]

[0129] For the weighted graph W c and W g The weighted processing yields the fused weighted graph W. f (x, y):

[0130]

[0131] In the formula, α∈[0, 1] is a parameter that is adaptively adjusted according to the mean of local variance. When the mean of local variance of the image is less than or equal to 5, α takes the value of 0.7; when the mean of local variance of the image is between 5 and 15, α takes the value of 0.5; when the mean of local variance of the image is greater than or equal to 15, α takes the value of 0.3.

[0132] In the fusion weight map W f Guided by (x, y), the left and right enhanced images I after CLAHE processing are processed. L (x, y) and I R We perform weighted summation on (x, y) to obtain the left and right final output images. , :

[0133]

[0134] .

[0135] Step 3: Calculate the disparity map by optimizing the SGM algorithm, and remove invalid disparities by combining connected component filtering and edge protection strategies. Specific operations include:

[0136] The pixel matching cost is calculated using a matching cost function based on absolute difference (SAD):

[0137]

[0138] In the formula, I mL (x, y), I mR (x, y) represents the grayscale value of pixel (x, y) in the left and right final output images; C(x, y, d) represents the matching cost of pixel (x, y) in the left final output image when the disparity is d; d is the disparity value; i and j are the relative offsets of pixels within the matching window; w is the window radius.

[0139] Then, based on C(x, y, d), the cost is aggregated in multiple directions using the SGM algorithm to obtain the final aggregated cost value S(x, y, d). For each pixel (x, y), a disparity value d that minimizes S(x, y, d) is selected and used as the disparity value of pixel (x, y), thus constructing a disparity map based on the disparity values ​​of all pixels.

[0140] Finally, a disparity post-processing strategy combining edge-preserving smoothing and connected component area filtering is introduced. First, the edge output image is obtained using Canny edge detection results, and the disparity map is smoothed in non-edge regions to suppress noise and preserve disparity boundaries. Then, based on connected component analysis, a minimum area threshold is set, and isolated mismatched regions with excessively small areas are removed, effectively eliminating invalid disparity points and thus obtaining the desired result. Figure 8 The final disparity map shown.

[0141] Step 4: Perform 3D reconstruction based on camera calibration parameters and effective parallax to generate an initial point cloud. Then, obtain a true-scale 3D point cloud model through adaptive depth scaling factor and underwater refractive index correction. Next, use y-axis constraints to filter spatial regions and combine dual-point positioning to extract local point clouds of the target cable. Specifically, this includes the following operations:

[0142] Step 4-1: Since binocular vision acquires two-dimensional images, it cannot obtain the suspension height information in three-dimensional space through two-dimensional means. Therefore, it is necessary to reconstruct the image in three dimensions to obtain three-dimensional point cloud data. The binocular vision image of the pile foundation cable is reconstructed in three dimensions to obtain the three-dimensional point cloud data of the pile foundation cable, which facilitates subsequent extraction of the cable point cloud and calculation of the suspension height.

[0143] The actual depth Z(x, y) is obtained from the disparity d(x, y) of each pixel (x, y) in the final disparity map:

[0144]

[0145] In the formula, f is the camera focal length; B is the length of the left and right baselines of the camera, both of which are parameters obtained from camera calibration;

[0146] Transform pixel coordinates into camera coordinates, that is, reconstruct X(x, y) and Y(x, y) in 3D coordinates based on pixel coordinates (x, y) and the actual depth Z(x, y):

[0147]

[0148] In the formula, f x f y c is the focal length of the image in the x and y directions; x cy These are the coordinates of the image center, all obtained from camera calibration.

[0149] Process each pixel according to the above steps to obtain its three-dimensional coordinates, thus constructing a complete three-dimensional point cloud model, such as... Figure 9 As shown.

[0150] An effective disparity mask is constructed, which only retains 3D point clouds with disparity d(x, y) > 0, and the left image is converted into an RGB image. Then, it is normalized to map the colors onto the 3D point cloud to obtain colored 3D point cloud data.

[0151] Step 4-2: The 3D point cloud generated by binocular vision contains a lot of divergent noise. Furthermore, the size of the 3D point cloud obtained by binocular vision varies depending on the shooting distance, which can lead to significant deviations in the estimated object height. To ensure accurate calculation of the cable suspension height, the point cloud data is denoised and its scale adjusted.

[0152] By directly retaining the point cloud data in the z-direction [0m, 0.3m], noise can be removed accurately and effectively. The denoised point cloud set is P(p i =(x, y, z));

[0153] An adaptive scaling factor is used to adjust the scale of the denoised point cloud data. The adaptive scaling factor depends on two factors: the scene depth range and the point cloud noise level. Assume the depth set of the entire point cloud set P is Z. c First, obtain its depth range R. Z :

[0154]

[0155] In the formula, P 95 P5 and P6 are the 95th and 5th percentiles of the depth value set, respectively, which are the depth values ​​located at the 95th and 5th percentiles after all depth values ​​are sorted from smallest to largest.

[0156] Quantize the point cloud noise level, and then define the noise standard deviation σ. z :

[0157]

[0158] In the formula, N is Z c The number of effective depth points in Z; m For Z c The median depth;

[0159] Then based on the depth range R Z To determine the distance between the camera and the pile foundation cable, the following three distance modes are defined:

[0160]

[0161] In the formula, R near R far Near and far thresholds;

[0162] The depth range adjustment factor k for each distance mode d Noise adjustment factor k n The empirical bias b is set as follows:

[0163]

[0164] Calculate the scaling factor α based on the identified pattern:

[0165]

[0166] In the formula, R0 is the empirical parameter for depth span; σ0 is the noise normalization parameter;

[0167] Finally, the point cloud set P is scaled up to obtain the point cloud set P. α ;

[0168] Step 4-3: Due to interference from noise points and non-target objects in the complex underwater environment, in order to obtain the point cloud P α To separate the seabed plane and achieve subsequent point cloud segmentation, a target point cloud extraction strategy combining manual prior constraints and axial interval screening is adopted.

[0169] In the left image of binocular vision, the highest and lowest pixels of the target cable are selected sequentially through manual interaction, and the validity of the selected pixels is checked to ensure that their corresponding regions have valid disparity values. Subsequently, based on the obtained final disparity map, the selected two-dimensional pixels are converted into three-dimensional spatial coordinates through disparity mapping, thereby obtaining the upper and lower boundary positions of the target cable in three-dimensional space. The upper and lower boundary positions are used as constraints to retain the point cloud P. α The point cloud located within the constraints constitutes the local point cloud set P of the target cable. g ,like Figure 10 As shown.

[0170] Step 5: Calculate the cable suspension height using the upper and lower plane fitting method. Specific operations include:

[0171] Step 5-1: To reduce point cloud density, alleviate computational burden, and preserve spatial structure information, voxel sampling is commonly used to downsample the point cloud. Voxel sampling effectively reduces the number of points and suppresses redundant information by dividing the point cloud space into regular cubic voxels and replacing the original point set with representative points within each voxel. However, fixing voxel sizes makes it difficult to balance sampling accuracy and computational efficiency in point cloud data of different scales. Therefore, this embodiment of the invention employs an adaptive voxel sampling method, adaptively determining the voxel side length based on the spatial range of the point cloud, enabling it to adapt to point cloud data of different scales.

[0172] Calculate the point cloud P g (p i =(p xi , p yi , p zi The range of b on the three coordinate axes for i∈1, 2, …, m) x ,b y , b z :

[0173]

[0174] Calculate the diagonal length L within the spatial range of the point cloud based on the range of the three coordinate axes:

[0175]

[0176] The voxel side length v is defined based on the diagonal length L:

[0177]

[0178] In the formula, k is an empirically set sampling resolution parameter, with a default value of 50.

[0179] The point cloud data space is divided into a cubic voxel mesh with side length v. For all points within each voxel, its geometric centroid is calculated as the representative point c of that voxel. j The calculated representative points form a new downsampling point cloud P. d :

[0180]

[0181] In the formula, V i Let V be the set of point clouds contained within the i-th voxel; i | represents the number of point clouds in this voxel; M represents the number of voxels.

[0182] Step 5-2: Due to the point cloud set P dThere may be anomalous point clouds and noise in the data. If the difference between the maximum and minimum height values ​​is used directly to estimate the height of an object, it will be affected by these point clouds, leading to unstable or large deviations in the estimated height. The point clouds at the top and bottom are more concentrated and flatter, making them suitable for fitting as planes. This can more accurately reflect the true upper and lower boundaries of the object. Therefore, planes are fitted separately for the upper and lower point clouds, and the height difference between the two planes is calculated as the cable suspension height.

[0183] For the downsampling point set P d (p i =(x i , y i , z i The 95th and 5th quantiles of all point cloud coordinates along the z-axis are calculated and used as thresholds t. top t bottom According to the threshold t top t bottom Select the top point cloud and bottom point cloud sets P respectively. top P bottom :

[0184]

[0185] For the top point cloud and bottom point cloud set P top P bottom Calculate the center point cloud c of the two point cloud sets respectively. top c bottom :

[0186]

[0187] Calculate the average distance d from each point cloud in the two point cloud sets to its respective center point cloud. top d bottom The two sets of average distances reflect the spatial dispersion of the point cloud within a local region and can be used as an adaptive distance threshold for RANSAC plane fitting.

[0188]

[0189] In the formula, ||·||2 is the L2 norm, representing the value of point p. i The centroid of the cloud at the top, c top Spatial distance.

[0190] The RANSAC algorithm is used to fit a planar model of the upper and lower point clouds to obtain normal vectors, which are then normalized. Finally, the normalized normal vector n is used as the basis for the calculation. c Calculate the projected distance between the two center point clouds along the normal vector direction, and use it as the suspension height h of the pile-foundation cable:

[0191]

[0192] This embodiment focuses on binocular images of wind turbine pile cables acquired using a binocular camera. After correcting the binocular images, edge detection and local contrast enhancement are performed to optimize image quality. The disparity map is calculated using an optimized SGM algorithm, and invalid disparities are eliminated by combining connected component filtering and edge protection strategies. A true-scale 3D point cloud model is obtained using 3D reconstruction, adaptive depth scaling factor, and underwater refractive index correction. Spatial regions are then filtered using y-axis constraints, and local point clouds of the target cable are extracted using dual-point localization. The suspension height of the pile cable is calculated using a top and bottom plane fitting method, thereby improving the detection efficiency and accuracy of the pile cable suspension height under binocular vision.

[0193] To verify the effectiveness of the proposed binocular vision-based method for detecting the suspended height of wind power pile foundation cables, the measurement objects were two black marked points on the cable and the suspended height of the cable apex from the bottom plane. The measurement was carried out in four scenarios: daytime, nighttime, nighttime with lights on, and outdoor lake scene.

[0194] Figure 11 Left and right images taken in three scenarios: daytime, nighttime, and nighttime with lights on. Figure 12 Here are the disparity maps for these three scenarios. Figure 13 , 14 Figures 1 and 15 show the results of comparing the disparity maps generated by the proposed method with those generated by the traditional SGM, BM, and SGBM algorithms under daytime, nighttime (with lights on), and nighttime scenarios, respectively. Figure 13 , 14 As can be seen from Figure 15, the BM algorithm exhibits poor matching stability under conditions of weak texture and strong lighting interference. Numerous holes and noise areas exist in the disparity map, making it difficult to obtain continuous and reliable depth information. The traditional SGM algorithm improves matching continuity to some extent, but significant disparity breaks and mismatches still occur in areas of strong reflection and sudden changes in local brightness. The SGBM algorithm performs relatively stably in smooth areas, but disparity blurring easily occurs at cable edges and shadow boundaries, and its ability to preserve details is limited.

[0195] In contrast, the optimized SGM algorithm proposed in this embodiment of the invention can effectively suppress matching interference caused by strong light reflection under complex lighting conditions such as nighttime lighting, and significantly reduce parallax noise and hole areas. The resulting parallax map has better continuity and integrity within the target area, with clear cable outlines and well-preserved edge structures, providing more stable and reliable depth information for subsequent 3D reconstruction and point cloud extraction.

[0196] Based on the disparity map obtained by the optimized SGM algorithm, the point cloud data of the cable model was reconstructed in 3D. Figure 11 In the three typical scenarios shown, based on the left image of each scenario, two feature points are manually selected from the top and bottom of the image. Specifically, two black marker points on the cable and one feature point at the cable vertex are selected first. Then, the point on the bottom plane that is vertically projected from this feature point is selected and mapped into a 3D point cloud space. Subsequently, all point cloud data located between these two 3D point clouds are extracted. The left image of each scenario is selected three times based on the two black marker points on the cable and the cable vertex; that is, three suspended region point clouds are selected in each group, as shown below. Figures 16-18 As shown, this method can effectively extract the desired point cloud region and maintain the geometric structural features of the target object well, achieving accurate reconstruction of the object's basic shape.

[0197] The cable suspension height for each group can be fully reconstructed and extracted in 3D. The final nine height selection results for the three scenarios are shown in Table 1. As can be seen from the table, the maximum detection error for the nine selection results under the proposed method is 6.40%, and the average detection error is 2.58%. For the nine selection results, the suspension height results calculated using the Y-axis range method and the 95th percentile height method were compared with the suspension results obtained by the proposed algorithm. The maximum detection errors for the two comparison methods were 26.43% and 28.96%, respectively, and the average detection errors were 12.14% and 11.94%, respectively. Compared to the error percentages of the Y-axis range method and the 95th percentile height method, the proposed method exhibits a lower and more stable error level for each selection result. This indicates that the proposed height detection algorithm can still effectively and accurately calculate the cable suspension height under different lighting conditions.

[0198] Table 1

[0199]

[0200] This embodiment presents a method for detecting the suspended height of wind turbine pile cables, designed to address the three binocular vision shooting scenarios described above. After acquiring left and right binocular images of the wind turbine pile cable area and performing stereo correction, the corrected images undergo edge detection and local contrast enhancement to reduce interference from factors such as uneven underwater illumination, image blur, and complex seabed textures on cable contour recognition, thus providing high-quality input for subsequent disparity calculation. Based on this, and considering the characteristics of point cloud scale variations under different observation distances and imaging conditions, stable and reliable 3D point cloud data is obtained through a combination of effective disparity filtering and depth interval constraints, reducing the impact of invalid disparities and environmental noise on subsequent processing. Furthermore, an adaptive scaling factor based on point cloud depth distribution is introduced to uniformly adjust the point cloud scale under different shooting conditions, reducing the estimation deviation of suspended height caused by changes in shooting distance and the accumulation of reconstruction errors, thereby ensuring the consistency of 3D reconstruction results under different scenarios. Subsequently, a target point cloud extraction strategy combining manual prior constraints and axial interval filtering was adopted. By selecting the highest and lowest pixels of the target cable in the binocular image to determine its upper and lower boundary positions in three-dimensional space, the point cloud was filtered accordingly, achieving accurate extraction of the target cable point cloud. Based on the extracted pile foundation cable point cloud and the corresponding seabed point cloud, a stable suspension height calculation model was constructed, effectively reducing the impact of outliers and local undulations on the height estimation results, thereby improving the accuracy and reliability of the suspension height detection of wind power pile foundation cables based on binocular vision.

[0201] Figure 17 These are left and right images of an outdoor lake scene. Figure 18 To perform stereo matching on outdoor lake binocular images and obtain disparity maps using the optimized SGM algorithm, Figure 19 Three selection results are presented for an outdoor lake scene. Based on the obtained disparity map, the point cloud data of the cable model is reconstructed in 3D. Figure 17 In (a), two feature points are manually selected, one above the other, and mapped onto a 3D point cloud space. Then, all point cloud data located between these two 3D point clouds are extracted, and manual selection is performed three times, again using the two black marked points and the cable vertex. A dual-plane height estimation method is used to calculate the cable height, and these three heights are used as the three possible suspension heights of the cable. The calculation results are shown in Table 2. As can be seen from the table, in an outdoor lake scene, the maximum detection error of the three selection results is 4.40%, and the average detection error is 3.40%. This indicates that the wind turbine pile cable suspension height detection method of this patent can still effectively and accurately calculate the suspension height in actual outdoor lake scenarios.

[0202] Table 2

[0203]

[0204] Therefore, the binocular vision-based method for detecting the suspension height of wind power pile cables in this embodiment achieves accurate detection of the suspension height of pile cables by accurately reconstructing the pile cables in three dimensions, thus ensuring efficient and timely operation and maintenance of pile cables.

Claims

1. A method for detecting the suspension height of wind turbine pile foundation cables based on binocular vision, characterized in that: Includes the following steps: Acquire left and right eye images of the cable area of ​​the pile foundation to be tested, and correct the left and right eye images so that the corresponding points in the corrected left and right eye images are at the same horizontal height; Edge detection and local contrast enhancement are performed on the corrected left and right eye images to obtain the final left and right output images; Based on the final output images on the left and right, a matching cost function based on absolute difference is used to calculate the pixel matching cost. According to the pixel matching cost, the cost is aggregated in multiple directions using the SGM algorithm to obtain the aggregated cost value. For each pixel, the disparity value that minimizes the aggregated cost value is selected and used as the disparity value of that pixel. A disparity map is constructed based on the disparity values ​​of all pixels. The actual depth is calculated based on the disparity value of each pixel in the disparity map. The three-dimensional coordinates of each pixel are then reconstructed based on the coordinates of each pixel in the disparity map and the actual depth, forming a complete three-dimensional point cloud data. The final output image is converted into an RGB image, and the colors are mapped onto the three-dimensional point cloud data to obtain colored three-dimensional point cloud data. After denoising and scaling the colored three-dimensional point cloud data, a point cloud set is obtained. The local point cloud set constituting the target cable is extracted from the point cloud set. Based on the spatial range of the local point cloud set, the voxel side length is adaptively determined, and the point cloud data space is divided into a cubic voxel grid with a side length equal to the voxel side length. For all points in each cubic voxel grid, its geometric centroid is calculated as the representative point of the cubic voxel grid, and all representative points form a downsampled point cloud set. Calculate the 95th and 5th percentiles of the coordinate set of all points in the downsampled point cloud set along the z-axis, and use them as the top threshold and bottom threshold, respectively. Select the top point cloud set and the bottom point cloud set based on the top threshold and the bottom threshold. Calculate the center point cloud of the top point cloud and the bottom point cloud, and calculate the suspension height of the cable of the pile foundation to be tested based on the center point cloud of the top point cloud and the bottom point cloud.

2. The method for detecting the suspension height of wind turbine pile foundation cables based on binocular vision according to claim 1, characterized in that: The process of performing edge detection and local contrast enhancement on the corrected left and right eye images to obtain the final left and right output images includes: The corrected left and right eye images are converted from RGB format to grayscale images using a grayscale conversion operation. The grayscale image is divided into multiple sub-regions, and the grayscale histogram of each sub-region is calculated. The grayscale histogram of each sub-region is then cropped and normalized in sequence to obtain the normalized cumulative histogram. Based on the normalized cumulative histogram, the enhanced gray values ​​are obtained. The gray values ​​in the sub-regions are mapped to the enhanced gray values, and bilinear interpolation is used to fuse adjacent sub-regions to obtain the left and right enhanced images. The Canny edge detection algorithm is used to extract the edge contours of the left and right enhanced images to obtain the edge output image; Based on the edge output image, image enhancement is performed on the left and right enhanced images to obtain the final left and right output images.

3. The method for detecting the suspension height of wind turbine pile foundation cables based on binocular vision according to claim 2, characterized in that: The method employs the Canny edge detection algorithm to extract the edge contours of the left and right enhanced images, obtaining an edge output image, including: A two-dimensional Gaussian filter is used to blur each pixel in the left and right enhanced images; A smoothed image is obtained by convolving a grayscale image with a Gaussian kernel. The Sobel algorithm is used to calculate the first-order gradient of a smoothed image in the x and y directions. Calculate the gradient magnitude and direction angle based on the first-order gradients in the x and y directions; Perform non-maximum suppression operation on the gradient magnitude to obtain edge points; A dual-threshold hysteresis connection strategy is adopted to determine the edge points, obtain the final edge points, and form the edge output image from the final edge points; The dual-threshold hysteresis connection strategy is expressed as follows: In the formula, T h For a high threshold, T l The threshold is set to low, and M(x, y) represents the gradient magnitude.

4. The method for detecting the suspension height of wind turbine pile foundation cables based on binocular vision according to claim 2, characterized in that: The step of enhancing the left and right enhanced images based on the edge output image to obtain the final left and right output images includes: A dilation operation is applied to the edge output image to obtain a dilated image; The dilated image is smoothed using a Gaussian filter to obtain a Gaussian-smoothed image; The Gaussian-filtered smoothed image is normalized to obtain the first weight map; The first weight map and the second weight map obtained based on the gradient direction cosine are weighted to obtain the fused weight map. Guided by the fusion weight map, the left and right enhanced images are weighted to obtain the final left and right output images.

5. The method for detecting the suspension height of wind turbine pile foundation cables based on binocular vision according to claim 2, characterized in that: The disparity map, constructed based on the disparity values ​​of all pixels, includes: An edge mask is constructed using the edge output image, and the disparity map is smoothed in non-edge regions; Based on the analysis of connected components, a minimum area threshold is set, and regions with areas smaller than the minimum area threshold are removed to obtain the final disparity map.

6. The method for detecting the suspension height of wind turbine pile foundation cables based on binocular vision according to claim 1, characterized in that: After denoising and scaling the color 3D point cloud data, a point cloud set is obtained, including: In the z-direction, the colored 3D point cloud data is denoised to obtain a denoised point cloud set; Based on the actual depth of the denoised point cloud, determine the depth range; based on the depth range, determine the distance between the binocular camera and the pile cable; define the adjustment coefficient corresponding to the distance. Define the noise standard deviation for quantizing point cloud noise levels; The adaptive scaling factor is determined based on the adjustment coefficients corresponding to near and far states and the noise standard deviation. The adaptive scaling factor is used to scale the denoised point cloud set to obtain the final point cloud set.

7. The method for detecting the suspension height of wind turbine pile foundation cables based on binocular vision according to claim 1, characterized in that: The extraction of the local point cloud set constituting the target cable from the point cloud set includes: In the left and right eye images of the cable area of ​​the pile foundation to be tested, the highest and lowest pixel points of the target cable are selected sequentially through manual interaction, and the validity of the selected pixel points is checked to ensure that the corresponding area has a valid parallax value. The selected pixels are converted into three-dimensional spatial coordinates through parallax mapping, thereby obtaining the upper and lower boundary positions of the target cable in three-dimensional space; Using the upper and lower boundary positions of the target cable in three-dimensional space as constraints, the point cloud set within the constraints is retained, and the local point cloud set constituting the target cable is extracted.

8. The method for detecting the suspension height of wind turbine pile foundation cables based on binocular vision according to claim 1, characterized in that: The adaptive determination of voxel side length based on the spatial range of the local point cloud includes: Calculate the extent of the local point cloud on the three coordinate axes; Calculate the diagonal length within the spatial range of the local point cloud based on the range on the three coordinate axes; The voxel side length is defined based on the diagonal length.

9. The method for detecting the suspension height of wind turbine pile foundation cables based on binocular vision according to claim 1, characterized in that: The calculation of the center point clouds of the top and bottom point cloud sets, and the calculation of the suspension height of the cable under test based on the center point clouds of the top and bottom point cloud sets, includes: Calculate the center point cloud of the top and bottom point clouds; Calculate the average distance from each point cloud in the top and bottom point clouds to its respective center point cloud, and use it as the adaptive distance threshold for RANSAC plane fitting; use the RANSAC plane fitting algorithm to fit the plane model of the top and bottom point clouds, obtain the normal vector, and normalize the normal vector. Based on the normalized normal vector, calculate the projection distance between the two center point clouds in the direction of the normal vector, which is used as the suspension height of the pile foundation cable.

Images