Underwater concrete three-dimensional reconstruction method based on binocular stereo vision in two-dimensional image domain

Abstract

The application discloses a two-dimensional image field-based binocular stereovision underwater concrete three-dimensional reconstruction method, and belongs to the technical field of binocular stereovision underwater concrete three-dimensional reconstruction. The method comprises the following steps: acquiring an underwater video sequence, extracting a key frame and constructing a two-dimensional time sequence motion model; based on the two-dimensional time sequence motion model, adopting two-dimensional motion-texture joint denoising to generate an optimized noise mask image; based on the optimized noise mask image, adopting a mask-guided stereomatching method to establish a disparity map, and constructing a three-dimensional point cloud according to the disparity map; and performing statistical filtering and voxel grid time sequence frequency screening on the three-dimensional point cloud to generate a pure underwater concrete three-dimensional point cloud. The application realizes the core target of 'time sequence denoising + detail reservation', has the advantages of low calculation cost, good real-time performance, and suitability for rapid reconstruction of massive underwater video data, solves the problem of resource waste caused by invalid calculation, and significantly improves the real-time performance of the algorithm.

Description

Technical Field

[0001] This invention belongs to the field of binocular stereo vision underwater concrete 3D reconstruction technology, specifically relating to a binocular stereo vision underwater concrete 3D reconstruction method based on a two-dimensional image domain. Background Technology

[0002] Binocular stereo vision technology, as a non-contact, high-precision measurement method, is widely used in the 3D reconstruction and surface defect detection of underwater concrete structures such as dams and bridge piers. However, the complex underwater environment, strong scattering effects, and interference from massive suspended particles pose severe challenges to 3D reconstruction, and different application requirements face drastically different technical bottlenecks.

[0003] For scenarios focusing on large-scale real-time inspection, traditional processes perform indiscriminate stereo matching and reconstruction on images containing a lot of noise. This not only leads to a significant waste of computing power on invalid data that needs to be removed, but also causes the high-frequency textures of suspended objects to severely interfere with the disparity calculation accuracy of stereo matching, making it difficult to meet the requirements of fast response and real-time processing. Existing technologies have the following drawbacks:

[0004] (1) Ineffective computation leads to resource waste: Existing technologies perform stereo matching and three-dimensional back projection on the whole map (containing a lot of noise), spending a lot of computing power to reconstruct the "garbage data" that will eventually be deleted, resulting in slow processing speed and difficulty in meeting the needs of large-scale inspection.

[0005] (2) Noise interference in stereo matching leads to mismatch: The high-frequency texture of suspended particles can interfere with stereo matching algorithms (such as SGBM), causing the algorithm to produce mismatches at the edges of structures and generate incorrect depth information.

[0006] (3) Single scale is difficult to deal with multiple types of suspended matter: underwater suspended matter varies in size, and single-scale detection methods are prone to missed detection or misjudgment.

[0007] (4) Two-dimensional denoising may damage the edges of the structure: radical two-dimensional image denoising (such as Gaussian blur) can easily blur the edges of concrete cracks, making it impossible to restore the clear outline of the disease in subsequent reconstruction. Summary of the Invention

[0008] To address the aforementioned shortcomings of existing technologies, the binocular stereo vision-based underwater concrete 3D reconstruction method based on a 2D image domain provided by this invention solves the following problems existing in the prior art:

[0009] (1) The problem of resource waste caused by invalid calculation.

[0010] (2) Noise interference in stereo matching leads to mismatch.

[0011] (3) A single scale is insufficient to address the problem of multiple types of suspended matter.

[0012] (4) Two-dimensional denoising may damage the edges of the structure.

[0013] To achieve the aforementioned objectives, the present invention employs the following technical solution: a binocular stereo vision-based three-dimensional reconstruction method for underwater concrete based on a two-dimensional image domain, comprising the following steps:

[0014] S1. Acquire underwater video sequences, extract key frames, and construct a two-dimensional temporal motion model;

[0015] S2. Based on a two-dimensional temporal motion model, a two-dimensional motion-texture joint denoising method is used to generate an optimized noise mask image;

[0016] S3. Based on the optimized noise mask image, a disparity map is established using a mask-guided stereo matching method, and a 3D point cloud is constructed based on the disparity map.

[0017] S4. Perform statistical filtering and voxel rasterization temporal frequency screening on the 3D point cloud to generate a pure underwater concrete 3D point cloud.

[0018] Furthermore, S1 includes the following sub-steps:

[0019] S11. Obtain underwater video sequences, and extract the most representative keyframe sequences by analyzing the number and distribution uniformity of SIFT feature point matching between adjacent frames.

[0020] S12. Based on the keyframe sequence, a multi-scale image is constructed for each keyframe using a Gaussian pyramid.

[0021] S13. Calculate the optical flow vectors of pixels between adjacent keyframes using the dense optical flow method;

[0022] S14. Calculate the pixel motion variance based on the optical flow vector, obtain the motion trajectory of each pixel, and construct a two-dimensional temporal motion model.

[0023] The beneficial effects of the above-mentioned further solutions are as follows: In order to adapt to underwater suspended particles of different sizes, the present invention constructs a multi-scale image pyramid for each key frame, analyzes the motion characteristics of pixels at multiple resolution levels, realizes full coverage detection of particles of different sizes, improves the integrity of denoising, and avoids missing small-scale particles or misjudging large-scale structural points.

[0024] Furthermore: In S12, the multi-scale image comprises several stacked image layers, each obtained by Gaussian smoothing and downsampling the previous layer. The specific expression for the pixel grayscale value of the multi-scale image is as follows:

[0025]

[0026] In the formula, The pixel grayscale value of the image at layer 0. Let be the coordinates of the pixel in the horizontal direction of the image. These are the coordinates of the pixel in the vertical direction of the image. For the first The pixel grayscale values ​​of the layer image. For Gaussian kernel function, For convolution operations, For convolution operation on the th The layer image is downsampled by 2 times. Keyframe images are input for the Gaussian pyramid;

[0027] In S13, the dense optical flow method is used to calculate the optical flow vectors of pixels between adjacent keyframes. The specific method is as follows:

[0028] like Then solve for the optical flow vector. ,in, for Moment image in grayscale value at that location for Moment image in grayscale value at that location The horizontal coordinate is... The vertical coordinate;

[0029]

[0030]

[0031] In the formula, for The coefficient matrix of the time-space polynomial expansion. for The coefficient matrix of the time-space polynomial expansion. It is the transpose symbol;

[0032] In S14, for continuous Frame optical flow vector Calculate pixel motion variance The specific expression is:

[0033]

[0034] In the formula, for Average displacement along the axial direction for Average displacement along the axial direction For the first The horizontal coordinate of the frame optical flow vector. For the first The vertical coordinate of the frame optical flow vector.

[0035] The beneficial effects of the above-mentioned further solutions are as follows: Based on the "constant brightness assumption" and the "spatial smoothness assumption", this invention fits the local image brightness distribution through polynomial expansion, solves the pixel motion displacement, and obtains a dense optical flow vector field, thereby characterizing the temporal motion characteristics of each pixel; the dense optical flow algorithm does not require feature point detection, can obtain the motion information of all pixels, and is suitable for subsequent global denoising processing.

[0036] Furthermore, S2 includes the following sub-steps:

[0037] S21. Set a motion variance threshold. If the motion variance of a pixel is greater than the motion variance threshold, then mark the pixel as a suspicious noise pixel and generate a noise mask image.

[0038] S22. For regions with a mask value of 0 in the noise mask image, perform adaptive histogram equalization to improve the texture clarity of the concrete target region.

[0039] S23. Perform morphological opening operation on the noise mask image to generate an opening-optimized noise mask image;

[0040] S24. Identify strong texture edge regions using Canny edge detection, reset the mask value corresponding to the identified region to 0, and generate an optimized noise mask image.

[0041] The beneficial effects of the above-mentioned further solutions are as follows: The present invention divides the image into multiple small regions through adaptive histogram equalization, performs histogram equalization on each region, and at the same time suppresses excessive noise enhancement by limiting contrast, which is suitable for texture enhancement of underwater low-contrast images. It can effectively enhance the contrast of detailed areas such as concrete cracks, and provide clearer image features for subsequent stereo matching.

[0042] Further: In S21, a noise mask image is generated. The specific expression is:

[0043]

[0044] In the formula, This is the motion variance threshold;

[0045] In S23, morphological opening operations are used to remove small-scale isolated noise while preserving large-scale connected regions, generating an opening-optimized noise mask image. The specific expression is:

[0046]

[0047] In the formula, The image is the noise mask before optimization. As a structural element, For erosion calculation, For expansion operation;

[0048] In S24, the method for identifying strong texture edge regions using Canny edge detection is as follows:

[0049] S241. Based on the keyframe images in the keyframe sequence, Gaussian smoothing is applied to the image. The image after Gaussian smoothing is... The specific expression is:

[0050]

[0051] S242. Calculate the gradient magnitude. and edge direction This yields the gradient image;

[0052]

[0053]

[0054] In the formula, for Sobel gradient in the axial direction, for Sobel gradient along the axial direction;

[0055] S243. Perform non-maximum suppression on the gradient image to generate a non-maximum suppressed gradient image. The specific expression is:

[0056]

[0057] S244. Set high and low thresholds, perform double-threshold hysteresis segmentation on the gradient image after non-maximum suppression, and generate a binary edge image to mark the strong texture edge regions that need to be forcibly preserved. The binary edge image... The specific expression is:

[0058]

[0059] In the formula, For high threshold, The threshold is low.

[0060] The beneficial effects of the above-mentioned further solutions are as follows: the present invention accurately identifies these regions through Canny edge detection and resets their corresponding mask values ​​to 0 to prevent them from being mistakenly marked as noise and removed; the dual threshold hysteresis segmentation ensures the continuity of the edges, avoids the breakage of strong edges, and achieves the goal of "denoising without losing key details".

[0061] Furthermore, S3 includes the following sub-steps:

[0062] S31. Based on the optimized noise mask image, the matching cost is corrected by the mask-guided improved SGBM stereo matching method to generate a disparity map.

[0063] S32. Perform bilateral filtering optimization on the disparity map, and use spatial proximity and color similarity to perform weighted smoothing on the disparity map to generate an optimized disparity map.

[0064] S33. For the optimized disparity map, perform 3D backprojection only on reliable disparity values ​​to generate a pre-denoised 3D point cloud.

[0065] The beneficial effects of the above-mentioned further solutions are as follows: For the noisy region with a mask value of 1, the present invention reduces the matching weight, narrows its disparity search range, and reduces invalid calculations; for the reliable region with a mask value of 0, normal matching weight and disparity search range are used to ensure matching accuracy.

[0066] Furthermore: In S31, the method for correcting the matching cost through the mask-guided improved SGBM stereo matching method is as follows:

[0067] In the cost calculation stage of SGBM stereo matching, the matching cost is corrected using a mask-guided weighted matching cost method. The weighted matching cost is then used for path aggregation and disparity selection. The specific expression is:

[0068]

[0069] In the formula, Pixels calculated for SGBM stereo matching In parallax The matching cost below, Matching weights guided by the mask;

[0070]

[0071] In the formula, For the preset small weight, The preset normal weights, For optimized noise masking images;

[0072] In S32, bilateral filtering optimization is performed on the disparity map, and the disparity values ​​after bilateral filtering are... The specific expression is:

[0073]

[0074] In the formula, These are the disparity values ​​of the disparity map. For filtering neighborhood windows, For the spatial Gaussian standard deviation, The standard deviation of grayscale Gaussian is given. The grayscale value of the neighboring pixels within the bilateral filtering neighborhood window. The gray value of the center pixel within the bilateral filtering neighborhood window. For normalized weighted sum;

[0075]

[0076] S33 specifically refers to: determining pixels Corresponding disparity value If the value is 0, then use its disparity value for 3D back projection to generate a valid point cloud; otherwise, discard the pixel directly.

[0077] The beneficial effects of the above-mentioned further solutions are as follows: The present invention uses mask-constrained 3D point cloud generation to remove the disparity values ​​corresponding to noisy pixels from the source and only performs 3D back projection on reliable disparity values. The generated point cloud has removed most of the noise from suspended particles, which greatly reduces the burden of subsequent point cloud post-processing, improves the overall real-time performance of the algorithm, and is suitable for rapid 3D reconstruction of large-scale underwater video sequences.

[0078] Furthermore, S4 includes the following sub-steps:

[0079] S41. Perform statistical filtering on the 3D point cloud to remove residual isolated outliers from the point cloud;

[0080] S42. The point cloud space is divided into a uniform voxel grid using a voxel rasterization method. The number of frames in which each voxel appears in multiple consecutive point cloud frames is counted. The voxels are then filtered based on their temporal frequency of occurrence to obtain a pure three-dimensional point cloud of underwater concrete.

[0081] Furthermore: In S42, the 3D point cloud space is divided into a uniform voxel mesh, with each voxel having a size of [missing information]. Midpoint of 3D point cloud Corresponding voxel index The specific expression is:

[0082]

[0083] In the formula, For floor operations, The minimum coordinate value in the point cloud space. The minimum coordinate value on the horizontal axis. The minimum coordinate value on the vertical axis. The minimum coordinate value on the vertical axis. For point x-axis coordinates For point The vertical axis coordinate, For point The vertical axis coordinates, As the first sub-index, For the second sub-index, For the third sub-index, The x-axis coordinate of the voxel. The vertical axis coordinate of the voxel. The vertical axis coordinate value of the voxel;

[0084] The method for screening voxels based on their temporal frequency is as follows:

[0085] Determine whether the number of frames in which a voxel appears in a point cloud over multiple consecutive frames is greater than or equal to the temporal occurrence frequency threshold. If yes, retain the voxel; otherwise, delete the voxel.

[0086] The beneficial effects of the above-mentioned further solutions are as follows: Based on the prior knowledge that "concrete structures are static regions with continuity in temporal point clouds, and the corresponding voxels appear in multiple frames; dynamic suspended particles are isolated regions, and the corresponding voxels appear only in a few frames," this invention discretizes the point cloud space through voxel rasterization, counts the temporal occurrence frequency of each voxel, and achieves the removal of residual dynamic noise; voxel rasterization can also achieve downsampling of the point cloud, reduce data redundancy, and improve the smoothness of the point cloud and the efficiency of subsequent applications.

[0087] The beneficial effects of this invention are as follows:

[0088] (1) This invention utilizes dense optical flow and pixel motion variance analysis to identify and remove noisy pixels before stereo matching. Subsequent 3D point cloud reconstruction steps are performed only on the selected "reliable pixels," thus avoiding invalid calculations of suspended particles from the source. Compared with traditional processes, this method significantly reduces the system's computational load, greatly improves processing efficiency, is suitable for the rapid reconstruction of massive underwater video data, solves the problem of resource waste caused by invalid calculations, and significantly improves the real-time performance of the algorithm.

[0089] (2) This invention proposes a mask-guided stereo matching technique. By dynamically adjusting the matching weight and disparity search range through a noise mask, the algorithm can automatically reduce the impact of noise regions on global path aggregation. This mechanism effectively blocks the interference of suspended objects on normal texture matching, reduces holes and mismatched regions in the disparity map, improves the robustness and accuracy of stereo matching, solves the problem of mismatch caused by noise interference in stereo matching, and optimizes the quality of the disparity map.

[0090] (3) This invention constructs a multi-scale image pyramid, simultaneously analyzing pixel motion characteristics at multiple levels from the original image to the low-resolution image. This strategy ensures that the algorithm can capture tiny particles at high-resolution levels and lock onto large-scale suspended matter at low-resolution levels, achieving comprehensive detection and removal of underwater noise of different sizes, improving the algorithm's adaptability to complex water quality environments, solving the problem that a single scale is insufficient to handle multiple types of suspended matter, and achieving noise suppression across all scales.

[0091] (4) This invention combines the Canny edge detection method with the edge filtering optimization method. When generating the noise mask, pixels with strong texture edges are forcibly retained; when optimizing parallax, bilateral filtering is used to consider both spatial and color differences. This combination ensures that while filtering out noise, the edge information of the concrete structure and cracks remains clear and sharp, and is not blurred due to denoising. It solves the problem that two-dimensional denoising may damage the structural edges, and balances denoising and edge protection.

[0092] In summary, this invention achieves the core objective of "temporal denoising + detail preservation," and its advantages include low computational cost, good real-time performance, and suitability for rapid processing of large-scale underwater video sequences. Attached Figure Description

[0093] Figure 1 This is a flowchart of the binocular stereo vision method for three-dimensional reconstruction of underwater concrete based on two-dimensional image domain according to the present invention. Detailed Implementation

[0094] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0095] like Figure 1 As shown, in one embodiment of the present invention, a binocular stereo vision method for three-dimensional reconstruction of underwater concrete based on a two-dimensional image domain includes the following steps:

[0096] S1. Acquire underwater video sequences, extract key frames, and construct a two-dimensional temporal motion model;

[0097] S2. Based on a two-dimensional temporal motion model, a two-dimensional motion-texture joint denoising method is used to generate an optimized noise mask image;

[0098] S3. Based on the optimized noise mask image, a disparity map is established using a mask-guided stereo matching method, and a 3D point cloud is constructed based on the disparity map.

[0099] S4. Perform statistical filtering and voxel rasterization temporal frequency screening on the 3D point cloud to generate a pure underwater concrete 3D point cloud.

[0100] S1 includes the following steps:

[0101] S11. Obtain underwater video sequences, and extract the most representative key frame sequences by analyzing the number and distribution uniformity of SIFT feature point matching between adjacent frames. Remove redundant frames (such as repeated frames with minimal changes between adjacent frames) to reduce the amount of data for subsequent processing.

[0102] S12. Based on the keyframe sequence, a multi-scale image is constructed for each keyframe using a Gaussian pyramid.

[0103] S13. Use dense optical flow methods (such as Farneback optical flow, which is suitable for underwater images) to calculate the optical flow vectors of pixels between adjacent keyframes;

[0104] S14. Calculate the pixel motion variance based on the optical flow vector, obtain the motion trajectory of each pixel, and construct a two-dimensional temporal motion model.

[0105] In S11, based on the assumption that "the more SIFT feature matches between adjacent frames, the more uniform the distribution, the smaller the inter-frame variation, and the higher the redundancy", a threshold for the number of matches and a threshold for the uniformity of distribution are set to eliminate redundant frames and retain key frames with rich information. This significantly reduces the amount of data and computational cost of subsequent processing while ensuring reconstruction accuracy.

[0106] In S12, the multi-scale image consists of several stacked image layers. Each image layer is obtained by Gaussian smoothing and downsampling the previous image layer. The specific expression for the pixel grayscale value of the multi-scale image is as follows:

[0107]

[0108] In the formula, The pixel grayscale value of the image at layer 0. Let be the coordinates of the pixel in the horizontal direction of the image. These are the coordinates of the pixel in the vertical direction of the image. For the first The pixel grayscale values ​​of the layer image. The kernel function is Gaussian, typically a 5×5 Gaussian kernel. For convolution operations, For convolution operation on the th The layer image is downsampled by 2 times (preserving even-numbered rows and columns of pixels). Keyframe images are input for the Gaussian pyramid;

[0109]

[0110] In the formula, The standard deviation is Gaussian smoothed and is usually taken as 1.0;

[0111] Suspended particles of different sizes behave differently on images of different resolutions. Large-scale particles can still be detected on low-resolution images, while small-scale particles are clearer on high-resolution images. To accommodate underwater suspended particles of different sizes (from micrometers to millimeters), this embodiment constructs a multi-scale image pyramid for each keyframe and analyzes the motion characteristics of pixels at multiple resolution levels (e.g., 4 levels, from the original image to 1 / 8 of the original image) to achieve full coverage detection of particles of different sizes, improve the integrity of denoising, and avoid missing small-scale particles or misjudging large-scale structural points.

[0112] In S13, the dense optical flow method is used to calculate the optical flow vectors of pixels between adjacent keyframes. The specific method is as follows:

[0113] like Then solve for the optical flow vector. ,in, for Moment image in grayscale value at that location for Moment image in grayscale value at that location The horizontal coordinate is... The vertical coordinate;

[0114]

[0115]

[0116] In the formula, for The coefficient matrix of the time-matter polynomial expansion (solved by local neighborhood least squares fitting). for The coefficient matrix of the time-space polynomial expansion. It is the transpose symbol;

[0117] In this embodiment, a dense optical flow algorithm is used to calculate the optical flow vector between adjacent keyframes. The optical flow vector field is optimized by global smoothing constraints to obtain the motion trajectory information of each pixel and construct a two-dimensional temporal motion model to provide a motion basis for subsequent noise detection.

[0118] This method differs fundamentally from traditional optical flow methods based on pixel difference or sparse feature matching. Traditional methods are susceptible to underwater scattering noise, leading to matching failures. In contrast, this method uses quadratic polynomial modeling of the neighborhood and leverages the coefficient changes of surface fitting to solve for displacement. This results in strong noise resistance and sub-pixel accuracy, generating a dense optical flow field across the entire image. This provides a reliable kinematic basis for subsequent accurate identification and removal of suspended particle noise. Solving for the optical flow vector... The specific process is as follows:

[0119] 1. Polynomial modeling of local images

[0120] First, assume that the local brightness distribution of the image at time t can be approximated by a quadratic polynomial:

[0121]

[0122] 2. Polynomial modeling at time t+1

[0123] Similarly, at time t+1, since the pixel has shifted (u, v), the brightness distribution at that point can be represented as:

[0124]

[0125] 3. Solution based on the assumption of constant brightness

[0126] Based on the assumption of constant brightness, the grayscale value of the same object point remains essentially unchanged at adjacent times.

[0127]

[0128] By simultaneously solving the two equations in process 1 and 2, the displacement (u, v) can be linked to the changes in the coefficient matrices A(t) and A(t+1).

[0129] 4. Construction and optimization of linear equation systems

[0130] By performing algebraic transformations on the coefficient matrix, a system of linear equations about (u, v) can be derived. In applications, to improve robustness, the algorithm incorporates a spatial smoothing constraint to optimize the optical flow vector field, obtaining the motion trajectory information of each pixel and constructing a two-dimensional temporal motion model, providing a motion basis for subsequent noise detection.

[0131] In S14, for continuous Frame optical flow vector Calculate pixel motion variance The specific expression is:

[0132]

[0133] In the formula, for Average displacement along the axial direction for Average displacement along the axial direction For the first The horizontal coordinate of the frame optical flow vector. For the first The vertical coordinate of the frame optical flow vector. A larger value indicates that the pixel motion is more unstable and is more likely to be suspended particle noise. The smaller the value, the more stable the motion, and the more likely it is to be a pixel corresponding to a concrete structure.

[0134] In this embodiment, based on the "constant brightness assumption" (the gray value of the same pixel remains unchanged at different times) and the "spatial smoothness assumption" (the motion of adjacent pixels is consistent), the local image brightness distribution is fitted by polynomial expansion to solve for the motion displacement of pixels and obtain a dense optical flow vector field, thereby characterizing the temporal motion characteristics of each pixel. The dense optical flow algorithm does not require feature point detection and can obtain the motion information of all pixels, making it suitable for subsequent global denoising processing.

[0135] S2 includes the following steps:

[0136] S21. Set a motion variance threshold. If the motion variance of a pixel is greater than the motion variance threshold, then mark the pixel as a suspicious noise pixel and generate a noise mask image.

[0137] S22. For regions with a mask value of 0 in the noise mask image, perform adaptive histogram equalization (CLAHE) to improve the texture clarity of the concrete target region and provide better image quality for subsequent stereo matching.

[0138] S23. Perform morphological opening operation on the noise mask image to remove small isolated noise points caused by image noise and optical flow errors, optimize the connectivity and integrity of the mask, avoid mislabeling, and generate a noise mask image optimized by the opening operation.

[0139] S24. Identify strong texture edge regions using Canny edge detection, and reset the mask value corresponding to the identified region to 0 (reliable pixels) to prevent key disease information from being mistakenly removed during the preprocessing stage, thereby achieving a "balance between noise reduction and detail preservation" and generating an optimized noise mask image.

[0140] In S21, the noise mask image is used to guide stereo matching and disparity calculation, generating the noise mask image. The specific expression is:

[0141]

[0142] In the formula, The motion variance threshold (which can be automatically solved using the Otsu binarization algorithm, based on the grayscale distribution of the mask image, maximizing the inter-class variance between the foreground and background) is used in noisy mask images. In the mask, a value of 1 indicates suspicious noise, and 0 indicates a reliable pixel.

[0143] In S22, traditional histogram equalization can over-enhance noise. This embodiment divides the image into multiple small regions (tiles) through adaptive histogram equalization, performs histogram equalization on each region, and suppresses excessive noise enhancement by limiting contrast (clip limit). It is suitable for texture enhancement of low-contrast underwater images and can effectively enhance the contrast of detailed areas such as concrete cracks, providing clearer image features for subsequent stereo matching.

[0144] In S23, morphological opening operations are used to remove small-scale isolated noise while preserving large-scale connected regions, generating an opening-optimized noise mask image. The specific expression is:

[0145]

[0146] In the formula, The image is the noise mask before optimization. It is a structural element (usually a 3×3 or 5×5 rectangular structural element). For the erosion operation (the structuring element slides across the image, and the center pixel is kept as 1 only when the structuring element completely matches the foreground region in the image, otherwise it is set to 0). For the dilation operation (the structuring element slides on the image; if the structuring element intersects with the foreground region in the image, the center pixel is set to 1; otherwise, it is kept as 0).

[0147] In the above formula, the erosion operation is used to shrink the foreground region and remove small-scale isolated noise; the dilation operation is used to restore the shape of the large-scale foreground region and compensate for the region shrinkage caused by the erosion operation. The combination of the two achieves the effect of "removing the small and keeping the large", optimizes the quality of the noise mask, and reduces the misjudgment of subsequent stereo matching.

[0148] In S24, Canny edge detection is a multi-stage edge detection algorithm with high detection rate, low false detection rate, and high localization accuracy. The specific method for identifying strongly textured edge regions using Canny edge detection is as follows:

[0149] S241. Based on the keyframe images in the keyframe sequence, Gaussian smoothing is applied to the image. The image after Gaussian smoothing is... The specific expression is:

[0150]

[0151] S242. Calculate the gradient magnitude. and edge direction This yields the gradient image;

[0152]

[0153]

[0154] In the formula, for Sobel gradient in the axial direction, for Sobel gradient along the axial direction;

[0155] S243. Perform non-maximum suppression on the gradient image to generate a non-maximum suppressed gradient image (sharpening edges and removing blurred edge areas). The non-maximum suppressed gradient image... The specific expression is:

[0156]

[0157] S244. Set high and low thresholds, perform double-threshold hysteresis segmentation on the gradient image after non-maximum suppression, and generate a binary edge image to mark the strong texture edge regions that need to be forcibly preserved. The binary edge image... The specific expression is:

[0158]

[0159] In the formula, For high threshold, The threshold is low.

[0160] Cracks and edges in concrete are key targets for 3D reconstruction. These areas appear as strong edges in 2D images. Based on this, this embodiment uses Canny edge detection to accurately identify these areas and reset their corresponding mask values ​​to 0 to prevent them from being mistakenly labeled as noise and removed. Dual threshold hysteresis segmentation ensures the continuity of edges and avoids the breakage of strong edges, achieving the goal of "denoising without losing key details".

[0161] S3 includes the following steps:

[0162] S31. Based on the optimized noise mask image, the matching cost is corrected by the mask-guided improved SGBM stereo matching method to generate a disparity map.

[0163] S32. Perform bilateral filtering optimization on the disparity map, and use spatial proximity and color similarity to perform weighted smoothing on the disparity map. While smoothing the noise in the disparity map, the clear edge information of the concrete structure is preserved, thereby improving the quality of the disparity map and generating an optimized disparity map.

[0164] S33. For the optimized disparity map, only the reliable disparity values ​​are subjected to 3D back projection to generate a pre-denoised 3D point cloud, which greatly reduces the burden and computational cost of subsequent point cloud post-processing.

[0165] In S31, the method for correcting the matching cost using the mask-guided improved SGBM stereo matching method is as follows:

[0166] In the cost calculation stage of SGBM stereo matching, the matching cost is corrected using a mask-guided weighted matching cost method. The weighted matching cost is then used for path aggregation and disparity selection. The specific expression is:

[0167]

[0168] In the formula, Pixels calculated for SGBM stereo matching In parallax The matching cost below, Matching weights guided by the mask;

[0169]

[0170] In the formula, For the preset small weight, The preset normal weights, For optimized noise masking images;

[0171] In this embodiment, the principle of weighting the matching cost using the optimized noise mask image is as follows:

[0172] The noise mask marks the pixels corresponding to underwater suspended particles. These pixels have low matching reliability and are prone to mismatches. By reducing their matching weight, their contribution to global cost aggregation is reduced, while the disparity search range is narrowed, reducing invalid calculations and improving the efficiency and accuracy of stereo matching. Reliable regions maintain normal matching parameters to ensure the matching quality of concrete structure regions, achieving a balance between efficiency and accuracy. Based on this, this invention reduces the matching weight (e.g., multiply by 0.2) and narrows the disparity search range (e.g., from the normal range [0,64] to [0,16]) for noise regions with a mask value of 1, reducing invalid calculations; for reliable regions with a mask value of 0, normal matching weight and disparity search range are used to ensure matching accuracy.

[0173] In S32, the disparity map is optimized using bilateral filtering. Bilateral filtering considers both spatial proximity and color similarity, which can smooth noise while preserving edges. The disparity values ​​after bilateral filtering are... The specific expression is:

[0174]

[0175] In the formula, These are the disparity values ​​of the disparity map. For filtering neighborhood windows (such as 5×5, 7×7). This represents the spatial Gaussian standard deviation (which controls the range of influence of spatial proximity; the larger the value, the wider the spatial range). The Gaussian standard deviation of grayscale (color) (controls the range of influence of color similarity; the larger the value, the higher the tolerance for color differences). For normalized weighted sum;

[0176]

[0177] In the formula, The grayscale value of the neighboring pixels within the bilateral filtering neighborhood window. This represents the grayscale value of the center pixel within the bilateral filtering neighborhood window.

[0178] In this embodiment, the principle of bilateral filtering in this invention is as follows: Unlike traditional Gaussian filtering, which only considers spatial proximity, bilateral filtering additionally considers color similarity. It assigns high weights to spatially adjacent and color-similar pixels, and low weights to spatially adjacent but color-different pixels (such as edge pixels). This smooths out noise in the disparity map while preserving edge information, avoiding edge blurring in the disparity map, and improving the accuracy of subsequent 3D reconstruction.

[0179] S33 specifically refers to: determining pixels Corresponding disparity value If the value is 0, then use its disparity value for 3D back projection to generate a valid point cloud; otherwise, discard the pixel directly.

[0180] In this embodiment, the present invention uses mask-constrained 3D point cloud generation to remove the disparity values ​​corresponding to noisy pixels from the source and only performs 3D back projection on reliable disparity values. The generated point cloud has removed most of the noise from suspended particles, which greatly reduces the burden of subsequent point cloud post-processing, improves the overall real-time performance of the algorithm, and is suitable for rapid 3D reconstruction of large-scale underwater video sequences.

[0181] S4 includes the following steps:

[0182] S41. Perform statistical filtering on the 3D point cloud to remove residual isolated outliers (such as tiny particles missed by 2D denoising).

[0183] In this embodiment, during the statistical filtering process of the 3D point cloud, the present invention uses a more relaxed confidence threshold to perform statistical filtering on the 3D point cloud, only removing obvious residual isolated points, thus avoiding the loss of concrete structural details due to over-filtering.

[0184] S42. The point cloud space is divided into a uniform voxel grid using a voxel rasterization method. The number of frames in which each voxel appears in multiple consecutive point cloud frames is counted. The voxels are then filtered based on their temporal frequency of occurrence to obtain a pure three-dimensional point cloud of underwater concrete.

[0185] Since denoising in the two-dimensional image domain has already removed most of the noise, this embodiment only requires lightweight post-processing of the three-dimensional point cloud to further remove residual noise and obtain a clean concrete three-dimensional point cloud.

[0186] In S42, the 3D point cloud space is divided into a uniform voxel mesh, and the size of each voxel is... (e.g., 0.5mm×0.5mm×0.5mm, set according to reconstruction accuracy requirements), midpoint of 3D point cloud Corresponding voxel index The specific expression is:

[0187]

[0188] In the formula, For floor operations, This represents the minimum coordinates in the point cloud space (the coordinates of the lower left corner of the bounding box). The minimum coordinate value on the horizontal axis. The minimum coordinate value on the vertical axis. The minimum coordinate value on the vertical axis. For point x-axis coordinates For point The vertical axis coordinate, For point The vertical axis coordinates, As the first sub-index, For the second sub-index, For the third sub-index, The x-axis coordinate of the voxel. The vertical axis coordinate of the voxel. The vertical axis coordinate value of the voxel;

[0189] In this embodiment, the principle of voxel rasterization temporal frequency filtering is as follows: based on the prior knowledge that "concrete structures are static regions with continuity in temporal point clouds, and the corresponding voxels appear in multiple frames; dynamic suspended particles are isolated regions, and the corresponding voxels appear only in a few frames", the point cloud space is discretized by voxel rasterization, and the temporal occurrence frequency of each voxel is statistically analyzed to remove residual dynamic noise; voxel rasterization can also achieve downsampling of the point cloud, reduce data redundancy, and improve the smoothness of the point cloud and the efficiency of subsequent applications.

[0190] The method for screening voxels based on their temporal frequency is as follows:

[0191] Determine whether the number of frames in which a voxel appears in a point cloud over multiple consecutive frames is greater than or equal to the temporal occurrence frequency threshold. If yes, retain the voxel; otherwise, delete the voxel.

[0192] In this embodiment, voxels appear frequently in the concrete structure region and infrequently in the dynamic particle region. Residual noise is eliminated by using a temporal occurrence frequency threshold, where the temporal occurrence frequency threshold is typically set to... , The total number of keyframes means that voxels must appear in at least 1 / 5 of the frames to be considered a static concrete structure area.

[0193] In the description of this invention, the above are merely preferred embodiments and are not intended to limit the scope of protection of this invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for three-dimensional reconstruction of underwater concrete based on binocular stereo vision in a two-dimensional image domain, characterized in that, Includes the following steps: S1. Acquire underwater video sequences, extract key frames, and construct a two-dimensional temporal motion model; S2. Based on a two-dimensional temporal motion model, a two-dimensional motion-texture joint denoising method is used to generate an optimized noise mask image; S3. Based on the optimized noise mask image, a disparity map is established using a mask-guided stereo matching method, and a 3D point cloud is constructed based on the disparity map. S4. Perform statistical filtering and voxel rasterization temporal frequency screening on the 3D point cloud to generate a pure underwater concrete 3D point cloud. S3 includes the following steps: S31. Based on the optimized noise mask image, the matching cost is corrected by the mask-guided improved SGBM stereo matching method to generate a disparity map. S32. Perform bilateral filtering optimization on the disparity map, and use spatial proximity and color similarity to perform weighted smoothing on the disparity map to generate an optimized disparity map. S33. For the optimized disparity map, perform 3D backprojection only on reliable disparity values ​​to generate a pre-denoised 3D point cloud. In S31, the method for correcting the matching cost using the mask-guided improved SGBM stereo matching method is as follows: In the cost calculation stage of SGBM stereo matching, the matching cost is corrected using a mask-guided weighted matching cost method. The weighted matching cost is then used for path aggregation and disparity selection. The specific expression is: In the formula, Pixels calculated for SGBM stereo matching In parallax The matching cost below, Matching weights guided by the mask; In the formula, For the preset small weight, The preset normal weights, For optimized noise masking images; In S32, bilateral filtering optimization is performed on the disparity map, and the disparity values ​​after bilateral filtering are... The specific expression is: In the formula, These are the disparity values ​​of the disparity map. For filtering the neighborhood window, The standard deviation of Gaussians in space. The standard deviation of grayscale Gaussian is given. The grayscale value of the neighboring pixels within the bilateral filtering neighborhood window. The gray value of the center pixel within the bilateral filtering neighborhood window. For normalized weighted sum; S33 specifically refers to: determining pixels Corresponding disparity value If the value is 0, then use its disparity value for 3D back projection to generate a valid point cloud; otherwise, discard the pixel directly.

2. The method for three-dimensional reconstruction of underwater concrete based on binocular stereo vision using a two-dimensional image domain according to claim 1, characterized in that, S1 includes the following steps: S11. Obtain underwater video sequences, and extract the most representative keyframe sequences by analyzing the number and distribution uniformity of SIFT feature point matching between adjacent frames. S12. Based on the keyframe sequence, a multi-scale image is constructed for each keyframe using a Gaussian pyramid. S13. Calculate the optical flow vectors of pixels between adjacent keyframes using the dense optical flow method; S14. Calculate the pixel motion variance based on the optical flow vector, obtain the motion trajectory of each pixel, and construct a two-dimensional temporal motion model.

3. The method for three-dimensional reconstruction of underwater concrete based on binocular stereo vision using a two-dimensional image domain according to claim 2, characterized in that, In S12, the multi-scale image consists of several stacked image layers. Each image layer is obtained by Gaussian smoothing and downsampling the previous image layer. The specific expression for the pixel grayscale value of the multi-scale image is as follows: In the formula, The pixel grayscale value of the image at layer 0. Let be the coordinates of the pixel in the horizontal direction of the image. These are the coordinates of the pixel in the vertical direction of the image. For the first The pixel grayscale values ​​of the layer image. For Gaussian kernel function, For convolution operations, For convolution operation on the th The layer image is downsampled by a factor of 2. Keyframe images are input for the Gaussian pyramid; In S13, the dense optical flow method is used to calculate the optical flow vectors of pixels between adjacent keyframes. The specific method is as follows: like Then solve for the optical flow vector. ,in, for Moment image in grayscale value at that location for Moment image in grayscale value at that location The horizontal coordinate is... The vertical coordinate; In the formula, for The coefficient matrix of the time-space polynomial expansion. for The coefficient matrix of the time-space polynomial expansion. It is the transpose symbol; In S14, for continuous Frame optical flow vector Calculate pixel motion variance The specific expression is: In the formula, for Average displacement along the axial direction for Average displacement along the axial direction For the first The horizontal coordinate of the frame optical flow vector. For the first The vertical coordinate of the frame optical flow vector.

4. The method for three-dimensional reconstruction of underwater concrete based on binocular stereo vision using a two-dimensional image domain according to claim 3, characterized in that, S2 includes the following steps: S21. Set a motion variance threshold. If the motion variance of a pixel is greater than the motion variance threshold, then mark the pixel as a suspicious noise pixel and generate a noise mask image. S22. For regions with a mask value of 0 in the noise mask image, perform adaptive histogram equalization to improve the texture clarity of the concrete target region. S23. Perform morphological opening operation on the noise mask image to generate an opening-optimized noise mask image; S24. Identify strong texture edge regions using Canny edge detection, reset the mask value corresponding to the identified region to 0, and generate an optimized noise mask image.

5. The method for three-dimensional reconstruction of underwater concrete based on binocular stereo vision using a two-dimensional image domain according to claim 4, characterized in that, In S21, a noise mask image is generated. The specific expression is: In the formula, This is the motion variance threshold; In S23, morphological opening operations are used to remove small-scale isolated noise while preserving large-scale connected regions, generating an opening-optimized noise mask image. The specific expression is: In the formula, The image is the noise mask before optimization. As a structural element, For erosion calculation, For expansion operation; In S24, the method for identifying strong texture edge regions using Canny edge detection is as follows: S241. Based on the keyframe images in the keyframe sequence, Gaussian smoothing is applied to the image. The image after Gaussian smoothing is... The specific expression is: S242. Calculate the gradient magnitude. and edge direction This yields the gradient image; In the formula, for Sobel gradient in the axial direction, for Sobel gradient along the axial direction; S243. Perform non-maximum suppression on the gradient image to generate a non-maximum suppressed gradient image. The specific expression is: S244. Set high and low thresholds, perform double-threshold hysteresis segmentation on the gradient image after non-maximum suppression, and generate a binary edge image to mark the strong texture edge regions that need to be forcibly preserved. The binary edge image... The specific expression is: In the formula, For high threshold, The threshold is low.

6. The method for three-dimensional reconstruction of underwater concrete based on binocular stereo vision using a two-dimensional image domain according to claim 5, characterized in that, S4 includes the following steps: S41. Perform statistical filtering on the 3D point cloud to remove residual isolated outliers from the point cloud; S42. The point cloud space is divided into a uniform voxel grid using a voxel rasterization method. The number of frames in which each voxel appears in multiple consecutive point cloud frames is counted. The voxels are then filtered based on their temporal frequency of occurrence to obtain a pure three-dimensional point cloud of underwater concrete.

7. The method for three-dimensional reconstruction of underwater concrete based on binocular stereo vision using a two-dimensional image domain according to claim 6, characterized in that, In S42, the 3D point cloud space is divided into a uniform voxel mesh, and the size of each voxel is... Midpoint of 3D point cloud Corresponding voxel index The specific expression is: In the formula, For floor operations, The minimum coordinate value in the point cloud space. The minimum coordinate value on the horizontal axis. The minimum coordinate value on the vertical axis. The minimum coordinate value on the vertical axis. For point x-axis coordinates For point The vertical axis coordinate, For point The vertical axis coordinates, As the first sub-index, For the second sub-index, For the third sub-index, The x-axis coordinate value of the voxel. The vertical axis coordinate of the voxel. The vertical axis coordinate value of the voxel; The method for screening voxels based on their temporal frequency is as follows: Determine whether the number of frames in which a voxel appears in a consecutive multi-frame point cloud is greater than or equal to the temporal occurrence frequency threshold. If yes, retain the voxel; otherwise, delete the voxel.

Images