A regional constraint binocular point cloud robust reconstruction method for complex water surface

Abstract

The application discloses a regional constraint binocular point cloud robust reconstruction method for complex water surface, which comprises the following steps: the processing range is limited by water surface area mask constraint; the quality of feature points in the region is screened; cross-eye neighborhood matching is established; multi-level false matching is removed through ratio test, RANSAC geometric consistency screening, matching distance statistical filtering and regional coverage constraint; the effective matching points are triangulated to generate candidate three-dimensional point cloud; stable water surface point cloud and plane parameters are obtained through limited value judgment, spatial outlier cleaning and closed loop plane fitting of "rough cleaning-fitting-precise cleaning-re-fitting"; the comprehensive reliability is calculated based on matching number, inlier proportion, plane residual and regional coverage; if the standard is met, the water surface point cloud, plane parameters and quality indicators are output; otherwise, a low reliability mark is output. The regional constraint is applied to the whole process of feature screening, matching, point cloud generation and output, which effectively reduces background false matching pollution and reduces the false matching probability under complex water surface conditions.

Description

Technical Field

[0001] This invention belongs to the field of machine vision technology, specifically relating to a robust binocular point cloud reconstruction method for region constraints on complex water surfaces. Background Technology

[0002] In water conservancy monitoring, experimental canal measurement, urban flooding analysis, and field river observation, 3D point clouds of water surfaces and water surface plane parameters can provide basic data for water level estimation, water surface attitude analysis, on-site verification, and subsequent statistical processing. Binocular vision has the advantages of being non-contact, low-cost, and capable of simultaneously acquiring image and spatial information, making it suitable for 3D reconstruction of water surface areas under certain observation conditions.

[0003] General methods such as stereo matching, local feature extraction, nearest neighbor matching, and geometric model estimation can be used for image matching, corresponding point construction, and 3D point cloud reconstruction. A typical processing flow is as follows: feature points are extracted from the entire image and descriptors are established; initial matching point pairs are obtained through nearest neighbor search; mismatches are eliminated using geometric consistency constraints (such as RANSAC fundamental matrix filtering); and finally, a 3D point cloud is generated through triangulation.

[0004] However, complex water surfaces differ from ordinary rigid objects or scenes with rich textures. They often exhibit characteristics such as weak textures, localized specular reflections, overexposure, undulating waves, reflections, floating objects, shoreline occlusion, and the coexistence of strong background textures. Directly applying the aforementioned general binocular matching method to process complex water surface scenes presents the following problems:

[0005] First, there is a lack of prior constraints for complex water surfaces, and strong texture points on non-water surfaces can easily contaminate matching and point clouds. General methods extract and match features from the entire image, and non-water surface areas such as shorelines, vegetation, bridge piers, distant buildings, and water reflections are prone to generating strong feature points, which are then incorrectly treated as water surface points in 3D reconstruction, causing the reconstruction results to deviate from the true water surface.

[0006] Second, the mismatch removal process is incomplete, making it difficult to simultaneously constrain regional origin, geometric consistency, matching distance statistics, and spatial distribution. Common methods typically only use single-level geometric consistency constraints to remove mismatches, leaving many mismatches in 3D reconstruction under complex water surface conditions.

[0007] Third, the point cloud cleaning and water surface plane fitting lack a closed loop, and 3D outliers may cause distortion in the plane and point cloud output. Common methods often only perform matching and screening at the 2D level, lacking spatial outlier detection and plane residual cleaning for 3D point clouds; when 3D outliers caused by reflections, waves, floating objects or mismatches participate in plane fitting, it will lead to distortion of plane parameters, and the distorted plane cannot, in turn, guide the removal of outliers.

[0008] Fourth, there is a lack of reliability assessment for water surface point clouds. Existing processing workflows often directly output the reconstruction results without explaining the reliability of the current water surface point clouds. This can easily lead to the output of misleading water surface point clouds under low-quality input.

[0009] Therefore, existing general-purpose binocular stereo matching methods have shortcomings in complex water surface scenarios, such as large regional interference, insufficient effective matching points, inadequate removal of mismatches, lack of closed loop between point cloud cleaning and plane fitting, and difficulty in judging the reliability of results, which urgently need to be improved. Summary of the Invention

[0010] The purpose of this invention is to overcome the shortcomings of binocular point cloud reconstruction on complex water surfaces, such as large regional interference, insufficient removal of mismatches, lack of closed loop in point cloud cleaning and plane fitting, and difficulty in judging the reliability of results, and to provide a robust binocular point cloud reconstruction method with regional constraints for complex water surfaces.

[0011] The technical solution of the present invention is as follows:

[0012] A robust binocular point cloud reconstruction method for region constraints on complex water surfaces includes the following steps:

[0013] Step 1: Receive left and right binocular images, and determine the effective water surface area mask in the left and right binocular images. The effective water surface area mask is used to limit subsequent processing to the water surface area.

[0014] Step 2: Extract local feature points within the area defined by the effective water surface area mask, and filter the local feature points according to the effective water surface area mask, boundary safety distance, feature response intensity, and spatial coverage to obtain candidate feature points;

[0015] Step 3: Perform cross-nearest neighbor matching on the descriptors of the candidate feature points to obtain a candidate nearest neighbor set, and remove fuzzy matches with insufficient distinguishability between the first and second nearest neighbors by ratio test to obtain candidate matching point pairs;

[0016] Step 4: Perform RANSAC in-point filtering based on geometric consistency model, anomaly filtering based on matching distance statistics, and water surface area coverage constraint on the candidate matching point pairs in sequence to obtain valid matching point pairs;

[0017] Step 5: Based on the effective matching point pairs, triangulate to generate candidate 3D point clouds. Perform finite value judgment, effective depth judgment, spatial range judgment, and spatial outlier cleaning on the candidate 3D point clouds in sequence to obtain cleaned candidate water surface point clouds.

[0018] Step 6: Perform RANSAC robust plane fitting on the cleaned candidate water surface point cloud to obtain an initial plane, calculate the residuals of each 3D point to the initial plane, update the inlier set based on the residual statistical threshold, and refit the plane to obtain the final water surface point cloud and water surface plane parameters.

[0019] Step 7: Calculate the overall credibility based on the number of valid matching points, the number of 3D points, the proportion of points in the plane, the root mean square of the plane residual, and the area coverage. If the overall credibility and each sub-indicator meet the preset conditions, output the final water surface point cloud, the inner / outer point markers, the water surface plane parameters, and the quality indicators; otherwise, output a low credibility marker or a reprocessing prompt.

[0020] Furthermore, the determination of the effective water surface area mask in step one includes: fusing the artificial ROI coarse constraint mask with the water surface segmentation mask to obtain the initial mask of the effective water surface area, and performing a safety shrinkage process on the boundary of the initial mask of the effective water surface area to exclude unstable pixels near the shoreline junction, the edge of floating objects, and the specular highlight boundary.

[0021] Furthermore, the local feature point extraction algorithm in step two includes SURF, ORB, SIFT, or AKAZE.

[0022] Furthermore, the filtering in step two includes: retaining only feature points located inside the effective area mask of the water surface, which are more than a preset safe distance from the mask boundary, have feature response intensity higher than a preset threshold, and are not excessively concentrated within the effective area mask of the water surface.

[0023] Furthermore, the ratio test described in step three satisfies:

[0024]

[0025] in, To prevent extremely small positive numbers with a denominator of zero; This represents the descriptor distance between the i-th feature point in the left eye and its first nearest neighbor, which is the distance between the i-th feature point and the most similar feature point found in the right eye image. This represents the descriptor distance between the i-th feature point of the left eye and its second nearest neighbor, which is the distance between the second most similar feature points after excluding the best match; This represents the threshold for the ratio test.

[0026] Furthermore, the RANSAC in-point screening based on the geometric consistency model described in step four includes: calculating the geometric residuals of each candidate matching point pair using the fundamental matrix, homography model, or epipolar constraints, and retaining matching point pairs whose geometric residuals are less than or equal to the geometric residual threshold.

[0027] The anomaly filtering based on matching distance statistics in step four includes: calculating the mean and standard deviation of the descriptor distances of matching point pairs after RANSAC filtering, and removing abnormal matches whose descriptor distances are greater than the mean plus a certain number of times the standard deviation;

[0028] The water surface area coverage constraint in step four includes: dividing the effective water surface area mask into several grids, counting the percentage of grids with matching point projections, and determining that the matching point spatial coverage is insufficient if the percentage of grids is lower than a preset threshold.

[0029] Furthermore, the spatial outlier cleaning described in step five includes statistical outlier removal or radius outlier removal.

[0030] Furthermore, the residual statistical threshold mentioned in step six is ​​the mean plus... Multiple standard deviations, of which The value ranges from 2 to 4; the refitted plane includes least squares fitting or weighted least squares fitting.

[0031] Furthermore, the overall credibility mentioned in step seven is calculated using a weighted scoring method:

[0032]

[0033] in, To assess overall credibility, These are the weighting coefficients for the match count score, inlier ratio, residual score, and region coverage, respectively. ; To effectively match the number of point pairs; To minimize the number of valid matches, The threshold for planar residuals; is the regional coverage; RMSE is the root mean square of the planar residuals.

[0034] Furthermore, the preset conditions in step seven include: the overall credibility is greater than or equal to the credibility threshold, the number of valid matching points is greater than or equal to the minimum number of valid matches, the proportion of points in the plane is greater than or equal to the proportion of points in the plane threshold, and the root mean square of the plane residual is less than or equal to the plane residual threshold.

[0035] Compared with the prior art, the present invention has at least the following beneficial effects:

[0036] This invention shifts from whole-image processing to water surface region constraint processing, reducing the contamination of water surface point clouds by background mismatches; it moves from simple feature extraction to regional quality screening, making points entering the matching chain more suitable for water surface reconstruction; it transforms from single-level matching to multi-level mismatch removal, reducing the probability of mismatches entering 3D reconstruction under complex water surface conditions; it extends from 2D screening to 3D point cloud cleaning, capable of identifying 3D outliers caused by reflections, waves, floating objects, or partial occlusion; it shifts from direct output to reliable output, avoiding the output of misleading water surface point clouds under low-quality input; and it outputs water surface point clouds, planar parameters, residual indices, and reliability scores, facilitating result evaluation and quality judgment. Attached Figure Description

[0037] Figure 1 This is a data processing flowchart of a robust binocular point cloud reconstruction method for complex water surfaces according to the present invention.

[0038] Figure 2 This is a flowchart of a robust binocular point cloud reconstruction method for complex water surfaces, based on the present invention. Detailed Implementation

[0039] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. Without departing from the inventive concept, those skilled in the art can make equivalent substitutions for feature algorithms, region constraint methods, geometric consistency models, point cloud cleaning methods, and plane fitting methods.

[0040] like Figure 1 , 2 As shown, the method of this invention executes the following sequence: receiving left and right binocular images—water surface region constraint—feature filtering within the region—binocular matching—multi-level mismatch removal—point cloud reconstruction and cleaning—water surface plane fitting—confidence judgment and output. There is a continuous data transfer relationship between each step. The water surface region constraint result is not only used to limit the feature point extraction range, but also for matching point retention, point cloud filtering, and result confidence judgment, thus ensuring that region constraint is integrated throughout the entire binocular point cloud reconstruction process.

[0041] The water surface region constraint module is used to determine candidate water surface regions in the left and right eye images, excluding non-water surface strong texture regions from subsequent processing.

[0042] Step S1: Receive left and right binocular images. These images can be provided via online input, offline import, or other methods; this invention does not limit the image source method. The left and right binocular images serve as the raw data source for subsequent processing.

[0043] Step S2: Determine candidate water surface areas. Since the entire image contains non-water surface areas such as sky, buildings, vegetation, and bridge piers, the strong texture points in these areas will pollute subsequent matching and point cloud reconstruction. Therefore, it is necessary to first determine candidate water surface areas to limit the processing scope. Depending on the scene conditions, one or a combination of the following methods can be used:

[0044] Method A: Manually select ROI (suitable for scenarios involving operator intervention)

[0045] Method B: Historical Task Area (Applicable to Fixed Monitoring Viewpoint)

[0046] Method C: Shoreline Priority (Applicable to scenarios where shoreline geographic information is known)

[0047] Method D: Water body segmentation mask (suitable for automatic segmentation scenarios)

[0048] Method E: A combination of the above methods

[0049] Step S3: When combining the ROI with the water surface segmentation mask, the initial mask for the effective water surface area is obtained through fusion. For the left-eye image... Right eye image This method can generate corresponding masks separately, or map one side's mask to generate the other side's mask based on the geometric relationship between the left and right images. Using the ROI alone results in coarse boundaries, and using the segmentation mask alone may misclassify distant shore reflections. Therefore, fusing the two methods can achieve both coarse and fine screening. The fused image is the left eye image:

[0050] (1)

[0051] The right eye image is obtained similarly:

[0052] (2)

[0053] in, These are the pixel coordinates (horizontal and vertical coordinates) in the image. This represents the coarse constraint mask of the ROI for the left eye image. The value is 0 or 1, indicating whether the pixel falls within the manually selected (or historical task area) region of interest. 1 indicates that it is within the ROI, and 0 indicates that it is not within the ROI. This represents the coarse constraint mask of the ROI in the right eye image. The value is 0 or 1, indicating whether the pixel falls within the region of interest selected manually (or the historical task area). 1 indicates that it is within the ROI, and 0 indicates that it is not within the ROI. This represents the water surface segmentation mask in the left eye image. The value is 0 or 1, indicating whether the pixel is identified as water by the water segmentation model (or color thresholding, etc.). 1 indicates water surface, and 0 indicates non-water surface. This represents the water surface segmentation mask in the right eye image. The value is 0 or 1, indicating whether the pixel is identified as water by the water segmentation model (or color thresholding, etc.). 1 indicates water surface, and 0 indicates non-water surface. This represents the effective water surface area mask in the left eye image. The value is 0 or 1, indicating whether the pixel ultimately belongs to the effective water surface area. 1 indicates that it belongs, and 0 indicates that it does not. This represents the effective water surface area mask in the right eye image. The value is 0 or 1, indicating whether the pixel ultimately belongs to the effective water surface area. 1 indicates that it belongs, and 0 indicates that it does not.

[0054] Step S4: Perform safe shrinkage on the initial mask boundary. Since the mask boundary obtained in S3 often contains shoreline blending pixels, floating object edges, or specular reflection boundaries, the pixel textures at these locations are unstable and easily misidentified as water surface feature points. Therefore, it is necessary to perform erosion shrinkage of the mask boundary by 3 to 30 pixels, shrinking the boundary inwards towards the water surface, thereby physically eliminating these unstable pixels and obtaining the final effective water surface region mask for subsequent feature extraction.

[0055] After the above S1-S4 steps, strong texture points in non-water surface areas are excluded from subsequent processing, fundamentally cutting off the source of background mismatch and contamination of water surface point clouds.

[0056] The regional feature selection and binocular matching module extracts feature points within the effective area of ​​the water surface and performs quality selection, retaining only candidate feature points suitable for 3D reconstruction of the water surface, providing a high-quality candidate matching set for subsequent multi-level mismatch elimination.

[0057] Step S5: Extract local feature points and descriptors within the masked region output in S4. SURF, ORB, SIFT, AKAZE, or other feature algorithms with local descriptive capabilities can be used.

[0058] Step S6: Based on the effective water surface area, boundary safety distance, feature response intensity, and spatial coverage constraints, filter feature points from the local feature points and descriptors in Step S5. Only retain feature points located within the effective water surface area and at a distance greater than the preset safety distance from the area boundary. Response strength Above the threshold Furthermore, the feature points are not excessively concentrated within the water surface area. The set of feature points within the water surface area is represented as:

[0059] (3)

[0060]

[0061] in, This represents the original set of feature points in the left eye image, i.e., all feature points extracted from the left eye image (without any filtering). This represents the i-th feature point in the left eye image, where each feature point contains its coordinates in the image. Response strength Attributes such as... This represents the minimum distance (pixel distance) from the i-th feature point to the boundary of the water surface mask. This represents the set of feature points within the water surface region of the left eye image after filtering. These feature points simultaneously meet the three conditions of being within the water surface region, having sufficient response intensity, and being sufficiently far from the boundary.

[0062] Step S7: Perform cross-eye nearest neighbor matching on the descriptors in the feature point sets after left and right eye filtering output from S6. For each feature point in the left eye set, search for the nearest and second nearest neighbors (K=2) in the right eye set using its descriptor, record the corresponding right eye feature point index and descriptor distance, and obtain a nearest neighbor candidate set sorted by distance.

[0063] Step S8: Remove fuzzy matches using a ratio test. The ratio test retains matches that satisfy the following formula:

[0064] Fuzzy matches with insufficient discriminative power are removed using a ratio test. For each query point output by S7, its first nearest neighbor distance and second nearest neighbor distance are compared, and only matches satisfying the following formula are retained:

[0065] (4)

[0066] in, To prevent extremely small positive numbers with a denominator of zero; This represents the descriptor distance between the i-th feature point in the left eye and its first nearest neighbor (best matching candidate), which is the distance between the i-th feature point and the most similar feature point found in the right eye image. This represents the descriptor distance between the i-th feature point of the left eye and its second nearest neighbor (the second best matching candidate), which is the distance between the second most similar feature points after excluding the best match. This represents the threshold for the ratio test (recommended 0.6 to 0.8, preferably 0.7).

[0067] After ratio testing, the retained matching point pairs have clear distinguishability at the descriptor level, but there may still be mismatches due to similar descriptors but incorrect geometric positions (such as incorrect correspondences between water reflections and real water surfaces, or repetitive texture structures). Therefore, further verification from the perspective of binocular geometric consistency is needed.

[0068] The multi-level mismatch removal module performs a four-level progressive mismatch removal on candidate matching point pairs, constraining the matching quality from four dimensions: descriptor similarity, geometric consistency, statistical distribution, and spatial coverage, which can solve the problem of incomplete mismatch removal links.

[0069] Step S9: Perform RANSAC interior point screening based on the binocular geometric consistency model. This utilizes the fundamental matrix, homography model, epipolar constraints, or other geometric models. Calculate the geometric residual of the i-th matching point pair:

[0070] (5)

[0071] in, and The first The pixel coordinates of each matching point pair in the left and right images. Represented by geometric model Mapping the left eye point to the corresponding right eye position. Retaining the condition that... The system can identify matching pairs and further eliminate outlier matches whose distance is greater than the mean plus several times the standard deviation. Let represent the geometric residual of the i-th matching point pair, indicating the degree of geometric consistency of the matching point pair. The smaller the value, the more reliable the match.

[0072] This step eliminates mismatches from the perspective of spatial geometric relationships: even if the descriptors of two points are highly similar (S8 has passed), if their relative positions in the left and right eyes violate the geometric rules of binocular imaging, they will still be judged as mismatches and eliminated.

[0073] Step S9's RANSAC screening uses geometric residuals as the criterion, which is robust to geometrically inconsistent matches, but may retain a small number of weak-quality matches with "excessively large descriptor distances that barely meet the geometric threshold." Furthermore, if matching points are excessively concentrated in local water surface regions, even if individual points are of acceptable quality, it is difficult to support reliable global planar fitting. Therefore, further constraints are needed from two dimensions: descriptor distance statistics and spatial distribution.

[0074] Step S10: Perform descriptor distance statistical filtering and region coverage constraints.

[0075] (1) Descriptor distance statistical filtering: Calculate the mean of the descriptor distances in the set of matching point pairs retained by S9. and standard deviation Remove values ​​that are more than a certain distance from the mean plus a certain number of standard deviations. (Or based on quantiles, adaptive thresholds) outlier matches. This step eliminates weak quality matches from the statistical dimension of descriptor quality: even if the geometric residual barely passes S9, if its descriptor distance deviates significantly from the overall distribution, it is still considered an unreliable match and eliminated.

[0076] (2) Area coverage constraint: Divide the effective water surface area into several grids, count the percentage of grids with matching point projections, and determine whether the retained matching points cover enough water surface area to avoid the entire water surface point cloud output being supported only by local small water surface. If it is lower than the preset threshold, it means that the matching points are overly concentrated in local small water surface and are difficult to support the reconstruction of the water surface point cloud of the entire image. At this time, it is marked as insufficient coverage.

[0077] Step S11: Determine if there are enough valid matches. If the number of valid matches is... Below the preset threshold If the area covered by the matching points is too small, the current image is considered insufficient to generate a reliable water surface point cloud, and the process jumps directly to step S18 to output a low-confidence marker or a reprocessing prompt; if the requirements are met, the process proceeds to the point cloud reconstruction step.

[0078] Step S12: For the image pairs determined to be sufficiently matched in S11, triangulation is performed on the retained valid matching point pairs to generate candidate 3D point clouds. Triangulation formula:

[0079] (6)

[0080] in, Let represent the three-dimensional spatial point obtained by triangulation reconstruction of the i-th matching point pair.

[0081] Step S13 involves preliminary filtering of the candidate point cloud generated in step S12. This includes finite value judgment, effective depth or effective distance judgment, and spatial range judgment, removing incalculable points, obvious outliers, and points exceeding the reasonable observation range. The points:

[0082] (7)

[0083] in, For candidate point cloud sets; This is the point cloud collection after preliminary filtering; Let be the depth coordinates of the i-th 3D point, which usually represents the distance along the camera's optical axis, i.e., the depth value; This represents the maximum effective depth / distance of a 3D point. judge Whether it is a finite value or not, triangulation calculation failure may produce infinite or illegal values, which need to be rejected.

[0084] Step S14: Perform spatial outlier cleaning on the point cloud obtained from the preliminary filtering in step S13. Use statistical outlier removal or radius outlier removal methods to remove spatially isolated points or points that are significantly deviated from the main water surface group.

[0085] Step S15: Perform robust water surface plane fitting on the point cloud cleaned in step S14 to obtain the initial water surface plane. First, RANSAC selects the initial set of interior points, and then least squares or weighted least squares is used to perform a second fitting on the interior point set to output the water surface plane parameters. The water surface plane equation is expressed as:

[0086] (8)

[0087] in, A set representing the parameters of the water surface. The three components of the normal vector to the water surface plane are represented. The constant term in the plane equation represents the perpendicular distance from the origin to the plane.

[0088] Step S16: Update the set of interior points based on the planar residuals and refit to obtain the final water surface point cloud and planar parameters.

[0089] The RANSAC in-point threshold of S15 is primarily aimed at ensuring recall, and may include a small number of marginal outliers. Therefore, this step calculates the residuals from all points to the plane based on the initial plane fitted by S15, and uses statistical thresholding. Perform a thorough cleaning.

[0090] Calculate the residuals from each 3D point to the water surface:

[0091] (9)

[0092] With residual threshold (Preferred) Clean the outer points to obtain the cleaned water surface point cloud:

[0093] (10)

[0094] in, This represents the set of water surface point clouds after cleaning, that is, the set of three-dimensional points that are considered to be truly located on the water surface after planar residual cleaning, and is also the final output water surface point cloud.

[0095] The updated set of interior points is used to refit the water surface plane, resulting in the final water surface point cloud and plane parameters. Through the closed loop of "cleaning → fitting → re-cleaning → re-fitting", the vicious cycle of outliers contaminating the plane and distorting the plane without cleaning outliers is avoided.

[0096] The credibility judgment and output distribution module establishes a comprehensive credibility judgment mechanism to avoid forcing unreliable results under low-quality input and solves the problem of lacking credibility judgment for water surface point clouds.

[0097] Step S17: Calculate the credibility index. The credibility index includes at least the number of valid matching points, the number of 3D points, the proportion of points in the plane, the root mean square of the plane residuals, and the region coverage. Wherein:

[0098] Point scale in the plane:

[0099] (11)

[0100] in, This indicates the number of points in the plane, i.e., the number of points ultimately retained after plane residual cleaning. The number of three-dimensional points in the data comes from S16; This represents the number of candidate 3D points, i.e., the point cloud set after initial filtering. The total number of points is derived from S13.

[0101] Root mean square of planar residuals:

[0102] (12)

[0103] Water surface area coverage The coverage can be obtained by dividing the water surface mask into several grids and statistically analyzing the proportion of grids with in-point projections. Preferably, the area coverage is calculated using the following formula:

[0104] (13)

[0105] in, This indicates the number of water surface grids with interior point projections. This represents the total number of grid cells corresponding to the effective water surface area. The overall reliability can be calculated using the following formula:

[0106] (14)

[0107] in, These are the weighting coefficients for the match count score, inlier ratio, residual score, and region coverage, respectively. , To effectively match the number of point pairs, from S11; To minimize the number of valid matches, This is the threshold for planar residuals. The weights corresponding to the planar residuals and interior point ratios can be set higher than the weights for region coverage, or they can be preset according to different water surface scenarios.

[0108] Step S18, confidence level splitting output. If satisfied... If the result is positive, the reconstruction of the water surface point cloud is deemed reliable, and the water surface point cloud, interior / exterior point markers, fitting plane parameters, and quality indicators are output; otherwise, the final reliable point cloud is not forced to be output, and only low-reliability markers, failure reasons, or reprocessing prompts are output.

[0109] The parameters in the above formulas have the following meanings: Table 1:

[0110] Table 1 Parameter Meaning

[0111]

[0112] Table 2 below provides recommended values ​​for the key technical parameters of each of the above modules:

[0113] Table 2 Recommended values ​​for key technical parameters

[0114]

[0115] In practice, the above-mentioned mask can be further combined with boundary erosion or safety distance constraints to eliminate unstable pixels near the mask edge.

[0116] Example 1: Single-frame complex water surface point cloud reconstruction. A set of left and right binocular images is received. The operator selects the Region of Interest (ROI) containing the main water surface in the left-eye image, uses a water segmentation model or an existing mask to generate the water surface region, and applies a safe shrinkage of 5 to 15 pixels to the boundary. Subsequently, SURF features are extracted within the effective water surface region; ORB features can also be used when it is necessary to reduce implementation dependencies or improve operational compatibility. Nearest neighbor matching is performed on the left and right eye descriptors, using... The ratio test retains the initial matches, and then RANSAC geometric consistency is used to remove mismatches. The geometric residual threshold is set to... Pixels. Then, based on the matching distance statistics, outlier points are removed, and it is checked whether the matching points cover a sufficient water surface area.

[0117] When the number of valid matching points meets When the area coverage meets the requirements, candidate 3D point clouds are generated based on valid matching points. After removing points that are not finite-valued, exceed the valid range, and are spatial outliers from the candidate point clouds, the remaining points are fitted to the water surface plane. The residual threshold is used to clean outliers, resulting in a water surface point cloud and planar parameters. If the inlier ratio, RMSE, and overall confidence level all meet the requirements, the water surface point cloud is output; otherwise, it indicates insufficient matching points, insufficient region coverage, or unreliable planar fitting, and a low-confidence label is output.

[0118] To support the feasibility of this invention, the aforementioned regionally constrained binocular point cloud reconstruction link was embedded as a point cloud computing module into the host computer software within a prototype shore-based water level measurement platform. This was verified in experimental canals, field water surfaces, and random water environments. The relevant data are used to illustrate that the point cloud reconstruction link of this invention can support subsequent water surface plane fitting and water level calculation, and are not intended to limit this invention to a specific hardware structure or external elevation benchmark.

[0119] The experimental results are shown in Table 3 below:

[0120] Table 3 Experimental Results

[0121]

[0122] The test results above demonstrate that the region constraint, mismatch removal, point cloud cleaning, plane fitting, and confidence output chain described in this invention can form a complete closed loop in actual water surface measurement processes. Especially in water surface scenarios with weak textures, reflections, and local interference, without region constraints and confidence splitting, existing processing procedures easily output misleading point clouds supported by strong background textures or local outliers. Using this invention, low-confidence prompts can be output when there are insufficient matching points, inadequate region coverage, or excessive plane residuals, avoiding directly providing unreliable results.

[0123] Those skilled in the art should understand that, without departing from the technical concept of this invention, equivalent substitutions can be made to the following aspects:

[0124] (1) The water surface area constraint step can use artificial ROI, SAM, U-Net, traditional color or texture thresholding, shoreline prior, or a combination of multiple methods, as long as the output can form an effective water surface area mask. For example, for experimental canals with simple backgrounds or near-shore still water, artificial ROI and color thresholding can be combined to obtain the water surface area; for rivers with complex shorelines or floating objects, a deep learning segmentation model can be used to obtain the water surface mask, and then the shoreline mixed pixels can be excluded by combining the boundary safety distance; for known fixed monitoring viewpoints, historical task areas can be used as the initial ROI, and corrected according to the current segmentation results in each processing.

[0125] (2) The feature extraction step can use SURF, ORB, SIFT, AKAZE or other local feature methods; the matcher can use FLANN, brute force matching, LSH nearest neighbor search or deep feature matching methods.

[0126] (3) The geometric consistency model can be selected from the basic matrix, epipolar constraint, homography matrix or other geometric error models according to the scenario.

[0127] (4) Point cloud cleaning can be performed using fixed range threshold, statistical outlier removal, radius outlier removal, planar residual threshold, or a combination of these methods.

[0128] (5) Plane fitting can be performed using RANSAC, least squares, weighted least squares, M estimation or multi-frame joint fitting; residual threshold can be a fixed threshold, a mean plus standard deviation threshold or an adaptive threshold based on water surface fluctuation estimation.

[0129] (6) Credibility calculation may use weighted scoring, rule-based judgment or learning-based quality assessment model, but should at least reflect multiple indicators among the number of matches, the proportion of points in the point cloud, the planar residual and the regional coverage.

[0130] (7) Multi-frame joint processing strategy: Under water surface conditions with significant changes in reflection and ripples, the complete processing chain of this invention (water surface region constraint, feature matching, mismatch removal, point cloud generation, point cloud cleaning, and plane fitting) can be executed separately for multiple sets of left and right eye images to obtain the water surface point cloud and plane confidence level of each frame. For frames with satisfactory confidence levels, their water surface point cloud and plane parameters are retained; for frames with insufficient confidence levels, they are marked as low confidence and excluded from the final statistics. If the number of valid frames reaches a preset ratio, the water surface point cloud or average plane parameters after multi-frame fusion can be output, and the residual standard deviation and the proportion of valid frames are used as stability indicators.

[0131] (8) Software System Integration Method: This invention can run as an independent algorithm process or be embedded in shore-based water surface elevation measurement software or other hydraulic visual perception platforms. In one software implementation, it can provide functions such as offline import of left and right eye images, online acquisition, parameter configuration, core calculation, image matching preview, 3D point cloud view, interior / exterior point display, and result export. After obtaining the left and right eye images, the software calls the various processing modules of this invention according to the user selection or task parameters, and finally outputs the water surface point cloud, plane equation, residual index, confidence score, and processing report.

Claims

1. A robust binocular point cloud reconstruction method with region constraints for complex water surfaces, characterized in that, Includes the following steps: Step 1: Receive left and right binocular images, and determine the effective water surface area mask in the left and right binocular images. The effective water surface area mask is used to limit subsequent processing to the water surface area. Step 2: Extract local feature points within the area defined by the effective water surface area mask, and filter the local feature points according to the effective water surface area mask, boundary safety distance, feature response intensity, and spatial coverage to obtain candidate feature points; Step 3: Perform cross-nearest neighbor matching on the descriptors of the candidate feature points to obtain a candidate nearest neighbor set, and remove fuzzy matches with insufficient distinguishability between the first and second nearest neighbors by ratio test to obtain candidate matching point pairs; Step 4: Perform RANSAC in-point filtering based on geometric consistency model, anomaly filtering based on matching distance statistics, and water surface area coverage constraint on the candidate matching point pairs in sequence to obtain valid matching point pairs; Step 5: Based on the effective matching point pairs, triangulate to generate candidate 3D point clouds. Then, sequentially perform finite value judgment, effective depth judgment, spatial range judgment, and spatial outlier cleaning on the candidate 3D point clouds to obtain cleaned candidate water surface point clouds. Step 6: Perform RANSAC robust plane fitting on the cleaned candidate water surface point cloud to obtain an initial plane, calculate the residuals of each 3D point to the initial plane, update the inlier set based on the residual statistical threshold, and refit the plane to obtain the final water surface point cloud and water surface plane parameters. Step 7: Calculate the overall credibility based on the number of valid matching points, the number of 3D points, the proportion of points in the plane, the root mean square of the plane residual, and the area coverage. If the overall credibility and each sub-indicator meet the preset conditions, output the final water surface point cloud, the inner / outer point markers, the water surface plane parameters, and the quality indicators; otherwise, output a low credibility marker or a reprocessing prompt.

2. The method according to claim 1, characterized in that, The determination of the effective water surface area mask in step one includes: fusing the artificial ROI coarse constraint mask with the water surface segmentation mask to obtain the initial mask of the effective water surface area, and performing a safety shrinkage process on the boundary of the initial mask of the effective water surface area to exclude unstable pixels near the shoreline boundary, the edge of floating objects, and the specular highlight boundary.

3. The method according to claim 1, characterized in that, The local feature point extraction algorithms mentioned in step two include SURF, ORB, SIFT, or AKAZE.

4. The method according to claim 1, characterized in that, The filtering in step two includes: retaining only feature points located inside the effective area mask of the water surface, which are more than a preset safe distance from the mask boundary, have a feature response intensity higher than a preset threshold, and are not excessively concentrated within the effective area mask of the water surface.

5. The method according to claim 1, characterized in that, The ratio test described in step three satisfies: in, To prevent extremely small positive numbers with a denominator of zero; This represents the descriptor distance between the i-th feature point in the left eye and its first nearest neighbor, which is the distance between the i-th feature point and the most similar feature point found in the right eye image. This represents the descriptor distance between the i-th feature point of the left eye and its second nearest neighbor, which is the distance between the second most similar feature points after excluding the best match; This represents the threshold for the ratio test.

6. The method according to claim 1, characterized in that, Step four, the RANSAC in-point screening based on the geometric consistency model, includes: calculating the geometric residuals of each candidate matching point pair using the fundamental matrix, homography model, or epipolar constraints, and retaining matching point pairs whose geometric residuals are less than or equal to the geometric residual threshold; The anomaly filtering based on matching distance statistics in step four includes: calculating the mean and standard deviation of the descriptor distances of matching point pairs after RANSAC filtering, and removing abnormal matches whose descriptor distances are greater than the mean plus a certain number of times the standard deviation; The water surface area coverage constraint in step four includes: dividing the effective water surface area mask into several grids, counting the percentage of grids with matching point projections, and determining that the matching point spatial coverage is insufficient if the percentage of grids is lower than a preset threshold.

7. The method according to claim 1, characterized in that, The spatial outlier cleaning described in step five includes statistical outlier removal or radius-based outlier removal.

8. The method according to claim 1, characterized in that, The residual statistical threshold mentioned in step six is ​​the mean plus... Multiple standard deviations, of which The value ranges from 2 to 4; the refitted plane includes least squares fitting or weighted least squares fitting.

9. The method according to claim 1, characterized in that, The overall credibility mentioned in step seven is calculated using a weighted scoring method: in, To assess overall credibility, These are the weighting coefficients for the match count score, inlier ratio, residual score, and region coverage, respectively. ; To effectively match the number of point pairs; To minimize the number of valid matches, The threshold for planar residuals; is the regional coverage; RMSE is the root mean square of the planar residuals.

10. The method according to claim 1, characterized in that, The preset conditions mentioned in step seven include: the overall credibility is greater than or equal to the credibility threshold, the number of valid matching points is greater than or equal to the minimum number of valid matching points, the proportion of points in the plane is greater than or equal to the proportion of points in the plane threshold, and the root mean square of the plane residual is less than or equal to the plane residual threshold.

Images