Aggregate three-dimensional reconstruction method and system based on multi-view vision

Abstract

The application provides a kind of aggregate three-dimensional reconstruction method and system based on multi-view vision, it is related to computer vision and three-dimensional reconstruction technical field, the method comprises: obtaining the multi-view image of aggregate;Feature matching processing is carried out to multi-view image, including metric reconstruction to original feature descriptor, dynamically adjusting matching threshold based on local texture statistics, performing spatial balancing processing of griding adaptive quota allocation, and removing outliers based on residual evaluation of geometric model, to obtain geometric consistency in-point matching set;Based on in-point matching set, multi-view geometry reconstruction is carried out to obtain dense point cloud;Dense point cloud is mapped to regular grid to carry out high-order interpolation and low-pass filtering smoothing processing, to obtain the continuous three-dimensional surface model of aggregate.The application improves the robustness of feature matching and the integrity of reconstruction model in aggregate scene by feature metric reconstruction and spatial balancing distribution strategy, and provides high-precision three-dimensional geometric basis for aggregate granularity analysis.

Description

Technical Field

[0001] This invention relates to the field of computer vision and 3D reconstruction technology, and in particular to a method and system for 3D reconstruction of aggregates based on multi-view vision. Background Technology

[0002] The particle size distribution and morphological characteristics of aggregates (such as crushed stone, slag, grain particles, and other bulk materials) are key indicators for evaluating their processing quality and performance. To achieve accurate quantitative analysis of aggregates, it is usually necessary to obtain a high-precision three-dimensional surface model of the aggregate pile.

[0003] In existing technologies, Structure from Motion (SfM) methods based on multi-view vision are widely used for 3D object reconstruction. These methods first establish correspondences between multi-view images through feature detection and matching algorithms, then calculate camera pose and generate 3D point clouds. However, aggregate surfaces generally exhibit complex features such as high texture repeatability, local weak textures, or specular reflections. Traditional fixed-threshold feature matching strategies based on Euclidean distance are prone to generating numerous mismatches and outliers in such scenarios, leading to holes, layering, or scale distortion in the reconstructed point cloud, severely restricting the integrity and measurement accuracy of the 3D model. Summary of the Invention

[0004] To address the technical problem of high feature matching ambiguity and poor reconstruction model integrity caused by repetitive and weak textures on aggregate surfaces, this invention provides a method and system for three-dimensional reconstruction of aggregates based on multi-view vision.

[0005] The first aspect of the present invention provides a method for three-dimensional reconstruction of aggregates based on multi-view vision, comprising: Acquire multi-view images of the aggregate; The feature matching process performed on the multi-view image includes: normalizing and power-law transforming the original feature descriptor vectors detected from the scale space; dynamically adjusting the matching discrimination threshold based on the local texture statistical features of the image, and generating a candidate matching set by combining bidirectional mutual verification; performing spatial equalization processing on the candidate matching set based on rasterization adaptive quota allocation and local nonmaximum suppression; and performing geometric model fitting and residual evaluation on the matching set after spatial equalization processing to remove outliers and obtain a geometrically consistent interior point matching set. Multi-view geometric reconstruction is performed based on the geometrically consistent in-point matching set to obtain a dense point cloud of the aggregate; The dense point cloud is mapped onto a regular grid on a preset projection plane, and the elevation values ​​of the regular grid nodes are reconstructed by high-order interpolation and smoothed by low-pass filtering to obtain a continuous three-dimensional surface model of the aggregate.

[0006] Furthermore, the normalization and power transformation of the original feature descriptor vectors detected from the scale space includes: Obtain the original feature descriptor vector Based on the transformation formula The original feature descriptor vector is transformed to map the Euclidean distance metric corresponding to the original feature descriptor vector to a Hellinger distance metric, thereby obtaining the reconstructed feature descriptor vector. ;in, This is a sign function used to preserve the sign information of the elements in the original feature descriptor vector. It takes the value 1 when the input value is greater than 0, -1 when it is less than 0, and 0 when it is equal to 0. This is a preset power transformation factor.

[0007] Furthermore, the step of dynamically adjusting the matching discrimination threshold based on local texture statistical features of the image and generating a candidate matching set by combining bidirectional mutual verification includes: Obtain the grayscale standard deviation of the neighborhood of the candidate matching point and the contrast statistical benchmark of the global image; Based on threshold function Dynamically calculate adaptive discrimination threshold ,in, Based on the threshold, As a regulating factor, The global grayscale mean. The global grayscale standard deviation, This is a numerical stability term used to prevent the denominator from becoming zero or numerically unstable when the global grayscale standard deviation is too small. The condition is determined when the ratio of the nearest neighbor distance to the second nearest neighbor distance is less than the adaptive discrimination threshold. When the corresponding matching pair is accepted, the intersection of the bidirectional matching results is taken to generate the candidate matching set.

[0008] Furthermore, spatial equalization processing based on rasterized adaptive quota allocation and local nonmaximum suppression is performed on the candidate matching set, including: The image domain is divided into multiple preset grids, and the cumulative response intensity of candidate matching points within each grid is calculated. ; According to the quota allocation formula Determine the feature point retention quota for each grid. ,in, To retain the expected total number of features, As a smoothing control factor, For the first Line 1 The cumulative response intensity of candidate matching points within a column grid. For the first in all grids Line 1 Cumulative value of response intensity of candidate matching points corresponding to column grids; Within each grid, feature points that meet the retention quota are retained in descending order of response intensity, and local nonmaximum suppression based on the dynamic radius of spatial distance is applied to eliminate redundant feature points.

[0009] Furthermore, the step of performing multi-view geometric reconstruction based on the geometrically consistent in-point matching set to obtain the dense point cloud of the aggregate includes: The scale correction factor is calculated using the proportional relationship between a reference object of known spatial scale and the reconstruction model. Based on the scale correction factor, the dense point cloud of the initial reconstruction is scaled and corrected to restore the true physical scale of the aggregate.

[0010] Furthermore, the dense point cloud is mapped onto a regular grid on a preset projection plane, and the elevation values ​​of the regular grid nodes are reconstructed using high-order interpolation and smoothed using low-pass filtering to obtain a continuous three-dimensional surface model of the aggregate, including: A two-dimensional regular grid with a preset spatial step size is constructed in the XY projection plane of the dense point cloud; The dense point cloud is mapped to the nodes of the two-dimensional regular grid, and cubic interpolation is performed on the elevation values ​​of the nodes to construct a surface elevation matrix with continuous first derivatives. The surface elevation matrix is ​​convolved with a Gaussian smoothing kernel with a preset standard deviation to suppress non-systematic high-frequency fluctuations and output the continuous three-dimensional surface model.

[0011] A second aspect of the present invention provides a multi-view vision-based three-dimensional reconstruction system for aggregates, comprising: The image acquisition module is used to acquire multi-view images of the aggregate; The feature matching module is used to perform feature matching processing on the multi-view image, including: normalizing and power-law transforming the original feature descriptor vectors detected from the scale space; dynamically adjusting the matching discrimination threshold based on the local texture statistical features of the image, and generating a candidate matching set by combining bidirectional mutual verification; performing spatial equalization processing on the candidate matching set based on rasterization adaptive quota allocation and local nonmaximum suppression; and performing geometric model fitting and residual evaluation on the matching set after spatial equalization processing to remove outliers and obtain a geometrically consistent interior point matching set. The point cloud reconstruction module is used to perform multi-view geometric reconstruction based on the geometrically consistent in-point matching set to obtain a dense point cloud of the aggregate. The surface reconstruction module is used to map the dense point cloud onto a regular grid on a preset projection plane, and to perform high-order interpolation reconstruction and low-pass filtering smoothing on the elevation values ​​of the regular grid nodes to obtain a continuous three-dimensional surface model of the aggregate.

[0012] A third aspect of the present invention provides an electronic device including a memory, a processor, and a program stored in the memory and running on the processor, wherein the processor executes the program to implement the steps in the multi-view vision-based three-dimensional reconstruction method for aggregates as described in the first aspect of the present invention.

[0013] A fourth aspect of the present invention provides a computer-readable storage medium having a program stored thereon that, when executed by a processor, implements the steps of the multi-view vision-based three-dimensional reconstruction method for aggregates as described in the first aspect of the present invention.

[0014] A fifth aspect of the present invention provides a computer program product comprising software code, wherein the program in the software code performs the steps of the multi-view vision-based three-dimensional reconstruction method for aggregates as described in the first aspect of the present invention.

[0015] Compared with existing technologies, the present invention provides a method and system for 3D reconstruction of aggregates based on multi-view vision, which has the following beneficial effects: It maps the Euclidean distance metric of the original SIFT descriptor to Hellinger distance, significantly suppressing explosive mismatches caused by high gradient responses in repetitive texture regions. Furthermore, it adaptively adjusts the matching threshold through local texture complexity, ensuring feature point recall in weak texture regions and achieving a highly robust geometrically consistent inner-point matching set output. By uniformly distributing feature points in the image domain, it avoids excessive clustering of feature points in high-contrast regions, improving the integrity and surface continuity of the reconstructed point cloud. Through cubic interpolation and Gaussian smoothing of the elevation matrix, it eliminates staircase artifacts and high-frequency noise caused by point cloud discreteness, obtaining a continuous 3D surface model with continuous first derivatives that truly reflects the macroscopic packing morphology of the aggregates. Attached Figure Description

[0016] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute an undue limitation of this disclosure.

[0017] Figure 1 The flowchart shows the material aggregate three-dimensional reconstruction method based on multi-view vision provided in Embodiment 1 of the present invention. Figure 2 This is a schematic diagram of multi-view vision image acquisition provided in Embodiment 1 of the present invention. Figure 2 In the image, 'a' represents the image taken by the camera on the left. Figure 2 In this context, 'b' represents the image captured by the intermediate camera. Figure 2 In the image, 'c' represents the image captured by the camera on the right. Figure 3 This is a comparison chart of the feature matching effects of different algorithms provided in Embodiment 1 of the present invention. Figure 3The 'a' in the equation uses the SIFT algorithm; Figure 3 The 'b' in the equation uses the ORB algorithm. Figure 3 The 'c' in the algorithm uses the AKAZE algorithm; Figure 3 The d in the algorithm uses the SIFT+GMS algorithm; Figure 3 The 'e' in the equation uses the DCFM algorithm; Figure 4 This is a sparse point cloud effect diagram of material collection provided in Embodiment 1 of the present invention; Figure 5 This is a rendering of the three-dimensional reverse reconstruction model of aggregates provided in Embodiment 1 of the present invention; Figure 6 This is a reconstruction effect diagram of the three-dimensional continuous surface model of the aggregate provided in Embodiment 1 of the present invention; Figure 7 This is a block diagram of a multi-view vision-based three-dimensional reconstruction system for aggregates provided in Embodiment 2 of the present invention. Detailed Implementation

[0018] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0019] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. Furthermore, it should be understood that the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0020] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0021] All data acquisition in this embodiment is carried out in accordance with laws and regulations and with user consent, and the data is used legally.

[0022] Example 1 like Figure 1 This embodiment provides a method for three-dimensional reconstruction of aggregates based on multi-view vision, including: S1. Obtain multi-view images of the aggregate.

[0023] Specifically, a three-camera spatial layout of "left-center-right" is used to acquire images of the material collection. The center camera is the primary viewpoint (optical axis perpendicular to the material drop surface), and the left and right cameras are auxiliary viewpoints (the optical axis forms a 45° angle with the center camera). A schematic diagram of the multi-view vision image acquisition system is shown below. Figure 2 As shown, where, Figure 2 In the image, 'a' represents the image taken by the camera on the left. Figure 2 In this context, 'b' represents the image captured by the intermediate camera. Figure 2 In the image, 'c' represents the image taken by the camera on the right.

[0024] For hardware specifications, an industrial camera (such as the Hikrobot MV-CA050-10GM) can be used, along with a low-distortion lens (25mm focal length), and the shooting distance can be set to 600-800mm.

[0025] When shooting, set the shutter speed to 1 / 2500s to eliminate motion blur, the aperture to F5.6-F8.0 to ensure depth of field, and the ISO to 400-800.

[0026] After acquiring the image, distortion correction, downsampling, and normalization are performed on the acquired image (pixel values ​​are mapped to [0,1]), which are then used as input for subsequent reconstruction.

[0027] S2. Perform feature matching processing on the multi-view image, including: normalizing and power-law transforming the original feature descriptor vectors detected from the scale space; dynamically adjusting the matching discrimination threshold based on the local texture statistical features of the image, and generating a candidate matching set by combining bidirectional mutual verification; performing spatial equalization processing on the candidate matching set based on rasterization adaptive quota allocation and local nonmaximum suppression; and performing geometric model fitting and residual evaluation on the matching set after spatial equalization processing to eliminate outliers and obtain a geometrically consistent interior point matching set.

[0028] In this embodiment, the above feature matching process is referred to as the Distribution-Consistent Feature Matching (DCFM) algorithm.

[0029] Specifically, this step includes the following processing logic: (1) Scale space construction and SIFT-DoG key point detection To ensure good stability of detected feature points at different imaging distances and object scales, a Difference of Gaussian (DoG) scale space is first constructed for extreme point detection. Let the input image be... Its scale space Defined as a two-dimensional Gaussian kernel Convolution with the input image:

[0030] In the formula, For the scale Two-dimensional Gaussian kernel, This represents the convolution operation.

[0031] Difference Gaussian image It is obtained by subtracting Gaussian images of adjacent scales:

[0032] Extreme points are retrieved within a 3×3 three-dimensional neighborhood (x, y, σ). To eliminate unstable edge response points, this invention constructs a Hessian matrix H for each candidate point at the current scale and uses the principal curvature ratio criterion for screening: if... If a keypoint is found to be identifiable, it is retained; otherwise, it is discarded. Preferably, the principal curvature ratio threshold r is set to 10. Subsequently, the principal directions of stable keypoints are calculated to ensure rotation invariance, and a 128-dimensional original SIFT feature descriptor vector is generated. .

[0033] (2) Feature descriptor metric reconstruction The original SIFT descriptor often uses Euclidean distance matching, which can easily lead to high ambiguity when dealing with repetitive textures or unevenly lit aggregate regions. To scale excessively high gradient responses and improve the discriminative power of the descriptor, this invention performs L1 normalization and power transformation on the original descriptor. The vector obtained after the transformation is defined as the reconstructed feature descriptor vector. The specific metric reconstruction formula is as follows:

[0034] in, For the original feature descriptor vector, α is the sign function, used to preserve the sign information of the original feature descriptor vector elements. It takes the value 1 when the input value is greater than 0, -1 when it is less than 0, and 0 when it is equal to 0. α is a preset power transformation factor. In this embodiment, α is 0.5 (i.e., Root-SIFT). To reconstruct the feature descriptor vectors, Euclidean distance is mapped to Hellinger distance, enhancing robustness to illumination variations and local noise. This transformation approximates the feature metric from Euclidean space to Hellinger distance space without altering the original feature matching framework, thereby significantly suppressing bursty false matches in repetitive texture regions and enhancing feature discriminative power in low-texture regions.

[0035] (3) Candidate matching generation and adaptive ratio discrimination After obtaining the reconstructed descriptor subset, a K-nearest neighbor (KNN, preferably K=2) search is performed to obtain the distances between the nearest and second nearest neighbors. To address the issue of missed or false matches caused by fixed thresholds in complex texture scenes, this invention introduces an adaptive ratio discrimination strategy, dynamically adjusting the matching threshold based on local image statistics. The grayscale standard deviation of each candidate point's neighborhood is calculated. And combined with the average contrast of the entire image Standard deviation of gray level of the whole image Normalization and adaptive threshold function The design is as follows:

[0036] Where τbase=0.7, λ=0.1, This is a numerical stability term used to prevent the denominator from becoming zero or numerically unstable when the global grayscale standard deviation is too small. This strategy allows the algorithm to appropriately relax the matching threshold in weakly textured regions to improve feature point recall, while automatically tightening the threshold in areas of strong or repetitive texture to enhance anti-ambiguity. A candidate match is only accepted if the ratio of the nearest neighbor distance to the second nearest neighbor distance is less than T.

[0037] To further eliminate accidental one-way matching and ensure geometric consistency, a two-way mutual check (SymmetricCheck) is performed: Let the set of nearest neighbors from image A to image B be MA→B, and the set from image B to A be MB→A. Then the final retained preliminary matching set is the intersection of the two, Msym = MA→B∩MB→A.

[0038] (4) Spatial Equilibrium: Multi-view 3D reconstruction places high demands on the spatial distribution density of feature points. To prevent feature points from becoming overly concentrated in high-contrast aggregate areas (causing local clustering) and to improve global coverage, this invention implements a spatial equalization mechanism. Specifically, it includes the following steps: ① The image domain is divided into Gx×Gy equally divided grids. In this embodiment, the number of grids is 8×8; ② Sum of the reliability of candidate matching points for each grid cell. And adaptively allocate the retention quota for each grid according to the score weight. :

[0039] in, To retain the desired total number of features, Used to control the smoothness of quota allocation. For the first Line 1 The cumulative response intensity of candidate matching points within a column grid. For the first in all grids Line 1 The cumulative response intensity of the candidate matching points corresponding to the column grid.

[0040] ③ Within each grid cell, candidate matching points are arranged in descending order of score, and local nonmaximum suppression (NMS) based on dynamic radius R is applied. Low-scoring points are eliminated within the suppression radius, which is adaptively calculated as follows:

[0041] This ensures that the number of matching points is not only sufficient, but also that they are evenly distributed throughout the image space.

[0042] (5) Geometric consistency verification and interior point output Finally, this invention employs a robust estimation method (USAC or MAGSAC algorithm) to fit a geometric model (such as the fundamental matrix F or the essential matrix E) to the candidate matching set selected through the above steps. The Sampson distance is used as a metric for evaluating local residuals.

[0043] The geometric consistency of each feature matching pair is determined based on the residuals, and outliers are removed. Finally, a high-precision geometric interior point matching set is output, providing high-quality input data for subsequent 3D point cloud inverse reconstruction.

[0044] Compared to the traditional SIFT algorithm, the DCFM algorithm of this invention improves the correct matching rate (CMR) to 99.22% in complex texture scenarios of asphalt mixtures, and increases the running speed by approximately 14.3%, effectively solving the mismatch problem caused by repeated textures. Through a spatial equalization mechanism, it significantly improves the uniformity of feature point distribution on the aggregate surface, avoiding local reconstruction voids. A comparison of the feature matching effects of different algorithms is provided below. Figure 3 As shown, where, Figure 3 In the image, 'a' represents the result of using the SIFT algorithm. Figure 3 In the diagram, 'b' represents the effect of using the ORB algorithm. Figure 3 In the image, 'c' represents the result of using the AKAZE algorithm. Figure 3 In the image, d represents the result achieved using the SIFT+GMS algorithm. Figure 3 In the figure, 'e' represents the effect of using the DCFM algorithm of this invention. It can be seen that the method of this invention significantly reduces mismatched lines and improves the uniformity of the distribution of matching points.

[0045] S3. Perform multi-view geometric reconstruction based on the geometrically consistent in-point matching set to obtain the dense point cloud of the aggregate.

[0046] Specifically, it includes the following sub-steps: S3.1 Sparse Reconstruction: Import the matching coordinate file output by DCFM into the SfM system (such as Visual SFM). Utilize multi-view geometric constraints to jointly solve for the camera pose (rotation matrix R and translation vector t) and the 3D coordinates of spatial points, generating a sparse point cloud. The resulting sparse point cloud of the reconstructed aggregate is shown in the image below. Figure 4 As shown.

[0047] S3.2 Dense Reconstruction: The CMVS / PMVS algorithm is used to generate high-density point clouds across views (minimum point cloud spacing is set to 0.01mm), and the local point clouds are then fused into the overall model through global registration. The final aggregate 3D reverse reconstruction model is as follows: Figure 5 As shown.

[0048] S3.3 Model Correction: Coordinate System Correction: Align the model coordinate system to the experimental reference plane to eliminate spatial tilt. Scale Calibration: Calculate the scale factor SS using a reference object of known size (such as the side length of the background cloth) to scale and correct the model, restoring the true physical scale of the aggregate.

[0049] S4. Map the dense point cloud onto a regular grid on a preset projection plane, and perform high-order interpolation reconstruction and low-pass filtering smoothing on the elevation values ​​of the regular grid nodes to obtain a continuous three-dimensional surface model of the aggregate.

[0050] Specifically, S4 includes the following sub-steps: S4.1 Regular Mesh Resampling: Construct a regular two-dimensional mesh in the XY projection plane of the point cloud, with a mesh density set to Δx=Δy=0.1 mm. Map the discrete point cloud to the mesh nodes to initially establish the elevation matrix.

[0051] S4.2 Cubic Interpolation Reconstruction: The elevation values ​​of the grid nodes are calculated using cubic interpolation. This method comprehensively considers the information of neighboring sampling points, ensuring the continuity of the reconstructed surface in function values ​​and first derivatives, and avoiding staircase artifacts.

[0052] S4.3 Gaussian Smoothing: A two-dimensional Gaussian filter is introduced onto the interpolated elevation surface to suppress local high-frequency noise (burrs) caused by matching errors or dust. The standard deviation parameter of the Gaussian kernel function is set to σ = 0.5 - 1.0. This parameter setting can eliminate outliers while preserving the macroscopic outline and elevation variation trend of the aggregate accumulation.

[0053] S4.4 Model Output: Generates a continuous 3D topological surface model containing accurate elevation information (exportable as a .obj file) for subsequent particle segmentation and virtual sieving analysis. The final generated aggregate 3D continuous surface model reconstruction effect is as follows: Figure 6As shown.

[0054] Example 2 like Figure 7 This embodiment provides a multi-view vision-based three-dimensional reconstruction system for aggregates, including: The image acquisition module is used to acquire multi-view images of the aggregate; The feature matching module is used to perform feature matching processing on the multi-view image, including: normalizing and power-law transforming the original feature descriptor vectors detected from the scale space; dynamically adjusting the matching discrimination threshold based on the local texture statistical features of the image, and generating a candidate matching set by combining bidirectional mutual verification; performing spatial equalization processing on the candidate matching set based on rasterization adaptive quota allocation and local nonmaximum suppression; and performing geometric model fitting and residual evaluation on the matching set after spatial equalization processing to remove outliers and obtain a geometrically consistent interior point matching set. The point cloud reconstruction module is used to perform multi-view geometric reconstruction based on the geometrically consistent in-point matching set to obtain a dense point cloud of the aggregate. The surface reconstruction module is used to map the dense point cloud onto a regular grid on a preset projection plane, and to perform high-order interpolation reconstruction and low-pass filtering smoothing on the elevation values ​​of the regular grid nodes to obtain a continuous three-dimensional surface model of the aggregate.

[0055] Example 3 Embodiment 3 of the present invention provides an electronic device.

[0056] An electronic device includes a memory, a processor, and a program stored in the memory and running on the processor. The processor includes, but is not limited to, at least one of a central processing unit (CPU), a graphics processing unit (GPU), a neural network processor (NPU), a tensor processor (TPU), or an artificial intelligence acceleration chip. When executing the program, the processor implements the steps in the multi-view vision-based three-dimensional reconstruction method for aggregates as described in Embodiment 1 of the present invention.

[0057] The detailed steps are the same as those of the multi-view vision-based three-dimensional reconstruction method for aggregates provided in Example 1, and will not be repeated here.

[0058] Example 4 Embodiment 4 of the present invention provides a computer-readable storage medium.

[0059] A computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in the multi-view vision-based three-dimensional reconstruction method for aggregates as described in Embodiment 1 of the present invention.

[0060] The detailed steps are the same as those of the multi-view vision-based three-dimensional reconstruction method for aggregates provided in Example 1, and will not be repeated here.

[0061] Example 5 Embodiment 5 of the present invention provides a computer program product.

[0062] A computer program product includes software code, wherein the program in the software code performs the steps of the aggregate three-dimensional reconstruction method based on multi-view vision as described in Embodiment 1 of the present invention.

[0063] The detailed steps are the same as those of the multi-view vision-based three-dimensional reconstruction method for aggregates provided in Example 1, and will not be repeated here.

[0064] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages. For example, in one implementation, the methods and systems can be developed based on deep learning frameworks (such as TensorFlow, PyTorch, etc.) and using the Python language. Those skilled in the art will understand that other suitable programming languages ​​or tools can also be used for implementation without departing from the core ideas of the present invention.

[0065] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0066] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0067] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0068] The above description is merely a preferred embodiment of this practice and is not intended to limit the scope of this practice. Various modifications and variations can be made to this practice by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this practice should be included within the protection scope of this practice.

Claims

1. A method for three-dimensional reconstruction of aggregates based on multi-view vision, characterized in that, include: Acquire multi-view images of the aggregate; The feature matching process performed on the multi-view image includes: normalizing and power-law transforming the original feature descriptor vectors detected from the scale space; dynamically adjusting the matching discrimination threshold based on the local texture statistical features of the image, and generating a candidate matching set by combining bidirectional mutual verification; performing spatial equalization processing on the candidate matching set based on rasterization adaptive quota allocation and local nonmaximum suppression; and performing geometric model fitting and residual evaluation on the matching set after spatial equalization processing to remove outliers and obtain a geometrically consistent interior point matching set. Multi-view geometric reconstruction is performed based on the geometrically consistent in-point matching set to obtain a dense point cloud of the aggregate; The dense point cloud is mapped onto a regular grid on a preset projection plane, and the elevation values ​​of the regular grid nodes are reconstructed by high-order interpolation and smoothed by low-pass filtering to obtain a continuous three-dimensional surface model of the aggregate.

2. The method according to claim 1, characterized in that, The normalization and power transformation of the original feature descriptor vectors detected from the scale space includes: Obtain the original feature descriptor vector Based on the transformation formula The original feature descriptor vector is transformed to map the Euclidean distance metric corresponding to the original feature descriptor vector to a Hellinger distance metric, thereby obtaining the reconstructed feature descriptor vector. ;in, This is a sign function used to preserve the sign information of the elements in the original feature descriptor vector. It takes the value 1 when the input value is greater than 0, -1 when it is less than 0, and 0 when it is equal to 0. This is the preset power transformation factor.

3. The method according to claim 1, characterized in that, The method of dynamically adjusting the matching threshold based on local texture statistical features of the image and generating a candidate matching set by combining bidirectional mutual verification includes: Obtain the grayscale standard deviation of the neighborhood of the candidate matching point and the contrast statistical benchmark of the global image; Based on threshold function Dynamically calculate adaptive discrimination threshold ,in, Based on the threshold, As a regulating factor, The global grayscale mean. The global grayscale standard deviation, This is a numerical stability term used to prevent the denominator from becoming zero or numerically unstable when the global grayscale standard deviation is too small. The condition is determined when the ratio of the nearest neighbor distance to the second nearest neighbor distance is less than the adaptive discrimination threshold. When the corresponding matching pair is accepted, the intersection of the bidirectional matching results is taken to generate the candidate matching set.

4. The method according to claim 1, characterized in that, Perform spatial equalization processing based on rasterized adaptive quota allocation and local nonmaximum suppression on the candidate matching set, including: The image domain is divided into multiple preset grids, and the cumulative response intensity of candidate matching points within each grid is calculated. ; According to the quota allocation formula Determine the feature point retention quota for each grid. ,in, To retain the expected total number of features, As a smoothing control factor, For the first Line number The cumulative response intensity of candidate matching points within a column grid. For the first in all grids Line number Cumulative value of response intensity of candidate matching points corresponding to column grids; Within each grid, feature points that meet the retention quota are retained in descending order of response intensity, and local nonmaximum suppression based on the dynamic radius of spatial distance is applied to eliminate redundant feature points.

5. The method according to claim 1, characterized in that, The process of performing multi-view geometric reconstruction based on the geometrically consistent in-point matching set to obtain a dense point cloud of aggregates includes: The scale correction factor is calculated using the proportional relationship between a reference object of known spatial scale and the reconstruction model. Based on the scale correction factor, the dense point cloud of the initial reconstruction is scaled and corrected to restore the true physical scale of the aggregate.

6. The method according to claim 1, characterized in that, The dense point cloud is mapped onto a regular grid on a preset projection plane, and the elevation values ​​of the regular grid nodes are reconstructed by high-order interpolation and smoothed by low-pass filtering to obtain a continuous three-dimensional surface model of the aggregate, including: A two-dimensional regular grid with a preset spatial step size is constructed in the XY projection plane of the dense point cloud; The dense point cloud is mapped to the nodes of the two-dimensional regular grid, and cubic interpolation is performed on the elevation values ​​of the nodes to construct a surface elevation matrix with continuous first derivatives. The surface elevation matrix is ​​convolved with a Gaussian smoothing kernel with a preset standard deviation to suppress non-systematic high-frequency fluctuations and output the continuous three-dimensional surface model.

7. A three-dimensional reconstruction system for aggregates based on multi-view vision, characterized in that, include: The image acquisition module is used to acquire multi-view images of the aggregate; The feature matching module is used to perform feature matching processing on the multi-view image, including: normalizing and power-law transforming the original feature descriptor vectors detected from the scale space; dynamically adjusting the matching discrimination threshold based on the local texture statistical features of the image, and generating a candidate matching set by combining bidirectional mutual verification; performing spatial equalization processing on the candidate matching set based on rasterization adaptive quota allocation and local nonmaximum suppression; and performing geometric model fitting and residual evaluation on the matching set after spatial equalization processing to remove outliers and obtain a geometrically consistent interior point matching set. The point cloud reconstruction module is used to perform multi-view geometric reconstruction based on the geometrically consistent in-point matching set to obtain a dense point cloud of the aggregate. The surface reconstruction module is used to map the dense point cloud onto a regular grid on a preset projection plane, and to perform high-order interpolation reconstruction and low-pass filtering smoothing on the elevation values ​​of the regular grid nodes to obtain a continuous three-dimensional surface model of the aggregate.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the program, it implements the steps of the multi-view vision-based three-dimensional reconstruction method for aggregates according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the multi-view vision-based three-dimensional reconstruction method for aggregates according to any one of claims 1 to 6.

10. A computer program product, comprising software code, characterized in that, The program in the software code executes the steps of the multi-view vision-based three-dimensional reconstruction method for aggregates according to any one of claims 1 to 6.

Images