Three-dimensional reconstruction method and system applied to 3D vision

Abstract

The invention discloses a three-dimensional reconstruction method and system applied to 3D vision, two-dimensional image data used for three-dimensional reconstruction are obtained through shooting of a camera with multiple visual angles, more comprehensive scene information can be provided through the multi-angle image data, the accuracy and integrity of subsequent three-dimensional reconstruction are facilitated, and meanwhile, the three-dimensional reconstruction method and system applied to 3D vision can be applied to 3D reconstruction. The feature points are key information points in the images, association between the images of different view angles can be established through matching of the feature points, the accuracy of three-dimensional reconstruction is improved, in addition, structural information of a three-dimensional space can be effectively recovered from the images of multiple view angles according to the multi-view geometric principle, dense point cloud data are generated, and the accuracy of three-dimensional reconstruction is improved. The point cloud data is an important basis for subsequent three-dimensional reconstruction, the overall quality of the point cloud data can be improved by optimizing the preliminarily recovered point cloud data, and in addition, a three-dimensional model can be presented to a user in a visual and vivid manner through visual operation, so that the user can conveniently observe, analyze and interact.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional reconstruction technology, and in particular to a three-dimensional reconstruction method and system applied to 3D vision. Background Art

[0002] In the current field of 3D vision and 3D reconstruction, although significant progress has been made, existing 3D reconstruction methods still face a series of technical challenges and defects, which limit the accuracy, robustness and real-time performance of 3D reconstruction, and thus affect its wide application in various fields (such as virtual reality, augmented reality, autonomous driving, medical imaging, etc.). First, traditional 3D reconstruction methods often rely on image data from a single perspective, which makes the reconstruction results vulnerable to occlusion, illumination changes and perspective limitations, making it difficult to fully and accurately restore the 3D scene. This limitation makes the 3D model lack details and accuracy in complex scenes and cannot meet the needs of high-precision applications. Second, feature matching is a key link in 3D reconstruction, but existing feature matching algorithms are prone to mismatching or missing matches when processing images with complex textures, repeated structures or low contrast, which not only reduces the accuracy of 3D reconstruction, but also increases the difficulty and computational complexity of subsequent processing. In addition, in terms of point cloud data processing, existing methods lack effective filtering and denoising mechanisms when processing point cloud data containing noise, outliers or redundant points, which results in the reconstructed 3D model surface being rough and not smooth, affecting the visual effect and subsequent application of the model. Finally, in terms of 3D model visualization, existing methods usually only provide basic rendering and display functions, lack interactivity and immersion, which limits the application potential of 3D models in virtual reality, augmented reality and other fields, and cannot meet users' needs for high-quality and highly immersive experience. Summary of the invention

[0003] In view of this, the present invention proposes a three-dimensional reconstruction method and system applied to 3D vision, which can solve the defects of the prior art such as insufficient accuracy, poor robustness, imperfect point cloud data processing and limited visualization effect.

[0004] The technical solution of the present invention is achieved in this way:

[0005] A three-dimensional reconstruction method applied to 3D vision, specifically comprising:

[0006] Acquire two-dimensional image data for three-dimensional reconstruction, wherein the two-dimensional image data is captured by cameras from multiple viewing angles;

[0007] Use feature matching algorithms to detect feature points in two-dimensional images, and match them between multiple two-dimensional images to find the corresponding relationship between feature points in different images;

[0008] Based on the correspondence between feature points, the point cloud data in three-dimensional space is restored using the principle of multi-view geometry;

[0009] Optimizing the point cloud data;

[0010] Based on the optimized point cloud data and combined with 3D reconstruction technology, the final 3D model is constructed;

[0011] The constructed three-dimensional model is visualized.

[0012] As a further optional solution of the three-dimensional reconstruction method applied to 3D vision, the feature matching algorithm is used to detect feature points in a two-dimensional image, and matching is performed between multiple two-dimensional images to find the corresponding relationship between feature points in different images, which specifically includes:

[0013] A scale-invariant feature transformation algorithm is applied to detect feature points in each 2D image;

[0014] For each detected feature point, the scale-invariant feature transformation algorithm calculates the gradient direction and magnitude around it, and generates a feature descriptor with rotation and scale invariance based on the gradient direction and magnitude;

[0015] Using the generated feature descriptors, match feature points between multiple two-dimensional images to find matching feature point pairs;

[0016] The matched feature point pairs are screened and optimized. After screening and optimization, the remaining matching point pairs represent the projections of the same three-dimensional points in different images on the two-dimensional plane, thereby finding the correspondence between the feature points in different images.

[0017] As a further optional solution of the three-dimensional reconstruction method applied to 3D vision, the point cloud data in the three-dimensional space is restored using the multi-view geometry principle based on the correspondence between feature points, specifically including:

[0018] Estimate the basic matrix based on the correspondence between feature points;

[0019] Using the basic matrix and the matching feature point pairs, the camera motion parameters and the positions of the 3D points in the scene are restored;

[0020] Triangulation is performed based on the camera's motion parameters and the positions of the three-dimensional points in the scene to calculate the coordinate pairs of the three-dimensional points.

[0021] As a further optional solution of the three-dimensional reconstruction method applied to 3D vision, estimating the basic matrix based on the correspondence between feature points specifically includes:

[0022] According to the correspondence between feature points, eight non-collinear matching point pairs are selected using the eight-point method;

[0023] Using the coordinate information of eight non-collinear matching point pairs, a linear equation system is constructed;

[0024] Solve the system of linear equations to obtain the fundamental matrix.

[0025] As a further optional solution of the three-dimensional reconstruction method applied to 3D vision, the final three-dimensional model is constructed based on the optimized point cloud data in combination with the three-dimensional reconstruction technology, specifically including:

[0026] Using the points in the point cloud data as input, the Delaunay triangulation algorithm is used to construct triangular meshes between the point clouds, and the constructed triangular meshes are optimized;

[0027] Acquire a texture image corresponding to the triangular mesh;

[0028] The collected texture image is mapped onto the triangular mesh through texture mapping technology to obtain the final three-dimensional model.

[0029] As a further optional solution of the three-dimensional reconstruction method applied to 3D vision, the method uses points in the point cloud data as input and adopts the Delaunay triangulation algorithm to construct a triangular mesh between the point clouds, specifically including:

[0030] Project the point cloud data onto a two-dimensional plane;

[0031] For each point in the point cloud, find its neighboring points according to the preset distance. These neighboring points will be used as candidate points for constructing the triangle mesh.

[0032] Select three points from the point cloud as the vertices of the initial triangle, and then gradually add more points to construct more triangles according to the Delaunay criterion;

[0033] After all triangles are constructed, they are optimized and merged to obtain the final triangle mesh.

[0034] As a further optional solution of the three-dimensional reconstruction method applied to 3D vision, the optimization processing of the point cloud data specifically includes:

[0035] The point cloud data is subjected to denoising, filtering and smoothing, down-sampling and outlier removal processing.

[0036] A three-dimensional reconstruction system applied to 3D vision, comprising:

[0037] An acquisition module, used to acquire two-dimensional image data for three-dimensional reconstruction, wherein the two-dimensional image data is captured by cameras from multiple viewing angles;

[0038] The feature detection and matching module is used to detect feature points in a two-dimensional image using a feature matching algorithm, and to match between multiple two-dimensional images to find the corresponding relationship between feature points in different images;

[0039] Point cloud data recovery module, used to recover point cloud data in three-dimensional space based on the correspondence between feature points and feature points using the principle of multi-view geometry;

[0040] A point cloud processing module, used for optimizing the point cloud data;

[0041] The construction module is used to construct the final 3D model based on the optimized point cloud data and combined with 3D reconstruction technology;

[0042] The visualization module is used to visualize the constructed three-dimensional model.

[0043] A computing device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned three-dimensional reconstruction method applied to 3D vision when executing the computer program.

[0044] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned three-dimensional reconstruction method applied to 3D vision.

[0045] The beneficial effects of the present invention are as follows: two-dimensional image data for three-dimensional reconstruction is acquired by shooting with cameras from multiple perspectives, and the multi-angle image data can provide more comprehensive scene information, which is helpful for the accuracy and completeness of subsequent three-dimensional reconstruction. At the same time, feature points are key information points in the image, and they have good recognizability and stability in the image. Through the matching of feature points, the association between images from different perspectives can be established, which provides an important basis for subsequent three-dimensional reconstruction. In addition, the principle of multi-view geometry can effectively recover the structural information of the three-dimensional space from images from multiple perspectives and generate dense point cloud data. These point cloud data are an important basis for subsequent three-dimensional reconstruction. Optimizing the initially restored point cloud data can remove noise points, fill in missing areas, and reduce redundant data, thereby improving the overall quality of the point cloud data. In addition, the visualization operation enables the three-dimensional model to be presented to the user in an intuitive and vivid manner, which is convenient for the user to observe, analyze and interact. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 A schematic diagram of a process flow of a three-dimensional reconstruction method applied to 3D vision provided by an embodiment of the present invention;

[0048] Figure 2 A schematic diagram of the composition of a three-dimensional reconstruction system applied to 3D vision provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] refer to Figure 1 to Figure 2 , a three-dimensional reconstruction method applied to 3D vision, specifically comprising:

[0051] Acquire two-dimensional image data for three-dimensional reconstruction, wherein the two-dimensional image data is captured by cameras from multiple viewing angles;

[0052] Use feature matching algorithms to detect feature points in two-dimensional images, and match them between multiple two-dimensional images to find the corresponding relationship between feature points in different images;

[0053] Based on the correspondence between feature points, the point cloud data in three-dimensional space is restored using the principle of multi-view geometry;

[0054] Optimizing the point cloud data;

[0055] Based on the optimized point cloud data and combined with 3D reconstruction technology, the final 3D model is constructed;

[0056] The constructed three-dimensional model is visualized.

[0057] In this embodiment, two-dimensional image data for three-dimensional reconstruction is acquired by shooting with cameras from multiple perspectives. The multi-angle image data can provide more comprehensive scene information, which is helpful for the accuracy and completeness of subsequent three-dimensional reconstruction. At the same time, feature points are key information points in the image. They have good recognizability and stability in the image. Through the matching of feature points, the association between images from different perspectives can be established, providing an important basis for subsequent three-dimensional reconstruction. In addition, the multi-view geometry principle can effectively recover the structural information of the three-dimensional space from images from multiple perspectives and generate dense point cloud data. These point cloud data are an important basis for subsequent three-dimensional reconstruction. Optimizing the initially restored point cloud data can remove noise points, fill in missing areas, and reduce redundant data, thereby improving the overall quality of the point cloud data. In addition, the visualization operation enables the three-dimensional model to be presented to the user in an intuitive and vivid way, which is convenient for the user to observe, analyze and interact.

[0058] Preferably, the method of using a feature matching algorithm to detect feature points in a two-dimensional image and matching between multiple two-dimensional images to find the corresponding relationship between feature points in different images specifically includes:

[0059] A scale-invariant feature transformation algorithm is applied to detect feature points in each 2D image;

[0060] For each detected feature point, the scale-invariant feature transformation algorithm calculates the gradient direction and magnitude around it, and generates a feature descriptor with rotation and scale invariance based on the gradient direction and magnitude;

[0061] Using the generated feature descriptors, match feature points between multiple two-dimensional images to find matching feature point pairs;

[0062] The matched feature point pairs are screened and optimized. After screening and optimization, the remaining matching point pairs represent the projections of the same three-dimensional points in different images on the two-dimensional plane, thereby finding the correspondence between the feature points in different images.

[0063] In this embodiment, the scale-invariant feature transform (SIFT) algorithm can accurately detect stable feature points in complex scenes because of its invariance to scale scaling, rotation and even brightness changes, which is crucial for subsequent 3D reconstruction because accurate feature points are the basis for building 3D models; the SIFT algorithm generates a unique descriptor for each feature point, which is based on the gradient direction and size around the feature point and has rotation and scale invariance, so that even in images taken from different perspectives, the same feature point can be found by matching descriptors, thereby enhancing the robustness of the algorithm; using the generated feature descriptors for feature point matching, it is possible to quickly find the corresponding feature points between multiple images. Mutually matching feature point pairs. At the same time, by screening and optimizing matching point pairs, false matches can be further removed, the matching accuracy can be improved, and more reliable data support can be provided for subsequent 3D reconstruction. The matching point pairs after screening and optimization represent the projections of the same 3D points in different images on the 2D plane. These correspondences are the key to recovering 3D spatial information. Accurate correspondences can ensure the accuracy of 3D reconstruction results and reduce errors. Feature point matching and the establishment of correspondences are the basis for multi-view geometry to restore 3D information. Accurate correspondences provide reliable data support for subsequent point cloud data recovery, 3D model construction and other steps, which helps to improve the efficiency and accuracy of the entire 3D reconstruction process.

[0064] It should be noted that the SIFT algorithm is used to independently detect local feature points in each two-dimensional image. These feature points are usually corner points, edge points, etc. in the image. They are unique and stable in the image and can remain consistent at different scales and rotation angles. The feature descriptor is a high-dimensional vector used in subsequent feature matching steps. By calculating the similarity between feature descriptors in different images (such as Euclidean distance, Hamming distance, etc.), matching feature point pairs are found. Due to the possibility of mismatching, the matched feature point pairs need to be screened and optimized. The screening and optimization methods include using algorithms such as RANSAC (random sampling consensus algorithm) to estimate the basic matrix or essential matrix, thereby removing matching point pairs that do not meet the multi-view geometric constraints.

[0065] In addition, in the SIFT algorithm, for each detected feature point, the specific steps of calculating the gradient direction and magnitude around it and generating a feature descriptor with rotation and scale invariance include:

[0066] First, a scale space pyramid of the image is constructed. Gaussian blur and downsampling are used to simulate the representation of the image at different scales. In the scale space pyramid, the grayscale values ​​of each pixel are compared with those of its neighbors (including the same scale and adjacent scales) to detect local extreme points in the scale space. These extreme points are candidate key points.

[0067] Accurately locate each candidate key point in the scale space, and determine the exact position and scale of the key point by fitting the Taylor series expansion;

[0068] Calculate the gradient direction and magnitude of the pixels in the scale neighborhood where the key point is located, and use the histogram to count the gradient direction and magnitude of the pixels in the neighborhood. The peak of the histogram represents the main direction of the key point. In order to enhance the robustness of the matching, the secondary peak in the histogram is usually selected as the auxiliary direction.

[0069] According to the position, scale and direction information of the key point, a 16x16 window is taken in its neighborhood and divided into 4x4 sub-regions. In each sub-region, the gradient histograms in 8 directions (i.e. 45 degrees in each direction) are calculated to form an 8-dimensional vector. Therefore, for each key point, a 4x4x8=128-dimensional feature descriptor can be generated. This feature descriptor is rotation and scale invariant and can be used in the subsequent feature point matching process.

[0070] Preferably, the method of restoring point cloud data in three-dimensional space using the principle of multi-view geometry based on the correspondence between feature points specifically includes:

[0071] Estimate the basic matrix based on the correspondence between feature points;

[0072] Using the basic matrix and the matching feature point pairs, the camera motion parameters and the positions of the 3D points in the scene are restored;

[0073] Triangulation is performed based on the camera's motion parameters and the positions of the three-dimensional points in the scene to calculate the coordinate pairs of the three-dimensional points.

[0074] In this embodiment, after the feature point matching is completed, we use the epipolar geometry principle in multi-view geometry to estimate the basic matrix by calculating the correspondence between the feature points based on the matched feature point pairs. The basic matrix describes the epipolar geometry relationship between the two views and is a very important step in 3D reconstruction. After having the basic matrix, we can further use it, combined with the camera's intrinsic parameter matrix, to restore the camera's motion parameters (i.e., the camera's position and posture in different views) by solving the essential matrix and the camera posture. At the same time, using these parameters and the feature point matching relationship, the position of the 3D points in the scene can be further restored. Triangulation uses the correspondence between feature points in at least two views and the camera's internal and external parameters to accurately estimate the position of a point in 3D space through geometric calculations. In this step, the previously restored camera motion parameters and 3D point positions (which may be preliminary estimates) are used to calculate more accurate 3D point coordinate pairs through triangulation.

[0075] Preferably, estimating the basic matrix based on the correspondence between the feature points specifically includes:

[0076] According to the correspondence between feature points, eight non-collinear matching point pairs are selected using the eight-point method;

[0077] Using the coordinate information of eight non-collinear matching point pairs, a linear equation system is constructed;

[0078] Solve the system of linear equations to obtain the fundamental matrix.

[0079] In this embodiment, by selecting eight non-collinear matching point pairs, this point selection method reduces the calculation error caused by collinearity and improves the stability of the basic matrix estimation. As a classic matrix estimation method, the eight-point method shows strong robustness when dealing with general scenes and can effectively cope with a certain degree of noise and interference; using eight matching point pairs to construct a linear equation group and solve the basic matrix, this process is relatively direct and computationally moderate, and can be completed in a reasonable time to meet the needs of real-time or quasi-real-time applications; this technical solution does not depend on specific scenes or objects, has strong versatility, and is suitable for a variety of three-dimensional reconstruction scenes, including object reconstruction in static scenes and dynamic scenes.

[0080] It should be noted that the specific steps of constructing a linear equation system using the coordinate information of eight non-collinear matching point pairs include:

[0081] According to the coordinates of the selected matching point pairs, a linear system of equations is constructed using the constraints of epipolar geometry (i.e., the coordinates of any pair of matching points p and p' in the two views should satisfy p'^T*F*p=0, where F is the basic matrix);

[0082] Solving the linear equations to obtain a preliminary solution of the fundamental matrix;

[0083] The obtained preliminary basic matrix is ​​normalized to eliminate the error introduced by the different image coordinate scales;

[0084] Force the base matrix to satisfy its intrinsic properties, such as the rank 2 constraint, usually through subsequent processing such as SVD (singular value decomposition);

[0085] Use more matching point pairs to verify the accuracy of the fundamental matrix and evaluate the quality of the matrix by calculating the proportion of inliers (points that meet the constraints of the fundamental matrix);

[0086] If necessary, the fundamental matrix can be further optimized, for example using a nonlinear optimization algorithm to minimize the inlier projection error.

[0087] Preferably, the method of constructing a final three-dimensional model based on the optimized point cloud data in combination with three-dimensional reconstruction technology specifically includes:

[0088] Using the points in the point cloud data as input, the Delaunay triangulation algorithm is used to construct triangular meshes between the point clouds, and the constructed triangular meshes are optimized;

[0089] Acquire a texture image corresponding to the triangular mesh;

[0090] The collected texture image is mapped onto the triangular mesh through texture mapping technology to obtain the final three-dimensional model.

[0091] In this embodiment, a Delaunay triangulation algorithm is used to construct a triangular mesh. This algorithm can generate triangles that are as uniform and well-shaped as possible, reducing errors in the triangulation process, thereby ensuring the geometric accuracy of the three-dimensional model. The constructed triangular mesh is optimized to further reduce errors caused by uneven mesh division or poor shape, thereby improving the accuracy of the three-dimensional model. A texture image corresponding to the triangular mesh is collected and mapped to the triangular mesh through texture mapping technology. This step significantly enhances the realism of the three-dimensional model. The texture image provides detailed information such as color and texture of the object surface, making the three-dimensional model more realistic visually. This technical solution is efficient. The algorithm and process design can achieve a faster processing speed while ensuring accuracy, especially in the processing of large-scale point cloud data. The efficiency of 3D reconstruction can be further improved by optimizing the algorithm and parallel processing technology. This technical solution has strong scalability and flexibility. For example, the parameters of Delaunay triangulation can be adjusted according to actual needs, or different texture mapping algorithms can be used to adapt to the needs of different application scenarios and model complexity. This 3D reconstruction technology based on point cloud data is not only suitable for the reconstruction of static scenes, but can also be extended to the tracking and reconstruction of dynamic objects. Combined with advanced technologies such as deep learning, it can further improve the real-time and robustness of 3D reconstruction and broaden its scope of application.

[0092] Preferably, the method uses points in the point cloud data as input and adopts a Delaunay triangulation algorithm to construct a triangular mesh between the point clouds, specifically including:

[0093] Project the point cloud data onto a two-dimensional plane;

[0094] For each point in the point cloud, find its neighboring points according to the preset distance. These neighboring points will be used as candidate points for constructing the triangle mesh.

[0095] Select three points from the point cloud as the vertices of the initial triangle, and then gradually add more points to construct more triangles according to the Delaunay criterion;

[0096] After all triangles are constructed, they are optimized and merged to obtain the final triangle mesh.

[0097] In this embodiment, first, the point cloud data is projected onto a two-dimensional plane. This step helps to reduce the computational complexity while retaining the spatial relationship of the point cloud, providing a basis for subsequent three-dimensional mesh construction. The neighboring points of each point are found as candidate points by presetting the distance, which helps to ensure that the constructed triangular mesh can fit closely to the original point cloud data, thereby improving the accuracy of the three-dimensional model. The Delaunay triangulation algorithm is selected based on its ability to generate the optimal triangular mesh, that is, to avoid the appearance of narrow and long triangles as much as possible, thereby improving the uniformity and stability of the mesh. Three points are selected from the point cloud as the vertices of the initial triangle, and more points are gradually added to construct more triangles in accordance with the Delaunay criterion (empty circumscribed circle criterion). This process ensures that the constructed triangular mesh has good geometric properties and topological structure; after all triangles are constructed, the mesh is optimized and merged to eliminate possible redundancy and overlap, further improving the quality of the mesh. This step also adjusts the shape and size of the triangles to make the final triangular mesh smoother and more uniform, which is beneficial to subsequent 3D model rendering and texture mapping. The triangular mesh constructed through the above steps can fit the original point cloud data closely, reducing the error caused by improper mesh division, thereby improving the accuracy of the 3D model. The Delaunay triangulation algorithm ensures the uniformity of the triangular mesh, avoids the appearance of narrow and long triangles, and improves the overall quality of the mesh. The mesh optimization and merging process further enhances the stability of the mesh and reduces deformation or distortion caused by unreasonable mesh structure.

[0098] Preferably, the optimizing process of the point cloud data specifically includes:

[0099] The point cloud data is subjected to denoising, filtering and smoothing, down-sampling and outlier removal processing.

[0100] In this embodiment, random error points introduced by factors such as equipment noise and environmental interference in the point cloud data are removed. These noise points will interfere with the accuracy of the three-dimensional model. After denoising, the accuracy and clarity of the model can be significantly improved; the point cloud data is smoothed by a filtering algorithm to reduce local fluctuations in the point cloud due to measurement errors or irregular surface of the object. The point cloud data after smoothing is more uniform, which helps to build a smoother three-dimensional surface; the dense point cloud data is downsampled to reduce the number of points, but the basic form and characteristics of the point cloud are kept unchanged, which can not only reduce the amount of data for subsequent processing and improve the processing speed, but also avoid calculation and storage problems caused by excessively large data. At the same time, appropriate downsampling can also reduce the impact of noise and outliers on the three-dimensional model to a certain extent; outlier points that obviously deviate from the overall data distribution or do not conform to physical laws are eliminated. These outliers may be caused by measurement errors, data corruption or special environmental factors. Their existence will seriously affect the accuracy and authenticity of the three-dimensional model. By eliminating outliers, the reliability and credibility of the three-dimensional model can be further improved.

[0101] A three-dimensional reconstruction system applied to 3D vision, comprising:

[0102] An acquisition module, used to acquire two-dimensional image data for three-dimensional reconstruction, wherein the two-dimensional image data is captured by cameras from multiple viewing angles;

[0103] The feature detection and matching module is used to detect feature points in a two-dimensional image using a feature matching algorithm, and to match between multiple two-dimensional images to find the corresponding relationship between feature points in different images;

[0104] Point cloud data recovery module, used to recover point cloud data in three-dimensional space based on the correspondence between feature points and feature points using the principle of multi-view geometry;

[0105] A point cloud processing module, used for optimizing the point cloud data;

[0106] The construction module is used to construct the final 3D model based on the optimized point cloud data and combined with 3D reconstruction technology;

[0107] The visualization module is used to visualize the constructed three-dimensional model.

[0108] A computing device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned three-dimensional reconstruction method applied to 3D vision when executing the computer program.

[0109] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned three-dimensional reconstruction method applied to 3D vision.

[0110] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions or improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0111] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A three-dimensional reconstruction method applied to 3D vision, characterized in that: Specifically include: Acquire two-dimensional image data for three-dimensional reconstruction, wherein the two-dimensional image data is captured by cameras from multiple viewing angles; Use feature matching algorithms to detect feature points in two-dimensional images, and match them between multiple two-dimensional images to find the corresponding relationship between feature points in different images; Based on the correspondence between feature points, the point cloud data in three-dimensional space is restored using the principle of multi-view geometry; Optimizing the point cloud data; Based on the optimized point cloud data and combined with 3D reconstruction technology, the final 3D model is constructed; The constructed three-dimensional model is visualized.

2. A three-dimensional reconstruction method applied to 3D vision according to claim 1, characterized in that: The method of using a feature matching algorithm to detect feature points in a two-dimensional image and matching between multiple two-dimensional images to find the corresponding relationship between feature points in different images specifically includes: A scale-invariant feature transformation algorithm is applied to detect feature points in each 2D image; For each detected feature point, the scale-invariant feature transformation algorithm calculates the gradient direction and magnitude around it, and generates a feature descriptor with rotation and scale invariance based on the gradient direction and magnitude; Using the generated feature descriptors, match feature points between multiple two-dimensional images to find matching feature point pairs; The matched feature point pairs are screened and optimized. After screening and optimization, the remaining matching point pairs represent the projections of the same three-dimensional points in different images on the two-dimensional plane, thereby finding the correspondence between the feature points in different images.

3. The three-dimensional reconstruction method applied to 3D vision according to claim 2, characterized in that: The method of restoring point cloud data in three-dimensional space using the multi-view geometry principle based on the correspondence between feature points includes: Estimate the basic matrix based on the correspondence between feature points; Using the basic matrix and the matching feature point pairs, the camera motion parameters and the positions of the 3D points in the scene are restored; Triangulation is performed based on the camera's motion parameters and the positions of the three-dimensional points in the scene to calculate the coordinate pairs of the three-dimensional points.

4. The three-dimensional reconstruction method applied to 3D vision according to claim 3, characterized in that: The estimating the basic matrix according to the correspondence between the feature points specifically includes: According to the correspondence between feature points, eight non-collinear matching point pairs are selected using the eight-point method; Using the coordinate information of eight non-collinear matching point pairs, a linear equation system is constructed; Solve the system of linear equations to obtain the fundamental matrix.

5. The three-dimensional reconstruction method applied to 3D vision according to claim 4, characterized in that: The optimized point cloud data is combined with 3D reconstruction technology to construct the final 3D model, which specifically includes: Using the points in the point cloud data as input, the Delaunay triangulation algorithm is used to construct triangular meshes between the point clouds, and the constructed triangular meshes are optimized; Acquire a texture image corresponding to the triangular mesh; The collected texture image is mapped onto the triangular mesh through texture mapping technology to obtain the final three-dimensional model.

6. The three-dimensional reconstruction method applied to 3D vision according to claim 5, characterized in that: The method uses points in the point cloud data as input and adopts the Delaunay triangulation algorithm to construct a triangular mesh between the point clouds, specifically including: Project the point cloud data onto a two-dimensional plane; For each point in the point cloud, find its neighboring points according to the preset distance. These neighboring points will be used as candidate points for constructing the triangle mesh. Select three points from the point cloud as the vertices of the initial triangle, and then gradually add more points to construct more triangles according to the Delaunay criterion; After all triangles are constructed, they are optimized and merged to obtain the final triangle mesh.

7. The three-dimensional reconstruction method applied to 3D vision according to claim 6, characterized in that: The optimizing process of the point cloud data specifically includes: The point cloud data is subjected to denoising, filtering and smoothing, down-sampling and outlier removal processing.

8. A three-dimensional reconstruction device for 3D vision, characterized in that: include: An acquisition module, used to acquire two-dimensional image data for three-dimensional reconstruction, wherein the two-dimensional image data is captured by cameras from multiple viewing angles; The feature detection and matching module is used to detect feature points in a two-dimensional image using a feature matching algorithm, and to match between multiple two-dimensional images to find the corresponding relationship between feature points in different images; Point cloud data recovery module, used to recover point cloud data in three-dimensional space based on the correspondence between feature points and feature points using the principle of multi-view geometry; A point cloud processing module, used for optimizing the point cloud data; The construction module is used to construct the final 3D model based on the optimized point cloud data and combined with 3D reconstruction technology; The visualization module is used to visualize the constructed three-dimensional model.

9. A computing device, characterized in that The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the three-dimensional reconstruction method applied to 3D vision as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the steps of the three-dimensional reconstruction method applied to 3D vision as described in any one of claims 1 to 7.

Images