A three-dimensional reconstruction system, method and terminal device

Abstract

The present invention discloses a three-dimensional reconstruction system, method and terminal device. The system includes a first image acquisition device, a second image acquisition device, a plurality of supporting columns and a terminal device. The plurality of supporting columns are used to support an object to be reconstructed. The first image acquisition device and the second image acquisition device are located on both sides of the object to be reconstructed. The first image acquisition device is used to obtain a first image information set of multiple different perspectives on one side of the object to be reconstructed. The second image acquisition device is used to obtain a second image information set of multiple different perspectives on the other side of the object to be reconstructed. The terminal device is used to establish a first point cloud model according to the first image information set; establish a second point cloud model according to the second image information set; and obtain a point cloud model of the object to be reconstructed according to the first point cloud model and the second point cloud model.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a three-dimensional reconstruction system, method and terminal equipment. Background Art

[0002] Particle geometry refers to the particle size, shape, color and other characteristics of the particles. It is an important microscopic parameter that affects the mechanical properties of particles and has received increasing research and attention in recent years. Rock particles are widely used in the construction of dams, railways, airports and other projects in western my country due to their abundant natural reserves and excellent engineering properties. The engineering macroscopic mechanical properties of rockfill structures, cemented gravel structures or coarse aggregate concrete structures formed by direct accumulation or cementation of rock particles are closely related to the microscopic geometry of the particles themselves. Previous research results have shown that particle geometry has a great influence on the compression characteristics, shear characteristics, dynamic characteristics, permeability characteristics, and crushing characteristics of granular materials. How to achieve accurate three-dimensional reconstruction of particle geometry and analyze its geometric morphology characteristics is the key to exploring the influence of particle shape on the macroscopic mechanical behavior of the engineering body it constitutes. It is currently a research hotspot and important challenge in the engineering field.

[0003] The existing methods for obtaining the geometric morphology of rock particles mainly include static image analysis, dynamic image analysis, computer tomography and three-dimensional laser scanning.

[0004] Static image analysis calculates morphological indicators by obtaining two-dimensional images of particles, and this method cannot obtain three-dimensional morphological indicators; dynamic image analysis takes two-dimensional images of particles from multiple perspectives during their movement, and takes the average value of the two-dimensional morphological indicators calculated using the obtained multi-perspective two-dimensional images to obtain the three-dimensional morphological indicators. The three-dimensional morphological indicators obtained by this method are only approximate values ​​and cannot reproduce the precise three-dimensional morphology of the particles; computer tomography and three-dimensional laser scanning can obtain accurate three-dimensional morphological indicators of particles, but these two technologies require manual flipping of the particles to obtain the completed three-dimensional reconstructed morphology when obtaining the three-dimensional model of the particles, and cannot include the color information of the particles. The color of the particles is an important reference for distinguishing the lithology of the particles and has an important influence on the mechanical properties such as the strength of the particles. In addition, these two technologies can only be used indoors and cannot be used outdoors, especially on engineering sites, which greatly limits their promotion in the engineering field. Summary of the invention

[0005] Therefore, in order to solve the deficiencies of the prior art, embodiments of the present invention provide a three-dimensional reconstruction system, method and terminal device.

[0006] According to a first aspect, an embodiment of the present invention discloses a three-dimensional reconstruction system, the system comprising a first image acquisition device, a second image acquisition device, a plurality of support columns and a terminal device, the plurality of support columns are used to support an object to be reconstructed, the first image acquisition device and the second image acquisition device are located on both sides of the object to be reconstructed;

[0007] A first image acquisition device, used to acquire a first image information set of multiple different viewing angles on one side of the object to be reconstructed;

[0008] A second image acquisition device, used to acquire a second image information set of multiple different viewing angles on the other side of the object to be reconstructed;

[0009] The terminal device is used to establish a first point cloud model according to a first image information set; establish a second point cloud model according to a second image information set; and obtain a point cloud model of an object to be reconstructed according to the first point cloud model and the second point cloud model.

[0010] Optionally, when executing the step of establishing the first point cloud model according to the first image information set, the terminal device is further configured to:

[0011] Extracting a feature point set of each image information in the first image information set;

[0012] Performing dimensionality reduction processing on the feature point set of the first image information to obtain a feature point set of the first image information after dimensionality reduction, wherein the first image information is any image information in the first image information set;

[0013] A first point cloud model is obtained based on each feature point in the feature point set after the dimension reduction of the first image information and a preset algorithm.

[0014] Optionally, when the terminal device performs the step of performing dimensionality reduction processing on the feature point set of the first image information to obtain the feature point set after dimensionality reduction of the first image information, the terminal device is further configured to:

[0015] According to the first feature point, a preset number of feature points are selected from the feature point set of the first image information to obtain a first feature point subset, where the first feature point is any one of the feature point set of the first image information;

[0016] Constructing an undirected graph of the first feature points according to the first feature point set and the first feature point subset;

[0017] Determine the weight between any two feature points in the undirected graph of the first feature point according to the connection relationship between each feature point in the undirected graph of the first feature point;

[0018] Determine a weight matrix corresponding to the undirected graph of the first feature point according to the weight between any two feature points in the undirected graph of the first feature point;

[0019] According to the weight matrix corresponding to the undirected graph of each feature point and the preset objective function, a feature point set after dimensionality reduction of the first image information is obtained.

[0020] Optionally, when executing the step of establishing a second point cloud model according to the second image information set, the terminal device is further configured to:

[0021] Extracting a feature point set of each image information in the second image information set;

[0022] Performing dimensionality reduction processing on the feature point set of the second image information to obtain a feature point set of the second image information after dimensionality reduction, wherein the second image information is any image information in the second image information set;

[0023] A second point cloud model is obtained based on each feature point in the feature point set after the dimension reduction of the second image information and a preset algorithm.

[0024] Optionally, when the terminal device performs the step of performing dimensionality reduction processing on the feature point set of the second image information to obtain the feature point set after dimensionality reduction of the second image information, the terminal device is further used to:

[0025] According to the second feature point, a preset number of feature points are selected from the feature point set of the second image information to obtain a second feature point subset, where the second feature point is any one of the feature point set of the second image information;

[0026] Constructing an undirected graph of the second feature points according to the second feature point set and the second feature point subset;

[0027] Determine the weight between any two feature points in the undirected graph of the second feature point according to the connection relationship between each feature point in the undirected graph of the second feature point;

[0028] Determine a weight matrix corresponding to the undirected graph of the second feature point according to the weight between any two feature points in the undirected graph of the second feature point;

[0029] According to the weight matrix corresponding to the undirected graph of each feature point and the preset objective function, a feature point set after dimensionality reduction of the second image information is obtained.

[0030] Optionally, the terminal device is further configured to:

[0031] Acquiring position information of at least two preset positions on the surface of the object to be reconstructed;

[0032] A point cloud model of the object to be reconstructed is obtained according to the position information of at least two preset positions, the first point cloud model and the second point cloud model.

[0033] Optionally, the terminal device is further configured to:

[0034] Acquire position information corresponding to a plurality of positions on the surface of the object to be reconstructed except for at least two preset positions;

[0035] Verifying the accuracy of the point cloud model according to position information corresponding to a plurality of other positions on the surface of the object to be reconstructed except for at least two preset positions;

[0036] If the accuracy is greater than the preset error, the point cloud model is rebuilt.

[0037] Optionally, the plurality of support posts are made of reflective material.

[0038] Optionally, each supporting column contacts the object to be reconstructed in a point manner.

[0039] Optionally, the system further comprises at least two laser emitters, and the at least two laser emitters emit lasers at different times;

[0040] A laser emitter, used for emitting laser light to the surface of the object to be reconstructed, so as to form an auxiliary mark on the surface of the object to be reconstructed;

[0041] The terminal device is also used to obtain a point cloud model of the object to be reconstructed based on the first point cloud model, the second point cloud model and the auxiliary markers.

[0042] Optionally, the system further comprises a leveling device and a turntable;

[0043] The leveling device is used to level the turntable;

[0044] The turntable is used to support the first image acquisition device and the second image acquisition device to rotate to acquire a first image information set and a second image information set at multiple different viewing angles.

[0045] According to a second aspect, an embodiment of the present invention further discloses a three-dimensional reconstruction method, which is executed by a terminal device in a three-dimensional reconstruction system in the first aspect or any optional embodiment of the first aspect, and the method includes:

[0046] Acquire a first image information set, where the first image information set is an image information set acquired by a first image acquisition device in the three-dimensional reconstruction system at multiple different viewing angles on one side of the object to be reconstructed and transmitted to a terminal device;

[0047] Acquire a second image information set, where the second image information set is an image information set acquired by a second image acquisition device in the three-dimensional reconstruction system from multiple different viewing angles on the other side of the object to be reconstructed and transmitted to the terminal device;

[0048] Establishing a first point cloud model according to the first image information set;

[0049] Establishing a second point cloud model according to the second image information set;

[0050] A point cloud model of the object to be reconstructed is obtained according to the first point cloud model and the second point cloud model.

[0051] According to the third aspect, an embodiment of the present invention also discloses a terminal device, comprising: at least one terminal device; and a memory communicatively connected to the at least one terminal device; wherein the memory stores instructions executable by the at least one terminal device, and the instructions are executed by the at least one terminal device so that the at least one terminal device performs the steps of the three-dimensional reconstruction method as in the first aspect or any optional embodiment of the first aspect.

[0052] According to the fourth aspect, an embodiment of the present invention further discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a terminal device, implements the steps of the three-dimensional reconstruction method of the first aspect or any optional embodiment of the first aspect.

[0053] The technical solution of the present invention has the following advantages:

[0054] The present invention provides a three-dimensional reconstruction system, method and terminal device, the system includes a first image acquisition device, a second image acquisition device, a plurality of support columns and a terminal device, wherein the plurality of support columns are used to support an object to be reconstructed, so that the first image acquisition device and the second image acquisition device can directly obtain image information of the object to be reconstructed, the first image acquisition device and the second image acquisition device are located on both sides of the object to be reconstructed, and can obtain image information of each angle of the object to be reconstructed; the first image acquisition device is used to obtain a first image information set of multiple different viewing angles of the object to be reconstructed on one side, and the second image acquisition device is used to obtain a second image information set of multiple different viewing angles of the object to be reconstructed on the other side. After obtaining the first image information set and the second image information set, the terminal device establishes a first point cloud model according to the first image information set, establishes a second point cloud model according to the second image information set, obtains a point cloud model of the object to be reconstructed according to the first point cloud model and the second point cloud model, and constructs a complete three-dimensional model of the object to be reconstructed with features such as color, and because the material to be reconstructed is supported by the plurality of support columns, a complete three-dimensional point cloud model can be constructed without turning over during the reconstruction process, which greatly improves the efficiency of model generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1is a schematic diagram of a specific example of a three-dimensional reconstruction system in an embodiment of the present invention;

[0057] Figure 2 is a schematic diagram of a specific example of a three-dimensional reconstruction system in an embodiment of the present invention;

[0058] Figure 3 is a schematic diagram of a specific example of a three-dimensional reconstruction system in an embodiment of the present invention;

[0059] Figure 4 is a flowchart of a specific example of a three-dimensional reconstruction method in an embodiment of the present invention;

[0060] Figure 5 FIG. 4 is a specific example diagram of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0063] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, it can also be the internal connection of two components, it can be a wireless connection, or it can be a wired connection. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0064] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0065] In view of the technical problems mentioned in the background technology, the present application embodiment provides a three-dimensional reconstruction system. Figure 1 As shown, the system includes a first image acquisition device 101, a second image acquisition device 102, a plurality of support columns 103 and a terminal device (not shown in the figure), the plurality of support columns 103 are used to support the object to be reconstructed, and the first image acquisition device 101 and the second image acquisition device 102 are located on both sides of the object to be reconstructed;

[0066] The first image acquisition device 101 is used to obtain a first image information set of multiple different viewing angles on one side of the object to be reconstructed.

[0067] The second image acquisition device 102 is used to obtain a second image information set of multiple different viewing angles on the other side of the object to be reconstructed.

[0068] The terminal device is used to establish a first point cloud model according to a first image information set; establish a second point cloud model according to a second image information set; and obtain a point cloud model of an object to be reconstructed according to the first point cloud model and the second point cloud model.

[0069] Exemplarily, the 3D reconstruction system of the present invention is applicable to the reconstruction of the 3D model of the object to be reconstructed. In the embodiment of the present invention, the object to be reconstructed is introduced by taking rock particles as an example. The first image acquisition device 101 and the second image acquisition device 102 in the embodiment of the present application can be a device with a shooting function such as a camera, specifically a GoPro sports camera, model: HERO 9Black; GoPro, USA, with a maximum resolution of 5120×2880 (5k), corresponding to a frame rate of 30fps; a maximum frame rate of 240fps, corresponding to a resolution of 1920×1080 (1080p).

[0070] The embodiment of the present invention does not limit the number of the plurality of supporting columns 103 as long as the object to be reconstructed can be supported. Of course, under the premise of being able to stably support the object to be reconstructed, the fewer the number of supporting columns 103, the better.

[0071] like Figure 1As shown, the first image acquisition device 101 and the second image acquisition device 102 respectively obtain the first image information set and the second image information set of multiple viewpoints of the object to be reconstructed in the upper and lower directions. Here, the provisions for different viewpoints can be adaptively adjusted according to the actual reconstruction requirements, as long as the consistency of the reconstructed three-dimensional model and the object to be reconstructed is ensured. For example, when actually obtaining the image information set, the image information of the upper and lower sides of the object to be reconstructed can be obtained as the image information set. Specifically, after obtaining the video of the object to be reconstructed in the upper and lower directions, the frame image is extracted at a fixed time interval to obtain the multi-view image of the particle. It should be noted that the extracted adjacent images should overlap with each other and the overlap rate should not be less than 50%. To meet this requirement, the camera frame rate, image extraction time interval, and shooting speed should be coordinated with each other. The resolution selected in this study is 5120×2880 (5k), the frame rate is 30fps, the time interval for extracting frames is 0.8s, and the single-side shooting of each particle takes about 1 minute, and 90 to 130 particle images of different viewpoints can be obtained.

[0072] After acquiring the image information set, the terminal device can respectively establish the corresponding first point cloud model and the second point cloud model according to the first image information set and the second image information set, and splice the first point cloud model and the second point cloud model to obtain a complete point cloud model of the object to be reconstructed. Specifically, after respectively establishing dense point cloud models for the two sets of image information of the upper and lower halves of the object to be reconstructed, the two models are input into the CloudCompare software, and the ICP (Iterative Closest Point) point cloud splicing algorithm is used for registration, so as to realize the splicing of the upper and lower part point clouds of the object to be reconstructed, and obtain a complete three-dimensional point cloud model of the object to be reconstructed.

[0073] In a preferred embodiment, the process of obtaining a point cloud model according to the acquired image information set may be:

[0074] Step a1, extracting a feature point set of each image information in the first image information set.

[0075] Step a2, performing dimensionality reduction processing on the feature point set of the first image information to obtain a feature point set after dimensionality reduction of the first image information, wherein the first image information is any image information in the first image information set.

[0076] Step a3, obtaining a first point cloud model according to each feature point in the feature point set after the dimensionality reduction of the first image information and a preset algorithm.

[0077] For example, the feature point set extracted from the image information can use the SURF (Speeded Up Robust Features, accelerated robust features, referred to as SURF) algorithm to extract features from the image information. Next, the feature points can be processed by dimensionality reduction, which can reduce the complexity of calculations, improve the efficiency of feature matching, and thus improve the efficiency of three-dimensional reconstruction. Finally, a point cloud model is constructed for the feature points after dimensionality reduction to achieve three-dimensional reconstruction of the object to be reconstructed.

[0078] Specifically, the process of building a point cloud model is as follows:

[0079] Step 1: Use the nearest neighbor matching method based on KD tree to match feature points.

[0080] (1) Construct a KD tree: Construct all feature points after dimensionality reduction into a KD tree so that the nearest neighbor of each point can be quickly found during the search process. The KD tree is a binary tree data structure. Each node represents a sample point. By recursively partitioning the data space, a tree structure is constructed so that each leaf node corresponds to a sample point. In the KD tree, each node corresponds to a feature point. According to the feature value represented by the node, the data set is divided into two subsets. The left subtree contains sample points whose feature values ​​are less than the node feature value, and the right subtree contains sample points whose feature values ​​are greater than the node feature value.

[0081] (2) Search for the nearest neighbor: For each feature point, search for the point closest to it in the KD tree. For a given query point, start searching from the root node downwards, determine the search direction based on the distance between the node's feature value and the query point, and finally find the leaf node closest to the query point.

[0082] Then, we start from the leaf node and trace back upwards to calculate the distance between the parent node and the query point. If the distance is smaller than the distance between the current nearest neighbor and the query point, we update the nearest neighbor. We continue to trace back upwards until we reach the root node, and finally get the nearest neighbor.

[0083] (3) Selecting a distance measurement method: When searching for the nearest neighbor point, it is necessary to calculate the distance between feature points. Here, it is necessary to select a suitable distance measurement method. The present invention adopts the Euclidean distance based on the L2 norm because the Euclidean distance can well reflect the similarity between feature vectors.

[0084] (4) Threshold processing: The nearest neighbor points searched may have mismatches, so a threshold needs to be set to determine whether it is a valid match. Generally, the distance of the nearest neighbor point can be compared with a pre-set threshold. If it is less than the threshold, it is considered a valid match.

[0085] Step 2: Use SFM and MVS to create a dense point cloud.

[0086] (1) For the points successfully matched in the first step, the basic matrix F is solved using the normalized 8-point algorithm, and the essential matrix E is solved using the 5-point method.

[0087] (2) Perform SVD decomposition on the obtained essential matrix E to recover the relative transformation matrices R and T of the motion state.

[0088] (3) Based on the calculation principle of the triangle method, the projection matrix is ​​calculated using the basic matrix F obtained in (1) and the relative transformation matrices R and T obtained in (2). The two-dimensional pixel points are mapped to the three-dimensional space through the projection matrix.

[0089] (4) The projection matrix is ​​iteratively optimized using feature matching points. After convergence, the final projection matrix and the three-dimensional spatial coordinates of the feature points are obtained to obtain a sparse point cloud model.

[0090] (5) Based on the obtained sparse point cloud model, the MVS method is used to generate a dense point cloud with real colors through matching, diffusion and filtering.

[0091] In another preferred embodiment, step a2 may be implemented in the following manner: a preset number of feature points are screened from a feature point set of the first image information according to the first feature point to obtain a first feature point subset, wherein the first feature point is any one of the feature point set of the first image information;

[0092] Step b1, constructing an undirected graph of the first feature points according to the first feature point set and the first feature point subset.

[0093] For example, each data point x in the first feature point set X i As a vertex of the undirected graph, take x according to the k-nearest neighbor method i The first k closest points are x i The k-nearest neighbor method sets the number of neighboring points of each data point to k in advance. j Belong to x i The first k closest points of , then we call x i With x j is the neighboring point. According to each neighboring point (the first feature point subset) and x i (First feature point) Construct an undirected graph.

[0094] Step b2, determining the weight between any two feature points in the undirected graph of the first feature points according to the connection relationship between each feature point in the undirected graph of the first feature points.

[0095] For example, for the neighbor graph of each data point constructed in step b1, two points x connected by an edge are i With x jThe weight W between ij Set to 1, the weight W between two points without edges connected ij Set to 0, so that the first feature point x can be obtained i The corresponding weight sparse matrix W m×n .

[0096] Step b3: determining a weight matrix corresponding to the undirected graph of the first feature point according to the weights between any two feature points in the undirected graph of the first feature point.

[0097] For example, since the neighboring points corresponding to each feature point may not include all feature points, there are two situations when determining the weight matrix. The first situation: the number of neighboring points of the first feature point set includes all feature points in the first feature point set, so the weight matrix can be obtained according to the undirected graph of the first feature point; the second situation: the number of neighboring points of the first feature point set cannot include all feature points in the first feature point set, so the undirected graph according to the first feature point cannot cover all feature points, so multiple undirected graphs are required to determine the weight matrix.

[0098] Step b4, obtaining a feature point set after dimensionality reduction of the first image information according to the weight matrix corresponding to the undirected graph of each feature point and a preset objective function.

[0099] Exemplarily, the preset objective function may be set as:

[0100]

[0101] Among them, yi and yj are two different feature points in the feature point set, W ij That is, the weight of the corresponding feature point in the weight matrix in step b3.

[0102] After transforming formula (1), we get:

[0103]

[0104] Where: A is the projection matrix, X is the first feature point set, L = DW is the Laplace matrix; D is the diagonal matrix of the weight matrix, and its diagonal elements are That is to satisfy:

[0105]

[0106] Add constraints as follows:

[0107] Y T DY=1→A T XD T A=1 (4)

[0108] Combining equations (2) and (4), the objective function is transformed into:

[0109] min A T XL T Ast A T XD T A=1 (5)

[0110] To solve the transformation projection matrix A, we can solve the generalized eigenvector of equation (3.2.10), that is:

[0111] XL T A=λXDX T A (6)

[0112] Let a1, a2, …, a d is the eigenvector of formula (6), and it is arranged from small to large according to the corresponding eigenvalue size, which is the required projection matrix A. Then the low-dimensional embedding Y obtained from the high-dimensional first feature point set X can be expressed as:

[0113] x i →y i =A T x i (7)

[0114] The first feature set Y after dimensionality reduction is obtained by formula (7).

[0115] Before feature matching, it is usually necessary to reduce the dimension of feature points from high-dimensional space to low-dimensional space, which helps to reduce computational complexity and improve the efficiency of feature matching. Manifold learning is a commonly used dimensionality reduction method that can map high-dimensional data to low-dimensional space while retaining the local structure and geometric features of the data.

[0116] The processing of the second image information set and the process of constructing the point cloud model are the same as above and will not be repeated here.

[0117] On the basis of the above embodiments, in order to make the point cloud model of the object to be reconstructed consistent with the actual size parameters of the object to be reconstructed, the terminal device of the three-dimensional reconstruction system of the present invention is also used to: obtain the position information of at least two preset positions on the surface of the object to be reconstructed; obtain the point cloud model of the object to be reconstructed according to the position information of at least two preset positions, the first point cloud model and the second point cloud model.

[0118] Exemplarily, to ensure that the point cloud model is a one-to-one restoration of the object to be reconstructed, the position information of at least two positions on the surface of the object to be reconstructed can be calibrated when acquiring the image information set. In this way, when constructing the point cloud model, the corresponding position information can be brought into the point cloud model to obtain a point cloud model that corresponds one-to-one to the object to be reconstructed.

[0119] To further determine the accuracy of the point cloud model, preferably, the terminal device is also used to obtain position information corresponding to multiple other positions on the surface of the object to be reconstructed except at least two preset positions; verify the accuracy of the point cloud model based on the position information corresponding to multiple other positions on the surface of the object to be reconstructed except at least two preset positions; if the accuracy is greater than the preset error, reconstruct the point cloud model.

[0120] Exemplarily, in order to determine the accuracy of the point cloud model, the position information of at least two positions may be determined in addition to the at least two positions mentioned above to verify the constructed point cloud model.

[0121] Specifically, before obtaining the image information set of the object to be reconstructed, two sets of marker points are pasted on the upper and lower halves of the object to be reconstructed, with two in each set. The point cloud model is converted to the real size using the straight-line distance of one set of marker points. At the same time, the conversion error is checked using the other set of marker points. If the error exceeds 5%, it means that the modeling has failed and needs to be remodeled.

[0122] In the process of reconstructing the three-dimensional model of the object to be reconstructed, the texture of the object to be reconstructed needs to be restored. Therefore, setting the support column 103 as a reflective material can reduce the impact in the reconstruction process of the three-dimensional model and improve the accuracy of the model.

[0123] Since the support pillars 103 will contact the surface of the object to be reconstructed while supporting the object to be reconstructed, the contacting places will be blocked when collecting image information. Therefore, in order to further improve the accuracy of the model, each support pillar 103 contacts the object to be reconstructed in a point manner. Specifically, each support pillar 103 can be set as a pointed head to reduce the contact with the object to be reconstructed.

[0124] When the first point cloud model and the second point cloud model are spliced, auxiliary marks are required for distinguishing them, so the system further includes at least two laser emitters, and the at least two laser emitters emit lasers at different times;

[0125] A laser emitter, used for emitting laser light to the surface of the object to be reconstructed, so as to form an auxiliary mark on the surface of the object to be reconstructed;

[0126] The terminal device is also used to obtain a point cloud model of the object to be reconstructed based on the first point cloud model, the second point cloud model and the auxiliary markers.

[0127] Exemplarily, a line laser emitter can be arranged in the middle of the brackets of the first image acquisition device 101 and the second image acquisition device 102, and can emit laser lines with a certain width. The laser emitter has a red laser emitter and a green laser emitter arranged side by side. When the turntable rotates to the first half circle, the red laser emitter works, and the object to be measured has a red laser line. When the turntable rotates to the second half circle, the green laser emitter works, and the object to be measured has a green laser line. When the turntable rotates one circle, the middle part of the finally established particle point cloud model is a red and green dividing line. This dividing line will be used for splicing the point cloud models established by the upper and lower image acquisition devices; the red and green pattern enables the splicing algorithm to quickly identify the RGB information at the dividing line. Compared with the traditional method of splicing using the texture of the particle surface, adding two-color auxiliary lines can greatly improve the splicing efficiency and accuracy.

[0128] In order to make it more convenient for the first image acquisition device 101 and the second image acquisition device 102 to obtain multiple image information sets from different perspectives, the system also includes a horizontal device and a turntable; the horizontal device is used to level the turntable; the turntable is used to support the first image acquisition device 101 and the second image acquisition device 102 to rotate to obtain the first image information sets and the second image information sets from multiple different perspectives.

[0129] Corresponding to the above embodiment, a specific example is used for introduction. Figure 2 and Figure 3As shown, the bottom of the system is a tripod (triangular bracket), the top of the tripod is a leveling knob, and the turntable is equipped with a level (horizontal device). At the operation site, the tripod can be set up first, and then the turntable can be adjusted to a horizontal level using the leveling knob and the level. Four top pointed support columns 103 are arranged in the middle of the turntable. The positions of these four columns are fixed and do not rotate with the turntable. The tops of the columns are pointed in order to reduce the contact area between the support columns 103 and the object to be measured. The columns are made of reflective stainless steel, so that The purpose of this is to reduce the interference to the camera during 3D reconstruction. This design allows the second camera (second image acquisition device 102) to capture most of the image information of the lower half of the object being measured, greatly reducing the occlusion area of ​​the support platform on the object being measured. In this way, by arranging two cameras, point cloud information of the upper and lower halves can be obtained at the same time, without the need to flip the particles for a second shot according to the traditional method, thereby improving work efficiency; the turntable is equipped with a camera bracket (mechanical arm) that can carry two Gopro sports cameras The machine, as the turntable rotates, the two cameras work simultaneously to obtain the video of the upper and lower surfaces of the measured particles at the same time, and use them for modeling to finally obtain the point cloud model; the camera is connected to the bracket by means of a slide rail and a rotating shaft to ensure that the position and rotation angle of the camera can be adjusted; a line laser emitter is arranged in the middle of the camera bracket, which can emit laser lines with a certain width. The line laser emitter has a red laser emitter and a green laser emitter, which are arranged side by side. When the turntable rotates to the first half of the circle, the red laser emitter works, and the object to be measured has a red laser line. When the turntable rotates to the second half of the circle, the green laser emitter works, and the object to be measured has a green laser line. When the turntable rotates one circle, the middle part of the particle point cloud model finally established is a red and green dividing line, which will be used for the splicing of the point cloud models established by the upper and lower cameras; the red and green mode enables the splicing algorithm to quickly identify the RGB information at the dividing line. Compared with the traditional method of splicing using the texture of the particle surface, adding two-color auxiliary lines can greatly improve the splicing efficiency and accuracy.

[0130] By running this system, the system includes a first image acquisition device 101, a second image acquisition device 102, a plurality of support columns 103 and a terminal device, wherein the plurality of support columns 103 are used to support the object to be reconstructed, so that the first image acquisition device 101 and the second image acquisition device 102 can directly obtain image information of the object to be reconstructed, and the first image acquisition device 101 and the second image acquisition device 102 are located on both sides of the object to be reconstructed, and can obtain image information of each angle of the object to be reconstructed; the first image acquisition device 101 is used to obtain a first image information set of multiple different perspectives of the object to be reconstructed on one side, and the second image acquisition device 101 is used to obtain a first image information set of multiple different perspectives of the object to be reconstructed on one side, and the second image acquisition device 102 ... The acquisition device 102 is used to obtain a second image information set of multiple different perspectives on the other side of the object to be reconstructed. After obtaining the first image information set and the second image information set, the terminal device establishes a first point cloud model according to the first image information set, and establishes a second point cloud model according to the second image information set. Based on the first point cloud model and the second point cloud model, a point cloud model of the object to be reconstructed is obtained, and a complete three-dimensional model of the object to be reconstructed with features such as color is constructed. Since the material to be reconstructed is supported by multiple supporting columns 103, the complete three-dimensional point cloud model can be constructed without turning it over during the reconstruction process, which greatly improves the efficiency of model generation.

[0131] The above are embodiments of the three-dimensional reconstruction system provided by the present application. Other embodiments of the three-dimensional reconstruction system provided by the present application are introduced below. Please refer to the following for details.

[0132] The embodiment of the present invention also discloses a three-dimensional reconstruction method, such as Figure 4 As shown, the method is executed by a terminal device in the above three-dimensional reconstruction system, and the method includes:

[0133] Step 401, obtaining a first image information set, where the first image information set is an image information set acquired by a first image acquisition device 101 in a 3D reconstruction system at multiple different viewing angles on one side of an object to be reconstructed and transmitted to a terminal device;

[0134] Step 402, obtaining a second image information set, where the second image information set is an image information set acquired by a second image acquisition device 102 in the 3D reconstruction system from multiple different viewing angles of the object to be reconstructed on the other side and transmitted to the terminal device;

[0135] Step 403, establishing a first point cloud model according to the first image information set;

[0136] Step 404, establishing a second point cloud model according to the second image information set;

[0137] Step 405: Obtain a point cloud model of the object to be reconstructed according to the first point cloud model and the second point cloud model.

[0138] As an optional embodiment of the present invention, the method further includes:

[0139] Acquiring position information of at least two preset positions on the surface of the object to be reconstructed;

[0140] A point cloud model of the object to be reconstructed is obtained according to the position information of at least two preset positions, the first point cloud model and the second point cloud model.

[0141] As an optional embodiment of the present invention, the method further includes:

[0142] Acquire position information corresponding to a plurality of positions on the surface of the object to be reconstructed except for at least two preset positions;

[0143] Verifying the accuracy of the point cloud model according to position information corresponding to a plurality of other positions on the surface of the object to be reconstructed except for at least two preset positions;

[0144] If the accuracy is greater than the preset error, the point cloud model is rebuilt.

[0145] The methods executed in each step of the three-dimensional reconstruction device provided in the embodiments of the present invention have been described in detail in any of the above method embodiments, and thus will not be described again here.

[0146] By executing this method, a first image information set of multiple different perspectives of the object to be reconstructed on one side is obtained, and a second image information set of multiple different perspectives of the object to be reconstructed on the other side is obtained. After obtaining the first image information set and the second image information set, a first point cloud model is established according to the first image information set, and a second point cloud model is established according to the second image information set. Based on the first point cloud model and the second point cloud model, a point cloud model of the object to be reconstructed is obtained, and a complete three-dimensional model of the object to be reconstructed with features such as color is constructed. Since the material to be reconstructed is supported by multiple supporting columns 103, the complete three-dimensional point cloud model can be constructed without turning over during the reconstruction process, which greatly improves the efficiency of model generation.

[0147] The embodiment of the present invention also provides a terminal device, such as Figure 5 As shown, the terminal device may include a processor 501 and a memory 502, wherein the processor 501 and the memory 502 may be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0148] The processor 501 may be a central processing unit (CPU). The processor 501 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0149] The memory 502 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the three-dimensional reconstruction method in the embodiment of the present invention. The processor 501 executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory 502, that is, implementing the three-dimensional reconstruction method in the above method embodiment.

[0150] The memory 502 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created by the processor 501, etc. In addition, the memory 502 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 502 may optionally include a memory remotely arranged relative to the processor 501, and these remote memories may be connected to the processor 501 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0151] One or more modules are stored in the memory 502, and when executed by the processor 501, the execution is as follows: Figure 1 The three-dimensional reconstruction method in the illustrated embodiment.

[0152] For details of the above computer equipment, please refer to Figure 1 The corresponding related descriptions and effects in the illustrated embodiments can be understood and will not be repeated here.

[0153] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0154] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A three-dimensional reconstruction system, characterized in that: The system comprises a first image acquisition device, a second image acquisition device, a plurality of supporting columns and a terminal device, wherein the plurality of supporting columns are used to support the object to be reconstructed, the first image acquisition device and the second image acquisition device are located on both sides of the object to be reconstructed, and the first image acquisition device and the second image acquisition device respectively acquire a first image information set and a second image information set of a plurality of viewing angles of the object to be reconstructed in upper and lower directions; The first image acquisition device is used to obtain a plurality of first image information sets of different viewing angles on one side of the object to be reconstructed, wherein adjacent images in the first image information set overlap with each other and the overlap rate is not less than 50%; The second image acquisition device is used to obtain a second image information set of multiple different viewing angles on the other side of the object to be reconstructed, wherein adjacent images in the second image information set overlap with each other and the overlap rate is not less than 50%; The terminal device is used to establish a first point cloud model according to the first image information set; establish a second point cloud model according to the second image information set; and obtain a point cloud model of the object to be reconstructed by splicing the first point cloud model and the second point cloud model; When constructing the point cloud model, the terminal device is also used to use the nearest neighbor matching method based on the KD tree to match feature points; and to use SFM and MVS to establish a dense point cloud; When executing the step of establishing the first point cloud model according to the first image information set, the terminal device is further used to: Extracting a feature point set of each first image information in the first image information set; Performing dimensionality reduction processing on the feature point set of the first image information to obtain a feature point set of the first image information after dimensionality reduction, wherein the first image information is any image information in the first image information set; Obtaining a first point cloud model according to each feature point in the feature point set after the dimensionality reduction of the first image information and a preset algorithm; When the terminal device performs the step of performing dimensionality reduction processing on the feature point set of the first image information to obtain the feature point set of the first image information after dimensionality reduction, the terminal device is further used to: According to the first feature point, a preset number of feature points are selected from the feature point set of the first image information to obtain a first feature point subset, wherein the first feature point is any one of the feature point set of the first image information; Constructing an undirected graph of first feature points according to the first feature point set and the first feature point subset; Determine a weight between any two feature points in the undirected graph of the first feature points according to a connection relationship between each feature point in the undirected graph of the first feature points; Determine a weight matrix corresponding to the undirected graph of the first feature points according to the weights between any two feature points in the undirected graph of the first feature points; Obtaining a set of feature points after dimensionality reduction of the first image information according to a weight matrix corresponding to an undirected graph of each feature point and a preset objective function; The system also includes a horizontal device and a turntable; the horizontal device is used to level the turntable; the turntable is used to support the first image acquisition device and the second image acquisition device to rotate to acquire the first image information set and the second image information set at multiple different viewing angles.

2. The system according to claim 1, characterized in that When executing the step of establishing a second point cloud model according to the second image information set, the terminal device is further used to: Extracting a feature point set of each second image information in the second image information set; Performing dimensionality reduction processing on the feature point set of the second image information to obtain a feature point set of the second image information after dimensionality reduction, wherein the second image information is any image information in the second image information set; A second point cloud model is obtained based on each feature point in the feature point set after the dimension reduction of the second image information and a preset algorithm.

3. The system according to claim 2, characterized in that When the terminal device performs the step of performing dimensionality reduction processing on the feature point set of the second image information to obtain the feature point set of the second image information after dimensionality reduction, the terminal device is further used to: According to the second feature point, a preset number of feature points are selected from the feature point set of the second image information to obtain a second feature point subset, wherein the second feature point is any one of the feature point set of the second image information; constructing an undirected graph of second feature points according to the second feature point set and the second feature point subset; Determine a weight between any two feature points in the undirected graph of the second feature points according to a connection relationship between each feature point in the undirected graph of the second feature points; Determine a weight matrix corresponding to the undirected graph of the second feature points according to the weights between any two feature points in the undirected graph of the second feature points; According to the weight matrix corresponding to the undirected graph of each feature point and the preset objective function, a feature point set after dimensionality reduction of the second image information is obtained.

4. The system according to claim 1, characterized in that The terminal device is also used to: Acquiring position information of at least two preset positions on the surface of the object to be reconstructed; A point cloud model of the object to be reconstructed is obtained according to the position information of the at least two preset positions, the first point cloud model and the second point cloud model.

5. The system according to claim 4, characterized in that The terminal device is also used for: Acquire position information corresponding to a plurality of positions on the surface of the object to be reconstructed except for at least two preset positions; Verifying the accuracy of the point cloud model according to position information corresponding to a plurality of other positions on the surface of the object to be reconstructed except for the at least two preset positions; If the accuracy is greater than a preset error, the point cloud model is reconstructed.

6. The system according to any one of claims 1 to 5, characterized in that: The plurality of support columns are made of reflective material.

7. The system according to claim 6, characterized in that Each of the supporting pillars contacts the object to be reconstructed in a point manner.

8. The system according to any one of claims 1 to 5, characterized in that: The system further comprises at least two laser emitters, wherein at least two of the laser emitters emit lasers at different times; The laser emitter is used to emit laser light to the surface of the object to be reconstructed, so as to form auxiliary marks on the surface of the object to be reconstructed; The terminal device is further used to obtain the point cloud model of the object to be reconstructed based on the first point cloud model, the second point cloud model and the auxiliary mark.

9. A three-dimensional reconstruction method, characterized in that: The method is executed by a terminal device in the three-dimensional reconstruction system according to any one of claims 1 to 8, and the method comprises: Acquire a first image information set, where the first image information set is an image information set acquired by a first image acquisition device in the three-dimensional reconstruction system at multiple different viewing angles on one side of the object to be reconstructed and transmitted to the terminal device, and adjacent images in the first image information set overlap with each other and the overlap rate is not less than 50%; Acquire a second image information set, where the second image information set is an image information set acquired by a second image acquisition device in the three-dimensional reconstruction system from multiple different viewing angles of the object to be reconstructed on the other side and transmitted to the terminal device, and adjacent images in the second image information set overlap with each other and the overlap rate is not less than 50%; Establishing a first point cloud model according to the first image information set; Establishing a second point cloud model according to the second image information set; splicing the first point cloud model and the second point cloud model to obtain a point cloud model of the object to be reconstructed; In the step of constructing the point cloud model, the nearest neighbor matching method based on KD tree is used for feature point matching; SFM and MVS are used to build a dense point cloud.

10. A terminal device, characterized in that: include: at least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the three-dimensional reconstruction method as described in claim 9.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a terminal device, the three-dimensional reconstruction method as claimed in claim 9 is implemented.

Images