Three-dimensional reconstruction method and laser scanner

Abstract

The invention discloses a three-dimensional reconstruction method and a laser scanner, and belongs to the field of laser scanning. The three-dimensional reconstruction method comprises the following steps: constructing three-dimensional point cloud data corresponding to a single image frame based on the image frame of a to-be-reconstructed object collected by a binocular camera in a laser scanner; the three-dimensional point cloud data comprises a plurality of reconstruction points and three-dimensional coordinates of the reconstruction points; the reconstruction points are space coordinate points obtained by performing three-dimensional reconstruction on pixel points in the image frame; and performing interpolation processing on at least part of reconstruction points in the three-dimensional point cloud data to obtain three-dimensional coordinates of a target interpolation point. According to the three-dimensional reconstruction method provided by the invention, the point cloud data far exceeding the resolution of the camera can be obtained, and the point output quantity of a single-frame image frame is improved, so that the scanning efficiency is improved; and the three-dimensional coordinates corresponding to the reconstructed target interpolation point accord with the surface change trend of the object to be reconstructed, so that the sampling precision and accuracy are relatively high, and the user experience is improved.

Description

Technical Field

[0001] The present application belongs to the field of laser scanning, and in particular to a three-dimensional reconstruction method and a laser scanner. Background Art

[0002] Binocular structured light 3D reconstruction is a technology that uses two cameras and a structured light projector to obtain the 3D shape and depth information of an object's surface. It is widely used in industrial inspection, intelligent manufacturing, autonomous driving, robot navigation, virtual reality, security monitoring and other fields.

[0003] In the related technology, binocular reconstruction is based on the points on the pixels of the image; that is, the number of points in a single frame is related to the image pixels, one pixel reconstructs a three-dimensional coordinate point, and then subsequent reconstruction and other operations are performed based on the reconstructed three-dimensional coordinate point; the reconstructed point cloud data distribution may include: 1) reconstructing a surface based on the reconstruction method of the entire surface radiation matching such as phase and speckle; 2) reconstructing a curve based on reconstruction methods such as laser lines and projection lines.

[0004] However, the output point quantity under the current reconstruction method is limited by the camera resolution, and the maximum reconstruction point quantity is fixed, resulting in the inability to increase the scanning speed, affecting the reconstruction effect and thus the user experience. Summary of the invention

[0005] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a three-dimensional reconstruction method and a laser scanner to increase the number of points of a single frame image, thereby improving the scanning efficiency.

[0006] In a first aspect, the present application provides a three-dimensional reconstruction method, the method comprising: Based on the image frames of the object to be reconstructed acquired by the binocular camera in the laser scanner, three-dimensional point cloud data corresponding to a single image frame is constructed; the three-dimensional point cloud data includes a plurality of reconstruction points and the three-dimensional coordinates of each of the reconstruction points; the reconstruction points are spatial coordinate points obtained by three-dimensionally reconstructing the pixel points in the image frame; Interpolation processing is performed on at least part of the reconstructed points in the three-dimensional point cloud data to obtain three-dimensional coordinates of target interpolation points.

[0007] According to the 3D reconstruction method of the present application, by interpolating the 3D point cloud data obtained based on a single-frame image frame, the target interpolation point and the 3D coordinates of the target interpolation point are obtained, so that the 3D coordinates of the new position on the object to be reconstructed other than the reconstruction point can be obtained, so that the output point amount is not limited by the resolution of the binocular camera, and point cloud data far exceeding the camera resolution is obtained, thereby increasing the output point amount of a single-frame image frame, thereby improving the scanning efficiency; and the 3D coordinates corresponding to the reconstructed target interpolation points are in line with the surface change trend of the object to be reconstructed, and have high sampling precision and accuracy, thereby improving the user experience.

[0008] According to an embodiment of the present application, the interpolation processing is performed on at least part of the reconstructed points in the three-dimensional point cloud data to obtain the three-dimensional coordinates of the target interpolation points, including: Fitting at least part of the reconstructed points in the three-dimensional point cloud data based on the three-dimensional point cloud data to obtain a fitting object; Interpolation processing is performed on at least part of the reconstructed points in the fitting object to obtain the three-dimensional coordinates of the target interpolation points.

[0009] According to an embodiment of the present application, fitting at least part of the reconstructed points in the three-dimensional point cloud data based on the three-dimensional point cloud data to obtain a fitting object includes: Perform surface fitting on at least part of the reconstructed points based on the three-dimensional point cloud data to obtain a target surface.

[0010] According to an embodiment of the present application, performing interpolation processing on at least part of the reconstructed points in the fitting object to obtain the three-dimensional coordinates of the target interpolation points includes: Performing interpolation processing on at least part of the reconstructed points in the target curved surface to obtain first coordinates of the target interpolation points; The first coordinates are transformed to obtain the three-dimensional coordinates of the target interpolation point.

[0011] According to an embodiment of the present application, performing interpolation processing on at least part of the reconstructed points in the target surface to obtain first coordinates of the target interpolation points includes: Acquire second coordinates of at least part of the reconstructed points in the three-dimensional point cloud data on the target surface; The second coordinate is interpolated to obtain the target interpolation point and the first coordinate of the target interpolation point on the target plane.

[0012] According to an embodiment of the present application, the transforming the first coordinate to obtain the three-dimensional coordinate of the target interpolation point includes: Using a three-dimensional polynomial fitting algorithm to perform surface fitting on the three-dimensional coordinates of some reconstructed points in the three-dimensional point cloud data to obtain a preset fitting formula; The first coordinate is processed based on the preset fitting formula to obtain the three-dimensional coordinate of the target interpolation point.

[0013] According to an embodiment of the present application, the three-dimensional polynomial fitting algorithm is used to perform surface fitting on the three-dimensional coordinates of some reconstructed points in the three-dimensional point cloud data to obtain a preset fitting formula, including: The projection coordinates of the reconstructed points on the target surface are converted into spatial coordinates based on the preset fitting formula; Solving the optimal value of the first coefficient of the preset fitting formula based on the three-dimensional point cloud data and the spatial coordinates to obtain a value of the first coefficient; The preset fitting formula is determined based on the value of the first coefficient.

[0014] According to an embodiment of the present application, fitting at least part of the reconstructed points in the three-dimensional point cloud data based on the three-dimensional point cloud data to obtain a fitting object includes: Curve fitting is performed on at least part of the reconstructed points based on the three-dimensional point cloud data to obtain a target curve.

[0015] According to an embodiment of the present application, performing interpolation processing on at least part of the reconstructed points in the fitting object to obtain the three-dimensional coordinates of the target interpolation points includes: Performing interpolation processing on at least part of the reconstructed points in the target curve to obtain first coordinates of the target interpolation points; The first coordinates are transformed to obtain the three-dimensional coordinates of the target interpolation point.

[0016] According to an embodiment of the present application, performing interpolation processing on at least part of the reconstructed points in the target curve to obtain the first coordinates of the target interpolation points includes: Acquire the third coordinates of at least part of the reconstructed points in the three-dimensional point cloud data in the target direction; The third coordinate is interpolated to obtain the target interpolation point and the first coordinate of the target interpolation point in the target direction.

[0017] According to an embodiment of the present application, the transforming the first coordinate to obtain the three-dimensional coordinate of the target interpolation point includes: Using a two-dimensional polynomial fitting algorithm, curve fitting is performed on the three-dimensional coordinates of some reconstructed points in the three-dimensional point cloud data to obtain a preset fitting formula; The first coordinate is processed based on the preset fitting formula to obtain the three-dimensional coordinate of the target interpolation point.

[0018] According to an embodiment of the present application, the two-dimensional polynomial fitting algorithm is used to perform curve fitting on the three-dimensional coordinates of some reconstructed points in the three-dimensional point cloud data to obtain a preset fitting formula, including: Based on the preset fitting formula, the projection coordinates of the reconstructed point in the target direction are converted into spatial coordinates in the target direction; Solving the optimal value of the second coefficient of the preset fitting formula based on the three-dimensional point cloud data and the spatial coordinates to obtain a value of the second coefficient; Based on the value of the second coefficient, a preset fitting formula corresponding to the target direction is determined.

[0019] According to an embodiment of the present application, performing interpolation processing on at least part of the reconstructed points in the fitting object to obtain the three-dimensional coordinates of the target interpolation points includes: Performing interpolation processing on at least part of the reconstructed points in the fitting object to obtain first coordinates of the target interpolation point; The first coordinate is processed based on a preset fitting formula to obtain the three-dimensional coordinate of the target interpolation point; the preset fitting formula is determined by fitting the three-dimensional point cloud data.

[0020] According to an embodiment of the present application, the interpolation processing is performed on at least part of the reconstructed points in the three-dimensional point cloud data to obtain the three-dimensional coordinates of the target interpolation points, including: A linear interpolation algorithm is used to interpolate at least part of the reconstructed points in the three-dimensional point cloud data to obtain the three-dimensional coordinates of the target interpolation points.

[0021] According to an embodiment of the present application, the interpolation processing is performed on at least part of the reconstructed points in the three-dimensional point cloud data to obtain the three-dimensional coordinates of the target interpolation points, including: A nearest neighbor interpolation algorithm is used to interpolate at least some of the reconstructed points based on the grayscale values ​​of at least some of the reconstructed points in the three-dimensional point cloud data to obtain the three-dimensional coordinates of the target interpolation points and the grayscale values ​​of the target interpolation points.

[0022] According to an embodiment of the present application, the interpolation processing is performed on at least part of the reconstructed points in the three-dimensional point cloud data to obtain the three-dimensional coordinates of the target interpolation points, including: Based on the sampling accuracy, determine the target number of interpolation points; The three-dimensional point cloud data is interpolated based on the target number of interpolation points to obtain the target number of interpolation points and the three-dimensional coordinates of each of the target interpolation points.

[0023] In a second aspect, the present application provides a three-dimensional reconstruction method, the method comprising: According to the three-dimensional reconstruction method as described in the first aspect, the three-dimensional coordinates of at least one target interpolation point are obtained: Based on the three-dimensional coordinates of the target interpolation point and the three-dimensional point cloud data, the object to be reconstructed is three-dimensionally reconstructed.

[0024] According to the 3D reconstruction method of the present application, by interpolating the 3D point cloud data obtained based on a single-frame image frame, the coordinates in the world coordinate system are obtained, so that the 3D coordinates of the new positions on the object to be reconstructed other than the reconstruction points can be obtained, so that the output point quantity is not limited by the resolution of the binocular camera, and point cloud data far exceeding the camera resolution is obtained, thereby increasing the output point quantity of a single-frame image frame, thereby improving the scanning efficiency; and the 3D coordinates corresponding to the reconstructed target interpolation points are in line with the surface change trend of the object to be reconstructed, with high sampling precision and accuracy, thereby improving the accuracy of the reconstructed 3D model and improving the scanning effect.

[0025] In a third aspect, the present application provides a three-dimensional reconstruction device, the device comprising: A first processing module is used to construct three-dimensional point cloud data corresponding to a single image frame based on the image frame of the object to be reconstructed collected by the binocular camera in the laser scanner; the three-dimensional point cloud data includes a plurality of reconstruction points and the three-dimensional coordinates of each of the reconstruction points; the reconstruction point is a spatial coordinate point obtained by three-dimensionally reconstructing the pixel points in the image frame; The second processing module is used to perform interpolation processing on at least part of the reconstructed points in the three-dimensional point cloud data to obtain the three-dimensional coordinates of the target interpolation points.

[0026] According to the 3D reconstruction device of the present application, by interpolating the 3D point cloud data obtained based on a single-frame image frame, a target interpolation point and the 3D coordinates of the target interpolation point are obtained, so that the 3D coordinates of new positions on the object to be reconstructed other than the reconstruction point can be obtained, so that the output point quantity is not limited by the resolution of the binocular camera, and point cloud data far exceeding the camera resolution is obtained, thereby increasing the output point quantity of a single-frame image frame, thereby improving the scanning efficiency; and the 3D coordinates corresponding to the reconstructed target interpolation points are in line with the surface change trend of the object to be reconstructed, and have high sampling precision and accuracy, thereby improving the user experience.

[0027] In a fourth aspect, the present application provides a three-dimensional reconstruction device, the device comprising: A third processing module is used to obtain the three-dimensional coordinates of at least one target interpolation point according to the three-dimensional reconstruction method according to the first aspect: The fourth processing module is used to perform three-dimensional reconstruction on the object to be reconstructed based on the three-dimensional coordinates of the target interpolation point and the three-dimensional point cloud data.

[0028] In a fifth aspect, the present application provides a laser scanner, comprising: A binocular camera, a laser group and a control device, wherein the control device is used to execute the three-dimensional reconstruction method as described in the first aspect or the three-dimensional reconstruction method as described in the second aspect.

[0029] In a sixth aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the three-dimensional reconstruction method as described in the first aspect or the three-dimensional reconstruction method as described in the second aspect.

[0030] In a seventh aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the three-dimensional reconstruction method as described in the first aspect or the three-dimensional reconstruction method as described in the second aspect.

[0031] In an eighth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the three-dimensional reconstruction method as described in the first aspect or the three-dimensional reconstruction method as described in the second aspect.

[0032] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects: By interpolating the three-dimensional point cloud data constructed based on a single-frame image frame, the target interpolation point and its three-dimensional coordinates are obtained, so that the three-dimensional coordinates of the new position on the object to be reconstructed other than the reconstruction point can be obtained, so that the output point quantity is not limited by the resolution of the binocular camera, and point cloud data far exceeding the camera resolution is obtained, which increases the output point quantity of a single-frame image frame, thereby improving the scanning efficiency; and the three-dimensional coordinates corresponding to the reconstructed target interpolation point are consistent with the surface change trend of the object to be reconstructed, with high sampling precision and accuracy, thereby improving the user experience.

[0033] Furthermore, by fitting the three-dimensional point cloud data lines constructed based on a single-frame image frame and performing interpolation processing on the obtained fitting object to obtain the three-dimensional coordinates of the target interpolation point on the object to be reconstructed, the interpolation accuracy can be improved, so that the target interpolation point obtained by upsampling is more in line with the surface change trend of the object to be reconstructed, and is suitable for three-dimensional reconstruction scenarios of irregular objects and objects with complex structures; it can also reduce single-point noise fluctuations, improve interpolation accuracy, and further improve the sampling effect, with high universality.

[0034] Furthermore, the reconstructed three-dimensional point cloud data is interpolated by a direct linear interpolation algorithm to obtain new interpolation points, which can increase the number of points, and the operation is simple, convenient and easy to implement.

[0035] Furthermore, by performing surface fitting on the reconstructed three-dimensional point cloud data and performing interpolation processing based on the fitted target surface, a surface that better conforms to the surface change law of the object to be reconstructed can be obtained, thereby improving the precision and accuracy of the interpolation points obtained by interpolation and improving the sampling effect.

[0036] Furthermore, by performing curve fitting on the reconstructed three-dimensional point cloud data and performing interpolation processing based on the fitted target curve, the operation is simple, fast and easy to implement, which can improve the sampling efficiency and thus improve the scanning efficiency.

[0037] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which: Figure 1 This is one of the flow charts of the three-dimensional reconstruction method provided in the embodiment of the present application; Figure 2 It is one of the intermediate result schematic diagrams of the three-dimensional reconstruction method provided in the embodiment of the present application; Figure 3 This is the second schematic diagram of the intermediate result of the three-dimensional reconstruction method provided in the embodiment of the present application; Figure 4 It is one of the structural schematic diagrams of the three-dimensional reconstruction device provided in the embodiment of the present application; Figure 5 This is the second flow chart of the three-dimensional reconstruction method provided in the embodiment of the present application; Figure 6 is a schematic diagram of the result of the three-dimensional reconstruction method provided in an embodiment of the present application; Figure 7 This is the second structural schematic diagram of the three-dimensional reconstruction device provided in the embodiment of the present application; Figure 8 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0040] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0041] The following, in conjunction with the accompanying drawings, describes in detail the 3D reconstruction method, 3D reconstruction device, electronic device and readable storage medium provided in the embodiments of the present application through specific embodiments and their application scenarios.

[0042] The three-dimensional reconstruction method may be applied to a terminal, and may be specifically executed by hardware or software in the terminal.

[0043] The 3D reconstruction method provided in the embodiment of the present application can be executed by an electronic device or a functional module or functional entity in the electronic device that can implement the 3D reconstruction method. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablet computers, computers, cameras and wearable devices. The 3D reconstruction method provided in the embodiment of the present application is described below using the electronic device as an example of the execution subject.

[0044] A handheld laser scanner is a high-precision 3D measurement device that can measure the surface features of an object by emitting a laser beam and receiving the reflected light. Handheld laser scanners can be used in industrial design, architectural measurement, archaeological excavation and other fields to obtain a 3D model of an object.

[0045] In the related art, when acquiring a three-dimensional model, the image frames are mainly collected through the binocular camera inside the handheld laser scanner, and the binocular reconstruction technology is used to construct three-dimensional point cloud data based on the image frames, wherein the binocular reconstruction is based on the points on the pixels of the image; that is, the number of points in a single frame is related to the image pixels, one pixel reconstructs a three-dimensional coordinate point, and then subsequent reconstruction and other operations are performed based on the reconstructed three-dimensional coordinate point; the reconstructed point cloud data distribution may include: 1) Reconstructing a surface based on the reconstruction method of the entire surface radiation matching such as phase and speckle; 2) Reconstructing a curve based on the reconstruction method such as laser lines and projection lines. However, the number of points reconstructed by the above method is limited by the camera resolution, and the maximum number of reconstructed points is fixed, resulting in the inability to increase the scanning speed and affecting the scanning efficiency; in addition, because the single-pixel reconstruction is unrelated to each other, the single-point noise fluctuates greatly, affecting the reconstruction precision and accuracy, thereby affecting the user experience. In order to solve at least one of the above technical problems, the embodiments of the present application provide a three-dimensional reconstruction method, a three-dimensional reconstruction method, a three-dimensional reconstruction device, a three-dimensional reconstruction device, an electronic device and a readable storage medium. The three-dimensional reconstruction method, the three-dimensional reconstruction method, the three-dimensional reconstruction device, the three-dimensional reconstruction device, the electronic device and the readable storage medium provided in the embodiments of the present application are described in detail below in combination with the accompanying drawings through specific embodiments and their application scenarios.

[0046] like Figure 1 As shown, the three-dimensional reconstruction method includes: step 110 and step 120.

[0047] Step 110: constructing three-dimensional point cloud data corresponding to a single image frame based on the image frame of the object to be reconstructed acquired by the binocular camera in the laser scanner; the three-dimensional point cloud data includes a plurality of reconstruction points and the three-dimensional coordinates of each reconstruction point; the reconstruction point is a spatial coordinate point obtained by three-dimensionally reconstructing the pixel points in the image frame; In this step, the laser scanner is a high-precision measuring device, which may include components such as a laser and a binocular camera. The laser can emit a laser beam, which is reflected after being irradiated on the surface of the scanned object. The binocular camera captures these reflected lasers and calculates the three-dimensional coordinate information of the object surface through binocular reconstruction technology. In the laser scanner, it may include a laser that emits a single laser beam; it may also include multiple lasers that can emit intersecting lasers. These laser lines intersect in space to form a cross shape for capturing the three-dimensional information of the object surface, so as to improve the accuracy and efficiency of the scanning.

[0048] The binocular camera includes two cameras, which are used to capture the reflections generated by the lasers emitted by each laser group to the object to be reconstructed, so as to obtain two frames of images. The binocular reconstruction technology is used to image any point on the object to be reconstructed in the two cameras respectively, and the pixel coordinates of the pixel points corresponding to the point in the two frames of images are obtained. Then, based on the parameter matrices of the two cameras, a coordinate transformation relationship is established to calculate the world coordinates of the point; by performing similar processing on multiple pixel points in the two frames of images, the world coordinates corresponding to multiple points on the object to be reconstructed (i.e., reconstruction points) can be calculated, thereby obtaining three-dimensional point cloud data. In some embodiments, the binocular camera can be two cameras with exactly the same internal parameters, and the two cameras are arranged in parallel, and the optical axes are parallel to each other.

[0049] In some embodiments, before step 110, based on the image frames of the object to be reconstructed acquired by the binocular camera in the laser scanner, constructing the three-dimensional point cloud data corresponding to the single image frame, the method may further include: Calibrate the binocular camera.

[0050] In this embodiment, before image frame acquisition, the external parameters of the dual-target cameras should be calibrated first, so as to reduce errors caused by different internal parameters of the dual-target cameras and different shooting states.

[0051] Step 120: interpolate at least some of the reconstructed points in the three-dimensional point cloud data to obtain three-dimensional coordinates of target interpolation points.

[0052] In this step, the number of output points corresponding to the three-dimensional point cloud data constructed by the binocular reconstruction algorithm is determined based on the resolution of the binocular camera. One pixel point in the image frame can correspond to the calculated three-dimensional coordinates of a reconstructed point; that is, the maximum value of the number of output points (equivalent to the maximum number of reconstructed points included in the three-dimensional point cloud data) is the number of pixels included in the image frame captured by the binocular camera.

[0053] At least part of the reconstruction points includes two reconstruction points or more than two reconstruction points.

[0054] The target interpolation point is a new point obtained by interpolating the three-dimensional point cloud data, which is different from the reconstructed point obtained by directly mapping the pixel points in the image frame. The target interpolation point may include one or more interpolation points. By interpolating at least part of the reconstructed points to obtain the interpolation points, the output point quantity can be increased.

[0055] In some embodiments, step 120, performing interpolation processing on at least part of the reconstructed points in the three-dimensional point cloud data to obtain the three-dimensional coordinates of the target interpolation points, may include: A linear interpolation algorithm is used to interpolate at least some of the reconstructed points in the three-dimensional point cloud data to obtain the three-dimensional coordinates of the target interpolation points.

[0056] In this embodiment, after reconstructing the three-dimensional point cloud data, the target interpolation point and the three-dimensional coordinates corresponding to the target interpolation point can be directly interpolated through a linear interpolation algorithm, such as by reconstructing the pixel x and y directions and directly performing linear interpolation to obtain a new interpolation point.

[0057] Of course, in other embodiments, other interpolation algorithms may also be used, such as cubic spline interpolation or Kriging interpolation algorithm, etc., which will not be elaborated in this application.

[0058] According to the three-dimensional reconstruction method provided in the embodiment of the present application, the reconstructed three-dimensional point cloud data is interpolated by a direct linear interpolation algorithm to obtain new interpolation points, which can increase the number of output points, and the operation is simple, convenient and easy to implement.

[0059] In some embodiments, step 120, performing interpolation processing on at least part of the reconstructed points in the three-dimensional point cloud data to obtain the three-dimensional coordinates of the target interpolation points, may include: A nearest neighbor interpolation algorithm is used to interpolate at least some of the reconstructed points based on the grayscale values ​​of at least some of the reconstructed points in the three-dimensional point cloud data to obtain the three-dimensional coordinates of the target interpolation points and the grayscale values ​​of the target interpolation points.

[0060] In this embodiment, after the target interpolation point is obtained, the grayscale value of the reconstruction point closest to the target interpolation point can be assigned to the target interpolation point using a nearest neighbor interpolation algorithm, thereby obtaining a grayscale value corresponding to a new interpolation point.

[0061] Of course, in other embodiments, the average value of the grayscale values ​​of the reconstructed points near the target interpolation point may also be determined as the grayscale value of the target interpolation point, and this application does not limit this.

[0062] According to the 3D reconstruction method provided in the embodiment of the present application, by interpolating the 3D point cloud data obtained based on a single-frame image frame, a target interpolation point and the 3D coordinates of the target interpolation point are obtained, so that the 3D coordinates of new positions on the object to be reconstructed other than the reconstruction point can be obtained, so that the output point quantity is not limited by the resolution of the binocular camera, and point cloud data far exceeding the camera resolution is obtained, thereby increasing the output point quantity of a single-frame image frame, thereby improving scanning efficiency; and the 3D coordinates corresponding to the reconstructed target interpolation points are in line with the surface change trend of the object to be reconstructed, and have high sampling precision and accuracy, thereby improving the user experience.

[0063] In some embodiments, step 120, performing interpolation processing on at least part of the reconstructed points in the three-dimensional point cloud data to obtain the three-dimensional coordinates of the target interpolation points, may include: Fitting at least part of the reconstructed points in the three-dimensional point cloud data based on the three-dimensional point cloud data to obtain a fitting object; Interpolation processing is performed on at least part of the reconstructed points in the fitting object to obtain three-dimensional coordinates of target interpolation points.

[0064] In this embodiment, before interpolation, a line or surface fitting may be performed based on some of the reconstructed points in the reconstructed three-dimensional point cloud data to obtain a fitting object, and then interpolation processing is performed on the fitting object to obtain a target interpolation point and the three-dimensional coordinates of the target interpolation point. The fitting object may include a fitting plane, a fitting surface, a fitting line or a fitting curve, etc. In the process of interpolating the fitting object, linear interpolation or other interpolation algorithms may also be used.

[0065] According to the 3D reconstruction method provided in the embodiment of the present application, by fitting the 3D point cloud data lines constructed based on a single-frame image frame, and performing interpolation processing on the obtained fitting object to obtain the 3D coordinates of the target interpolation point on the object to be reconstructed, the interpolation accuracy can be improved, so that the target interpolation point obtained by upsampling is more in line with the surface change trend of the object to be reconstructed, and it is suitable for 3D reconstruction scenarios of irregular objects and objects with complex structures; it can also reduce single-point noise fluctuations, improve interpolation accuracy, and further improve the sampling effect, and has high universality.

[0066] In some embodiments, performing interpolation processing on at least part of the reconstructed points in the fitting object to obtain the three-dimensional coordinates of the target interpolation points may include: Performing interpolation processing on at least part of the reconstructed points in the fitting object to obtain first coordinates of the target interpolation points; The first coordinate is processed based on a preset fitting formula to obtain the three-dimensional coordinates of the target interpolation point.

[0067] In this embodiment, the preset fitting formula is a formula used to characterize the surface change trend of the object to be reconstructed.

[0068] The preset fitting formula is determined by fitting the three-dimensional point cloud data.

[0069] The preset fitting formula may include but is not limited to a cubic polynomial, a quartic polynomial or a quintic polynomial, etc. In the process of solving the polynomial, the least square method or other extreme value solving algorithm may be used to solve the polynomial to obtain a preset fitting formula with a better fitting effect, and obtain a fitting object that better conforms to the surface change trend of the object to be reconstructed.

[0070] According to the three-dimensional reconstruction method provided in the embodiment of the present application, polynomial fitting is performed, and the operation is simple and convenient.

[0071] The following is a detailed description of the embodiments of the present application from two different perspectives of fitting objects.

[0072] 1. The fitting object is a surface In some embodiments, fitting at least part of the reconstructed points in the three-dimensional point cloud data based on the three-dimensional point cloud data to obtain the fitting object may include: Based on the three-dimensional point cloud data, surface fitting is performed on at least part of the reconstructed points to obtain a target surface.

[0073] In this embodiment, the surface fitting can be performed based on the three-dimensional coordinates corresponding to some adjacent reconstruction points in the three-dimensional point cloud data to obtain the target surface, such as by reconstructing the x direction and y direction of the pixel. Figure 2 As shown, one grid point corresponds to one reconstruction point, and the grid point stores the three-dimensional coordinates of the reconstruction point corresponding to the grid point. Figure 2The reconstructed points within a certain range in the image are fitted to obtain a high-order surface, and the high-order surface is used as the target surface to obtain the target interpolation point by upsampling on the high-order surface.

[0074] In some embodiments, performing interpolation processing on at least part of the reconstructed points in the fitting object to obtain the three-dimensional coordinates of the target interpolation points may include: Performing interpolation processing on at least part of the reconstructed points in the target surface to obtain first coordinates of the target interpolation points; The first coordinate is transformed to obtain the three-dimensional coordinate of the target interpolation point.

[0075] In this embodiment, the first coordinate may be coordinates in any two directions of the three-dimensional coordinates, such as an x ​​coordinate and a y coordinate.

[0076] After the first coordinate is obtained, the z coordinate can be determined based on the x coordinate and the y coordinate through the fitting parameters of the target surface, thereby obtaining the three-dimensional coordinate of the target interpolation point on the object to be reconstructed.

[0077] In some embodiments, performing interpolation processing on at least part of the reconstructed points in the target surface to obtain first coordinates of the target interpolation points may include: Obtaining second coordinates of at least some of the reconstructed points in the three-dimensional point cloud data on the target surface; The second coordinate is interpolated to obtain a target interpolation point and a first coordinate of the target interpolation point on the target surface.

[0078] In this embodiment, the target surface may be an orthographic projection surface formed by the x-axis direction and the y-axis direction, so as to ensure that the coordinates (x, y) on the projection surface are the same as the (x, y) in the three-dimensional space coordinates.

[0079] The second coordinate is the projection coordinate obtained by projecting the three-dimensional coordinate of the reconstructed point onto the target surface. The second coordinate is the coordinate in the same dimension as the first coordinate. For example, the second coordinate can be the coordinate of the reconstructed point in the x-axis direction and the y-axis direction.

[0080] Continue to refer to Figure 2 The high-order surface shown in the figure can be interpolated to obtain the fitted surface. The nearest neighbor points can be found for multiple reconstruction points, and new points can be obtained by linear interpolation. Taking the reconstruction points A (x, y), B (x+1, y), C (x, y+1) and D (x+1, y+1) as examples, the target interpolation point G can be obtained by interpolating each reconstruction point: Pi=(P<x,y> +P<x+1,y> +P<x+1,y+1> +P<x,y+1> ) / 4; Wherein, Pi is the first coordinate of the target interpolation point G.

[0081] In some embodiments, two adjacent reconstruction points can be selected in the fitted high-order surface, and linear interpolation is performed on the two adjacent reconstruction points to obtain a new point, and then the new point is obtained. Figure 2 , taking the reconstruction point A (x, y) and the reconstruction point C (x, y+1) as examples, the target interpolation point I can be obtained by interpolating the reconstruction point A and the reconstruction point B: Pi=(P<x,y> +P<x,y+1> ) / 2; Wherein, Pi is the first coordinate of the target interpolation point I.

[0082] In some embodiments, transforming the first coordinate to obtain the three-dimensional coordinate of the target interpolation point may include: A three-dimensional polynomial fitting algorithm is used to perform surface fitting on the three-dimensional coordinates of some reconstructed points in the three-dimensional point cloud data to obtain a preset fitting formula; The first coordinate is processed based on a preset fitting formula to obtain the three-dimensional coordinates of the target interpolation point.

[0083] In this embodiment, a three-dimensional polynomial fitting algorithm is used to fit a high-order surface, and a preset fitting formula is used to characterize the correlation between coordinate values ​​in any two dimensions of the three-dimensional coordinates and the coordinate value in another dimension.

[0084] During the fitting process, cubic, quartic, quintic or other degree polynomials can be selected based on known three-dimensional point cloud data, and the least squares method can be used for fitting to obtain a fitting polynomial with better fitting effect as a preset fitting formula.

[0085] In some embodiments, a three-dimensional polynomial fitting algorithm is used to perform surface fitting on the three-dimensional coordinates of some reconstructed points in the three-dimensional point cloud data to obtain a preset fitting formula, which may include: The projection coordinates of the reconstructed points on the target surface are converted into spatial coordinates based on a preset fitting formula; Based on the three-dimensional point cloud data and the spatial coordinates, solving the optimal value of the first coefficient of the preset fitting formula to obtain the value of the first coefficient; Based on the value of the first coefficient, a preset fitting formula is determined.

[0086] In this embodiment, the surface equation (i.e., the preset fitting formula) is: h(u,v)=au 2 +buv+cv 2 +du+ev+f Among them, a, b, c, d, e and f are the first coefficients, which are used to describe the fitting coefficients of the fitting surface; (ui, vi, hi) is the three-dimensional coordinate of the i-th reconstruction point, (ui, vi, hi)∈C, i=1, 2,..., n, n is a positive integer, and C is the centroid set.

[0087] List the linear equations as:

[0088] The above linear equations can be simplified to: AX=h; Solve using the linear least squares method, that is, , the first coefficient can be obtained, thereby obtaining the surface equation, that is, obtaining a preset fitting formula for characterizing the correlation relationship between the projection coordinates on the target surface and the coordinate value of another dimension in the three-dimensional coordinates.

[0089] Substituting the first coordinate (u, v) corresponding to the target interpolation point into the preset fitting formula, the corresponding h coordinate can be obtained, thereby obtaining the three-dimensional coordinate of the target interpolation point.

[0090] According to the three-dimensional reconstruction method provided in the embodiment of the present application, by performing surface fitting on the reconstructed three-dimensional point cloud data and performing interpolation processing based on the fitted target surface, a surface that better conforms to the surface change law of the object to be reconstructed can be obtained, thereby improving the precision and accuracy of the interpolation points obtained by interpolation and improving the sampling effect.

[0091] 2. The fitting object is a curve In some embodiments, fitting at least part of the reconstructed points in the three-dimensional point cloud data based on the three-dimensional point cloud data to obtain the fitting object may include: Curve fitting is performed on at least part of the reconstructed points based on the three-dimensional point cloud data to obtain a target curve.

[0092] In this embodiment, reconstruction points within a certain range may be extracted first to perform polynomial fitting to obtain a fitting curve, and then the fitting curve may be upsampled to obtain new target interpolation points.

[0093] Figure 3 An example of a target curve is shown.

[0094] In some embodiments, performing interpolation processing on at least part of the reconstructed points in the fitting object to obtain the three-dimensional coordinates of the target interpolation points may include: Performing interpolation processing on at least part of the reconstructed points in the target curve to obtain first coordinates of the target interpolation points; The first coordinate is transformed to obtain the three-dimensional coordinate of the target interpolation point.

[0095] In this embodiment, the first coordinate may be a projection coordinate in the x-axis direction, a projection coordinate in the y-axis direction, or a projection coordinate in the z-axis direction.

[0096] Continue to refer to Figure 3The target curve shown in the figure takes the reconstruction point E(t) and the reconstruction point F(t+1) as examples. The target interpolation point H can be obtained by interpolating the reconstruction point E and the reconstruction point F: Pi=(P <t>+P<t+1> ) / 2; Wherein, Pi is the first coordinate of the target interpolation point H.

[0097] After the first coordinates are obtained, the first coordinates are transformed based on the fitting parameters to obtain the three-dimensional coordinates of the target interpolation point on the object to be reconstructed.

[0098] In some embodiments, performing interpolation processing on at least part of the reconstructed points in the target curve to obtain first coordinates of the target interpolation points may include: Obtaining the third coordinates of at least some of the reconstructed points in the three-dimensional point cloud data in the target direction; The third coordinate is interpolated to obtain a target interpolation point and a first coordinate of the target interpolation point in a target direction.

[0099] In this embodiment, the third coordinate corresponds one-to-one to the dimension corresponding to the first coordinate.

[0100] The target direction may include an x-direction, a y-direction, and a z-direction.

[0101] In the actual implementation process, the reconstructed points in the three-dimensional point cloud data should be projected along the x-direction, y-direction and z-direction respectively to obtain the third coordinates in each target direction; then, in each target direction, the same projection points are interpolated to obtain the first coordinates of the target interpolation points in different target directions.

[0102] In some embodiments, transforming the first coordinate to obtain the three-dimensional coordinate of the target interpolation point may include: Using a two-dimensional polynomial fitting algorithm, curve fitting is performed on the three-dimensional coordinates of some reconstructed points in the three-dimensional point cloud data to obtain a preset fitting formula; The first coordinate is processed based on a preset fitting formula to obtain the three-dimensional coordinates of the target interpolation point.

[0103] In this embodiment, a two-dimensional polynomial fitting algorithm is used to fit the curve, and a preset fitting formula is used to characterize the correlation between the projection coordinates in the target direction and the three-dimensional coordinates.

[0104] During the fitting process, cubic, quartic, quintic or other degree polynomials can be selected based on known three-dimensional point cloud data, and the least squares method can be used for fitting to obtain a fitting polynomial with better fitting effect as a preset fitting formula.

[0105] In some embodiments, a two-dimensional polynomial fitting algorithm is used to perform curve fitting on the three-dimensional coordinates of some reconstructed points in the three-dimensional point cloud data to obtain a preset fitting formula, which may include: The projection coordinates of the reconstructed points in the target direction are converted into spatial coordinates in the target direction based on a preset fitting formula; Based on the three-dimensional point cloud data and the spatial coordinates, solving the optimal value of the second coefficient of the preset fitting formula to obtain the value of the second coefficient; Based on the value of the second coefficient, a preset fitting formula corresponding to the target direction is determined.

[0106] In this embodiment, the target direction may include an x-axis direction, a y-axis direction, and a z-axis direction.

[0107] Each direction corresponds to a preset fitting formula.

[0108] Assume the curve equation (i.e. the preset fitting formula) is: x=f(t); y=g(t); z = h(t); Where t is the projection coordinate in the target direction.

[0109] In the actual fitting process, you can customize the polynomial for fitting. Taking cubic polynomial fitting as an example, the curve equation can be obtained as follows: x(t)=a0+a1*t+a2*t 2 +a3*t 3 ; y(t)=b0+b1*t+b2*t 2 +b3*t 3 ; z(t)=c0+c1*t+c2*t 2 +c3*t 3 ; Among them, a0, a1, a2, a3, b0, b1, b2, b3, c0, c1, c2 and c3 are the second coefficients used to describe the fitting coefficients of the fitting curve, ti is the projection coordinate of the i-th reconstruction point in the target direction, i = 1, 2, ..., n, n is a positive integer.

[0110] Substitute the known three-dimensional point cloud data into the curve equation, use the least squares method and other methods to solve the second coefficient, so as to obtain an optimal polynomial fitting curve, that is, obtain the preset fitting formula; then substitute the first coordinate t corresponding to the target interpolation point in each target direction into the preset fitting formula corresponding to the target direction, and the three-dimensional coordinate of the target interpolation point can be obtained.

[0111] According to the three-dimensional reconstruction method provided in the embodiment of the present application, curve fitting is performed on the reconstructed three-dimensional point cloud data to perform interpolation processing based on the fitted target curve. The operation is simple, fast and easy to implement, which can improve the sampling efficiency and thus improve the scanning efficiency.

[0112] This application provides a variety of different interpolation methods, which can be flexibly selected based on actual needs.

[0113] In some embodiments, step 120, performing interpolation processing on at least part of the reconstructed points in the three-dimensional point cloud data to obtain the three-dimensional coordinates of the target interpolation points, may include: Based on the sampling accuracy, determine the target number of interpolation points; The three-dimensional point cloud data is interpolated based on the target number of interpolation points to obtain the target number of interpolation points and the three-dimensional coordinates of each target interpolation point.

[0114] In this embodiment, the target number is the number of outgoing points that need to be increased.

[0115] In some embodiments, the target number may be based on user customization, or may be determined based on sampling or scanning accuracy.

[0116] The amount of outgoing points to be added may be selected according to actual needs, so that a corresponding number of interpolation points and the first coordinates corresponding to each interpolation point are obtained by performing interpolation processing according to the amount of outgoing points to be added.

[0117] According to the three-dimensional reconstruction method provided in the embodiment of the present application, a corresponding number of interpolation points can be added according to the number of output points required in the actual application process, so that the number of output points is not limited by the resolution of the binocular camera, and point cloud data far exceeding the camera resolution is obtained, thereby improving scanning efficiency.

[0118] The 3D reconstruction method provided in the embodiment of the present application can be executed by a 3D reconstruction device. In the embodiment of the present application, the 3D reconstruction method performed by a 3D reconstruction device is taken as an example to illustrate the 3D reconstruction device provided in the embodiment of the present application.

[0119] The embodiment of the present application also provides a three-dimensional reconstruction device.

[0120] like Figure 4 As shown, the three-dimensional reconstruction device includes: a first processing module 410 and a second processing module 420 .

[0121] The first processing module 410 is used to construct three-dimensional point cloud data corresponding to a single image frame based on the image frame of the object to be reconstructed collected by the binocular camera in the laser scanner; the three-dimensional point cloud data includes multiple reconstruction points and the three-dimensional coordinates of each reconstruction point; the reconstruction point is a spatial coordinate point obtained by three-dimensionally reconstructing the pixel points in the image frame; The second processing module 420 is used to perform interpolation processing on at least part of the reconstructed points in the three-dimensional point cloud data to obtain the three-dimensional coordinates of the target interpolation points.

[0122] According to the 3D reconstruction device provided in the embodiment of the present application, by interpolating the 3D point cloud data obtained based on a single-frame image frame, a target interpolation point and the 3D coordinates of the target interpolation point are obtained, so that the 3D coordinates of new positions on the object to be reconstructed other than the reconstruction point can be obtained, so that the output point quantity is not limited by the resolution of the binocular camera, and point cloud data far exceeding the camera resolution is obtained, thereby increasing the output point quantity of a single-frame image frame, thereby improving the scanning efficiency; and the 3D coordinates corresponding to the reconstructed target interpolation points are in line with the surface change trend of the object to be reconstructed, and have high sampling precision and accuracy, thereby improving the user experience.

[0123] In some embodiments, the second processing module 420 may also be used to: Fitting at least part of the reconstructed points in the three-dimensional point cloud data based on the three-dimensional point cloud data to obtain a fitting object; Interpolation processing is performed on at least part of the reconstructed points in the fitting object to obtain three-dimensional coordinates of target interpolation points.

[0124] In some embodiments, the second processing module 420 may also be used to: Based on the three-dimensional point cloud data, surface fitting is performed on at least part of the reconstructed points to obtain a target surface.

[0125] In some embodiments, the second processing module 420 may also be used to: Performing interpolation processing on at least part of the reconstructed points in the target surface to obtain first coordinates of the target interpolation points; The first coordinate is transformed to obtain the three-dimensional coordinate of the target interpolation point.

[0126] In some embodiments, the second processing module 420 may also be used to: Obtaining second coordinates of at least some of the reconstructed points in the three-dimensional point cloud data on the target surface; The second coordinate is interpolated to obtain a target interpolation point and a first coordinate of the target interpolation point on the target surface.

[0127] In some embodiments, the second processing module 420 may also be used to: A three-dimensional polynomial fitting algorithm is used to perform surface fitting on the three-dimensional coordinates of some reconstructed points in the three-dimensional point cloud data to obtain a preset fitting formula; The first coordinate is processed based on a preset fitting formula to obtain the three-dimensional coordinates of the target interpolation point.

[0128] In some embodiments, the second processing module 420 may also be used to: The projection coordinates of the reconstructed points on the target surface are converted into spatial coordinates based on a preset fitting formula; Based on the three-dimensional point cloud data and the spatial coordinates, solving the optimal value of the first coefficient of the preset fitting formula to obtain the value of the first coefficient; Based on the value of the first coefficient, a preset fitting formula is determined.

[0129] In some embodiments, the second processing module 420 may also be used to: Curve fitting is performed on at least part of the reconstructed points based on the three-dimensional point cloud data to obtain a target curve.

[0130] In some embodiments, the second processing module 420 may also be used to: Performing interpolation processing on at least part of the reconstructed points in the target curve to obtain first coordinates of the target interpolation points; The first coordinate is transformed to obtain the three-dimensional coordinate of the target interpolation point.

[0131] In some embodiments, the second processing module 420 may also be used to: Obtaining the third coordinates of at least some of the reconstructed points in the three-dimensional point cloud data in the target direction; The third coordinate is interpolated to obtain a target interpolation point and a first coordinate of the target interpolation point in a target direction.

[0132] In some embodiments, the second processing module 420 may also be used to: Using a two-dimensional polynomial fitting algorithm, curve fitting is performed on the three-dimensional coordinates of some reconstructed points in the three-dimensional point cloud data to obtain a preset fitting formula; The first coordinate is processed based on a preset fitting formula to obtain the three-dimensional coordinates of the target interpolation point.

[0133] In some embodiments, the second processing module 420 may also be used to: The projection coordinates of the reconstructed points in the target direction are converted into spatial coordinates in the target direction based on a preset fitting formula; Based on the three-dimensional point cloud data and the spatial coordinates, solving the optimal value of the second coefficient of the preset fitting formula to obtain the value of the second coefficient; Based on the value of the second coefficient, a preset fitting formula corresponding to the target direction is determined.

[0134] In some embodiments, the second processing module 420 may also be used to: Performing interpolation processing on at least part of the reconstructed points in the fitting object to obtain first coordinates of the target interpolation points; The first coordinate is processed based on a preset fitting formula to obtain the three-dimensional coordinate of the target interpolation point; the preset fitting formula is determined by fitting the three-dimensional point cloud data.

[0135] In some embodiments, the second processing module 420 may also be used to: A linear interpolation algorithm is used to interpolate at least some of the reconstructed points in the three-dimensional point cloud data to obtain the three-dimensional coordinates of the target interpolation points.

[0136] In some embodiments, the second processing module 420 may also be used to: A nearest neighbor interpolation algorithm is used to interpolate at least some of the reconstructed points based on the grayscale values ​​of at least some of the reconstructed points in the three-dimensional point cloud data to obtain the three-dimensional coordinates of the target interpolation points and the grayscale values ​​of the target interpolation points.

[0137] In some embodiments, the second processing module 420 may also be used to: Based on the sampling accuracy, determine the target number of interpolation points; The three-dimensional point cloud data is interpolated based on the target number of interpolation points to obtain the target number of interpolation points and the three-dimensional coordinates of each target interpolation point.

[0138] The three-dimensional reconstruction device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc. It can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0139] The 3D reconstruction device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0140] The three-dimensional reconstruction device provided in the embodiment of the present application can achieve Figures 1 to 3 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0141] The embodiment of the present application also provides a three-dimensional reconstruction method.

[0142] like Figure 5 As shown, the three-dimensional reconstruction method includes: step 510 and step 520.

[0143] Step 510: According to the 3D reconstruction method described in any of the above embodiments, obtain the 3D coordinates of at least one target interpolation point: In this step, each image frame captured by the binocular camera can be upsampled based on a similar method to obtain one or more target interpolation points and the three-dimensional coordinates of the target interpolation points.

[0144] Step 520: Perform three-dimensional reconstruction on the object to be reconstructed based on the three-dimensional coordinates of the target interpolation point and the three-dimensional point cloud data.

[0145] In this step, by shooting multiple image frames, such as collecting multiple image frames of the object to be reconstructed at different angles, the three-dimensional point cloud data corresponding to each image frame and the target interpolation points obtained by interpolation based on the three-dimensional point cloud data and the three-dimensional coordinates of the target interpolation points are calculated, and the three-dimensional model of the object to be reconstructed can be reconstructed by combining all the three-dimensional coordinates. Figure 6 shown.

[0146] Of course, in other embodiments, after obtaining the three-dimensional coordinates of the target interpolation point on the object to be reconstructed, data fusion, filtering and other scanning and measurement work can be performed based on the three-dimensional coordinates of the target interpolation point on the object to be reconstructed and the three-dimensional point cloud data.

[0147] According to the 3D reconstruction method provided in the embodiment of the present application, by interpolating the 3D point cloud data obtained based on a single-frame image frame, the coordinates in the world coordinate system are obtained, so that the 3D coordinates of the new positions on the object to be reconstructed other than the reconstruction points can be obtained, so that the output point quantity is not limited by the resolution of the binocular camera, and point cloud data far exceeding the camera resolution is obtained, thereby increasing the output point quantity of a single-frame image frame, thereby improving the scanning efficiency; and the 3D coordinates corresponding to the reconstructed target interpolation points are in line with the surface change trend of the object to be reconstructed, and have high sampling precision and accuracy, thereby improving the accuracy of the reconstructed 3D model and improving the scanning effect.

[0148] The 3D reconstruction method provided in the embodiment of the present application can be executed by a 3D reconstruction device. In the embodiment of the present application, the 3D reconstruction method performed by a 3D reconstruction device is taken as an example to illustrate the 3D reconstruction device provided in the embodiment of the present application.

[0149] The embodiment of the present application also provides a three-dimensional reconstruction device.

[0150] like Figure 7 As shown, the three-dimensional reconstruction device includes: a fourth processing module 710 and a fifth processing module 720 .

[0151] The fourth processing module 710 is used to obtain the three-dimensional coordinates of at least one target interpolation point according to the three-dimensional reconstruction method described in any of the above embodiments: The fifth processing module 720 is used to perform three-dimensional reconstruction on the object to be reconstructed based on the three-dimensional coordinates of the target interpolation point on the object to be reconstructed and the three-dimensional point cloud data.

[0152] According to the 3D reconstruction device provided in the embodiment of the present application, by interpolating the 3D point cloud data obtained based on the single-frame image frame, the coordinates in the world coordinate system are obtained, so that the 3D coordinates of the new positions on the object to be reconstructed except for the reconstruction points can be obtained, so that the output point quantity is not limited by the resolution of the binocular camera, and point cloud data far exceeding the camera resolution is obtained, thereby increasing the output point quantity of the single-frame image frame, thereby improving the scanning efficiency; and the 3D coordinates corresponding to the reconstructed target interpolation points are in line with the surface change trend of the object to be reconstructed, with high sampling precision and accuracy, thereby improving the accuracy of the reconstructed 3D model and improving the scanning effect.

[0153] The three-dimensional reconstruction device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc. It can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0154] The 3D reconstruction device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0155] The three-dimensional reconstruction device provided in the embodiment of the present application can achieve Figure 5 and Figure 6 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0156] An embodiment of the present application also provides a laser scanner.

[0157] The laser scanner comprises a binocular camera, a laser group and a control device.

[0158] In this embodiment, the laser group includes a plurality of lasers emitting cross-beam lasers.

[0159] The binocular camera is electrically connected to the control device, and the control device is used to execute the three-dimensional reconstruction method described in any of the above embodiments or the three-dimensional reconstruction method described in any of the above embodiments.

[0160] The binocular camera includes two cameras, which are used to respectively capture the reflections generated by the lasers emitted by each laser of the laser group to the object to be reconstructed, so as to obtain two image frames.

[0161] A control device is used to adopt binocular reconstruction technology, by imaging any point on the object to be reconstructed in two cameras respectively, to obtain the pixel coordinates of the pixel point corresponding to the point in the two image frames, and then based on the parameter matrices of the two cameras, by establishing a coordinate transformation relationship to calculate the world coordinates of the point; by performing similar processing on multiple pixel points in the two image frames, the world coordinates corresponding to multiple reconstruction points can be calculated, thereby obtaining three-dimensional point cloud data, and based on the obtained three-dimensional point cloud data, the three-dimensional reconstruction method described in any of the above embodiments is executed to perform upsampling, and one or more interpolation points and the three-dimensional coordinates of the interpolation points are obtained.

[0162] The obtained reconstruction points and interpolation points are used to represent the three-dimensional coordinate points of the object to be reconstructed in the three-dimensional space.

[0163] In some embodiments, the control device is also used to perform three-dimensional reconstruction of the object to be reconstructed based on the three-dimensional coordinates of the interpolation points and the three-dimensional point cloud data after obtaining one or more interpolation points and the three-dimensional coordinates of the interpolation points, so as to obtain a three-dimensional model of the object to be reconstructed, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0164] The controller may be the electronic device mentioned above.

[0165] According to the laser scanner provided in the embodiment of the present application, by interpolating the three-dimensional point cloud data constructed based on a single-frame image frame to obtain the coordinates of the interpolation point in the world coordinate system, it is possible to obtain the three-dimensional coordinates of the new position in addition to the reconstructed points obtained by direct mapping based on the pixel points, so that the output point quantity is not limited by the resolution of the binocular camera, and point cloud data far exceeding the camera resolution is obtained, thereby increasing the output point quantity of a single-frame image frame, thereby improving the scanning efficiency; and the three-dimensional coordinates corresponding to the reconstructed target interpolation points are in line with the surface change trend of the object to be reconstructed, and have high sampling precision and accuracy, thereby improving the user experience.

[0166] In some embodiments, Figure 8 As shown, an embodiment of the present application also provides an electronic device 800, including a processor 801, a memory 802, and a computer program stored in the memory 802 and executable on the processor 801. When the program is executed by the processor 801, the above-mentioned three-dimensional reconstruction method or each process of the three-dimensional reconstruction method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0167] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0168] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned three-dimensional reconstruction method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0169] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0170] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned three-dimensional reconstruction method when executed by a processor.

[0171] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0172] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned three-dimensional reconstruction method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0173] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0174] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0175] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0176] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0177] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0178] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.< / t>

Claims

1. A three-dimensional reconstruction method, characterized in that: include: Based on the image frames of the object to be reconstructed acquired by the binocular camera in the laser scanner, three-dimensional point cloud data corresponding to the single image frame is constructed; The three-dimensional point cloud data includes a plurality of reconstruction points and the three-dimensional coordinates of each of the reconstruction points; the reconstruction points are spatial coordinate points obtained by three-dimensionally reconstructing the pixel points in the image frame; Interpolation processing is performed on at least part of the reconstructed points in the three-dimensional point cloud data to obtain three-dimensional coordinates of target interpolation points.

2. The three-dimensional reconstruction method according to claim 1, characterized in that: The interpolation processing is performed on at least part of the reconstructed points in the three-dimensional point cloud data to obtain the three-dimensional coordinates of the target interpolation points, including: Fitting at least part of the reconstructed points in the three-dimensional point cloud data based on the three-dimensional point cloud data to obtain a fitting object; Interpolation processing is performed on at least part of the reconstructed points in the fitting object to obtain the three-dimensional coordinates of the target interpolation points.

3. The three-dimensional reconstruction method according to claim 2, characterized in that: The step of fitting at least part of the reconstructed points in the three-dimensional point cloud data based on the three-dimensional point cloud data to obtain a fitting object includes: Perform surface fitting on at least part of the reconstructed points based on the three-dimensional point cloud data to obtain a target surface.

4. The three-dimensional reconstruction method according to claim 3, characterized in that: The interpolation processing is performed on at least part of the reconstructed points in the fitting object to obtain the three-dimensional coordinates of the target interpolation points, including: Performing interpolation processing on at least part of the reconstructed points in the target curved surface to obtain first coordinates of the target interpolation points; The first coordinates are transformed to obtain the three-dimensional coordinates of the target interpolation point.

5. The three-dimensional reconstruction method according to claim 4, characterized in that: The interpolation processing is performed on at least part of the reconstructed points in the target surface to obtain the first coordinates of the target interpolation points, including: Acquire second coordinates of at least part of the reconstructed points in the three-dimensional point cloud data on the target surface; The second coordinate is interpolated to obtain the target interpolation point and the first coordinate of the target interpolation point on the target plane.

6. The three-dimensional reconstruction method according to claim 4, characterized in that: The converting the first coordinate to obtain the three-dimensional coordinate of the target interpolation point includes: Using a three-dimensional polynomial fitting algorithm to perform surface fitting on the three-dimensional coordinates of some reconstructed points in the three-dimensional point cloud data to obtain a preset fitting formula; The first coordinate is processed based on the preset fitting formula to obtain the three-dimensional coordinate of the target interpolation point.

7. The three-dimensional reconstruction method according to claim 6, characterized in that: The three-dimensional polynomial fitting algorithm is used to perform surface fitting on the three-dimensional coordinates of some reconstructed points in the three-dimensional point cloud data to obtain a preset fitting formula, including: The projection coordinates of the reconstructed points on the target surface are converted into spatial coordinates based on the preset fitting formula; Solving the optimal value of the first coefficient of the preset fitting formula based on the three-dimensional point cloud data and the spatial coordinates to obtain a value of the first coefficient; The preset fitting formula is determined based on the value of the first coefficient.

8. The three-dimensional reconstruction method according to claim 2, characterized in that: The step of fitting at least part of the reconstructed points in the three-dimensional point cloud data based on the three-dimensional point cloud data to obtain a fitting object includes: Curve fitting is performed on at least part of the reconstructed points based on the three-dimensional point cloud data to obtain a target curve.

9. The three-dimensional reconstruction method according to claim 8, characterized in that: The interpolation processing is performed on at least part of the reconstructed points in the fitting object to obtain the three-dimensional coordinates of the target interpolation points, including: Performing interpolation processing on at least part of the reconstructed points in the target curve to obtain first coordinates of the target interpolation points; The first coordinates are transformed to obtain the three-dimensional coordinates of the target interpolation point.

10. The three-dimensional reconstruction method according to claim 9, characterized in that: The interpolation processing is performed on at least part of the reconstructed points in the target curve to obtain the first coordinates of the target interpolation points, including: Acquire the third coordinates of at least part of the reconstructed points in the three-dimensional point cloud data in the target direction; The third coordinate is interpolated to obtain the target interpolation point and the first coordinate of the target interpolation point in the target direction.

11. The three-dimensional reconstruction method according to claim 9, characterized in that: The converting the first coordinate to obtain the three-dimensional coordinate of the target interpolation point includes: Using a two-dimensional polynomial fitting algorithm, curve fitting is performed on the three-dimensional coordinates of some reconstructed points in the three-dimensional point cloud data to obtain a preset fitting formula; The first coordinate is processed based on the preset fitting formula to obtain the three-dimensional coordinate of the target interpolation point.

12. The three-dimensional reconstruction method according to claim 11, characterized in that: The two-dimensional polynomial fitting algorithm is used to perform curve fitting on the three-dimensional coordinates of some reconstructed points in the three-dimensional point cloud data to obtain a preset fitting formula, including: Based on the preset fitting formula, the projection coordinates of the reconstructed point in the target direction are converted into spatial coordinates in the target direction; Solving the optimal value of the second coefficient of the preset fitting formula based on the three-dimensional point cloud data and the spatial coordinates to obtain a value of the second coefficient; Based on the value of the second coefficient, a preset fitting formula corresponding to the target direction is determined.

13. The three-dimensional reconstruction method according to claim 2, characterized in that: The interpolation processing is performed on at least part of the reconstructed points in the fitting object to obtain the three-dimensional coordinates of the target interpolation points, including: Performing interpolation processing on at least part of the reconstructed points in the fitting object to obtain first coordinates of the target interpolation point; The first coordinate is processed based on a preset fitting formula to obtain the three-dimensional coordinate of the target interpolation point; the preset fitting formula is determined by fitting the three-dimensional point cloud data.

14. The three-dimensional reconstruction method according to claim 1, characterized in that: The interpolation processing is performed on at least part of the reconstructed points in the three-dimensional point cloud data to obtain the three-dimensional coordinates of the target interpolation points, including: A linear interpolation algorithm is used to interpolate at least part of the reconstructed points in the three-dimensional point cloud data to obtain the three-dimensional coordinates of the target interpolation points.

15. The three-dimensional reconstruction method according to any one of claims 1 to 14, characterized in that: The interpolation processing is performed on at least part of the reconstructed points in the three-dimensional point cloud data to obtain the three-dimensional coordinates of the target interpolation points, including: A nearest neighbor interpolation algorithm is used to interpolate at least some of the reconstructed points based on the grayscale values ​​of at least some of the reconstructed points in the three-dimensional point cloud data to obtain the three-dimensional coordinates of the target interpolation points and the grayscale values ​​of the target interpolation points.

16. The three-dimensional reconstruction method according to any one of claims 1 to 14, characterized in that: The interpolation processing is performed on at least part of the reconstructed points in the three-dimensional point cloud data to obtain the three-dimensional coordinates of the target interpolation points, including: Based on the sampling accuracy, determine the target number of interpolation points; The three-dimensional point cloud data is interpolated based on the target number of interpolation points to obtain the target number of interpolation points and the three-dimensional coordinates of each of the target interpolation points.

17. A three-dimensional reconstruction device, characterized in that: include: A first processing module is used to construct three-dimensional point cloud data corresponding to a single image frame based on the image frame of the object to be reconstructed collected by the binocular camera in the laser scanner; The three-dimensional point cloud data includes a plurality of reconstruction points and the three-dimensional coordinates of each of the reconstruction points; the reconstruction points are spatial coordinate points obtained by three-dimensionally reconstructing the pixel points in the image frame; The second processing module is used to perform interpolation processing on at least part of the reconstructed points in the three-dimensional point cloud data to obtain the three-dimensional coordinates of the target interpolation points.

18. A laser scanner, characterized in that: include: A binocular camera, a laser group and a control device, wherein the control device is used to execute the three-dimensional reconstruction method as described in any one of claims 1 to 16.

19. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the three-dimensional reconstruction method according to any one of claims 1 to 16 is implemented.

20. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the three-dimensional reconstruction method as described in any one of claims 1 to 16 is implemented.

21. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the three-dimensional reconstruction method according to any one of claims 1 to 16 is implemented.

Images