Method and apparatus for splicing three-dimensional data, storage medium and electronic device

Abstract

The present invention provides a method and apparatus for splicing three-dimensional data, a storage medium, and an electronic device, belonging to the field of three-dimensional scanning. The method includes: determining the laser three-dimensional data corresponding to the current frame and the current frame texture map; performing three-dimensional coordinate reconstruction based on the laser three-dimensional data and a preset three-dimensional interpolation algorithm to obtain each three-dimensional point of the current frame; determining the mapping relationship between each three-dimensional point of the current frame and the current frame texture map; determining the feature matching relationship between the current frame texture map and the texture maps of other frames; determining the relative position relationship between each three-dimensional point of the current frame and each three-dimensional point of other frames according to the mapping relationship and the feature matching relationship; and performing three-dimensional data splicing on each three-dimensional point of the current frame and each three-dimensional point of other frames according to the relative position relationship. By applying the method of the present invention, the key feature points for data splicing can be identified through texture features, and three-dimensional data splicing can be completed without performing point pasting operations. The scanning process is simple, and the scanning efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional scanning, and particularly to a method and device for stitching three-dimensional data, a storage medium, and an electronic device. Background Art

[0002] A handheld laser scanner is one of the common scanning devices in three-dimensional scanning scenarios. In the application scenarios of handheld laser scanners, it is usually necessary to scan an object from different perspectives, and then stitch the three-dimensional data measured from different scanning perspectives to construct a final three-dimensional model.

[0003] Currently, it is usually artificial to deploy multiple marked points with certain identifications on the surface of the object to be scanned. When scanning the object, each marked point will be repeatedly captured from different scanning perspectives. When stitching the three-dimensional data of different scanning perspectives, according to the same marked points captured from these scanning perspectives, the three-dimensional data of these scanning perspectives is stitched.

[0004] In the existing three-dimensional data stitching method, the stitching of three-dimensional data depends on the positions of the marked points. In actual application scenarios, objects with a large surface area or a complex shape structure are often encountered, and complex point pasting operations are required, making the process of three-dimensional scanning work more cumbersome and the scanning work efficiency lower. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method for stitching three-dimensional data to solve the problem that the existing three-dimensional data stitching method depends on the deployment of marked points on the object surface, requires manual complex point pasting operations, makes the process of three-dimensional scanning work more cumbersome, and has a lower efficiency.

[0006] Embodiments of the present invention also provide a device for stitching three-dimensional data to ensure the implementation and application of the above method in practice.

[0007] To achieve the above object, embodiments of the present invention provide the following technical solutions:

[0008] A method for stitching three-dimensional data includes:

[0009] Determine the laser three-dimensional data corresponding to the current frame and the current frame texture map;

[0010] Perform three-dimensional coordinate reconstruction based on the laser three-dimensional data and a preset three-dimensional interpolation algorithm to obtain each current frame three-dimensional point;

[0011] Determine the mapping relationship between each current frame three-dimensional point and the current frame texture map;

[0012] Determine the feature matching relationship between the current frame texture map and other frame texture maps;

[0013] Determine the relative position relationship between each of the current-frame three-dimensional points and each of the other-frame three-dimensional points according to the mapping relationship and the feature matching relationship;

[0014] Perform three-dimensional data stitching on each of the current-frame three-dimensional points and each of the other-frame three-dimensional points according to the relative position relationship.

[0015] For the above method, optionally, the three-dimensional coordinate reconstruction based on the laser three-dimensional data and a preset three-dimensional interpolation algorithm to obtain each current-frame three-dimensional point includes:

[0016] Obtain the three-dimensional coordinates corresponding to each laser imaging point in the laser three-dimensional data;

[0017] Apply the three-dimensional interpolation algorithm to perform three-dimensional interpolation processing on the three-dimensional coordinates corresponding to each laser imaging point to obtain the three-dimensional coordinates corresponding to each interpolation point;

[0018] Use the three-dimensional coordinates corresponding to each laser imaging point and the three-dimensional coordinates corresponding to each interpolation point as each of the current-frame three-dimensional points.

[0019] For the above method, optionally, the determination of the feature matching relationship between the current-frame texture map and other-frame texture maps includes:

[0020] Determine each first texture feature corresponding to the current-frame texture map;

[0021] Determine each second texture feature corresponding to the other-frame texture maps;

[0022] For each of the first texture features, perform feature matching processing on this first texture feature with each of the second texture features respectively to obtain the feature matching result corresponding to this first texture feature;

[0023] Determine the feature matching relationship between the current-frame texture map and the other-frame texture maps according to the feature matching results corresponding to each of the first texture features.

[0024] For the above method, optionally, the determination of the relative position relationship between each of the current-frame three-dimensional points and each of the other-frame three-dimensional points according to the mapping relationship and the feature matching relationship includes:

[0025] Determine the target region image in the current-frame texture map according to the mapping relationship;

[0026] Determine the target texture feature in the target region image that matches the other-frame texture maps according to the feature matching relationship;

[0027] Among the current frame three-dimensional points, determine the current frame target points corresponding to the target texture features;

[0028] Among the other frame three-dimensional points, determine the other frame target points corresponding to the target texture features;

[0029] Based on the three-dimensional coordinates of the current frame target points and the three-dimensional coordinates of the other frame target points, determine the relative position relationship between each of the current frame three-dimensional points and each of the other frame three-dimensional points.

[0030] For the above method, optionally, the three-dimensional data stitching of each of the current frame three-dimensional points and each of the other frame three-dimensional points based on the relative position relationship includes:

[0031] Based on the relative position relationship, construct the initial stitching matrix corresponding to each of the current frame three-dimensional points;

[0032] According to a preset point cloud registration algorithm, perform data optimization processing on the initial stitching matrix to obtain an optimized matrix corresponding to the initial stitching matrix;

[0033] Determine the other frame three-dimensional matrices corresponding to each of the other frame three-dimensional points;

[0034] Perform stitching processing on the optimized matrix corresponding to the initial stitching matrix and the other frame three-dimensional matrices to obtain a stitching result, and use the stitching result as the three-dimensional data stitching result of each of the current frame three-dimensional points and each of the other frame three-dimensional points.

[0035] For the above method, optionally, the step of performing data optimization processing on the initial stitching matrix according to a preset point cloud registration algorithm to obtain an optimized matrix corresponding to the initial stitching matrix includes:

[0036] For each three-dimensional coordinate in the initial stitching matrix, determine whether the three-dimensional point corresponding to the three-dimensional coordinate is coordinate data obtained by laser measurement. If the three-dimensional point corresponding to the three-dimensional coordinate is coordinate data obtained by laser measurement, then use the three-dimensional coordinate as the original coordinate; if the three-dimensional point corresponding to the three-dimensional coordinate is not coordinate data obtained by laser measurement, then use the three-dimensional coordinate as the derived coordinate;

[0037] Use the preset first weight as the optimization weight corresponding to each of the original coordinates, and use the preset second weight as the optimization weight corresponding to each of the derived coordinates; the first weight is greater than the second weight;

[0038] Based on the optimization weights corresponding to the three-dimensional coordinates in the initial stitching matrix, apply the iterative closest point algorithm to perform three-dimensional data optimization on the initial stitching matrix, and use the optimization result as the optimized matrix corresponding to the initial stitching matrix.

[0039] Optionally, the above method further includes:

[0040] After the three-dimensional scanning process is completed, determine the global stitching matrix;

[0041] Perform global optimization on the global stitching matrix to obtain a globally optimized matrix, and use the globally optimized matrix as the global three-dimensional matrix of the scanned object.

[0042] A three-dimensional data stitching device, comprising:

[0043] A first determination unit for determining the laser three-dimensional data corresponding to the current frame and the current frame texture map;

[0044] A coordinate reconstruction unit for performing three-dimensional coordinate reconstruction based on the laser three-dimensional data and a preset three-dimensional interpolation algorithm to obtain each current frame three-dimensional point;

[0045] A second determination unit for determining the mapping relationship between each of the current frame three-dimensional points and the current frame texture map;

[0046] A third determination unit for determining the feature matching relationship between the current frame texture map and other frame texture maps;

[0047] A fourth determination unit for determining the relative position relationship between each of the current frame three-dimensional points and each of the other frame three-dimensional points according to the mapping relationship and the feature matching relationship;

[0048] A data stitching unit for performing three-dimensional data stitching on each of the current frame three-dimensional points and each of the other frame three-dimensional points according to the relative position relationship.

[0049] A storage medium, the storage medium includes stored instructions, wherein when the instructions run, they control the device where the storage medium is located to execute the three-dimensional data stitching method as described above.

[0050] An electronic device, comprising a memory, and one or more instructions, wherein one or more instructions are stored in the memory and are configured to be executed by one or more processors to execute the three-dimensional data stitching method as described above.

[0051] A method for stitching three-dimensional data provided by the embodiments of the present invention includes: determining the laser three-dimensional data corresponding to the current frame and the current frame texture map; performing three-dimensional coordinate reconstruction based on the laser three-dimensional data and a preset three-dimensional interpolation algorithm to obtain each three-dimensional point of the current frame; determining the mapping relationship between each three-dimensional point of the current frame and the current frame texture map; determining the feature matching relationship between the current frame texture map and other frame texture maps; determining the relative position relationship between each three-dimensional point of the current frame and each three-dimensional point of other frames according to the mapping relationship and the feature matching relationship; and performing three-dimensional data stitching on each three-dimensional point of the current frame and each three-dimensional point of other frames according to the relative position relationship. By applying the method provided by the embodiments of the present invention, each three-dimensional point corresponding to the current frame can be mapped to the texture map corresponding to the current frame, and the texture map of the current frame can be feature-matched with the texture maps of other frames. According to the mapping relationship between the three-dimensional points of the current frame and the texture map of the current frame and the feature matching relationship between the texture map of the current frame and the texture maps of other frames, the three-dimensional points associated with the three-dimensional points of the current frame can be found among the three-dimensional points of other frames, thereby determining the relative position relationship between the three-dimensional points of the current frame and the three-dimensional points of other frames, and thus realizing three-dimensional data stitching. During the data stitching process, the key feature points of the data stitching can be identified through texture features, and it is not necessary to complete the three-dimensional data stitching through the fiducial points deployed on the object surface. Therefore, no manual pasting operation is required in the three-dimensional scanning work, the scanning process is relatively simple, which is beneficial to improving the scanning efficiency and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0053] Figure 1 It is a flowchart of a method for stitching three-dimensional data provided by the embodiments of the present invention;

[0054] Figure 2 It is a schematic structural diagram of a laser scanner provided by the embodiments of the present invention;

[0055] Figure 3 It is a schematic diagram of the three-dimensional data stitching process during the three-dimensional scanning provided by the embodiments of the present invention;

[0056] Figure 4 It is a schematic structural diagram of a three-dimensional data stitching device provided by the embodiments of the present invention;

[0057] Figure 5 It is a schematic structural diagram of an electronic device provided by the embodiments of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0058] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0059] In this application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "including one..." does not exclude the presence of additional identical elements in the process, method, article or device including the said element.

[0060] The embodiment of the present invention provides a method for stitching three-dimensional data. The method can be applied to a data processing system of a laser scanner, and its execution subject can be a processor of the data processing system. The method flowchart of the method is as Figure 1 shown and includes:

[0061] S101: Determine the laser three-dimensional data corresponding to the current frame and the current frame texture map;

[0062] The method provided by the embodiments of the present invention can be applied to the scanning process of a handheld laser scanner. The user can hold the scanner and continuously scan the object to be scanned from different scanning perspectives. The scanner can collect corresponding data at different scanning positions for measuring the three-dimensional data of the corresponding scanning positions, and splice the three-dimensional data of each scanning position to obtain the complete point cloud data of the object. In the method provided by the embodiments of the present invention, the three-dimensional data splicing is achieved by combining the laser three-dimensional data and the texture map on the object surface. Therefore, when the scanner performs the scanning operation, it is necessary to collect the two-dimensional laser line map and the texture map on the object surface. The two-dimensional laser line map refers to the image obtained by using a black-and-white camera to photograph the object surface when a laser beam is projected onto the object surface. The texture map refers to the image obtained by using a color texture camera to photograph the object surface when no laser beam is projected onto the object surface. Specifically, the scanner can be equipped with a laser projector, a black-and-white camera, and a color texture camera. When performing a scanning operation on the current position of the object, the laser projector projects a laser beam onto the surface of the object to be scanned, and the black-and-white camera photographs the current position of the object to obtain the two-dimensional laser line map of the current position of the object. Then, the projection of the laser beam is stopped, and the color texture camera photographs the current position of the object to obtain the texture map of the current position of the object. The two-dimensional laser line map and the texture map collected for the current position of the object during this operation process are used as the image data of the same frame. It should be noted that during the collection process of the two-dimensional laser line map, multiple black-and-white cameras can be used to photograph the object surface simultaneously. There can be multiple two-dimensional laser line maps corresponding to the current frame, and subsequent three-dimensional coordinate measurements can be combined with multiple two-dimensional laser line maps.

[0063] The data processing system of the scanner can process the collected image data in real time during the scanning process. For example, when the scanning starts and the first-frame two-dimensional laser line map and texture map are collected, the three-dimensional data corresponding to the first frame can be measured according to the image data of the first frame. When the second-frame two-dimensional laser line map and texture map are received, the three-dimensional data corresponding to the second frame can be measured according to the image data of the second frame, and then the three-dimensional data of the first frame and the three-dimensional data of the second frame are spliced to obtain the current splicing data. When the third-frame two-dimensional laser line map and texture map are received, the three-dimensional data corresponding to the third frame is further spliced with the existing splicing data. The subsequent processing process is the same.

[0064] It should be noted that the deployment quantity and deployment position of the laser projector, black-and-white camera, and color physical camera in the scanner can be set according to actual needs, as long as the image collection of the two-dimensional laser line map and the texture map on the object surface can be achieved. The structure of the scanner does not affect the implementation of the functions of the method provided by the embodiments of the present invention.

[0065] In the method provided by the embodiments of the present invention, when it is necessary to process the image data of a certain frame, this frame can be used as the current frame. The system can obtain the two-dimensional laser line map corresponding to the current frame and the texture map corresponding to the current frame (i.e., the current frame texture map), and measure the three-dimensional coordinates of the laser line imaging points at the corresponding object surface positions according to the two-dimensional laser line map corresponding to the current frame, so as to obtain the laser three-dimensional data of the current frame. Specifically, the principle of laser triangulation can be used to calculate the three-dimensional coordinates of the object surface positions corresponding to the laser imaging points in the two-dimensional laser line map. That is, the positions of the camera for shooting the two-dimensional laser line map and the laser projector are known. When the laser beam is projected onto the object surface, a corresponding laser line contour will be formed on the object surface. The laser line contour will present a corresponding shape due to the surface structure of the object. That is to say, the position change of the light contour of the laser line reflects the positions at different heights of the laser line on the object surface. Therefore, according to the position of the laser line contour of the laser imaging in the two-dimensional laser line map, combined with the positions of the laser projector and the camera, the three-dimensional coordinates of each point where the laser line is imaged on the object surface can be calculated, that is, the three-dimensional coordinates of the corresponding positions on the object surface, so as to obtain the laser three-dimensional data corresponding to the current frame, that is, the three-dimensional coordinates of each laser imaging point.

[0066] S102: Perform three-dimensional coordinate reconstruction based on the laser three-dimensional data and a preset three-dimensional interpolation algorithm to obtain each current frame three-dimensional point;

[0067] In the method provided by the embodiments of the present invention, on the basis of the laser three-dimensional data, the three-dimensional data is further interpolated and optimized in combination with the three-dimensional interpolation algorithm, so as to perform three-dimensional coordinate reconstruction on the scanning area, and the three-dimensional data after interpolation and optimization is used as each current frame three-dimensional point corresponding to the current frame.

[0068] S103: Determine the mapping relationship between each of the current frame three-dimensional points and the current frame texture map;

[0069] In the method provided by the embodiments of the present invention, the mapping relationship between each pixel point in the two-dimensional laser line map and each pixel point in the texture map can be determined according to the relative positions of the camera for shooting the two-dimensional laser line map and the camera for shooting the texture map. And each current frame three-dimensional point is the three-dimensional coordinate calculated based on the two-dimensional laser line map. Each current frame three-dimensional point actually represents the position of a certain object surface point in the two-dimensional laser line map. Therefore, there is also a corresponding mapping relationship between each current frame three-dimensional point and each pixel point in the two-dimensional laser line map. According to the mapping relationship between each current frame three-dimensional point and the pixel points in the two-dimensional laser line map, and the mapping relationship between the pixel points in the two-dimensional laser line map and the pixel points in the current frame texture map, the mapping relationship between each current frame three-dimensional point and the pixel points in the current frame texture map can be converted, so as to obtain the mapping relationship between each current frame three-dimensional point and the current frame texture map.

[0070] S104: Determine the feature matching relationship between the current frame texture map and other frame texture maps;

[0071] In the method provided by the embodiments of the present invention, the data frame that needs to be stitched with the three-dimensional data of the current frame currently is used as other frames. The other frames can be the previous frame or the next frame of the current frame, or can be data frames with a certain interval from the current frame, or can be global frames obtained by stitching the existing three-dimensional data currently. During the data processing of other frames, each three-dimensional point and texture map corresponding to the other frames will also be obtained. Each three-dimensional point corresponding to the other frames is regarded as each other-frame three-dimensional point, and the texture map corresponding to the other frames is regarded as other-frame texture maps.

[0072] It should be noted that if there is no three-dimensional data that needs to be stitched with the three-dimensional data corresponding to the current frame currently, then a corresponding three-dimensional data matrix is constructed based on each current-frame three-dimensional point corresponding to the current frame, and wait to be stitched with subsequent three-dimensional data.

[0073] In the method provided by the embodiments of the present invention, each texture feature corresponding to the current frame texture map can be feature-matched with each texture feature corresponding to the other frame texture maps to determine the feature matching relationship between the current frame texture map and the other frame texture maps. This feature matching relationship characterizes whether the texture feature in the current frame texture map matches a certain texture feature in the other frame texture maps.

[0074] S105: Determine the relative position relationship between each of the current-frame three-dimensional points and each of the other-frame three-dimensional points according to the mapping relationship and the feature matching relationship;

[0075] In the method provided by the embodiments of the present invention, the corresponding relationship between the current-frame three-dimensional points and the texture features in the current frame texture map can be determined according to the mapping relationship between the current-frame three-dimensional points and the current frame texture map. According to this corresponding relationship and the feature matching relationship between the current frame texture map and the other frame texture maps, the mutually related current-frame three-dimensional points and other-frame three-dimensional points can be found among each of the current-frame three-dimensional points and each of the other-frame three-dimensional points. According to the coordinate transformation relationship between the mutually related current-frame three-dimensional points and other-frame three-dimensional points, the relative position relationship between the current-frame three-dimensional points and the other-frame three-dimensional points can be calculated.

[0076] S106: Perform three-dimensional data stitching on each of the current-frame three-dimensional points and each of the other-frame three-dimensional points according to the relative position relationship.

[0077] In the method provided by the embodiments of the present invention, a preset data fusion algorithm can be applied to perform data fusion on each current-frame three-dimensional point and other-frame three-dimensional points according to the relative position relationship between the current-frame three-dimensional points and the other-frame three-dimensional points, and use the data fusion result as the data stitching result corresponding to the current frame. The specific data fusion algorithm can be set according to actual requirements. For example, the data fusion algorithm can be set based on a machine learning model, can be set based on the Iterative Closest Point (ICP) algorithm, can be set based on Gaussian Mixture Model (GMM) registration, or can be set based on other methods capable of realizing three-dimensional data fusion, without affecting the function implementation of the method provided by the embodiments of the present invention.

[0078] Based on the method provided by the embodiments of the present invention, when it is necessary to stitch the three-dimensional data corresponding to the current frame with the previous three-dimensional data, determine the laser three-dimensional data corresponding to the current frame and the current-frame texture map; perform three-dimensional coordinate reconstruction based on the laser three-dimensional data and a preset three-dimensional interpolation algorithm to obtain each current-frame three-dimensional point; determine the mapping relationship between each current-frame three-dimensional point and the current-frame texture map; determine the feature matching relationship between the current-frame texture map and other-frame texture maps; based on the mapping relationship and the feature matching relationship, determine the relative position relationship between each current-frame three-dimensional point and each other-frame three-dimensional point; and perform three-dimensional data stitching on each current-frame three-dimensional point and each other-frame three-dimensional point according to the relative position relationship. By applying the method provided by the embodiments of the present invention, each three-dimensional point corresponding to the current frame can be mapped to the current-frame texture map, and the current-frame texture map can be feature-matched with other-frame texture maps. According to the mapping relationship between the current-frame three-dimensional points and the current-frame texture map and the feature matching relationship between the current-frame texture map and other-frame texture maps, the three-dimensional points associated with the current-frame three-dimensional points can be found among the other-frame three-dimensional points, thereby determining the relative position relationship between the current-frame three-dimensional points and the other-frame three-dimensional points, and thus realizing three-dimensional data stitching. During the data stitching process, the key feature points of the data stitching can be identified through texture features, and there is no need to complete three-dimensional data stitching through the landmark points deployed on the object surface. Therefore, there is no need for manual point-pasting operation in the three-dimensional scanning work, the scanning process is relatively simple, which is beneficial to improving the scanning efficiency and improving the user experience.

[0079] In Figure 1 Based on the method shown above, in the method provided by the embodiments of the present invention, the process of performing three-dimensional coordinate reconstruction based on the laser three-dimensional data and a preset three-dimensional interpolation algorithm to obtain each current-frame three-dimensional point mentioned in step S102 includes:

[0080] Obtain the three-dimensional coordinates corresponding to each laser imaging point in the laser three-dimensional data;

[0081] In the method provided by the embodiments of the present invention, in the process of determining the laser three-dimensional data corresponding to the current frame, first, based on the two-dimensional laser line diagram corresponding to the current frame, the laser triangulation method can be applied to measure the three-dimensional coordinates of each laser imaging point in the two-dimensional laser line diagram, that is, calculate the three-dimensional coordinates corresponding to each laser imaging point. According to the positions of each laser imaging point in the two-dimensional laser line diagram and their corresponding three-dimensional coordinates, a depth map is reconstructed to obtain an original depth map containing the three-dimensional coordinates corresponding to each laser imaging point, and the laser three-dimensional data is represented in the form of the original depth map. It can be understood that each pixel point in the original depth map corresponds one-to-one with each pixel point in the two-dimensional laser line diagram. The pixel points in the two-dimensional laser line diagram may be laser imaging points or may not be laser imaging points. Each pixel point in the original depth map represents a set of depth data, that is, three-dimensional coordinates. Regarding the pixel points in the original depth map, if the pixel point corresponds to a laser imaging point, then the depth data of the pixel point is the three-dimensional coordinate corresponding to the laser imaging point. If the pixel point does not correspond to a laser imaging point, then the depth data of the pixel point is not a valid three-dimensional coordinate and can be configured as a null value.

[0082] Apply the three-dimensional interpolation algorithm to perform three-dimensional interpolation processing on the three-dimensional coordinates corresponding to each of the laser imaging points to obtain the three-dimensional coordinates corresponding to each interpolation point;

[0083] In the method provided by the embodiments of the present invention, based on the three-dimensional coordinates corresponding to each laser imaging point, a preset three-dimensional interpolation algorithm can be applied to perform three-dimensional interpolation processing on the depth data in the original depth map, that is, according to the existing three-dimensional coordinates in the original depth map, solve the depth data of the pixel points without valid three-dimensional coordinates, so as to obtain the three-dimensional coordinates corresponding to each interpolation point in the original depth map. The preset three-dimensional interpolation algorithm can be set according to actual needs. For example, polynomial fitting can be used for three-dimensional interpolation, the nearest neighbor interpolation algorithm can be used for three-dimensional interpolation, or other three-dimensional interpolation algorithms can be used, which does not affect the implementation of the functions of the method provided by the embodiments of the present invention.

[0084] Take the three-dimensional coordinates corresponding to each of the laser imaging points and the three-dimensional coordinates corresponding to each of the interpolation points as each of the current frame three-dimensional points.

[0085] In the method provided by the embodiments of the present invention, the three-dimensional coordinates corresponding to each laser imaging point calculated through three-dimensional coordinate measurement and the three-dimensional coordinates corresponding to each interpolation point calculated through three-dimensional interpolation are used as the three-dimensional data corresponding to the current frame, that is, each of the current frame three-dimensional points.

[0086] Based on the method provided by the embodiments of the present invention, during the process of reconstructing three-dimensional data, the original three-dimensional data can be interpolated and optimized by means of three-dimensional interpolation, which can make the final three-dimensional data smoother, facilitate splicing with other three-dimensional data, and improve the accuracy of data splicing.

[0087] Based on Figure 1 On the basis of the method shown, in the method provided by the embodiments of the present invention, the process of determining the feature matching relationship between the current frame texture map and other frame texture maps mentioned in step S104 includes:

[0088] Determine each first texture feature corresponding to the current frame texture map;

[0089] In the method provided by the embodiments of the present invention, a preset texture feature analysis method can be applied to analyze the texture information in the current frame texture map to identify the texture features of the surface positions of each object in the current frame texture map. Specifically, global analysis can be performed on the current frame texture map, or according to the mapping relationship between the current frame three-dimensional points and the current frame texture map, texture feature analysis can be performed only on the texture map area mapped by the current frame three-dimensional points. Each texture feature in the current frame texture map is used as each first texture feature. The preset texture feature analysis method can be set according to actual needs. For example, machine learning algorithms can be used to extract texture features in images, algorithms such as Local Binary Patterns algorithms for extracting image texture features can be used to extract texture features, or other algorithms capable of realizing texture feature extraction can be used, which does not affect the implementation of the functions of the method provided by the embodiments of the present invention.

[0090] Determine each second texture feature corresponding to the other frame texture maps;

[0091] In the method provided by the embodiments of the present invention, during the processing of other frame texture maps, the preset texture feature analysis method is also applied to analyze the texture information in the other frame texture maps, and each texture feature in the other frame texture maps is used as the second texture feature.

[0092] For each of the first texture features, perform feature matching processing on this first texture feature with each of the second texture features respectively to obtain the feature matching result corresponding to this first texture feature;

[0093] In the method provided by the embodiments of the present invention, for each first texture feature, perform feature matching processing on this first texture feature with each of the second texture features respectively. If there is a second texture feature that matches this first texture feature, establish the matching relationship between this first texture feature and its matching second texture feature, and thus obtain the feature matching result of the first texture feature, that is, the result indicating whether this first texture feature matches each of the second texture features.

[0094] Determine the feature matching relationship between the current frame texture map and the other frame texture maps according to the feature matching results corresponding to each of the first texture features.

[0095] In the method provided by the embodiments of the present invention, according to the feature matching results of whether each first texture feature matches each second texture feature, determine the feature matching relationship between the current frame texture map and the other frame texture maps.

[0096] In Figure 1 Based on the method shown, in the method provided by the embodiments of the present invention, the process of determining the relative position relationship between each of the current frame three-dimensional points and each of the other frame three-dimensional points according to the mapping relationship and the feature matching relationship mentioned in step S105 includes:

[0097] Determine the target area image in the current frame texture map according to the mapping relationship;

[0098] In the method provided by the embodiments of the present invention, according to each current frame three-dimensional point, in the current frame texture map, find the image corresponding to the object surface area represented by each current frame three-dimensional point, and use the image of this area as the target area image. It can be understood that each current frame three-dimensional point represents the three-dimensional coordinates of the points in the corresponding object surface area, and the target area image is the image corresponding to the object surface area in the current frame texture map.

[0099] In the method provided by the embodiments of the present invention, for each pixel point in the current frame texture map, according to the mapping relationship between each current frame three-dimensional point and the texture image pixel point, determine whether the pixel point is a pixel point corresponding to the three-dimensional point. If the pixel point is a pixel point corresponding to the three-dimensional point, then use this pixel point as the target pixel point. Use the image area corresponding to each target pixel point in the current frame texture map as the target area image.

[0100] Determine the target texture feature in the target area image that matches the other frame texture maps according to the feature matching relationship;

[0101] In the method provided by the embodiments of the present invention, according to the feature matching relationship between the current frame texture map and the other frame texture maps, determine the feature matching results of each texture feature in the target area image and the other frame texture maps. For each texture feature in the target area image, if the texture feature matches a certain texture feature in the other frame texture maps, then use this texture feature in the target area image as the target texture feature.

[0102] Among each of the current frame three-dimensional points, determine the current frame target point corresponding to the target texture feature;

[0103] In the method provided by the embodiment of the present invention, according to the mapping relationship between the three-dimensional points of the current frame and the texture map of the current frame and the corresponding relationship between the target texture feature and the texture map of the current frame, the three-dimensional points of the current frame corresponding to the target texture feature can be determined, and the three-dimensional points of the current frame corresponding to the target texture feature are used as the target points of the current frame.

[0104] Among all the other-frame three-dimensional points, determine the other-frame target points corresponding to the target texture feature;

[0105] In the method provided by the embodiment of the present invention, the texture feature matching the target texture feature in the other-frame texture map can be used as the associated texture feature corresponding to the target texture feature. It can be understood that similar to the mapping relationship between each current-frame three-dimensional point and the texture feature of the current-frame texture map, there is also a corresponding mapping relationship between each other-frame three-dimensional point and the texture feature in the other-frame texture map. According to the mapping relationship between the other-frame three-dimensional point and the other-frame texture map, the other-frame three-dimensional points mapped by the associated texture feature corresponding to the target texture feature can be determined, and the other-frame three-dimensional points corresponding to the associated texture feature are used as the other-frame target points corresponding to the target texture feature.

[0106] Based on the three-dimensional coordinates of the current-frame target point and the three-dimensional coordinates of the other-frame target points, determine the relative position relationship between each current-frame three-dimensional point and each other-frame three-dimensional point.

[0107] In the method provided by the embodiment of the present invention, according to the three-dimensional coordinates of the current-frame target point and the three-dimensional point coordinates of the other-frame target points, the coordinate conversion relationship between the current-frame target point and the other-frame target points can be determined, and the relative position relationship between the current-frame three-dimensional points and the other-frame three-dimensional points can be determined according to this coordinate conversion relationship.

[0108] In Figure 1 Based on the method shown, in the method provided by the embodiment of the present invention, the process of performing three-dimensional data stitching on each current-frame three-dimensional point and each other-frame three-dimensional point according to the relative position relationship mentioned in step S106 includes:

[0109] Construct an initial stitching matrix corresponding to each current-frame three-dimensional point according to the relative position relationship;

[0110] In the method provided by the embodiment of the present invention, the three-dimensional coordinates of each current-frame three-dimensional point can be converted according to the relative position relationship between the current-frame three-dimensional point and the other-frame three-dimensional point, so as to align each current-frame three-dimensional point and each other-frame three-dimensional point to the same coordinate system. The current-frame three-dimensional point after coordinate conversion is used as the converted three-dimensional point, and a corresponding three-dimensional data matrix is constructed based on the three-dimensional coordinates of each converted three-dimensional point. The constructed three-dimensional data matrix is used as the initial stitching matrix.

[0111] According to a preset point cloud registration algorithm, perform data optimization processing on the initial stitching matrix to obtain an optimized matrix corresponding to the initial stitching matrix;

[0112] In the method provided by the embodiments of the present invention, during the three-dimensional data stitching process, a preset point cloud registration algorithm can be first applied to perform registration optimization on the three-dimensional data in the initial stitching matrix, and the initial stitching matrix after registration optimization is used as the optimized matrix. The preset point cloud registration algorithm can be set according to actual needs. For example, the iterative closest point (ICP) algorithm can be used as the point cloud registration algorithm, a point cloud registration algorithm based on a deep learning network can be used, or other point cloud registration methods can be used, which does not affect the implementation of the function of the method provided by the embodiments of the present invention.

[0113] Determine other frame three-dimensional matrices corresponding to each of the other frame three-dimensional points;

[0114] In the method provided by the embodiments of the present invention, a three-dimensional data matrix composed of each of the other frame three-dimensional points is used as the other frame three-dimensional matrix.

[0115] Perform stitching processing on the optimized matrix corresponding to the initial stitching matrix and the other frame three-dimensional matrix to obtain a stitching result, and use the stitching result as the three-dimensional data stitching result of each of the current frame three-dimensional points and each of the other frame three-dimensional points.

[0116] In the method provided by the embodiments of the present invention, data fusion is performed on the optimized matrix of the initial stitching matrix and the other frame three-dimensional matrix to achieve data stitching, and the matrix obtained by stitching the optimized matrix and the other frame three-dimensional matrix (i.e., the stitching result) is used as the three-dimensional data stitching result corresponding to the current frame.

[0117] Based on the method provided by the embodiments of the present invention, the initial stitching matrix can be further registered and optimized through a point cloud registration algorithm, and the optimized matrix is fused with the other frame three-dimensional matrix, which can further improve the accuracy of data stitching.

[0118] On the basis of the method provided in the above embodiments, in the method provided by the embodiments of the present invention, the process of performing data optimization processing on the initial stitching matrix according to a preset point cloud registration algorithm to obtain an optimized matrix corresponding to the initial stitching matrix includes:

[0119] For each three-dimensional coordinate in the initial stitching matrix, determine whether the three-dimensional point corresponding to the three-dimensional coordinate is coordinate data obtained by laser measurement. If the three-dimensional point corresponding to the three-dimensional coordinate is coordinate data obtained by laser measurement, then use the three-dimensional coordinate as the original coordinate; if the three-dimensional point corresponding to the three-dimensional coordinate is not coordinate data obtained by laser measurement, then use the three-dimensional coordinate as the derived coordinate;

[0120] In the method provided by the embodiment of the present invention, during the three-dimensional coordinate reconstruction mentioned in step S102, each three-dimensional point is obtained by performing three-dimensional interpolation on the laser three-dimensional data. That is, among the finally obtained current-frame three-dimensional points, it includes the three-dimensional coordinates of each laser imaging point obtained by laser measurement, and also includes the three-dimensional coordinates of each interpolation point obtained by three-dimensional interpolation.

[0121] During the process of optimizing the initial stitching matrix, for each three-dimensional coordinate in the initial stitching matrix, it is determined whether the three-dimensional point corresponding to the three-dimensional coordinate is coordinate data obtained by laser measurement, that is, it is identified whether the three-dimensional coordinate is a coordinate converted from the three-dimensional coordinates of the laser imaging point. If the three-dimensional point corresponding to the three-dimensional coordinate is coordinate data obtained by laser measurement, then the three-dimensional coordinate is marked as an original coordinate. If the three-dimensional point corresponding to the three-dimensional coordinate is not coordinate data obtained by laser measurement, that is, the three-dimensional point corresponding to the three-dimensional coordinate is coordinate data obtained by three-dimensional interpolation, then the three-dimensional coordinate is marked as a derived coordinate. Overall, that is, for each three-dimensional coordinate in the initial stitching matrix, the three-dimensional coordinate converted from the three-dimensional coordinates of the laser imaging point is used as the original coordinate, and the three-dimensional coordinate converted from the three-dimensional coordinates of the interpolation point is used as the derived coordinate.

[0122] Respectively use the preset first weight as the optimization weight corresponding to each of the original coordinates, and use the preset second weight as the optimization weight corresponding to each of the derived coordinates; the first weight is greater than the second weight;

[0123] In the method provided by the embodiment of the present invention, two weights can be preset in advance, that is, the first weight and the second weight, where the first weight is greater than the second weight. During the data optimization process, the first weight is used as the optimization weight corresponding to the original coordinate, and the second weight is used as the optimization weight corresponding to the derived coordinate, that is, a relatively high optimization weight is given to the original coordinate, and a relatively low optimization weight is given to the derived coordinate.

[0124] Based on the optimization weights corresponding to each three-dimensional coordinate in the initial stitching matrix, apply the iterative closest point algorithm to perform three-dimensional data optimization on the initial stitching matrix, and use the optimization result as the optimized matrix corresponding to the initial stitching matrix.

[0125] In the method provided by the embodiment of the present invention, based on the optimization weights corresponding to each three-dimensional coordinate (original coordinate and derived coordinate) in the initial stitching matrix, apply the iterative closest point (ICP) algorithm to register and optimize the three-dimensional coordinates in the initial stitching matrix with the corresponding weights, and use the matrix obtained after optimization (that is, the optimization result) as the optimized matrix corresponding to the initial stitching matrix.

[0126] Based on the method provided by the embodiments of the present invention, in the case of constructing three-dimensional coordinates by combining the laser measurement principle and the three-dimensional interpolation method, different weights are assigned to the three-dimensional data obtained by laser measurement and the three-dimensional data of the interpolation points during the point cloud registration process, making the point cloud registration focus more on the three-dimensional data obtained by actual measurement, which is beneficial to improving the accuracy of point cloud registration.

[0127] On the basis of the method shown in Figure 1 the method provided by the embodiments of the present invention further includes:

[0128] After the three-dimensional scanning process is completed, determine the global stitching matrix;

[0129] In the method provided by the embodiments of the present invention, during the three-dimensional scanning process, the system can process the image data corresponding to each frame in sequence based on Figure 1 the method shown to obtain the three-dimensional data corresponding to each frame, and stitch the three-dimensional data of each frame. After the overall three-dimensional scanning process is completed, a three-dimensional matrix obtained by stitching all the three-dimensional data will be obtained. At this time, the system can use the finally stitched three-dimensional matrix as the global stitching matrix.

[0130] Perform global optimization on the global stitching matrix to obtain a globally optimized matrix, and use the globally optimized matrix as the global three-dimensional matrix of the scanned object.

[0131] In the method provided by the embodiments of the present invention, the three-dimensional data in the global stitching matrix can be globally optimized through a preset global optimization method to obtain a globally optimized matrix, and the globally optimized matrix is used as the global three-dimensional matrix of the scanned object, that is, the point cloud data of the whole scanned object. For example, the principle of assigning different weights to different types of three-dimensional coordinates and applying the ICP algorithm for data optimization mentioned in the previous embodiments can be adopted to optimize the data in the global stitching matrix. Specifically, for each three-dimensional coordinate in the global stitching matrix, it can be determined whether the three-dimensional coordinate is coordinate data obtained based on laser measurement. If the three-dimensional coordinate is coordinate data obtained based on laser measurement, the three-dimensional coordinate is marked as the original global coordinate. If the three-dimensional coordinate is not coordinate data obtained based on laser measurement, that is, the three-dimensional coordinate is coordinate data obtained based on three-dimensional interpolation, the three-dimensional coordinate is marked as the derived global coordinate. The preset third weight is used as the optimization weight corresponding to each original global coordinate, and the preset fourth weight is used as the optimization weight corresponding to each derived global coordinate, where the third weight is greater than the fourth weight. Based on the optimization weights corresponding to each three-dimensional coordinate in the global stitching matrix, global refinement and weighted fusion are performed on the global stitching matrix to achieve global optimization.

[0132] Based on the method provided by the embodiments of the present invention, after obtaining the global stitching matrix, the global stitching matrix can be further globally optimized, which can further improve the accuracy of data stitching.

[0133] To better illustrate the method provided by the embodiments of the present invention, based on the methods provided in the foregoing embodiments and in combination with an actual application scenario, the embodiments of the present invention provide another method for stitching three-dimensional data. The method provided by the embodiments of the present invention is applied to a handheld laser scanner, and a laser projector, a color texture camera, and two or more (more than two) black-and-white cameras are provided on the laser scanner. The structure of the laser scanner can be as Figure 2 shown. In the laser scanner, there are a black-and-white camera 201, a color texture camera 202, a laser projector 203, and a black-and-white camera 204. A white LED lamp is deployed on the color texture camera 202. It should be noted that Figure 2 is only a brief schematic illustration for better explaining the embodiments of the present invention, and it is only one embodiment, and does not limit the actual device structure and form. In an actual application scenario, the structure of the laser scanner can be designed according to actual needs.

[0134] In the embodiments of the present invention, the laser projector of the laser scanner can project multi-line lasers. The number of laser beams is preferably not less than 15, and the imaging spacing between the beams on the image is preferably within 8 pixels. Next, an example of the image acquisition method during scanning will be given. For example, if the laser band projected by the laser projector is the infrared band (the color texture camera can filter the band), then during scanning, the laser projector projects laser beams onto the object surface, and each black-and-white camera takes pictures of the object surface to obtain a two-dimensional laser line image. At the same time, the color texture camera takes pictures of the object surface to obtain a texture image, and the LED lamp of the color texture camera can blink synchronously to shield the interference of ambient light. If the laser band projected by the laser projector is a band that the color texture camera cannot filter, then during scanning, first the laser projector projects laser beams onto the object surface, and each black-and-white camera takes pictures of the object surface to obtain a two-dimensional laser line image. Then the laser projector is turned off, and within a certain time (for example, within 5 ms), the color texture camera takes pictures of the object surface to obtain a texture image, and the LED lamp of the color texture camera can blink synchronously to shield the interference of ambient light. The two-dimensional laser line image and the texture image collected during a single data acquisition process (that is, the black-and-white camera and the color texture camera take pictures of the same area once) according to the foregoing image acquisition method are used as the image data of the same frame. After obtaining the image data of each frame, the data processing system processes the image data for stitching three-dimensional data.

[0135] In the embodiments of the present invention, the process of stitching three-dimensional data can be as Figure 3As shown in the figure, specifically including:

[0136] S301: Obtain the two-dimensional laser line map and texture map corresponding to the current frame;

[0137] In the method provided by the embodiment of the present invention, when it is necessary to process the image data corresponding to the current frame, the two-dimensional laser line map and texture map corresponding to the current frame can be obtained.

[0138] S302: Reconstruct the depth map based on the two-dimensional laser line map;

[0139] In the method provided by the embodiment of the present invention, three-dimensional coordinate reconstruction can be performed based on the two-dimensional laser line map to obtain a depth map containing the three-dimensional coordinates of each laser imaging point.

[0140] S303: Perform three-dimensional interpolation processing on the basis of the depth map to obtain a three-dimensional data set, and mark the original data points and interpolation data points in the three-dimensional data set;

[0141] In the method provided by the embodiment of the present invention, interpolation processing can be performed based on the three-dimensional coordinates of each laser imaging point in the depth map to obtain a three-dimensional data set. Specifically, polynomial fitting can be used for three-dimensional interpolation processing. Mark the three-dimensional coordinates of each laser imaging point in the three-dimensional data set as the original data points, and mark the three-dimensional coordinates of each interpolation point obtained by three-dimensional interpolation in the three-dimensional data set as the interpolation data points. Through three-dimensional interpolation, the depth data of the depth map can be improved, which is beneficial to making the subsequent stitching process smoother and more fluent.

[0142] S304: Map the three-dimensional data set to the texture map;

[0143] In the method provided by the embodiment of the present invention, according to the mapping relationship between the two-dimensional laser line map and the three-dimensional data set, and the mapping relationship between the two-dimensional laser line map and the texture map, the three-dimensional data set is mapped to the texture map.

[0144] S305: Identify the image area in the texture map that has a mapping relationship with the three-dimensional data set, and extract texture features from this image area to obtain each texture feature of the current frame;

[0145] In the method provided by the embodiment of the present invention, the image area mapped to the three-dimensional data set is extracted in the texture map, and texture features are extracted based on the texture information of this image area. The extracted texture features are used as the texture features of the current frame.

[0146] S306: Perform texture feature matching between each texture feature of the current frame and each previous texture feature to determine the target texture feature among each texture feature of the current frame;

[0147] In the method provided by the embodiments of the present invention, each texture feature extracted during the previous data processing process (such as the texture feature of the previous frame or the current global texture feature) is matched with each texture feature of the current frame. If there is a texture feature in each texture feature of the current frame that matches the previous texture feature, then this texture feature is used as the target texture feature.

[0148] S307: Obtain the three-dimensional data corresponding to the target texture feature in the three-dimensional dataset, and solve the initial stitching matrix based on the three-dimensional data corresponding to the target texture feature;

[0149] In the method provided by the embodiments of the present invention, according to the mapping relationship between the three-dimensional data in the three-dimensional dataset and each texture feature of the current frame, the three-dimensional data corresponding to the target texture feature is extracted in the three-dimensional dataset. The three-dimensional data corresponding to the target texture feature is used as the key feature points for data stitching, and the initial stitching matrix corresponding to the three-dimensional dataset is solved accordingly.

[0150] S308: Assign high weights to the data associated with the original data points in the initial stitching matrix, assign low weights to the data associated with the interpolated data points, and optimize the data of the initial stitching matrix based on this to obtain the optimized stitching matrix;

[0151] In the method provided by the embodiments of the present invention, the optimization weight of the three-dimensional data associated with the original data points in the initial stitching matrix is set to the first weight, and the optimization weight of the three-dimensional data associated with the interpolated data points is set to the second weight, where the first weight is greater than the second weight. Based on the optimization weights of the three-dimensional data, the ICP algorithm is applied to optimize the three-dimensional data in the initial stitching matrix in real time to obtain the optimized stitching matrix. By assigning higher weights to the original data points and lower weights to the interpolated data points, the point cloud registration can focus more on the processing of the original data points, which is beneficial to improving the accuracy of data stitching.

[0152] S309: Stitch and fuse the optimized stitching matrix with the previous three-dimensional matrix to obtain the stitched three-dimensional matrix;

[0153] In the method provided by the embodiments of the present invention, the optimized stitching matrix is fused with the previous three-dimensional matrix (the three-dimensional matrix of the previous frame or the current global three-dimensional matrix) to obtain the stitched three-dimensional matrix.

[0154] S310: Determine whether the three-dimensional scanning process is over;

[0155] In the method provided by the embodiments of the present invention, it is possible to monitor whether the current three-dimensional scanning process is over, that is, to determine whether there is still three-dimensional data that needs to be stitched. If the current three-dimensional scanning process has ended, then go to step S312; if the current three-dimensional scanning process has not ended, then go to step S311.

[0156] S311: Wait for the next frame of data to be processed;

[0157] In the method provided by the embodiments of the present invention, if the current scanning process has not ended, wait for the image data of the next frame to be processed. When the image data of the next frame is received, then perform data stitching in the manner of steps S301 to S309.

[0158] S312: Perform global refinement and weighted fusion on the three-dimensional matrix after data stitching is completed.

[0159] In the method provided by the embodiments of the present invention, if the current three-dimensional scanning process has ended, then for the three-dimensional matrix after the overall data stitching is completed, that is, the current global stitching matrix, apply the ICP algorithm to perform global refinement and weighted fusion on the three-dimensional data in the global stitching matrix in the manner of assigning different optimization weights to different types of three-dimensional data in step S308 to complete global data optimization. Take the optimized three-dimensional matrix as the final point cloud data and perform subsequent three-dimensional modeling based on this.

[0160] Based on the method provided by the embodiments of the present invention, three-dimensional data stitching is achieved through the texture features on the surface of the object. During the three-dimensional scanning process, there is no need to perform complex point pasting operations, and there is no need to use auxiliary devices such as trackers, which is beneficial to improving the convenience of three-dimensional scanning, improving the scanning efficiency, improving the user experience, and reducing the equipment cost.

[0161] Corresponding to Figure 1 a method for stitching three-dimensional data shown, the embodiments of the present invention further provide a device for stitching three-dimensional data, used for the Figure 1 specific implementation of the method shown in Figure 4 shown, including:

[0162] The first determination unit 401 is used to determine the laser three-dimensional data corresponding to the current frame and the current frame texture map;

[0163] The coordinate reconstruction unit 402 is used to perform three-dimensional coordinate reconstruction based on the laser three-dimensional data and a preset three-dimensional interpolation algorithm to obtain each current frame three-dimensional point;

[0164] The second determination unit 403 is used to determine the mapping relationship between each current frame three-dimensional point and the current frame texture map;

[0165] The third determination unit 404 is used to determine the feature matching relationship between the current frame texture map and other frame texture maps;

[0166] A fourth determination unit 405, configured to determine the relative position relationship between each of the current-frame three-dimensional points and each of the other-frame three-dimensional points according to the mapping relationship and the feature matching relationship;

[0167] A data splicing unit 406, configured to perform three-dimensional data splicing on each of the current-frame three-dimensional points and each of the other-frame three-dimensional points according to the relative position relationship.

[0168] By applying the device provided in the embodiment of the present invention, each three-dimensional point corresponding to the current frame can be mapped to the texture map corresponding to the current frame, and the texture map of the current frame can be feature-matched with the texture maps of other frames. According to the mapping relationship between the current-frame three-dimensional points and the current-frame texture map and the feature matching relationship between the current-frame texture map and the texture maps of other frames, the three-dimensional points associated with the current-frame three-dimensional points can be found among the other-frame three-dimensional points, thereby determining the relative position relationship between the current-frame three-dimensional points and the other-frame three-dimensional points, and thus realizing three-dimensional data splicing. During the data splicing process, the key feature points of the data splicing can be identified through texture features, and it is not necessary to complete the three-dimensional data splicing through the fiducial points deployed on the object surface. Therefore, in the three-dimensional scanning work, there is no need for manual fiducial point pasting operation, the scanning process is relatively simple, which is beneficial to improving the scanning efficiency and improving the user experience.

[0169] Based on Figure 4 the device shown above, the device provided in the embodiment of the present invention can further expand multiple units. For the functions of each unit, reference can be made to the descriptions in the respective embodiments provided for the three-dimensional data splicing method above, and no further examples will be given here.

[0170] The embodiment of the present invention further provides a storage medium, where the storage medium includes stored instructions, and when the instructions are running, the device where the storage medium is located is controlled to execute the three-dimensional data splicing method as described above.

[0171] The embodiment of the present invention further provides an electronic device, the structural schematic diagram of which is as Figure 5 shown, and specifically includes a memory 501 and one or more instructions 502. One or more instructions 502 are stored in the memory 501 and are configured to be executed by one or more processors 503 to perform the following operations:

[0172] Determine the laser three-dimensional data corresponding to the current frame and the current-frame texture map;

[0173] Perform three-dimensional coordinate reconstruction based on the laser three-dimensional data and a preset three-dimensional interpolation algorithm to obtain each current-frame three-dimensional point;

[0174] Determine the mapping relationship between each of the current-frame three-dimensional points and the current-frame texture map;

[0175] Determine the feature matching relationship between the current frame texture map and other frame texture maps;

[0176] According to the mapping relationship and the feature matching relationship, determine the relative position relationship between each current frame three-dimensional point and each other frame three-dimensional point;

[0177] According to the relative position relationship, perform three-dimensional data stitching on each current frame three-dimensional point and each other frame three-dimensional point.

[0178] Each embodiment in this specification is described in a progressive manner. For the same or similar parts between each embodiment, reference can be made to each other. The key point of each embodiment is to illustrate the differences from other embodiments. In particular, for a system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, reference can be made to the partial description of the method embodiment. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0179] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this article can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0180] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A three-dimensional data splicing method, characterized in that: include: Determine the laser three-dimensional data corresponding to the current frame and the texture map of the current frame; Reconstructing three-dimensional coordinates based on the laser three-dimensional data and a preset three-dimensional interpolation algorithm to obtain three-dimensional points of each current frame; Determine a mapping relationship between each of the current frame three-dimensional points and the current frame texture map; Determine a feature matching relationship between the current frame texture image and other frame texture images; Determining the relative position relationship between each of the three-dimensional points of the current frame and each of the three-dimensional points of other frames according to the mapping relationship and the feature matching relationship; According to the relative position relationship, three-dimensional data stitching is performed on each of the three-dimensional points of the current frame and each of the three-dimensional points of the other frames; in the three-dimensional data stitching process, among the three-dimensional points of the current frame, the optimization weight corresponding to the three-dimensional data obtained based on laser measurement is greater than the optimization weight corresponding to the three-dimensional data not obtained based on laser measurement; Wherein, determining the feature matching relationship between the current frame texture image and other frame texture images includes: Determine each first texture feature corresponding to the current frame texture map; Determine each second texture feature corresponding to the other frame texture images; For each of the first texture features, performing feature matching processing on the first texture feature and each of the second texture features respectively to obtain a feature matching result corresponding to the first texture feature; A feature matching relationship between the current frame texture map and the other frame texture maps is determined according to the feature matching results corresponding to each of the first texture features.

2. The three-dimensional data splicing method according to claim 1, characterized in that: The three-dimensional coordinate reconstruction based on the laser three-dimensional data and a preset three-dimensional interpolation algorithm to obtain each current frame three-dimensional point includes: Obtaining the three-dimensional coordinates corresponding to each laser imaging point in the laser three-dimensional data; Applying the three-dimensional interpolation algorithm to perform three-dimensional interpolation processing on the three-dimensional coordinates corresponding to each of the laser imaging points to obtain the three-dimensional coordinates corresponding to each interpolation point; The three-dimensional coordinates corresponding to each of the laser imaging points and the three-dimensional coordinates corresponding to each of the interpolation points are used as each of the current frame three-dimensional points.

3. The three-dimensional data splicing method according to claim 1, characterized in that: Determining the relative position relationship between each of the three-dimensional points of the current frame and each of the three-dimensional points of other frames according to the mapping relationship and the feature matching relationship includes: Determining a target area image in the current frame texture map according to the mapping relationship; Determining target texture features in the target area image that match the other frame texture images based on the feature matching relationship; Determine the current frame target point corresponding to the target texture feature among the current frame three-dimensional points; Determine, among the three-dimensional points of the other frames, target points of the other frames corresponding to the target texture features; The relative position relationship between each of the three-dimensional points in the current frame and each of the three-dimensional points in the other frames is determined according to the three-dimensional coordinates of the target point in the current frame and the three-dimensional coordinates of the target points in the other frames.

4. The three-dimensional data splicing method according to claim 1, characterized in that: The step of performing three-dimensional data stitching on each of the three-dimensional points of the current frame and each of the three-dimensional points of the other frames according to the relative position relationship includes: According to the relative position relationship, construct an initial stitching matrix corresponding to each of the three-dimensional points of the current frame; According to a preset point cloud registration algorithm, the initial stitching matrix is ​​subjected to data optimization processing to obtain an optimized matrix corresponding to the initial stitching matrix; Determine the three-dimensional matrix of other frames corresponding to each of the three-dimensional points of other frames; The optimization matrix corresponding to the initial stitching matrix is ​​stitched with the three-dimensional matrix of the other frames to obtain a stitching result, and the stitching result is used as the three-dimensional data stitching result of each three-dimensional point of the current frame and each three-dimensional point of the other frames.

5. The three-dimensional data splicing method according to claim 4, characterized in that: The step of performing data optimization processing on the initial stitching matrix according to a preset point cloud registration algorithm to obtain an optimized matrix corresponding to the initial stitching matrix includes: For each three-dimensional coordinate in the initial stitching matrix, determine whether the three-dimensional point corresponding to the three-dimensional coordinate is coordinate data obtained by laser measurement, if the three-dimensional point corresponding to the three-dimensional coordinate is coordinate data obtained by laser measurement, use the three-dimensional coordinate as the original coordinate, if the three-dimensional point corresponding to the three-dimensional coordinate is not coordinate data obtained by laser measurement, use the three-dimensional coordinate as the derived coordinate; Using a preset first weight as the optimization weight corresponding to each of the original coordinates, and using a preset second weight as the optimization weight corresponding to each of the derived coordinates; the first weight is greater than the second weight; Based on the optimization weights corresponding to the three-dimensional coordinates in the initial splicing matrix, an iterative closest point algorithm is applied to perform three-dimensional data optimization on the initial splicing matrix, and the optimization result is used as the optimization matrix corresponding to the initial splicing matrix.

6. The three-dimensional data splicing method according to claim 1, characterized in that: Also includes: When the three-dimensional scanning process is completed, the global stitching matrix is ​​determined; The global stitching matrix is ​​globally optimized to obtain a global optimization matrix, and the global optimization matrix is ​​used as the global three-dimensional matrix of the scanned object.

7. A three-dimensional data splicing device, characterized in that: include: A first determination unit, used to determine the laser three-dimensional data corresponding to the current frame and the texture map of the current frame; A coordinate reconstruction unit, used to perform three-dimensional coordinate reconstruction based on the laser three-dimensional data and a preset three-dimensional interpolation algorithm to obtain three-dimensional points of each current frame; A second determining unit, used to determine a mapping relationship between each of the current frame 3D points and the current frame texture map; A third determining unit, used to determine a feature matching relationship between the current frame texture image and other frame texture images; A fourth determining unit, configured to determine a relative position relationship between each of the three-dimensional points of the current frame and each of the three-dimensional points of other frames according to the mapping relationship and the feature matching relationship; A data stitching unit is used to stitch the three-dimensional data of each of the three-dimensional points of the current frame and each of the three-dimensional points of the other frames according to the relative position relationship; in the three-dimensional data stitching process, among the three-dimensional points of the current frame, the optimization weight corresponding to the three-dimensional data obtained based on the laser measurement is greater than the optimization weight corresponding to the three-dimensional data not obtained based on the laser measurement; The third determining unit is specifically used to: determine each first texture feature corresponding to the texture map of the current frame; Determine each second texture feature corresponding to the other frame texture images; for each of the first texture features, perform feature matching processing on the first texture feature with each of the second texture features to obtain a feature matching result corresponding to the first texture feature; and determine a feature matching relationship between the current frame texture image and the other frame texture images based on the feature matching results corresponding to each of the first texture features.

8. A storage medium, characterized in that: The storage medium includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the three-dimensional data splicing method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: The system comprises a memory and one or more instructions, wherein the one or more instructions are stored in the memory and configured to be executed by one or more processors to perform the three-dimensional data stitching method according to any one of claims 1 to 6.

Images