Three-dimensional data splicing method and device, storage medium and electronic equipment

Abstract

The invention provides a three-dimensional data splicing method and device, a storage medium and electronic equipment, and belongs to the field of three-dimensional scanning, and the method comprises the steps: determining laser three-dimensional data corresponding to a current frame and a texture image of the current frame; performing three-dimensional coordinate reconstruction based on the laser three-dimensional data and a preset three-dimensional interpolation algorithm to obtain three-dimensional points of each current frame; determining a mapping relation between each current frame three-dimensional point and the current frame texture map; determining a feature matching relationship between the current frame texture image and other frame texture images; according to the mapping relationship and the feature matching relationship, determining a relative position relationship between each current frame three-dimensional point and each other frame three-dimensional point; and according to the relative position relationship, performing three-dimensional data splicing on each current frame three-dimensional point and each other frame three-dimensional point. By applying the method, the key feature points of data splicing can be recognized through the texture features, three-dimensional data splicing is completed, point pasting operation is not needed, the scanning process is simple and convenient, and the scanning efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional scanning technology, and in particular to a three-dimensional data splicing method and device, a storage medium and an electronic device. Background Art

[0002] Handheld laser scanners are one of the common scanning devices in three-dimensional scanning scenarios. In the application scenarios of handheld laser scanners, it is usually necessary to scan objects at different perspectives, and then splice the three-dimensional data measured at different scanning perspectives to construct the final three-dimensional model.

[0003] At present, it is usually necessary to manually deploy multiple marking points with certain identification on the surface of the scanned object. When scanning the object, each marking point will be repeatedly captured at different scanning angles. When splicing the three-dimensional data of different scanning angles, the three-dimensional data of these scanning angles are spliced ​​according to the same marking points captured at these scanning angles.

[0004] In the existing three-dimensional data stitching method, the stitching of three-dimensional data depends on the position of the landmark points. In actual application scenarios, objects with large surface areas or complex shapes and structures are often encountered, which require complex point-sticking operations, making the three-dimensional scanning process more cumbersome and the scanning efficiency low. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a three-dimensional data splicing method to solve the problem that the existing three-dimensional data splicing method relies on the deployment of landmark points on the surface of the object and requires manual complex point-pasting operations, making the three-dimensional scanning process more cumbersome and inefficient.

[0006] The embodiment of the present invention also provides a three-dimensional data splicing device to ensure the actual implementation and application of the above method.

[0007] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0008] A three-dimensional data splicing method, comprising:

[0009] Determine the laser three-dimensional data corresponding to the current frame and the texture map of the current frame;

[0010] Reconstructing three-dimensional coordinates based on the laser three-dimensional data and a preset three-dimensional interpolation algorithm to obtain three-dimensional points of each current frame;

[0011] Determine a mapping relationship between each of the current frame three-dimensional points and the current frame texture map;

[0012] Determine a feature matching relationship between the current frame texture image and other frame texture images;

[0013] Determining the relative position relationship between each of the three-dimensional points of the current frame and each of the three-dimensional points of other frames according to the mapping relationship and the feature matching relationship;

[0014] Based on the relative position relationship, three-dimensional data splicing is performed on each of the three-dimensional points of the current frame and each of the three-dimensional points of the other frames.

[0015] Optionally, the above method, performing three-dimensional coordinate reconstruction based on the laser three-dimensional data and a preset three-dimensional interpolation algorithm to obtain each current frame three-dimensional point includes:

[0016] Obtaining the three-dimensional coordinates corresponding to each laser imaging point in the laser three-dimensional data;

[0017] Applying the three-dimensional interpolation algorithm to perform three-dimensional interpolation processing on the three-dimensional coordinates corresponding to each of the laser imaging points to obtain the three-dimensional coordinates corresponding to each interpolation point;

[0018] The three-dimensional coordinates corresponding to each of the laser imaging points and the three-dimensional coordinates corresponding to each of the interpolation points are used as each of the current frame three-dimensional points.

[0019] In the above method, optionally, determining the feature matching relationship between the current frame texture image and other frame texture images includes:

[0020] Determine each first texture feature corresponding to the current frame texture map;

[0021] Determine each second texture feature corresponding to the other frame texture images;

[0022] For each of the first texture features, performing feature matching processing on the first texture feature and each of the second texture features respectively to obtain a feature matching result corresponding to the first texture feature;

[0023] A feature matching relationship between the current frame texture map and the other frame texture maps is determined according to the feature matching results corresponding to each of the first texture features.

[0024] Optionally, the method described above, determining the relative position relationship between each of the three-dimensional points of the current frame and each of the three-dimensional points of other frames based on the mapping relationship and the feature matching relationship, includes:

[0025] Determining a target area image in the current frame texture map according to the mapping relationship;

[0026] Determining target texture features in the target area image that match the other frame texture images based on the feature matching relationship;

[0027] Determine the current frame target point corresponding to the target texture feature among the current frame three-dimensional points;

[0028] Determine, among the three-dimensional points of the other frames, target points of the other frames corresponding to the target texture features;

[0029] The relative position relationship between each of the three-dimensional points in the current frame and each of the three-dimensional points in the other frames is determined according to the three-dimensional coordinates of the target point in the current frame and the three-dimensional coordinates of the target points in the other frames.

[0030] In the above method, optionally, performing three-dimensional data stitching on each of the three-dimensional points of the current frame and each of the three-dimensional points of the other frames based on the relative position relationship includes:

[0031] According to the relative position relationship, construct an initial stitching matrix corresponding to each of the three-dimensional points of the current frame;

[0032] According to a preset point cloud registration algorithm, the initial stitching matrix is ​​subjected to data optimization processing to obtain an optimized matrix corresponding to the initial stitching matrix;

[0033] Determine the three-dimensional matrix of other frames corresponding to each of the three-dimensional points of other frames;

[0034] The optimization matrix corresponding to the initial stitching matrix is ​​stitched with the three-dimensional matrix of the other frames to obtain a stitching result, and the stitching result is used as the three-dimensional data stitching result of each three-dimensional point of the current frame and each three-dimensional point of the other frames.

[0035] Optionally, the method described above includes performing data optimization processing on the initial stitching matrix according to a preset point cloud registration algorithm to obtain an optimized matrix corresponding to the initial stitching matrix, including:

[0036] For each three-dimensional coordinate in the initial stitching matrix, determine whether the three-dimensional point corresponding to the three-dimensional coordinate is coordinate data obtained by laser measurement, if the three-dimensional point corresponding to the three-dimensional coordinate is coordinate data obtained by laser measurement, use the three-dimensional coordinate as the original coordinate, if the three-dimensional point corresponding to the three-dimensional coordinate is not coordinate data obtained by laser measurement, use the three-dimensional coordinate as the derived coordinate;

[0037] Using a preset first weight as the optimization weight corresponding to each of the original coordinates, and using a preset second weight as the optimization weight corresponding to each of the derived coordinates; the first weight is greater than the second weight;

[0038] Based on the optimization weights corresponding to the three-dimensional coordinates in the initial splicing matrix, an iterative closest point algorithm is applied to perform three-dimensional data optimization on the initial splicing matrix, and the optimization result is used as the optimization matrix corresponding to the initial splicing matrix.

[0039] The above method may optionally further include:

[0040] When the three-dimensional scanning process is completed, the global stitching matrix is ​​determined;

[0041] The global stitching matrix is ​​globally optimized to obtain a global optimization matrix, and the global optimization matrix is ​​used as the global three-dimensional matrix of the scanned object.

[0042] A three-dimensional data splicing device, comprising:

[0043] A first determination unit, used to determine the laser three-dimensional data corresponding to the current frame and the texture map of the current frame;

[0044] A coordinate reconstruction unit, used to perform three-dimensional coordinate reconstruction based on the laser three-dimensional data and a preset three-dimensional interpolation algorithm to obtain three-dimensional points of each current frame;

[0045] A second determining unit, used to determine a mapping relationship between each of the current frame 3D points and the current frame texture map;

[0046] A third determining unit, used to determine a feature matching relationship between the current frame texture image and other frame texture images;

[0047] A fourth determining unit, configured to determine a relative position relationship between each of the three-dimensional points of the current frame and each of the three-dimensional points of other frames according to the mapping relationship and the feature matching relationship;

[0048] The data stitching unit is used to perform three-dimensional data stitching on each of the three-dimensional points of the current frame and each of the three-dimensional points of the other frames according to the relative position relationship.

[0049] A storage medium includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the above-mentioned three-dimensional data splicing method.

[0050] An electronic device includes a memory and one or more instructions, wherein the one or more instructions are stored in the memory and are configured to be executed by one or more processors to perform the three-dimensional data stitching method as described above.

[0051] A three-dimensional data splicing method based on the above-mentioned embodiment of the present invention includes: determining the laser three-dimensional data and the texture map of the current frame corresponding to the current frame; reconstructing the three-dimensional coordinates based on the laser three-dimensional data and a preset three-dimensional interpolation algorithm to obtain each three-dimensional point of the current frame; determining the mapping relationship between each three-dimensional point of the current frame and the texture map of the current frame; determining the feature matching relationship between the texture map of the current frame and the texture maps of other frames; determining the relative position relationship between each three-dimensional point of the current frame and each three-dimensional point of other frames based on the mapping relationship and the feature matching relationship; and splicing the three-dimensional data of each three-dimensional point of the current frame and each three-dimensional point of other frames based on the relative position relationship. By applying the method provided by the embodiment of the present invention, each three-dimensional point corresponding to the current frame can be mapped to the texture map corresponding to the current frame, and the texture map of the current frame can be feature matched with the texture map of other frames. According to the mapping relationship between the three-dimensional point of the current frame and the texture map of the current frame and the feature matching relationship between the texture map of the current frame and the texture maps of other frames, a three-dimensional point associated with the three-dimensional point of the current frame is found in the three-dimensional points of other frames, thereby determining the relative position relationship between the three-dimensional point of the current frame and the three-dimensional points of other frames, thereby realizing three-dimensional data splicing. In the process of data stitching, the key feature points of data stitching can be identified through texture features, and there is no need to complete the three-dimensional data stitching through the landmark points deployed on the surface of the object. Therefore, there is no need for manual point sticking operation in the three-dimensional scanning work. The scanning process is relatively simple, which is conducive to improving scanning efficiency and improving user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0053] Figure 1 A method flow chart of a three-dimensional data splicing method provided by an embodiment of the present invention;

[0054] Figure 2 A schematic diagram of the structure of a laser scanner provided by an embodiment of the present invention;

[0055] Figure 3 A schematic diagram of a three-dimensional data stitching process in a three-dimensional scanning process provided by an embodiment of the present invention;

[0056] Figure 4 A schematic diagram of the structure of a three-dimensional data splicing device provided by an embodiment of the present invention;

[0057] Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0060] The embodiment of the present invention provides a method for splicing three-dimensional data. The method can be applied to a data processing system of a laser scanner. The execution subject can be a processor of the data processing system. The method flow chart of the method is as follows: Figure 1 As shown, including:

[0061] S101: Determine the laser three-dimensional data corresponding to the current frame and the texture map of the current frame;

[0062] The method provided by the embodiment of the present invention can be applied to the scanning process of a handheld laser scanner. The user can hold the scanner to continuously scan the scanned object from different scanning angles. The scanner can collect corresponding data at different scanning positions to measure the three-dimensional data of the corresponding scanning position, and splice the three-dimensional data of each scanning position to obtain complete point cloud data of the object. The method provided by the embodiment of the present invention combines the laser three-dimensional data and texture map of the object surface to achieve three-dimensional data splicing. Therefore, when the scanner performs a scanning operation, it is necessary to collect a two-dimensional laser line map and a texture map of the object surface. The two-dimensional laser line map refers to an image obtained by photographing the object surface with a black and white camera when a laser beam is projected onto the object surface. The texture map refers to an image obtained by photographing the object surface with a color texture camera when no laser beam is projected onto the object surface. Specifically, the scanner can be equipped with a laser projector, a black and white camera, and a color texture camera. When performing a scanning operation on the current position of the object, the laser projector projects a laser beam to the surface of the scanned object, and the black and white camera shoots the current position of the object to obtain a two-dimensional image of the laser line at the current position of the object. Then the laser beam projection is stopped, and the color texture camera shoots the current position of the object to obtain a texture map of the current position of the object. The two-dimensional image and texture map of the laser line at the current position of the object collected during this operation are used as image data of the same frame. It should be noted that in the process of collecting the two-dimensional image of the laser line, multiple black and white cameras can be used to simultaneously shoot the surface of the object. There can be multiple two-dimensional images of the laser line corresponding to the current frame, and the multiple two-dimensional images of the laser line can be combined to perform three-dimensional coordinate measurement later.

[0063] The data processing system of the scanner can process the collected image data in real time during the scanning process. For example, when the scanning starts, when the two-dimensional laser line image and texture map of the first frame are collected, the three-dimensional data corresponding to the first frame can be measured according to the image data of the first frame. When the two-dimensional laser line image and texture map of the second frame are received, the three-dimensional data corresponding to the second frame can be measured according to the image data of the second frame. The three-dimensional data of the first frame and the three-dimensional data of the second frame are then spliced ​​to obtain the current spliced ​​data. When the two-dimensional laser line image and texture map of the third frame are received, the three-dimensional data corresponding to the third frame is further spliced ​​with the existing spliced ​​data, and the subsequent processing process is the same.

[0064] It should be noted that the number and positions of the laser projectors, black and white cameras and color physical cameras in the scanner can be set according to actual needs, and image acquisition of two-dimensional laser line images and texture images of the object surface can be realized. The structure of the scanner does not affect the function of the method provided in the embodiment of the present invention.

[0065] In the method provided by the embodiment of the present invention, when it is necessary to process the image data of a certain frame, the frame can be used as the current frame. The system can obtain the laser line two-dimensional image corresponding to the current frame and the texture image corresponding to the current frame (i.e., the texture image of the current frame), and measure the three-dimensional coordinates of the laser line imaging point at the corresponding object surface position according to the laser line two-dimensional image corresponding to the current frame, and obtain the laser three-dimensional data of the current frame. Specifically, the principle of laser triangulation can be used to calculate the three-dimensional coordinates of the object surface position corresponding to each laser imaging point in the laser line two-dimensional image, that is, the camera position and the laser projector position for shooting the laser line two-dimensional image are known. When the laser beam is projected onto the surface of the object, a corresponding laser line profile will be formed on the surface of the object. The laser line profile will present a corresponding shape due to the surface structure of the object, that is, the light profile position of the laser line reflects the position change of the laser line at different heights on the surface of the object. Therefore, according to the laser line profile position of the laser imaging in the laser line two-dimensional image, combined with the laser projector position and the camera position, the three-dimensional coordinates of each point of the laser line imaging on the object surface can be calculated, that is, the three-dimensional coordinates of the corresponding position on the object surface, so as to obtain the laser three-dimensional data corresponding to the current frame, that is, the three-dimensional coordinates of each laser imaging point.

[0066] S102: reconstructing three-dimensional coordinates based on the laser three-dimensional data and a preset three-dimensional interpolation algorithm to obtain three-dimensional points of each current frame;

[0067] In the method provided in an embodiment of the present invention, based on the laser three-dimensional data, the three-dimensional data is further interpolated and optimized in combination with a three-dimensional interpolation algorithm, so as to reconstruct the three-dimensional coordinates of the scanning area, and the three-dimensional data after interpolation optimization is used as the three-dimensional points of the current frame corresponding to the current frame.

[0068] S103: Determine a mapping relationship between each of the current frame 3D points and the current frame texture map;

[0069] In the method provided by the embodiment of the present invention, the mapping relationship between each pixel point in the laser line two-dimensional image and each pixel point in the texture image can be determined according to the relative position of the camera that shoots the laser line two-dimensional image and the camera that shoots the texture image. Each current frame three-dimensional point is a three-dimensional coordinate calculated based on the laser line two-dimensional image. Each current frame three-dimensional point actually represents the position of a surface point of an object in the laser line two-dimensional image. Therefore, there is also a corresponding mapping relationship between each current frame three-dimensional point and each pixel point in the laser line two-dimensional image. According to the mapping relationship between each current frame three-dimensional point and the pixel point of the laser line two-dimensional image, and the mapping relationship between the pixel point of the laser line two-dimensional image and the pixel point of the current frame texture image, the mapping relationship between each current frame three-dimensional point and the pixel point in the current frame texture image can be converted, thereby obtaining the mapping relationship between each current frame three-dimensional point and the current frame texture image.

[0070] S104: Determine a feature matching relationship between the current frame texture image and other frame texture images;

[0071] In the method provided by the embodiment of the present invention, the data frame that needs to be spliced ​​with the three-dimensional data of the current frame is taken as the other frame, and the other frame can be the previous frame or the next frame of the current frame, or a data frame with a certain interval from the current frame, or a global frame obtained by splicing the current existing three-dimensional data. In the data processing process of other frames, the three-dimensional points and texture maps corresponding to other frames are also obtained, and the three-dimensional points corresponding to other frames are regarded as the three-dimensional points of other frames, and the texture maps corresponding to other frames are regarded as the texture maps of other frames.

[0072] It should be noted that if there is no three-dimensional data that needs to be spliced ​​with the three-dimensional data corresponding to the current frame, then a corresponding three-dimensional data matrix is ​​constructed based on each current frame three-dimensional point corresponding to the current frame, and then it can be spliced ​​with subsequent three-dimensional data.

[0073] In the method provided by the embodiment of the present invention, each texture feature corresponding to the current frame texture map can be feature matched with each texture feature corresponding to other frame texture maps to determine the feature matching relationship between the current frame texture map and other frame texture maps. The feature matching relationship indicates whether the texture feature in the current frame texture map matches a certain texture feature in other frame texture maps.

[0074] S105: determining the relative position relationship between each of the three-dimensional points of the current frame and each of the three-dimensional points of other frames according to the mapping relationship and the feature matching relationship;

[0075] In the method provided by the embodiment of the present invention, the correspondence between the current frame 3D point and the texture feature in the current frame texture map can be determined according to the mapping relationship between the current frame 3D point and the current frame texture map. According to the correspondence and the feature matching relationship between the current frame texture map and other frame texture maps, the mutually related current frame 3D points and other frame 3D points can be found in each current frame 3D point and each other frame 3D point. According to the coordinate transformation relationship between the mutually related current frame 3D points and other frame 3D points, the relative position relationship between the current frame 3D points and other frame 3D points can be calculated.

[0076] S106: performing three-dimensional data splicing on each of the three-dimensional points of the current frame and each of the three-dimensional points of the other frames according to the relative position relationship.

[0077] In the method provided in the embodiment of the present invention, a preset data fusion algorithm can be applied to perform data fusion on each current frame 3D point and other frame 3D points according to the relative position relationship between the current frame 3D points and other frame 3D points, and use the data fusion result as the data stitching result corresponding to the current frame. The specific data fusion algorithm can be set according to actual needs, for example, the data fusion algorithm can be set based on a machine learning model, the data fusion algorithm can be set based on an iterative closest point (ICP) algorithm, the data fusion algorithm can be set based on Gaussian mixture model (GMM) registration, and the data fusion algorithm can also be set based on other methods that can achieve three-dimensional data fusion, which does not affect the function of the method provided in the embodiment of the present invention.

[0078] Based on the method provided by the embodiment of the present invention, when it is necessary to splice the three-dimensional data corresponding to the current frame with the previous three-dimensional data, the laser three-dimensional data corresponding to the current frame and the texture map of the current frame are determined; the three-dimensional coordinates are reconstructed based on the laser three-dimensional data and the preset three-dimensional interpolation algorithm to obtain each three-dimensional point of the current frame; the mapping relationship between each three-dimensional point of the current frame and the texture map of the current frame is determined; the feature matching relationship between the texture map of the current frame and the texture maps of other frames is determined; based on the mapping relationship and the feature matching relationship, the relative position relationship between each three-dimensional point of the current frame and each three-dimensional point of other frames is determined; based on the relative position relationship, the three-dimensional data splicing of each three-dimensional point of the current frame and each three-dimensional point of other frames is performed. By applying the method provided by the embodiment of the present invention, each three-dimensional point corresponding to the current frame can be mapped with the texture map corresponding to the current frame, and the texture map of the current frame can be feature matched with the texture map of other frames. According to the mapping relationship between the three-dimensional point of the current frame and the texture map of the current frame and the feature matching relationship between the texture map of the current frame and the texture maps of other frames, the three-dimensional point associated with the three-dimensional point of the current frame is found in the three-dimensional points of other frames, thereby determining the relative position relationship between the three-dimensional point of the current frame and the three-dimensional points of other frames, thereby realizing three-dimensional data splicing. In the process of data stitching, the key feature points of data stitching can be identified through texture features, and there is no need to complete the three-dimensional data stitching through the landmark points deployed on the surface of the object. Therefore, there is no need for manual point sticking operation in the three-dimensional scanning work. The scanning process is relatively simple, which is conducive to improving scanning efficiency and improving user experience.

[0079] exist Figure 1 On the basis of the method shown, in the method provided by the embodiment of the present invention, the process of reconstructing three-dimensional coordinates based on the laser three-dimensional data and a preset three-dimensional interpolation algorithm to obtain three-dimensional points of each current frame mentioned in step S102 includes:

[0080] Obtaining the three-dimensional coordinates corresponding to each laser imaging point in the laser three-dimensional data;

[0081] In the method provided by the embodiment of the present invention, in the process of determining the laser three-dimensional data corresponding to the current frame, the laser line two-dimensional map corresponding to the current frame can be used to firstly measure the three-dimensional coordinates of each laser imaging point in the laser line two-dimensional map by applying the laser triangulation method, that is, to calculate the three-dimensional coordinates corresponding to each laser imaging point, and to reconstruct the depth map according to the position of each laser imaging point in the laser line two-dimensional map and its corresponding three-dimensional coordinates, so as to obtain the original depth map containing the three-dimensional coordinates corresponding to each laser imaging point, and to represent the laser three-dimensional data in the form of the original depth map. It can be understood that each pixel point in the original depth map corresponds to each pixel point in the laser line two-dimensional map one by one, and the pixel point in the laser line two-dimensional map may be a laser imaging point or may not be a laser imaging point. Each pixel point in the original depth map represents a set of depth data, that is, a three-dimensional coordinate. With respect to the pixel point in the original depth map, if the pixel point corresponds to a laser imaging point, then the depth data of the pixel point is the three-dimensional coordinate corresponding to the laser imaging point, and if the pixel point does not correspond to a laser imaging point, then the depth data of the pixel point is not a valid three-dimensional coordinate and can be configured as a null value.

[0082] Applying the three-dimensional interpolation algorithm to perform three-dimensional interpolation processing on the three-dimensional coordinates corresponding to each of the laser imaging points to obtain the three-dimensional coordinates corresponding to each interpolation point;

[0083] In the method provided by the embodiment of the present invention, a preset three-dimensional interpolation algorithm can be applied to perform three-dimensional interpolation processing on the depth data in the original depth map based on the three-dimensional coordinates corresponding to each laser imaging point, that is, the depth data of the pixel points without valid three-dimensional coordinates are solved according to the existing three-dimensional coordinates in the original depth map, so as to obtain the three-dimensional coordinates corresponding to each interpolation point in the original depth map. The preset three-dimensional interpolation algorithm can be set according to actual needs, for example, a polynomial fitting method can be used for three-dimensional interpolation, a nearest neighbor interpolation algorithm can be used for three-dimensional interpolation, and other three-dimensional interpolation algorithms can be used without affecting the function of the method provided by the embodiment of the present invention.

[0084] The three-dimensional coordinates corresponding to each of the laser imaging points and the three-dimensional coordinates corresponding to each of the interpolation points are used as each of the current frame three-dimensional points.

[0085] In the method provided by an embodiment of the present invention, the three-dimensional coordinates corresponding to each laser imaging point obtained by three-dimensional coordinate measurement and the three-dimensional coordinates corresponding to each interpolation point obtained by three-dimensional interpolation calculation are used as the three-dimensional data corresponding to the current frame, that is, the three-dimensional points of the current frame.

[0086] Based on the method provided in the embodiment of the present invention, in the process of reconstructing three-dimensional data, the original three-dimensional data can be interpolated and optimized by three-dimensional interpolation, which can make the final three-dimensional data smoother, facilitate splicing with other three-dimensional data, and improve the accuracy of data splicing.

[0087] exist Figure 1 On the basis of the method shown, in the method provided by the embodiment of the present invention, the process of determining the feature matching relationship between the current frame texture image and other frame texture images mentioned in step S104 includes:

[0088] Determine each first texture feature corresponding to the current frame texture map;

[0089] In the method provided by the embodiment of the present invention, a preset texture feature analysis method can be applied to analyze the texture information in the current frame texture map to identify the texture features of the surface positions of each object in the current frame texture map. Specifically, a global analysis can be performed on the current frame texture map, or texture feature analysis can be performed only on the texture map area mapped by the current frame three-dimensional points based on the mapping relationship between the current frame three-dimensional points and the current frame texture map. The various texture features in the current frame texture map are used as various first texture features. The preset texture feature analysis method can be set according to actual needs. For example, a machine learning algorithm can be used to extract texture features in an image, and an algorithm for extracting image texture features such as a local binary pattern (Local Binary Patterns) algorithm can be used to extract texture features. Other algorithms that can realize texture feature extraction can also be used without affecting the function of the method provided by the embodiment of the present invention.

[0090] Determine each second texture feature corresponding to the other frame texture images;

[0091] In the method provided by the embodiment of the present invention, during the processing of other frame texture images, a preset texture feature analysis method is also applied to analyze the texture information in other frame texture images, and each texture feature in other frame texture images is used as the second texture feature.

[0092] For each of the first texture features, performing feature matching processing on the first texture feature and each of the second texture features respectively to obtain a feature matching result corresponding to the first texture feature;

[0093] In the method provided by an embodiment of the present invention, for each first texture feature, feature matching processing is performed on the first texture feature and each second texture feature respectively. If there is a second texture feature that matches the first texture feature, a matching relationship between the first texture feature and the second texture feature that matches it is established, thereby obtaining a feature matching result of the first texture feature, that is, a result that characterizes whether the first texture feature matches each second texture feature.

[0094] A feature matching relationship between the current frame texture map and the other frame texture maps is determined according to the feature matching results corresponding to each of the first texture features.

[0095] In the method provided by the embodiment of the present invention, the feature matching relationship between the current frame texture image and other frame texture images is determined according to the feature matching result of whether each first texture feature matches each second texture feature.

[0096] exist Figure 1 On the basis of the method shown, in the method provided by the embodiment of the present invention, the process of determining the relative position relationship between each of the current frame 3D points and each of the other frame 3D points according to the mapping relationship and the feature matching relationship mentioned in step S105 includes:

[0097] Determining a target area image in the current frame texture map according to the mapping relationship;

[0098] In the method provided by the embodiment of the present invention, according to each current frame 3D point, in the current frame texture map, an image corresponding to the surface area of ​​the object represented by each current frame 3D point is found, and the image of the area is used as the target area image. It can be understood that each current frame 3D point represents the 3D coordinates of the point of the corresponding object surface area, and the target area image is the image corresponding to the surface area of ​​the object in the current frame texture map.

[0099] In the method provided by the embodiment of the present invention, for each pixel point in the texture map of the current frame, it is possible to determine whether the pixel point is a pixel point corresponding to the three-dimensional point according to the mapping relationship between each three-dimensional point of the current frame and the pixel point of the texture map, and if the pixel point is a pixel point corresponding to the three-dimensional point, the pixel point is used as the target pixel point. The image area corresponding to each target pixel point in the texture map of the current frame is used as the target area image.

[0100] Determining target texture features in the target area image that match the other frame texture images based on the feature matching relationship;

[0101] In the method provided by the embodiment of the present invention, the feature matching results of each texture feature in the target area image and the feature matching results of other frame texture maps can be determined based on the feature matching relationship between the current frame texture map and the other frame texture maps. For each texture feature in the target area image, if the texture feature matches a texture feature in other frame texture maps, the texture feature in the target area image is used as the target texture feature.

[0102] Determine the current frame target point corresponding to the target texture feature among the current frame three-dimensional points;

[0103] In the method provided by an embodiment of the present invention, the current frame 3D point corresponding to the target texture feature can be determined based on the mapping relationship between the current frame 3D point and the current frame texture map and the corresponding relationship between the target texture feature and the current frame texture map, and the current frame 3D point corresponding to the target texture feature can be used as the current frame target point.

[0104] Determine, among the three-dimensional points of the other frames, target points of the other frames corresponding to the target texture features;

[0105] In the method provided by the embodiment of the present invention, the texture features in other frame texture maps that match the target texture features can be used as the associated texture features corresponding to the target texture features. It can be understood that, similar to the mapping relationship between each current frame 3D point and the texture features of the current frame texture map, each other frame 3D point also has a corresponding mapping relationship with the texture features in the other frame texture maps. According to the mapping relationship between other frame 3D points and other frame texture maps, other frame 3D points mapped by the associated texture features corresponding to the target texture features can be determined, and the other frame 3D points corresponding to the associated texture features are used as other frame target points corresponding to the target texture features.

[0106] The relative position relationship between each of the three-dimensional points in the current frame and each of the three-dimensional points in the other frames is determined according to the three-dimensional coordinates of the target point in the current frame and the three-dimensional coordinates of the target points in the other frames.

[0107] In the method provided by an embodiment of the present invention, the coordinate transformation relationship between the current frame target point and the other frame target points can be determined based on the three-dimensional coordinates of the current frame target point and the three-dimensional point coordinates of the other frame target points, and the relative position relationship between the current frame three-dimensional point and the other frame three-dimensional point can be determined based on the coordinate transformation relationship.

[0108] exist Figure 1 On the basis of the method shown, in the method provided by the embodiment of the present invention, the process of performing three-dimensional data stitching on each of the three-dimensional points of the current frame and each of the three-dimensional points of the other frames according to the relative position relationship mentioned in step S106 includes:

[0109] According to the relative position relationship, construct an initial stitching matrix corresponding to each of the three-dimensional points of the current frame;

[0110] In the method provided by the embodiment of the present invention, the three-dimensional coordinates of each current frame three-dimensional point can be transformed according to the relative position relationship between the current frame three-dimensional point and the three-dimensional points of other frames, so as to align each current frame three-dimensional point with each other frame three-dimensional point to the same coordinate system, and the current frame three-dimensional point that has undergone coordinate transformation is used as the transformed three-dimensional point, and a corresponding three-dimensional data matrix is ​​constructed based on the three-dimensional coordinates of each transformed three-dimensional point, and the constructed three-dimensional data matrix is ​​used as the initial stitching matrix.

[0111] According to a preset point cloud registration algorithm, the initial stitching matrix is ​​subjected to data optimization processing to obtain an optimized matrix corresponding to the initial stitching matrix;

[0112] In the method provided by the embodiment of the present invention, in the process of three-dimensional data splicing, a preset point cloud registration algorithm can be first applied to optimize the registration of the three-dimensional data in the initial splicing matrix, and the initial splicing matrix after registration optimization is used as the optimization matrix. The preset point cloud registration algorithm can be set according to actual needs. For example, an iterative closest point (ICP) algorithm can be used as a point cloud registration algorithm, a point cloud registration algorithm based on a deep learning network can be used, and other point cloud registration methods can also be used without affecting the function of the method provided by the embodiment of the present invention.

[0113] Determine the three-dimensional matrix of other frames corresponding to each of the three-dimensional points of other frames;

[0114] In the method provided by the embodiment of the present invention, the three-dimensional data matrix composed of three-dimensional points of each other frame is used as the three-dimensional matrix of other frames.

[0115] The optimization matrix corresponding to the initial stitching matrix is ​​stitched with the three-dimensional matrix of the other frames to obtain a stitching result, and the stitching result is used as the three-dimensional data stitching result of each three-dimensional point of the current frame and each three-dimensional point of the other frames.

[0116] In the method provided by the embodiment of the present invention, data fusion is performed on the optimized matrix of the initial splicing matrix and the three-dimensional matrices of other frames to achieve data splicing, and the matrix obtained by splicing the optimized matrix with the three-dimensional matrices of other frames (i.e., the splicing result) is used as the three-dimensional data splicing result corresponding to the current frame.

[0117] Based on the method provided in the embodiment of the present invention, the initial stitching matrix can be further optimized by a point cloud registration algorithm, and the optimized matrix can be fused with other frame three-dimensional matrices to further improve the accuracy of data stitching.

[0118] On the basis of the method provided in the above embodiment, in the method provided in the embodiment of the present invention, the process of performing data optimization processing on the initial stitching matrix according to a preset point cloud registration algorithm to obtain an optimized matrix corresponding to the initial stitching matrix includes:

[0119] For each three-dimensional coordinate in the initial stitching matrix, determine whether the three-dimensional point corresponding to the three-dimensional coordinate is coordinate data obtained by laser measurement, if the three-dimensional point corresponding to the three-dimensional coordinate is coordinate data obtained by laser measurement, use the three-dimensional coordinate as the original coordinate, if the three-dimensional point corresponding to the three-dimensional coordinate is not coordinate data obtained by laser measurement, use the three-dimensional coordinate as the derived coordinate;

[0120] In the method provided by an embodiment of the present invention, in the process of three-dimensional coordinate reconstruction mentioned in step S102, each three-dimensional point is obtained by performing three-dimensional interpolation on the laser three-dimensional data, that is, the three-dimensional points of the current frame finally obtained include the three-dimensional coordinates of each laser imaging point obtained by laser measurement, and also include the three-dimensional coordinates of each interpolation point obtained by three-dimensional interpolation.

[0121] In the process of optimizing the initial stitching matrix, for each three-dimensional coordinate in the initial stitching matrix, it is determined whether the three-dimensional point corresponding to the three-dimensional coordinate is the coordinate data obtained by laser measurement, that is, it is identified whether the three-dimensional coordinate is the coordinate obtained by transforming the three-dimensional coordinate of the laser imaging point. If the three-dimensional point corresponding to the three-dimensional coordinate is the coordinate data obtained by laser measurement, the three-dimensional coordinate is marked as the original coordinate. If the three-dimensional point corresponding to the three-dimensional coordinate is not the coordinate data obtained by laser measurement, that is, the three-dimensional point corresponding to the three-dimensional coordinate is the coordinate data obtained by three-dimensional interpolation, then the three-dimensional coordinate is marked as a derived coordinate. In general, that is, for each three-dimensional coordinate in the initial stitching matrix, the three-dimensional coordinate obtained by transforming the three-dimensional coordinate of the laser imaging point is used as the original coordinate, and the three-dimensional coordinate obtained by transforming the three-dimensional coordinate of the interpolation point is used as the derived coordinate.

[0122] Using a preset first weight as the optimization weight corresponding to each of the original coordinates, and using a preset second weight as the optimization weight corresponding to each of the derived coordinates; the first weight is greater than the second weight;

[0123] In the method provided by the embodiment of the present invention, two weights may be pre-set, namely, a first weight and a second weight, wherein the first weight is greater than the second weight. In the data optimization process, the first weight is used as the optimization weight corresponding to the original coordinates, and the second weight is used as the optimization weight corresponding to the derived coordinates, that is, a relatively high optimization weight is assigned to the original coordinates, and a relatively low optimization weight is assigned to the derived coordinates.

[0124] Based on the optimization weights corresponding to the three-dimensional coordinates in the initial splicing matrix, an iterative closest point algorithm is applied to perform three-dimensional data optimization on the initial splicing matrix, and the optimization result is used as the optimization matrix corresponding to the initial splicing matrix.

[0125] In the method provided by the embodiment of the present invention, based on the optimization weights corresponding to each three-dimensional coordinate (original coordinates and derived coordinates) of the initial stitching matrix, an iterative closest point (ICP) algorithm is applied to perform registration optimization on the three-dimensional coordinates in the initial stitching matrix with corresponding weights, and the matrix obtained after optimization (i.e., the optimization result) is used as the optimization matrix corresponding to the initial stitching matrix.

[0126] Based on the method provided in an embodiment of the present invention, when three-dimensional coordinates are constructed by combining the laser measurement principle and the three-dimensional interpolation method, in the point cloud registration process, different weights are assigned to the three-dimensional data obtained by laser measurement and the three-dimensional data of the interpolation points, so that the point cloud registration focuses more on the three-dimensional data obtained by the actual measurement, which is conducive to improving the accuracy of the point cloud registration.

[0127] exist Figure 1 On the basis of the method shown, the method provided in the embodiment of the present invention further includes:

[0128] When the three-dimensional scanning process is completed, the global stitching matrix is ​​determined;

[0129] In the method provided in the embodiment of the present invention, during the three-dimensional scanning process, the system can be based on Figure 1 The method shown processes the image data corresponding to each frame in turn to obtain the three-dimensional data corresponding to each frame, and splices the three-dimensional data of each frame. After completing the overall three-dimensional scanning process, a three-dimensional matrix obtained by splicing all three-dimensional data will be obtained. At this time, the system can use the final spliced ​​three-dimensional matrix as the global splicing matrix.

[0130] The global stitching matrix is ​​globally optimized to obtain a global optimization matrix, and the global optimization matrix is ​​used as the global three-dimensional matrix of the scanned object.

[0131] In the method provided by the embodiment of the present invention, the three-dimensional data in the global stitching matrix can be globally optimized by a preset global optimization method to obtain a global optimization matrix, and the global optimization matrix is ​​used as the global three-dimensional matrix of the scanned object, that is, the point cloud data of the scanned object as a whole. For example, the data in the global stitching matrix can be optimized by assigning different weights to different types of three-dimensional coordinates and applying the ICP algorithm to optimize the data as mentioned in the previous embodiment. Specifically, for each three-dimensional coordinate in the global stitching matrix, it can be determined whether the three-dimensional coordinate is the coordinate data obtained based on laser measurement. If the three-dimensional coordinate is the coordinate data obtained based on laser measurement, the three-dimensional coordinate is marked as the original global coordinate. If the three-dimensional coordinate is not the coordinate data obtained based on laser measurement, that is, the three-dimensional coordinate is the coordinate data obtained based on three-dimensional interpolation, the three-dimensional coordinate is marked as a derived global coordinate. The preset third weight is used as the optimization weight corresponding to each original global coordinate, and the preset fourth weight is used as the optimization weight corresponding to each derived global coordinate, wherein the third weight is greater than the fourth weight. Based on the optimization weights corresponding to each three-dimensional coordinate in the global stitching matrix, the global stitching matrix is ​​globally refined and weighted fused to achieve global optimization.

[0132] Based on the method provided by the embodiment of the present invention, after the global splicing matrix is ​​obtained, the global splicing matrix can be further globally optimized to further improve the accuracy of data splicing.

[0133] In order to better illustrate the method provided by the embodiment of the present invention, based on the methods provided by the previous embodiments and combined with actual application scenarios, the embodiment of the present invention provides another method for stitching three-dimensional data. The method provided by the embodiment of the present invention is applied to a handheld laser scanner, which is provided with a laser projector, a color texture camera, and two or more (more than two) black and white cameras. The structure of the laser scanner can be as follows: Figure 2 As shown, the laser scanner is provided with a black and white camera 201, a color texture camera 202, a laser projector 203 and a black and white camera 204, wherein a white LED light is deployed on the color texture camera 202. It should be noted that, Figure 2 The above is only a brief schematic description for better illustrating the embodiment of the present invention, and is only an embodiment, and is not intended to limit the actual device structure and form. In actual application scenarios, the structure of the laser scanner can be designed according to actual needs.

[0134] In an embodiment of the present invention, the laser projector of the laser scanner can project multi-line lasers, and the number of laser beams is preferably not less than 15, and the imaging spacing between the beams on the image is preferably within 8 pixels. Next, an example of the image acquisition method during the scanning process is given. For example, if the laser band projected by the laser projector is an infrared band (a band that can be filtered by a color texture camera), then during the scanning process, the laser projector projects a laser beam to the surface of the object, and the surface of the object is photographed by each black and white camera to obtain a two-dimensional image of the laser line. At the same time, the color texture camera photographs the surface of the object to obtain a texture map, and the LED light of the color texture camera can flash synchronously to shield the interference of ambient light. If the laser band projected by the laser projector is a band that cannot be filtered by the color texture camera, then during the scanning process, the laser projector first projects a laser beam to the surface of the object, and the surface of the object is photographed by each black and white camera to obtain a two-dimensional image of the laser line, and then the laser projector is turned off, and within a certain time (for example, within 5ms), the surface of the object is photographed by the color texture camera to obtain a texture map, and the LED light of the color texture camera can flash synchronously to shield the interference of ambient light. The laser line two-dimensional image and texture image collected in a single data collection process (i.e., the black and white camera and the color texture camera shoot the same area once) according to the aforementioned image collection method are used as the image data of the same frame. After obtaining the image data of each frame, the data processing system processes the image data and performs three-dimensional data splicing.

[0135] In the embodiment of the present invention, the splicing process of the three-dimensional data can be as follows: Figure 3As shown, specifically including:

[0136] S301: Obtaining a laser line two-dimensional image and a texture image corresponding to the current frame;

[0137] In the method provided by the implementation of the present invention, when it is necessary to process the image data corresponding to the current frame, the two-dimensional image and texture image of the laser line corresponding to the current frame can be obtained.

[0138] S302: Reconstructing a depth map based on the laser line two-dimensional map;

[0139] In the method provided in the embodiment of the present invention, three-dimensional coordinates can be reconstructed based on the two-dimensional image of the laser line to obtain a depth map including the three-dimensional coordinates of each laser imaging point.

[0140] S303: performing three-dimensional interpolation processing on the basis of the depth map to obtain a three-dimensional data set, and marking original data points and interpolated data points in the three-dimensional data set;

[0141] In the method provided by the embodiment of the present invention, interpolation processing can be performed based on the three-dimensional coordinates of each laser imaging point in the depth map to obtain a three-dimensional data set. Specifically, three-dimensional interpolation processing can be performed by polynomial fitting. The three-dimensional coordinates of each laser imaging point in the three-dimensional data set are marked as original data points, and the three-dimensional coordinates of each interpolation point obtained by three-dimensional interpolation in the three-dimensional data set are marked as interpolation data points. The depth data of the depth map can be improved through three-dimensional interpolation, which is conducive to making the subsequent stitching process smoother and more fluent.

[0142] S304: Mapping the three-dimensional data set to a texture map;

[0143] In the method provided by the embodiment of the present invention, the three-dimensional data set is mapped to the texture map according to the mapping relationship between the two-dimensional laser line map and the three-dimensional data set and the mapping relationship between the two-dimensional laser line map and the texture map.

[0144] S305: Identify an image region in the texture map that has a mapping relationship with the three-dimensional data set, and extract texture features of the image region to obtain various texture features of the current frame;

[0145] In the method provided by the embodiment of the present invention, an image area mapped to a three-dimensional data set is extracted from a texture map, texture features are extracted based on texture information of the image area, and the extracted texture features are used as texture features of the current frame.

[0146] S306: performing texture feature matching on each texture feature of the current frame and each previous texture feature to determine a target texture feature among the texture features of the current frame;

[0147] In the method provided by an embodiment of the present invention, each texture feature extracted in the previous data processing process (for example, the texture feature of the previous frame or the current global texture feature) is matched with each texture feature of the current frame. If there is a texture feature among the texture features of the current frame that matches the previous texture feature, then the texture feature is used as the target texture feature.

[0148] S307: Acquire three-dimensional data corresponding to the target texture feature in the three-dimensional data set, and solve an initial splicing matrix based on the three-dimensional data corresponding to the target texture feature;

[0149] In the method provided by the embodiment of the present invention, according to the mapping relationship between the three-dimensional data in the three-dimensional data set and each texture feature of the current frame, the three-dimensional data corresponding to the target texture feature is extracted from the three-dimensional data set. The three-dimensional data corresponding to the target texture feature is used as the key feature point of data splicing, so as to solve the initial splicing matrix corresponding to the three-dimensional data set.

[0150] S308: assigning high weights to data associated with original data points in the initial splicing matrix, and assigning low weights to data associated with interpolated data points, and based on this, performing data optimization on the initial splicing matrix to obtain an optimized splicing matrix;

[0151] In the method provided by the embodiment of the present invention, the optimization weight of the three-dimensional data associated with the original data points in the initial stitching matrix is ​​set to a first weight, and the optimization weight of the three-dimensional data associated with the interpolated data points is set to a second weight, wherein the first weight is greater than the second weight. Based on the optimization weight of the three-dimensional data, the ICP algorithm is applied to perform real-time optimization on the three-dimensional data in the initial stitching matrix to obtain an optimized stitching matrix. By assigning a higher weight to the original data points and a lower weight to the interpolated data points, the point cloud registration can be made to focus more on the processing of the original data points, which is conducive to improving the accuracy of data stitching.

[0152] S309: performing data splicing and fusion of the optimized splicing matrix and the previous three-dimensional matrix to obtain a spliced ​​three-dimensional matrix;

[0153] In the method provided by the embodiment of the present invention, the optimized splicing matrix is ​​fused with the previous three-dimensional matrix (the three-dimensional matrix of the previous frame or the current global three-dimensional matrix) to obtain a spliced ​​three-dimensional matrix.

[0154] S310: Determine whether the three-dimensional scanning process is completed;

[0155] In the method provided in the embodiment of the present invention, it is possible to monitor whether the current 3D scanning process is finished, that is, to determine whether there is still 3D data that needs to be spliced. If the current 3D scanning process is finished, the process proceeds to step S312; if the current 3D scanning process is not finished, the process proceeds to step S311.

[0156] S311: Waiting for processing the next frame of data;

[0157] In the method provided by the embodiment of the present invention, if the scanning process is not completed currently, the image data of the next frame is waited to be processed. When the image data of the next frame is received, the data is spliced ​​according to the method of steps S301 to S309.

[0158] S312: Perform global refinement and weighted fusion on the three-dimensional matrix after data splicing.

[0159] In the method provided by the embodiment of the present invention, if the three-dimensional scanning process has been completed, the three-dimensional matrix that completes the overall data stitching, that is, the current global stitching matrix, is subjected to global refinement and weighted fusion of the three-dimensional data in the global stitching matrix using the ICP algorithm in accordance with the manner of assigning different optimization weights to different types of three-dimensional data in step S308, so as to complete the global data optimization, and the optimized three-dimensional matrix is ​​used as the final point cloud data for subsequent three-dimensional modeling.

[0160] Based on the method provided in the embodiment of the present invention, the splicing of three-dimensional data is achieved through the texture features of the object surface. During the three-dimensional scanning process, there is no need to perform complicated point-sticking operations or use auxiliary equipment such as trackers, which is beneficial to improving the convenience of three-dimensional scanning, improving scanning efficiency, improving user experience and reducing equipment costs.

[0161] and Figure 1 Corresponding to the three-dimensional data splicing method shown in FIG. 1 , an embodiment of the present invention further provides a three-dimensional data splicing device for Figure 1 The specific implementation of the method shown in is shown in the structural diagram Figure 4 As shown, including:

[0162] The first determination unit 401 is used to determine the laser three-dimensional data corresponding to the current frame and the texture map of the current frame;

[0163] A coordinate reconstruction unit 402 is used to perform three-dimensional coordinate reconstruction based on the laser three-dimensional data and a preset three-dimensional interpolation algorithm to obtain three-dimensional points of each current frame;

[0164] A second determining unit 403 is used to determine the mapping relationship between each of the current frame 3D points and the current frame texture map;

[0165] The third determining unit 404 is used to determine the feature matching relationship between the current frame texture image and other frame texture images;

[0166] A fourth determining unit 405 is used to determine the relative position relationship between each of the current frame 3D points and each of the other frame 3D points according to the mapping relationship and the feature matching relationship;

[0167] The data stitching unit 406 is used to perform 3D data stitching on each of the 3D points of the current frame and each of the 3D points of the other frames according to the relative position relationship.

[0168] By using the device provided by the embodiment of the present invention, each 3D point corresponding to the current frame can be mapped with the texture map corresponding to the current frame, and the texture map of the current frame can be feature matched with the texture maps of other frames. According to the mapping relationship between the 3D point of the current frame and the texture map of the current frame and the feature matching relationship between the texture map of the current frame and the texture maps of other frames, the 3D point associated with the 3D point of the current frame can be found in the 3D points of other frames, thereby determining the relative position relationship between the 3D point of the current frame and the 3D points of other frames, thereby realizing 3D data splicing. In the process of data splicing, the key feature points of data splicing can be identified by texture features, and there is no need to complete the 3D data splicing through the marker points deployed on the surface of the object. Therefore, there is no need to manually perform point pasting operations in the 3D scanning work, and the scanning process is relatively simple, which is conducive to improving scanning efficiency and user experience.

[0169] exist Figure 4 Based on the device shown, the device provided by the embodiment of the present invention can be further expanded into multiple units. The function of each unit can refer to the description of each embodiment provided in the previous text for the three-dimensional data splicing method, and no further examples will be given here.

[0170] An embodiment of the present invention further provides a storage medium, which includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the three-dimensional data splicing method as described above.

[0171] The embodiment of the present invention further provides an electronic device, the structural diagram of which is shown in FIG. Figure 5 As shown, it specifically includes a memory 501 and one or more instructions 502, wherein the one or more instructions 502 are stored in the memory 501 and are configured to be executed by one or more processors 503 to perform the following operations:

[0172] Determine the laser three-dimensional data corresponding to the current frame and the texture map of the current frame;

[0173] Reconstructing three-dimensional coordinates based on the laser three-dimensional data and a preset three-dimensional interpolation algorithm to obtain three-dimensional points of each current frame;

[0174] Determine a mapping relationship between each of the current frame three-dimensional points and the current frame texture map;

[0175] Determine a feature matching relationship between the current frame texture image and other frame texture images;

[0176] Determining the relative position relationship between each of the three-dimensional points of the current frame and each of the three-dimensional points of other frames according to the mapping relationship and the feature matching relationship;

[0177] Based on the relative position relationship, three-dimensional data splicing is performed on each of the three-dimensional points of the current frame and each of the three-dimensional points of the other frames.

[0178] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0179] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0180] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A three-dimensional data splicing method, characterized in that: include: Determine the laser three-dimensional data corresponding to the current frame and the texture map of the current frame; Reconstructing three-dimensional coordinates based on the laser three-dimensional data and a preset three-dimensional interpolation algorithm to obtain three-dimensional points of each current frame; Determine a mapping relationship between each of the current frame three-dimensional points and the current frame texture map; Determine a feature matching relationship between the current frame texture image and other frame texture images; Determining the relative position relationship between each of the three-dimensional points of the current frame and each of the three-dimensional points of other frames according to the mapping relationship and the feature matching relationship; Based on the relative position relationship, three-dimensional data splicing is performed on each of the three-dimensional points of the current frame and each of the three-dimensional points of the other frames.

2. The three-dimensional data splicing method according to claim 1, characterized in that: The three-dimensional coordinate reconstruction based on the laser three-dimensional data and a preset three-dimensional interpolation algorithm to obtain each current frame three-dimensional point includes: Obtaining the three-dimensional coordinates corresponding to each laser imaging point in the laser three-dimensional data; Applying the three-dimensional interpolation algorithm to perform three-dimensional interpolation processing on the three-dimensional coordinates corresponding to each of the laser imaging points to obtain the three-dimensional coordinates corresponding to each interpolation point; The three-dimensional coordinates corresponding to each of the laser imaging points and the three-dimensional coordinates corresponding to each of the interpolation points are used as each of the current frame three-dimensional points.

3. The three-dimensional data splicing method according to claim 1, characterized in that: Determining the feature matching relationship between the current frame texture image and other frame texture images includes: Determine each first texture feature corresponding to the current frame texture map; Determine each second texture feature corresponding to the other frame texture images; For each of the first texture features, performing feature matching processing on the first texture feature and each of the second texture features respectively to obtain a feature matching result corresponding to the first texture feature; A feature matching relationship between the current frame texture map and the other frame texture maps is determined according to the feature matching results corresponding to each of the first texture features.

4. The three-dimensional data splicing method according to claim 1, characterized in that: Determining the relative position relationship between each of the three-dimensional points of the current frame and each of the three-dimensional points of other frames according to the mapping relationship and the feature matching relationship includes: Determining a target area image in the current frame texture map according to the mapping relationship; Determining target texture features in the target area image that match the other frame texture images based on the feature matching relationship; Determine the current frame target point corresponding to the target texture feature among the current frame three-dimensional points; Determine, among the three-dimensional points of the other frames, target points of the other frames corresponding to the target texture features; The relative position relationship between each of the three-dimensional points in the current frame and each of the three-dimensional points in the other frames is determined according to the three-dimensional coordinates of the target point in the current frame and the three-dimensional coordinates of the target points in the other frames.

5. The three-dimensional data splicing method according to claim 1, characterized in that: The step of performing three-dimensional data stitching on each of the three-dimensional points of the current frame and each of the three-dimensional points of the other frames according to the relative position relationship includes: According to the relative position relationship, construct an initial stitching matrix corresponding to each of the three-dimensional points of the current frame; According to a preset point cloud registration algorithm, the initial stitching matrix is ​​subjected to data optimization processing to obtain an optimized matrix corresponding to the initial stitching matrix; Determine the other frame three-dimensional matrix corresponding to each of the other frame three-dimensional points; The optimization matrix corresponding to the initial stitching matrix is ​​stitched with the three-dimensional matrix of the other frames to obtain a stitching result, and the stitching result is used as the three-dimensional data stitching result of each three-dimensional point of the current frame and each three-dimensional point of the other frames.

6. The three-dimensional data splicing method according to claim 5, characterized in that: The step of performing data optimization processing on the initial stitching matrix according to a preset point cloud registration algorithm to obtain an optimized matrix corresponding to the initial stitching matrix includes: For each three-dimensional coordinate in the initial stitching matrix, determine whether the three-dimensional point corresponding to the three-dimensional coordinate is coordinate data obtained by laser measurement, if the three-dimensional point corresponding to the three-dimensional coordinate is coordinate data obtained by laser measurement, use the three-dimensional coordinate as the original coordinate, if the three-dimensional point corresponding to the three-dimensional coordinate is not coordinate data obtained by laser measurement, use the three-dimensional coordinate as the derived coordinate; Using a preset first weight as the optimization weight corresponding to each of the original coordinates, and using a preset second weight as the optimization weight corresponding to each of the derived coordinates; the first weight is greater than the second weight; Based on the optimization weights corresponding to the three-dimensional coordinates in the initial splicing matrix, an iterative closest point algorithm is applied to perform three-dimensional data optimization on the initial splicing matrix, and the optimization result is used as the optimization matrix corresponding to the initial splicing matrix.

7. The three-dimensional data splicing method according to claim 1, characterized in that: Also includes: When the three-dimensional scanning process is completed, the global stitching matrix is ​​determined; The global stitching matrix is ​​globally optimized to obtain a global optimization matrix, and the global optimization matrix is ​​used as the global three-dimensional matrix of the scanned object.

8. A three-dimensional data splicing device, characterized in that: include: A first determination unit, used to determine the laser three-dimensional data corresponding to the current frame and the texture map of the current frame; A coordinate reconstruction unit, used to perform three-dimensional coordinate reconstruction based on the laser three-dimensional data and a preset three-dimensional interpolation algorithm to obtain three-dimensional points of each current frame; A second determining unit, used to determine a mapping relationship between each of the current frame 3D points and the current frame texture map; A third determining unit, used to determine a feature matching relationship between the current frame texture image and other frame texture images; A fourth determining unit, configured to determine a relative position relationship between each of the three-dimensional points of the current frame and each of the three-dimensional points of other frames according to the mapping relationship and the feature matching relationship; The data stitching unit is used to perform three-dimensional data stitching on each of the three-dimensional points of the current frame and each of the three-dimensional points of the other frames according to the relative position relationship.

9. A storage medium, characterized in that: The storage medium includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the three-dimensional data splicing method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: It comprises a memory and one or more instructions, wherein the one or more instructions are stored in the memory and are configured to be executed by one or more processors to perform the three-dimensional data splicing method as described in any one of claims 1 to 7.

Images