Data processing method, apparatus, system, and storage medium

Abstract

The application discloses a data processing method, device, system and storage medium. The method comprises the following steps: determining feature points in first point cloud data and feature points in second point cloud data, wherein the first point cloud data and the second point cloud data are used for representing different parts of a same object; performing feature matching on the first point cloud data and the second point cloud data to determine feature points between the first point cloud data and the second point cloud data that meet a feature matching condition, thereby forming a plurality of feature point pairs; determining a transformation matrix between the feature points in the feature point pairs, wherein the transformation matrix is used for determining a spatial distance between the feature points in the feature point pairs that meets a proximity condition; and performing coordinate transformation on one or more feature point pairs in the plurality of feature point pairs through the transformation matrix, so as to register the first point cloud data and the second point cloud data. According to the data processing method provided in the embodiment of the application, the accuracy and stability of object surface feature registration can be improved.

Description

Technical Field

[0001] This invention relates to the field of computers, and more particularly to a data processing method, apparatus, system, and storage medium. Background Technology

[0002] In both Virtual Reality (VR) and Augmented Reality (AR) fields, registering collected three-dimensional (3D) single-view point clouds to form a complete 3D model is a crucial step in 3D display. The collection of point data on the outer surface of an object is called a point cloud. Since a 3D single-view point cloud can only reflect the 3D object information from that single viewpoint, to obtain complete, i.e., full-view 3D information about the object, multiple 3D single-view point clouds need to be registered, a process known as point cloud registration.

[0003] Registration based on object surface features is often limited by the accuracy of feature matching and is severely affected by noise, resulting in poor stability. Summary of the Invention

[0004] This invention provides a data processing method, apparatus, system, and storage medium that can improve the accuracy and stability of object surface feature registration.

[0005] According to one aspect of the present invention, a data processing method is provided, comprising:

[0006] The feature points in the first point cloud data and the feature points in the second point cloud data are determined. The first point cloud data and the second point cloud data are used to represent different parts of the same object.

[0007] Feature matching is performed on the first point cloud data and the second point cloud data to determine the feature points that meet the feature matching conditions between the first point cloud data and the second point cloud data, thus forming multiple feature point pairs;

[0008] For one or more feature point pairs among multiple feature point pairs, determine the transformation matrix that satisfies the proximity condition for the spatial distance between feature points in the feature point pair.

[0009] By using a transformation matrix, coordinate transformation is performed on one or more feature point pairs from multiple feature point pairs to register the first point cloud data with the second point cloud data.

[0010] According to another aspect of the present invention, a data processing apparatus is provided, comprising:

[0011] The feature point acquisition module is used to determine the feature points in the first point cloud data and the feature points in the second point cloud data. The first point cloud data and the second point cloud data are used to represent different parts of the same object.

[0012] The feature matching module is used to perform feature matching on the first point cloud data and the second point cloud data to determine the feature points that meet the feature matching conditions between the first point cloud data and the second point cloud data, thus forming multiple feature point pairs.

[0013] The feature point pair filtering module is used to determine, for one or more feature point pairs among multiple feature point pairs, the transformation matrix in which the spatial distance between feature points in the feature point pair meets the proximity condition.

[0014] The data registration module is used to perform coordinate transformation on one or more feature point pairs from multiple feature point pairs through a transformation matrix, so as to register the first point cloud data with the second point cloud data.

[0015] According to another aspect of the present invention, a data processing system is provided, comprising: a memory and a processor; the memory is used to store a program; the processor is used to read executable program code stored in the memory to execute the above-described data processing method.

[0016] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores instructions that, when executed on a computer, cause the computer to perform the data processing methods described above.

[0017] The data processing method, apparatus, system, and storage medium according to the embodiments of the present invention can improve the accuracy and stability of object surface feature registration.

[0018] According to another aspect of the present invention, a data processing method is provided, comprising:

[0019] The object in the real scene is scanned in three dimensions from multiple shooting angles to obtain multiple point data of the object.

[0020] Based on the multiple point data, multiple point cloud data of the object are constructed under multiple shooting perspectives, and the multiple point cloud data includes point cloud data of at least two coordinate systems;

[0021] The point cloud data of the at least two coordinate systems are processed to unify the coordinate system, thereby obtaining a three-dimensional point cloud data model of the object.

[0022] In one or more possible embodiments, the step of unifying the coordinate system of the point cloud data from the at least two coordinate systems to obtain a three-dimensional point cloud data model of the object includes:

[0023] Using a unified coordinate system, coordinate transformation is performed on the point cloud data of at least two coordinate systems to obtain a three-dimensional point cloud data model of the object.

[0024] In another or more possible embodiments, the unified coordinate system corresponds to the transformation matrix, the transformation matrix including rotation and translation components; the step of performing coordinate transformation on the point cloud data of the at least two coordinate systems according to the unified coordinate system to obtain the three-dimensional point cloud data model of the object includes:

[0025] Based on the rotation and translation components, coordinate transformation is performed on the point cloud data of the at least two coordinate systems to obtain a three-dimensional point cloud data model of the object.

[0026] The rotation component is used to characterize the rotational relationship between point cloud data in each of the at least two coordinate systems; the translation component is used to characterize the translational relationship between point cloud data in each of the at least two coordinate systems.

[0027] In yet another possible embodiment, the point cloud data of each of the at least two coordinate systems corresponds to a polygonal patch model, the polygonal patch model includes multiple polygonal patches, the polygonal patches include point data, and the unified coordinate system corresponds to the transformation matrix;

[0028] The step of unifying the coordinate system of the point cloud data from the at least two coordinate systems to obtain the three-dimensional point cloud data model of the object includes:

[0029] The polygonal patch model corresponding to the point cloud data of each coordinate system is taken as the patch to be processed;

[0030] The point data of each facet in the multiple facets of each polygon facet model are transformed using the transformation matrix to obtain the three-dimensional point cloud data model of the object.

[0031] In one or more possible embodiments, the point cloud data of each of the at least two coordinate systems corresponds to a polygonal patch model.

[0032] The step of unifying the coordinate system of the point cloud data from the at least two coordinate systems to obtain the three-dimensional point cloud data model of the object includes:

[0033] The polygonal patch model corresponding to the point cloud data of each coordinate system is taken as the patch to be processed to obtain multiple patches to be processed of the object. The multiple patches to be processed include a first patch to be processed and a second patch to be processed.

[0034] Determine the feature points in the first surface patch to be processed, and determine the feature points in the second surface patch to be processed;

[0035] Feature matching is performed on the surface features of the feature points in the first surface to be processed and the surface features of the feature points in the second surface to be processed, respectively, to obtain feature point pairs that meet the feature matching conditions for the first surface to be processed and the second surface to be processed.

[0036] From the feature point pairs that meet the feature matching conditions of the first and second facets, remove the erroneous feature point pairs between the first and second facets to obtain the feature point matching results of the first and second facets.

[0037] Based on the feature point matching results of the first and second facets to be processed, a three-dimensional point cloud data model of the object is generated.

[0038] In one or more possible embodiments, determining the feature points in the first surface patch to be processed and determining the feature points in the second surface patch to be processed includes:

[0039] The surface features of the object are extracted from the first surface to be processed, and the surface features of the object are extracted from the second surface to be processed.

[0040] By extracting the surface features of the first surface to be processed, feature points in the first surface to be processed are determined; and by extracting the surface features of the second surface to be processed, feature points in the second surface to be processed are determined.

[0041] In one or more possible embodiments, the surface features include geometric features, which at least include the normal vector or curvature of the sampled points on the surface of the object;

[0042] The step of determining feature points in the first surface patch by extracting the surface features of the first surface patch to be processed includes:

[0043] Feature points in the first surface patch are extracted using the normal vectors of the sampling points in the first surface patch; or...

[0044] Feature points in the first surface patch to be processed are extracted by utilizing the curvature of the sampling points in the first surface patch to be processed.

[0045] In one or more possible embodiments, extracting feature points in the first surface patch by utilizing the normal vectors of the sampling points in the first surface patch includes:

[0046] Obtain the sampling points in the first surface patch to be processed;

[0047] Calculate the normal vector of the sampling point in the first patch to be processed;

[0048] Calculate the gradient of the normal vector of the sampling point in the first patch to be processed. The gradient of the normal vector is used to represent the direction of change of the normal vector of the sampling point, and the gradient value of the normal vector is used to measure the rate of change of the normal vector of the sampling point.

[0049] When the gradient of the normal vector at the sampling point reaches an extreme value, it is determined that the normal vector of the first patch to be processed undergoes a sudden change at the sampling point.

[0050] When the normal vector of the first patch to be processed changes abruptly at the sampling point, the sampling point is taken as a feature point in the first patch to be processed.

[0051] In one or more possible embodiments, extracting feature points in the first surface patch by utilizing the curvature of the sampling points in the first surface patch includes:

[0052] Obtain the sampling points in the first surface patch to be processed;

[0053] Calculate the curvature values ​​of the sampling points in the first surface patch to be processed;

[0054] Calculate the gradient of curvature of the sampling points in the first surface to be processed. The gradient of curvature is used to represent the direction of curvature change of the sampling points, and the gradient value of curvature is used to measure the rate of curvature change of the sampling points.

[0055] When the gradient of the curvature at the sampling point reaches an extreme value, it is determined that the curvature of the first patch to be processed has abruptly changed at the sampling point;

[0056] When it is determined that the curvature of the first surface to be processed changes abruptly at the sampling point, the sampling point is taken as a feature point in the first surface to be processed.

[0057] In one or more possible embodiments, the surface feature includes a texture feature, which includes at least the brightness or grayscale of the sampled points on the surface of the object;

[0058] The step of determining feature points in the first surface patch by extracting the surface features of the first surface patch to be processed includes:

[0059] Feature points in the first surface patch are extracted based on the brightness variation of sampling points within the patch; or...

[0060] By extracting the texture features of the first surface to be processed, the feature points in the first surface to be processed are determined.

[0061] In one or more possible embodiments, extracting feature points in the first surface patch by measuring the brightness variation of sampling points in the first surface patch includes:

[0062] Obtain the sampling points in the first surface patch to be processed;

[0063] Calculate the brightness value of the sampling point in the first area to be processed;

[0064] Calculate the gradient of the brightness value of the sampling point in the first area to be processed, whereby the gradient of the brightness value is used to represent the direction of the fastest brightness change at the sampling point;

[0065] When the gradient of the brightness at the sampling point reaches an extreme value, it is determined that the brightness of the first patch to be processed at the sampling point has changed abruptly.

[0066] When it is determined that the brightness of the first patch to be processed changes abruptly at the sampling point, the sampling point is taken as a feature point in the first patch to be processed.

[0067] In one or more possible embodiments, the feature point pairs of the first and second patches to be processed that meet the feature matching conditions are used to represent the object relationship between the feature points of the first and second patches to be processed; the feature matching conditions include the minimum difference in the feature values ​​of the feature points or the difference being within a preset feature threshold range.

[0068] The step of performing feature matching on the surface features of feature points in the first surface to be processed and the surface features of feature points in the second surface to be processed, respectively, to obtain feature point pairs that meet the feature matching conditions for the first surface to be processed and the second surface to be processed, includes:

[0069] Based on the same specified features, feature matching is performed on the surface features of feature points in the first facet to be processed and the surface features of feature points in the second facet to be processed, respectively. The specified features include at least one of the following: structural features and texture features.

[0070] If the difference between the feature values ​​of the feature points in the first surface to be processed and the feature values ​​of the feature points in the second surface to be processed is minimized, or if the difference is within a preset feature threshold range, then the feature point pair that meets the feature matching condition for the first surface to be processed and the second surface to be processed is determined.

[0071] In one or more possible embodiments, the method further includes:

[0072] Obtain the first feature point in the second surface patch to be processed;

[0073] Using the feature value of the first feature point, a second feature point is found in the first area to be processed, and the feature value of the second feature point matches the feature matching condition with the feature value of the first feature point.

[0074] When the third feature point and the second feature point in the first surface patch to be processed coincide, the third feature point and the second feature point are determined to be the same feature point;

[0075] The third feature point in the first surface to be processed and the first feature point in the second surface to be processed are determined as feature point pairs that meet the feature matching conditions between the first surface to be processed and the second surface to be processed.

[0076] In one or more possible embodiments, the feature point pairs include erroneous feature point pairs and valid feature point pairs;

[0077] The step of removing erroneous feature point pairs between the first and second surfaces from the feature point pairs that meet the feature matching conditions to obtain the feature point matching results for the first and second surfaces includes:

[0078] An evaluation model is constructed based on spatial distance and precision control parameters;

[0079] Using the evaluation model, the spatial distance between the matching feature point pairs between the first and second facets that meet the feature matching conditions is calculated.

[0080] Among the feature point pairs that meet the feature matching conditions of the first and second face patches to be processed, the feature point pairs that correspond to the spatial distance are determined as valid feature point pairs.

[0081] From the feature point pairs that meet the feature matching conditions of the first and second facets to be processed, remove the erroneous feature point pairs other than the valid feature point pairs to obtain the feature point matching results of the first and second facets to be processed.

[0082] In one or more possible embodiments, the evaluation model includes an objective function and constraints, the objective function being used to reduce the number of erroneous feature point pairs, and the constraints being accuracy control parameters used to control the accuracy of the calculation of the spatial distance;

[0083] The step of calculating the spatial distance between matching feature point pairs between the first and second face patches that meet the feature matching conditions using the evaluation model includes:

[0084] Based on the objective function, the precision control parameters, the set of feature points in the first surface to be processed, the set of feature points in the second surface to be processed, and the transformation matrix of feature points between the first surface to be processed and the second surface to be processed, calculate the spatial distance between the matching feature point pairs between the first surface to be processed and the second surface to be processed.

[0085] In one or more possible embodiments, determining the feature point pairs corresponding to the spatial distance among the feature point pairs of the first and second facets that meet the feature matching conditions as valid feature point pairs includes:

[0086] Set the initial value of the precision control parameter;

[0087] The spatial distance between the matching feature point pairs between the first and second facets to be processed is calculated using the evaluation model.

[0088] The value of the precision control parameter is gradually reduced, and the spatial distance between the matching feature point pairs between the first and second facets to be processed is iteratively calculated using the evaluation model.

[0089] When the minimum spatial distance between the first and second facets to be processed is calculated, the feature point pair corresponding to the spatial distance is determined as the effective feature point pair.

[0090] In one or more possible embodiments, the method further includes:

[0091] By using 3D reconstruction technology, the point cloud data of each coordinate system is reconstructed to obtain a polygonal patch model corresponding to the point cloud data of each coordinate system.

[0092] In one or more possible embodiments, the patch to be processed includes point cloud data;

[0093] The step of extracting surface features of the object from the first surface patch to be processed, and extracting surface features of the object from the second surface patch to be processed, includes:

[0094] The point cloud data in the first and second facets to be processed are determined as the point set of the surface features of the object.

[0095] The surface features of the object are analyzed to obtain the surface features of the first surface to be processed and the surface features of the second surface to be processed.

[0096] In one or more possible embodiments, the step of performing a three-dimensional scan of an object in a real scene according to multiple shooting angles to obtain multiple point data of the object includes: projecting structured light onto the object according to multiple shooting angles;

[0097] By acquiring feedback images from the multiple shooting perspectives, multiple point data of the object are obtained.

[0098] In one or more possible embodiments, the plurality of point data includes point data for each of the plurality of shooting angles;

[0099] The process of constructing multiple point cloud data of the object from multiple shooting perspectives based on the multiple point data includes:

[0100] The data set of point data for each of the multiple shooting perspectives is determined as the point cloud data for each shooting perspective;

[0101] The point cloud data of each shooting angle is defined as point cloud data in a coordinate system. Attached Figure Description

[0102] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0103] Figure 1a This illustrates a first surface model to be processed corresponding to an object from a first shooting perspective, according to an embodiment of the present invention.

[0104] Figure 1b This illustrates a second surface model to be processed corresponding to an object from a second shooting perspective, according to an embodiment of the present invention.

[0105] Figure 1c This illustrates a third surface model to be processed corresponding to an object from a third shooting perspective, according to an embodiment of the present invention.

[0106] Figure 1d This illustrates a fourth surface model to be processed corresponding to an object from a fourth shooting perspective, according to an embodiment of the present invention.

[0107] Figure 1e This is a schematic diagram illustrating the effect of a three-dimensional point cloud data model obtained by the data processing method according to an embodiment of the present invention.

[0108] Figure 2 This is a flowchart illustrating a data processing method according to an embodiment of the present invention;

[0109] Figure 3 This is a schematic diagram illustrating the structure of a data processing apparatus according to an embodiment of the present invention;

[0110] Figure 4 This is a structural diagram illustrating an exemplary hardware architecture of a computing device that can implement the data processing method and apparatus according to embodiments of the present invention. Detailed Implementation

[0111] The features and exemplary embodiments of various aspects of the present invention will now be described in detail. To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present invention by illustrating examples of the invention.

[0112] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0113] In this embodiment of the invention, AR technology can capture real-world images and videos through the system's own camera terminal, use image registration and other techniques to estimate the three-dimensional pose of virtual objects in the images and videos, and then place the virtual objects or scenes into predetermined positions in the real environment. Finally, the scene is rendered through the camera lens.

[0114] In this embodiment of the invention, VR technology is a computer simulation system in which virtual worlds can be created and experienced. Essentially, the system utilizes a computer to generate a simulated environment that includes an interactive, three-dimensional dynamic visual scene based on multi-source information fusion and a system simulation of entity behavior, achieving an immersive experience.

[0115] In the above embodiments, when augmented reality and virtual reality technologies require the use of three-dimensional scenes, the following steps may be included:

[0116] First, obtain the point cloud data of the object.

[0117] In this embodiment of the invention, instruments and equipment capable of 3D scanning, such as 3D scanners, laser or radar scanners, and stereo cameras, can be used to project structured light onto objects in a real scene to perform 3D scanning from multiple perspectives, such as 360 degrees, and obtain multiple point data of objects in the real scene. These point data constitute point cloud data corresponding to different perspectives.

[0118] Secondly, point cloud registration is performed based on the point cloud data of the object corresponding to different viewpoints.

[0119] As an example, in a camera coordinate system, objects can be scanned in 3D from different shooting angles. Point cloud data from different shooting angles may sometimes exhibit rotational or translational misalignment. Therefore, to obtain complete point cloud data, multiple point clouds need to be registered.

[0120] In the following description of the embodiments of the present invention, multiple point cloud data obtained from different shooting angles when the object is rotated or translated during a three-dimensional scan can be regarded as having different angles or in different coordinate systems.

[0121] Therefore, in this embodiment of the invention, point cloud registration can be understood as: performing a unified coordinate system operation on point cloud data, transforming the point cloud data of objects taken from different perspectives or in different coordinate systems through rotation and / or translation operations, so as to achieve consistent alignment of point cloud data in a unified coordinate system and form a complete three-dimensional point cloud data model.

[0122] In this embodiment of the invention, coordinate transformation between point cloud data can be performed using a transformation matrix T.

[0123] As an example, the transformation matrix can be represented as T = [R t], where R is the rotation component, which can be represented as a 3×3 matrix, representing the rotation relationship between two point cloud data from different viewpoints; t is the translation component, which can be represented as a 1×3 matrix, representing the translation relationship between two point cloud data from different viewpoints. By using this transformation matrix T, the coordinates of the point cloud data corresponding to different viewpoints are transformed to a unified coordinate system.

[0124] In the point cloud data registration process described above, the surface reconstruction method of 3D reconstruction technology can be used to obtain a polygonal patch model of the object's first point cloud data based on the acquired first point cloud data. This polygonal patch model includes multiple polygonal patches, and each polygonal patch can uniquely identify a polygonal plane.

[0125] In this embodiment of the invention, point cloud data of the same object under different viewpoints or different coordinate systems can be registered. Specifically, the polygonal patch model corresponding to each point cloud model obtained through 3D reconstruction can be used as the patch to be processed. Using a transformation matrix, the point data in each patch to be processed under different viewpoints or coordinate systems can be transformed to a unified coordinate system to obtain a complete 3D point cloud data model of the object.

[0126] To better understand this invention, taking a shoe as a specific object, a polygonal patch model of the object from different perspectives is obtained using point cloud models. This polygonal patch model is then used as the patch model to be processed, and the data processing method according to an embodiment of the invention is described. It should be understood that this description should not be construed as limiting the scope or implementation possibilities of this solution. For data processing methods of specific objects other than shoes, the method should be combined with… Figures 1a to 1e The specific implementation of the described data processing methods remains consistent.

[0127] Figure 1a A first surface model to be processed, corresponding to an object with a first shooting perspective, is shown according to an embodiment of the present invention. Figure 1b A second surface model to be processed, corresponding to an object from a second shooting perspective, is shown according to an embodiment of the present invention. Figure 1c A third surface model to be processed is shown according to an embodiment of the present invention, corresponding to an object from a third shooting perspective; Figure 1d A fourth surface model to be processed, corresponding to an object with a fourth shooting perspective, is shown according to an embodiment of the present invention. Figure 1e A schematic diagram illustrating the effect of a three-dimensional point cloud model obtained by the data processing method according to an embodiment of the present invention is shown.

[0128] exist Figures 1a to 1d In the diagram, the first patch model to be processed is the reconstruction result of multiple first point cloud data obtained from a first-view perspective using 3D reconstruction technology; the second patch model to be processed is the reconstruction result of multiple second point cloud data obtained from a second-view perspective using 3D reconstruction technology; the third patch model to be processed is the reconstruction result of multiple third point cloud data obtained from a third-view perspective using 3D reconstruction technology; and the fourth patch model to be processed is the reconstruction result of multiple fourth point cloud data obtained from a fourth-view perspective using 3D reconstruction technology. Combined with... Figures 1a to 1eThis allows for a detailed explanation of the data processing method's process and a illustrative demonstration of its effects.

[0129] In the following description, the patch model to be processed can be referred to simply as the patch to be processed.

[0130] In this embodiment of the invention, two surfaces to be processed can be selected from the surfaces to be processed of the object, for example... Figure 1a The first surface to be processed shown and Figure 1b The second surface to be processed, as shown, is used to perform the data processing method 100 of this embodiment of the invention.

[0131] In this embodiment, the data processing method 100 may include:

[0132] Step S110: Determine the feature points in the first surface to be processed and determine the feature points in the second surface to be processed.

[0133] Specifically, firstly, the surface features of the object can be extracted from the first surface to be processed, and the surface features of the object can be extracted from the second surface to be processed; then, the feature points in the first surface to be processed can be determined based on the extracted surface features, and the feature points in the second surface to be processed can be determined based on the extracted surface features.

[0134] In this embodiment of the invention, the point cloud data in each surface patch to be processed can be understood as a set of points representing the surface features of an object. By analyzing the point cloud data, the surface features of the object can be obtained.

[0135] Specifically, surface features can include geometric features and texture features. Geometric features can at least include the normal vector or curvature of the sampled points on the object's surface, and texture features can at least include the brightness or grayscale of the sampled points on the object's surface.

[0136] As an example, feature points in the first patch to be processed can be extracted using the normal vectors of the sampling points in the first patch to be processed.

[0137] In this example, the degree of change in the normal vector of the sampling points of the surface to be processed can be used to measure the undulation or flatness of the surface. The greater the degree of change in the normal vector of the sampling point, the greater the undulation of the region where that point is located.

[0138] Specifically, the sampling points in the first surface to be processed are obtained; the normal vectors of the sampling points in the first surface to be processed are calculated; the gradient of the normal vector of the sampling point in the first surface to be processed is calculated. The gradient of the normal vector can be used to represent the direction of change of the normal vector of the current sampling point, and the gradient value of the normal vector can be used to measure the rate of change of the normal vector of the current sampling point. When the gradient of the normal vector of the sampling point reaches an extreme value, it indicates that the normal vector of the first surface to be processed has changed abruptly at the current sampling point, and the sampling point can be used as a feature point in the first surface to be processed.

[0139] As another example, feature points in the first patch to be processed can be extracted by utilizing the curvature of the sampling points of the first patch to be processed.

[0140] In this example, the degree of curvature variation at the sampling points of the surface to be processed can be used to measure the degree of surface curvature. The greater the degree of curvature variation, the greater the degree of surface curvature, i.e., the lower the smoothness of the surface.

[0141] Specifically, sampling points in the first surface to be processed are obtained; the curvature values ​​of the sampling points in the first surface to be processed are calculated; the gradient of the curvature of the sampling points in the first surface to be processed is calculated. The gradient of curvature can be used to represent the direction of curvature change of the current sampling point, and the gradient value of curvature can be used to measure the rate of curvature change of the current sampling point. When the gradient of the curvature of the sampling point reaches an extreme value, it indicates that the curvature of the first surface to be processed has changed abruptly at the current sampling point, and the sampling point can be used as a feature point in the first surface to be processed.

[0142] In another embodiment, feature points in the first patch to be processed can be determined by extracting the texture features of the first patch to be processed.

[0143] As an example, feature points in the first patch to be processed can be extracted by utilizing the degree of brightness variation of the sampling points of the first patch to be processed.

[0144] Specifically, sampling points in the first area to be processed can be obtained; the brightness value of the sampling points in the first area to be processed can be calculated; the gradient of the brightness value of the sampling points in the first area to be processed can be calculated. The gradient of the brightness value can be used to represent the direction of the fastest change in brightness of the current sampling point. When the gradient of the brightness of the sampling point reaches an extreme value, it indicates that the brightness of the first area to be processed has changed abruptly at the current sampling point, and the sampling point can be used as a feature point in the first area to be processed.

[0145] As an example, feature points in the first surface to be processed can be extracted by utilizing the grayscale variation of the sampling points. This process is basically the same as the principle and steps of extracting feature points in the first surface to be processed by utilizing the brightness variation of the sampling points, and will not be repeated here.

[0146] Through step S110, feature points in the first patch to be processed can be obtained, for example... Figure 1a Feature points A1, A2, and A3 are shown.

[0147] like Figure 1b As shown, the steps for extracting feature points through the above embodiments are used to... Figure 1b The second surface to be processed is subjected to feature point extraction operation to obtain feature points such as feature point B1, feature point B2 and feature point B3 in the second surface to be processed.

[0148] like Figure 1c As shown, the steps for extracting feature points through the above embodiments are used to... Figure 1c The third surface to be processed is subjected to feature point extraction operation to obtain feature points such as feature point C1, feature point C2 and feature point C3 in the third surface to be processed.

[0149] like Figure 1d As shown, the steps for extracting feature points through the above embodiments are used to... Figure 1d The fourth surface to be processed is subjected to feature point extraction operation to obtain feature points such as feature point D1, feature point D2 and feature point D3 in the fourth surface to be processed.

[0150] Step S120: Perform feature matching on the surface features of the feature points in the first surface to be processed and the surface features of the feature points in the second surface to be processed to obtain feature point pairs that meet the feature matching conditions for the first surface to be processed and the second surface to be processed.

[0151] In this step, feature point pairs represent the object relationship between feature points of the first and second facets to be processed.

[0152] In one embodiment, the feature matching condition is: based on the same specified feature, the difference between the feature value of the feature point in the first patch to be processed and the feature value of the feature point in the second patch to be processed is the smallest, or the difference is within a preset feature threshold range.

[0153] As can be seen from the above embodiments, the specified feature can be a structural feature or a texture feature.

[0154] If the specified feature is curvature, as an example, such as Figure 1a and 1b As shown, when the difference between the curvature value of feature point A1 in the first surface to be processed and the curvature value of feature point B1 in the second surface to be processed is the smallest, or when the difference is within the preset curvature threshold range, feature point A1 in the first surface to be processed and feature point B1 in the second surface to be processed are considered to have similar structures.

[0155] As an example, if the specified feature is brightness, when the difference between the brightness value of feature point A1 in the first area to be processed and the brightness value of feature point B1 in the second area to be processed is the smallest, or when the difference is within a preset brightness threshold range, feature point A1 in the first area to be processed and feature point B1 in the second area to be processed are considered to have similar textures.

[0156] As another example, one or more specified features can be combined to determine the similarity between feature point A1 in the first facet to be processed and feature point B1 in the second facet to be processed.

[0157] Specifically, the curvature and normal vector of the sampling points can be combined to determine whether feature point A1 in the first surface to be processed and feature point B1 in the second surface to be processed are structurally similar.

[0158] Specifically, the brightness and grayscale of the sampling points can be combined to determine whether feature point A1 in the first area to be processed and feature point B1 in the second area to be processed have similar textures.

[0159] Specifically, the structural and texture features of the sampling points can be combined to determine whether feature point A1 in the first patch to be processed and feature point B1 in the second patch to be processed are structurally and texturally similar.

[0160] In one embodiment, feature point A1 in the first surface to be processed is obtained, and feature point B1 in the second surface to be processed is found using the feature value of feature point A1. Based on the same specified feature, the feature value of feature point B1 and the feature value of feature point A1 meet the feature matching condition.

[0161] In other words, feature point B1 in the second facet to be processed can be considered as a matching point of feature point A1 in the first facet to be processed, and feature point A1 and feature point B1 form a feature point pair.

[0162] To obtain more accurate feature point pairs, the matching relationship between feature point A1 and feature point B1 can be further verified.

[0163] Specifically, in one embodiment, feature point B1 in the second surface to be processed in the above embodiment is obtained, and feature point A1 is further searched in the first surface to be processed using the feature value of feature point B1. ′ (Not shown in the figure) Feature point A1 ′ The feature value of feature point B1 meets the feature matching condition of this embodiment of the invention.

[0164] In this embodiment, when feature point A1 and feature point A1 in the first patch to be processed ′ When they coincide, that is, feature point A1 and feature point A1′ When the feature points are the same, feature point A1 in the first facet to be processed and feature point B1 in the second facet to be processed can be regarded as a feature point pair that meets the feature matching conditions between the first facet to be processed and the second facet to be processed.

[0165] Continue to refer to Figure 1a and Figure 1b By using the method described in the above embodiments, other feature point pairs of the first and second facets to be processed are obtained.

[0166] As an example, other feature point pairs may include, for example, feature point A2 in the first facet to be processed and feature point B2 in the second facet to be processed, and feature point A3 in the first facet to be processed and feature point B3 in the second facet to be processed, etc.

[0167] The method for finding feature point pairs in the above embodiments can improve the reliability of the obtained feature point pairs.

[0168] Step S130: Remove erroneous feature point pairs between the first and second face patches to be processed, and purify and optimize the feature point pairs.

[0169] In this step, the feature point pairs obtained through step S120 can be used as initial feature point pairs. Due to the influence of noise, erroneous feature point pairs will inevitably appear in the initial feature point pairs. Not all feature point pairs obtained through the above steps represent accurate correspondences between feature points of two patches to be processed.

[0170] Since erroneous feature point pairs can negatively impact the registration results between two surfaces to be processed, such as reducing registration accuracy, causing misalignment of the overall data model, and affecting visual effects, it is necessary to refine and optimize the correspondence between feature points represented by the initial feature point pairs based on step S120, removing erroneous feature point pairs from the initial feature point pairs.

[0171] In this embodiment of the invention, the basic idea of ​​purifying and optimizing feature point pairs is to construct an evaluation model and calculate the spatial distance of the initial registration point pairs in the above embodiment based on the constructed evaluation model.

[0172] Specifically, when the spatial distance is calculated to be at its minimum value, the coordinate transformation matrix corresponding to the minimum value of the spatial distance is obtained, as well as the feature point pair in the initial feature point pair that corresponds to the spatial distance. The coordinate transformation matrix is ​​used as the coordinate transformation matrix T, and the aforementioned feature point pair that corresponds to the spatial distance is used as the effective feature point pair.

[0173] In one embodiment, to improve the accuracy and stability of the calculated coordinate transformation matrix and effective feature point pairs, the evaluation model may include an objective function and constraints. The constraints can be used to control the accuracy of the spatial distance calculation, and the objective function can be used to reduce the number of erroneous feature point pairs and suppress the impact of erroneous feature point pairs on the registration accuracy.

[0174] In one embodiment, the objective function can be a penalty function, such as the error function of the Welsh function, the error function of the Tukey function, or the error function of the Gaman-McClure function. The following example, using the Gaman-McClure error function as the objective function, illustrates a method for purifying and optimizing feature point pairs according to an embodiment of the present invention.

[0175] In one embodiment, the constraint can be a precision control parameter, which controls the precision of spatial distance calculation within its range.

[0176] In one embodiment, the objective function for evaluating the model can be expressed by the following formula (1):

[0177]

[0178] In the above formula (1), f(x) represents the objective function of the evaluation model, x represents the spatial distance between the feature point pairs of the two patches to be matched, and μ represents the accuracy control parameter.

[0179] In this embodiment, the value of μ represents the accuracy of solving the objective function. The smaller the value of μ, the higher the accuracy of solving the objective function, which means that the optimization and purification process of feature point pairs can remove more erroneous feature point pairs.

[0180] In one embodiment, the spatial distance between feature point pairs of two patches to be matched can be represented by the following formula (2):

[0181] x=||P-TQ|| (2)

[0182] In the above formula (2), P represents the set of feature points in the first facet to be processed, Q represents the set of feature points in the second facet to be processed, T represents the transformation matrix of feature points between the first facet to be processed and the second facet to be processed, and x represents the spatial distance between the matching feature point pairs between the first facet to be processed and the second facet to be processed.

[0183] In one embodiment, the spatial distance x can be solved using the above formulas (1) and (2) through a nonlinear least squares algorithm.

[0184] As an example, to achieve faster and more accurate solutions, the spatial distance x can also be calculated using, for example, the Gauss-Newton method.

[0185] In this embodiment, the process of solving for the spatial distance x between the matching feature point pairs between the first and second facets using a nonlinear least squares algorithm, such as the Gauss-Newton method, can be expressed as:

[0186] Set an initial value for the precision control parameter μ, and use the evaluation model to calculate the spatial distance x. Gradually reduce the value of the precision control parameter μ, and use the evaluation model to iteratively calculate the spatial distance x. When the minimum value of the spatial distance x is obtained, obtain the coordinate transformation matrix corresponding to the minimum value of the spatial distance x and the feature point pair corresponding to the spatial distance x. Use the coordinate transformation matrix as the coordinate transformation matrix T, and use the feature point pair as the effective feature point pair.

[0187] As a concrete example, using a nonlinear least squares algorithm such as the Gauss-Newton method to solve for the spatial distance x based on the above formulas (1) and (2) can include:

[0188] S01, obtain the initial value μ0 of the precision control parameter μ, calculate the spatial distance x using the above formulas (1) and (2) based on the initial value μ0 of the precision control parameter, and determine the spatial coordinate transformation matrix T1 by using the calculated spatial distance x.

[0189] In this step, the effective feature point pairs between the first and second facets are transformed to a unified coordinate system using the coordinate transformation matrix T1, and the spatial distance x can be obtained. As an example, the spatial distance x includes, for example, x1, x2, x3, x4, and x5.

[0190] S02, reduce the precision control parameter to obtain a new precision parameter value μ1. Based on the new precision parameter value μ1, use the above formula (1) and the above formula (2) to calculate the spatial distance x.

[0191] In this step, when the precision control parameter is reduced, the reduced precision control parameter value μ1 can be used to improve the accuracy of the calculation of the spatial distance x in the above formula (1).

[0192] Specifically, when the precision control parameter decreases from μ0 to μ1, the spatial distance x is calculated using the above formulas (1) and (2) based on the precision control parameter value μ1, and the spatial coordinate transformation matrix T2 is determined by the calculated spatial distance x.

[0193] In this step, the effective feature point pairs between the first and second facets are transformed to a unified coordinate system using the coordinate transformation matrix T2, and the spatial distance x can be obtained. As an example, the spatial distance x includes, for example, x1, x3, and x4.

[0194] Therefore, under the control of the precision control parameter, when the calculation accuracy is improved by reducing the precision control parameter, feature point pairs with spatial distance values ​​greater than the spatial distance threshold will be removed. This spatial distance threshold is related to the value of the precision control parameter. Based on this spatial distance threshold related to the value of the precision control parameter, the smaller the value of the precision control parameter, the more feature point pairs with spatial distance values ​​greater than the related spatial distance threshold will be removed.

[0195] Step S03: Within the range of values ​​for the precision control parameter, iteratively decrease the precision control parameter, and then, based on the decreased precision control parameter (e.g., μ), perform the iterative process. n Continue to use the above formula (1) to solve for the spatial distance x in the above formula (2), and obtain the coordinate transformation matrix corresponding to the minimum value of the spatial distance x as the coordinate transformation matrix Tn.

[0196] In this step, the effective feature point pairs between the first and second facets are transformed to a unified coordinate system using the coordinate transformation matrix Tn, and the spatial distance x, including x1, can be obtained, where x1 is the minimum value of the spatial distance x.

[0197] As an example, within the range of the precision control parameter, when the minimum value of the spatial distance is calculated, the spatial coordinate transformation matrix Tn corresponding to the minimum value of the spatial distance x is used as the spatial coordinate transformation matrix.

[0198] In this embodiment of the invention, feature point pairs between the first and second facets to be processed are obtained. When the spatial distance between the feature point pairs meets the spatial distance threshold related to the value of the precision control parameter, the spatial distance between the feature point pairs is considered to meet the proximity condition.

[0199] In other words, the spatial coordinate transformation matrix can also be called the transformation matrix that satisfies the proximity condition.

[0200] In this embodiment, the precision control parameter can be gradually reduced in several ways. For example, the precision control parameter can be gradually reduced based on a fixed precision control parameter step size, or the precision control parameter can be calculated according to the above embodiment based on a pre-set precision control parameter value in descending order.

[0201] In this embodiment of the invention, the coordinate transformation matrix T obtained in the above embodiment is used to transform the effective feature point pairs to the same coordinate system, thereby realizing the registration of the effective feature point pairs between the first surface to be processed and the second surface to be processed.

[0202] Specifically, in this embodiment, a transformation matrix that meets the proximity condition can be used to perform coordinate transformation on the set of feature points Q in the second surface to be processed using the above formula (2). That is, using the rotation matrix T in the above embodiment, a rotation operation based on the rotation component R and a translation operation based on the translation component t are performed on each sampling point of the set of feature points Q, so that the set of feature points Q after coordinate transformation and the set of feature points P in the first surface to be processed are transformed to the same viewpoint or the same coordinate system, thereby realizing the registration of the first point cloud data in the first surface to be processed and the second point cloud data in the second surface to be processed.

[0203] As an example, after multiple point cloud data are registered, the registered point cloud data can be overlapped in the same coordinate system by displacement to obtain a complete three-dimensional point cloud data model.

[0204] As a specific example, the data processing method of this invention can be used, according to... Figure 1a The first point cloud data shown and Figure 1b The second point cloud data shown is used to perform point cloud registration on the first and second areas to be processed, resulting in the first point cloud registration result (not shown in the figure); and the same method is used to... Figure 1c The third facet to be processed shown and Figure 1d Point cloud registration is performed on the fourth surface to be processed, as shown, to obtain the second point cloud registration result (not shown in the figure); using the same method, point cloud data from the first point cloud registration result is registered with the second point cloud registration result to obtain the following result: Figure 1e The point cloud data model is shown.

[0205] Figure 1e In the point cloud data model shown, feature point A1 (B1) represents the coincidence of feature point A1 and feature point B1 after the first point cloud data and the second point cloud data are registered. When feature point A1 and feature point B1 coincide, the spatial distance between feature point A1 and feature point B1 is minimized.

[0206] In some embodiments, when processing multiple point cloud data, two point cloud data can be arbitrarily selected from the multiple point cloud data for processing to obtain new point cloud data; then, two more point cloud data can be arbitrarily selected from the new point cloud data and other unregistered point cloud data in the multiple point cloud data, and the data processing method of the embodiments of the present invention can be executed until all point cloud data are registered to obtain a point cloud data. Figure 1eThe complete 3D point cloud data model is shown.

[0207] In this embodiment, after the purification and optimization of feature point pairs in step S130, the accuracy of feature point pairs between the surfaces to be processed is high, and it has good noise resistance. Extensive practical experience shows that, using the data processing method of this embodiment, when the signal-to-noise ratio for feature point matching between the surfaces to be processed is less than or equal to 0.7 or less than or equal to 0.4, high-precision and high-accuracy feature point matching results can be obtained, thereby obtaining a stable and accurate 3D point cloud data model.

[0208] Figure 2 This is a flowchart illustrating a data processing method according to an embodiment of the present invention. Figure 2 As shown, the data processing method 200 in this embodiment of the invention includes the following steps:

[0209] Step S210: Determine the feature points in the first point cloud data and the feature points in the second point cloud data. The first point cloud data and the second point cloud data are used to represent different parts of the same object.

[0210] In one embodiment, step S210 may specifically include:

[0211] Step S211: Extract feature points that meet the selection criteria from the first point cloud data according to the specified feature selection criteria; and

[0212] Step S212: Extract feature points that meet the selection criteria from the second point cloud data.

[0213] Step S220: Perform feature matching on the first point cloud data and the second point cloud data to determine the feature points that meet the feature matching conditions between the first point cloud data and the second point cloud data, thus forming multiple feature point pairs.

[0214] In one embodiment, step S220 may specifically include:

[0215] Step S221: Obtain the first feature point in the first point cloud data, and use the feature value of the first feature point to find the second feature point in the second point cloud data. The feature value of the second feature point and the feature value of the first feature point satisfy the feature value threshold condition.

[0216] Step S222: Using the feature value of the second feature point, find the third feature point in the first point cloud data. The feature value of the third feature point and the feature value of the second feature point satisfy the feature value threshold condition.

[0217] Step S223: When the first feature point coincides with the third feature point, the first feature point and the second feature point are determined to be feature points that meet the feature matching conditions, thus forming multiple feature point pairs.

[0218] Step S230: For one or more feature point pairs among multiple feature point pairs, determine the transformation matrix that satisfies the proximity condition for the spatial distance between feature points in the feature point pair.

[0219] In one embodiment, step S230 may specifically include:

[0220] Step S231: Based on spatial distance and precision control parameters, construct an evaluation model for one or more feature point pairs among multiple feature point pairs;

[0221] Step S232: Under the control of the precision control parameters, the spatial distance is iteratively processed using the evaluation model to obtain the effective feature point pairs in the feature point pairs and the transformation matrix between the effective feature point pairs. The transformation matrix between the effective feature point pairs is used as the transformation matrix of the spatial distance between the feature points in the feature point pairs that meets the proximity condition.

[0222] Specifically, step S232 may include:

[0223] By reducing the precision control parameters, a new evaluation model is constructed using the evaluation model and the reduced precision control parameters. During the process of solving the spatial distance between feature point pairs using the new evaluation model, the precision control parameters are further reduced and iterated until the spatial distance is minimized. This yields the effective feature point pairs in the feature point pair and the transformation matrix between the effective feature point pairs.

[0224] Through the above step S230, the feature point pairs can be purified and optimized to remove erroneous feature point pairs caused by noise interference. By using the purified and optimized feature point pairs for registration, the accuracy and precision of the feature point matching results can be improved, thereby obtaining a stable and accurate three-dimensional point cloud data model.

[0225] Step S240: Using a transformation matrix, perform coordinate transformation on one or more feature point pairs among multiple feature point pairs to register the first point cloud data with the second point cloud data.

[0226] Through the above steps S210-S240, the first point cloud data and the second point cloud data of the object can be registered. In order to obtain a complete three-dimensional point cloud data model, the data processing method 200 may further include:

[0227] Step S250: The registered first point cloud data and the second point cloud data are used as the first point cloud data. The feature points in the first point cloud data are re-determined until all point cloud data of the object are registered, and a complete three-dimensional point cloud data model of the object is obtained.

[0228] The data processing method of this invention can achieve relatively stable registration results even when the signal-to-noise ratio of feature point matching is low.

[0229] The data processing apparatus according to embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0230] Figure 3 A schematic diagram of the structure of a data processing apparatus provided according to an embodiment of the present invention is shown.

[0231] like Figure 4 As shown, the data processing device 300 includes:

[0232] The feature point acquisition module 310 is used to determine the feature points in the first point cloud data and the feature points in the second point cloud data. The first point cloud data and the second point cloud data are used to represent different parts of the same object.

[0233] The feature matching module 320 is used to perform feature matching on the first point cloud data and the second point cloud data to determine the feature points that meet the feature matching conditions between the first point cloud data and the second point cloud data, thereby forming multiple feature point pairs.

[0234] The feature point pair filtering module 330 is used to determine, for one or more feature point pairs among multiple feature point pairs, a transformation matrix in which the spatial distance between feature points in the feature point pair meets the proximity condition.

[0235] The data registration module 340 is used to perform coordinate transformation on one or more feature point pairs from multiple feature point pairs through a transformation matrix, so as to register the first point cloud data with the second point cloud data.

[0236] In one embodiment, the feature point acquisition module 310 may include:

[0237] The first feature point selection unit is used to extract feature points that meet the selection criteria from the first point cloud data according to the specified feature selection criteria; and the second feature point selection unit is used to extract feature points that meet the selection criteria from the second point cloud data.

[0238] In this embodiment, the specified features include at least geometric features or color features.

[0239] In one embodiment, the feature matching module 320 may include:

[0240] The first feature point search unit is used to obtain the first feature point in the first point cloud data, and use the feature value of the first feature point to search for the second feature point in the second point cloud data. The feature value of the second feature point and the feature value of the first feature point satisfy the feature value threshold condition.

[0241] The second feature point search unit is used to search for a third feature point in the first point cloud data using the feature values ​​of the second feature point, wherein the feature values ​​of the third feature point and the feature values ​​of the second feature point satisfy a feature value threshold condition; and

[0242] The feature point pair matching determination unit is used to determine the first feature point and the second feature point as feature points that meet the feature matching conditions when the first feature point and the third feature point coincide, thereby forming multiple feature point pairs.

[0243] In one embodiment, the feature point pair filtering module 330 may include:

[0244] An evaluation model building unit is used to construct an evaluation model for one or more feature point pairs from multiple feature point pairs based on spatial distance and accuracy control parameters; and

[0245] The transformation matrix calculation unit is used to iteratively process the spatial distance using the evaluation model under the control of precision control parameters, to obtain the effective feature point pairs in the feature point pairs and the transformation matrix between the effective feature point pairs. The transformation matrix between the effective feature point pairs is used as the transformation matrix of the spatial distance between the feature points in the feature point pair that meets the proximity condition.

[0246] In one embodiment, the transformation matrix calculation unit may further include:

[0247] The precision control subunit is used to reduce the precision control parameters and construct a new evaluation model using the evaluation model and the reduced precision control parameters.

[0248] The transformation matrix calculation unit is also used to continue to reduce the precision control parameters during the process of solving the spatial distance of feature point pairs using the new evaluation model, and to iterate until the spatial distance is minimized, so as to obtain the effective feature point pairs in the feature point pairs and the transformation matrix between the effective feature point pairs.

[0249] In one embodiment, the data processing apparatus 300 may further include:

[0250] The iterative registration module is used to take the registered first point cloud data and the second point cloud data as the first point cloud data, redetermine the feature points in the first point cloud data, until all the point cloud data of the object is completed, and obtain the complete three-dimensional point cloud data model of the object.

[0251] The data processing apparatus according to embodiments of the present invention can register point cloud data from different viewpoints or different coordinate systems to obtain registration results with high stability and accuracy.

[0252] Further details of the data processing apparatus according to embodiments of the present invention are combined with the above. Figures 1a-1eThe data processing method described in the embodiments of the present invention is similar and will not be repeated here.

[0253] Combination Figures 1a-1e as well as Figure 3 The data processing method and apparatus described in the embodiments of the present invention can be implemented by a computing device.

[0254] Figure 4 This is a structural diagram illustrating an exemplary hardware architecture of a computing device capable of implementing the data processing method and apparatus according to embodiments of the present invention.

[0255] like Figure 4 As shown, the computing device 400 includes an input device 401, an input interface 402, a processor 403, a memory 404, an output interface 405, and an output device 406. The input interface 402, processor 403, memory 404, and output interface 405 are interconnected via a bus 410. The input device 401 and output device 406 are connected to the bus 410 via the input interface 402 and output interface 405, respectively, and thus connected to other components of the computing device 400. Specifically, the input device 401 receives input information from an external source (e.g., a 3D scanner) and transmits the input information to the processor 403 via the input interface 402. The processor 403 processes the input information based on computer-executable instructions stored in the memory 404 to generate output information, temporarily or permanently stores the output information in the memory 404, and then transmits the output information to the output device 406 via the output interface 405. The output device 406 outputs the output information to the outside of the computing device 400 for user use.

[0256] In one embodiment, Figure 4 The computing device 400 shown can be implemented as a data processing system, including: a memory and a processor; the memory is used to store executable program code; the processor is used to read the executable program code stored in the memory to execute the data processing method of the above embodiments.

[0257] The image device of this invention can register point cloud data from different viewpoints or coordinate systems, and can obtain registration results with high stability and accuracy.

[0258] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, in the form of a computer program product or a computer-readable storage medium. The computer program product or computer-readable storage medium includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid state disks (SSDs)).

[0259] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0260] The above description is merely a specific embodiment of the present invention. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the protection scope of the present invention.

Claims

1. A data processing method, comprising: The object in the real scene is scanned in three dimensions from multiple shooting angles to obtain multiple point data of the object. Based on the multiple point data, multiple point cloud data of the object are constructed under multiple shooting perspectives, and the multiple point cloud data includes point cloud data of at least two coordinate systems; A unified coordinate system processing is performed on the point cloud data of the at least two coordinate systems to obtain a three-dimensional point cloud data model of the object. The three-dimensional point cloud data model is obtained by performing coordinate transformation on the point cloud data of the at least two coordinate systems according to the transformation matrix corresponding to the point cloud data of the at least two coordinate systems. The transformation matrix is ​​the coordinate transformation matrix corresponding to the minimum value of the spatial distance of multiple feature point pairs contained in the point cloud data. The minimum value is determined by repeatedly reducing the value of the precision control parameter within the range of the precision control parameter and calculating the spatial distance using an evaluation model. The multiple feature point pairs are determined based on the geometric structure features and texture features of the feature points in the polygonal patch corresponding to the point cloud data of each coordinate system. The geometric structure features include at least the normal vector and curvature of the sampling points in the polygonal patch, and the texture features include at least the brightness and grayscale of the sampling points.

2. The method according to claim 1, wherein, The step of unifying the coordinate system of the point cloud data from the at least two coordinate systems to obtain the three-dimensional point cloud data model of the object includes: Using a unified coordinate system, coordinate transformation is performed on the point cloud data of at least two coordinate systems to obtain a three-dimensional point cloud data model of the object.

3. The method according to claim 2, wherein, The unified coordinate system corresponds to the transformation matrix, which includes rotation and translation components. The step of performing coordinate transformation on the point cloud data of at least two coordinate systems according to a unified coordinate system to obtain the three-dimensional point cloud data model of the object includes: Based on the rotation and translation components, coordinate transformation is performed on the point cloud data of the at least two coordinate systems to obtain a three-dimensional point cloud data model of the object. The rotation component is used to characterize the rotational relationship between point cloud data in each of the at least two coordinate systems; the translation component is used to characterize the translational relationship between point cloud data in each of the at least two coordinate systems.

4. The method according to any one of claims 1-3, wherein, The point cloud data of the at least two coordinate systems each correspond to a polygonal patch model, the polygonal patch model includes multiple polygonal patches, the polygonal patches include point data, and the unified coordinate system corresponds to the transformation matrix. The step of unifying the coordinate system of the point cloud data from the at least two coordinate systems to obtain the three-dimensional point cloud data model of the object includes: The polygonal patch model corresponding to the point cloud data of each coordinate system is taken as the patch to be processed; The point data of each facet in the multiple facets of each polygon facet model are transformed using the transformation matrix to obtain the three-dimensional point cloud data model of the object.

5. The method according to any one of claims 1-3, wherein, The step of unifying the coordinate system of the point cloud data from the at least two coordinate systems to obtain the three-dimensional point cloud data model of the object includes: The polygonal patch model corresponding to the point cloud data of each coordinate system is taken as the patch to be processed to obtain multiple patches to be processed of the object. The multiple patches to be processed include a first patch to be processed and a second patch to be processed. Determine the feature points in the first surface patch to be processed, and determine the feature points in the second surface patch to be processed; Feature matching is performed on the surface features of the feature points in the first surface to be processed and the surface features of the feature points in the second surface to be processed, respectively, to obtain feature point pairs that meet the feature matching conditions for the first surface to be processed and the second surface to be processed. From the feature point pairs that meet the feature matching conditions of the first and second facets, remove the erroneous feature point pairs between the first and second facets to obtain the feature point matching results of the first and second facets. Based on the feature point matching results of the first and second facets to be processed, a three-dimensional point cloud data model of the object is generated.

6. The method according to claim 5, wherein, The steps of determining feature points in the first surface patch to be processed and determining feature points in the second surface patch to be processed include: The surface features of the object are extracted from the first surface to be processed, and the surface features of the object are extracted from the second surface to be processed. By extracting the surface features of the first surface to be processed, feature points in the first surface to be processed are determined; and by extracting the surface features of the second surface to be processed, feature points in the second surface to be processed are determined.

7. The method according to claim 5, wherein, The feature point pairs include erroneous feature point pairs and valid feature point pairs; The step of removing erroneous feature point pairs between the first and second surfaces from the feature point pairs that meet the feature matching conditions to obtain the feature point matching results for the first and second surfaces includes: An evaluation model is constructed based on spatial distance and precision control parameters; Using the evaluation model, the spatial distance between the matching feature point pairs between the first and second facets that meet the feature matching conditions is calculated. Among the feature point pairs that meet the feature matching conditions of the first and second face patches to be processed, the feature point pairs that correspond to the spatial distance are determined as valid feature point pairs. From the feature point pairs that meet the feature matching conditions of the first and second facets to be processed, remove the erroneous feature point pairs other than the valid feature point pairs to obtain the feature point matching results of the first and second facets to be processed.

8. The method according to claim 5, wherein, The method further includes: By using 3D reconstruction technology, the point cloud data of each coordinate system is reconstructed to obtain a polygonal patch model corresponding to the point cloud data of each coordinate system.

9. The method according to claim 5, wherein, The surface patch to be processed includes point cloud data; The step of extracting surface features of the object from the first surface patch to be processed, and extracting surface features of the object from the second surface patch to be processed, includes: The point cloud data in the first and second facets to be processed are determined as the point set of the surface features of the object. The surface features of the object are analyzed to obtain the surface features of the first surface to be processed and the surface features of the second surface to be processed.

10. The method according to any one of claims 1-3, wherein, The step of performing a three-dimensional scan of an object in a real scene from multiple shooting angles to obtain multiple point data of the object includes: projecting structured light onto the object from multiple shooting angles. By acquiring feedback images from the multiple shooting perspectives, multiple point data of the object are obtained.

11. The method according to any one of claims 1-3, wherein, The multiple point data includes point data for each of the multiple shooting angles; The process of constructing multiple point cloud data of the object from multiple shooting perspectives based on the multiple point data includes: The data set of point data for each of the multiple shooting perspectives is determined as the point cloud data for each shooting perspective; The point cloud data of each shooting angle is defined as point cloud data in a coordinate system.

12. A data processing method, comprising: Using virtual reality or augmented reality technology, a three-dimensional scan of an object in a real scene is performed to obtain multiple point data of the object from each shooting perspective in multiple shooting perspectives. Based on the multiple point data, multiple point cloud data of the object are constructed under multiple shooting perspectives, and the multiple point cloud data includes point cloud data of at least two coordinate systems; A unified coordinate system processing is performed on the point cloud data of the at least two coordinate systems to obtain a three-dimensional point cloud data model of the object for virtual reality or augmented reality technology. The three-dimensional point cloud data model is obtained by performing coordinate transformation on the point cloud data of the at least two coordinate systems according to the transformation matrix corresponding to the point cloud data of the at least two coordinate systems. The transformation matrix is ​​the coordinate transformation matrix corresponding to the minimum value of the spatial distance of multiple feature point pairs contained in the point cloud data. The minimum value is determined by repeatedly reducing the value of the precision control parameter within the range of the precision control parameter and calculating the spatial distance using an evaluation model. The multiple feature point pairs are determined based on the geometric structure features and texture features of the feature points in the polygonal patch corresponding to the point cloud data of each coordinate system. The geometric structure features include at least the normal vector and curvature of the sampling points in the polygonal patch, and the texture features include at least the brightness and grayscale of the sampling points.

13. A data processing device, comprising: Memory and processor The memory is used to store computer programs; The processor is configured to execute a computer program stored in the memory, wherein when the computer program is executed, the processor performs the steps of the data processing method as described in any one of claims 1 to 11, or the steps of the data processing method as described in claim 12.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed on a computer, cause the computer to perform the steps of the data processing method as claimed in any one of claims 1 to 11, or the steps of the data processing method as claimed in claim 12.

Images