Data processing method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a data processing method and device, electronic equipment and storage medium, and relates to the technical field of computer, wherein the method comprises: generating a tree structure according to the first positions of a plurality of sampling points; determining a first leaf node and a second leaf node from a plurality of leaf nodes in the tree structure, the consistency degree of the first normal vector between the sampling points in the first leaf node and the sampling points in the adjacent sampling area is less than the consistency degree of the first normal vector between the sampling points in the second leaf node and the sampling points in the adjacent sampling area; determining the vector field by using different precision filters for the sampling points in the first leaf node and the sampling points in the second leaf node respectively; and reconstructing the target surface according to the vector field of the first leaf node and the vector field of the second leaf node to obtain and display the reconstructed target surface image. Thus, the reconstruction efficiency can be improved, and the method can be applied to mobile terminal side with high speed requirement, thereby improving the applicability of the method.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a data processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] Surface reconstruction refers to the technology of accurately restoring the three-dimensional surface shape of an object using point cloud data. Surface reconstruction can be applied to AR (Augmented Reality) displays on mobile terminals, such as displaying the appearance of new products through AR; or to high-definition map displays in the field of autonomous driving, that is, restoring high-definition three-dimensional road maps in the digital twin world and displaying them visually; or to metaverse-VR (Virtual Reality) displays, that is, reconstructing real-world objects in the digital world through 3D reconstruction technology and then visually displaying them on a screen; or it can be applied to other fields as well.

[0003] In related technologies, the Poisson surface reconstruction algorithm can be used to reconstruct the three-dimensional surface of an object. However, the reconstruction speed of the Poisson surface reconstruction algorithm is relatively slow, making it difficult to run on mobile terminals. Summary of the Invention

[0004] This disclosure aims to at least partially address one of the technical problems in the related art.

[0005] Therefore, the present disclosure proposes the following technical solution:

[0006] The first aspect of this disclosure provides a data processing method, including:

[0007] Based on the point cloud map obtained by sampling the target surface in three-dimensional space, the first position of multiple sampling points in the point cloud map and the first normal vector of the multiple sampling points are obtained.

[0008] A tree structure is generated based on the first position of the plurality of sampling points, wherein the tree structure includes a plurality of leaf nodes, and the plurality of leaf nodes correspond one-to-one with the plurality of sampling regions divided in the three-dimensional space, wherein each leaf node includes at least one sampling point whose first position is located within the corresponding sampling region;

[0009] A first leaf node and a second leaf node are determined from the plurality of leaf nodes, wherein the consistency of the first normal vector between the sampling point in the first leaf node and the sampling point in the adjacent sampling region is less than the consistency of the first normal vector between the sampling point in the second leaf node and the sampling point in the adjacent sampling region.

[0010] The vector field is determined by applying filters of different precision to the sampling points in the first leaf node and the sampling points in the second leaf node, respectively.

[0011] Based on the vector fields of the first leaf node and the second leaf node, the target surface is reconstructed to obtain a reconstructed target surface image.

[0012] Displays a reconstructed image of the target surface.

[0013] A second aspect of this disclosure provides a data processing apparatus, comprising:

[0014] The acquisition module is used to acquire the first position of multiple sampling points in the point cloud map and the first normal vector of the multiple sampling points based on the point cloud map obtained by sampling the target surface in three-dimensional space.

[0015] The generation module is used to generate a tree structure based on the first position of the plurality of sampling points, wherein the tree structure includes a plurality of leaf nodes, the plurality of leaf nodes correspond one-to-one with the plurality of sampling regions divided in the three-dimensional space, and each leaf node includes at least one sampling point whose first position is located in the corresponding sampling region.

[0016] The first determining module is used to determine a first leaf node and a second leaf node from the plurality of leaf nodes, wherein the consistency of the first normal vector between the sampling point in the first leaf node and the sampling point in the adjacent sampling region is less than the consistency of the first normal vector between the sampling point in the second leaf node and the sampling point in the adjacent sampling region.

[0017] The second determining module is used to determine the vector field by applying filters of different precision to the sampling points in the first leaf node and the sampling points in the second leaf node respectively;

[0018] The reconstruction module is used to reconstruct the target surface based on the vector field of the first leaf node and the vector field of the second leaf node, so as to obtain a reconstructed target surface image.

[0019] The display module is used to display the reconstructed target surface image.

[0020] A third aspect of this disclosure provides an electronic device comprising:

[0021] processor;

[0022] A memory for storing executable instructions of a processor; wherein the processor is configured to invoke and execute the executable instructions stored in the memory to implement a data processing method as proposed in the first aspect of the present disclosure.

[0023] A fourth aspect of this disclosure provides a non-transitory computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the data processing method as described in the first aspect of this disclosure.

[0024] A fifth aspect of this disclosure provides a computer program product that, when instructions in the computer program product are executed by a processor, performs a data processing method as described in a first aspect of this disclosure.

[0025] The technical solution disclosed herein involves obtaining the first positions and first normal vectors of multiple sampling points in a point cloud image obtained by sampling a target surface in three-dimensional space; generating a tree structure based on the first positions of the multiple sampling points, wherein the tree structure includes multiple leaf nodes, each leaf node corresponding to multiple sampling regions divided in three-dimensional space, and each leaf node includes at least one sampling point whose first position is located within the corresponding sampling region; determining a first leaf node and a second leaf node from the multiple leaf nodes, wherein the consistency of the first normal vector between the sampling points in the first leaf node and the sampling points in the adjacent sampling regions is less than the consistency of the first normal vector between the sampling points in the second leaf node and the sampling points in the adjacent sampling regions; determining the vector fields of the sampling points in the first leaf node and the sampling points in the second leaf node using filters of different precision; reconstructing the target surface based on the vector fields of the first leaf node and the second leaf node to obtain a reconstructed target surface image; and displaying the reconstructed target surface image. Therefore, based on the consistency of the normal vectors between the sampling points in each leaf node and the sampling points in the adjacent sampling regions, filters of different precision can be selected to determine the vector field of each leaf node. Thus, the target surface can be reconstructed based on the vector field of each leaf node. This method can accelerate the calculation speed of the vector field by using relatively low precision filters while meeting the reconstruction accuracy requirements, thereby improving the reconstruction efficiency. It can be applied to mobile terminals with high requirements for reconstruction speed, thus improving the applicability of the method.

[0026] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description

[0027] The above and / or additional aspects and advantages of this disclosure will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, in which:

[0028] Figure 1 This is a schematic diagram of PR in a two-dimensional case;

[0029] Figure 2 This is a schematic flowchart of a data processing method provided in an embodiment of the present disclosure;

[0030] Figure 3 This is a schematic flowchart illustrating a data processing method provided in another embodiment of the present disclosure;

[0031] Figure 4 This is a schematic flowchart illustrating a data processing method provided in another embodiment of the present disclosure;

[0032] Figure 5 A schematic diagram of the basis functions for a quadratic B-spline centered at 1.5;

[0033] Figure 6 This is a schematic diagram of normal cancellation in PR;

[0034] Figure 7 This is a schematic flowchart illustrating a data processing method provided in another embodiment of the present disclosure;

[0035] Figure 8 This is a schematic flowchart illustrating a data processing method provided in another embodiment of the present disclosure;

[0036] Figure 9 Here is a schematic diagram of the normals at each point;

[0037] Figure 10 This is a schematic flowchart of the error-adaptive reconstruction method proposed in the embodiments of this disclosure;

[0038] Figure 11 A schematic diagram of a quadtree with a triangle shape;

[0039] Figure 12 A schematic diagram illustrating the error test results of various data processing methods provided in the embodiments of this disclosure;

[0040] Figure 13 This is a schematic diagram of the structure of a data processing apparatus provided in an embodiment of the present disclosure;

[0041] Figure 14 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. Detailed Implementation

[0042] Embodiments of this disclosure are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this disclosure, and should not be construed as limiting this disclosure.

[0043] To facilitate understanding, the terminology used in this disclosure will be introduced first.

[0044] 1. Point cloud

[0045] Point clouds are obtained by scanning real-world objects using depth cameras or 3D scanners. Each sampled point in a point cloud contains its corresponding position, normal, color, and other information on the object's surface. The number of sampled points in a point cloud is enormous, and it can be used to create 3D meshes or other models for applications in CAD, quality inspection, and medical imaging.

[0046] As the price of 3D scanners gradually decreases and their size shrinks (and their portability increases), point clouds are appearing in many consumer devices. For example, smartphones have built-in structured light 3D facial recognition cameras that can collect 3D facial information to unlock the phone and make secure payments. Another example is autonomous vehicles equipped with LiDAR or millimeter-wave radar, which can output continuous point clouds to represent the vehicle's surroundings. Furthermore, point clouds can be applied to virtual reality and augmented reality to simulate realistic environments.

[0047] 2. Surface Reconstruction

[0048] Surface reconstruction is a classic problem that has been studied for many years, which can fit the curved surface of an object from a point cloud. Since point cloud data is often obtained by scanning real-world objects with a 3D scanner, it contains a lot of noise, outliers or missing data, and the shape of the object surface to be fitted is also arbitrary. Therefore, an ideal reconstruction method should be robust.

[0049] Surface reconstruction can generate object surfaces or curved surfaces based on scanned sampling points. These sampling points are often characterized by noise, non-uniform distribution, and layering along the normal direction, requiring algorithms to reduce noise, fill voids, and merge layers. Surface reconstruction algorithms can be broadly classified into two categories: combined function methods and implicit function methods. Combined function methods often use a subset of sampling points for triangulation, discretizing the three-dimensional space using tetrahedrons, and then dividing the tetrahedron into surface interior and exterior using element analysis, eigenvector calculation, or graphical cut methods. In the presence of noise, implicit function methods typically use zero isosurfaces to fit the sampling points, such as radial basis functions and / or piecewise polynomial functions.

[0050] 3. Poisson surface reconstruction (PR)

[0051] PR is a widely used surface reconstruction method that generates watertight surfaces from point clouds containing location and normal information. PR reconstructs a smooth indicator function of the scanned object by matching the gradient with the input normal and outputting an isosurface (level set) extracted from the indicator function. Therefore, surface reconstruction can be viewed as solving the Poisson equation, which can be solved using a finite element solver.

[0052] One known problem with Poisson Reconstruction (PR) comes from smoothing filters. To address this, positional constraints can be added to employ Screened Poisson Surface Reconstruction (SPR) to make the reconstructed surface closer to the point cloud, thus avoiding the problem of PR being too smooth.

[0053] PR generates watertight surfaces from directed points (points with position and normals (x, y, z, nx, ny, nz)). The basic idea is to recover the indicator function from the gradient, and the surface can be regarded as the indicator function χ of the scanned object M. M The level set, except on the surface Outside the boundary of M, the gradient It is zero (because χ) M It is discontinuous at the surface, so the gradient does not exist, but if for χ M Using Gaussian filtering for blurring allows us to calculate its gradient, surface normal, and... On Since they are in the same direction, the vector field can be recovered from the input normal.

[0054]

[0055] Applying a gradient operator to both sides of equation (1), we obtain the Poisson equation as follows:

[0056]

[0057] in, It refers to the gradient. Δ refers to divergence.

[0058] Formula (2) can be solved using the finite element method to obtain χ. M , because χ M In actual calculations, it is blurred; its... The value at that location is 0.5, so the final extraction of χ... M The 0.5 isosurface is sufficient.

[0059] As an example, a PR diagram in a two-dimensional case can be shown as follows: Figure 1 As shown, according to Figure 1 It can be seen that the vector field fitted based on the input normal... It is an approximation of the gradient field.

[0060] SPR addresses the oversmoothing problem of PR by introducing positional constraints. The formula for masked Poisson reconstruction is as follows:

[0061]

[0062] Here, I is an appropriately defined operator used to "mask" the Δ operator, and α is the masking factor.

[0063] Since this disclosure only involves the improvement of the Poisson equation when reconstructing object surfaces in different application scenarios, and does not involve the finite element solution of the Poisson equation and the extraction of isosurfaces, the following will only describe in detail how to obtain the Poisson equation:

[0064] 1. The PR algorithm first compresses the input points (i.e., sampling points) and their normals into an octree O, giving it a maximum depth D. This depth is related to the presence of data points; that is, where there are input points, the node is subdivided to depth D, and where there are no input points, a larger node (i.e., the minimum depth D_min) is used. Then, the input points within the same node are averaged into a single data point, meaning one node corresponds to one data point. A node may contain at least one input point (i.e., sampling point), and the data point is the average of all input points within that node. The octree O is an efficient method for storing and processing data; all subsequent operations are performed on the octree O.

[0065] 2. Because the input normal (i.e., the compressed normal in octree O) is a sample of the normal on the surface, the normal can be fitted to the following vector field.

[0066]

[0067] Where q refers to a node in octree O. It refers to the vector field at node q. It is a Gaussian smoothing filter centered at q. It is the normal at point p.

[0068] Since the input normals are not uniformly distributed and are mainly distributed near the object surface, the normals are "Gaussian smoothed" using formula (4) to obtain the vector field. therefore Also known as a smooth normal field, it is defined on all octree O nodes, including both inside and outside the object, but specifically the "normal" inside the object. The absolute value is small because of the Gaussian smoothing filter. It tends to 0 when |pq| is relatively large.

[0069] 3. Substitute formula (4) into formula (2) to obtain the Poisson equation.

[0070] In actual calculations, the integral sign of formula (4) can be discretized into the summation sign Σ. Replace each data point, and the equation in formula (2) is also discretized into a finite element form for solution.

[0071] Among state-of-the-art surface reconstruction algorithms, SPR (Surface Reconstruction Process) performs well in extracting 3D surfaces from point clouds. To better handle non-uniform point clouds, both SPR and PR use a fixed-size filter kernel to estimate the point cloud density, thereby constructing a vector field that masks the Poisson equation. Then solve the shielded Poisson equation.

[0072] However, the overall reconstruction speed of the above reconstruction algorithms is too slow, making it difficult to run on mobile terminals, while the reconstruction effect of reconstruction algorithms that can run on mobile terminals is not good; moreover, a major source of error in the above reconstruction algorithms is the problem of "normal cancellation", that is, the normals in the edges and sharp areas cancel each other out, resulting in a large reconstruction error.

[0073] To address the aforementioned problems, this disclosure proposes a data processing method, apparatus, electronic device, and storage medium.

[0074] The data processing method, apparatus, electronic device, and storage medium of this disclosure are described below with reference to the accompanying drawings.

[0075] Figure 2 This is a schematic flowchart of a data processing method provided in an embodiment of the present disclosure.

[0076] The data processing method of this disclosure can be applied to any electronic device so that the electronic device can perform surface reconstruction functions.

[0077] In this context, an electronic device is a user-side entity used to receive or transmit signals, such as a mobile phone. Electronic devices can also be called terminals, user equipment (UE), mobile stations (MS), mobile terminals (MT), etc. Electronic devices can be communication-enabled vehicles, smart cars, mobile phones, wearable devices, tablets, televisions, TV sticks, computers with wireless transceiver capabilities, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminal devices in industrial control, wireless terminal devices in self-driving, wireless terminal devices in remote medical surgery, wireless terminal devices in smart grids, wireless terminal devices in transportation safety, wireless terminal devices in smart cities, wireless terminal devices in smart homes, and so on. The embodiments disclosed herein do not limit the specific technologies or device forms used in the electronic devices.

[0078] like Figure 2 As shown, the data processing method may include the following steps:

[0079] Step 101: Based on the point cloud map obtained by sampling the target surface in three-dimensional space, obtain the first position of multiple sampling points in the point cloud map and the first normal vector of multiple sampling points.

[0080] In this embodiment of the disclosure, the target surface in three-dimensional space can be sampled to obtain a point cloud map. For example, a three-dimensional sensor can be used to scan the surface of the target object (such as a face, object, etc.) (referred to as the target surface in this disclosure) to obtain a point cloud map, thereby obtaining the first position of multiple sampling points in the point cloud map and the first normal vector of multiple sampling points.

[0081] Step 102: Generate a tree structure based on the first position of multiple sampling points. The tree structure includes multiple leaf nodes, which correspond one-to-one with multiple sampling regions divided in three-dimensional space. Each leaf node includes at least one sampling point whose first position is located within the corresponding sampling region.

[0082] In this embodiment of the disclosure, a tree structure can be generated based on the first positions of multiple acquired sampling points. The tree structure includes multiple leaf nodes, each corresponding to a sampling region, and each leaf node includes at least one sampling point whose first position is located within the sampling region corresponding to that leaf node. The sampling region can be obtained by dividing the aforementioned three-dimensional space. For example, the aforementioned three-dimensional space can be divided to obtain a tree structure conforming to a first predetermined depth (e.g., D), where the leaf nodes can be the leaf nodes with the maximum depth D in the tree structure.

[0083] It should be noted that when the value of D is small, the reconstructed surface image is relatively blurry, while when the value of D is large, the reconstruction speed is slow. In order to balance reconstruction accuracy and reconstruction speed, the value of D can be between 6 and 12.

[0084] For example, taking an octree with a first set depth (e.g., D) as an example, the leaf nodes can be leaf nodes at depth D in the octree.

[0085] As an example, an octree O can be constructed based on the acquired multiple sampling points. The maximum depth of octree O is D, meaning the octree has D levels of nodes, with the last level's nodes being leaf nodes. Multiple sampling points can be compressed into octree O. An octree is a data structure for partitioning 3D space, similar to a binary tree in 1D and a quadtree in 2D.

[0086] Step 103: Determine the first leaf node and the second leaf node from multiple leaf nodes, wherein the consistency of the first normal vector between the sampling point in the first leaf node and the sampling point in the adjacent sampling region is less than the consistency of the first normal vector between the sampling point in the second leaf node and the sampling point in the adjacent sampling region.

[0087] In this embodiment of the disclosure, a first leaf node and a second leaf node can be determined from each leaf node based on the first normal vector of the sampling point in each leaf node, wherein the number of first leaf nodes can be at least one, and the number of second leaf nodes can also be at least one.

[0088] Among them, the consistency between the first normal vector of the sampling point in the first leaf node and the first normal vector of the sampling point in the adjacent sampling region is relatively low, while the consistency between the first normal vector of the sampling point in the second leaf node and the first normal vector of the sampling point in the adjacent sampling region is relatively high. That is, the consistency between the first normal vector of the sampling point in the first leaf node and the sampling point in the adjacent sampling region is less than the consistency between the first normal vector of the sampling point in the second leaf node and the sampling point in the adjacent sampling region.

[0089] The consistency between the normal vectors of two sampling points indicates the similarity of their directions. The consistency is highest when the directions of the normal vectors of two sampling points are exactly the same, and lowest when the directions of the normal vectors of two sampling points are completely opposite.

[0090] Step 104: Use filters of different precision to determine the vector field for the sampling points in the first leaf node and the sampling points in the second leaf node.

[0091] In this embodiment of the disclosure, for each first leaf node, the vector field corresponding to the first leaf node can be determined by using a filter of first precision based on the sampling points in the first leaf node. For each second leaf node, the vector field corresponding to the second leaf node can be determined by using a filter of second precision based on the sampling points in the second leaf node.

[0092] In this context, the vector field of a leaf node can be understood as the weighted sum of the normal vectors of the sampled points within that leaf node and the normal vectors of the sampled points in all nodes within its neighborhood. When calculating the vector field of a leaf node, the neighborhood range traversed by the filter can be adjusted by changing the filter's precision. The vector field of the leaf node is determined by the weighted sum of the normal vectors of the sampled points in the neighborhood range and the normal vectors of the sampled points within that leaf node.

[0093] Step 105: Based on the vector fields of the first leaf node and the second leaf node, the target surface is reconstructed to obtain the reconstructed target surface image.

[0094] In this embodiment of the disclosure, the target surface of the target object can be reconstructed based on the vector fields of each first leaf node and each second leaf node to obtain a reconstructed target surface image.

[0095] As an example, the vector fields of each first leaf node and each second leaf node can be substituted into formula (2) and formula (2) can be solved to obtain the reconstructed surface (i.e., the isosurface) χ. M .

[0096] Step 106: Display the reconstructed target surface image.

[0097] In this embodiment of the disclosure, the reconstructed target surface image can be visualized.

[0098] As an application scenario, this method can be applied to mobile terminals. Users can use 3D sensors in mobile terminals (such as LiDAR scanners in rear camera arrays, Time of Flight (TOF) sensors, etc.) to scan objects in the real world, obtain point cloud maps, and reconstruct the 3D surface images of objects in the virtual world based on the point cloud maps.

[0099] It should be noted that surface reconstruction generally involves two algorithms. The first algorithm is a real-time, low-quality reconstruction, which scans the object and reconstructs the object's surface simultaneously, allowing users to obtain the final reconstruction result in a timely manner. The second algorithm is a non-real-time, high-precision reconstruction of the object's surface after scanning the object at least once. However, due to missed scans and low scan quality, the reconstructed surface is not fully restored in certain corners and orientations, requiring rescanning of areas with low restoration accuracy, which is time-consuming.

[0100] The method provided in this disclosure enables rapid and accurate 3D surface reconstruction. If a user observes that the reconstruction effect of a certain area is not good, the user can rescan that area in real time and online, achieving an interactive, real-time feedback, and WYSIWYG reconstruction process.

[0101] As another application scenario, the 3D sensor in the mobile terminal can be used to scan the environment such as the desktop, and the 3D model of the new product can be "placed" on the desktop in the camera to realize AR shopping, so that users can intuitively feel the 360-degree appearance and size of the new product.

[0102] The method disclosed herein can quickly reconstruct the surface of objects, especially irregular objects on a desktop, such as a mouse or a water glass, preventing overlap and clipping between new products and environmental objects in AR, making AR more "realistic".

[0103] As another application scenario, a front-facing 3D sensor (such as a front-facing depth sensor) in a mobile terminal can be used to scan a face, obtain a point cloud map, and quickly reconstruct a 3D face based on the point cloud map, so as to unlock and make payments based on the reconstructed 3D face image.

[0104] The data processing method of this disclosure includes: obtaining the first positions and first normal vectors of multiple sampling points in a point cloud map obtained by sampling a target surface in three-dimensional space; generating a tree structure based on the first positions of the multiple sampling points, wherein the tree structure includes multiple leaf nodes, each leaf node corresponding to multiple sampling regions divided in three-dimensional space, and each leaf node includes at least one sampling point whose first position is located within the corresponding sampling region; determining a first leaf node and a second leaf node from the multiple leaf nodes, wherein the consistency of the first normal vector between the sampling points in the first leaf node and the sampling points in the adjacent sampling regions is less than the consistency of the first normal vector between the sampling points in the second leaf node and the sampling points in the adjacent sampling regions; determining the vector fields of the sampling points in the first leaf node and the sampling points in the second leaf node using filters of different precision; reconstructing the target surface based on the vector fields of the first leaf node and the second leaf node to obtain a reconstructed target surface image; and displaying the reconstructed target surface image. Therefore, based on the consistency of the normal vectors between the sampling points in each leaf node and the sampling points in the adjacent sampling regions, filters of different precision can be selected to determine the vector field of each leaf node. Thus, the target surface can be reconstructed based on the vector field of each leaf node. This method can accelerate the calculation speed of the vector field by using relatively low precision filters while meeting the reconstruction accuracy requirements, thereby improving the reconstruction efficiency. It can be applied to mobile terminals with high requirements for reconstruction speed, thus improving the applicability of the method.

[0105] To clearly illustrate how a tree structure is generated based on the first positions of multiple sampling points in any embodiment of this disclosure, this disclosure also proposes a data processing method.

[0106] Figure 3 This is a schematic flowchart of a data processing method provided in another embodiment of the present disclosure.

[0107] like Figure 3 As shown, the data processing method may include the following steps:

[0108] Step 201: Based on the point cloud map obtained by sampling the target surface in three-dimensional space, obtain the first position of multiple sampling points in the point cloud map and the first normal vector of multiple sampling points.

[0109] The explanation of step 201 can be found in the relevant description in any embodiment of this disclosure, and will not be repeated here.

[0110] Step 202: Divide the sampling area based on three-dimensional space to obtain a tree structure that conforms to the first set depth.

[0111] In this embodiment of the disclosure, the first set depth is a pre-set depth, for example, the first set depth is marked as D.

[0112] In this embodiment of the disclosure, the above-mentioned three-dimensional space can be divided into sampling regions to obtain a tree structure that conforms to a first set depth. The tree structure includes each leaf node and at least one non-leaf node that has a parent-child relationship with at least one leaf node. Among the at least two nodes with a parent-child relationship, the sampling region corresponding to the parent node in the above-mentioned three-dimensional space is the sum of the sampling regions corresponding to the child nodes. The sampling region of each leaf node is the sampling region of the corresponding leaf node.

[0113] For example, let's take an octree as an example. An octree is a tree-like structure that describes three-dimensional space. Each node in an octree represents a volume element of a cube, and each node has eight child nodes. The sum of the volume elements represented by these eight child nodes equals the volume of the parent node. We can first divide the three-dimensional space into eight equal parts to obtain the eight nodes of the first level. Then, we can divide each node of the first level into eight equal parts to obtain the nodes of the second level. After that, we can divide each node of the second level into eight equal parts to obtain the nodes of the third level, and so on, until we obtain the leaf nodes of the last level. The depth from the leaf node of this level to the root node is the first predetermined depth.

[0114] It should be noted that each sampling region is obtained by dividing three-dimensional space. Each sampling region can have a corresponding position range in three-dimensional space. For each sampling point, the sampling region to which the sampling point belongs can be determined based on the first position of the sampling point. For example, when the first position of the sampling point is located within the position range corresponding to sampling region A, the sampling region to which the sampling point belongs is sampling region A.

[0115] Step 203: Determine the sampling points included in each leaf node based on the sampling points contained in the sampling area corresponding to each leaf node.

[0116] In this embodiment of the disclosure, for any leaf node in the tree structure, the sampling points contained in the sampling area corresponding to the leaf node can be used as the sampling points included in the leaf node.

[0117] Step 204: Determine the sampling points included in each non-leaf node based on the sampling points contained in the sampling area corresponding to each non-leaf node.

[0118] In this embodiment of the disclosure, for any non-leaf node in the tree structure, the sampling points contained in the sampling region corresponding to the non-leaf node can be used as the sampling points included in the non-leaf node.

[0119] Step 205: Determine the first leaf node and the second leaf node from multiple leaf nodes, wherein the consistency of the first normal vector between the sampling point in the first leaf node and the sampling point in the adjacent sampling region is less than the consistency of the first normal vector between the sampling point in the second leaf node and the sampling point in the adjacent sampling region.

[0120] Step 206: Use filters of different precision to determine the vector field for the sampling points in the first leaf node and the sampling points in the second leaf node.

[0121] Step 207: Based on the vector fields of the first leaf node and the second leaf node, the target surface is reconstructed to obtain the reconstructed target surface image.

[0122] Step 208: Display the reconstructed target surface image.

[0123] The explanation of steps 205 to 208 can be found in the relevant description in any embodiment of this disclosure, and will not be repeated here.

[0124] The data processing method of this disclosure obtains a tree structure conforming to a first predetermined depth by dividing the sampling region based on three-dimensional space; determines the sampling points included in each leaf node based on the sampling points contained in the sampling region corresponding to each leaf node; and determines the sampling points included in each non-leaf node based on the sampling points contained in the sampling region corresponding to each non-leaf node. Thus, a tree structure can be effectively constructed based on the first positions of multiple sampling points in the acquired point cloud image.

[0125] To clearly illustrate how the first leaf node and the second leaf node are determined from multiple leaf nodes in any embodiment of this disclosure, this disclosure also proposes a data processing method.

[0126] Figure 4 This is a schematic flowchart of a data processing method provided in another embodiment of the present disclosure.

[0127] like Figure 4 As shown, the data processing method may include the following steps:

[0128] Step 301: Based on the point cloud map obtained by sampling the target surface in three-dimensional space, obtain the first position of multiple sampling points in the point cloud map and the first normal vector of multiple sampling points.

[0129] Step 302: Generate a tree structure based on the first position of multiple sampling points. The tree structure includes multiple leaf nodes, which correspond one-to-one with multiple sampling regions divided in three-dimensional space. Each leaf node includes at least one sampling point whose first position is located within the corresponding sampling region.

[0130] The explanation of steps 301 to 302 can be found in the relevant description in any embodiment of this disclosure, and will not be repeated here.

[0131] Step 303: Determine multiple target nodes with a depth of a second set depth from the non-leaf nodes of the tree structure.

[0132] In this embodiment of the disclosure, the second set depth is a pre-set depth, and the second set depth is less than the first set depth. For example, the second set depth can be Dm, where m can be a positive integer, such as 1, 2, 3, ...

[0133] In this embodiment of the disclosure, multiple target nodes with a depth of a second predetermined depth can be determined from the non-leaf nodes of the tree structure. That is, multiple target nodes at a level of the second predetermined depth can be determined from the non-leaf nodes of the tree structure. Taking a second predetermined depth of 3 as an example, a node located at the third level in the tree structure can be determined as a target node.

[0134] Step 304: Determine the second position corresponding to each target node based on the first position of at least one sampling point in the sampling area corresponding to each target node.

[0135] In this embodiment of the disclosure, for any one of a plurality of target nodes, the second position corresponding to the target node can be determined based on the first position of at least one sampling point in the sampling area corresponding to the target node.

[0136] As an example, the average of the first positions of at least one sampling point in the sampling area corresponding to the target node can be calculated to obtain the second position corresponding to the target node.

[0137] For example, suppose the sampling region corresponding to target node A contains two sampling points, namely sampling point 1 and sampling point 2. Suppose the first position of sampling point 1 is (x1, y1, z1) and the first position of sampling point 2 is (x2, y2, z2). Then the second position of target node A is ((x1+x2) / 2, (y1+y2) / 2, (z1+z2) / 2).

[0138] As another example, the first position of at least one sampling point in the sampling area corresponding to the target node can be weighted and summed to obtain the second position corresponding to the target node.

[0139] As another example, the maximum, minimum, or intermediate position of the first position of at least one sampling point in the sampling area corresponding to the target node can be used as the second position corresponding to the target node. This disclosure does not limit this.

[0140] Step 305: Determine the second normal vector corresponding to each target node based on the first normal vector of at least one sampling point in the sampling region corresponding to each target node.

[0141] In this embodiment of the disclosure, for any one of a plurality of target nodes, the second normal vector corresponding to the target node can be determined based on the first normal vector of at least one sampling point in the sampling region corresponding to the target node.

[0142] As an example, the mean of the first normal vector of at least one sampling point in the sampling region corresponding to the target node can be calculated to obtain the second normal vector corresponding to the target node.

[0143] As another example, the first normal vectors of at least one sampling point in the sampling region corresponding to the target node can be weighted and summed to obtain the second normal vector corresponding to the target node.

[0144] As another example, any one of the first normal vectors of at least one sampling point in the sampling region corresponding to the target node can be used as the second normal vector corresponding to the target node.

[0145] Step 306: Determine the vector field of each target node based on its second position and second normal vector.

[0146] In this embodiment of the disclosure, for any one of a plurality of target nodes, the vector field of the target node can be determined based on the second position and the second normal vector corresponding to the target node.

[0147] As an example, the vector field of each leaf node in a tree structure can be determined according to the following formula:

[0148]

[0149] Where S is the set of leaf nodes, and points s in S are leaf nodes, each point s is associated with a sampling region P. s Correspondingly, sp is the position of point s, sN is the normal (i.e., the normal vector of point s), |P s |is P s The area. Because |P s |Unknown, the density estimator in PR can be used to estimate the local sampling density:

[0150]

[0151] Wherein, the sampling density W and |P s | is inversely proportional, therefore there is an approximate relationship. Where c is a constant independent of s, PR assumes c = 1 (because c can be extracted and written as cV, and in formula (2) it is divided by c, and then χ M / c is considered a new indicator function, whose value inside the object is 1 / c, instead of the previous 1).

[0152] The above It is a Gaussian filter with center q. Since the support set of the Gaussian filter is the entire space and the calculation range is large, the basis functions of the quadratic B-spline can be used instead in actual calculations. This is because the range of the basis functions of the quadratic B-spline is limited and only involves a few neighborhoods.

[0153] Accordingly, when calculating the vector field of the target node in the tree structure, s in formula (5) can be replaced with s', S can be replaced with S', and P can be replaced with S'. s Change to P s’ Where S' refers to the target node set, and point s' in S' is a target node, and the sampling region P of each point s' is... s’ , is the sum of the sampling regions corresponding to the child nodes of point s'.

[0154] As an example, the basis functions of a quadratic B-spline with center 1.5 are as follows: Figure 5 As shown, this basis function approximates the Gaussian function F. 1.5 (p).

[0155] In summary, vector fields It is the weighted sum of the normals of several nodes in the neighborhood of point q:

[0156]

[0157] Here, weight is the weight.

[0158] As an example, let point s be... Figure 6 The triangle shown on the left is used as an example; each point s has a normal (i.e., a normal vector). Figure 6 The middle section shows the divergence of the vector field V at each point s. The divergence of the vector field V is smaller at the three vertices of the triangle because the directions of the normals near the vertices are opposite, canceling each other out in the summation corresponding to formula (7). The gradient (i.e., V) tends to 0, producing an error. Therefore, the calculation of the vector field in formula (2) is inaccurate, and the reconstructed surface (level set) Precision is also lost at the vertices, such as... Figure 6 As shown on the right.

[0159] Step 307: Determine the reference error of each target node based on the magnitude of the vector field of each target node; wherein the reference error is used to indicate the degree of inconsistency between the second normal vector of the corresponding target node and the second normal vector of the adjacent target node.

[0160] In this embodiment of the disclosure, for any one of the target nodes, the reference error of the target node can be determined based on the magnitude of the vector field of the target node. The reference error can also be called the estimated error (EE), which is used to indicate the degree of inconsistency between the second normal vector of the target node and the second normal vector of the corresponding adjacent target node. The reference error is positively correlated with the degree of inconsistency, that is, the larger the reference error, the higher the degree of inconsistency, and vice versa.

[0161] As one possible implementation, the magnitude of the vector field of the target node can be normalized, and the reference error of the target node can be determined based on the magnitude of the normalized vector field.

[0162] As an example, in order to reduce Figure 6 The errors present in the target node can be addressed by introducing a reference error. The reference error EE of the target node can be determined using the following formula:

[0163]

[0164] In formula (8), the numerator on the right side is The more severely the normal vector is canceled out, The smaller the value of , the larger the reference error EE. The denominator on the right side of the equation is used for normalization to ensure that EE is between 0 and 1. From formula (5), it can be seen that when there is no normal cancellation at all, i.e. Since the directions are consistent, it can be extracted from the integral. Also, since the magnitude of the normal (i.e., the normal vector) is 1, the denominator equals the numerator. When the normals completely cancel each other out, the numerator on the right side of the equation is 0, and EE reaches its maximum value of 1. Therefore, EE can accurately predict the error caused by the cancellation of normals.

[0165] Step 308: Determine the first target node and the second target node from multiple target nodes based on the reference error.

[0166] In this embodiment of the disclosure, a first target node and a second target node can be determined from each target node based on the reference error corresponding to each target node, wherein the reference error of the first target node is greater than the reference error of the second target node.

[0167] As one possible implementation, the target nodes can be sorted according to the reference error from largest to smallest, and the target nodes with the highest sorting ratio can be designated as the first target nodes, while the remaining target nodes other than the first target nodes can be designated as the second target nodes.

[0168] The set ratio is a pre-set ratio, such as 5%, 10%, 15%, etc.

[0169] As another possible implementation, target nodes with reference errors greater than a set threshold can be designated as first target nodes, and target nodes with reference errors less than or equal to the set threshold can be designated as second target nodes.

[0170] The threshold setting is a pre-defined threshold, such as 45%, 50%, 55%, etc.

[0171] Step 309: Select the leaf nodes of the first target node that have a parent-child relationship as the first leaf node, and select the leaf nodes of the second target node that have a parent-child relationship as the second leaf node.

[0172] Among them, the consistency of the first normal vector between the sampling point in the first leaf node and the sampling point in the adjacent sampling region is less than the consistency of the first normal vector between the sampling point in the second leaf node and the sampling point in the adjacent sampling region.

[0173] Step 310: Use filters of different precision to determine the vector field for the sampling points in the first leaf node and the sampling points in the second leaf node.

[0174] Step 311: Based on the vector fields of the first leaf node and the second leaf node, the target surface is reconstructed to obtain the reconstructed target surface image.

[0175] Step 312: Display the reconstructed target surface image.

[0176] The explanation of steps 310 to 312 can be found in the relevant description in any embodiment of this disclosure, and will not be repeated here.

[0177] The data processing method of this disclosure can determine a second target node with a relatively small reference error and a first target node with a relatively large reference error from each node according to the reference error of each node in the tree structure. Thus, the leaf nodes of the second target node with a relatively small reference error that have a parent-child relationship can be regarded as the second leaf nodes with a relatively high degree of normal vector consistency, and the leaf nodes of the first target node with a relatively large reference error that have a parent-child relationship can be regarded as the first leaf nodes with a relatively low degree of normal vector consistency.

[0178] To clearly illustrate how the vector field of each leaf node is determined in any embodiment of this disclosure, this disclosure also proposes a data processing method.

[0179] Figure 7This is a schematic flowchart illustrating a data processing method provided in another embodiment of this disclosure.

[0180] like Figure 7 As shown, the data processing method may include the following steps:

[0181] Step 401: Based on the point cloud map obtained by sampling the target surface in three-dimensional space, obtain the first position of multiple sampling points in the point cloud map and the first normal vector of multiple sampling points.

[0182] Step 402: Generate a tree structure based on the first position of multiple sampling points. The tree structure includes multiple leaf nodes, which correspond one-to-one with multiple sampling regions divided in three-dimensional space. Each leaf node includes at least one sampling point whose first position is located within the corresponding sampling region.

[0183] For example, the above three-dimensional space can be divided to obtain a tree structure that conforms to a first set depth (e.g., D), where the leaf nodes can be the leaf nodes with the maximum depth D in the tree structure.

[0184] Step 403: Determine the first leaf node and the second leaf node from multiple leaf nodes, wherein the consistency of the first normal vector between the sampling point in the first leaf node and the sampling point in the adjacent sampling region is less than the consistency of the first normal vector between the sampling point in the second leaf node and the sampling point in the adjacent sampling region.

[0185] The explanations of steps 401 to 403 can be found in the relevant descriptions in any embodiment of this disclosure, and will not be repeated here.

[0186] Step 404: Apply a Gaussian filter with a variance of the first variance to the sampling points in the first leaf node to determine the vector field, and apply a Gaussian filter with a variance of the second variance to the sampling points in the second leaf node to determine the vector field.

[0187] In this embodiment of the disclosure, the first variance is less than the second variance.

[0188] As one possible implementation, the first variance can be determined based on a first predetermined depth corresponding to the tree structure, and the second variance can be determined based on the difference between the first predetermined depth and a fixed value, where the fixed value is preset, for example, 1, 2, etc. Taking a first predetermined depth of D and a fixed value of 1 as an example, the first variance could be 2. -D The second variance can be 2. -(D-1) .

[0189] In this embodiment of the disclosure, for any first leaf node, a Gaussian filter with a first variance can be applied to the sampling points in the first leaf node to determine the vector field corresponding to the first leaf node. For any second leaf node, a Gaussian filter with a second variance can be applied to the sampling points in the second leaf node to determine the vector field corresponding to the second leaf node.

[0190] As an example, using a fixed value of 1, the `setNormalField` function can be used to calculate the depth as the first set depth D (with a first variance of 2). -D The vector field at the first leaf node of a vector field can be calculated using the `setNormalField` function, with a depth of D⁻¹ (second variance of 2). -(D-1) The vector field at the second leaf node of ).

[0191] Using an octree as an example, the variance of the smoothing filter (i.e., the Gaussian filter) F can be adjusted by changing the depth D of the octree nodes. Since the variance of the Gaussian filter is set to 2 in Premiere Pro... -D Therefore, the variance of the Gaussian filter can be changed by altering D. The depth of the second leaf node with a relatively low reference error is set to D-1, while the depth of the first leaf node with a relatively high reference error remains at the original D. For the first leaf node, the vector field at that first leaf node can be calculated using the setNormalField function at depth D. For the second leaf node, its depth can be reduced to D-1, and the vector field at the second leaf node can be calculated using the setNormalField function at depth D-1.

[0192] Step 405: Based on the vector fields of the first leaf node and the second leaf node, the target surface is reconstructed to obtain the reconstructed target surface image.

[0193] Step 406: Display the reconstructed target surface image.

[0194] The explanation of steps 405 to 406 can be found in the relevant description in any embodiment of this disclosure, and will not be repeated here.

[0195] The data processing method of this disclosure determines the vector field by applying a Gaussian filter with a variance of a first variance to the sampling points in the first leaf node, and applying a Gaussian filter with a variance of a second variance to the sampling points in the second leaf node; the first variance is less than the second variance. Therefore, the vector fields of the first and second leaf nodes can be effectively calculated based on Gaussian filters with different variances.

[0196] To clearly illustrate any of the above embodiments, this disclosure also proposes a data processing method.

[0197] Figure 8 This is a schematic flowchart illustrating a data processing method provided in another embodiment of this disclosure.

[0198] like Figure 8 As shown, the data processing method may include the following steps:

[0199] Step 501: Based on the point cloud map obtained by sampling the target surface in three-dimensional space, obtain the first position of multiple sampling points in the point cloud map and the first normal vector of multiple sampling points.

[0200] Step 502: Generate a tree structure based on the first positions of multiple sampling points. The tree structure includes multiple leaf nodes, which correspond one-to-one with multiple sampling regions divided in three-dimensional space. Each leaf node includes at least one sampling point whose first position is located within the corresponding sampling region.

[0201] For example, the above three-dimensional space can be divided to obtain a tree structure that conforms to a first set depth (e.g., D), where the leaf nodes can be the leaf nodes with the maximum depth D in the tree structure.

[0202] Step 503: Determine the first leaf node and the second leaf node from multiple leaf nodes, wherein the consistency of the first normal vector between the sampling point in the first leaf node and the sampling point in the adjacent sampling region is less than the consistency of the first normal vector between the sampling point in the second leaf node and the sampling point in the adjacent sampling region.

[0203] The explanation of steps 501 to 503 can be found in the relevant description in any embodiment of this disclosure, and will not be repeated here.

[0204] Step 504: Determine the third position corresponding to each leaf node based on the first position of at least one sampling point in each leaf node.

[0205] In this embodiment of the disclosure, for any leaf node among all leaf nodes, the third position corresponding to the leaf node can be determined based on the first position of at least one sampling point in the leaf node.

[0206] As an example, the mean of the first position of at least one sampling point in the leaf node can be calculated to obtain the third position corresponding to the leaf node.

[0207] For example, suppose leaf node A contains two sampling points, namely sampling point 1 and sampling point 2. Suppose the first position of sampling point 3 is (x3, y3, z3) and the first position of sampling point 4 is (x4, y4, z4). Then the third position of leaf node A is ((x3+x4) / 2, (y3+y4) / 2, (z3+z4) / 2).

[0208] As another example, the first positions of at least one sampling point in the leaf node can be weighted and summed to obtain the third position corresponding to the leaf node.

[0209] As another example, the maximum, minimum, or intermediate position of the first position of at least one sampling point in the leaf node can be used as the third position corresponding to the leaf node, and this disclosure does not limit this.

[0210] Step 505: Determine the third normal vector corresponding to each leaf node based on the first normal vector of at least one sampling point in each leaf node.

[0211] In this embodiment of the disclosure, for any leaf node among all leaf nodes, the third normal vector corresponding to the leaf node can be determined based on the first normal vector of at least one sampling point in the leaf node.

[0212] As an example, the mean of the first normal vector of at least one sampling point in the leaf node can be taken to obtain the third normal vector corresponding to the leaf node.

[0213] As another example, the first normal vector of at least one sampling point in the leaf node can be weighted and summed to obtain the third normal vector corresponding to the leaf node.

[0214] As another example, any one of the first normal vectors of at least one sampling point in the leaf node can be used as the third normal vector corresponding to the leaf node.

[0215] Step 506: Determine the vector field using a Gaussian filter with the first variance based on the third position and the third normal vector corresponding to each first leaf node.

[0216] In this embodiment of the disclosure, for any first leaf node, the vector field corresponding to the first leaf node can be determined by using a Gaussian filter with a first variance based on the third position and the third normal vector corresponding to the first leaf node.

[0217] As an example, for the first leaf node, the setNormalField function can be used at depth D to calculate the vector field at the first leaf node based on the third position and the third normal vector corresponding to that first leaf node.

[0218] Step 507: Determine the vector field using a Gaussian filter with second variance based on the third position and third normal vector corresponding to each second leaf node.

[0219] The second variance is greater than the first variance. It should be noted that the explanations of the first and second variances in the previous embodiments also apply to this embodiment, and will not be repeated here.

[0220] In this embodiment of the disclosure, for any second leaf node, the vector field of the second leaf node can be determined by using a Gaussian filter with a second variance based on the third position and the third normal vector corresponding to the second leaf node.

[0221] As an example, for the second leaf node, the setNormalField function can be used in D-1 to calculate the vector field at the second leaf node based on the third position and the third normal vector corresponding to that second leaf node.

[0222] In other words, in this disclosure, after calculating the EE corresponding to each node, for nodes with relatively large EE, the variance of the Gaussian filter F can be reduced, thereby reducing the neighborhood range traversed by formula (7), excluding normals in different directions, and thus reducing the situation of normal cancellation.

[0223] As an example, such as Figure 9 As shown, the dashed circle represents the range where the Gaussian filter F has a relatively high weight, i.e., the neighborhood in formula (7), which incorrectly includes the normal of the other side. Moving the normal in the dashed circle to point P, it can be seen that the normal cancellation is quite severe, and the integral / accumulation is close to 0. If the variance of the Gaussian filter F is reduced, the dashed circle will become smaller, excluding the downward-facing side, which can avoid the problem of normal cancellation as much as possible, and thus the correct vector field can be calculated.

[0224] Step 508: Based on the vector fields of the first leaf node and the second leaf node, the target surface is reconstructed to obtain the reconstructed target surface image.

[0225] Step 509: Display the reconstructed target surface image.

[0226] The explanation of steps 508 to 509 can be found in the relevant description in any embodiment of this disclosure, and will not be repeated here.

[0227] As an example, for the vector field in formula (4) The discrete form, given a point set S, can be... Divide the data into different small blocks (i.e., sampling regions) such that each point s (i.e., leaf node) in S corresponds to a small block (i.e., a 5-sampling region) P. s Then the vector field As shown in formula (5), that is:

[0228]

[0229] Because of |P in formula (5) s |Unknown, the density estimator in PR can be used to estimate the local sampling density:

[0230]

[0231] Wherein, the sampling density W and |P s | is inversely proportional, therefore there is an approximate relationship.

[0232] Simply put, It is the weighted sum of the normals of several nodes in the neighborhood of point q:

[0233]

[0234] As an example, let point s be... Figure 6 The triangle shown on the left is used as an example; each point s has a normal (i.e., a normal vector). Figure 6 The middle section shows the divergence of the vector field V at each point s. The divergence of the vector field V is smaller at the three vertices of the triangle because the directions of the normals near the vertices are opposite, canceling each other out in the summation corresponding to formula (7). The gradient (i.e., V) tends to 0, producing an error. Therefore, the calculation of the vector field in formula (2) is inaccurate, and the reconstructed surface (level set) Precision is also lost at the vertices, such as... Figure 6 As shown on the right.

[0235] To reduce the above errors, the following reference error EE can be defined:

[0236]

[0237] In formula (8), the numerator on the right side is The more severely the normal vector is canceled out, The smaller the value of , the larger the reference error EE. The denominator on the right side of the equation is used for normalization to ensure that EE is between 0 and 1. From formula (5), it can be seen that when there is absolutely no normal (i.e., normal vector) to cancel each other out, i.e. Since the directions are consistent, it can be extracted from the integral. Also, since the magnitude of the normal (i.e., the normal vector) is 1, the denominator equals the numerator. When the normal vectors (i.e., the normal lines) completely cancel each other out, the numerator on the right side of the equation is 0, and EE reaches its maximum value of 1. Therefore, EE can accurately predict the error caused by the cancellation of normal vectors.

[0238] Furthermore, the computation of EE does not take much time because the PR algorithm itself requires the computation of vector fields. And the molecules in EE are precisely Simply calculate one more modulus; and the denominator is equivalent to The special case where the vector field is always (1,0,0) can be solved by performing the vector field operation once more. The calculation, where let Take the first component of the result. Finally, iterate through each node, performing a division and a subtraction to obtain the EE for each node. Since... The calculations and arithmetic operations account for a small percentage of the time spent in PR, therefore, the calculations of EE will not add much time.

[0239] After calculating the EE of each node, the variance of the Gaussian filter F can be reduced at nodes with larger EE, thereby reducing the neighborhood range traversed by formula (7), excluding normals in different directions, and thus reducing the situation of normal cancellation.

[0240] Optionally, an example is given using an octree O as the tree structure, such as... Figure 10 As shown, the surface reconstruction process can be explained in detail through the following steps:

[0241] (1) Construct an octree and compute vector fields using the original PR.

[0242] 1. The input to the algorithm is a set of input points {(x_i.e., normal vectors) with normals. i ,y i ,z i ,nx i ,ny i ,nz i )|i=0,1,…,n}, where n is the number of sampling points, (x i ,y i ,z i (nx) represents the coordinates of sampling point i, and (nx) represents the coordinates of sampling point i. i ,ny i ,nz i ) is the normal vector of sampling point i. These input point sets can be obtained by scanning the surface of the target object (such as a face, object, etc.) with a 3D sensor. In this disclosure, the reference error can be divided into two categories: large error and small error. The following β represents the proportion of the small error, which is between 0 and 1, and is generally defaulted to 0.9.

[0243] 2. Construct an octree O (depth is a predefined depth, e.g., D) on the input point set. The maximum depth of the octree is D, meaning the minimum width of the tree nodes is 2. -D Where there are no data points, the nodes of the octree are at depth 3; where there are data points, the nodes of the octree are at depth D.

[0244] 3. Compress the input point set into an octree O. Traverse all nodes in octree O. If a node contains multiple sampling points, average the coordinates and normal vectors of each sampling point to obtain the coordinates and normal vectors of the node. Through the above operations, we can obtain the point set S = {s = (x...} i ,y i ,z i ,nx i ,ny i ,nz i ,num i ,node)|i=0,1,…,n}, where num i s is the number of sampling points in a node, and node is the address of the node containing sampling point s. Therefore, each node in octree O can include 1 or 0 points.

[0245] Similar to the above operation, a set S' at a second predetermined depth (e.g., D-2) can be obtained. S' is coarser than S, meaning that S contains the leaf nodes, while S' contains the target nodes. Subsequent operations can be performed on the octree O and S and S' without using the input point set. The purpose of establishing S' is to quickly estimate the EE of each node in the octree O according to formula (8).

[0246] 4. Calculate the sampling density W at the second set depth according to formula (6).

[0247] 5. If β>=1, then use the original PR method and use the setNormalField function on S to calculate the vector field of each node at the first set depth D.

[0248] Among them, setNormalField calculates the vector field on the octree. The function, for each node q, can be obtained by weighted summation according to formulas (5)-(7). Function declaration is Where S is the set of points compressed into an octree (depth D), W is calculated by formula (6), D is the depth of the calculation, and the output is a vector field defined on the octree O.

[0249] 6. Substitute the vector field of each node into formula (2) and solve formula (2) to extract the solution χ.M The isosurface is obtained and output.

[0250] (2) Reference error

[0251] 7. If β < 1, the error-adaptive reconstruction method provided in this disclosure is used to estimate the EE of each node. To balance reconstruction speed and robustness, the EE can be calculated at nodes at a second predetermined depth (D-2).

[0252] 8. In S' at depth D-2, calculate the vector field of each target node using formula (5):

[0253]

[0254] beg Even if you use the setNormalField function in PR to calculate the vector field

[0255] That is, in formula (5), s can be changed to s', S can be changed to S', and P can be changed to S'. s Change to P s’ Where S' refers to the target node set, and point s' in S' is a target node, and the sampling region P of each point s' is... s’ , is the sum of the sampling regions corresponding to the child nodes of point s', thus the vector field of each target node can be calculated.

[0256] 9. A similar method can be used to find the denominator in EE. Simply set all the normals in the setNormalField function to (1,0,0), and then take the first component of the result.

[0257] 10. Traverse all nodes: Using the numerator and denominator calculated in steps 8 and 9, calculate the EE of each node. Each node only needs to be calculated by one division and one subtraction. Finally, the reference error EE of each node can be obtained.

[0258] 11. Sort the nodes according to the order of reference error from largest to smallest, select the nodes with a set percentage (1-100β) at the top of the sort as high error nodes, and select the nodes with a percentage (100β) at the bottom of the sort as low error nodes. For example, when β is 0.9, select the first 10% of the sorted nodes as high error nodes and select the last 90% of the sorted nodes as low error nodes.

[0259] The threshold can be calculated based on the EE of the last 100β nodes in the sorting. If a node's EE is greater than the threshold, the node is considered a high-error node; if a node's EE is less than or equal to the threshold, the node is considered a low-error node.

[0260] (3) Error-adaptive octree / Gaussian filter

[0261] This disclosure allows adjustment of the variance of the smoothing filter F by changing the depth D of the octree nodes, since the variance of the filter is set to 2 in PR. -D Therefore, the variance of the filter can be changed by changing D. The depth of the node with low error is set to D-1, and the depth of the node with high error is the original D.

[0262] As an example, we will use the two-dimensional case of an octree, namely a quadtree, as shown in Figure 11. Figure 11 In the diagram, (a) and (c) are quadtrees from the original PR at depths of 4 and 5, respectively. The quadtree nodes have the same width, but it's unnecessary to use the same width around the sides and corners of the triangles because the reconstruction error is mainly concentrated at the corners of the triangles. Figure 11 As shown in (c) in the figure. Figure 11 In (b), the error-adaptive quadtree is at a depth of 5. If the reference error of a node is relatively high, the corresponding node is located on a grid at a depth of 5. If the reference error of a node is relatively low, the corresponding node is located on a grid at a depth of 4.

[0263] 12. Traverse each node in the octree O.

[0264] 13. If the reference error EE of a node is large, the node is considered a high-error node. The vector field at that node can be calculated using the setNormalField function at depth D.

[0265] 14. If the reference error EE of a certain node is relatively small, then the node is considered a low-error node. The depth of the node can be reduced to D-1 (i.e., the difference between the first set depth and the fixed value), and then the setNormalField function can be used to calculate the vector field at the node.

[0266] 15. End the traversal and obtain the vector field of each node under error adaptive conditions.

[0267] 16. Returning to the original PR process, convert the vector fields of each node... Substitute into formula (2) and solve formula (2) to extract the solution χ. M The isosurface is obtained and output.

[0268] The error-adaptive reconstruction method disclosed herein can be applied to both PR and SPR, denoted by CAPR (Calculated Error Adaptive PR) and CASPR (Calculated Error Adaptive SPR), respectively. Since SPR is an improved version of PR, the CASPR method is recommended, while CAPR is only used to verify the improvement effect of error adaptation.

[0269] The following experimental data demonstrates that CASPR can improve time and space efficiency without affecting reconstruction accuracy, making it more suitable for operation on mobile devices:

[0270] First, accuracy

[0271] The reconstruction accuracy under an octree of depth 9 is evaluated through benchmark testing. This disclosure employs five models (Anchor, Daratch, Dancing, Gargoyle, and Quasimodo), each generating numerous point clouds through virtual scanning. Each point cloud is reconstructed using methods such as surface shaded display (SSD), SPR, and CASPR. Test results are shown below. Figure 12 As shown, the test results include the angle and distance errors of SSD, SPR, and CASPR relative to PR. The test value is the error of other methods (i.e., SSD, SPR, and CASPR) divided by the error of PR. If the test value is less than 1, it means that other methods have made improvements relative to PR.

[0272] Figure 12 The test results include the ratio of the average angle (top) error of each method to the angle error of PR, and the ratio of the average distance (bottom) error to the distance error of PR. For example... Figure 12 It can be seen that the error adaptive method improves the accuracy of SPR. The angle error of CASPR is smaller than that of SPR, and the distance errors of CASPR and SPR are similar, both of which are better than the error of PR (i.e., Figure 12 (The horizontal black line in the middle).

[0273] Second, speed and space utilization.

[0274] Table 1 shows the reconstruction efficiency of CAPR and CASPR. It is noteworthy that CASPR's angular error is smaller when the number of vertices on the output surface is less. The number of vertices on the output surface is directly proportional to the space occupied by the output 3D model, with a scaling factor of approximately 10 bytes (the three float types in the position (x,y,z) alone occupy 12 bytes, plus several triangles referencing this vertex).

[0275] Table 1 shows the reconstruction efficiency of the Neptune model. This disclosure tests the running time and the number of vertices on the output surface of each method. The CAPR and CASPR proposed in this disclosure improve the efficiency in terms of time and space, reducing the running time to about 70% of the original and the number of vertices on the output surface to about 50% of the original.

[0276] Table 1

[0277]

[0278] The data processing method of this disclosure determines the vector field by applying a Gaussian filter with a variance of a first variance to the sampling points in the first leaf node, and applying a Gaussian filter with a variance of a second variance to the sampling points in the second leaf node; the first variance is less than the second variance. Therefore, the vector fields of the first and second leaf nodes can be effectively calculated based on Gaussian filters with different variances.

[0279] With the above Figures 2 to 8 Corresponding to the data processing method provided in the embodiments, this disclosure also provides a data processing apparatus. Because the data processing apparatus provided in the embodiments of this disclosure is similar to the one described above… Figures 2 to 8 The data processing method provided in the embodiments corresponds to the data processing apparatus provided in the embodiments of this disclosure, and will not be described in detail in the embodiments of this disclosure.

[0280] Figure 13 This is a schematic diagram of the structure of a data processing apparatus provided in an embodiment of the present disclosure.

[0281] like Figure 13 As shown, the data processing device 1300 may include: an acquisition module 1301, a generation module 1302, a first determination module 1303, a second determination module 1304, a reconstruction module 1305, and a display module 1306.

[0282] The acquisition module 1301 is used to acquire the first position of multiple sampling points and the first normal vector of multiple sampling points in the point cloud map obtained by sampling the target surface in three-dimensional space.

[0283] The generation module 1302 is used to generate a tree structure based on the first position of multiple sampling points. The tree structure includes multiple leaf nodes, and the multiple leaf nodes correspond one-to-one with multiple sampling regions divided in three-dimensional space. Each leaf node includes at least one sampling point whose first position is located in the corresponding sampling region.

[0284] The first determining module 1303 is used to determine a first leaf node and a second leaf node from multiple leaf nodes, wherein the consistency of the first normal vector between the sampling point in the first leaf node and the sampling point in the adjacent sampling region is less than the consistency of the first normal vector between the sampling point in the second leaf node and the sampling point in the adjacent sampling region.

[0285] The second determining module 1304 is used to determine the vector field by applying filters of different precision to the sampling points in the first leaf node and the sampling points in the second leaf node.

[0286] The reconstruction module 1305 is used to reconstruct the target surface based on the vector field of the first leaf node and the vector field of the second leaf node, so as to obtain the reconstructed target surface image.

[0287] Display module 1306 is used to display the reconstructed target surface image.

[0288] In one possible implementation of this disclosure, the generation module 1302 is configured to: divide the sampling region based on three-dimensional space to obtain a tree structure conforming to a first predetermined depth; wherein the tree structure includes each leaf node and at least one non-leaf node having a parent-child relationship with at least one leaf node; among the at least two nodes having a parent-child relationship, the sampling region corresponding to the parent node in three-dimensional space is the sum of the sampling regions corresponding to the child nodes; determine the sampling points included in each leaf node based on the sampling points contained in the sampling regions corresponding to each leaf node; and determine the sampling points included in each non-leaf node based on the sampling points contained in the sampling regions corresponding to each non-leaf node.

[0289] In one possible implementation of this disclosure, the first determining module 1303 is configured to: determine a plurality of target nodes with a depth of a second predetermined depth from the non-leaf nodes of the tree structure; wherein the second predetermined depth is less than the first predetermined depth; determine a second position corresponding to each target node based on a first position of at least one sampling point in the sampling area corresponding to each target node; determine a second normal vector corresponding to each target node based on a first normal vector of at least one sampling point in the sampling area corresponding to each target node; determine a vector field of each target node based on the second position and the second normal vector corresponding to each target node; determine a reference error of each target node based on the magnitude of the vector field of each target node; wherein the reference error is used to indicate the degree of inconsistency between the second normal vector of the corresponding target node and the second normal vector of the adjacent target node; determine a first target node and a second target node from the plurality of target nodes based on the reference error; wherein the reference error of the first target node is greater than the reference error of the second target node; and designate the leaf nodes of the first target node with a parent-child relationship as the first leaf nodes, and designate the leaf nodes of the second target node with a parent-child relationship as the second leaf nodes.

[0290] In one possible implementation of this disclosure, the first determining module 1303 is used to: normalize the magnitude of the vector field of each target node; and use the normalized magnitude of the vector field of each target node as the reference error of each target node.

[0291] In one possible implementation of this disclosure, the first determining module 1303 is configured to: sort the target nodes according to the reference error from largest to smallest; take the target nodes with a predetermined proportion at the top of the sort as the first target nodes, and take the remaining target nodes as the second target nodes.

[0292] In one possible implementation of this disclosure, the second determining module 1304 is configured to: apply a Gaussian filter with a variance of the first variance to the sampling points in the first leaf node to determine the vector field, and apply a Gaussian filter with a variance of the second variance to the sampling points in the second leaf node to determine the vector field; wherein the first variance is less than the second variance.

[0293] In one possible implementation of this disclosure, the data processing apparatus 1300 may further include:

[0294] The third determining module is used to determine the first variance based on the first set depth of the tree structure.

[0295] The fourth determining module is used to determine the second variance based on the difference between the first set depth and the fixed value.

[0296] In one possible implementation of this disclosure, the second determining module 1304 is configured to: determine the third position corresponding to each leaf node based on the first position of at least one sampling point in each leaf node; determine the third normal vector corresponding to each leaf node based on the first normal vector of at least one sampling point in each leaf node; determine the vector field using a Gaussian filter with a first variance based on the third position and the third normal vector corresponding to each first leaf node; and determine the vector field using a Gaussian filter with a second variance based on the third position and the third normal vector corresponding to each second leaf node.

[0297] The data processing apparatus of this embodiment obtains the first positions and first normal vectors of multiple sampling points in a point cloud map obtained by sampling a target surface in three-dimensional space; generates a tree structure based on the first positions of the multiple sampling points, wherein the tree structure includes multiple leaf nodes, each leaf node corresponding to multiple sampling regions divided in three-dimensional space, and each leaf node includes at least one sampling point whose first position is located within the corresponding sampling region; determines a first leaf node and a second leaf node from the multiple leaf nodes, wherein the consistency of the first normal vector between the sampling points in the first leaf node and the sampling points in the adjacent sampling regions is less than the consistency of the first normal vector between the sampling points in the second leaf node and the sampling points in the adjacent sampling regions; determines the vector field by applying filters of different precision to the sampling points in the first leaf node and the sampling points in the second leaf node; reconstructs the target surface based on the vector fields of the first leaf node and the second leaf node to obtain a reconstructed target surface image; and displays the reconstructed target surface image. Therefore, based on the consistency of the normal vectors between the sampling points in each leaf node and the sampling points in the adjacent sampling regions, filters of different precision can be selected to determine the vector field of each leaf node. Thus, the target surface can be reconstructed based on the vector field of each leaf node. This method can accelerate the calculation speed of the vector field by using relatively low precision filters while meeting the reconstruction accuracy requirements, thereby improving the reconstruction efficiency. It can be applied to mobile terminals with high requirements for reconstruction speed, thus improving the applicability of the method.

[0298] To implement the above embodiments, this disclosure also proposes an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the data processing method proposed in any of the foregoing embodiments of this disclosure.

[0299] To implement the above embodiments, this disclosure also proposes a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data processing method as proposed in any of the foregoing embodiments of this disclosure.

[0300] To implement the above embodiments, this disclosure also proposes a computer program product that, when the instructions in the computer program product are executed by a processor, performs a data processing method as proposed in any of the foregoing embodiments of this disclosure.

[0301] Figure 14 This is a block diagram illustrating an electronic device according to an exemplary embodiment. For example, the electronic device 1400 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0302] Reference Figure 14 The electronic device 1400 may include one or more of the following components: processing component 1402, memory 1404, power component 1406, multimedia component 1408, audio component 1410, input / output (I / O) interface 1412, sensor component 1414, and communication component 1416.

[0303] Processing component 1402 typically controls the overall operation of electronic device 1400, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 1402 may include one or more processors 1420 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 1402 may include one or more modules to facilitate interaction between processing component 1402 and other components. For example, processing component 1402 may include a multimedia module to facilitate interaction between multimedia component 1408 and processing component 1402.

[0304] Memory 1404 is configured to store various types of data to support the operation of electronic device 1400. Examples of this data include instructions for any application or method operating on electronic device 1400, contact data, phonebook data, messages, pictures, videos, etc. Memory 1404 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0305] Power component 1406 provides power to various components of electronic device 1400. Power component 1406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 1400.

[0306] Multimedia component 1408 includes a screen that provides an output interface between electronic device 1400 and a user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 1408 includes a front-facing camera and / or a rear-facing camera. When electronic device 1400 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0307] Audio component 1410 is configured to output and / or input audio signals. For example, audio component 1410 includes a microphone (MIC) configured to receive external audio signals when electronic device 1400 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 1404 or transmitted via communication component 1416. In some embodiments, audio component 1410 also includes a speaker for outputting audio signals.

[0308] I / O interface 1412 provides an interface between processing component 1402 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0309] Sensor assembly 1414 includes one or more sensors for providing state assessments of various aspects of electronic device 1400. For example, sensor assembly 1414 may detect the on / off state of electronic device 1400, the relative positioning of components such as the display and keypad of electronic device 1400, changes in position of electronic device 1400 or a component of electronic device 1400, the presence or absence of user contact with electronic device 1400, orientation or acceleration / deceleration of electronic device 1400, and temperature changes of electronic device 1400. Sensor assembly 1414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 1414 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 1414 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0310] Communication component 1416 is configured to facilitate wired or wireless communication between electronic device 1400 and other devices. Electronic device 1400 can access wireless networks based on communication standards, such as WiFi, 4G, or 5G, or combinations thereof. In one exemplary embodiment, communication component 1416 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 1416 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0311] In an exemplary embodiment, the electronic device 1400 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0312] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1404 including instructions, which can be executed by a processor 1420 of an electronic device 1400 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0313] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0314] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0315] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain.

[0316] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0317] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0318] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware, and the program can be stored in a computer-readable storage medium. When executed, the program includes one or a combination of the steps of the method embodiments.

[0319] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0320] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A data processing method, characterized in that, The method includes: Based on the point cloud map obtained by sampling the target surface in three-dimensional space, the first position of multiple sampling points in the point cloud map and the first normal vector of the multiple sampling points are obtained. A tree structure is generated based on the first position of the plurality of sampling points, wherein the tree structure includes a plurality of leaf nodes, and the plurality of leaf nodes correspond one-to-one with the plurality of sampling regions divided in the three-dimensional space, wherein each leaf node includes at least one sampling point whose first position is located within the corresponding sampling region; The first leaf node and the second leaf node are determined from multiple leaf nodes, wherein the consistency of the first normal vector between the sampling point in the first leaf node and the sampling point in the adjacent sampling region is less than the consistency of the first normal vector between the sampling point in the second leaf node and the sampling point in the adjacent sampling region. The vector field is determined by applying filters of different precision to the sampling points in the first leaf node and the sampling points in the second leaf node, respectively. Based on the vector fields of the first leaf node and the second leaf node, the target surface is reconstructed to obtain a reconstructed target surface image. Displays a reconstructed image of the target surface.

2. The method according to claim 1, characterized in that, The step of generating a tree structure based on the first positions of the plurality of sampling points includes: The sampling area is divided based on the three-dimensional space to obtain a tree structure that conforms to a first set depth; wherein, the tree structure includes each of the leaf nodes and at least one non-leaf node that has a parent-child relationship with at least one of the leaf nodes; among the at least two nodes with a parent-child relationship, the sampling area corresponding to the parent node in the three-dimensional space is the sum of the sampling areas corresponding to the child nodes; The sampling points included in each leaf node are determined based on the sampling points contained in the sampling area corresponding to each leaf node. The sampling points included in each non-leaf node are determined based on the sampling points contained in the sampling area corresponding to each non-leaf node.

3. The method according to claim 2, characterized in that, Determining the first leaf node and the second leaf node from the plurality of leaf nodes includes: From the non-leaf nodes of the tree structure, determine a plurality of target nodes with a depth of a second predetermined depth; wherein the second predetermined depth is less than the first predetermined depth; The second position corresponding to each target node is determined based on the first position of at least one sampling point in the sampling area corresponding to each target node; Based on the first normal vector of at least one sampling point in the sampling region corresponding to each target node, determine the second normal vector corresponding to each target node; Based on the second position and the second normal vector corresponding to each target node, determine the vector field of each target node; Based on the magnitude of the vector field of each target node, the reference error of each target node is determined; wherein, the reference error is used to indicate the degree of inconsistency between the second normal vector of the corresponding target node and the second normal vector of the adjacent target node; Based on the reference error, a first target node and a second target node are determined from the plurality of target nodes; wherein the reference error of the first target node is greater than the reference error of the second target node; The leaf nodes of the first target node that have a parent-child relationship are designated as the first leaf nodes, and the leaf nodes of the second target node that have a parent-child relationship are designated as the second leaf nodes.

4. The method according to claim 3, characterized in that, The step of determining the reference error of each target node based on the magnitude of the vector field of each target node includes: Normalize the magnitude of the vector field of each target node; The normalized magnitude of the vector field of each target node is used as the reference error of each target node.

5. The method according to claim 3, characterized in that, The step of determining the first target node and the second target node from the plurality of target nodes based on the reference error includes: The target nodes are sorted according to the reference error from largest to smallest. The target nodes that are ranked first by a predetermined proportion are designated as the first target nodes, and the remaining target nodes are designated as the second target nodes.

6. The method according to any one of claims 1-5, characterized in that, The step of determining the vector field by applying filters of different precision to the sampling points in the first leaf node and the sampling points in the second leaf node includes: A Gaussian filter with a variance of the first variance is applied to the sampling points in the first leaf node to determine the vector field, and a Gaussian filter with a variance of the second variance is applied to the sampling points in the second leaf node to determine the vector field. Wherein, the first variance is less than the second variance.

7. The method according to claim 6, characterized in that, The method further includes: The first variance is determined based on the first predetermined depth of the tree structure; The second variance is determined based on the difference between the first set depth and the fixed value.

8. The method according to claim 6, characterized in that, The step of applying a Gaussian filter with a variance of the first variance to the sampling points in the first leaf node to determine the vector field, and applying a Gaussian filter with a variance of the second variance to the sampling points in the second leaf node to determine the vector field, includes: Based on the first position of at least one sampling point in each leaf node, determine the third position corresponding to each leaf node; Based on the first normal vector of at least one sampling point in each leaf node, determine the third normal vector corresponding to each leaf node; Based on the third position and third normal vector corresponding to each first leaf node, a Gaussian filter with the first variance is used to determine the vector field. The vector field is determined using a Gaussian filter with the second variance based on the third position and the third normal vector corresponding to each of the second leaf nodes.

9. A data processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire the first position of multiple sampling points in the point cloud map and the first normal vector of the multiple sampling points based on the point cloud map obtained by sampling the target surface in three-dimensional space. The generation module is used to generate a tree structure based on the first position of the plurality of sampling points, wherein the tree structure includes a plurality of leaf nodes, the plurality of leaf nodes correspond one-to-one with the plurality of sampling regions divided in the three-dimensional space, and each leaf node includes at least one sampling point whose first position is located in the corresponding sampling region. The first determining module is used to determine a first leaf node and a second leaf node from the plurality of leaf nodes, wherein the consistency of the first normal vector between the sampling point in the first leaf node and the sampling point in the adjacent sampling region is less than the consistency of the first normal vector between the sampling point in the second leaf node and the sampling point in the adjacent sampling region. The second determining module is used to determine the vector field by applying filters of different precision to the sampling points in the first leaf node and the sampling points in the second leaf node respectively; The reconstruction module is used to reconstruct the target surface based on the vector field of the first leaf node and the vector field of the second leaf node, so as to obtain a reconstructed target surface image. The display module is used to display the reconstructed target surface image.

10. The apparatus according to claim 9, characterized in that, The generation module is used for: The sampling region is divided based on the three-dimensional space to obtain a tree structure that conforms to a first set depth; wherein, the tree structure includes each of the leaf nodes and at least one non-leaf node that has a parent-child relationship with at least one of the leaf nodes; among the at least two nodes with a parent-child relationship, the sampling region corresponding to the parent node in the three-dimensional space is the sum of the sampling regions corresponding to the child nodes; The sampling points included in each leaf node are determined based on the sampling points contained in the sampling area corresponding to each leaf node. The sampling points included in each non-leaf node are determined based on the sampling points contained in the sampling area corresponding to each non-leaf node.

11. The apparatus according to claim 10, characterized in that, The first determining module is used for: From the non-leaf nodes of the tree structure, determine a plurality of target nodes with a depth of a second predetermined depth; wherein the second predetermined depth is less than the first predetermined depth; The second position corresponding to each target node is determined based on the first position of at least one sampling point in the sampling area corresponding to each target node; Based on the first normal vector of at least one sampling point in the sampling region corresponding to each target node, determine the second normal vector corresponding to each target node; Based on the second position and the second normal vector corresponding to each target node, determine the vector field of each target node; Based on the magnitude of the vector field of each target node, the reference error of each target node is determined; wherein, the reference error is used to indicate the degree of inconsistency between the second normal vector of the corresponding target node and the second normal vector of the adjacent target node; Based on the reference error, a first target node and a second target node are determined from the plurality of target nodes; wherein the reference error of the first target node is greater than the reference error of the second target node; The leaf nodes of the first target node that have a parent-child relationship are designated as the first leaf nodes, and the leaf nodes of the second target node that have a parent-child relationship are designated as the second leaf nodes.

12. The apparatus according to claim 11, characterized in that, The first determining module is used for: Normalize the magnitude of the vector field of each target node; The normalized magnitude of the vector field of each target node is used as the reference error of each target node.

13. The apparatus according to claim 11, characterized in that, The first determining module is used for: The target nodes are sorted according to the reference error from largest to smallest. The target nodes that are ranked first by a predetermined proportion are designated as the first target nodes, and the remaining target nodes are designated as the second target nodes.

14. The apparatus according to any one of claims 9-13, characterized in that, The second determining module is used for: A Gaussian filter with a variance of the first variance is applied to the sampling points in the first leaf node to determine the vector field, and a Gaussian filter with a variance of the second variance is applied to the sampling points in the second leaf node to determine the vector field. Wherein, the first variance is less than the second variance.

15. The apparatus according to claim 14, characterized in that, The device further includes: The third determining module is used to determine the first variance based on the first set depth of the tree structure; The fourth determining module is used to determine the second variance based on the difference between the first set depth and the fixed value.

16. The apparatus according to claim 14, characterized in that, The second determining module is used for: Based on the first position of at least one sampling point in each leaf node, determine the third position corresponding to each leaf node; Based on the first normal vector of at least one sampling point in each leaf node, determine the third normal vector corresponding to each leaf node; Based on the third position and third normal vector corresponding to each first leaf node, a Gaussian filter with the first variance is used to determine the vector field. The vector field is determined using a Gaussian filter with the second variance based on the third position and the third normal vector corresponding to each of the second leaf nodes.

17. An electronic device, characterized in that, include: processor; A memory for storing executable instructions of the processor; wherein the processor is configured to invoke and execute the executable instructions stored in the memory to implement the data processing method as described in any one of claims 1-8.

18. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the data processing method as described in any one of claims 1-8.

19. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor, the data processing method as described in any one of claims 1-8 is performed.

Images