Grid extraction method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a grid extraction method and device, electronic equipment and a storage medium, and relates to the field of point cloud processing, and the method comprises the steps: determining a to-be-processed point cloud, carrying out the normalization of the to-be-processed point cloud, obtaining a first to-be-processed point cloud, calculating the point cloud average density of the first to-be-processed point cloud, calculating the normal of each point in the first to-be-processed point cloud, and obtaining a first to-be-processed point cloud; and based on the point cloud average density and the normal of each point, performing grid extraction through a rolling ball algorithm to obtain the Mesh grid of the to-be-processed point cloud, so that the collision detection precision can be ensured under the condition of reducing the construction difficulty.

Description

Technical Field

[0001] The present invention relates to the field of point cloud processing, and in particular to a grid extraction method, device, electronic device and storage medium. Background Art

[0002] The current mainstream methods for generating collision models are divided into several categories: bounding volume method, space segmentation and voxelization method, surface reconstruction and mesh generation, and machine learning assisted method.

[0003] The bounding volume method improves collision detection efficiency by simplifying the geometric structure and mainly includes three technologies: the convex hull algorithm converts the point cloud into a minimum convex polyhedron, which is suitable for robot path planning and has the advantage of simple calculation but limited accuracy in high-dimensional scenes; the bounding sphere hierarchy (BSH) adopts a recursive sphere tree structure to balance efficiency and accuracy, supports dynamic updates, and is suitable for large-scale point cloud scenes (such as cultural relics protection); the bounding box (AABB / OBB) quickly filters non-collision areas through cube partitioning and is widely used in industrial robot joint collision detection.

[0004] Spatial segmentation and voxelization methods improve collision detection efficiency through structured partitioning, mainly including two technologies: Octomap (octree map) constructs point clouds into octree voxel grids, which are suitable for dynamic obstacle detection in autonomous driving and robot navigation, combining memory efficiency and adaptability to dynamic scenes; Voxel Grid (Voxel Grid) generates uniform / adaptive three-dimensional grids, and realizes collision threshold analysis between robot end-tools and point clouds in industrial scenarios through volume intersection detection.

[0005] The surface reconstruction and mesh generation method achieves high-precision collision detection through geometric modeling technology, which mainly includes two technologies: Mesh generation (Poisson reconstruction) converts point clouds into triangular mesh models to support accurate collision detection; CAD model conversion generates parametric CAD models through point cloud fitting, which is used for accurate collision analysis in engineering simulation.

[0006] Machine learning-assisted methods improve collision detection accuracy through intelligent feature analysis, mainly including two technologies: feature extraction and deep learning use networks such as PointNet and PointGPT to extract point cloud features, combined with classifiers to realize collision risk prediction, which is suitable for dynamic obstacle avoidance in autonomous driving; real-time semantic segmentation divides the obstacle area in the point cloud and combines it with the bounding volume to generate a dynamic collision model, providing accurate collision detection support for complex scenarios.

[0007] Objective shortcomings of existing technologies: The shortcomings of bounding volume detection are mainly reflected in three aspects: the convex hull algorithm uses the minimum convex polyhedron approximation, which makes it difficult to accurately restore the shape of complex objects; although the bounding sphere hierarchy balances efficiency and accuracy through the sphere tree structure, the geometric characteristics of the bounding sphere itself lead to limited fit, which may generate redundant space and affect detection accuracy; bounding boxes (such as AABB / OBB) use regular shapes to simplify calculations, resulting in a large amount of invalid space inside the bounding box, which may misjudge irrelevant objects as potential collision objects, increasing the computational burden of subsequent precise detection; convex bounding boxes cannot fit the geometric features of concave objects. When processing objects with concave structures, it is necessary to rely on multi-layer nested bounding boxes or complex combination structures, which significantly increases the computational complexity.

[0008] The main disadvantages of spatial segmentation and voxelization methods include: resolution sensitivity leading to loss of details or a sharp increase in memory consumption, and the discretization loss of geometric features (such as edges and holes) during the voxelization process, which requires a trade-off between accuracy and computational resources.

[0009] The main disadvantages of surface reconstruction and mesh generation methods include: Mesh generation (such as Poisson reconstruction) requires high computing resources and is sensitive to point cloud noise, which can easily lead to incomplete meshes; Poisson reconstruction is suitable for complex unstructured surface modeling, and constructing octrees and solving the Poisson equation are very time-consuming, and normal vector calculation errors will directly affect the accuracy of the Poisson equation solution; CAD model conversion relies on parametric fitting, which may lose original point cloud details (such as surface features or irregular structures), resulting in deviations in collision analysis in engineering simulations, and the cost of model updates in dynamic scenarios is high.

[0010] The main disadvantages of machine learning-assisted methods include: feature extraction and deep learning rely on large amounts of labeled data for training, and the model is sensitive to interference such as noise and occlusion in complex scenarios, which may lead to misjudgment of collision risks.

[0011] Common shortcomings of collision detection methods include: high computational resource consumption in high-density point clouds or complex scenes (e.g., voxel grids and Poisson reconstruction); limited detection accuracy due to loss of geometric features (e.g., discretization, approximate fitting, or projective simplification); environmental interference such as sensor noise, occlusion, and illumination changes that can easily lead to misjudgments; and significant data dependency (e.g., machine learning requires large amounts of labeled data and traditional methods are sensitive to point cloud quality), which limits generalization in complex scenes. Furthermore, bounding volume methods suffer from the inherent contradiction between spatial redundancy and poor concave shape adaptation, while voxelization methods face a trade-off between resolution and memory. Summary of the Invention

[0012] The object of the present invention is to provide a grid extraction method, device, electronic device and storage medium, which can ensure collision detection accuracy while reducing construction difficulty.

[0013] In order to achieve the above objectives, the technical solutions adopted in the embodiments of the present application are as follows:

[0014] In a first aspect, an embodiment of the present application provides a grid extraction method, the method comprising:

[0015] Determine the point cloud to be processed;

[0016] Normalizing the to-be-processed point cloud to obtain a first to-be-processed point cloud;

[0017] Calculating the average density of the first point cloud to be processed;

[0018] Calculating the normal of each point in the first to-be-processed point cloud;

[0019] Based on the average density of the point cloud and the normal of each point, a mesh is extracted using a rolling ball algorithm to obtain a Mesh of the point cloud to be processed.

[0020] In an optional embodiment, the method further comprises:

[0021] Based on the average density of the point cloud, grid extraction is performed using an octree algorithm to obtain a voxel grid of the point cloud to be processed.

[0022] In an optional embodiment, the step of performing mesh extraction based on the average density of the point cloud and the normal of each point by a rolling ball algorithm to obtain a Mesh of the point cloud to be processed includes:

[0023] Based on the normals, construct a plurality of initial seed triangles, wherein the normal directions of the points included in the initial seed triangles are consistent;

[0024] constructing a first spherical surface based on each of the initial seed triangles and the average density of the point cloud;

[0025] Obtaining non-popular edges and non-popular points in the first spherical surface;

[0026] The non-popular edges and the non-popular points are deleted from the first spherical surface to obtain a Mesh grid of the point cloud to be processed.

[0027] In an optional embodiment, the method further comprises:

[0028] Determine the mesh resolution of the point cloud to be processed;

[0029] comparing the grid resolution with a preset grid resolution;

[0030] When the grid resolution is greater than the preset grid resolution, adjusting the average density of the point cloud;

[0031] A Mesh grid of the point cloud to be processed is obtained based on the adjusted average density of the point cloud.

[0032] In an optional embodiment, the step of calculating the average point cloud density of the first point cloud to be processed includes:

[0033] For each point in the first point cloud to be processed, calculating distances between the point and other points in the first point cloud to be processed except the point;

[0034] Obtaining a minimum distance from each of the distances;

[0035] Calculating an average of the minimum distances;

[0036] Based on the average value, the point cloud average density of the first to-be-processed point cloud is calculated.

[0037] In an optional embodiment, the average density of the point cloud is calculated using the following formula:

[0038]

[0039] Among them, r represents the average density of the point cloud, Γ is the value of the gamma function, and d is the average value of each minimum distance.

[0040] In an optional embodiment, the step of performing grid extraction based on the average density of the point cloud by an octree algorithm to obtain a voxel grid of the point cloud to be processed includes:

[0041] Constructing an octree of the first to-be-processed point cloud based on the average density of the point cloud;

[0042] Determining the vertices of each voxel in the octree;

[0043] Generate a triangle mesh index of each voxel based on the vertices of each voxel;

[0044] A voxel grid of the point cloud to be processed is constructed based on each of the indexes.

[0045] In a second aspect, an embodiment of the present application provides a grid extraction device, the device comprising:

[0046] A determination module, used to determine the point cloud to be processed;

[0047] a normalization module, configured to normalize the point cloud to be processed to obtain a first point cloud to be processed;

[0048] a calculation module, configured to calculate an average density of the first point cloud to be processed; and calculate a normal of each point in the first point cloud to be processed;

[0049] The mesh extraction module is used to extract the mesh based on the average density of the point cloud and the normal of each point through the rolling ball algorithm to obtain the Mesh mesh of the point cloud to be processed.

[0050] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the grid extraction method when executing the computer program.

[0051] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the grid extraction method when executed by a processor.

[0052] This application has the following beneficial effects:

[0053] This application determines the point cloud to be processed, normalizes the point cloud to be processed to obtain a first point cloud to be processed, calculates the average point cloud density of the first point cloud to be processed, calculates the normal of each point in the first point cloud to be processed, and extracts the mesh based on the average point cloud density and the normal of each point through the rolling ball algorithm to obtain the Mesh mesh of the point cloud to be processed, which can reduce the construction difficulty and ensure the accuracy of collision detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0055] Figure 1 A block diagram of an electronic device provided by an embodiment of the present invention;

[0056] Figure 2 One of the flow charts of a grid extraction method provided in an embodiment of the present invention;

[0057] Figure 3 A second flow chart of a grid extraction method provided by an embodiment of the present invention;

[0058] Figure 4 A schematic diagram of a mesh extracted using the rolling ball algorithm.

[0059] Figure 5 A third flow chart of a grid extraction method provided by an embodiment of the present invention;

[0060] Figure 6A fourth flow chart of a grid extraction method provided in an embodiment of the present invention;

[0061] Figure 7 A fifth flow chart of a grid extraction method provided in an embodiment of the present invention;

[0062] Figure 8 Schematic diagram of voxel grid based on grid extraction using octree method;

[0063] Figure 9 A schematic structural diagram of a grid extraction device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0064] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0065] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention.

[0066] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not require further definition or explanation in subsequent drawings.

[0067] In the description of the present invention, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the accompanying drawings, or is the orientation or position relationship in which the product of the invention is usually placed when in use. It is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it should not be understood as a limitation on the present invention.

[0068] In addition, the terms "first", "second", etc., if used, are merely used to distinguish and describe, and should not be understood as indicating or implying relative importance.

[0069] It should also be noted that, in the description of this application, unless otherwise expressly specified or limited, the terms "disposed," "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to internal connections between two components. Those skilled in the art will understand the specific meanings of the above terms in this application based on the specific circumstances.

[0070] After extensive research, the inventors found that the existing technology has objective shortcomings: the shortcomings of bounding volume detection are mainly reflected in three aspects: the convex hull algorithm uses the minimum convex polyhedron approximation, which makes it difficult to accurately restore the shape of complex objects; although the bounding sphere hierarchy balances efficiency and accuracy through the sphere tree structure, the geometric characteristics of the bounding sphere itself lead to limited fit, which may generate redundant space and affect detection accuracy; the bounding box (such as AABB / OBB) uses regular shapes to simplify calculations, resulting in a large amount of invalid space inside the bounding box, which may misjudge irrelevant objects as potential collision objects, increasing the computational burden of subsequent precise detection; the convex bounding box cannot fit the geometric features of concave objects. When processing objects with concave structures, it is necessary to rely on multi-layer nested bounding boxes or complex combination structures, which significantly increases the computational complexity.

[0071] The main disadvantages of spatial segmentation and voxelization methods include: resolution sensitivity leading to loss of details or a sharp increase in memory consumption, and the discretization loss of geometric features (such as edges and holes) during the voxelization process, which requires a trade-off between accuracy and computational resources.

[0072] The main disadvantages of surface reconstruction and mesh generation methods include: Mesh generation (such as Poisson reconstruction) requires high computing resources and is sensitive to point cloud noise, which can easily lead to incomplete meshes; Poisson reconstruction is suitable for complex unstructured surface modeling, and constructing octrees and solving the Poisson equation are very time-consuming, and normal vector calculation errors will directly affect the accuracy of the Poisson equation solution; CAD model conversion relies on parametric fitting, which may lose original point cloud details (such as surface features or irregular structures), resulting in deviations in collision analysis in engineering simulations, and the cost of model updates in dynamic scenarios is high.

[0073] The main disadvantages of machine learning-assisted methods include: feature extraction and deep learning rely on large amounts of labeled data for training, and the model is sensitive to interference such as noise and occlusion in complex scenarios, which may lead to misjudgment of collision risks.

[0074] Common shortcomings of collision detection methods include: high computational resource consumption in high-density point clouds or complex scenes (e.g., voxel grids and Poisson reconstruction); limited detection accuracy due to loss of geometric features (e.g., discretization, approximate fitting, or projective simplification); environmental interference such as sensor noise, occlusion, and illumination changes that can easily lead to misjudgments; and significant data dependency (e.g., machine learning requires large amounts of labeled data and traditional methods are sensitive to point cloud quality), which limits generalization in complex scenes. Furthermore, bounding volume methods suffer from the inherent contradiction between spatial redundancy and poor concave shape adaptation, while voxelization methods face a trade-off between resolution and memory.

[0075] In view of the discovery of the above problems, the present embodiment provides a mesh extraction method, device, electronic device and storage medium, which can determine the point cloud to be processed, normalize the point cloud to be processed to obtain a first point cloud to be processed, calculate the average point cloud density of the first point cloud to be processed, calculate the normal of each point in the first point cloud to be processed, and extract the mesh based on the average point cloud density and the normal of each point through the rolling ball algorithm to obtain the Mesh mesh of the point cloud to be processed. It can reduce the construction difficulty while ensuring the accuracy of collision detection. The solution provided by this embodiment is described in detail below.

[0076] This embodiment provides an electronic device capable of extracting a grid. In one possible implementation, the electronic device may be a user terminal, such as, but not limited to, a server, a smartphone, a personal computer (PC), a tablet computer, a personal digital assistant (PDA), a mobile internet device (MID), etc.

[0077] Please refer to Figure 1 , Figure 1 The electronic device 100 provided in the embodiment of the present application is shown in FIG. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0078] The electronic device 100 includes a grid extraction device 110 , a memory 120 , and a processor 130 .

[0079] The memory 120 and processor 130 are electrically connected to each other, directly or indirectly, to enable data transmission or interaction. For example, these components may be electrically connected via one or more communication buses or signal lines. The grid extraction device 110 includes at least one software functional module that can be stored in the memory 120 in the form of software or firmware or embedded in the operating system (OS) of the electronic device 100. The processor 130 is configured to execute the executable modules stored in the memory 120, such as the software functional modules and computer programs included in the grid extraction device 110.

[0080] The memory 120 may be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. The memory 120 is used to store a program, and the processor 130 executes the program after receiving an execution instruction.

[0081] Please refer to Figure 2 , Figure 2 For application Figure 1 A flowchart of a grid extraction method for an electronic device 100 is shown, and the method including each step is described in detail below.

[0082] S201: Determine the point cloud to be processed.

[0083] S202: Normalize the point cloud to be processed to obtain a first point cloud to be processed.

[0084] S203: Calculate the average density of the first point cloud to be processed.

[0085] S204: Calculate the normal of each point in the first point cloud to be processed.

[0086] S205: Based on the average density of the point cloud and the normal of each point, mesh extraction is performed using a rolling ball algorithm to obtain a Mesh of the point cloud to be processed.

[0087] Normalizing a point cloud to be processed means adjusting its coordinate range to a uniform interval. Point cloud data from different sources may have different scale ranges. Normalization can bring all points in the point cloud to a standard scale for easier comparison and analysis.

[0088] Many point cloud processing algorithms are sensitive to the scale of the data. After normalization, the performance and efficiency of the algorithm are usually improved.

[0089] When performing mathematical operations (such as matrix operations, distance calculations, etc.), normalized data can avoid numerical overflow or precision problems.

[0090] There are many ways to normalize the point cloud to be processed, such as range normalization. Range normalization is to scale each coordinate component of the point cloud to be processed to a specified interval, usually [0, 1].

[0091] The formula for range normalization is:

[0092]

[0093] Where x is the original coordinate value, x′ is the normalized coordinate value, min(x) and max(x) are the minimum and maximum values ​​of the coordinate component, respectively.

[0094] For each coordinate component (such as x, y, z) in the point cloud to be processed, calculate its minimum and maximum values ​​respectively, and use the above formula to normalize the coordinates of each point.

[0095] Another way to normalize the point cloud is:

[0096] Mean normalization,mean normalization is to subtract the mean of each coordinate component of the point cloud data to be processed, and then divide it by its range, which is composed of the maximum and minimum values.

[0097] The formula for mean normalization is:

[0098] Where mean(x) is the mean of the coordinate component.

[0099] Another way to normalize the point cloud is to use Z-Score normalization. Z-Score normalization is to subtract the mean of each coordinate component of the point cloud data and then divide it by its standard deviation.

[0100] The formula for Z-Score standardization is:

[0101]

[0102] where std(x) is the standard deviation of the coordinate component.

[0103] The average density of the first point cloud to be processed and the normal of each point in the first point cloud to be processed are calculated. The average density of the point cloud is used as the radius of sphere surface reconstruction, and the Mesh grid of the point cloud to be processed is obtained by the rolling ball algorithm.

[0104] Another way to determine the grid of the point cloud to be processed is to extract the grid using an octree algorithm based on the average density of the point cloud to obtain a voxel grid of the point cloud to be processed.

[0105] Based on the average density of the point cloud and the normal of each point, the mesh is extracted by the rolling ball algorithm to obtain the Mesh mesh of the point cloud to be processed. There are many ways to implement it. In one implementation, for example Figure 3 As shown, the following steps are included:

[0106] S301: Constructing multiple initial seed triangles based on each normal.

[0107] Among them, the normal directions of the points contained in the initial seed triangle are consistent.

[0108] S302: Constructing a first spherical surface based on the initial seed triangles and the average density of the point cloud.

[0109] S303: Obtain non-popular edges and non-popular points in the first spherical surface.

[0110] S304: Delete non-popular edges and non-popular points from the first sphere to obtain a Mesh of the point cloud to be processed.

[0111] The initial seed triangle must have three points with consistent normals to ensure the ball extends along the surface. For example, by comparing the angles between the three-point normals and the line connecting the center of the ball, candidate points with consistent normals can be selected to prevent the ball from falling into internal cavities.

[0112] The rolling ball algorithm is a commonly used point cloud surface reconstruction algorithm that uses the average density of the point cloud as the basis. Its basic principle is to let the ball roll on the surface. Once the surface of the ball touches three points, a triangular mesh is generated until the end, and the first spherical surface is obtained.

[0113] Delete the non-popular edges and non-popular points in the first sphere to obtain the Mesh of the point cloud to be processed.

[0114] Non-manifold edges are edges shared by three or more faces (e.g., two cubes share an edge to form a "T" structure). In the rolling ball algorithm, this happens when multiple faces incorrectly share the same edge when expanding in different directions, or when the mesh is topologically inconsistent during stitching.

[0115] Non-popular edges will destroy the manifold properties of the mesh, leading to calculation errors in physical simulation and 3D printing.

[0116] Non-manifold points are points where a single vertex connects multiple non-adjacent faces or edges (e.g., faces share only a single vertex, not an edge). This can occur when the mesh is stitched but vertices are not merged correctly, or when localized topological breaks are not repaired.

[0117] Non-popular points cause the mesh to be unable to be closed, and lighting rendering anomalies (such as inconsistent normal directions), which affects the machinability of the model.

[0118] A method for determining non-popular edges and non-popular points in the first sphere may be to call the DetectNonManifoldEdges method in the vcg::tri::Clean class in the Vcglib library to detect non-popular points and non-manifold edges.

[0119] After deleting the non-popular edges and non-popular points from the first sphere, the reconstructed Mesh is obtained and the open source library VcgLib or VTK library is used to realize the Mesh visualization. The visualized Mesh is obtained and it is judged whether the visualized Mesh meets the requirements. If the visualized Mesh meets the requirements, the point cloud unit is converted (mm to m) and exported as a Mesh in .obj format, such as Figure 4 As shown in FIG, it is a schematic diagram of converting the first point cloud to be processed into a Mesh grid.

[0120] The mesh generation in this application uses the rolling ball algorithm to generate Mesh meshes, which also supports accurate collision testing; the rolling ball algorithm has a fast reconstruction speed for regular geometric bodies (such as bearings and ball valves in industrial parts), low algorithm complexity, and only requires fitting the coordinates of the sphere center and radius parameters.

[0121] The nearest neighbor statistics method is used to calculate the average density of the point cloud. The average density of the point cloud is used as the initial radius. The rolling ball algorithm is executed to visualize the mesh and adjust the model resolution. This means that the average density of the point cloud is expanded or reduced. This adjustment can stabilize the mesh model resolution and prevent resolution sensitivity from causing loss of details or a sharp increase in memory consumption.

[0122] There are many ways to determine whether the visual mesh meets the requirements. In one implementation, Figure 5 As shown, the following steps are included:

[0123] S401: Determine the mesh resolution of the point cloud to be processed.

[0124] S402: Compare the grid resolution with the preset grid resolution.

[0125] S403: When the grid resolution is greater than the preset grid resolution, the average density of the point cloud is adjusted.

[0126] S404: Obtaining a Mesh of the point cloud to be processed based on the adjusted average density of the point cloud.

[0127] It should be noted that the preset grid resolution is set according to the actual project.

[0128] For the Mesh grid extracted by the rolling ball algorithm, when the grid resolution is greater than the preset grid resolution, it indicates that the visualized Mesh grid does not meet the requirements, and it is determined that the average density of the point cloud needs to be adjusted. The adjustment method can be to adjust the average density of the point cloud to 2n times the average density of the original point cloud, and reconstruct the ball surface based on the adjusted average density of the point cloud to obtain the Mesh grid of the point cloud to be processed.

[0129] For the voxel grid extracted by the octree algorithm, when the grid resolution is greater than the preset grid resolution, it indicates that the visual voxel grid does not meet the requirements, and it is determined that the average density of the point cloud needs to be adjusted. The adjustment method can be to adjust the average density of the point cloud to 2n times the average density of the original point cloud, and reconstruct the spherical surface based on the adjusted average density of the point cloud to obtain the voxel grid of the point cloud to be processed.

[0130] There are many ways to calculate the average density of the first point cloud to be processed. In one implementation, for example, Figure 6 As shown, the following steps are included:

[0131] S501: For each point in the first point cloud to be processed, calculate the distances between the point and other points in the first point cloud to be processed except the point.

[0132] S502: Obtain the minimum distance from each distance.

[0133] S503: Calculate the average value of each minimum distance.

[0134] S504: Calculate the average point cloud density of the first to-be-processed point cloud based on the average value.

[0135] Calculate the nearest neighbor distance of each point in the first point cloud to be processed: For each point in the first point cloud to be processed, find the distance to the nearest point other than the point in the first point cloud to be processed, recorded as di.

[0136] Calculate the average of all di, denoted as d`:

[0137]

[0138] Wherein, N is the total number of points in the first point cloud to be processed.

[0139] According to the derivation of the three-dimensional Poisson point process, the average density r can be expressed as Where Γ(4 / 3)≈0.89298 is the gamma function value. After substituting the gamma function value, the formula for calculating the average density of the point cloud is obtained:

[0140] The calculated point cloud average density is mainly used for subsequent Mesh resolution. The mesh resolution setting is related to the current point cloud average density. To prevent the resolution from being too high or too low, which may cause memory explosion, the default resolution is the point cloud average density. The point cloud average density can be adjusted manually later.

[0141] Adapt the rolling ball radius of the rolling ball algorithm to the average density of the point cloud: the rolling ball radius can be set to be slightly larger than the average density of the point cloud to ensure that the rolling ball can stably form a triangle between the three points.

[0142] The method for calculating the normal of each point in the first point cloud to be processed can be: calculating the center of mass of each point, and centralizing each center of mass, calculating the covariance matrix based on the center of mass after centralization, performing eigenvalue decomposition based on the covariance matrix, and obtaining the eigenvector corresponding to the minimum eigenvalue from the decomposed eigenvalues ​​as the normal.

[0143] Specifically, the center of mass can be calculated by the following formula: where p i points in the neighborhood, and k is the number of neighborhood points.

[0144] The way to center the centroid is:

[0145] The covariance matrix satisfies the following formula:

[0146]

[0147] The eigenvalue satisfies the following formula: C = E∧E T , where ∧ is the eigenvalue diagonal matrix and E is the eigenvector matrix.

[0148] Take the eigenvector corresponding to the minimum eigenvalue as the normal n.

[0149] Normals are used to adjust the radius and direction of the rolling ball in the rolling ball algorithm, ensuring that the triangle mesh is properly connected.

[0150] The consistency of normal direction can help determine the local density distribution of the point cloud. For example, if the normal direction changes suddenly in a local area (such as an edge or noise point), it may be necessary to dynamically adjust the radius to avoid abnormal points (such as reducing the radius to avoid misconnection).

[0151] There are many ways to extract the grid based on the average density of the point cloud by using the octree algorithm to obtain the voxel grid of the point cloud to be processed. In one implementation, Figure 7 As shown, the following steps are included:

[0152] S601: Constructing an octree of a first point cloud to be processed based on the average density of the point cloud.

[0153] S602: Determine the vertices of each voxel in the octree.

[0154] S603: Generate a triangle mesh index of each voxel based on the vertices of each voxel.

[0155] S604: Construct a voxel grid of the point cloud to be processed based on each index.

[0156] The method of constructing a voxel grid of a point cloud to be processed based on an octree includes: determining a point cloud to be processed, normalizing the point cloud to be processed to obtain a first point cloud to be processed, calculating an average point cloud density of the first point cloud to be processed, constructing an octree based on the average point cloud density and the first point cloud to be processed, determining the vertices of each voxel in the octree, generating a triangular grid index of the voxel based on the vertices of each voxel, constructing a voxel grid of the point cloud to be processed based on each index, visualizing the voxel grid to obtain a visualized voxel grid, judging whether the visualized voxel grid meets the requirements, and if so, performing unit conversion on the visualized voxel grid and exporting it to .obj format as the voxel grid of the point cloud to be processed.

[0157] If the visual voxel grid does not meet the requirements, the average density of the point cloud is adjusted. The adjustment method can be to adjust the average density of the point cloud to 2n times the average density of the original point cloud, reconstruct the octree based on the adjusted average density of the point cloud, and obtain the Mesh grid of the point cloud to be processed based on the reconstructed octree again.

[0158] An octree is a tree-like data structure used for 3D spatial data management. Each node represents a cubic volume (called a voxel), which recursively divides the space into eight child nodes to form a hierarchical spatial partition. The average density of the point cloud can control the octree voxel granularity and spatial partitioning accuracy.

[0159] The method of constructing an octree based on the average density of the point cloud and the first point cloud to be processed can be: determining the voxel resolution of the octree, that is, the average density of the point cloud of the first point cloud to be processed. If the average density of the point cloud causes the points in the point cloud to be too dense, downsampling can be performed through the VoxelGrid filter of PCL to match the voxel size of the octree with the density to avoid redundant calculations.

[0160] An octree constructed based on the average density of the point cloud and the first point cloud to be processed can be constructed by determining an initial root node from the first point cloud to be processed and determining the spatial extent of the root node based on the point cloud bounding box. In a tree data structure, the root node is the topmost node of the tree and the entry point of the entire data structure. All other nodes are derived from the root node.

[0161] Based on the average point cloud density, the current node's cube space is split into eight sub-cubes (eight voxels) along the midpoints of the x / y / z axes. Points are assigned to intersecting voxels based on the point cloud distribution. If the number of points within a voxel exceeds a threshold, the octree is constructed by recursively partitioning until the preset maximum depth is reached, the voxel edge length is less than or equal to the average point cloud density, or the number of points within the node falls below the threshold.

[0162] Determine the vertices of each voxel in the octree. When the voxel is a cube or a cuboid, determine that each voxel contains 8 vertices. Based on each vertex, 12 triangular faces can be formed. One triangular face uses three vertices, so 36 triangular mesh indexes can be obtained. Since three of the 36 triangular meshes share one vertex, in order to reduce the storage of duplicate vertices, 8 triangular mesh indexes are generated. Based on each index and the triangular mesh corresponding to the index, the voxel mesh of the point cloud to be processed is constructed, as shown in the following example. Figure 8 , which is a schematic diagram of a voxel grid of a point cloud to be processed constructed based on the octree algorithm.

[0163] The voxel grid can be visualized by determining the boundaries of each voxel in the octree. This can be done by traversing all voxels in the octree and obtaining the minimum corner coordinates and side length of each voxel. For each voxel, six rectangular faces are generated, with vertex coordinates of (x, y, z), (x + Δ, y, z), ... (Δ is the voxel side length). Ultimately, the boundaries of all voxels in the octree are obtained, and the voxel grid is visualized based on these boundaries.

[0164] This application is based on the average density of point cloud, and uses the octree algorithm to extract the mesh to obtain the mesh of the point cloud to be processed. The average density of the point cloud is calculated using the nearest neighbor statistics method, and the average density of the point cloud is used as the initial mesh resolution. The octree algorithm is executed to visualize the mesh and adjust the model resolution (enlarge or reduce the original resolution by r*2 n multiples), which can be adjusted to stably control the mesh model resolution (preventing resolution sensitivity from causing loss of details or a sharp increase in memory consumption).

[0165] After the octree is built, eight vertices and cube face indices of the cube mesh are generated. Shared vertex indices are used, with 12 triangular faces per cube and a total of 8 indices. This reduces the storage of duplicate vertices and saves memory space.

[0166] Please refer to Figure 9 The embodiment of the present application also provides an application Figure 1 The grid extraction device 110 of the electronic device 100 includes:

[0167] A determination module 111 is used to determine a point cloud to be processed;

[0168] A normalization module 112 is configured to normalize the to-be-processed point cloud to obtain a first to-be-processed point cloud;

[0169] The calculation module 113 is configured to calculate the average density of the first point cloud to be processed; and calculate the normal of each point in the first point cloud to be processed;

[0170] The mesh extraction module 114 is configured to perform mesh extraction based on the average density of the point cloud and the normal of each point using a rolling ball algorithm to obtain a Mesh of the point cloud to be processed.

[0171] The present application further provides an electronic device 100, which includes a processor 130 and a memory 120. The memory 120 stores computer-executable instructions, which, when executed by the processor 130, implement the grid extraction method.

[0172] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by the processor 130, the grid extraction method is implemented.

[0173] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0174] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part. If the function is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0175] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0176] The above descriptions are merely examples of various embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any modifications or substitutions that can be readily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included within the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A grid extraction method, characterized in that: The method comprises: Determine the point cloud to be processed; Normalizing the to-be-processed point cloud to obtain a first to-be-processed point cloud; Calculating the average density of the first point cloud to be processed; Calculating the normal of each point in the first to-be-processed point cloud; Based on the average density of the point cloud and the normal of each point, a mesh is extracted using a rolling ball algorithm to obtain a Mesh of the point cloud to be processed.

2. The method according to claim 1, characterized in that The method further comprises: Based on the average density of the point cloud, grid extraction is performed using an octree algorithm to obtain a voxel grid of the point cloud to be processed.

3. The method according to claim 1, characterized in that The step of extracting a mesh based on the average density of the point cloud and the normal of each point by a rolling ball algorithm to obtain a Mesh of the point cloud to be processed includes: Based on the normals, construct a plurality of initial seed triangles, wherein the normal directions of the points included in the initial seed triangles are consistent; constructing a first spherical surface based on each of the initial seed triangles and the average density of the point cloud; Obtaining non-popular edges and non-popular points in the first spherical surface; The non-popular edges and the non-popular points are deleted from the first spherical surface to obtain a Mesh grid of the point cloud to be processed.

4. The method according to claim 1, wherein The method further comprises: Determine the mesh resolution of the point cloud to be processed; comparing the grid resolution with a preset grid resolution; When the grid resolution is greater than the preset grid resolution, adjusting the average density of the point cloud; A Mesh grid of the point cloud to be processed is obtained based on the adjusted average density of the point cloud.

5. The method according to claim 1, wherein The step of calculating the average density of the first point cloud to be processed includes: For each point in the first point cloud to be processed, calculating distances between the point and other points in the first point cloud to be processed except the point; Obtaining a minimum distance from each of the distances; Calculating an average of the minimum distances; Based on the average value, the point cloud average density of the first to-be-processed point cloud is calculated.

6. The method according to claim 5, characterized in that The average density of the point cloud is calculated using the following formula: Among them, r represents the average density of the point cloud, Γ is the value of the gamma function, and d is the average value of each minimum distance.

7. The method according to claim 2, characterized in that The step of performing grid extraction based on the average density of the point cloud by an octree algorithm to obtain a voxel grid of the point cloud to be processed includes: Constructing an octree of the first to-be-processed point cloud based on the average density of the point cloud; Determining the vertices of each voxel in the octree; Generate a triangle mesh index of each voxel based on the vertices of each voxel; A voxel grid of the point cloud to be processed is constructed based on each of the indexes.

8. A grid extraction device, characterized in that: The device comprises: A determination module, used to determine the point cloud to be processed; a normalization module, configured to normalize the point cloud to be processed to obtain a first point cloud to be processed; a calculation module, configured to calculate an average density of the first point cloud to be processed; and calculate a normal of each point in the first point cloud to be processed; The mesh extraction module is used to extract the mesh based on the average density of the point cloud and the normal of each point through the rolling ball algorithm to obtain the Mesh mesh of the point cloud to be processed.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method according to any one of claims 1 to 7 when executing the computer program.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images