Point cloud plane segmentation method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a point cloud plane segmentation method and device, electronic equipment and a storage medium, and the method comprises the steps: carrying out the grid division of a three-dimensional space corresponding to point cloud data through employing a phase distribution-based grid division mode, and obtaining a plurality of grids; performing plane fitting on points in each grid in the plurality of grids in a parallel processing mode to obtain a plurality of plane seeds; and through a parallel processing mode, performing neighborhood plane growth on the plurality of plane seeds to obtain a plurality of planes corresponding to the point cloud data. According to the embodiment of the invention, a plurality of planes corresponding to the point cloud data can be quickly and accurately determined, so that the point cloud plane segmentation efficiency is improved.

Description

Technical Field

[0001] The present disclosure relates to digital signal processing technologies, and in particular, to a method, apparatus, electronic device, and storage medium for segmenting a point cloud plane. Background Art

[0002] The point cloud plane segmentation method can be applied to Virtual Reality (VR), enabling the VR acquisition data to be processed more beautifully.

[0003] In related technologies, the neighborhood growth method is used to segment the point cloud plane. When segmenting the point cloud plane by the neighborhood growth method, any vertex of the point cloud is used as the initial plane center, and the normal vector of this vertex is used as the initial plane normal vector. Starting from this vertex, the plane range is gradually expanded until it cannot be expanded; then another vertex is selected arbitrarily, and so on until the segmentation of the point cloud plane is completed. This method for segmenting the point cloud plane assumes that the point cloud is distributed on a smooth surface. However, the actual VR acquisition scenario is complex and variable, and the assumption of a smooth surface does not conform to the real environment, resulting in varying degrees of degradation for actual data.

[0004] For non-smooth surface scenarios, the selection of the initial vertex will bring uncertainty to the segmentation method, and there will be perturbations and noises in the normal vector of the initial vertex itself. For incremental and greedy algorithms, the initial state will affect the final plane segmentation result to a certain extent, resulting in incorrect results. Summary of the Invention

[0005] Embodiments of the present disclosure provide a method, apparatus, electronic device, and storage medium for segmenting a point cloud plane to solve the above problems.

[0006] In a first aspect of embodiments of the present disclosure, a method for segmenting a point cloud plane is provided, including:

[0007] Performing grid division on the three-dimensional space corresponding to the point cloud data by using a grid division method based on phase distribution to obtain a plurality of grids;

[0008] For grids with the same phase among the plurality of grids, performing parallel processing to perform plane fitting on the points within each grid among the plurality of grids to obtain a plurality of plane seeds;

[0009] Performing neighborhood plane growth on the plurality of plane seeds by means of parallel processing to obtain a plurality of planes corresponding to the point cloud data.

[0010] In some embodiments of the present disclosure, the performing grid division on the three-dimensional space corresponding to the point cloud data by using a grid division method based on phase distribution to obtain a plurality of grids includes:

[0011] Determine anchor points based on the positions of the points in the point cloud data;

[0012] Based on the anchor points, perform multi-scale grid division on the three-dimensional space corresponding to the point cloud data using the grid division method to obtain the multiple grids.

[0013] In some embodiments of the present disclosure, after performing multi-scale grid division on the three-dimensional space corresponding to the point cloud data using the grid division method based on the anchor points to obtain the multiple grids, it further includes:

[0014] Determine the grid indices of the points in the point cloud data based on the spatial coordinates of the anchor points and the spatial coordinates of the points in the point cloud data;

[0015] Perform hash calculation based on the grid indices of the points in the point cloud data to obtain the hash values of the points in the point cloud data;

[0016] Determine the phase numbers of the points in the point cloud data based on the grid indices of the points in the point cloud data;

[0017] Use a hash list to store the hash values of the points in the point cloud data and the phase numbers of the points in the point cloud data.

[0018] In some embodiments of the present disclosure, the step of performing plane fitting on the points in each grid of the multiple grids to obtain the plane seeds of each grid includes:

[0019] Perform the following steps for each grid of the multiple grids:

[0020] Randomly select a point from the current grid, perform plane fitting on the currently selected point and the points within the spherical neighborhood, obtain the plane fitting result and increment the plane fitting count by one, where the radius of the spherical neighborhood is equal to the preset grid side length, and the plane fitting result includes the plane seeds generated when the plane fitting is successful;

[0021] Loop through the step of randomly selecting a point from the current grid, performing plane fitting on the currently selected point and the points within the spherical neighborhood, obtaining the plane fitting result and incrementing the plane fitting count by one until the plane fitting count of the current grid reaches the preset count threshold to obtain all the plane seeds of the current grid.

[0022] In some embodiments of the present disclosure, the step of performing neighborhood plane growth on the multiple plane seeds to obtain the multiple planes corresponding to the point cloud data includes:

[0023] Perform the following steps for each plane seed of the multiple plane seeds in a loop until the multiple planes corresponding to the point cloud data are obtained:

[0024] Select a first target point from the current plane seeds, and obtain each point within the neighborhood of the first target point, where the first target point and each point within the neighborhood of the first target point belong to the points in the point cloud data;

[0025] Based on the normal vector angle and distance between each point within the neighborhood of the first target point and the current plane seeds, as well as the number of points within the neighborhood of the first target point, determine whether each point within the neighborhood of the first target point joins the current plane seeds;

[0026] In response to at least one point being added to the current plane seeds, update the current plane seeds.

[0027] In some embodiments of the present disclosure, the determining whether each point within the neighborhood of the first target point joins the current plane seeds based on the normal vector angle and distance between each point within the neighborhood of the first target point and the current plane seeds, as well as the number of points within the neighborhood of the first target point, includes:

[0028] Select a second target point from within the neighborhood of the first target point;

[0029] In response to the normal vector angle between the second target point and the current plane seeds being less than a preset normal vector angle threshold, the distance between the second target point and the current plane seeds being less than a preset distance threshold, and the number of points within the spherical neighborhood of the first target point being greater than a preset number threshold, add the second target point to the current plane seeds.

[0030] In some embodiments of the present disclosure, during the process of neighborhood plane growth for the multiple plane seeds, it further includes:

[0031] Estimate the plane equation based on the positions of the points within each plane seed during the neighborhood plane growth process to obtain the plane equations of the respective plane seeds;

[0032] Remove outliers from the respective plane seeds based on the plane equations of the respective plane seeds to obtain at least one outlier.

[0033] In a second aspect of the embodiments of the present disclosure, there is provided a point cloud plane segmentation device, including:

[0034] A grid division module, configured to perform grid division on the three-dimensional space corresponding to the point cloud data by using a grid division method based on phase distribution to obtain multiple grids;

[0035] A plane fitting module, configured to perform parallel processing on the grids with the same phase among the multiple grids, and perform plane fitting on the points within each grid among the multiple grids to obtain multiple plane seeds;

[0036] A neighborhood plane growth module, configured to perform neighborhood plane growth on the multiple plane seeds in a parallel processing manner to obtain multiple planes corresponding to the point cloud data.

[0037] In some embodiments of the present disclosure, the mesh division module is configured to determine anchor points based on the positions of the points in the point cloud data; the mesh division module is further configured to perform multi-scale mesh division on the three-dimensional space corresponding to the point cloud data based on the anchor points by using the mesh division method to obtain the multiple meshes.

[0038] In some embodiments of the present disclosure, it further includes:

[0039] A mesh index determination module, configured to determine the mesh indices of the points in the point cloud data based on the spatial coordinates of the anchor points and the spatial coordinates of the points in the point cloud data;

[0040] A hash calculation module, configured to perform hash calculation based on the mesh indices of the points in the point cloud data to obtain the hash values of the points in the point cloud data;

[0041] A phase number determination module, configured to determine the phase numbers of the points in the point cloud data based on the mesh indices of the points in the point cloud data;

[0042] A storage module, configured to store the hash values of the points in the point cloud data and the phase numbers of the points in the point cloud data by using a hash list.

[0043] In some embodiments of the present disclosure, the plane fitting module is configured to perform the following steps for each of the multiple meshes:

[0044] Randomly select a point from the current mesh, perform plane fitting on the currently selected point and the points within the spherical neighborhood, obtain a plane fitting result and increment the plane fitting count by one, where the radius of the spherical neighborhood is equal to the preset mesh side length, and the plane fitting result includes the plane seeds generated when the plane fitting is successful;

[0045] Loop to execute the step of randomly selecting a point from the current mesh, performing plane fitting on the currently selected point and the points within the spherical neighborhood, obtaining a plane fitting result and incrementing the plane fitting count by one until the plane fitting count of the current mesh reaches a preset count threshold to obtain all the plane seeds of the current mesh.

[0046] In some embodiments of the present disclosure, the neighborhood plane growth module is configured to loop to execute the following steps for each of the multiple plane seeds until multiple planes corresponding to the point cloud data are obtained:

[0047] Select a first target point from the current planar seed, and obtain each point within the neighborhood of the first target point, where the first target point and each point within the neighborhood of the first target point belong to the points in the point cloud data;

[0048] Based on the normal vector angle and distance between each point within the neighborhood of the first target point and the current planar seed, as well as the number of points within the spherical neighborhood of the first target point, determine whether each point within the neighborhood of the first target point is added to the current planar seed;

[0049] In response to at least one point being added to the current planar seed, update the current planar seed.

[0050] In some embodiments of the present disclosure, the neighborhood planar growth module is configured to select a second target point from within the neighborhood of the first target point; the neighborhood planar growth module is further configured to, in response to the normal vector angle between the second target point and the current planar seed being less than a preset normal vector angle threshold, and the distance between the second target point and the current planar seed being less than a preset distance threshold, and the number of points within the spherical neighborhood of the first target point being greater than a preset number threshold, add the second target point to the current planar seed.

[0051] In some embodiments of the present disclosure, the neighborhood planar growth module is further configured to estimate a plane equation based on the positions of the points within each planar seed during the neighborhood planar growth process when performing neighborhood planar growth on the multiple planar seeds, to obtain the plane equations of the respective planar seeds; the neighborhood planar growth module is further configured to perform outlier rejection on the respective planar seeds based on the plane equations of the respective planar seeds to obtain at least one outlier.

[0052] A third aspect of the embodiments of the present disclosure provides an electronic device, including:

[0053] A memory for storing a computer program product;

[0054] A processor for executing the computer program product stored in the memory, and when the computer program product is executed, implementing the above-mentioned method for segmenting a point cloud plane.

[0055] A fourth aspect of the embodiments of the present disclosure provides a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, implementing the above-mentioned method for segmenting a point cloud plane.

[0056] A fifth aspect of the embodiments of the present disclosure provides a computer program product, including computer program instructions, and when the computer program instructions are run by a processor, causing the processor to execute the above-mentioned method for segmenting a point cloud plane.

[0057] The method, device, electronic device, and storage medium for segmenting a point cloud plane according to an embodiment of the present disclosure can perform grid division on the three-dimensional space corresponding to the point cloud data by using a grid division method based on phase distribution to obtain multiple groups of grids with phase consistency, because the phase distribution method can make different spaces have different phases. Multiple plane seeds can be obtained by parallel processing of grids with the same phase in multiple groups of grids with phase consistency. Furthermore, the neighborhood plane growth can be performed according to the spatial position relationship between each plane seed and the points in the neighborhood by using a parallel processing method, so that multiple planes corresponding to the point cloud data can be obtained quickly and accurately, thereby improving the efficiency of segmenting the point cloud plane.

[0058] The technical solutions of the present disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The drawings forming a part of the specification illustrate embodiments of the present disclosure and, together with the description, are used to explain the principles of the present disclosure.

[0060] Referring to the accompanying drawings, the present disclosure can be more clearly understood according to the following detailed description, where:

[0061] Figure 1 is a schematic flowchart of a method for segmenting a point cloud plane according to an embodiment of the present disclosure;

[0062] Figure 2 is a schematic flowchart of step S1 according to an embodiment of the present disclosure;

[0063] Figure 3 is a schematic diagram of a partial flowchart after step S1-2 according to an embodiment of the present disclosure;

[0064] Figure 4 is a schematic diagram of the phase arrangement in a two-dimensional plane according to an example of the present disclosure;

[0065] Figure 5 is a schematic diagram of the corresponding relationship of a data storage list according to an example of the present disclosure;

[0066] Figure 6 is a schematic diagram of the corresponding relationship between the data storage list and the point cloud data according to an example of the present disclosure;

[0067] Figure 7 is a schematic flowchart of step S2 according to an embodiment of the present disclosure;

[0068] Figure 8 is a schematic flowchart of step S3 according to an embodiment of the present disclosure;

[0069] Figure 9 is a schematic flowchart of step S3-2 according to an embodiment of the present disclosure;

[0070] Figure 10 The structural block diagram of the point cloud plane segmentation device according to an embodiment of the present disclosure;

[0071] Figure 11 The structural block diagram of the point cloud plane segmentation device according to another embodiment of the present disclosure;

[0072] Figure 12 The structural block diagram of an electronic device according to an embodiment of the present disclosure. Detailed implementation manners

[0073] Now, various exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. It should be noted that: unless otherwise specifically stated, the relative arrangements of components and steps, numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present disclosure.

[0074] Those skilled in the art can understand that terms such as "first", "second", etc. in the embodiments of the present disclosure are only used to distinguish different steps, devices, or modules, etc., and do not represent any specific technical meaning, nor do they indicate an inevitable logical order between them.

[0075] It should also be understood that in the embodiments of the present disclosure, "a plurality of" may refer to two or more, and "at least one" may refer to one, two, or more.

[0076] It should also be understood that for any component, data, or structure mentioned in the embodiments of the present disclosure, without clear limitation or contrary indication in the context, it is generally understood as one or more.

[0077] In addition, the term "and / or" in the present disclosure is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in the present disclosure generally represents an "or" relationship between the associated objects before and after.

[0078] It should also be understood that the present disclosure emphasizes the differences between various embodiments. The same or similar parts can be referred to each other. For the sake of brevity, they will not be described in detail one by one.

[0079] The following description of at least one exemplary embodiment is actually merely illustrative and in no way limits the present disclosure or its application or use.

[0080] Technologies, methods, and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, the said technologies, methods, and devices should be regarded as part of the specification.

[0081] It should be noted that like reference numerals and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.

[0082] Embodiments of the present disclosure can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate together with many other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, servers, etc. include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, and so on.

[0083] Terminal devices, computer systems, servers and other electronic devices can be described in the general context of computer system-executable instructions (such as program modules) executed by a computer system. Generally, program modules may include routines, programs, target programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. The computer system / server can be implemented in a distributed cloud computing environment where tasks are executed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0084] It should be noted that the point cloud plane segmentation method and device in the embodiments of the present disclosure can both be applied to a distributed system. In the following embodiments, the steps clearly described to be processed in a parallel processing manner can greatly improve the data processing efficiency, and the remaining steps can also be processed in a parallel processing manner to improve the data processing efficiency.

[0085] Figure 1 This is a flowchart of the point cloud plane segmentation method in an embodiment of the present disclosure. As Figure 1 shown, the point cloud plane segmentation method includes:

[0086] S1: Using a grid division method based on phase distribution, divide the three-dimensional space corresponding to the point cloud data into grids to obtain a plurality of grids.

[0087] For the point cloud data to be subjected to plane segmentation, determine a grid division method based on phase distribution. According to the positions of the points in the three-dimensional space in the point cloud data, determine the position occupied by the point cloud data in the three-dimensional space, and then divide the position occupied by the point cloud data in the three-dimensional space into grids by using a grid division method based on phase distribution to obtain a plurality of grids.

[0088] In some examples of the present disclosure, the grid division method based on phase distribution divides the grid into 27 phases in a three-dimensional space. Among them, the distribution of the 27 phases is similar to the distribution of a 3x3x3 Rubik's Cube. That is, given the grid side length μ, each grid is a cube with side length μ, and the three-dimensional space is divided into grids with 3μ×3μ×3μ as the unit, so that the closest distance between two grids with the same phase is 2μ. By the grid division method based on phase distribution, it is helpful for subsequent steps to perform parallel processing on grids with the same phase, and thus helps to improve the efficiency of segmenting the point cloud plane.

[0089] It should be noted that dividing into 27 phases is only an example, and those skilled in the art can also divide into other numbers of phases and then make corresponding adjustments to the phase distribution. In addition, each grid is not limited to a cube structure, and can also be a cuboid structure or other structures.

[0090] S2: For the grids with the same phase among multiple grids, adopt a parallel processing method to perform plane fitting on the points in each grid among the multiple grids to obtain multiple plane seeds.

[0091] The grid division method based on phase distribution makes there be grids with the same phase among the multiple grids obtained by grid division. Therefore, a distributed system can be used to adopt a parallel processing method for the grids with the same phase among the multiple grids obtained by phase division, perform plane fitting on the points in all grids, obtain the plane seed corresponding to each grid, and thus obtain multiple plane seeds. Among them, the plane seed is the initial plane generated for the point cloud data in each grid and not expanded.

[0092] S3: Through a parallel processing method, perform neighborhood plane growth on multiple plane seeds to obtain multiple planes corresponding to the point cloud data.

[0093] A distributed system can be used to perform neighborhood growth in parallel for each plane seed to obtain multiple planes corresponding to the point cloud data.

[0094] In this embodiment, since the phase distribution method can make different spaces have different phases, the grid division method based on phase distribution is used to divide the three-dimensional space corresponding to the point cloud data into grids, and multiple groups of grids with phase consistency are obtained. Multiple plane seeds can be obtained by performing parallel processing on grids with the same phase among multiple groups of grids with phase consistency. Furthermore, a parallel processing method is adopted to perform neighborhood plane growth according to the spatial position relationship between each plane seed and the points in the neighborhood, and multiple planes corresponding to the point cloud data can be obtained quickly and accurately, thereby improving the efficiency of point cloud plane segmentation.

[0095] Figure 2 is a schematic flowchart of step S1 in an embodiment of the present disclosure. As Figure 2 shown, step S2 includes the following steps:

[0096] S1-1: Determine an anchor point based on the positions of the points in the point cloud data.

[0097] The center point of the space where the point cloud data is located or other preset position points can be determined as the anchor point. Among them, the center point of the space where the point cloud data is located can be determined by the following method: adding the three-dimensional space coordinates of each point in the point cloud data and then calculating the average value to obtain the three-dimensional space coordinates of the anchor point.

[0098] S1-2: Based on the anchor point, perform multi-scale grid division on the three-dimensional space corresponding to the point cloud data by using the above grid division method to obtain multiple grids.

[0099] Continuing with the example of step S1, when the grid side length μ takes different values, the three-dimensional space is divided into grids with 3μ×3μ×3μ as the unit based on the anchor point, so as to achieve multi-scale grid division, obtain the grid division results corresponding to different scales, and further obtain multiple grids for plane fitting through parallel processing.

[0100] The multi-scale grid division method in step S1-2 at least includes: performing grid division on the three-dimensional space corresponding to the point cloud data by using the above grid division method based on the anchor point and the first grid side length to obtain a first grid set; performing grid division on the three-dimensional space corresponding to the point cloud data by using the above grid division method based on the anchor point and the second grid side length to obtain a second grid set. Among them, the first grid side length and the second grid side length are different, and the multiple grids for plane fitting through parallel processing include the first grid set and the second grid set.

[0101] In this embodiment, since the plane seeds fitted by grids of different scales are different, multi-scale grid division is performed on the three-dimensional space corresponding to the point cloud data by using the grid division method according to the anchor point, so that multi-scale grids can be obtained, and further multi-scale plane seeds can be obtained, which helps to improve the adaptability of the plane seeds to complex environments.

[0102] Figure 3 is a schematic diagram of a part of the process after step S1-2 in an embodiment of the present disclosure. As Figure 3 shown, after step S1-2, the following steps are further included:

[0103] S1-3: Determine the grid index of each point in the point cloud data based on the spatial coordinates of the anchor point and the spatial coordinates of each point in the point cloud data.

[0104] For any point p in the point cloud datai =(x i , y i , z i ) of the grid index is calculated as follows:

[0105]

[0106] Where c i =(k i , l i , m i ) represents the grid index of p i , (x o , y o , z o ) represents the spatial coordinates of the anchor point, and μ represents the grid side length.

[0107] S1-4: Perform hash calculation based on the grid indices of each point in the point cloud data to obtain the hash values of each point in the point cloud data.

[0108] In order to represent a large amount of space with a small amount of memory, the embodiments of the present disclosure use spatial hashing to represent the grid data structure. For the grid (cell) index c i =(k i , l i , m i ), its hash value h is calculated as follows:

[0109] h=(k i ·p 1 ) xor (l i ·p 2 ) xor (m i ·p 3 )

[0110] Where p 1 , p 2 , p 3 are three preset prime numbers, for example, 73856093, 19349663, and 83492791 respectively.

[0111] S1-5: Determine the phase numbers of each point in the point cloud data based on the grid indices of each point in the point cloud data.

[0112] The data storage list stores the number of points and the starting point position under the same grid index. Each grid index c i has a phase number. In three-dimensional space, 27 phase numbers are set so that the distance between the grids of each phase and its nearest grid is 2μ, enabling parallel processing of all grids within a single phase without mutual influence.

[0113] Figure 4 It is a schematic diagram of the phase arrangement in a two-dimensional plane in an example of the present disclosure. As Figure 4 shown, 0-8 represent 9 phases. The points on the solid line in the solid-line graph (i.e., the graph similar to a rabbit) can represent the points of the point cloud data on the two-dimensional plane. Through Figure 6 the arrangement method of the 9 phases shown, the phase numbers of the points on the solid line can be obtained.

[0114] S1-6: Use a hash list to store the hash values of each point in the point cloud data and the phase numbers of each point in the point cloud data.

[0115] Use a secondary data structure to store a data storage list. The data storage list includes hash units, and the hash units store the indexes of the list, so as to improve cache hit and reduce memory application and release.

[0116] Figure 5 It is a schematic diagram of the corresponding relationship of the data storage list in an example of the present disclosure. As Figure 5 shown, m represents the number of hash values calculated according to the grid index, n represents the number of all occupied grids in the space, and k represents the number of points in a certain grid.

[0117] In addition, the data entities will be pre-parallel sorted so that the points located at the same grid index can be tightly arranged together.

[0118] Figure 6 It is a schematic diagram of the corresponding relationship between the data storage list and the point cloud data in an example of the present disclosure. As Figure 6 shown, the key represents the grid index; the quantity represents the number of points contained in the grid and points to a certain value in an arranged array. For example, the first row of the data storage list executes the grid 111 in the arranged point cloud data, and the second row of the data storage list executes the grid 387 in the arranged point cloud data.

[0119] Figure 7 It is a schematic flowchart of step S2 in an embodiment of the present disclosure. As Figure 7 shown, step S2 includes the following steps:

[0120] For the grids with the same phase among the multiple grids obtained by phase division, adopt a parallel processing method, and execute the following steps for each grid among the multiple grids:

[0121] S2-1: Randomly select a point from the current grid, perform plane fitting on the currently selected point and the points in the spherical neighborhood, obtain the plane fitting result and increment the plane fitting times by one.

[0122] Among them, the radius of the spherical neighborhood is equal to the preset grid side length. In this case, the generated plane seeds are not affected by noise. The plane fitting result includes: the plane seeds generated when the plane fitting is successful, and the points in the current grid that cannot be fitted into the plane seeds.

[0123] S2-2: Repeat step S2-1 until the number of plane fittings of the current grid reaches the preset number threshold, and all plane seeds of the current grid are obtained.

[0124] In this embodiment, for an object with a complex surface structure, since it is difficult to fit the point clouds in all grids of the object into the seed plane, a preset number threshold is set, and when the number of plane fittings of the current grid reaches the preset number threshold, the current grid is no longer fitted to the plane, thus avoiding consuming a large amount of time for plane fitting.

[0125] Figure 8 It is a schematic flowchart of step S3 in an embodiment of the present disclosure. As Figure 8 shown, through parallel processing, for each plane seed of multiple plane seeds, the following steps are cyclically executed until multiple planes corresponding to the point cloud data are obtained:

[0126] S3-1: Select a first target point from the current plane seed, and obtain the points in the neighborhood of the first target point. Among them, the first target point and the points in the neighborhood of the first target point all belong to the points in the point cloud data.

[0127] A point can be randomly selected from the current plane seed as the first target point. Taking the grid side length μ as the radius of the spherical neighborhood of the first target point, all points in the neighborhood of the first target point are obtained.

[0128] S3-2: Based on the normal vector angles and distances between the points in the neighborhood of the first target point and the current plane seed, and the number of points in the neighborhood of the first target point, determine whether the points in the neighborhood of the first target point are added to the current plane seed.

[0129] Calculate the normal vector angles between the points in the spherical neighborhood of the first target point and the current plane seed according to the normal vectors of the points in the spherical neighborhood of the first target point and the normal vector of the current plane seed. Calculate the distances between the points in the spherical neighborhood of the first target point and the current plane seed according to the spatial coordinates of the points in the spherical neighborhood of the first target point and the plane coordinates of the current plane seed. Determine which points in the point cloud data are located in the spherical neighborhood of the first target point, and then obtain the number of points in the spherical neighborhood of the first target point.

[0130] According to the set conditions regarding the normal vector angle, distance, and number of points when a point can be added to the planar seed. If the normal vector angle between a certain point within the spherical neighborhood of the first target point and the current planar seed, and the distance between this point and the current planar seed, as well as the number of points within the spherical neighborhood of the first target point satisfy the above set conditions, then this point is added to the current planar seed; otherwise, this point is not added to the current planar seed and is recorded.

[0131] S3-3: In response to at least one point being added to the current planar seed, update the current planar seed.

[0132] After at least one point within the spherical neighborhood of the first target point is added to the current planar seed, update the points included in the current planar seed.

[0133] In this embodiment, since the normal vector angle between a point and the planar seed can represent the degree of fit between the point and the planar seed in the vector direction, and the phenomenon of planar degeneration is likely to occur on a relatively smooth and curved surface, and the density of points within the neighborhood helps to distinguish non-connected planar regions in space. Therefore, when performing neighborhood planar growth on the planar seed, it is determined whether to add a point to the corresponding planar seed based on the normal vector angle between the point and the planar seed, the distance between the point and the planar seed, and the number of points within the neighborhood, which can not only reduce the probability of planar degeneration but also help to distinguish non-connected planar regions in space.

[0134] Figure 9 is a flowchart of step S3-2 in an embodiment of the present disclosure. As Figure 9 shown, step S3-2 includes the following steps:

[0135] S3-2-1: Select a second target point from the neighborhood of the first target point. Among them, the second target point can be a point randomly selected from the neighborhood of the first target point.

[0136] S3-2-2: In response to the normal vector angle between the second target point and the current planar seed being less than the preset normal vector angle threshold, the distance between the second target point and the current planar seed being less than the preset distance threshold, and the number of points within the spherical neighborhood of the first target point being greater than the preset number threshold, add the second target point to the current planar seed.

[0137] Set the preset normal vector angle threshold, preset distance threshold, and preset number threshold in the set conditions regarding the normal vector angle, distance, and number of points when a point can be added to the planar seed.

[0138] If the included angle between the normal vector of the second target point and the current plane seed is less than the preset normal vector included angle threshold, the distance between the second target point and the current plane seed is less than the preset distance threshold, and the number of points within the spherical neighborhood of the first target point is greater than the preset number threshold, then the second target point is added to the current plane seed; otherwise, the second target point is not added to the current plane seed and is recorded.

[0139] In this embodiment, when performing neighborhood plane growth on the plane seed, based on whether the included angle between the normal vector of the point and the plane seed is less than the preset normal vector included angle threshold, whether the distance between the point and the plane seed is less than the preset distance threshold, and whether the number of points within the neighborhood is greater than the preset number threshold, it is possible to reasonably determine whether the point is added to the corresponding plane seed, which can not only reduce the probability of plane degradation but also help distinguish non-connected plane regions in space.

[0140] In some embodiments of the present disclosure, during the process of performing neighborhood plane growth on multiple plane seeds, the following steps are further included: estimating the plane equation based on the positions of the points within each plane seed during the neighborhood plane growth process to obtain the plane equations of each plane seed; and removing outliers from each plane seed based on the plane equations of each plane seed to obtain at least one outlier.

[0141] For each plane seed during the neighborhood plane growth process, based on the positions of all the points within the current plane seed, the plane equation can be re-estimated using the principal component analysis method. All the outliers that do not belong to the current plane are determined according to the re-estimated plane equation, all the outliers are removed from the current plane seed, and the outliers can be set to the unclassified state, that is, the state of not belonging to the current plane seed. Among them, the method of re-estimating the plane equation using the principal component analysis method may include: calculating the covariance matrix according to the positions of all the points within the current plane seed, and determining the plane equation corresponding to the principal plane according to the covariance matrix.

[0142] Figure 10 This is the structural block diagram of the point cloud plane segmentation device according to an embodiment of the present disclosure. As Figure 10 shown, the point cloud plane segmentation device includes:

[0143] A grid division module 100, configured to perform grid division on the three-dimensional space corresponding to the point cloud data using a grid division method based on phase distribution to obtain a plurality of grids;

[0144] A plane fitting module 200, configured to perform parallel processing on the grids with the same phase among the plurality of grids, and perform plane fitting on the points within each grid among the plurality of grids to obtain a plurality of plane seeds;

[0145] The neighborhood plane growth module 300 is used to perform neighborhood plane growth on multiple plane seeds in a parallel processing manner to obtain multiple planes corresponding to the point cloud data.

[0146] In some embodiments of the present disclosure, the mesh division module 100 is used to determine anchor points based on the positions of the points in the point cloud data; the mesh division module 100 is further used to perform multi-scale mesh division on the three-dimensional space corresponding to the point cloud data based on the anchor points by using a mesh division method to obtain multiple meshes.

[0147] Figure 11 It is a structural block diagram of a point cloud plane segmentation device in another embodiment of the present disclosure. As Figure 11 shown, the point cloud plane segmentation device may further include:

[0148] The mesh index determination module 400 is used to determine the mesh index of each point in the point cloud data based on the spatial coordinates of the anchor points and the spatial coordinates of each point in the point cloud data;

[0149] The hash calculation module 500 is used to perform hash calculation based on the mesh index of each point in the point cloud data to obtain the hash value of each point in the point cloud data;

[0150] The phase number determination module 600 is used to determine the phase number of each point in the point cloud data based on the mesh index of each point in the point cloud data;

[0151] The storage module 700 is used to store the hash value of each point in the point cloud data and the phase number of each point in the point cloud data using a hash list.

[0152] In some embodiments of the present disclosure, the plane fitting module 200 is used to perform the following steps for each of the multiple meshes:

[0153] Randomly select a point from the current mesh, perform plane fitting on the currently selected point and the points within the spherical neighborhood, obtain the plane fitting result and increment the plane fitting count by one, where the radius of the spherical neighborhood is equal to the preset mesh side length, and the plane fitting result includes the plane seeds generated when the plane fitting is successful;

[0154] Loop through the steps of randomly selecting a point from the current mesh, performing plane fitting on the currently selected point and the points within the spherical neighborhood, obtaining the plane fitting result and incrementing the plane fitting count by one until the plane fitting count of the current mesh reaches the preset count threshold to obtain all the plane seeds of the current mesh.

[0155] In some embodiments of the present disclosure, the neighborhood plane growth module 300 is used to loop through the following steps for each of the multiple plane seeds until multiple planes corresponding to the point cloud data are obtained:

[0156] Select a first target point from the current planar seed, and obtain each point within the neighborhood of the first target point, where the first target point and each point within the neighborhood of the first target point belong to the points in the point cloud data;

[0157] Based on the normal vector angle and distance between each point within the neighborhood of the first target point and the current planar seed, as well as the number of points within the spherical neighborhood of the first target point, determine whether each point within the neighborhood of the first target point is added to the current planar seed;

[0158] In response to at least one point being added to the current planar seed, update the current planar seed.

[0159] In some embodiments of the present disclosure, the neighborhood plane growth module 300 is configured to select a second target point from within the neighborhood of the first target point; the neighborhood plane growth module 300 is further configured to, in response to the normal vector angle between the second target point and the current planar seed being less than a preset normal vector angle threshold, the distance between the second target point and the current planar seed being less than a preset distance threshold, and the number of points within the spherical neighborhood of the first target point being greater than a preset number threshold, add the second target point to the current planar seed.

[0160] In some embodiments of the present disclosure, the neighborhood plane growth module 300 is further configured to estimate a plane equation based on the positions of the points within each planar seed during the neighborhood plane growth process when performing neighborhood plane growth on multiple planar seeds, to obtain the plane equations of each planar seed; the neighborhood plane growth module 300 is further configured to perform outlier rejection on each planar seed based on the plane equations of each planar seed, to obtain at least one outlier.

[0161] It should be noted that the specific implementation manner of the point cloud plane segmentation device in the embodiments of the present disclosure is similar to the specific implementation manner of the point cloud plane segmentation method in the embodiments of the present disclosure, and the technical effects of the point cloud plane segmentation device in the embodiments of the present disclosure are similar to the technical effects of the point cloud plane segmentation method in the embodiments of the present disclosure. For specific reference, see the description in the part of the point cloud plane segmentation method. To reduce redundancy, it will not be elaborated here.

[0162] In addition, the embodiments of the present disclosure further provide an electronic device, including:

[0163] A memory for storing a computer program;

[0164] A processor for executing the computer program stored in the memory, and when the computer program is executed, implementing the point cloud plane segmentation method described in any one of the above embodiments of the present disclosure.

[0165] Next, refer to Figure 12 to describe the electronic device according to the embodiments of the present disclosure. As Figure 12 shown, the electronic device includes one or more processors and a memory.

[0166] The processor can be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and can control other components in the electronic device to perform desired functions.

[0167] The memory can store one or more computer program products. The memory can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory can include, for example, random access memory (RAM) and / or cache memory, etc. The non-volatile memory can include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program products can be stored on the computer-readable storage media, and the processor can run the computer program products to implement the point cloud plane segmentation method of various embodiments of the present disclosure described above and / or other desired functions.

[0168] In one example, the electronic device may further include: an input device and an output device, and these components are interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0169] In addition, the input device may further include, for example, a keyboard, a mouse, and so on.

[0170] The output device can output various information to the outside, including the determined distance information, direction information, etc. The output device can include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, and so on.

[0171] Of course, for simplicity, Figure 12 only some of the components related to the present disclosure in the electronic device are shown, and components such as buses, input / output interfaces, etc. are omitted. In addition, according to specific application scenarios, the electronic device may further include any other appropriate components.

[0172] In addition to the above methods and devices, the embodiments of the present disclosure may also be computer program products, which include computer program instructions that, when run by the processor, cause the processor to execute the steps in the point cloud plane segmentation method according to various embodiments of the present disclosure described in the above part of this specification.

[0173] The computer program product may be written in any combination of one or more programming languages for executing the program code of the operations of the embodiments of the present disclosure. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0174] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are run by a processor, the processor is caused to execute the steps in the method for segmenting a point cloud plane according to various embodiments of the present disclosure described in the foregoing part of this specification.

[0175] The computer-readable storage medium may employ any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may, for example, include but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0176] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, benefits, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, benefits, effects, etc. are essential for each embodiment of the present disclosure. In addition, the above-mentioned specific details are only for the purposes of illustration and facilitating understanding, rather than limitations, and the above details do not limit the present disclosure to necessarily adopt the above specific details for implementation.

[0177] Each embodiment in this specification is described in a progressive manner, and the key point of each embodiment is to illustrate the differences from other embodiments. For the same or similar parts among the embodiments, reference may be made to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and reference may be made to the partial description of the method embodiment for the relevant parts.

[0178] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any way. Words such as "including", "comprising", "having", etc. are open-ended terms meaning "including but not limited to" and can be used interchangeably with each other. The words "or" and "and" used herein refer to the phrase "and / or" and can be used interchangeably with it, unless the context clearly indicates otherwise. The phrase "such as" used herein refers to the phrase "such as but not limited to" and can be used interchangeably with it.

[0179] The methods and apparatuses of this disclosure can be implemented in many ways. For example, the methods and apparatuses of this disclosure can be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of the steps for the method is only for illustration, and the steps of the methods of this disclosure are not limited to the specific order described above, unless otherwise specifically stated. In addition, in some embodiments, this disclosure can also be implemented as a program recorded in a recording medium, and these programs include machine-readable instructions for implementing the methods according to this disclosure. Therefore, this disclosure also covers the recording medium storing the programs for executing the methods according to this disclosure.

[0180] It should also be noted that in the apparatuses, equipment, and methods of this disclosure, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of this disclosure.

[0181] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects are very obvious to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.

[0182] The above description has been given for purposes of illustration and description. In addition, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions, and sub-combinations thereof.

Claims

1. A point cloud plane segmentation method, characterized in that: include: Using a grid division method based on phase distribution, the three-dimensional space corresponding to the point cloud data is grid-divided to obtain a plurality of grids; For grids of the same phase in the multiple grids, a parallel processing method is used to perform plane fitting on points in each grid in the multiple grids to obtain multiple plane seeds; By means of parallel processing, neighborhood plane growth is performed on the multiple plane seeds to obtain multiple planes corresponding to the point cloud data.

2. The method according to claim 1, characterized in that: The grid division method based on phase distribution is used to grid the three-dimensional space corresponding to the point cloud data to obtain multiple grids, including: Determining an anchor point based on the position of each point in the point cloud data; Based on the anchor point, the three-dimensional space corresponding to the point cloud data is divided into multi-scale grids using the grid division method to obtain the multiple grids.

3. The method according to claim 2, characterized in that After the three-dimensional space corresponding to the point cloud data is divided into multi-scale grids based on the anchor point and the grid division method is adopted to obtain the plurality of grids, the method further includes: Determine a grid index of each point in the point cloud data based on the spatial coordinates of the anchor point and the spatial coordinates of each point in the point cloud data; Performing hash calculation based on the grid index of each point in the point cloud data to obtain a hash value of each point in the point cloud data; Determining a phase number of each point in the point cloud data based on a grid index of each point in the point cloud data; A hash list is used to store the hash value of each point in the point cloud data and the phase number of each point in the point cloud data.

4. The method according to any one of claims 1 to 3, characterized in that: The performing plane fitting on the points in each of the plurality of grids to obtain the plane seeds of each grid includes: For each of the plurality of grids, the following steps are performed: Randomly select a point from the current grid, perform plane fitting on the currently selected point and the points in the spherical neighborhood, obtain a plane fitting result and increase the number of plane fitting times by one, wherein the radius of the spherical neighborhood is equal to the preset grid side length, and the plane fitting result includes a plane seed generated when the plane fitting is successful; The steps of randomly selecting a point from the current grid, performing plane fitting on the currently selected point and the points in the spherical neighborhood, obtaining a plane fitting result and increasing the number of plane fitting times by one are executed cyclically until the number of plane fitting times of the current grid reaches a preset threshold, thereby obtaining all plane seeds of the current grid.

5. The method according to any one of claims 1 to 3, characterized in that: The performing neighborhood plane growth on the multiple plane seeds to obtain multiple planes corresponding to the point cloud data includes: The following steps are performed cyclically for each plane seed of the plurality of plane seeds until a plurality of planes corresponding to the point cloud data are obtained: Selecting a first target point from the current plane seed, and acquiring each point in a neighborhood of the first target point, wherein the first target point and each point in the neighborhood of the first target point are points in the point cloud data; Determining whether to add each point in the neighborhood of the first target point to the current plane seed based on the normal vector angle and distance between each point in the neighborhood of the first target point and the current plane seed, and the number of points in the neighborhood of the first target point; In response to at least one point being added to the current plane seed, the current plane seed is updated.

6. The method according to claim 5, characterized in that The determining whether to add each point in the neighborhood of the first target point to the current plane seed based on the normal vector angle and distance between each point in the neighborhood of the first target point and the current plane seed, and the number of points in the neighborhood of the first target point, comprises: Selecting a second target point from within the neighborhood of the first target point; In response to the normal vector angle between the second target point and the current plane seed being less than a preset normal vector angle threshold, the distance between the second target point and the current plane seed being less than a preset distance threshold, and the number of points within the spherical neighborhood of the first target point being greater than a preset number threshold, the second target point is added to the current plane seed.

7. The method according to any one of claims 1 to 3, characterized in that: The process of performing neighborhood plane growth on the plurality of plane seeds further includes: Estimating the plane equation based on the position of each point in each plane seed during the neighborhood plane growth process to obtain the plane equation of each plane seed; Outliers are eliminated from each plane seed based on the plane equation of each plane seed to obtain at least one outlier.

8. A point cloud plane segmentation device, characterized in that: include: A grid division module, used to grid the three-dimensional space corresponding to the point cloud data using a grid division method based on phase distribution to obtain a plurality of grids; A plane fitting module, for performing plane fitting on points in each of the multiple grids in a parallel processing manner for grids of the same phase in the multiple grids, to obtain multiple plane seeds; The neighborhood plane growth module is used to perform neighborhood plane growth on the multiple plane seeds in a parallel processing manner to obtain multiple planes corresponding to the point cloud data.

9. An electronic device, characterized in that: include: a memory for storing a computer program product; A processor is used to execute the computer program product stored in the memory, and when the computer program product is executed, it implements the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method described in any one of claims 1 to 7 is implemented.

Images