Point cloud data segmentation method, device, computer equipment and storage medium

Abstract

The present application relates to a method, apparatus, computer device, storage medium, and computer program product for segmenting point cloud data. The method comprises: mapping point cloud data to voxel space, projecting the data in the voxel space along the principal axis of the point cloud data to obtain multiple layers of images arranged in sequence; performing connected domain analysis on each layer of the image to obtain a connected domain of each layer of the image; performing voxel connected domain analysis on each layer of the image arranged in sequence based on the connected domain of each layer of the image to obtain a voxel space connected domain; and segmenting the point cloud data using the voxel space connected domain. This method can reduce the amount of computation by constructing a voxel space and performing two-dimensional connected domain calculations, so that the computational amount of point cloud segmentation is linearly related to the amount of data. The algorithm has low complexity and is easy to segment and process large amounts of point cloud data.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, apparatus, computer equipment, storage medium, and computer program product for segmenting point cloud data. Background Art

[0002] With the development of artificial intelligence (AI) technology, robots have been widely used in various industries. In industrial applications, robots are equipped with 3D perception systems. Using the 3D information acquired by these systems, robots can control end-effectors to perform actions such as machining and assembly. Simply put, the 3D perception system is equivalent to the human eye, and the end-effector is equivalent to the human hand. Through the coordination between the hand and the eye, pre-programmed tasks are completed.

[0003] The processing of scattered point clouds requires a lot of time to build a tree structure. At the same time, the seed filling method is used to add adjacent points in three dimensions to the same mark, and the adjacent neighbors are added to the end of the stack for the next calculation, so the amount of calculation is relatively large. Summary of the Invention

[0004] Based on this, it is necessary to provide a point cloud data segmentation method, device, computer equipment, computer-readable storage medium and computer program product that can reduce the amount of calculation to address the above technical problems.

[0005] In a first aspect, the present application provides a method for segmenting point cloud data. The method comprises:

[0006] Mapping the point cloud data to a voxel space, and projecting the data in the voxel space along the principal axis direction of the point cloud data to obtain sequentially arranged multi-layer images;

[0007] Performing connected domain analysis on the images of each layer respectively to obtain connected domains of the images of each layer;

[0008] Based on the connected domains of the images of each layer, voxel connected domains are analyzed on the images of each layer arranged in sequence to obtain voxel space connected domains;

[0009] The point cloud data is segmented by the voxel spatial connected domain.

[0010] In one embodiment, mapping the point cloud data to a voxel space, and projecting the data in the voxel space along the principal axis direction of the point cloud data to obtain sequentially arranged multi-layer images, includes:

[0011] Determine the principal axis direction and the corresponding bounding box based on the point cloud data;

[0012] constructing a plurality of voxels in a voxel space based on the bounding box;

[0013] Calculate the index of the point cloud data in the voxel space to obtain the mapped voxel space;

[0014] In the mapped voxel space, the point cloud data is projected layer by layer along the principal axis dimension to obtain sequentially adjacent multi-layer images.

[0015] In one embodiment, determining the principal axis direction and the corresponding bounding box based on the point cloud data includes:

[0016] Analyze the principal components of the point cloud data to obtain the information content of the point cloud data in different dimensions;

[0017] Determining the main axis direction based on the information amount of the different dimensions;

[0018] A bounding box of the point cloud data is determined according to the main axis direction.

[0019] In one embodiment, performing connected domain analysis on each layer of the image to obtain the connected domain of each layer of the image includes:

[0020] Performing connected domain analysis on the images of each layer respectively to obtain image connected domain analysis results;

[0021] Determine a global connected component label based on the connected component labels and offset values ​​within each layer of the image;

[0022] Based on the global connected domain label, the image connected domain analysis result is labeled to obtain the image connected domains of each layer.

[0023] In one embodiment, performing voxel connected domain analysis on sequentially arranged layers of images based on the connected domains of the layers of images respectively includes:

[0024] Based on the pixel positions in the connected domain of each layer of images and the pixel position set of the connected domain of the adjacent layer of images, it is determined in sequence whether the adjacent connected domains of images to which each pixel position belongs match, and adjacent connected domain matching results are obtained in sequence; wherein the pixel position set of the connected domain of the adjacent layer of images includes the target pixel position that is the same as the pixel position, and / or the neighboring pixel position of the target pixel position in the connected domain of the adjacent layer of images;

[0025] Based on the matching results of the connected domains of each adjacent image obtained in sequence, it is determined whether the adjacent voxels of the connected domains of the adjacent images in each layer arranged in sequence match each other.

[0026] In one embodiment, the determining whether adjacent voxels of adjacent images to which pixel positions of each layer are sequentially arranged match based on the connected domain matching results of each adjacent image sequentially obtained includes:

[0027] Determining a plurality of associated image connected domains based on the matching results of the adjacent image connected domains;

[0028] Determining the voxel connected domain value of each of the image connected domains in sequence based on the order in which the associated multiple image connected domains are arranged;

[0029] Based on the connectivity domain value of each voxel, it is determined whether each adjacent voxel matches.

[0030] In one embodiment, determining the voxel connectivity domain value of each of the image connected domains in sequence based on the order in which the associated multiple image connected domains are arranged includes:

[0031] Based on the order in which the associated multiple image connected domains are arranged, sequentially determining the associated multiple image connected domains as target image connected domains;

[0032] When a value of one of the image connected domains of the left neighboring image connected domain or the upper neighboring image connected domain of the target image connected domain is a valid value, updating the voxel connected domain value of the image connected domain based on the valid value;

[0033] When the image connected domain values ​​in the left neighboring image connected domain and the upper neighboring image connected domain of the target image connected domain are both valid values, the image connected domain values ​​of the left neighboring image connected domain and the upper neighboring image connected domain are compared, and the voxel connected domain value of the target image connected domain is determined based on the comparison result.

[0034] In a second aspect, the present application further provides a device for segmenting point cloud data. The device comprises:

[0035] An image generation module, configured to map the point cloud data to a voxel space, and project the data in the voxel space along the principal axis direction of the point cloud data to obtain a sequentially arranged multi-layer image;

[0036] An image connected domain generation module is used to perform connected domain analysis on the images of each layer to obtain connected domains of the images of each layer;

[0037] a voxel connected domain generation module, configured to perform voxel connected domain analysis on the sequentially arranged layers of images based on the connected domains of the layers of images, to obtain voxel spatial connected domains;

[0038] The point cloud data segmentation module is used to segment the point cloud data according to the voxel space connected domain.

[0039] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the step of segmenting the point cloud data in any of the above embodiments is implemented.

[0040] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the step of segmenting point cloud data in any of the above embodiments.

[0041] In a fifth aspect, the present application further provides a computer program product, which includes a computer program that, when executed by a processor, implements the step of segmenting point cloud data in any of the above embodiments.

[0042] The above-mentioned point cloud data segmentation method, device, computer equipment, storage medium and computer program product map the point cloud data to voxel space, integrate the scattered point cloud data into data in the voxel space, project the data in the voxel space according to the main axis direction of the point cloud data, and obtain multi-layer images arranged in sequence, so that the multi-dimensional data is converted into multi-layer images arranged according to the main axis direction, and the point cloud data of any data is not deleted to perform two-dimensional connected domain analysis, and the connected domains of each layer of images corresponding to each two-dimensional image are obtained, and then the connected domains of each layer of images are used as units in the voxel space for voxel connected domain analysis, which also belongs to the calculation of two-dimensional data. Therefore, the process only involves the calculation amount of constructing the voxel space and two-dimensional images. The calculation amount is linearly related to the amount of data. The algorithm complexity is low and it is easy to segment and process a large amount of point cloud data. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A diagram showing an application environment of a method for segmenting point cloud data in one embodiment;

[0044] Figure 2 1 is a flow chart of a method for segmenting point cloud data in one embodiment;

[0045] Figure 3 is a flow chart of a method for segmenting point cloud data in another embodiment;

[0046] Figure 4 A schematic diagram of a process for generating voxel connectivity domain values ​​in one embodiment;

[0047] Figure 5 is a flow chart of a method for segmenting point cloud data in another embodiment;

[0048] Figure 6 A schematic diagram of a process for matching connected domains of associated images in one embodiment;

[0049] Figure 7 is a structural block diagram of a device for segmenting point cloud data in one embodiment;

[0050] Figure 8 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0052] The point cloud data segmentation method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers. The terminal 102 maps the point cloud data to the voxel space, projects the data in the voxel space according to the main axis direction of the point cloud data, and obtains multiple layers of images arranged in sequence, performs connected domain analysis on each layer of images, and obtains connected domains of each layer of images. Based on the connected domains of each layer of images, voxel connected domain analysis is performed on each layer of images arranged in sequence to obtain voxel space connected domains, and the point cloud data is segmented by the voxel space connected domains.

[0053] Terminal 102 may include, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices may include smart speakers, smart TVs, smart air conditioners, and smart car devices. Portable wearable devices may include smart watches, smart bracelets, and head-mounted devices. Server 104 may be implemented as a standalone server or a server cluster consisting of multiple servers.

[0054] In one embodiment, Figure 2 As shown in the figure, a method for segmenting point cloud data is provided, which is applied to Figure 1 Taking the terminal 102 in FIG. 1 as an example, the method includes the following steps:

[0055] Step 202 : Map the point cloud data to the voxel space, and project the data in the voxel space along the principal axis direction of the point cloud data to obtain a multi-layer image arranged in sequence.

[0056] Point cloud data is a set of multidimensional point data obtained by the robot through a multidimensional perception system. The coordinate data of each point in the multidimensional data point set represents the position of the corresponding point in space.

[0057] In one embodiment, floating-point rounding is performed on the data of each dimension of the point cloud data, and the index of the corresponding voxel is determined based on the determined integer, and then the corresponding spatial position area is represented based on the index of the corresponding voxel. The spatial position area is determined by each voxel having a corresponding voxel specification threshold, and the voxel specification threshold is a set of critical values ​​of the length of the voxel in each dimension. The voxel specification threshold is determined based on the point cloud data. When the point cloud data represents an object to be segmented, the voxel specification threshold is greater than the resolution of the point cloud data; when the point cloud data represents multiple objects to be segmented, the voxel specification threshold is greater than the resolution of the point cloud data and less than the distance between any two objects to be segmented.

[0058] In one embodiment, before mapping the point cloud data to voxel space, the method further includes: calculating the bounding box length and subtracting the minimum value of each dimension from the point cloud data in each dimension to ensure that the mapped value in each dimension is positive; and calculating the ratio of the mapped value of the point cloud data in each dimension and the bounding box length to the voxel specification threshold to obtain the voxel specification in the voxel space. Therefore, each point cloud data can have its own corresponding voxel specification, and a voxel specification can define the length of the voxel in each dimension.

[0059] In the process of mapping point cloud data to voxel space, the coordinate information of each point in the point cloud data and the voxel specification threshold are processed to convert the point cloud data into voxel space and obtain its position information in voxel space. For example, when the point cloud data is three-dimensional data, the length of the three-dimensional data in the X, Y, and Z dimensions is divided by the length of the voxel in the corresponding dimension to obtain the position information of the point cloud data in the X, Y, and Z dimensions mapped to the three-dimensional voxel space. In other words, after the point cloud data is mapped, its index in voxel space is obtained.

[0060] When data in voxel space is sequentially projected along the principal axis, terminal 102 controls the sequential conversion of the multidimensional data in the voxel space, resulting in a sequentially arranged multi-layer image. When data in voxel space is sequentially projected along the principal axis and in accordance with a certain projection rule, the multidimensional data in the voxel space is converted according to the certain projection rule, resulting in a sequentially arranged multi-layer image. For example, a robot may slice the voxel space along the principal axis to obtain a multi-layer image. When the number of midpoints in each voxel corresponding to the image is non-zero, the corresponding pixel in the two-dimensional image is set to 1, and the pixel size of the projected image is consistent with the length and width of the voxel XY.

[0061] In one embodiment, point cloud data is mapped to voxel space, and the data in the voxel space is projected along the main axis direction of the point cloud data to obtain multiple layers of images arranged in sequence, including: determining the main axis direction and the corresponding bounding box based on the point cloud data; constructing multiple voxels in the voxel space based on the bounding box; calculating the index of the point cloud data in the voxel space to obtain the mapped voxel space; in the mapped voxel space, the point cloud data is sequentially projected along the main axis direction to obtain multiple layers of images that are adjacent in sequence.

[0062] A bounding box is a spatial geometric body used to segment point cloud data into discrete point sets. The volume of this spatial geometric body is larger than any point in the point cloud data, and its parameters such as length, volume, and surface area are measurable.

[0063] For constructing multiple voxels in a voxel space based on bounding boxes, each voxel is determined based on the boundary of at least one bounding box; wherein the bounding box type is one or more of any bounding boxes such as an axial bounding box (AABB bounding box), a bounding sphere (Sphere), a directional bounding box (OBB), a fixed-direction convex hull FDH (k-DOP), etc.

[0064] In one embodiment, determining the principal axis direction and the corresponding bounding box boundary based on point cloud data includes: analyzing the principal components of the point cloud data to obtain the amount of information in different dimensions of the point cloud data; determining the principal axis direction based on the amount of information in different dimensions; and determining the bounding box of the point cloud data according to the principal axis direction. The principal component analysis of the point cloud data is performed based on principal component analysis (PCA). The principal axis direction determined by the principal component analysis is the direction with the thinnest point cloud and the least amount of information, which helps to reduce the distribution range of the data to be projected and reduce the amount of subsequent calculations.

[0065] Step 204 : Perform connected domain analysis on each layer of images to obtain connected domains of each layer of images.

[0066] In one embodiment, a connected domain analysis is performed on each layer of projected two-dimensional images. Here, the two-pass algorithm for two-dimensional images can be directly borrowed, and the starting value of the connected domain mark of each layer of image is determined based on the specifications of the two-dimensional connected domain, and is incremented from the starting value to ensure that the connected domain mark of each layer is unique.

[0067] In one embodiment, the connected domain analysis of each layer of images is performed based on a two-pass algorithm. Accordingly, the connected domain analysis is performed on each layer of images respectively, including: the terminal analyzes the pixels in each layer of images one by one based on the order in which the connected domains of the multi-layer images are arranged; when the image connected domain value of the neighboring pixels of the pixel under analysis is an invalid value, the image connected domain value of the pixel under analysis is set to an image connected domain setting value related to the pixel setting number; when the image connected domain value of the neighboring pixels of the pixel under analysis is a valid value, the image connected domain value of the pixel is updated based on the valid value.

[0068] In one embodiment, a connected domain analysis is performed on each layer of an image based on a two-pass algorithm, specifically including: determining each pixel in the associated multiple connected domains of the associated multiple layers as a target pixel based on the order in which the connected domains of the associated multiple layers of images are arranged; when the image connected domain values ​​of the left neighboring pixel and the upper neighboring pixel of the target pixel are invalid values, setting the image connected domain value of the target pixel based on the number of times the image connected domain value of each pixel is set; when at least one pixel value of the left neighboring pixel or the upper neighboring pixel of the target pixel is a valid value, updating the image connected domain value of the pixel based on the valid value; when the image connected domain values ​​of the left neighboring pixel and the upper neighboring pixel of the target pixel are both valid values, comparing the image connected domain values ​​of the left neighboring pixel and the upper neighboring pixel, and determining the image connected domain value of the target pixel based on the comparison result. In particular, when the image connected domain value of the image connected domain value increases with the number of times the image connected domain value of each pixel is set, the smaller value of the image connected domain value of the left neighboring pixel and the upper neighboring pixel is used as the image connected domain value.

[0069] In one embodiment, connected domain analysis is performed on each layer of images to obtain connected domains of each layer of images, including: performing connected domain analysis on each layer of images to obtain image connected domain analysis results; determining global connected domain labels based on the intra-layer connected domain labels and offset values ​​of each layer of images; and labeling the image connected domain analysis results based on the global connected domain labels to obtain connected domains of each layer of images.

[0070] The image connected domain analysis results are used to characterize whether each pixel area in each layer of the image constitutes the corresponding image connected domain. However, repeated intra-layer connected domain labels may appear between image connected domains of different layers. Therefore, the image connected domain analysis results are based on the intra-layer connected domain labels and offset values ​​so that the image connected domains of each layer have globally unique labels, which is convenient for subsequent analysis.

[0071] Step 206 : Based on the connected domains of the images of each layer, voxel connected domains are analyzed on the sequentially arranged layers of images to obtain voxel spatial connected domains.

[0072] The connected domains of each layer of images belong to different two-dimensional connected domains. These two-dimensional connected domains are regarded as the basis for constructing three-dimensional connected domains. Three-dimensional voxel connected domain analysis is performed to determine the voxel space connected domain to which each two-dimensional connected domain belongs.

[0073] By performing multiple matches between the image connected domain and the voxel space connected domain, the related process of building a tree structure is omitted, and the two-step method of two-dimensional images is borrowed to reduce the amount of calculation, and the amount of calculation is linearly related to the amount of data. In particular, compared with the seed filling method, the method of this embodiment does not need to rely on OcTree or KdTree to find domain relationships. For the processing of scattered point clouds, it does not need to spend more time to build a tree structure, nor does it need to add adjacent points in three dimensions to the same label as the seed filling method, so as to add the adjacent neighbors to the end of the stack for the next calculation, so the seed filling method has a relatively large amount of calculation. Therefore, this method theoretically has advantages in big data processing.

[0074] In one embodiment, voxel connected domain analysis is performed on sequentially arranged layers of images based on connected domains of each layer of images, including:

[0075] Based on the pixel positions in the image connected domain of each layer and the pixel position set of the image connected domain of the adjacent layer, it is judged in turn whether the adjacent image connected domains to which each pixel position of each layer belongs match, and the adjacent image connected domain matching results are obtained in turn; based on the adjacent image connected domain matching results obtained in turn, it is judged whether the voxel connectivity thresholds of the adjacent image connected domains of each layer arranged in turn match; wherein, the pixel position set of the image connected domain of the adjacent layer includes the target pixel position that is the same as the pixel position, and / or the target pixel position is at the neighborhood pixel position of the image connected domain of the adjacent layer.

[0076] In one embodiment, voxel connectivity domain analysis is performed on sequentially arranged layers of images, including: a terminal analyzes multiple image connectivity domains one by one based on the order in which the multiple layers of image connectivity domains are arranged, and when the voxel connectivity domain value of the adjacent image connectivity domain of the image connectivity domain under analysis is a valid value, the voxel connectivity domain value of the image connectivity domain is updated based on the valid value.

[0077] For example, each 2D image connected domain in each layer of the projected 2D image is traversed to check whether the target pixel position in the next layer of the image has inconsistent image connected domain labels with the neighborhood of the target pixel position. If so, the adjacent image connected domain in the association item is recorded. After the image connected domain of each layer is first determined, the smallest image connected domain value is selected from the adjacent image connected domains in the association item recorded for each layer as the voxel connected domain value. The voxel connected domain value is a connected domain label in the voxel space that is used to indicate the voxel connected domain to which each voxel belongs.

[0078] Step 208 : Segment the point cloud data by voxel space connected domains.

[0079] After determining the 2D image connected domain corresponding to each voxel in the voxel space, the voxel space connected domain to which each pixel in each 2D image connected domain belongs is determined, and the point cloud data belonging to the same voxel space connected domain is then determined as the same part after segmentation, thereby segmenting the point cloud data. For example, when related image connected domains have the same voxel label, the point cloud data corresponding to these image connected domains are combined, and each combination is used as a part of the point cloud data, thereby achieving point cloud data segmentation.

[0080] In the above-mentioned point cloud data segmentation method, the point cloud data is mapped to voxel space, integrating the scattered point cloud data into data in voxel space. The data in voxel space is projected along the principal axis of the point cloud data to obtain sequentially arranged multi-layer images, converting the multi-dimensional data into multi-layer images arranged along the principal axis. A two-dimensional connected domain analysis is then performed to obtain the connected domains of each layer of the two-dimensional image. The connected domains of each layer of the image are then used as units in voxel space for voxel connected domain analysis, which also involves two-dimensional data calculations. Therefore, this process only involves the computational effort of constructing the voxel space and two two-dimensional images. This computational effort is linearly proportional to the amount of data, resulting in a low algorithmic complexity and ease of segmentation for large amounts of point cloud data. This reduced computational effort helps robots accurately process object information from acquired point cloud data, providing preprocessing for subsequent posture calculations and object recognition, and instructing the robot's end-user to complete pre-set motion tasks. Specifically, when analyzing information of interest in three-dimensional data, connected domain segmentation technology is required to enable the robot to obtain coordinate information based on the segmented object information.

[0081] In one embodiment, Figure 3 As shown in the figure, a method for segmenting point cloud data is provided, which is applied to Figure 1 The terminal 102, the server 104, or the interaction between the terminal 102 and the server 104 is performed. This embodiment takes the terminal 102 as an example for description, and includes the following steps:

[0082] Step 302 : Map the point cloud data to the voxel space, and generate a multi-layer image corresponding to the voxel space arranged sequentially according to the main axis direction of the point cloud data.

[0083] In step 304, based on the pixel positions in the image connected domain of each layer and the pixel position set of the image connected domain of the adjacent layer, it is determined in sequence whether the adjacent image connected domains of each layer match, and the adjacent image connected domain matching results are obtained in sequence.

[0084] The pixel position set of the connected domain of the adjacent layer image includes the target pixel position that is the same as the pixel position, and / or the target pixel position is a neighboring pixel position of the connected domain of the adjacent layer image. Steps 302 to 304 may refer to steps 202 to 204.

[0085] Step 306 : Based on the sequentially obtained matching results of the connected domains of the adjacent images, it is determined whether the voxel connected domain values ​​of the adjacent connected domains of the sequentially arranged pixel positions of the layers match.

[0086] In one embodiment, based on the sequentially obtained results of connected domain matching of each adjacent image, determining whether adjacent voxels belonging to adjacent images of each pixel position in each sequentially arranged layer match includes: determining multiple associated image connected domains based on the results of connected domain matching of each adjacent image; sequentially determining voxel connected domain values ​​of each image connected domain based on the order in which the multiple associated image connected domains are arranged; determining whether the voxel connected domain values ​​of the sequentially arranged image connected domains of each layer are consistent based on the voxel connected domain values; if they are consistent, the voxel connected domain values ​​are matched; if they are inconsistent, bidirectional recording is performed. Furthermore, the method further includes: performing matching based on the bidirectionally recorded image connected domains, and determining the voxel connected domain values ​​based on the matching results.

[0087] Specifically, when two 2D images have adjacent pixels, they likely correspond to the same voxel connected domain. Therefore, the pixels in the next 2D image layer below each pixel are analyzed in five directions: above, below, left, right, and center. This determines whether they belong to the same image connected domain. To better record the corresponding data, a sufficiently large queue is constructed, with its index number representing the global connected domain marker. Members store markers adjacent to this index number. In other words, when different image connected domains exist in the five directions of the 2D image, their values ​​are stored in the queue members, allowing comparisons to be made based on the queue members to determine the voxel connected domain value.

[0088] In one embodiment, based on the order in which the associated multiple image connected domains are arranged, determining the voxel connected domain value of each image connected domain in sequence includes:

[0089] Based on the order in which the associated multiple image connected domains are arranged, sequentially determining the associated multiple image connected domains as target image connected domains;

[0090] When a value of one of the image connected domains of the left neighboring image connected domain or the upper neighboring image connected domain of the target image connected domain is a valid value, updating the voxel connected domain value of the image connected domain based on the valid value;

[0091] When the image connected domain values ​​in the left neighboring image connected domain and the upper neighboring image connected domain of the target image connected domain are both valid values, the image connected domain values ​​of the left neighboring image connected domain and the upper neighboring image connected domain are compared, and the voxel connected domain value of the target image connected domain is determined based on the comparison result.

[0092] The image connectivity domain values ​​of the left-neighboring image connectivity domain and the upper-neighboring image connectivity domain are compared, and the voxel connectivity domain value of the target image connectivity domain is determined based on the comparison result, including: determining that the image connectivity domain setting value increases with the setting number of image connectivity domains, then comparing the image connectivity domain values ​​of the left-neighboring image connectivity domain and the upper-neighboring image connectivity domain, and taking the smaller image connectivity domain value as the voxel connectivity domain value of the target image connectivity domain.

[0093] Step 308 : Based on the order in which the associated multiple image connected domains are arranged, determine the voxel connected domain value of each image connected domain in sequence.

[0094] Specifically, the global connected domain labels associated in the queue are unified, and the minimum value in the associated items is used as a representative to determine the 3D voxel connected domain to which each image connected domain belongs. Figure 4 As shown in (a), the elements in the first column represent the labels of the connected domains of the image arranged in sequence. Each row starting from the second element is a bidirectional record that has an adjacent relationship with the first element. The second element represents an image connected domain. The values ​​in each cell starting from the second element indicate that it belongs to a different connected domain from the first element. Figure 4 (a) is matched with each adjacent image connected domain, and each selected adjacent image connected domain is matched in turn, and the minimum value of the adjacent image connected domain is selected in the matching process to determine the voxel connected domain. Figure 4 As shown in (b), after matching the adjacent image connected domains, the image connected domain values ​​0 to 3 are replaced by the voxel connected domain value 0, and the image connected domain values ​​4 and 5 are replaced by the voxel connected domain value 4, finally obtaining two three-dimensional voxel connected domains with voxel connected domain values ​​0 and 4.

[0095] Step 310 : Segment the point cloud data by voxel space connected domains.

[0096] The embodiment of step 310 refers to the embodiment of step 210 .

[0097] In this embodiment, the related items are summarized by sorting, and only voxels need to be constructed. The amount of calculation is linearly related to the amount of data. At the same time, the two-step method of two-dimensional images is used to reduce the amount of calculation. In theory, there will be advantages in big data processing.

[0098] Furthermore, in order to more clearly discuss the overall solution of the present application, the image connected domain setting value increases with the number of times the image connected domain is set. The overall process executed by the terminal is described through a more specific embodiment, such as Figure 5 As shown, it includes:

[0099] The point cloud data acquired by the 3D acquisition device is input into the terminal;

[0100] Process point cloud data based on PCA to reduce the amount of point cloud AABB bounding box data;

[0101] Calculate the data length of the bounding box, and convert the point cloud data into a positive point cloud data mapping value based on the minimum value of each dimension of the point cloud data and the value of each point data;

[0102] Divide the bounding box data length and the point cloud data mapping value by the voxel specification threshold to obtain the voxel space;

[0103] Rounding the point cloud data and storing it in the corresponding voxel grid in the voxel space;

[0104] Project the data of the principal axis direction in each voxel grid onto the corresponding plane to obtain a projection image;

[0105] Perform two-dimensional image connectivity domain analysis on the projected image to obtain the corresponding image connectivity domain value;

[0106] Traversing all two-dimensional images along the main axis direction to detect the target pixel position of the next layer and the image connected domains to which the neighboring pixels of the target pixel position belong, and storing the inconsistent image connected domain values ​​into the queue members in the queue;

[0107] Substitute the queue members into the recursive function and determine the minimum value among the associated items determined by the recursive function;

[0108] Replace the connected domain values ​​of all two-dimensional images with the minimum value of the connected domain values ​​of the associated images;

[0109] The points in the voxels with the same two-dimensional image connectivity domain value are combined into a point cloud connectivity domain to complete the segmentation preprocessing of the point cloud data.

[0110] Furthermore, in order to more clearly explain the generation process of voxel connected domains, the image connected domain setting value increases with the number of image connected domain settings, and the adjacent image connected domains of each image connected domain are stored in a queue, and the matching process of the associated image connected domains is recursively processed in a stack manner. Please refer to Figure 6 . Figure 6There are two for or do...while loops, loop A and loop B. Loop A is used to determine the image connected domain for bidirectional recording, and loop B is used to match and compare the image connected domains in loop A. The details are as follows:

[0111] In loop A, the image connected domain queue in the bidirectional record is obtained and traversed. Before reaching the end of the image connected domain queue, queue members are sequentially removed from the queue. If the removed queue member is empty, traversal continues; otherwise, the queue member that is not empty is input into loop B. A non-empty queue member means that there is an adjacent image connected domain in the associated item, that is, the target image connected domain has at least one of the left adjacent image connected domain and the upper adjacent image connected domain.

[0112] In loop B, the index number of the queue member input by A is obtained, and a member set is obtained based on the queue input by loop A. If the member set is a non-empty set, loop B includes: comparing the input queue member with the member set of its associated item in sequence, returning the index number of the smaller queue member, and replacing the index number of the queue member input by loop A with the index number of the smaller queue member; wherein the index number is used to represent the voxel, and the queue member corresponding to the index number includes the image connected domain.

[0113] In loop B, the index number of the queue member input by A is obtained, and according to the queue input by loop A; when the member set is an empty set, loop B includes: clearing the associated items of the queue member input by loop A to avoid repeated calculations in an infinite loop, recursively looping the pattern of B, and saving the calculated minimum associated items through the stack; among which, the minimum associated items are used to represent the voxel connected domain.

[0114] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0115] Based on the same inventive concept, embodiments of the present application also provide a point cloud data segmentation device for implementing the aforementioned point cloud data segmentation method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more point cloud data segmentation device embodiments provided below can be found in the aforementioned limitations of the point cloud data segmentation method and will not be further elaborated here.

[0116] In one embodiment, Figure 7 As shown, a point cloud data segmentation device is provided, comprising: an image generation module 702, an image connected domain generation module 704, a voxel connected domain generation module 706 and a point cloud data segmentation module 708, wherein:

[0117] An image generation module 702 is configured to map the point cloud data to a voxel space, project the data in the voxel space along the principal axis of the point cloud data, and obtain a sequentially arranged multi-layer image.

[0118] An image connected domain generation module 704 is configured to perform connected domain analysis on each layer of the image to obtain a connected domain of each layer of the image;

[0119] A voxel connected domain generation module 706 is configured to perform voxel connected domain analysis on the sequentially arranged layers of images based on the connected domains of the layers of images to obtain voxel spatial connected domains;

[0120] The point cloud data segmentation module 708 is configured to segment the point cloud data according to the voxel spatial connected domain.

[0121] In one embodiment, the image generation module 702 includes:

[0122] A preprocessing module is used to determine the main axis direction and the corresponding bounding box based on the point cloud data;

[0123] a voxel index determination module, configured to construct a plurality of voxels in a voxel space based on the bounding box;

[0124] The point cloud data index acquisition module is used to calculate the index of the point cloud data in the voxel space and obtain the mapped voxel space;

[0125] The voxel space projection module is used to project the point cloud data in layers along the main axis direction in the mapped voxel space to obtain sequentially adjacent multi-layer images.

[0126] In one embodiment, the pre-processing module includes:

[0127] an information content determination unit, configured to analyze the principal components of the point cloud data to obtain the information content of the point cloud data in different dimensions;

[0128] a main axis determination unit, configured to determine a main axis direction based on the information amounts of the different dimensions;

[0129] A single bounding box construction element is used to determine the bounding box of the point cloud data according to the main axis direction.

[0130] In one embodiment, the image connected domain generation module 704 includes:

[0131] An image connected domain analysis unit is used to perform connected domain analysis on the images of each layer to obtain image connected domain analysis results;

[0132] A global label generation unit, configured to determine a global connected component label based on the connected component labels and offset values ​​within each layer of the image;

[0133] The image connected domain determining unit of each layer is used to mark the image connected domain analysis result based on the global connected domain label to obtain the image connected domain of each layer.

[0134] In one embodiment, the voxel connected domain generation module 706 includes:

[0135] an associated item determination unit, configured to determine, based on the pixel positions in the connected domain of each layer of images and the set of pixel positions of the connected domain of the adjacent layer of images, whether the adjacent connected domains of images to which each pixel position belongs match, and to obtain matching results of adjacent connected domains of images in sequence; wherein the set of pixel positions of the connected domain of the adjacent layer of images includes a target pixel position that is the same as the pixel position, and / or a neighboring pixel position of the target pixel position in the connected domain of the adjacent layer of images;

[0136] The associated item matching unit is used to determine whether adjacent voxels of adjacent image connected domains in each layer arranged in sequence are matched based on the matching results of each adjacent image connected domain obtained in sequence.

[0137] In one embodiment, the associated item matching unit includes:

[0138] a sorting subunit, configured to determine a plurality of associated image connected domains based on the matching results of the adjacent image connected domains;

[0139] a voxel connected domain value determining subunit, configured to determine the voxel connected domain value of each of the image connected domains in sequence based on the order in which the associated multiple image connected domains are arranged;

[0140] The voxel matching subunit to which the pixel belongs is used to determine whether each of the adjacent voxels matches based on the connected domain value of each voxel.

[0141] In one embodiment, the voxel connectivity domain value determination subunit includes:

[0142] a voxel connected domain value sequence determining subunit, configured to determine the associated multiple image connected domains as target image connected domains in sequence based on the order in which the associated multiple image connected domains are arranged;

[0143] a first voxel connected domain value determining subunit, configured to update the voxel connected domain value of the target image connected domain based on a valid value when a value of one of the image connected domains to the left or above of the target image connected domain is a valid value;

[0144] The second voxel connected domain value determination subunit is used to compare the image connected domain values ​​of the left adjacent image connected domain and the upper adjacent image connected domain of the target image connected domain when the image connected domain values ​​in the left adjacent image connected domain and the upper adjacent image connected domain are both valid values, and determine the voxel connected domain value of the target image connected domain based on the comparison result.

[0145] Each module in the aforementioned point cloud data segmentation device may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a computer device memory in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0146] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 8As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for segmenting point cloud data is implemented. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0147] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0148] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0149] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0150] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0151] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0152] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0153] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0154] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for segmenting point cloud data, characterized in that: The method comprises: Determining a principal axis direction and a corresponding bounding box based on the point cloud data; constructing a plurality of voxels in a voxel space based on the bounding box; calculating an index of the point cloud data in the voxel space to obtain a mapped voxel space; and projecting the point cloud data in layers along the principal axis dimension in the mapped voxel space to obtain sequentially adjacent multi-layer images; Performing connected domain analysis on the images of each layer respectively to obtain connected domains of the images of each layer; Based on the connected domains of the images of each layer, voxel connected domains are analyzed on the images of each layer arranged in sequence to obtain voxel space connected domains; The point cloud data is segmented by the voxel spatial connected domain.

2. The method according to claim 1, characterized in that The determining of the principal axis direction and the corresponding bounding box based on the point cloud data includes: Analyze the principal components of the point cloud data to obtain the information content of the point cloud data in different dimensions; Determining the main axis direction based on the information amount of the different dimensions; A bounding box of the point cloud data is determined according to the main axis direction.

3. The method according to claim 1, characterized in that The connected domain analysis is performed on the images of each layer to obtain the connected domain of each layer of images, including: Performing connected domain analysis on the images of each layer respectively to obtain image connected domain analysis results; Determine a global connected component label based on the connected component labels and offset values ​​within each layer of the image; Based on the global connected domain label, the image connected domain analysis result is labeled to obtain the image connected domains of each layer.

4. The method according to claim 1, wherein The voxel connected domain analysis is performed on the sequentially arranged layers of images based on the connected domains of the layers of images, including: Based on the pixel positions in the connected domain of each layer of images and the pixel position set of the connected domain of the adjacent layer of images, it is determined in sequence whether the adjacent connected domains of images to which each pixel position belongs match, and adjacent connected domain matching results are obtained in sequence; wherein the pixel position set of the connected domain of the adjacent layer of images includes the target pixel position that is the same as the pixel position, and / or the neighboring pixel position of the target pixel position in the connected domain of the adjacent layer of images; Based on the matching results of the connected domains of each adjacent image obtained in sequence, it is determined whether the adjacent voxels of the connected domains of the adjacent images in each layer arranged in sequence match each other.

5. The method according to claim 4, characterized in that The method of judging whether adjacent voxels of adjacent images to which pixel positions of each layer are sequentially arranged match based on the connected domain matching results of each adjacent image obtained in sequence includes: Determining a plurality of associated image connected domains based on the matching results of the adjacent image connected domains; Determining the voxel connected domain value of each of the image connected domains in sequence based on the order in which the associated multiple image connected domains are arranged; Based on the connectivity domain value of each voxel, it is determined whether each adjacent voxel matches.

6. The method according to claim 5, characterized in that The step of sequentially determining the voxel connectivity domain value of each of the image connectivity domains based on the order in which the associated multiple image connectivity domains are arranged comprises: Based on the order in which the associated multiple image connected domains are arranged, sequentially determining the associated multiple image connected domains as target image connected domains; When a value of one of the image connected domains of the left neighboring image connected domain or the upper neighboring image connected domain of the target image connected domain is a valid value, updating the voxel connected domain value of the image connected domain based on the valid value; When the image connected domain values ​​in the left neighboring image connected domain and the upper neighboring image connected domain of the target image connected domain are both valid values, the image connected domain values ​​of the left neighboring image connected domain and the upper neighboring image connected domain are compared, and the voxel connected domain value of the target image connected domain is determined based on the comparison result.

7. A point cloud data segmentation device, characterized in that: The device comprises: An image generation module is configured to determine a principal axis direction and a corresponding bounding box based on the point cloud data; construct a plurality of voxels in a voxel space based on the bounding box; calculate an index of the point cloud data in the voxel space to obtain a mapped voxel space; and project the point cloud data in layers along the principal axis dimension in the mapped voxel space to obtain sequentially adjacent multi-layer images. An image connected domain generation module is used to perform connected domain analysis on the images of each layer to obtain connected domains of the images of each layer; a voxel connected domain generation module, configured to perform voxel connected domain analysis on the sequentially arranged layers of images based on the connected domains of the layers of images, to obtain voxel spatial connected domains; The point cloud data segmentation module is used to segment the point cloud data according to the voxel space connected domain.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images