Point cloud segmentation method and device and electronic equipment

Abstract

The disclosure provides a point cloud segmentation method, comprising: acquiring first point cloud feature information corresponding to an input point cloud, the first point cloud feature information comprising initial feature information of each target point in the input point cloud; the first point cloud feature information is processed through a target operator, second point cloud feature information is obtained, and the target operator has a frequency domain conversion processing process in the operation process; determining third point cloud feature information according to the second point cloud feature information, wherein the third point cloud feature information comprises transformation feature information corresponding to each target point; and determining a point cloud segmentation result of the input point cloud according to the third point cloud feature information. The invention further provides a point cloud segmentation device.

Description

Technical Field

[0001] The present disclosure relates to the field of neural network technology, and in particular to a point cloud segmentation method, device, electronic device, medium and product. Background Art

[0002] With the rapid development of artificial intelligence technology, autonomous driving technology has become the current research focus. Semantic segmentation of three-dimensional high-precision map scenes based on lidar can help autonomous vehicles better understand complex environments. However, the point cloud that makes up the three-dimensional scene is different from the grid structure of the image. It has characteristics such as disorder and unevenness, and image-related algorithms cannot be directly applied for semantic segmentation. Summary of the invention

[0003] In view of this, the present disclosure provides a point cloud segmentation method, device, electronic device, medium and product.

[0004] One aspect of the present disclosure provides a point cloud segmentation method, including: obtaining first point cloud feature information corresponding to an input point cloud, the first point cloud feature information including initial feature information of each target point in the input point cloud; processing the first point cloud feature information by a target operator to obtain second point cloud feature information, the target operator having a frequency domain conversion processing process during the operation; determining third point cloud feature information based on the second point cloud feature information, the third point cloud feature information including transformation feature information corresponding to each target point; and determining a point cloud segmentation result of the input point cloud based on the third point cloud feature information.

[0005] According to an embodiment of the present disclosure, determining third point cloud feature information based on second point cloud feature information includes: acquiring first point set feature information, the first point set feature information including feature information of each target point set, each target point set corresponding to a target point; processing the first point set feature information by a target operator to obtain second point set feature information; determining third point cloud feature information based on the second point cloud feature information and the second point set feature information.

[0006] According to an embodiment of the present disclosure, determining the third point cloud feature information based on the second point cloud feature information and the second point set feature information includes: obtaining the first position information corresponding to each target point; obtaining the second position information corresponding to each target point set; determining the position feature information based on the first position information and the second position information; determining the third point cloud feature information based on the position feature information, the second point cloud feature information and the second point set feature information.

[0007] According to an embodiment of the present disclosure, the third point cloud feature information is determined based on the position feature information, the second point cloud feature information and the second point set feature information, including: determining the first fusion feature information based on the second point cloud feature information and the second point set feature information; obtaining the second fusion feature information based on the position feature information and the first fusion feature information; determining the target weight information based on the second fusion information, the target weight information characterizing the degree of association between each target point and the corresponding target point set; and determining the third point cloud feature information based on the target weight information and the first point set feature information.

[0008] According to an embodiment of the present disclosure, determining position feature information based on first position information and second position information includes: concatenating a vector subtraction result and a scalar subtraction result between each first position information and the corresponding second position information to obtain the position feature information.

[0009] According to an embodiment of the present disclosure, obtaining the first point set feature information includes: performing a neighboring point search for each target point in the input point cloud to obtain a target point set corresponding to each target point; and determining to obtain the first point set feature information based on the target point set.

[0010] According to an embodiment of the present disclosure, the first point cloud feature information is processed respectively by a target operator to obtain the second point cloud feature information, including: dividing the target space corresponding to the input point cloud into evenly distributed grid subspaces; determining the feature sub-data corresponding to each grid subspace according to the target point in each grid subspace; determining the first point cloud feature information according to the feature sub-data; processing the first point cloud feature information by a target operator to obtain the second point cloud feature information.

[0011] According to an embodiment of the present disclosure, the first point cloud feature information is processed by a target operator to obtain the second point cloud feature information, including: performing a frequency domain transformation on the first point cloud feature information to obtain frequency domain point cloud feature information; performing a high-frequency signal filtering on the frequency domain point cloud feature information to obtain filtered point cloud feature information; and performing an inverse frequency domain transformation on the filtered point cloud feature information to obtain the second point cloud feature information.

[0012] According to an embodiment of the present disclosure, the target operator has weight parameters, and the weight parameters are obtained through a target training process. The target training process is a process of training a target point cloud segmentation network, and the target operator is a part of the target point cloud segmentation network.

[0013] Another aspect of the present disclosure also provides a point cloud segmentation device, including: an acquisition module, used to acquire first point cloud feature information corresponding to an input point cloud, the first point cloud feature information including initial feature information of each target point in the input point cloud; a processing module, used to process the first point cloud feature information through a target operator to obtain second point cloud feature information, the target operator having a frequency domain conversion processing process during the operation; a feature determination module, used to determine third point cloud feature information based on the second point cloud feature information, the third point cloud feature information including transformation feature information corresponding to each target point; a result determination module, used to determine the point cloud segmentation result of the input point cloud based on the third point cloud feature information.

[0014] Another aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above method.

[0015] Another aspect of the present disclosure further provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to perform the above method.

[0016] Another aspect of the present disclosure further provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0017] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0019] Figure 1 is a schematic diagram of an exemplary system architecture to which a point cloud segmentation method and apparatus can be applied according to an embodiment of the present disclosure;

[0020] Figure 2 A flowchart of a point cloud segmentation method according to an embodiment of the present disclosure is schematically shown;

[0021] Figure 3 A schematic diagram of a target point cloud segmentation network according to an embodiment of the present disclosure is schematically shown;

[0022] Figure 4 A flowchart of processing first point cloud feature information according to an embodiment of the present disclosure is schematically shown;

[0023] Figure 5A schematic diagram schematically shows a point cloud segmentation method according to an embodiment of the present disclosure;

[0024] Figure 6 A structural block diagram of a point cloud segmentation device according to an embodiment of the present disclosure is schematically shown; and

[0025] Figure 7 A schematic block diagram of an electronic device that can be used to implement the point cloud segmentation method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0026] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered as merely exemplary. Therefore, it should be appreciated by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Likewise, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0027] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision, disclosure and application of the data involved (including but not limited to user personal information) comply with the provisions of relevant laws and regulations, take necessary confidentiality measures, and do not violate public order and good morals.

[0028] Figure 1 is a schematic diagram of an exemplary system architecture to which a point cloud segmentation method and apparatus can be applied according to an embodiment of the present disclosure. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0029] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0030] The user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send image data, point cloud data, etc. The terminal devices 101, 102, 103 can be electronic devices or vehicles with display screens and point cloud acquisition devices, including but not limited to smartphones, tablet computers, desktop computers, and autonomous vehicles, etc.

[0031] Server 105 can be a server that provides various services, such as segmenting the three-dimensional lidar point cloud sent by the user using the terminal devices 101, 102, and 103, analyzing and processing the received point cloud, image and other data, and feeding back the processed segmentation results to the terminal device.

[0032] It should be noted that the point cloud segmentation method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the point cloud segmentation device provided in the embodiment of the present disclosure can generally be set in the server 105. The point cloud segmentation method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the point cloud segmentation device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0033] Figure 2 A flowchart of a point cloud segmentation method according to an embodiment of the present disclosure is schematically shown.

[0034] like Figure 2 As shown, the point cloud segmentation method of this embodiment includes operations S210 to S240.

[0035] In operation S210, first point cloud feature information corresponding to the input point cloud is obtained, where the first point cloud feature information includes initial feature information of each target point in the input point cloud.

[0036] In the embodiments of the present disclosure, the input point cloud may be three-dimensional or two-dimensional point cloud data acquired by an acquisition device such as a laser radar or a stereo camera. The target point may refer to a plurality of points obtained by downsampling the input point cloud, or may be part of the point data received from the previous segmentation network in the U-net architecture. The initial feature information may refer to feature information acquired by the acquisition device, including geometric information, color, texture, direction, etc.

[0037] In operation S220, the first point cloud feature information is processed by a target operator to obtain second point cloud feature information, and the target operator has a frequency domain conversion processing process during the operation process.

[0038] In the embodiments of the present disclosure, the target operator may be a component module in the point cloud segmentation network, which has the function of converting information into the frequency domain for further processing. The second point cloud feature information may refer to feature information obtained after intercepting, filtering, linearly transforming, etc. the first point cloud feature information in the frequency domain.

[0039] For example, the target operator may be a neural operator having a frequency domain conversion processing function, such as FNO (Fourier Neural Operators) and AFNO (Adapted Fourier Neural Operators).

[0040] For example, the first point cloud feature information is input into the target operator, and the target operator first converts the first point cloud feature information from the spatial domain to the frequency domain, and performs interception processing on the converted information. Then, the information is converted from the frequency domain to the spatial domain to obtain the processed second point cloud feature information.

[0041] In operation S230, third point cloud feature information is determined according to the second point cloud feature information, where the third point cloud feature information includes transformation feature information corresponding to each target point.

[0042] In the embodiment of the present disclosure, the third point cloud feature information refers to feature information output by the segmentation network. The transformed feature information may refer to feature information obtained by transforming feature learning based on the second point cloud feature information.

[0043] For example, the third point cloud feature information may be feature information obtained by processing the second point cloud feature information based on a multi-layer perceptron, a convolution or other method.

[0044] In operation S240 , a point cloud segmentation result of the input point cloud is determined according to the third point cloud feature information.

[0045] In the embodiments of the present disclosure, semantic segmentation may be performed based on the third point cloud features output by the segmentation network to obtain a point cloud segmentation result of the input point cloud. Alternatively, the third point cloud feature information output by each layer of the segmentation network may be obtained in the U-net architecture, and processed to obtain the final point cloud segmentation result.

[0046] According to the embodiments of the present disclosure, by converting the feature information of the input point cloud from the spatial domain to the frequency domain for processing, and then converting it back to the spatial domain for subsequent feature learning, semantic segmentation and other processing, the segmentation network can learn the point cloud features more effectively, thereby improving the accuracy of the lidar point cloud segmentation.

[0047] Figure 3 A schematic diagram of a target point cloud segmentation network according to an embodiment of the present disclosure is schematically shown.

[0048] like Figure 3As shown, the point cloud segmentation method of this embodiment can be implemented by a point cloud segmentation network 300. The first position information 301, the second position information 302, the first point cloud feature information 303 and the first point set feature information 304 are inputs of the point cloud segmentation network 300, and the point cloud segmentation network processes the above information to output third point cloud feature information 311.

[0049] According to an embodiment of the present disclosure, the first point set feature information 304 can be obtained by the following method: performing a neighboring point search for each target point in the input point cloud to obtain a target point set corresponding to each target point; and determining to obtain the first point set feature information 304 based on the target point set.

[0050] In the embodiment of the present disclosure, the target point can be obtained by grouping or randomly sampling the input point cloud. Through the adjacent point search, multiple points located near the target point and close to the target point feature information can be obtained from the input point cloud to form a target point set corresponding to the target point. The first point set feature information 304 includes feature information of each point in the target point set.

[0051] For example, a KNN (K-Nearest Neighbor) algorithm may be used to obtain K points closest to the target point to form a target point set, and the feature information of the target point and the corresponding K points is used as the first point set feature information 304 .

[0052] According to an embodiment of the present disclosure, determining the third point cloud feature information 310 based on the second point cloud feature information 305 includes: acquiring the first point set feature information 304, the first point set feature information 304 including feature information of each target point set, each target point set corresponding to a target point; processing the first point set feature information 304 by a target operator to obtain the second point set feature information 306; and determining the third point cloud feature information 311 based on the second point cloud feature information 305 and the second point set feature information 306.

[0053] In the embodiment of the present disclosure, the second point cloud feature information 305 is obtained by processing the first point cloud feature information 303 based on the target operator. Similarly, the second point set feature information 306 is obtained by processing the first point set feature information 304 based on the target operator.

[0054] For example, the first point set feature information 304 is input into the target operator, and the target operator first converts the first point set feature information 304 from the spatial domain to information in the frequency domain, and performs processing such as interception, filtering, and linear transformation on the converted information. Then, the information is converted from the frequency domain to the spatial domain to obtain the processed second point set feature information 306.

[0055] According to an embodiment of the present disclosure, the third point cloud feature information 311 is determined based on the second point cloud feature information 305 and the second point set feature information 306, including: obtaining the first position information 301 corresponding to each target point; obtaining the second position information 302 corresponding to each target point set; determining the position feature information 310 based on the first position information 301 and the second position information 302; determining the third point cloud feature information based on the position feature information, the second point cloud feature information and the second point set feature information.

[0056] In the embodiment of the present disclosure, the first position information 301 may refer to the position information of each target point in the input point cloud. The second position information 302 may refer to the position information of each point in each target point set. The position feature information 310 may characterize the spatial relationship between points determined based on the first position information 301 and the second position information 302.

[0057] For example, vector addition, vector subtraction, encoding and other calculations may be performed on the first position information 301 and the second position information 302 to obtain the position feature information 310 .

[0058] According to an embodiment of the present disclosure, the third point cloud feature information 311 is determined according to the position feature information 307, the second point cloud feature information 305 and the second point set feature information 306, including: determining the first fusion feature information 308 according to the second point cloud feature information 305 and the second point set feature information 306; obtaining the second fusion feature information 309 according to the position feature information 307 and the first fusion feature information 308; determining the target weight information 310 according to the second fusion information 309, the target weight information 310 characterizing the degree of association between each target point and the corresponding target point set; and determining the third point cloud feature information 311 according to the target weight information 309 and the first point set feature information 304.

[0059] In the embodiment of the present disclosure, the second point cloud feature information and the second point set feature information can be subjected to vector addition, vector subtraction, dot product, convolution and other calculations through the relationship function to obtain the first fused feature information. Subsequently, the first fused feature information and the position feature information are subjected to vector addition, vector subtraction, dot product, convolution and other calculations through the relationship function to obtain the second fused feature information.

[0060] For example, the first fused feature information is obtained by performing vector subtraction on the second point cloud feature information and the second point set feature information, and then the second fused feature information is obtained by performing vector addition on the first fused feature information and the position feature information.

[0061] In the embodiment of the present disclosure, the target weight information 310 refers to a weight vector generated based on the second fused feature information. The weight score can be generated by a fixed formula or a learnable parameter, and the weight score is normalized according to the softmax function to obtain the target weight information 310. The target weight information 310 and the first point set feature information 304 are input into the grouping aggregation module. After the input first point set feature information 304 is grouped, the first point set feature information 304 is aggregated by calculating the average value, the maximum value, the weighted sum, etc. to obtain the third point cloud feature information 311. The target weight information represents the importance of different points in the input point cloud through the weight vector, thereby improving the accuracy of point cloud segmentation.

[0062] According to the embodiments of the present disclosure, the feature information of the point cloud and the point set is processed in the frequency domain through the target operator, and the point cloud features are fused with the position information of the point cloud, so as to realize learning of the point cloud features in the frequency domain and the spatial domain respectively, thereby improving the accuracy of the lidar point cloud segmentation.

[0063] In some embodiments of the present disclosure, the third point cloud feature information can also be determined based on the self-attention module. The target operator is used to replace the query matrix and the key matrix in the self-attention module, and the first point cloud feature information 303 and the first point set feature information 304 are processed respectively to obtain the processed second point cloud feature information 305 and the second point set feature information 306. After vector calculation and weight encoding according to the second point cloud feature information 305, the second point set feature information 306 and the position feature information 307, the target weight information 310 is obtained. The first point set feature information 304 is input into the value matrix calculation module for multiplication to obtain the value vector, and the value vector and the target weight information 310 are grouped and aggregated to obtain the third point cloud feature information after processing based on the self-attention module.

[0064] According to the embodiments of the present disclosure, a self-attention mechanism is introduced to process the feature information of the point cloud, thereby helping the segmentation network to learn the point cloud features more effectively, and through learnable parameters, a dynamic weight score that can be adjusted based on the model is generated for the fused feature information, thereby further improving the accuracy of the point cloud segmentation.

[0065] According to an embodiment of the present disclosure, determining position feature information based on first position information and second position information includes: concatenating a vector subtraction result and a scalar subtraction result between each first position information and the corresponding second position information to obtain the position feature information.

[0066] In the embodiment of the present disclosure, the concatenation information of the first position information and the corresponding second position information can also be input into a multi-layer perceptron for processing, and the position code obtained after the processing is used as the position feature information. The position code can be calculated by the following formula:

[0067]

[0068] in, is the position encoding of the i-th target point, MLP represents a multi-layer perceptron, represents the vector subtraction result of the position information of the i-th target point and the second position information of the j-th point in the corresponding target point set, Represents the scalar subtraction result of the position information of the i-th target point and the second position information of the j-th point in the corresponding target point set.

[0069] According to an embodiment of the present disclosure, the target operator has weight parameters, and the weight parameters are obtained through a target training process. The target training process is a process of training a target point cloud segmentation network. The target operator is a part of the target point cloud segmentation network 300.

[0070] In the embodiments of the present disclosure, the weight parameters in the target operator can dynamically process the input first point cloud feature information and the first point set feature information. For example, the linear change processing of the first point cloud feature information after frequency domain conversion can be determined based on the weight parameters. When training the target point cloud segmentation network, the weight parameters in the target operator will also be adjusted based on the iteration of the model, so as to realize the dynamic processing of the point cloud feature information by the target operator.

[0071] Next, combine Figure 4 The target operator processing feature information is further explained.

[0072] Figure 4 A flowchart for processing first point cloud feature information according to an embodiment of the present disclosure is schematically shown.

[0073] According to the embodiments of the present disclosure, Figure 4 As shown, the first point cloud feature information 401 is processed by the target operator 410 to obtain the second point cloud feature information 404, including: performing frequency domain transformation on the first point cloud feature information 401 to obtain frequency domain point cloud feature information 402; performing high frequency signal filtering on the frequency domain point cloud feature information 402 to obtain filtered point cloud feature information 403; performing inverse frequency domain transformation on the filtered point cloud feature information 403 to obtain the second point cloud feature information 404.

[0074] In the embodiment of the present disclosure, after the first point cloud feature information 401 is transformed in the frequency domain to obtain the frequency domain point cloud feature information 402 , the frequency domain point cloud feature information 402 may be processed based on the weight parameter of the target operator 410 .

[0075] For example, a filtering threshold is determined based on a weight parameter, a high-frequency signal portion higher than the filtering threshold is filtered, and a low-frequency signal portion lower than the filtering threshold is linearly changed to obtain processed filtered point cloud feature information.

[0076] In the embodiments of the present disclosure, the feature information of the point cloud is processed in the frequency domain by the target operator, which can help the segmentation network to learn the point cloud features more effectively and improve the accuracy of the lidar point cloud segmentation.

[0077] In the embodiment of the present disclosure, since the target operator needs to process the input point cloud features in the frequency domain, the input data should be in an ordered structure. A point cloud with disordered characteristics cannot be directly input to the target operator for processing, so the point cloud data needs to be processed.

[0078] According to an embodiment of the present disclosure, the first point cloud feature information is processed respectively by the target operator to obtain the second point cloud feature information, including: dividing the target space corresponding to the input point cloud into uniformly distributed grid subspaces; determining the feature sub-data corresponding to each grid subspace according to the target point in each grid subspace; determining the input point cloud feature information according to the feature sub-data; processing the input point cloud feature information by the target operator to obtain the second point cloud feature information.

[0079] Next, combine Figure 5 The point cloud segmentation method of the present application is further explained.

[0080] Figure 5 A schematic diagram of a point cloud segmentation method according to an embodiment of the present disclosure is schematically shown.

[0081] like Figure 5 As shown, when inputting the segmentation network, the input point cloud 501 needs to be downsampled first. Therefore, after the space where the input point cloud is located is rasterized in the three-dimensional space where the input point cloud is located, a uniformly distributed grid subspace is obtained, and only one point cloud is retained in each grid subspace, and the rasterized input point cloud 511 is obtained. After downsampling, the sequence number of the grid corresponding to each point cloud can be obtained, and the index of the point cloud matrix can be obtained according to the sequence number, thereby converting the disordered point cloud into an ordered three-dimensional grid structure.

[0082] After the input point cloud is rasterized, it is input to other processing modules in the point cloud segmentation network for processing to obtain processed point cloud feature information 512 and third point cloud feature information 502 obtained after removing the rasterized form.

[0083] According to an embodiment of the present disclosure, the input point cloud is rasterized and serially encoded to convert the disordered point cloud into an ordered point cloud capable of frequency domain conversion, thereby converting the point cloud features to the frequency domain space for processing and feature learning, thereby improving the accuracy of point cloud segmentation.

[0084] Figure 6 The structural block diagram of the point cloud segmentation device according to an embodiment of the present disclosure is schematically shown.

[0085] like Figure 6 As shown, the point cloud segmentation device 600 of this embodiment includes an acquisition module 610 , a processing module 620 , a feature determination module 630 and a result determination module 640 .

[0086] The acquisition module 610 is used to acquire first point cloud feature information corresponding to the input point cloud, where the first point cloud feature information includes initial feature information of each target point in the input point cloud. In one embodiment, the acquisition module 610 can be used to perform the operation S210 described above, which will not be described in detail here.

[0087] The processing module 620 is used to process the first point cloud feature information through the target operator to obtain the second point cloud feature information. The target operator has a frequency domain conversion process during the operation. In one embodiment, the processing module 620 can be used to perform the operation S220 described above, which will not be repeated here.

[0088] The feature determination module 630 is used to determine the third point cloud feature information according to the second point cloud feature information, wherein the third point cloud feature information includes the transformation feature information corresponding to each target point. In one embodiment, the feature determination module 630 can be used to perform the operation S230 described above, which will not be described in detail here.

[0089] The result determination module 640 is used to determine the point cloud segmentation result of the input point cloud according to the third point cloud feature information. In one embodiment, the result determination module 640 can be used to perform the operation S240 described above, which will not be described in detail here.

[0090] According to an embodiment of the present disclosure, the feature determination module 630 includes a point set feature acquisition submodule, a point set feature processing submodule and a point cloud feature determination submodule. The point set feature acquisition submodule is used to acquire first point set feature information, the first point set feature information includes feature information of each target point set, and each target point set corresponds to a target point. The point set feature processing submodule is used to process the first point set feature information through a target operator to obtain second point set feature information. The point cloud feature determination submodule is used to determine third point cloud feature information based on the second point cloud feature information and the second point set feature information.

[0091] According to an embodiment of the present disclosure, the point cloud feature determination submodule includes a point cloud position acquisition unit, a point set position acquisition unit, a position feature determination unit and a point cloud feature determination unit. The point cloud position acquisition unit is used to obtain the first position information corresponding to each target point. The point set position acquisition unit is used to obtain the second position information corresponding to each target point set. The position feature determination unit is used to determine the position feature information according to the first position information and the second position information. The point cloud feature determination unit is used to determine the third point cloud feature information according to the position feature information, the second point cloud feature information and the second point set feature information.

[0092] According to an embodiment of the present disclosure, a point cloud feature determination unit includes a first fusion subunit, a second fusion subunit, a weight determination unit, and a point cloud feature determination unit. The first fusion subunit is used to determine first fusion feature information based on second point cloud feature information and second point set feature information. The second fusion subunit is used to obtain second fusion feature information based on position feature information and first fusion feature information. The weight determination unit is used to determine target weight information based on the second fusion information, and the target weight information characterizes the degree of association between each target point and the corresponding target point set. The point cloud feature determination unit is used to determine third point cloud feature information based on the target weight information and the first point set feature information.

[0093] According to an embodiment of the present disclosure, the position feature determination unit includes an information splicing subunit, which is used to splice the vector subtraction result and the scalar subtraction result between each first position information and the corresponding second position information to obtain the position feature information.

[0094] According to an embodiment of the present disclosure, the point set feature acquisition submodule includes a search unit for performing a neighboring point search for each target point in the input point cloud to obtain a target point set corresponding to each target point; and determining to obtain first point set feature information based on the target point set.

[0095] According to an embodiment of the present disclosure, the processing module 620 includes a space division submodule, a target point determination submodule, a feature determination submodule and an information processing submodule. The space division submodule is used to divide the target space corresponding to the input point cloud into uniformly distributed grid subspaces. The target point determination submodule is used to determine the feature sub-data corresponding to each grid subspace according to the target point in each grid subspace. The feature determination submodule is used to determine the input point cloud feature information according to the feature sub-data. The information processing submodule is used to process the input point cloud feature information through the target operator to obtain the second point cloud feature information.

[0096] According to an embodiment of the present disclosure, the information processing submodule includes a transformation unit, a filtering unit and an inverse transformation unit. The transformation unit is used to perform a frequency domain transformation on the input point cloud feature information to obtain frequency domain point cloud feature information. The filtering unit is used to perform a high frequency signal filtering on the frequency domain point cloud feature information to obtain filtered point cloud feature information. The inverse transformation unit is used to perform an inverse frequency domain transformation on the filtered point cloud feature information to obtain a second point cloud feature information.

[0097] According to an embodiment of the present disclosure, the target operator has weight parameters, and the weight parameters are obtained through a target training process. The target training process is a process of training a target point cloud segmentation network, and the target operator is a part of the target point cloud segmentation network.

[0098] Figure 7 A schematic block diagram of an electronic device that can be used to implement the point cloud segmentation method according to an embodiment of the present disclosure is schematically shown.

[0099] like Figure 7 As shown, the electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 to a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include an onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0100] In RAM 703, various programs and data required for the operation of electronic device 700 are stored. Processor 701, ROM 702 and RAM 703 are connected to each other via bus 704. Processor 701 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 702 and / or RAM 703. It should be noted that the program can also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 can also implement the method provided by the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0101] According to an embodiment of the present disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to the bus 704. The electronic device 700 may further include one or more of the following components connected to the I / O interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 708 including a hard disk, etc.; and a communication portion 709 including a network interface card such as a LAN card, a modem, etc. The communication portion 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage portion 708 as needed.

[0102] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0103] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 702 and / or RAM 703 described above and / or one or more memories other than ROM 702 and RAM 703.

[0104] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method provided by the embodiment of the present disclosure.

[0105] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 701. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0106] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 709, and / or installed from the removable medium 711. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0107] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0108] It should be noted that the collection, storage, use, processing, transmission, provision, disclosure and application of user personal information involved in the technical solution of this disclosure are in compliance with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and they do not violate public order and good morals. In the technical solution of this disclosure, the user's authorization or consent is obtained before obtaining or collecting user personal information.

[0109] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0110] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0111] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or coupled in a variety of ways, even if such combinations and / or couplings are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or coupled in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or couplings fall within the scope of the present disclosure.

[0112] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination. The scope of the present disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A point cloud segmentation method, comprising: Acquire first point cloud feature information corresponding to the input point cloud, wherein the first point cloud feature information includes initial feature information of each target point in the input point cloud; Processing the first point cloud feature information by a target operator to obtain second point cloud feature information, wherein the target operator has a frequency domain conversion processing process during the operation; Determine third point cloud feature information according to the second point cloud feature information, wherein the third point cloud feature information includes transformation feature information corresponding to each of the target points; A point cloud segmentation result of the input point cloud is determined according to the third point cloud feature information.

2. The method according to claim 1, wherein determining the third point cloud feature information according to the second point cloud feature information comprises: Acquire first point set feature information, where the first point set feature information includes feature information of each target point set, and each target point set corresponds to one target point; Processing the first point set feature information by the target operator to obtain second point set feature information; The third point cloud feature information is determined according to the second point cloud feature information and the second point set feature information.

3. The method according to claim 2, wherein determining the third point cloud feature information according to the second point cloud feature information and the second point set feature information comprises: Obtaining first position information corresponding to each of the target points; Obtaining second position information corresponding to each of the target point sets; Determine location feature information according to the first location information and the second location information; The third point cloud feature information is determined according to the position feature information, the second point cloud feature information and the second point set feature information.

4. The method according to claim 3, wherein determining the third point cloud feature information according to the position feature information, the second point cloud feature information and the second point set feature information comprises: Determine first fusion feature information according to the second point cloud feature information and the second point set feature information; Obtaining second fused feature information according to the position feature information and the first fused feature information; Determine target weight information according to the second fusion information, where the target weight information represents the degree of association between each target point and the corresponding target point set; The third point cloud feature information is determined according to the target weight information and the first point set feature information.

5. The method according to claim 3, wherein determining the location feature information according to the first location information and the second location information comprises: The vector subtraction result and the scalar subtraction result between each of the first position information and the corresponding second position information are concatenated to obtain the position feature information.

6. The method according to claim 2, wherein obtaining the first point set feature information comprises: Performing a neighboring point search for each of the target points in the input point cloud to obtain the target point set corresponding to each of the target points; The first point set feature information is obtained according to the target point set.

7. The method according to claim 1, wherein the step of processing the first point cloud feature information by a target operator to obtain the second point cloud feature information comprises: Dividing the target space corresponding to the input point cloud into evenly distributed grid subspaces; Determine the feature sub-data corresponding to each of the grid sub-spaces according to the target point in each of the grid sub-spaces; Determine the first point cloud feature information according to the feature sub-data; The first point cloud feature information is processed by the target operator to obtain the second point cloud feature information.

8. The method according to claim 7, wherein the step of processing the first point cloud feature information by the target operator to obtain the second point cloud feature information comprises: Performing frequency domain transformation on the first point cloud feature information to obtain frequency domain point cloud feature information; Performing high-frequency signal filtering on the frequency domain point cloud feature information to obtain filtered point cloud feature information; Perform an inverse frequency domain transformation on the filtered point cloud feature information to obtain the second point cloud feature information.

9. According to the method of claim 1, the target operator has a weight parameter, and the weight parameter is obtained through a target training process, and the target training process is a process of training a target point cloud segmentation network, and the target operator is a part of the target point cloud segmentation network.

10. A point cloud segmentation device, comprising: An acquisition module, used to acquire first point cloud feature information corresponding to the input point cloud, wherein the first point cloud feature information includes initial feature information of each target point in the input point cloud; A processing module, used for processing the first point cloud feature information through a target operator to obtain second point cloud feature information, wherein the target operator has a frequency domain conversion processing process during the operation process; a feature determination module, configured to determine third point cloud feature information according to the second point cloud feature information, wherein the third point cloud feature information includes transformation feature information corresponding to each of the target points; A result determination module is used to determine the point cloud segmentation result of the input point cloud according to the third point cloud feature information.

Images