Point cloud-based semantic segmentation method, device, system and storage medium

Abstract

The present invention provides a point cloud-based semantic segmentation method, device, system, and storage medium. The method comprises: acquiring point cloud data of a target object; sequentially performing feature aggregation and sampling on different ranges of points in the point cloud data to obtain aggregated features of the sampling points; fusing the aggregated features of the sampling points into the point set before sampling the sampling points to obtain updated features; and obtaining recognition results of the target to be detected based on the updated features. According to the method, device, system, and storage medium of the present invention, the receptive field of a point is expanded by sequentially sampling and feature aggregation on point sets of different ranges, thereby obtaining richer spatial structure information, thereby achieving better performance and results in point cloud semantic segmentation.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and more particularly to semantic segmentation based on point clouds. Background Art

[0002] In recent years, research on three-dimensional point clouds has developed rapidly. Compared to 2D images, three-dimensional point clouds have shown superior performance in applications such as autonomous driving, home robots, and three-dimensional reconstruction. However, the three-dimensional visual data collected by data acquisition devices is highly irregular. Traditional methods typically convert three-dimensional point cloud data into regular data representations, such as voxel grids, multi-view images, and projections. These regular data are then processed using traditional convolutional neural networks (CNNs). However, these operations do not fully utilize the sparse nature of point clouds, resulting in unnecessary large memory usage and computational overhead. Furthermore, the conversion to regular data inevitably loses the spatial structural characteristics of the point cloud, which deprives point clouds, which are rich in spatially structured information, of their inherent characteristics. Summary of the Invention

[0003] The present invention is proposed in view of the above problems. The present invention provides a semantic segmentation method, device, system and computer storage medium based on point cloud to solve at least one of the above problems.

[0004] According to a first aspect of the present invention, a semantic segmentation method based on point cloud is provided, comprising:

[0005] Obtain point cloud data of the target object;

[0006] Performing feature aggregation and sampling on different ranges of points in the point cloud data in sequence to obtain aggregated features of the sampling points;

[0007] Fusing the aggregated features of the sampling points into the point set before sampling the sampling points to obtain updated features;

[0008] A recognition result of the target to be detected is obtained based on the updated features.

[0009] Optionally, feature aggregation and sampling are sequentially performed on different ranges of points in the point cloud data to obtain aggregated features of the sampling points, including:

[0010] The different ranges of points in the point cloud data are sequentially processed to perform feature aggregation and sampling, and the processing result of the previous range is used as the input of the current range; wherein, for each range,

[0011] Establishing a first spherical coordinate system with a radius R with each point as the center, dividing the spherical area of ​​the first spherical coordinate system into a plurality of first areas and setting corresponding first weights;

[0012] updating the feature of each point based on the first weight of the first area;

[0013] Sampling the updated points to obtain sampling points;

[0014] Based on the sampling point, adjacent points are searched in the first spherical coordinate system to obtain an adjacent point set, and a maximum value in the adjacent point set is used as an aggregate feature of the sampling point.

[0015] Optionally, updating the feature of each point based on the first weight of the first area includes:

[0016] The feature of each point in each first region is multiplied by the weight of the first region and the sum is calculated, and then divided by the number of points in the range to update the feature of each point in the range.

[0017] Optionally, sampling the updated points to obtain sampling points includes:

[0018] The sampling point is obtained based on the farthest point sampling after the updated point sampling.

[0019] Optionally, different ranges have different expansion rates and / or different radii R of the first spherical coordinate system.

[0020] Optionally, the aggregated features of the sampling points are fused into the point set before the sampling points are sampled to obtain updated features, including:

[0021] The aggregated features of the sampling points are sequentially processed and fused into the point set before the sampling point is sampled; wherein, for each fusion,

[0022] Determine the sampling source point set of the input point set;

[0023] Establishing a second spherical coordinate system on the input point set with a point in the sampling source point set as an origin, dividing the spherical area of ​​the second spherical coordinate system into a plurality of second areas, and setting corresponding second weights;

[0024] Based on the second weight of the second area, the feature of each point within the source range is updated.

[0025] Optionally, the determining the sampling source point set of the input point set includes: determining by retrieval that the input point set is obtained by sampling the sampling source point set.

[0026] According to a second aspect of the present invention, a point cloud-based semantic segmentation apparatus is provided, comprising:

[0027] include:

[0028] Point cloud module, used to obtain point cloud data of the target object;

[0029] A feature aggregation layer is used to perform feature aggregation and sampling on different ranges of points in the point cloud data in sequence to obtain aggregated features of the sampling points;

[0030] A feature propagation layer is used to fuse the aggregated features of the sampling points into the point set before sampling at the sampling points to obtain updated features;

[0031] The recognition module is used to obtain the recognition result of the target to be detected based on the updated features.

[0032] According to a third aspect of the present invention, a point cloud-based semantic segmentation system is provided, comprising a memory, a processor, and a computer program stored on the memory and running on the processor, wherein the processor implements the steps of the method according to the first aspect when executing the computer program.

[0033] According to a fourth aspect of the present invention, there is provided a computer storage medium having a computer program stored thereon, wherein the computer program implements the steps of the method according to the first aspect when executed by a computer.

[0034] According to the point cloud-based semantic segmentation method, device, system and storage medium of the embodiments of the present invention, the receptive field of the point is expanded by sampling and feature aggregation of point sets in different ranges in sequence, so that richer spatial structure information can be obtained to achieve better performance and effects in point cloud semantic segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The above and other objects, features, and advantages of the present invention will become more apparent through a more detailed description of the embodiments of the present invention with reference to the accompanying drawings. The accompanying drawings are provided to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and are not intended to limit the present invention. In the drawings, the same reference numerals generally represent the same components or steps.

[0036] Figure 1 is a schematic flowchart of a point cloud-based semantic segmentation method according to an embodiment of the present invention;

[0037] Figure 2 is a schematic block diagram of a point cloud-based semantic segmentation apparatus according to an embodiment of the present invention;

[0038] Figure 3 is an example of a spherical coordinate system according to an embodiment of the present invention;

[0039] Figure 4 is a two-dimensional schematic diagram of multiple regions according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the present invention more apparent, exemplary embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described in the present invention, all other embodiments obtained by those skilled in the art without creative work should fall within the scope of protection of the present invention.

[0041] Graph convolutional neural networks can be broadly categorized into two types: spectral networks and spatial networks. The spectral network transforms two spatial signals into spectral signals through a Fourier transform on the graph. Multiplying the two spectral signals and then performing an inverse Fourier transform yields a value equivalent to the convolution of the two spatial signals. The Fourier transform basis of the graph is determined by the Laplacian matrix. The spatial network performs convolution directly. However, applying spectral convolution to point clouds requires calculating and analyzing the Laplacian matrix for each point. This process is computationally expensive and difficult to implement for real-world point clouds. Therefore, in practical applications, spatial-domain graph convolution is more attractive than spectral-domain graph convolution.

[0042] In order to better capture the spatial structural features of point clouds, constructing three-dimensional point clouds can capture the spatial structural features of sparse point clouds to the greatest extent. However, designing effective modules, such as convolution, pooling, and upsampling layers, has become a major challenge facing graph-based convolutional networks. These modules are expected to perform pixel operations on irregular point clouds similar to those of CNNs on regular images. In early theoretical research, M. Defferrard et al. implemented graph convolutional neural networks. However, these traditional graph convolutional neural networks cannot effectively process real-world point clouds, and no feasible solution has been found to solve this problem in point clouds.

[0043] Spatial graph convolution is challenged by the unavailability of discrete convolution kernels in three-dimensional space. To circumvent this problem, some existing methods use micro-networks to dynamically generate filters for each edge. However, this incurs a large amount of computational overhead, which can be avoided if discrete kernels are available. Among them, the PointCNN proposed by Y. Li et al. uses convolution kernels to process point clouds. However, its kernel is essentially a micro-network, which incurs a large computational cost. In addition, the network is very sensitive to the order of neighborhood points, which makes the network unable to adapt to the transformation of the point cloud scale. This ultimately leads to poor performance and performance in processing point clouds in the real world.

[0044] Based on the above considerations, the embodiment of the present invention provides a semantic segmentation method based on point cloud, and refers to Figure 1 A schematic flow chart of a semantic segmentation method based on point cloud according to an embodiment of the present invention is described. Figure 1 As shown, a semantic segmentation method based on point cloud 1 includes:

[0045] Step S1-1, obtaining point cloud data of the target object;

[0046] Step S1-2, performing feature aggregation and sampling on different ranges of points in the point cloud data in sequence to obtain aggregated features of the sampling points;

[0047] Step S1-3, fusing the aggregated features of the sampling points into the point set before sampling the sampling points to obtain updated features;

[0048] Step S1-4: obtaining a recognition result of the target to be detected based on the updated features.

[0049] Among them, traditional methods using spherical kernel-based graph convolutional networks on point clouds divide the adjacent space around a point into several distinct regions and assign a learnable weight to each region. This operation strictly limits the size of the point's receptive field and results in a high computational load. The point cloud-based object recognition method provided by embodiments of the present invention utilizes sequential sampling and feature aggregation of point sets from different ranges to expand the point's receptive field, thereby obtaining richer spatial structural information. In this case, only a portion of the several distinct regions are assigned learnable weights, while the rest are discarded. This method effectively reduces the computational load. Furthermore, the radius of the spherical kernel is increased to expand the receptive field of the point. However, since only a portion of the spherical kernel is selectively utilized, even if the size of the spherical kernel increases the total number of regions, the computational load does not increase significantly. This method is suitable for widespread application in any application requiring point cloud-based object recognition.

[0050] See also Figure 2 , Figure 2 FIG. 1 shows a schematic block diagram of a semantic segmentation device based on point cloud according to an embodiment of the present invention. Figure 2 As shown, the semantic segmentation device based on point cloud may include:

[0051] Point cloud module 101, used to obtain point cloud data of the target object;

[0052] Feature aggregation layer modules 102-105 are used to perform feature aggregation and sampling on different ranges of points in the point cloud data in sequence to obtain aggregated features of the sampling points;

[0053] Feature propagation layers 106-109 are used to fuse the aggregated features of the sampling points into the point set before sampling at the sampling points to obtain updated features;

[0054] The recognition module 110 is configured to obtain a recognition result of the target to be detected based on the updated features.

[0055] Only the main functional modules of the point cloud-based semantic segmentation device are described here. The point cloud-based semantic segmentation device according to an embodiment of the present invention is used to implement the point cloud-based semantic segmentation method according to an embodiment of the present invention, and the repeated parts will not be repeated here.

[0056] The following combination Figure 1 and Figure 2 Provide detailed explanation.

[0057] According to an embodiment of the present invention, in step S1 - 1 , point cloud data of a target object is acquired.

[0058] Point cloud data is generated by 3D vision sensors (such as Lidar and Matterport). These sensors can continuously, automatically, contactlessly, and rapidly collect large amounts of 3D point cloud data on the surface of target objects.

[0059] According to an embodiment of the present invention, Figure 2 As shown, in step S1-1, the method may further include: obtaining a point set of a point cloud scene and segmenting the point set to obtain a plurality of (ie, at least one) point cloud blocks;

[0060] Each point cloud block is used as the point cloud data.

[0061] In some embodiments, the segmentation may be performed by a point cloud module.

[0062] Specifically, since a point cloud scene usually has tens of millions of laser scattered points, it is impossible to directly input such a large-scale point cloud into the network. It is possible to divide a point cloud scene into several blocks through the point cloud module, and the number of point clouds in each block is about 10,000 points, for example. Each time, these 10,000 or so points are randomly sampled to obtain 8192 points as input data and processed according to the method according to the embodiment of the present invention. In addition, each point in each point cloud can be initially characterized as an xyz coordinate value, which can also be used as input data for processing according to the method according to the embodiment of the present invention.

[0063] According to an embodiment of the present invention, in step S1-2, feature aggregation and sampling are performed on different ranges of points in the point cloud data in sequence to obtain aggregated features of the sampling points.

[0064] Wherein, step S1-2 may be performed by at least one feature aggregation layer.

[0065] In some embodiments, sampling and feature aggregating point sets in different ranges in the point cloud data in sequence to obtain aggregated features of the point sets in different ranges includes:

[0066] The different ranges of points in the point cloud data are sequentially processed to perform feature aggregation and sampling, and the processing result of the previous range is used as the input of the current range; wherein, for each range,

[0067] Establishing a first spherical coordinate system with a radius R with each point as the center, dividing the spherical area of ​​the first spherical coordinate system into a plurality of first areas and setting corresponding first weights;

[0068] updating the feature of each point based on the first weight of the first area;

[0069] Sampling the updated points to obtain sampling points;

[0070] Based on the sampling point, adjacent points are searched in the first spherical coordinate system to obtain an adjacent point set, and a maximum value in the adjacent point set is used as an aggregate feature of the sampling point.

[0071] In some embodiments, updating the feature of each point based on the first weight of the first region includes:

[0072] The feature of each point in each first region is multiplied by the weight of the first region and the sum is calculated, and then divided by the number of points in the range to update the feature of each point in the range.

[0073] In some embodiments, sampling the updated points to obtain sampling points includes:

[0074] The sampling point is obtained based on the farthest point sampling after the updated point sampling.

[0075] In some embodiments, different ranges have different expansion rates and / or different radii of the first spherical coordinate system.

[0076] Specifically, if Figure 2 As shown, the feature aggregation layer modules 102-105 can aggregate the information of points in the point cloud data so that the points can obtain richer features.

[0077] The first feature aggregation layer module 102 may execute steps 102-1 to 102-2, including:

[0078] Step 102-1: Establish a graph structure. Specifically, Pa represents the point cloud data obtained in the above step S1-1, for example, including 8192 points. First, a spherical coordinate system is established for each point in a spherical area with a radius of R = 0.1m, and the spherical area is divided into blocks. Figure 3 , Figure 3 An example of a spherical coordinate system according to an embodiment of the present invention is shown in FIG. Figure 3 As shown, the spherical coordinate system can include three quantities: azimuth angle θ, elevation angle Φ, and radius r. The azimuth angle can be divided into n equal parts, i.e., each part is 2π / n; the elevation angle can be divided into p equal parts, i.e., each part is π / p; the radius can be divided into q equal parts, i.e., each part is r / q. So for each point in a spherical area with radius R = 0.1m, this area can be divided into Since the origin of the coordinate system is not included in any of the regions, the point itself is divided into a region. Therefore, the spherical region of each point is divided into See Figure 4 , Figure 4 FIG. 2 shows a two-dimensional schematic diagram of multiple regions according to an embodiment of the present invention, such as Figure 4 As shown, these block areas can be numbered from Among them, the numbering can be based on the following formula, where the number is k:

[0079] ,

[0080] ,

[0081] While performing spherical segmentation and numbering, it is possible to record which points are in the point cloud data and which block the point belongs to.

[0082] Step 102-2: Update the point-by-point feature module. You can use a dilated convolution based on a fixed spherical kernel for processing. Define d as the dilation rate, such as Figure 4As shown, when d = 1, each region numbered 012345678 is assigned a weight; when d = 2, each region numbered 02468 is assigned a weight; and when d = 3, each region numbered 0369 is assigned a weight. In the feature aggregation layer module 102, the case of d = 1 is used. Let count be the sum of the number of points in the weighted region. The features of each point in the spherical region are multiplied by their corresponding weights. Finally, the features of all points are summed according to the corresponding channels and divided by count to update the new features of the coordinate origin. The new features of the coordinate origin are first concatenated with the previous features before the update, which is called a residual connection. The features of the coordinate origin after the residual connection are input into the MLP structure to further refine the features of the coordinate origin, and the output is still the features of the coordinate origin. It can be seen that although the dilated convolution can expand the range of the receptive field, it can also cause independence between the regions. In this case, the above problems can be avoided by adding a residual network mechanism, which can also prevent the network from overfitting.

[0083] Step 102-3: Downsampling point cloud module. Prior to this, only point-by-point features were updated, so there are still 8192 points, Pa. By using the farthest point sampling, Pa is sampled as evenly as possible to obtain a smaller set of points, denoted by Pb.

[0084] Step 102-4: Update the sampling point feature module. For each point in Pb, search for nearby points within a radius R in the large-scale point cloud of Pa, with Pb as the coordinate origin. Assuming that the set of nearby points for point A is A-set, use the max pooling method to select the maximum value of the feature of the same channel in the set of nearby points A-set as the value of that channel for point A. The output is the updated feature of point A. Then, use the above method to update the features of each point in Pb.

[0085] The second feature aggregation layer module 103 takes as input the coordinates and features of Pb. Except for the dilation rate d = 2 and R = 0.2m, all other parameters are the same as in step 102. Step 103 outputs the coordinates and features of the sampled point set Pc.

[0086] The third feature aggregation layer module 104 takes as input the coordinates and features of Pc. Except for the dilation rate d = 2 and R = 0.4m, all other parameters are the same as in step 102. Step 104 outputs the coordinates and features of the sampled point set Pd.

[0087] The fourth feature aggregation layer module 105 takes as input the coordinates and features of Pd. Except for the dilation rate d = 3 and R = 0.8m, all other parameters are the same as in step 102. Step 105 outputs the coordinates and features of the sampled point set Pe.

[0088] According to an embodiment of the present invention, step S1-3, fusing the aggregated features of the sampling points into the point set before sampling the sampling points to obtain updated features, includes:

[0089] The aggregated features of the sampling points are sequentially processed and fused into the point set before the sampling point is sampled; wherein, for each fusion,

[0090] Determine the sampling source point set of the input point set;

[0091] Establishing a second spherical coordinate system on the input point set with a point in the sampling source point set as an origin, dividing the spherical area of ​​the second spherical coordinate system into a plurality of second areas, and setting corresponding second weights;

[0092] Based on the second weight of the second area, the feature of each point within the source range is updated.

[0093] It should be understood that the first weight and the second weight can be set as needed and are not limited here.

[0094] In some embodiments, determining the sampling source point set of the input point set includes: determining by retrieval that the input point set is obtained by sampling the sampling source point set.

[0095] Specifically, see Figure 2 , the feature propagation modules 106-109 can propagate the features of the post-sampling point set Pe to the pre-sampling point set Pd.

[0096] The first feature propagation module 106 may execute steps 106 - 1 to 106 - 3 , including:

[0097] Step 106-1: Upsampling point cloud module: For the input point set Pe, it is searched to find out whether it is obtained by sampling the point set Pd, and the features of Pd are obtained at the same time.

[0098] Step 106-2: Establish the graph structure module. For the coordinates and features of Pe, as well as the coordinates and features of Pd, input from the upsampled point cloud module, establish a spherical coordinate system on the Pe point set, with each point in Pd as the origin. R = 0.8 m. The remaining steps are the same as in step 102-1. The difference between the two is that the coordinate system established in step 102-1 is established on the same point set, while the coordinate system established in step 106-2 uses one point set, Pd, as the origin and is established on another point set, Pe.

[0099] Step 106-3: Update the point-by-point feature module. The dilation rate d = 3, and the other operations are the same as step 102-2. The entire module ultimately outputs the coordinates of Pd and the updated features of Pd.

[0100] The second feature propagation module 107 takes as input the coordinates of Pd and its updated features. A spherical coordinate system is established on the point set Pd, with each point in Pc as the origin. The operations in this section are identical to those in step 106, except for d = 2 and R = 0.4 m. This module outputs the coordinates of Pc and its updated features.

[0101] The third feature propagation module 108 takes as input the coordinates of Pc and its updated features. A spherical coordinate system is established on the point set Pc, with each point in Pb as the origin. The operations in this section are identical to those in step 106, except for d = 2 and R = 0.2 m. This module outputs the coordinates of Pb and its updated features.

[0102] The fourth feature propagation module 109 inputs the coordinates of Pb and its updated features. A spherical coordinate system is established on the point set Pb, with each point in Pa as the origin. The operations in this section are identical to those in step 106, except for d = 1 and R = 0.1 m. This module outputs the coordinates of Pa and its updated features.

[0103] It should be understood that the above expansion ratio d and spherical radius R are merely examples and are not intended to be limiting. The above expansion ratio d and spherical radius R can be set as needed and are not limited here.

[0104] According to an embodiment of the present invention, in step S1-4, obtaining the recognition result of the target to be detected based on the updated features may include:

[0105] The updated features are classified to obtain the recognition result.

[0106] Specifically, see Figure 2 For the input coordinates of Pa and features of Pa, the recognition module 110 can input the features of Pa based on a fully connected layer classification network and output the category of each point in Pa. For example, the category of the point is table or wall.

[0107] According to another aspect of the present invention, a point cloud-based semantic segmentation device system is provided, comprising a memory and a processor;

[0108] The memory stores program codes for implementing corresponding steps in the point cloud-based semantic segmentation method according to an embodiment of the present invention;

[0109] The processor is configured to run the program code stored in the memory to execute corresponding steps of the above point cloud-based semantic segmentation method according to an embodiment of the present invention.

[0110] In one embodiment, when the program code is executed by the processor, corresponding steps of the aforementioned point cloud-based semantic segmentation method according to the embodiment of the present invention are executed.

[0111] In addition, according to another aspect of the present invention, a computer-readable storage medium is provided, on which program instructions are stored. When the program instructions are run by a computer or processor, they are used to execute the corresponding steps of the point cloud-based semantic segmentation method according to an embodiment of the present invention, and are used to implement the point cloud-based semantic segmentation system according to an embodiment of the present invention.

[0112] Illustratively, the computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0113] In one embodiment, the computer program instructions, when executed by a computer, may implement the aforementioned point cloud-based semantic segmentation method according to an embodiment of the present invention.

[0114] It can be seen from this that the point cloud-based semantic segmentation method, device, system and storage medium according to the embodiments of the present invention expand the receptive field of the points by sampling and feature aggregation of point sets in different ranges in sequence, so as to obtain richer spatial structure information and achieve better performance and effects in point cloud semantic segmentation.

[0115] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely illustrative and are not intended to limit the scope of the present invention. Various changes and modifications may be made therein by those skilled in the art without departing from the scope and spirit of the present invention. All such changes and modifications are intended to be included within the scope of the present invention as claimed in the appended claims.

[0116] Those skilled in the art will understand that, except where mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstract, and drawings), as well as all processes or units of any method or apparatus disclosed herein, may be combined in any combination. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that provides the same, equivalent, or similar purpose.

[0117] The various component embodiments of the present invention may be implemented in hardware, as data modules running on one or more processors, or as a combination thereof. Those skilled in the art will appreciate that, in practice, a microprocessor or digital signal processor (DSP) may be used to implement some or all of the functions of some modules in the article analysis device according to embodiments of the present invention. The present invention may also be implemented as a device program (e.g., a computer program or computer program product) for executing part or all of the methods described herein. Such a program implementing the present invention may be stored on a computer-readable medium or in the form of one or more signals. Such signals may be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0118] The foregoing description is merely a specific embodiment of the present invention or an illustration of a specific embodiment. The scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be readily conceived by a person skilled in the art within the technical scope disclosed in the present invention are intended to be encompassed by the scope of protection of the present invention. The scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A point cloud-based target recognition method, characterized in that: The method comprises: Obtain point cloud data of the target object; Performing feature aggregation and sampling on different ranges of points in the point cloud data in sequence to obtain aggregated features of the sampling points; Specifically include: Different ranges of points in the point cloud data are processed serially to perform feature aggregation and sampling, and the processing result of the previous range is used as the input of the current range: wherein, for each range, Establishing a first spherical coordinate system with a radius R with each point as the center, dividing the spherical area of ​​the first spherical coordinate system into a plurality of first areas and setting corresponding first weights; Based on the first weight of the first region, updating the feature of each point, specifically, multiplying the feature of each point in each first region by the weight of the first region, summing the results, and then dividing the sum by the number of points in the range, so as to update the feature of each point in the range; The new features of the coordinate origin are first concatenated with the previously updated features, and the features of the coordinate origin after residual connection are input into the MLP structure to refine the features of the coordinate origin; Sampling the updated point to obtain a sampling point, that is, sampling the farthest point based on the updated point to obtain the sampling point; Searching for adjacent points in the first spherical coordinate system based on the sampling point to obtain an adjacent point set, and taking a maximum value in the adjacent point set as an aggregate feature of the sampling point; The aggregated features of the sampling points are fused into the point set before the sampling point is sampled to obtain updated features, including: The aggregated features of the sampling points are sequentially processed and fused into the point set before the sampling point is sampled: wherein, for each fusion, Determining a sampling source point set of an input point set, and determining by searching that the input point set is obtained by sampling the sampling source point set; Establishing a second spherical coordinate system on the input point set with a point in the sampling source point set as an origin, dividing the spherical area of ​​the second spherical coordinate system into a plurality of second areas, and setting corresponding second weights; updating the feature of each point within the source range based on the second weight of the second region; A recognition result of the target to be detected is obtained based on the updated features.

2. The method according to claim 1, wherein Different ranges have different expansion rates and / or different radii R of the first spherical coordinate system.

3. A semantic segmentation device based on point cloud, characterized in that: The device comprises: Point cloud module, used to obtain point cloud data of the target object; The feature aggregation layer is used to perform feature aggregation and sampling on different ranges of points in the point cloud data in turn to obtain the aggregated features of the sampling points: Specifically include: Different ranges of points in the point cloud data are processed serially to perform feature aggregation and sampling, and the processing result of the previous range is used as the input of the current range: wherein, for each range, Establishing a first spherical coordinate system with a radius R with each point as the center, dividing the spherical area of ​​the first spherical coordinate system into a plurality of first areas and setting corresponding first weights; Based on the first weight of the first region, updating the feature of each point, specifically, multiplying the feature of each point in each first region by the weight of the first region, summing the results, and then dividing the sum by the number of points in the range, so as to update the feature of each point in the range; The new features of the coordinate origin are first concatenated with the previously updated features, and the features of the coordinate origin after residual connection are input into the MLP structure to refine the features of the coordinate origin; Sampling the updated point to obtain a sampling point, that is, sampling the farthest point based on the updated point to obtain the sampling point; Searching for adjacent points in the first spherical coordinate system based on the sampling point to obtain an adjacent point set, and taking a maximum value in the adjacent point set as an aggregate feature of the sampling point; The feature propagation layer is used to fuse the aggregated features of the sampling points into the point set before sampling at the sampling points to obtain updated features, including: The aggregated features of the sampling points are sequentially processed and fused into the point set before the sampling point is sampled: wherein, for each fusion, Determining a sampling source point set of an input point set, and determining by searching that the input point set is obtained by sampling the sampling source point set; Establishing a second spherical coordinate system on the input point set with a point in the sampling source point set as an origin, dividing the spherical area of ​​the second spherical coordinate system into a plurality of second areas, and setting corresponding second weights; updating the feature of each point within the source range based on the second weight of the second region; The recognition module is used to obtain a recognition result of the target to be detected based on the updated features.

4. A semantic segmentation system based on point cloud, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 2 when executing the computer program.

5. A computer storage medium having a computer program stored thereon, wherein when the computer program is executed by a computer, the steps of the method according to any one of claims 1 to 2 are implemented.

Images