Point Cloud Semantic Segmentation Method and System Based on Adaptive Feature Sampling

Abstract

The present invention discloses a point cloud semantic segmentation method and system based on adaptive feature sampling, including: performing color channel conversion on point cloud data; performing farthest point sampling on the converted point cloud data and conducting neighborhood search; performing adaptive feature transformation on each neighborhood; performing non-local feature transformation on the features of all points; fusing the features after the above-mentioned downsampling, adaptive features and non-local features; repeating downsampling three times; performing three-layer upsampling based on neighboring point interpolation; using the softmax function to predict the result and conducting model training based on the cross-entropy loss function. Using the trained model to predict the point cloud data to be segmented. The method of the present invention re-weights the neighborhood range of the downsampled points, adaptively adjusts the sampled points, and at the same time constructs the correlation between long-distance sampled points through the fusion of local and non-local features, effectively reducing the influence of noise points on the segmentation result and improving the segmentation effect of data-imbalanced objects.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional point cloud data semantic segmentation, and particularly relates to a method and system for point cloud data semantic segmentation based on adaptive feature sampling. Background Art

[0002] With the popularization of three-dimensional data acquisition devices such as lidar and depth cameras, three-dimensional scene reconstruction and scene understanding have received increasing attention. Semantic segmentation is a key step in realizing scene understanding and a necessary condition for subsequent refined data processing. Point cloud, as the main form of three-dimensional data, its semantic segmentation is an essential task for scene understanding. With the progress of point cloud processing technology, point cloud segmentation methods are also constantly evolving. Researchers at home and abroad have proposed segmentation methods applicable to different scenarios in theory and engineering. Although current traditional point cloud segmentation methods and deep learning-based point cloud segmentation methods have achieved numerous progresses and a large number of profound research results have emerged, such as PointNet, PointNet++, RSNet, PointCNN, etc., there is still a problem of unbalanced point cloud data features in scene segmentation. This problem is reflected in the following three aspects: First, in the same scene, for objects of different categories, the feature differences are large or small, and the number of points represented is also different; Second, the structural inhomogeneity caused by different materials and functions of different objects; Third, the inhomogeneity of effective information of human-object interaction. Unbalanced point cloud data of categories will lead to unsatisfactory segmentation effects for objects of low-frequency small target categories during semantic segmentation.

[0003] At the same time, when the point cloud acquisition device acquires the target, due to the influence of factors such as the acquisition device, the surrounding environment, the target characteristics, and human disturbances, the sampled point cloud data is disordered and sparse, and there are inevitably outliers, missing points or noise points. These data outliers not only destroy the real information of the point cloud image, but also affect the visual effect of the point cloud image, which poses a great challenge to the ability of the point cloud semantic segmentation network. Summary of the Invention

[0004] The purpose of the present invention is to provide a method and system for point cloud semantic segmentation based on adaptive feature sampling to overcome the problem that the segmentation effect is affected due to unbalanced point cloud data features and the existence of data outliers in the prior art. The present invention fully considers the neighborhood information of point cloud data in the downsampling layer of the network, re-weights the neighborhood points of the initial sampling points during the farthest point sampling of the network, adaptively adjusts the sampling points, and at the same time constructs the correlation between long-distance sampling points through the fusion of non-local features and local features, effectively reducing the influence of noise points on the scene segmentation result and improving the segmentation effect of data-unbalanced objects.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] A point cloud semantic segmentation method based on adaptive feature sampling, comprising the following steps:

[0007] Step 1: Perform color channel conversion on the point cloud data containing color information, convert the RGB channels therein to HSV channels, and retain the three-dimensional spatial coordinates of the original point cloud data to obtain the converted point cloud data;

[0008] Step 2: Downsample the converted point cloud data using the farthest point sampling method to obtain a point set and the features corresponding to the point set;

[0009] Step 3: Perform neighborhood search on each point in the point set to obtain k neighbor points;

[0010] Step 4: Perform adaptive feature transformation on the features of the k neighbor points obtained in Step 3 to obtain the adaptive features within each neighborhood after adaptive adjustment;

[0011] Step 5: Associate the adaptive features within each neighborhood obtained in Step 4 using a weight-sharing multi-layer perceptron and normalize to obtain the adaptive feature of a certain point. Perform non-local feature transformation on the adaptive feature of this point, all points in the point set, and the corresponding feature space to obtain the non-local feature of this point;

[0012] Step 6: Combine the adaptive feature and non-local feature of a certain point obtained in Step 5, and the features in Step 2 to obtain the output feature of this downsampling process;

[0013] Step 7: Repeat Step 2 to Step 6, perform downsampling again and obtain the adaptive feature and non-local feature respectively until the network performs three downsamplings and three adaptive feature transformations and non-local feature transformations, thereby establishing three downsampling layers of the network;

[0014] Step 8: Perform three upsamplings through the method of nearest neighbor point weighted interpolation and backtracking, restore the point cloud data obtained by downsampling in Step 7 to the original point cloud scale and fuse features using a multi-layer perceptron; Use softmax to judge the probability of each point in the restored point cloud data belonging to the background or category, and obtain the segmentation result;

[0015] Step 9: Select the cross-entropy loss function to calculate the error of training the network, minimize the network loss through the gradient descent method, train the learnable network parameters, make the network result converge, and obtain the trained point cloud segmentation network;

[0016] Step Ten: Given new point cloud data to be segmented, according to the trained point cloud segmentation network parameters in Step Nine, execute Steps One to Eight to obtain the segmentation result of the point cloud data.

[0017] Further, Step Two is specifically as follows: For the converted point cloud data, use the farthest point sampling method for downsampling to obtain a point set where N represents the number of points in the downsampled point set, 3 represents the three-dimensional spatial coordinates of each point, and the corresponding feature is represented as where N represents the number of points in the downsampled point set, 6 represents the three-dimensional spatial coordinates and color features H, S, V of each point, and are represented by x i and f i represent each point's three-dimensional spatial coordinates and its features in the point set

[0018] Further, Step Three is specifically as follows: Use the K-nearest neighbor algorithm to perform neighborhood search for each point x in the point set, and obtain k neighbor points and the features of the k neighbor points. Among them, the k neighbor points are represented as i The features of the k neighbor points are represented as

[0019] Further, Step Four is specifically as follows: Perform adaptive feature transformation on the features of the k neighbor points obtained in Step Three to obtain the adaptive features within each neighborhood after adaptive adjustment

[0020]

[0021]

[0022] where: f i，c is the c-th neighbor point feature in the neighborhood where the point sets and the point x i are located. After being adjusted by the adaptive feature transformation, it obtains the adaptive feature f' i，c , The function calculates the distance-weighted dot product by enumerating each neighbor point x i of the point x i，j , (j = 1,..., k) and the c-th neighbor point x i of x i，c to measure the feature similarity between the c-th neighbor point of the point x i and all neighbor points. θ and δ are three independent convolution operations, whose purpose is to convert the dimensions of point coordinates or features to enable matrix multiplication, and γ is a pooling operation. ​

[0023] Furthermore, step five is specifically as follows: the adaptive features in each neighborhood obtained after adaptive adjustment in step four are Use a weight-sharing multi-layer perceptron to associate and normalize to obtain point x i Adaptive feature f' i , for point x i Adaptive feature f' i and point set All points x1,...,x N and its features f1,...,f N Perform non-local feature transformation to obtain point x i The non-local feature f ni :

[0024]

[0025]

[0026]

[0027] Where H(f' i , f j ) function is used to complete the point x i Adaptive feature f' i and point set Other point features f j The similarity measure of θ′ and g′ are three spatial mapping operations that project entities in a space into a linear vector space. g , and W θ is a learnable weight matrix.

[0028] Furthermore, step six is as follows: the point x obtained in step five is i Adaptive feature f' i With the non-local feature f ni , and the feature f i Combined, the output feature f of the final downsampling process is obtained F :

[0029] f F =[f' i , f ni , f i ].

[0030] Point cloud semantic segmentation system based on adaptive feature sampling, including:

[0031] Point cloud data conversion module: It is used to perform color channel conversion on point cloud data containing color information, convert the RGB channels therein into HSV channels, and retain the three-dimensional spatial coordinates of the original point cloud data to obtain the converted point cloud data;

[0032] Downsampling module: It is used to downsample the converted point cloud data using the farthest point sampling method to obtain a point set and the features corresponding to the point set;

[0033] Neighborhood search module: It is used to perform neighborhood search on each point in the point set to obtain k neighbor points;

[0034] Adaptive feature transformation module: It is used to perform adaptive feature transformation on the features of k neighbor points to obtain the adaptive features within each neighborhood after adaptive adjustment;

[0035] Non-local feature acquisition module: It is used to associate the adaptive features within each neighborhood after adaptive adjustment using a weight-sharing multi-layer perceptron, and normalize to obtain the adaptive feature of a certain point. Perform non-local feature transformation on the adaptive feature of this point and all points in the point set and their corresponding features to obtain the non-local feature of this point;

[0036] Output feature acquisition module for the downsampling process: It is used to combine the adaptive feature and non-local feature of a certain point, as well as the features in the downsampling module, to obtain the output feature of this downsampling process;

[0037] Three-layer downsampling layer establishment module of the network: It is used to repeat the running processes of the downsampling module, neighborhood search module, adaptive feature transformation module, non-local feature acquisition module, and output feature acquisition module for the downsampling process, perform downsampling again and obtain the adaptive feature and non-local feature respectively until the network has performed three downsamplings and three adaptive feature transformations and non-local feature transformations, thereby establishing the three-layer downsampling layer of the network;

[0038] Upsampling module: It performs three upsamplings through the method of nearest neighbor point weighted interpolation and backtracking, restores the downsampled point cloud data to the original point cloud scale and fuses features using a multi-layer perceptron; uses softmax to judge the probability of each point in the restored point cloud data belonging to the background or category, and obtains the segmentation result;

[0039] Training module: It is used to select the cross-entropy loss function to calculate the error of training the network, minimize the network loss through the gradient descent method, train the learnable network parameters, so that the network result converges, and obtain the trained point cloud segmentation network;

[0040] Segmentation module: Given new point cloud data to be segmented, it runs the point cloud data conversion module, downsampling module, neighborhood search module, adaptive feature transformation module, non-local feature acquisition module, output feature acquisition module of the downsampling process, and upsampling module to obtain the segmentation result of the point cloud data.

[0041] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the point cloud semantic segmentation method based on adaptive feature sampling.

[0042] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, it implements the steps of the point cloud semantic segmentation method based on adaptive feature sampling.

[0043] Compared with the prior art, the present invention has the following beneficial technical effects:

[0044] Currently, most of the point cloud data segmentation methods based on deep learning focus on studying geometric features or exploring deeper network layers, and there is less research on the problem of class imbalance caused by different object categories and volumes in the data. Moreover, there is a problem of insufficient consideration of outliers in the point cloud data. To address the above problems, the present invention fully considers the neighborhood information of the point cloud data, re-weights within the neighborhood range of the downsampled points, and adaptively adjusts the positions of the sampled points. For abnormal sampled points or noises, the weighted adjustment within the neighborhood range can make them more tend to the correct sampling positions, effectively alleviating the impact of data outliers on the segmentation result. At the same time, non-local features within the entire point cloud range are introduced, and through the fusion with local features, the correlation between long-distance sampled points is constructed, which is beneficial to the segmentation of low-frequency small target category objects. Description of the Drawings

[0045] The schematic diagrams in the specification are used to provide a further understanding of the present invention, and constitute a part of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention.

[0046] Figure 1 is the flowchart of the point cloud semantic segmentation network based on adaptive feature sampling;

[0047] Figure 2 is the framework diagram of the point cloud semantic segmentation network based on adaptive feature sampling;

[0048] Figure 3 is the schematic structural diagram of the adaptive feature transformation module;

[0049] Figure 4Schematic diagram of the non-local feature acquisition module structure;

[0050] Figure 5 It is the segmentation result of the point cloud semantic segmentation method based on adaptive feature sampling on the S3DIS dataset, where (a) is the label of indoor scene 1, (b) is the prediction of indoor scene 1, (c) is the label of indoor scene 2, and (d) is the prediction of indoor scene 2. Detailed implementation manners

[0051] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0052] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0053] The technical solutions of the present invention will be further described in detail below with reference to the accompanying drawings:

[0054] The point cloud semantic segmentation method based on adaptive feature sampling of the present invention is divided into the following ten steps, and each step is as follows:

[0055] Step 1: Perform color channel conversion on the point cloud data containing color information, convert the RGB channels therein to HSV channels, and retain the spatial coordinates of the original point cloud data. The conversion formula is as follows:

[0056] V = max(R, G, B)

[0057]

[0058]

[0059] if H < 0, H = H + 1

[0060] The reasons for performing color channel conversion in Step 1 are as follows:

[0061] The point cloud data in HSV format has independent color features relative to the point cloud data in RGB format, which can be used as preprocessing for ordering the point cloud data;

[0062] Step 2: Downsample the converted point cloud data using FPS (Farthest Point Sampling) to obtain a point set where N represents the number of points in the downsampled point set, 3 represents the three-dimensional spatial coordinates of each point, and the corresponding feature is expressed as where N represents the number of points in the downsampled point set, 6 represents the three-dimensional spatial coordinates and color features H, S, V of each point, and are represented by x i and f i represent the three-dimensional spatial coordinates and their features of each point in the point set

[0063] Step 3: Use the K-nearest neighbor algorithm to perform neighborhood search for each point x in the point set i to obtain k neighbor points and the features of the k neighbor points. Among them, the k neighbor points are represented as The features of the k neighbor points are represented as

[0064] Specifically, Step 4 is as follows: As Figure 3 shown, perform adaptive feature transformation on the features of the k neighbor points obtained in Step 3 to obtain the adaptive features within each neighborhood after adaptive adjustment

[0065]

[0066]

[0067] where: f i，c is the feature of the c-th neighbor point in the neighborhood where the point x in the point set is located. After being adjusted by adaptive feature transformation, it obtains the adaptive feature f' i , i，c , The function calculates the distance-weighted dot product by enumerating each neighbor point x i of point x i，j , (j = 1,..., k) and the c-th neighbor point x i of x i，c to measure the feature similarity between the c-th neighbor point of point x i and all neighbor points,​ θ and δ are three independent convolution operations whose purpose is to transform the point coordinates or feature dimensions so that matrix multiplication can be performed. γ is a pooling operation.

[0068] Step 5 is as follows: Figure 4 As shown, the adaptive features in each neighborhood after adaptive adjustment obtained in step 4 are Use a weight-sharing multi-layer perceptron to associate and normalize to obtain point x i Adaptive feature f' i , for point x i Adaptive feature f' i and point set All points x1,...,x N and its features f1,...,f N Perform non-local feature transformation to obtain point x i The non-local feature f ni :

[0069]

[0070]

[0071]

[0072] Where H(f' i , f j ) function is used to complete the point x i Adaptive feature f' i and point set Other point features f j The similarity measure of θ′ and g′ are three spatial mapping operations that project entities in a space into a linear vector space. g , and W θ is a learnable weight matrix.

[0073] Step 6 is as follows: the point x obtained in step 5 is i The adaptive feature f′ i With the non-local feature f ni , and the downsampled features f in step 2 i Combined, the output feature f of the final downsampling process is obtained F :

[0074] f F =[f' i , f ni , f i ]

[0075] Step 7: Repeat Step 2 to Step 6 to perform downsampling again and obtain adaptive features and non-local features respectively, until the network has performed three downsamplings and three adaptive feature transformations and non-local feature transformations, thereby establishing three downsampling layers of the network;

[0076] Step 8: Perform three upsamplings through the method of nearest neighbor weighted interpolation and backtracking, restore the downsampled point cloud data to the original point cloud scale and fuse features using a multi-layer perceptron; Use the softmax function to judge the probability of each point belonging to the background or category point by point to obtain the segmentation result. The calculation formula of the nearest neighbor weighted interpolation is as follows:

[0077]

[0078] Distance weight w i (x) is calculated as follows

[0079]

[0080] f(x) is the restored feature interpolated from the features f of the three nearest points i C′ is the number of channels of the output feature;

[0081] Step 9: Select the cross-entropy loss function to calculate the error of the training network, minimize the network loss through the gradient descent method, and train the learnable network parameters to make the network result converge;

[0082] Step 10: After training the above point cloud segmentation network, given new point cloud data to be segmented, execute Step 1 to Step 8 to obtain the segmentation result of the point cloud data.

[0083] This invention presents a point cloud semantic segmentation method based on adaptive feature sampling and has been verified on the Stanford publicly available dataset S3DIS. Figure 5 The segmentation effect of the segmentation method in this invention on this dataset is given. In the figure, (a) and (c) give the scene labels on the dataset, and (b) and (d) are the corresponding segmentation results. It is not difficult to see that the point cloud semantic segmentation method based on adaptive feature sampling can segment the point cloud scene data more accurately, especially the segmentation effect on low-frequency small target objects is significantly improved, which plays a very important role in the scene understanding or map construction of indoor robots.

[0084] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code.

[0085] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for realizing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0086] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing devices to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that realize the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0087] These computer program instructions can also be loaded onto a computer or other programmable data processing devices, such that a series of operation steps are executed on the computer or other programmable devices to generate a computer-implemented process, so that the instructions executed on the computer or other programmable devices provide steps for realizing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the scope of its protection. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that after reading the present invention, various changes, modifications, or equivalent replacements can still be made to the specific implementation manners of the invention. However, these changes, modifications, or equivalent replacements are all within the scope of the claimed rights of the invention pending approval.

Claims

1. A point cloud semantic segmentation method based on adaptive feature sampling, characterized in that, It includes the following steps: Step 1: Perform color channel conversion on the point cloud data containing color information, convert the RGB channels therein to HSV channels, and retain the three-dimensional spatial coordinates of the original point cloud data to obtain the converted point cloud data; Step 2: Downsample the converted point cloud data using the farthest point sampling method to obtain a point set and the features corresponding to the point set; Step 3: Conduct neighborhood search for each point in the point set to obtain k neighbor points; Step 4: Perform adaptive feature transformation on the features of the k neighbor points obtained in Step 3 to obtain the adaptive features within each neighborhood after adaptive adjustment; Specifically: the features of the k neighbor points obtained in step three are subjected to adaptive feature transformation to obtain the adaptive features within each neighborhood after adaptive adjustment where: f i,c is the c-th neighbor point feature within the neighborhood of point x in the point set i and after being adjusted by the adaptive feature transformation, the adaptive feature f’ i,c is obtained, where c = 1, …, k The function enumerates each neighbor point x i of point x i,j , (j = 1, …, k) and the c-th neighbor point x i of x i,c to perform a distance-weighted dot product calculation, thereby measuring the feature similarity between the c-th neighbor point of point x i and all neighbor points θ and δ are three independent convolution operations, whose purpose is to transform the dimensions of point coordinates or features so that matrix multiplication can be performed, and γ is a pooling operation; Step 5: Associate the adaptive features within each neighborhood obtained after adaptive adjustment in Step 4 using a weight-sharing multi-layer perceptron, and normalize to obtain the adaptive feature of a certain point. Perform non-local feature transformation on the adaptive feature of this point, all points in the point set, and the corresponding feature space to obtain the non-local feature of this point; Step 6: Combine the adaptive feature and non-local feature of a certain point obtained in Step 5, and the features in Step 2 to obtain the output features of this downsampling process; Step 7: Repeat Step 2 to Step 6, perform downsampling again and obtain the adaptive features and non-local features respectively until the network has performed three downsamplings and three adaptive feature transformations and non-local feature transformations, thereby establishing three downsampling layers of the network; Step 8: Perform three upsamplings through the method of weighted interpolation of neighboring points and backtracking, restore the point cloud data obtained by downsampling in Step 7 to the original point cloud scale and fuse features using a multi-layer perceptron; Use softmax to judge the probability of each point in the restored point cloud data belonging to the background or category, and obtain the segmentation result; Step 9: Select the cross-entropy loss function to calculate the error of training the network, minimize the network loss through the gradient descent method, and train the learnable network parameters to make the network result converge to obtain the trained point cloud segmentation network; Step 10: Given new point cloud data to be segmented, according to the trained point cloud segmentation network parameters in Step 9, execute Step 1 to Step 8 to obtain the segmentation result of the point cloud data.

2. The method for point cloud semantic segmentation based on adaptive feature sampling according to claim 1, wherein Step 2 specifically involves: for the converted point cloud data, downsampling is performed using the farthest point sampling method to obtain a point set where N represents the number of points in the downsampled point set, 3 represents the three-dimensional spatial coordinates of each point, and the corresponding feature representation is 6 represents the three-dimensional spatial coordinates of each point and the color features H, S, V, which are represented by x i and f i represent the point set the three-dimensional spatial coordinates of each point in and its features.

3. The method for point cloud semantic segmentation based on adaptive feature sampling according to claim 2, characterized in that, Step 3 specifically is: Using the K-nearest neighbor algorithm to perform neighborhood search on each point x in the point set i to obtain k neighbor points and the features of the k neighbor points. Among them, the k neighbor points are denoted as The features of the k neighbor points are denoted as 4. The method for point cloud semantic segmentation based on adaptive feature sampling according to claim 3, wherein Step five is specifically as follows: adaptively adjust the adaptive features in each neighborhood obtained in step four and associate them with a multi-layer perceptron with shared weights, and normalize to obtain the adaptive feature f' of point x i . For the adaptive feature f' of point x i , perform non-local feature transformation on the adaptive feature f' of point x i and all points x1, …, x i in the point set along with their features f1, …, f N to obtain the non-local feature f N of point x i : ni ​ where the function H(f’ i , f j ) is used to complete the similarity measurement between the adaptive feature f’ i of point x i and the features f s of other points in the point set P j . θ′ and g′ are three spatial mapping operations that project entities in one space into a linear vector space. W g , and W θ are learnable weight matrices.

5. The method for point cloud semantic segmentation based on adaptive feature sampling according to claim 4, wherein Step 6 specifically is: taking the adaptive feature f’ i of the point x obtained in Step 5 i and combining it with the non-local feature f ni , as well as the feature f i to obtain the output feature f F of this downsampling process finally: f F = [f' i , f ni , f i .

6. A point cloud semantic segmentation system based on adaptive feature sampling, characterized in that, It includes: Point cloud data conversion module: Used to perform color channel conversion on the point cloud data containing color information, convert the RGB channels therein to HSV channels, and retain the three-dimensional spatial coordinates of the original point cloud data to obtain the converted point cloud data; Downsampling module: Used to downsample the converted point cloud data using the farthest point sampling method to obtain a point set and the features corresponding to the point set; Neighborhood search module: Used to conduct neighborhood search for each point in the point set to obtain k neighbor points; Adaptive feature transformation module: Used to perform adaptive feature transformation on the features of k neighbor points to obtain the adaptive features within each neighborhood after adaptive adjustment; Specifically: the features of the obtained k neighbor points are subjected to an adaptive feature transformation to obtain the adaptive features within each neighborhood after adaptive adjustment where: f i,c is the c-th neighbor point feature in the neighborhood where the point x in the point set i is located, and the adaptive feature f' is obtained after being adjusted by the adaptive feature transformation i,c , c = 1, …, k, The function enumerates each neighbor point x i of the point x i,j , (j = 1, …, k) and the c-th neighbor point x i of x i,c to perform a distance-weighted dot product calculation to measure the feature similarity between the c-th neighbor point of the point x i and all neighbor points, θ and δ are three independent convolution operations, the purpose of which is to transform the dimensions of the point coordinates or features so that matrix multiplication can be performed, and γ is a pooling operation; Non-local Feature Acquisition Module: It is used to associate the adaptive features in each neighborhood after obtaining adaptive adjustment by using a multi-layer perceptron with shared weights, and normalize to obtain the adaptive feature of a certain point. Perform non-local feature transformation on the adaptive feature of this point and all points and their corresponding features in the point set to obtain the non-local feature of this point; Output Feature Acquisition Module of the Downsampling Process: It is used to combine the adaptive feature and non-local feature of a certain point with the features in the downsampling module to obtain the output feature of this downsampling process; Three-layer Downsampling Layer Establishment Module of the Network: It is used to repeat the running processes of the downsampling module, neighborhood search module, adaptive feature transformation module, non-local feature acquisition module, and output feature acquisition module of the downsampling process, perform downsampling again and obtain the adaptive feature and non-local feature respectively until the network has performed three downsamplings and three adaptive feature transformations and non-local feature transformations, thereby establishing the three-layer downsampling layer of the network; Upsampling Module: Perform upsampling three times through the method of weighted interpolation of neighboring points and backtracking, restore the point cloud data obtained by downsampling to the original point cloud scale and fuse features using a multi-layer perceptron; Use softmax to judge the probability of each point in the restored point cloud data belonging to the background or category, and obtain the segmentation result; Training Module: It is used to select the cross-entropy loss function to calculate the error of training the network, minimize the network loss through the gradient descent method, train the learnable network parameters, so that the network result converges to obtain a trained point cloud segmentation network; Segmentation Module: It is used to give new point cloud data to be segmented, and run the point cloud data conversion module, downsampling module, neighborhood search module, adaptive feature transformation module, non-local feature acquisition module, output feature acquisition module of the downsampling process, and upsampling module to obtain the segmentation result of the point cloud data.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the point cloud semantic segmentation method based on adaptive feature sampling according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the point cloud semantic segmentation method based on adaptive feature sampling according to any one of claims 1 to 5.

Images