A method and device for semantic segmentation of a point cloud, and a storage medium

Abstract

This invention provides a semantic segmentation method, apparatus, and storage medium for point clouds. The method includes: S1, a multi-scale encoding / decoding network samples the input point cloud P, defines a neighborhood range based on the sampled points, extracts features of each point within the neighborhood range, aggregates the features of each sampled point to obtain aggregated features, and then obtains the multi-scale features and semantic category probability distribution of each point; S2, a circular convolutional network segments the input point cloud P according to the radius range. t The point cloud P is divided into multiple rings, and the features of each ring are obtained. 3D sparse convolution and a point voxel index table are used to obtain the features of each point, thereby obtaining the semantic category probability distribution; S3, for the point cloud P... t For each unlabeled point in the dataset, the maximum value in the semantic category probability distribution of that unlabeled point is taken as the updated pseudo-label. Using this technical solution, data utilization efficiency can be improved, and dependence on 3D point cloud annotation data can be reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision, in particular to a point cloud semantic segmentation method and device and storage medium. BACKGROUND

[0002] Three-dimensional point cloud semantic segmentation is an important task of environment perception. However, the cost of three-dimensional point cloud data annotation is huge. Most of the existing deep learning-based semantic segmentation methods rely on fully annotated point cloud data, and only a small part of the research focuses on unsupervised or weakly supervised point cloud semantic segmentation. Existing weakly supervised semantic segmentation methods can be roughly divided into three categories: consistency regularization methods, pseudo-label methods and contrast pre-training methods. These methods have their own advantages, but still have some defects. For example: some methods involve multi-stage pre-training, which increases the difficulty of training compared with end-to-end methods; the domain features and semantic features of the point cloud are not fully extracted; and the annotated data is not fully utilized to assist the network in learning the features of the unannotated data. SUMMARY

[0003] Embodiments of the present application provide a point cloud semantic segmentation method, device and storage medium to reduce the dependence on three-dimensional point cloud annotation data, and enable weakly supervised semantic segmentation using a small amount of annotation data.

[0004] To achieve the above purpose, in one aspect, a point cloud semantic segmentation method is provided, comprising:

[0005] S1, a multi-scale coding and decoding network samples an input first point cloud P, wherein P = {P G , P P}, P G is a point cloud formed by labeled points, the number of points in P G is less than a predetermined number, P P is a point cloud formed by unlabeled points, the points in P G have real labels, the points in P P have pseudo-labels generated on the fly, a neighborhood range is defined based on each sampling point, the features of the points in the neighborhood range are extracted in the form of point Transformer within the neighborhood range, then the features of the points in the neighborhood range are aggregated to each sampling point to obtain the aggregated features of each sampling point, and based on the aggregated features, a multi-scale feature of each point in the point cloud P and a semantic class probability distribution of each point in the point cloud P are obtained, wherein i represents the i-th point.

[0006] S2, according to a predetermined radius range, a circular convolution network samples an input second point cloud P tThe point cloud P is divided into multiple annuli, features of the annuli are obtained, 3D sparse convolution is used to extract voxel features of the annuli, and the voxel features are projected to the point cloud P through a point voxel index table t to obtain point-by-point features and further obtain semantic category probability distribution of each point in the point cloud P t wherein i represents the i-th point; wherein the annulus is a three-dimensional annulus, and the point cloud P t is the same as the point cloud P or a point cloud obtained by a predetermined transformation on the point cloud P;

[0007] S3, for each unlabeled point in the point cloud P t , the maximum value in the semantic category probability distribution of each unlabeled point is taken as the updated pseudo-label of the unlabeled point, wherein the updated pseudo-label is represented as

[0008] wherein the multi-scale encoding-decoding network and the annular convolution network are pre-trained, and the loss function used in the training includes:

[0009] and

[0010]

[0011] wherein h(·) represents Jensen-Shannon divergence, m is the number of labeled points in the point cloud P and Pt, and n is the number of unlabeled points.

[0012] Preferably, the semantic segmentation method, wherein the point cloud P t corresponds to the points in the point cloud P one by one.

[0013] Preferably, the semantic segmentation method, wherein the step S2 comprises the following steps of obtaining the point cloud P t :

[0014] rotating the point cloud P at a random angle to obtain the point cloud P r .

[0015] scaling the point cloud P r according to a fixed ratio to obtain the point cloud P s .

[0016] translating the point cloud P s to obtain the data-enhanced point cloud P t .

[0017] Preferably, the semantic segmentation method, wherein the multi-scale encoding-decoding network comprises multiple encoding layers and multiple decoding layers, and in the step S1, the encoding layers sample the point cloud by using the farthest point sampling method and obtain the sampled point cloud.​​

[0018] Preferably, in the semantic segmentation method, step S1 further includes, after sampling:

[0019] For each sampling point, a ball query method is used to find neighboring points P within the neighborhood of the sampling point. neig Neighborhood range The value is set by the neighborhood radius r, and r is set as a learnable parameter.

[0020] For each point within its neighborhood, features are extracted using a point Transformer approach. The extracted features are represented as F. PT , of which F PT =PT(P neig ), PT represents the Transformer mode of the point;

[0021] Using hybrid pooling to divide feature F PT Aggregate to each sampling point to obtain aggregated feature F A The formula is as follows:

[0022]

[0023] Where ζ1 is an adjustable parameter. Let j represent the neighborhood range. The j-th point in the middle.

[0024] Preferably, in the semantic segmentation method, step S1, after obtaining the aggregated features of each sampling point, further includes:

[0025] The decoding layer performs point cloud upsampling and feature backpropagation through linear interpolation.

[0026] The output features are obtained by splicing the features of the encoding layer and the corresponding decoding layer through skip connections;

[0027] The output features are input into a multilayer perceptron of a multiscale encoding / decoding network to obtain the multiscale features of point cloud P.

[0028] Multiscale features Inputting the fully connected layer of a multi-scale encoder-decoder network yields the semantic category probability distribution of each point in the point cloud P.

[0029] Preferably, in the semantic segmentation method, the circular convolutional network includes multiple fully connected layers, and step S2 further includes the following before dividing the multiple circular rings:

[0030] Circular convolutional networks will transform point cloud P t Input multiple fully connected layers to extract point cloud Pt point feature F m , denoted as F m = FCN(P t ), where FCN denotes the operation of a fully connected layer;

[0031] transforming points on the point cloud P t into a polar coordinate system representation;

[0032] setting a radius range, dividing the point cloud in the polar coordinate system representation into different size annuli;

[0033] redistributing point features F m according to the annuli, aggregating the features of points belonging to the same annulus together through mixed pooling to obtain annulus features F C for the corresponding annulus, denoted as:

[0034]

[0035] where ζ2 is an adjustable parameter, is the annulus, k denotes the kth point in the annulus .

[0036] applying 3D sparse convolution on the representation of the annulus features F C to extract voxel features F V , where is where D, R, θ, H respectively denote feature dimension, radius of the annulus, distance azimuth angle and height of the corresponding point;

[0037] projecting the voxel features F V to each point of the point cloud P t through a point-voxel index table, to obtain point-wise features t of the point cloud P t .

[0038] inputting the point-wise features into a fully connected layer to obtain a semantic class probability distribution

[0039] Preferably, the semantic segmentation method, wherein the loss function used in the training further comprises:

[0040] cross-entropy loss for labeled points and

[0041] self-training loss for unlabeled points

[0042] where y ij denotes the real semantic label, a semantic label predicted by a network, a pseudo-real semantic label, a semantic label predicted by a network, wherein c represents the number of categories;

[0043] The total loss function used in the training is: L=L c +L p +L ce +L s .

[0044] In another aspect, a semantic segmentation device of point cloud is provided, comprising a memory and a processor, the memory stores at least one program, and the at least one program is executed by the processor to implement the method of any one of the above.

[0045] In yet another aspect, a computer readable storage medium is provided, wherein the storage medium stores at least one program, and the at least one program is executed by a processor to implement the method of any one of the above.

[0046] The above technical solutions have the following technical effects:

[0047] The technical solution of the embodiment of the present application uses multi-scale coding branches to fully extract the neighborhood features and multi-scale features of the point cloud, and uses a circular convolution branch to divide into a circular ring partition and extract semantic features by using 3D sparse convolution and a point voxel index table, so that the features of the point cloud can be more fully extracted by using the labeled data, the utilization efficiency of the data is improved, and the dependence on three-dimensional point cloud labeled data is reduced.

[0048] The technical solution of another embodiment of the present application uses four different loss functions to supervise the training of the network, optimizes the feature learning of unlabeled data, further reduces the dependence on three-dimensional point cloud labeled data, and improves the accuracy of semantic segmentation. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 It is a flowchart of the semantic segmentation method of point cloud of the embodiment of the present application.

[0050] Figure 2 It is a flowchart of the semantic segmentation method of point cloud of another embodiment of the present application.

[0051] Figure 3 It is a structural schematic diagram of the semantic segmentation device of point cloud of the embodiment of the present application. DETAILED DESCRIPTION

[0052] To further illustrate the various embodiments, the present invention provides accompanying drawings. These drawings are part of the disclosure of the present invention, primarily used to illustrate the embodiments and to explain the operating principles of the embodiments in conjunction with the relevant descriptions in the specification. With reference to these drawings, those skilled in the art should be able to understand other possible implementations and the advantages of the present invention. Components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0053] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments.

[0054] Example 1:

[0055] Figure 1 This is a flowchart illustrating a semantic segmentation method for point clouds according to an embodiment of the present invention. Figure 1 The semantic segmentation method for point clouds in this embodiment includes the following steps:

[0056] S1, the multi-scale encoding / decoding network samples the first point cloud P from the input, where P = {P G ,P P}, P G P is the point cloud formed by the labeled points. G The number of midpoints is less than the predetermined number, P P P is a point cloud formed from unlabeled points. G The points in P have real labels. P The points in the point cloud P have real-time generated pseudo-labels. A neighborhood is defined for each sampling point, and features of each point within that neighborhood are extracted using a point Transformer. These features are then aggregated to the features of each sampling point, resulting in aggregated features for each point. Based on these aggregated features, multi-scale features for each point in the point cloud P are obtained. The semantic category probability distribution of each point in the point cloud P Where i represents the i-th point;

[0057] S2, the circular convolutional network, based on a preset radius range, converts the input second point cloud P... t The data is divided into multiple rings, and the features of each ring are obtained. 3D sparse convolution is used to extract the voxel features of each ring, and the voxel features are projected onto the point cloud P using a voxel index table. t To obtain point-by-point features from each point And thus obtain point cloud P t Semantic category probability distribution of each point Where i represents the i-th point; and the annulus is a three-dimensional annulus, and the point cloud P t The point cloud P is the same as or is obtained from the point cloud P through a predetermined transformation; preferably, the point cloud P tcorresponding to the points in the point cloud P;

[0058] S3, for each unlabeled point in the point cloud P t , taking the maximum value in the semantic class probability distribution of each unlabeled point as the updated pseudo-label of the unlabeled point, wherein the updated pseudo-label is represented as

[0059] wherein the multi-scale encoding-decoding network and the torus convolution network are pre-trained, and the loss function used in the training includes:

[0060] and

[0061]

[0062] wherein h(·) represents Jensen-Shannon divergence, m is the number of labeled points in the point cloud P and P t , and n is the number of unlabeled points.

[0063] The semantic segmentation method of the point cloud according to the embodiment of the application extracts the neighborhood features and multi-scale features of the point cloud by using the multi-scale encoding-decoding network, i.e., the multi-scale encoding-decoding branch, and extracts the features of the point cloud by dividing the point cloud into torus partitions by using the torus convolution network, i.e., the torus convolution branch, and using 3D sparse convolution and point voxel index table. By using the two branches to extract the features of the point cloud, the semantic features of the point cloud can be more fully extracted using fewer labeled points, so that the number of labeled points containing real labels in the point cloud can be small.

[0064] Figure 2 is a flowchart of the semantic segmentation method of the point cloud according to another embodiment of the application. The semantic segmentation method of the point cloud according to the embodiment is a weakly supervised point cloud semantic segmentation method based on multi-scale encoding-decoding and torus convolution, as shown in Figure 2 , the method according to the embodiment includes the following steps:

[0065] S101, input a point cloud with very few real labels, represented as P={P G ,P P}, P G is a point cloud containing real labels, and P P is a point cloud containing pseudo-labels generated in real time; wherein very few here can be measured by being less than a predetermined number;

[0066] S102, the point cloud P G is represented as The point cloud P P is represented as wherein represents a real labeled point,​ representing the real label, for unlabeled points, for instantaneously generated pseudo-labels, m represents the number of labeled points, and n represents the number of unlabeled points;

[0067] S103, randomly rotating the point cloud P to obtain a point cloud P r ;

[0068] S104, scaling the point cloud P r according to a fixed ratio to obtain a point cloud P s ;

[0069] S105, translating the point cloud P s to obtain a data-augmented point cloud P t ;

[0070] The above steps S103 to S105 are used for data augmentation processing of the point cloud P, so as to optimize subsequent point-by-point feature extraction. The data augmentation method can be adjusted according to the input data type.

[0071] S106, inputting the point cloud P into a multi-scale encoding and decoding network, i.e., a multi-scale encoding and decoding branch, which includes multiple encoding layers and multiple decoding layers; preferably, four encoding layers and four decoding layers are included.

[0072] S107, the encoding layer first samples the point cloud P using the farthest point sampling method to obtain a sampled point cloud P sam , wherein P sam is represented as: P sam = FPS(P).

[0073] Preferably, each encoding layer samples the point cloud using the farthest point sampling method, which can obtain a sampled point cloud of different scales or ranges.

[0074] S108, based on each sampling point, a ball query method is used to find neighbor points P neig in the neighborhood range of the sampling point, the neighborhood range D r is set by the neighborhood radius r, and r is set as a learnable parameter, i.e., the neighborhood range is determined by a learnable radius.

[0075] S109, for each point in the neighborhood range, a point Transformer is used to extract features, and the extracted features are represented as F PT , specifically, F PT = PT(P neig ), PT represents the operation of the point Transformer method.

[0076] S110, a hybrid pooling method is used to pool the features F PTaggregated features of each sampling point A , is expressed as ζ1 is an adjustable parameter, is a neighborhood range, j represents a neighborhood range a point in the neighborhood range;

[0077] The method of the embodiment of the application can obtain a multi-scale point cloud through the farthest point sampling and the sphere query of the above-mentioned encoding layer, and the extracted features are correspondingly multi-scale features, such as multi-scale semantic features.

[0078] S111, the decoding layer realizes point cloud upsampling and feature return through linear interpolation operation;

[0079] S112, the features of the encoding layer and the corresponding decoding layer are spliced through the jump connection;

[0080] S113, the decoder output feature is input into a multi-layer perception to obtain multi-scale features i represents the ith point in the point cloud P, which can be any point;

[0081] S114, the multi-scale features are input into a fully connected layer to obtain the semantic class probability distribution of each point

[0082] S115, the point cloud P t is input into a torus convolution network, i.e., a torus convolution branch, to perform feature extraction;

[0083] S116, the torus convolution branch inputs the point cloud P t into a multi-layer fully connected layer to extract the features F of each point m , which is expressed as F m =FCN(P t ), FCN represents the operation of the fully connected layer;

[0084] S117, the points on the point cloud P t are converted into a polar coordinate system, i.e., the points (x, y, z) in a rectangular coordinate system are converted into points (β, θ, z) in a polar coordinate system, wherein the coordinates (x, y) in a rectangular coordinate system xoy are converted into a polar coordinate system (β, θ), wherein β is a polar radius, θ is a polar angle, and the third dimension still adopts a z-axis of the rectangular coordinate system;

[0085] S118, a radius range is set, and the point cloud represented in the polar coordinate system is divided into different size tori, which are three-dimensional tori here;

[0086] S119, the above-mentioned point features F m, the ring features F of each ring are obtained by aggregating the features of the points in each ring through mixed pooling C , wherein ζ2 is an adjustable parameter, is the ring, k represents a point in the ring .

[0087] S120, 3D sparse convolution is applied on the representation of the ring, i.e. the three-dimensional ring, to extract voxel features, and the extracted voxel features are represented as F V , wherein wherein D, R, θ, H represent feature dimension, radius of the ring, distance azimuth angle and height of the corresponding point respectively;

[0088] S121, the voxel features F V are projected to each point through the point voxel index table to obtain point-by-point features i represents a point therein, i.e. the ith point;

[0089] S122, the point-by-point features are input into a fully connected layer to obtain semantic class probability distribution t of each point in the point cloud P

[0090] S123, the maximum value in the semantic class probability distribution of each unlabeled point is taken as the updated pseudo-label of the unlabeled point , which is represented as wherein i represents a point therein;

[0091] The multi-scale coding and decoding network or branch and the ring convolution network or branch in the embodiment of the application need to be trained before use, and the two branch networks also perform the above steps in the training. The loss function used in the training in this embodiment is as follows:

[0092] S124, the point-level consistency loss of the two branches is represented as wherein h(·) represents Jensen-Shannon divergence, m is the number of labeled points contained in the point cloud P or Pt, and n is the number of unlabeled points contained in the point cloud P or Pt. t

[0093] S125, the point-level feature consistency loss of the two branches is represented as:

[0094]

[0095] S126, the semantic prediction supervision of the labeled points in the two branches all adopts cross-entropy loss wherein y ij ​represent a real semantic label, represent a predicted semantic label, c represents a number of categories, and j represents a jth category here;

[0096] In S127, the semantic prediction supervision of the unlabeled points in the two branches is performed by using a self-training loss wherein represent a pseudo real semantic label, represent a predicted semantic label, c represents a number of categories, and j represents a jth category here;

[0097] In S128, the total loss L includes the above four items, and is represented as L=L c +L p +L ce +L s .

[0098] In the method of the embodiment of the application, four different loss functions are used to supervise network training, the feature learning of unlabeled data is optimized, the utilization efficiency of labeled data can be further improved, and the dependence on three-dimensional point cloud labeled data is reduced.

[0099] In the method of the embodiment of the application, for the convenience of description, the serial numbers of the steps are arranged in sequence when the flow is described, but in actual implementation, the steps do not necessarily have to be executed in the above-mentioned numbering order, the order of some steps can be adjusted, some steps can be executed in parallel, and various order changes all fall within the protection scope of the embodiment of the application.

[0100] Embodiment three:

[0101] The application further provides a device for semantic segmentation of point clouds, as shown in the accompanying drawings, the device comprises a processor 301, a memory 302, a bus 303, and a computer program stored in the memory 302 and executable on the processor 301, the processor 301 comprises one or more than one processing core, the memory 302 is connected to the processor 301 through the bus 303, the memory 302 is used for storing program instructions, and the processor executes the computer program to realize the steps in the above method embodiments of the embodiment one of the application. Figure 3

[0102] ​Further, as an executable solution, the device for semantic segmentation of point cloud can be a computer unit, which can be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like. The computer unit can include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the above-mentioned constituent structure of the computer unit is only an example of the computer unit, and does not constitute a limitation on the computer unit, and can include more or fewer components than the above, or combine certain components, or different components. For example, the computer unit can also include an input / output device, a network access device, a bus, and the like, and the embodiments of the present application do not limit this.

[0103] Further, as an executable solution, the processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and the like. The general-purpose processor can be a microprocessor or any conventional processor, and the like. The processor is the control center of the computer unit, and connects various parts of the computer unit through various interfaces and lines.

[0104] The memory can be used to store the computer program and / or modules, and the processor realizes various functions of the computer unit by running or executing the computer program and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required by a function; the data storage area can store data created according to the use of the mobile phone, and the like. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory devices.

[0105] Embodiment Four

[0106] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the method of the above-mentioned embodiments of the application.

[0107] The modules / units integrated with the computer unit, if realized in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, the application can also be implemented by a computer program to instruct related hardware to complete all or part of the processes of the above-mentioned embodiments, and the computer program can be stored in a computer readable storage medium. The computer program can be executed by a processor to implement the steps of each method embodiment. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms, etc. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM) and software distribution medium, etc. It should be noted that the computer readable medium can include appropriate additions or subtractions according to the requirements of legislation and patent practice in the jurisdiction.

[0108] Although the application is specifically shown and described in connection with the preferred embodiments, it should be understood by those skilled in the art that various changes in form and details can be made to the application without departing from the spirit and scope of the application as defined in the appended claims.

Claims

1. A method for semantic segmentation of a point cloud, the method comprising: Comprise: S1, a multi-scale coding and decoding network samples a first point cloud P input, wherein P={P G , P P}, P G is a point cloud formed by labeled points, the number of points in P G is less than a predetermined number, P P is a point cloud formed by unlabeled points, the points in P G have real labels, the points in P P have pseudo labels generated on the spot, a neighborhood range is demarcated based on each sampling point, features of points in the neighborhood range are extracted in a point Transformer manner, then the features of points in the neighborhood range are aggregated to each sampling point to obtain aggregated features of each sampling point, and multi-scale features of each point in the point cloud P and semantic class probability distribution of each point in the point cloud P are obtained based on the aggregated features, wherein i represents the i-th point; S2, according to a preset radius range, the annular convolution network projects the input second point cloud P t into multiple annuli, obtains features of each annulus, adopts 3D sparse convolution to extract voxel features of each annulus, and projects the voxel features to each point of the point cloud P t by a point-voxel index table to obtain point-by-point features and further obtains semantic category probability distribution of each point in the point cloud P t wherein i represents the i-th point; wherein the annulus is a three-dimensional annulus, and the point cloud P t is the same as the point cloud P or a point cloud obtained by a predetermined transformation on the point cloud P.​ S3, for each unlabeled point in the point cloud P t taking the maximum value in the semantic class probability distribution of each unlabeled point as the updated pseudo label of the unlabeled point, wherein the updated pseudo label is denoted as Wherein, the multi-scale coding and decoding network and the circular convolution network are trained in advance, and the loss function used in the training comprises: and where h(·) represents the Jensen-Shannon divergence, m is the number of the annotated points in the point cloud P and P t annotated points, and n is the number of the unannotated points.

2. The semantic segmentation method of claim 1, wherein, The point cloud P t corresponds to a point in the point cloud P.

3. The semantic segmentation method of claim 2, wherein, The step S2 comprises the following steps of obtaining the point cloud Pt: performing a random angle rotation on the point cloud P to obtain a point cloud P r ; to the point cloud P r scaling the point cloud P s ; to the point cloud P s translation of the point cloud P t .

4. The semantic segmentation method of claim 1, wherein, The multi-scale coding and decoding network comprises a plurality of encoding layers and a plurality of decoding layers, and in the step S1, the encoding layers sample the point cloud by using the farthest point sampling method and obtain the sampled point cloud.

5. The semantic segmentation method of claim 4, wherein, In the step S1, after sampling, it further comprises: Based on each sampling point, the neighboring points P in the neighborhood range of the sampling point are found by using a sphere query method neig , the neighborhood range is set by the neighborhood radius r, which is set as a learnable parameter; For each point in the neighborhood range, the point Transformer is used to extract features, and the extracted features are denoted as F PT , wherein F PT = PT(P neig ), PT represents the point Transformer. The features F are aggregated to each sampling point to obtain aggregated features F PT The features F are aggregated to each sampling point to obtain aggregated features F A The formula is as follows: where ζ1is an adjustable parameter, is the neighborhood range, j denotes the jth point in the neighborhood range .

6. The semantic segmentation method of claim 5, wherein, In the step S1, after obtaining the aggregated features of each sampling point, it further comprises: The decoding layer realizes the point cloud upsampling and feature back through linear interpolation operation; The output features are obtained by splicing the features of the encoding layer and the decoding layer corresponding to the encoding layer through the jump connection; inputting the output feature into a multi-layer perceptron of the multi-scale coding-decoding network to obtain a multi-scale feature of the point cloud P obtaining the multi-scale feature of each point in the point cloud P obtaining the semantic class probability distribution of each point in the point cloud P by inputting the full connection layer of the multi-scale coding network 7. The semantic segmentation method of claim 1, wherein, The circular convolution network comprises a plurality of full connection layers, and in the step S2, before dividing the plurality of circular rings, it further comprises: The circular convolutional network will process the point cloud P t Input the multi-layer fully connected layer to extract the point cloud P t Point features F m , represented as F m =FCN(P t ), where FCN represents the operation of the fully connected layer; transforming points on the point cloud P t into a polar coordinate system representation; Setting the radius range, the point cloud represented by the polar coordinate system is divided into circular rings of different sizes; According to the characteristic F of the circular redistribution point m The features of points belonging to the same ring are aggregated together using mixed pooling to obtain the ring feature F of the corresponding ring. C , is represented as: where ζ2is an adjustable parameter, is a circle, k denotes the kth point in the circle . In the circular ring feature F C The 3D sparse convolution is applied to the representation to extract the voxel feature F V where F is where D, R, θ, H represent feature dimension, radius of the circular ring, distance azimuth angle and height of the corresponding point, respectively. by a point voxel index table V projected onto the point cloud P t each point of the point cloud P t point-wise features of the point cloud P the point-wise feature inputting a fully connected layer to obtain the point cloud P t the semantic class probability distribution of each point 8. The semantic segmentation method of claim 1, wherein, The loss function used in the training further comprises: For annotated point cross-entropy loss and Self-training loss for unlabelled points wherein y ij represents a true semantic label, represents a semantic label predicted by a network, represents a pseudo true semantic label, represents a semantic label predicted by a network, wherein c represents a class number; Wherein, the total loss function used in training is: L=L c +L p +L ce +L s .

9. An apparatus for semantic segmentation of a point cloud, the apparatus comprising: The memory stores at least one program, and the processor executes the at least one program to realize the method of any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The storage medium stores at least one program, and the processor executes the at least one program to realize the method of any one of claims 1 to 8.

Images