Point cloud weakly supervised semantic segmentation method and device based on spherical projection and medium

Abstract

The application discloses a kind of point cloud weak supervision semantic segmentation method, device and medium based on spherical projection, belong to intelligent identification technical field.The method includes: obtaining the point cloud data of laser radar;Mapping point cloud data into image space;The data after mapping is input into the semantic segmentation model after training and is handled, obtains dense and sparse prediction result, realizes the semantic segmentation of point cloud data.The application projects point cloud data into spherical space, and the features of spherical space are input into the deep neural network added with contrast learning module, to process target point cloud data to obtain prediction result, which can effectively reduce the dependence of the model on data labels, improve the accuracy of point cloud segmentation results under the condition of label scarcity.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent identification, and in particular to a point cloud weakly supervised semantic segmentation method and device based on spherical projection and a medium. BACKGROUND

[0002] Currently, in many applications such as automatic driving and robots, the semantic segmentation task is crucial. The point cloud data obtained from the laser radar can provide the position information and intensity information of the objects in the scene, and the application based on the laser radar can understand the 3D scene in a fine-grained manner. However, since the dense point-by-point semantic label is very laborious and expensive to obtain, more and more semantic segmentation tasks try to use weakly supervised or unsupervised methods to reduce the cost of data label acquisition. Compared with the supervised learning method, the weakly supervised method uses several orders of magnitude less labels, reducing the dependence of the method on data labels. The existing point cloud weakly supervised method mainly processes the point cloud directly, which has higher accuracy, but the training and inference speed is slower; and in the existing supervised learning method, the training and inference speed is faster at the cost of a small amount of accuracy. However, since there is a big difference between the supervised learning method and the unsupervised learning method, how to effectively combine the two methods is still to be solved. SUMMARY

[0003] To at least partially solve one of the technical problems existing in the prior art, the purpose of the present application is to provide a point cloud weakly supervised semantic segmentation method, device and medium based on spherical projection.

[0004] The technical solution adopted by the present application is:

[0005] A point cloud weakly supervised semantic segmentation method based on spherical projection, comprising the following steps:

[0006] Obtaining point cloud data of a laser radar;

[0007] Mapping the point cloud data into an image space;

[0008] Inputting the mapped data into a trained semantic segmentation model for processing to obtain dense and sparse prediction results, and realizing semantic segmentation of the point cloud data;

[0009] Wherein, the point cloud data is mapped into the image space by the following way:

[0010] According to the spherical projection algorithm, the point cloud data is projected into the image space; each point after projection corresponds to a five-dimensional feature (d, x, y, z, r), x, y, and z represent coordinate values, d represents depth, and r represents reflection intensity.

[0011] Further, the point cloud weakly supervised semantic segmentation method further comprises a step of training the semantic segmentation model, and the step comprises:

[0012] A neural network is constructed using a convolutional layer, a batch normalization layer, a linear rectifier layer, and a pooling layer.

[0013] A contrastive loss function is inserted into the neural network, and the contrastive loss function and a Multi-class focalloss and a Lov′asz softmax loss constitute a target loss function.

[0014] The neural network is trained according to the target loss function to obtain a semantic segmentation model.

[0015] Further, the neural network is constructed using a convolutional layer, a batch normalization layer, a linear rectifier layer, and a pooling layer, and comprises:

[0016] A basic neural network is constructed using a convolutional layer, a batch normalization layer, a linear rectifier layer, and a pooling layer.

[0017] A contrastive learning module is inserted into the basic neural network to construct a weakly supervised neural network that simultaneously uses a supervised learning method and an unsupervised learning method as a final neural network.

[0018] Further, the expression of the contrastive loss function is as follows:

[0019]

[0020] In the contrastive learning module, an anchor point is denoted as For each anchor point a i , a prototype corresponding to the anchor point type is selected, and is denoted as a positive key point and a negative key point τ is a temperature hyperparameter; N a is the number of anchor points.

[0021] Further, the anchor points in the contrastive learning module are obtained in the following manner:

[0022] For a feature map projected into an image space, the quality of each pixel in the feature map is calculated, and the calculation formula of the quality is as follows:

[0023]

[0024] Where f(x i )∈R K is the class prediction output of the basic neural network for the pixel x i ; and K is the number of classes.

[0025] For the same category of pixels, the sampling probability of the pixel is calculated, and the calculation formula of the sampling probability is as follows:

[0026]

[0027] Wherein, x j is the pixel point of the same category, and X is the set of pixel points of the same category;

[0028] For each category of pixels, N a pixels are obtained as anchor points of the category through weight sampling, denoted as

[0029] Further, the key points in the contrast learning module are obtained in the following way:

[0030] For each category of pixels, N p generic of dimension D is initialized, denoted as:

[0031]

[0032] In each round of training, for category k, N k pixels are sampled according to the probability, and the cosine value between the pixels and the generic is calculated to obtain the cost matrix

[0033] Using Sinkhorn optimal transport method, the cost matrix C is calculated to obtain the transport matrix T;

[0034] N k pixels are updated to N p generic through the transport matrix, and the calculation process is as follows:

[0035]

[0036] Wherein, That is, the subscript of the maximum value of the i-th row of the transport matrix T. In order to facilitate mathematical calculation in the training process, we use the differentiable formula Gumbel-Softmax (τ=0.5) instead of the original argmax function.

[0037] Further, the expression of the target loss function is:

[0038]

[0039] Wherein, L foc is Multi-class focal loss, L lov is Lov′asz softmax loss, and L nce is the contrast loss function; λfoc , λ lov , λ nce are weights, and i represents the serial number of the pixel point participating in the calculation;

[0040] In the process of training the network, the random gradient descent algorithm is used to make the target loss function converge.

[0041] Another technical solution adopted by the application is:

[0042] A point cloud weak supervision semantic segmentation system based on spherical projection, comprising:

[0043] The acquisition module is configured to acquire point cloud data of a laser radar.

[0044] The mapping module is configured to map the point cloud data into an image space.

[0045] The segmentation module is configured to input the mapped data into a trained semantic segmentation model for processing to obtain dense and sparse prediction results, thereby realizing semantic segmentation of the point cloud data.

[0046] The point cloud data is mapped into the image space by the following method:

[0047] According to the spherical projection algorithm, the point cloud data is projected into the image space. Each point after projection corresponds to a five-dimensional feature (d, x, y, z, r), wherein x, y and z represent coordinate values, d represents depth, and r represents reflection intensity.

[0048] Another technical solution adopted by the application is:

[0049] A point cloud weak supervision semantic segmentation device based on spherical projection, comprising:

[0050] At least one processor;

[0051] At least one memory for storing at least one program;

[0052] When the at least one program is executed by the at least one processor, the at least one processor implements the above-mentioned method.

[0053] Another technical solution adopted by the application is:

[0054] A computer-readable storage medium, wherein a processor-executable program is stored, and the processor-executable program is used to execute the above-mentioned method when executed by a processor.

[0055] The beneficial effects of the present application are: the present application projects point cloud data into spherical space, and inputs the features of the spherical space into a deep neural network added with a contrast learning module, processes the target point cloud data to obtain a prediction result, which can effectively reduce the dependence of the model on data labels, and improve the accuracy of the point cloud segmentation result under the condition of label scarcity. BRIEF DESCRIPTION OF DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following introduces the drawings of the related technical solutions in the embodiments of the present application or the prior art. It should be understood that the drawings in the following introduction are only for the convenience of clearly describing part of the embodiments of the technical solutions in the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0057] Figure 1 is a flowchart of a weakly supervised semantic segmentation method based on spherical projection and contrast learning in an embodiment of the present application;

[0058] Figure 2 is a flowchart of a projection method in an embodiment of the present application;

[0059] Figure 3 is a method diagram of a projection method in an embodiment of the present application;

[0060] Figure 4 is a structural block diagram of a weakly supervised semantic segmentation system based on spherical projection and contrast learning in an embodiment of the present application;

[0061] Figure 5 is a structural block diagram of a weakly supervised semantic segmentation device based on spherical projection and contrast learning in an embodiment of the present application;

[0062] Figure 6 is a specific structural diagram of a weakly supervised semantic segmentation system based on spherical projection and contrast learning in an embodiment of the present application;

[0063] Figure 7 is a result diagram of the SemanticKITTI data set in an embodiment of the present application. DETAILED DESCRIPTION

[0064] The embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0065] In the description of this invention, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., are based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0066] In the description of this invention, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0067] Furthermore, in the description of this invention, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0068] In the description of this invention, unless otherwise explicitly defined, terms such as "set up," "install," and "connect" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this invention in conjunction with the specific content of the technical solution.

[0069] like Figure 1 As shown, this embodiment provides a weakly supervised semantic segmentation method based on spherical projection and contrastive learning to reduce reliance on 3D point cloud labeled data by utilizing unlabeled point cloud data. The method specifically includes the following steps:

[0070] S1. Acquire point cloud data from the LiDAR, and project the given 3D point cloud onto the image space using a spherical projection algorithm. For example... Figure 2 As shown, Figure 2A schematic diagram of the projection process.

[0071] As shown in Figure 3 , we transform the point cloud coordinates into the elevation angle a and azimuth angle b in the spherical coordinate system through the inverse trigonometric function. It should be noted that the laser radar only rotates in the horizontal direction, and its scanning angle in the vertical direction is fixed as [fov_up, fov_down]. Therefore, for a 2D projection target image with a length H and a width W, the coordinates of a certain point in the 2D projection target image are obtained through h = a / (fov_up-fov_down)·H and w = b / π·W. After projecting the point cloud into the 2D projection target image, an additional dimension of depth feature is calculated for each projected point Finally, each point projected in the 2D projection target image has five-dimensional features (d, x, y, z, r), where r represents the reflection intensity. It should be noted that in actual applications, additional input features can be added according to specific needs to improve accuracy, such as the RGB color information of the image.

[0072] S2, with the help of the existing deep neural network based on supervised learning, a contrast learning based unsupervised module is added to obtain a target weakly supervised deep neural network.

[0073] In this embodiment, for the contrast learning module, the method of the embodiment is based on the anchor points and the key points to construct a contrast learning module. Among them, the anchor points in the contrast learning module are written as For each anchor point, select the prototype corresponding to the anchor point type, which are positive key points and negative key points, respectively, written as and τ is a temperature hyperparameter used to control the data distribution of the anchor point and key point product.

[0074] The acquisition process of the anchor points in the contrast learning module includes: for the feature map projected into the image space, the quality of each pixel in the feature map is calculated using the formula , wherein f(x i )∈R K is the class prediction output of the basic neural network for the pixel x i ; then, for the pixels of the same class, the sampling probability of the pixel is calculated using the following formula:

[0075]

[0076] where X represents the pixel points of the same class. Finally, for each class, we obtain N a pixels as anchor points of the class through weight sampling, written as

[0077] The key point acquisition process in the contrast learning module includes: for each category, initializing N p generic types with D dimensions, written as During each round of training, for category k, N k pixels are randomly sampled, and the cosine value between the pixels and the generic types is calculated to obtain the cost matrix Then, the Sinkhorn optimal transport method is used to obtain the transport matrix Then, the formula is used to update the N k pixels to N p generic types. Wherein, To facilitate mathematical calculations during training, we use the differentiable formula Gumbel-Softmax (τ = 0.5) instead of the original argmax function.

[0078] The features in the input contrast learning module are obtained according to the following steps: first, input the features obtained by projection into an encoder to obtain multiple feature maps of different scales. Then, the feature maps of different scales are obtained by linear interpolation to obtain feature maps with the same size as the image, and the feature maps are spliced; then, use two 1x1 2D convolution kernels to map the spliced features to 256 dimensions. Finally, the features are regularized by L2 norm.

[0079] S3, add a contrast loss function to the basic deep neural network to form a target loss function together with the point cloud loss function, and the target loss function is as follows:

[0080]

[0081] Where, L foc is a multi-class focal loss, L lov is a Lov′asz softmax loss, L nce is a contrast loss function; λ is the weight of different loss functions, and i represents the serial number of the pixel point participating in the calculation.

[0082]

[0083]

[0084] S4, train the pre-trained model with input data and target loss function.

[0085] The number of pseudo labels can be increased in a voxel expansion manner. In order to avoid manual labeling or the influence of one object and one point, a random down-sampling method is used on the original data set to obtain a target data set. AdamW is used to optimize the model, and 1.0, 1.0 and 0.1 are set as, and, respectively. It should be noted that, since the determination of the anchor point in the early stage is easily affected by noise, the contrast module is not added to train the model in the first 5 epochs to accelerate the convergence of the model. foc lov nce It should be noted that, since the determination of the anchor point in the early stage is easily affected by noise, the contrast module is not added to train the model in the first 5 epochs to accelerate the convergence of the model.

[0086] S5, using the trained model to predict the input data.

[0087] The input 3D point cloud data, the network outputs a dense prediction O, and then the dense prediction result is obtained through

[0088] S6, according to the projection relationship, the prediction result of the sparse point cloud is obtained from the dense prediction result output by the network.

[0089] After obtaining the two-dimensional dense prediction result, the two-dimensional dense prediction is converted into the original three-dimensional point cloud coordinate system by means of spherical projection, that is, the point with three-dimensional coordinates (x, y, z) obtains the prediction result of the corresponding image space pixel.

[0090] The dense prediction result refers to that each pixel in an image has a prediction value; and the sparse prediction result refers to that only part of the pixels in an image has a prediction value.

[0091] The method provided by the embodiment of the application is applied to obtain the point cloud data to be recognized; the point cloud data is projected into a spherical space; the features of the spherical space are input into a deep neural network to which a contrast learning module is added; and a classifier of a target prediction model is used to process the target point cloud data to obtain a prediction result.

[0092] The point cloud data to be recognized is obtained and input into a deep neural network to which a contrast learning module is added. Since the target network adds a contrast learning module that can utilize unlabeled data, the accuracy of the point cloud segmentation result can be improved in the case of only a small amount of labeled data. Meanwhile, since the projection matrix is used to process the point cloud data into a form that can be processed by 2D convolution, the calculation amount of the network can be greatly reduced. Further, since the projection matrix is used to process the point cloud data into a form that can be processed by 2D convolution, the prediction method provided by the embodiment of the application based on the fusion of laser radar and image information can be applied to products such as automatic driving and robots that have high real-time calculation requirements. ​​​

[0093] Referring to Figure 7 , Figure 7 are results of the embodiment on the SemanticKITTI dataset. The point cloud semantic segmentation method described above is applied to the SemanticKITTI autonomous driving semantic segmentation dataset, and the semantic segmentation result accuracy is shown in Table 1:

[0094] Table 1

[0095]

[0096] As can be seen from the above, by adding the unsupervised method based on contrast learning, the model can effectively reduce the dependence on data labels, and in addition, due to the use of a lightweight model framework, the speed of the method during training and inference is improved.

[0097] As shown in Figure 4 , the embodiment also provides a semantic segmentation system based on spherical projection and contrast learning, comprising:

[0098] The acquisition module is configured to acquire point cloud data of a laser radar.

[0099] The mapping module is configured to map the point cloud data into an image space.

[0100] The segmentation module is configured to input the mapped data into a trained semantic segmentation model for processing to obtain dense and sparse prediction results, thereby realizing semantic segmentation of the point cloud data.

[0101] The point cloud data is mapped into the image space by the following method:

[0102] According to the spherical projection algorithm, the point cloud data is projected into the image space. Each point after projection corresponds to a five-dimensional feature (d, x, y, z, r), wherein x, y and z represent coordinate values, d represents depth, and r represents reflection intensity.

[0103] The semantic segmentation system based on spherical projection and contrast learning of the embodiment can execute the semantic segmentation method based on spherical projection and contrast learning provided by the method embodiment of the application, can execute the implementation steps of any combination of the method embodiment, and has the corresponding functions and beneficial effects of the method.

[0104] As shown in Figure 5 , the embodiment also provides a semantic segmentation device based on spherical projection and contrast learning, comprising:

[0105] At least one processor D2;

[0106] At least one memory D1 for storing at least one program;

[0107] When the at least one program is executed by the at least one processor, the at least one processor implements the method as shown. Figure 1 The method as shown.

[0108] The semantic segmentation device based on spherical projection and contrast learning can execute the semantic segmentation method based on spherical projection and contrast learning provided by the method embodiment of the application, can execute the steps of any combination of the method embodiment, and has the corresponding functions and beneficial effects of the method.

[0109] As an implementation form, refer to Figure 6 , Figure 6 The specific structural diagram of the image recognition device based on a deep neural network model provided by the embodiment can have a large difference due to different configurations or performances, and can include one or more processors (central processing units, CPUs) (for example, one or more processors) and a memory, one or more storage media (for example, one or more mass storage devices) for storing application programs or data. Among them, the memory and the storage medium can be temporary storage or persistent storage. The program stored in the storage medium can include one or more modules (not shown in the figure), and each module can include a series of instruction operations in the data processing device. Further, the central processing unit can be configured to communicate with the storage medium and execute a series of instruction operations in the storage medium on the image recognition device based on the deep neural network model.

[0110] The image recognition device based on the deep neural network model can also include one or more power supplies, one or more wired or wireless network interfaces, one or more input / output interfaces, and / or one or more operating systems. For example, Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0111] The steps in the image recognition method based on the deep neural network model described above can be implemented by the structure of the image recognition device based on the deep neural network model.

[0112] The embodiment of the application also discloses a computer program product or a computer program, which comprises computer instructions stored in a computer readable storage medium. The processor of the computer device can read the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes Figure 1 The method as shown.

[0113] The embodiment also provides a storage medium storing instructions or programs for implementing the method provided in the embodiment of the application, which can execute any combination of the method embodiments to implement the steps and have the corresponding functions and advantages of the method when the instructions or programs are executed.

[0114] In some alternative embodiments, the functions / operations mentioned in the block diagrams can not occur in the order mentioned in the operation diagram. For example, depending on the functions / operations involved, two blocks shown in succession can actually be executed substantially simultaneously or the blocks can sometimes be executed in reverse order. In addition, the embodiments presented and described in the flowcharts of the present application are provided by way of example, with the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operations and logical flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and in which sub-operations described as part of a larger operation are independently executed.

[0115] In addition, although the present application is described in the context of functional modules, it should be understood that one or more of the functions and / or features described can be integrated in a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules, unless otherwise specified. It can also be understood that a detailed discussion of the actual implementation of each module is unnecessary for an understanding of the present application. Rather, the properties, functions and internal relationships of the various functional modules in the devices disclosed herein are considered within the ordinary skill of an engineer in view of the present disclosure. Therefore, those skilled in the art can implement the present application as set forth in the claims without undue experimentation using ordinary skill. It can also be understood that the disclosed specific concepts are merely illustrative and are not intended to limit the scope of the present application, which is determined by the full scope of the appended claims and their equivalents.

[0116] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the technical solutions that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0117] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered a list of executable instructions for implementing logic functions, and can be specifically embodied in any computer-readable medium for use by an instruction execution system, apparatus, or device, such as a computer-based system, a system including a processor, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions, or in conjunction with these instructions execution systems, apparatuses, or devices. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport programs for use by an instruction execution system, apparatus, or device, or in conjunction with these instruction execution systems, apparatuses, or devices.

[0118] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection having one or more wires (electrical devices), a portable computer diskette (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium can even be paper or other suitable medium on which the program can be printed, because the program can be electronically obtained, for example, by optical scanning of the paper or other medium, followed by editing, interpreting, or otherwise processing, if necessary, in other suitable ways, to be electronically obtained, and then stored in the computer memory.

[0119] It should be understood that various parts of the present application can be implemented in hardware, software, firmware or a combination thereof. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the following technologies, known in the art, or their equivalents, can be employed for implementation: discrete logic circuitry having logic gates for implementing logic functions upon data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and the like.

[0120] In the above description of the present specification, the description of the terms "one embodiment / one example", "another embodiment / another example", or "certain embodiments / certain examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiments or examples are included in at least one embodiment or example of the present application. In the present specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Also, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0121] Although the embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made thereto without departing from the principles and spirit of the present application, the scope of which is defined by the claims and their equivalents.

[0122] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the above-described embodiments, and those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present application, and these equivalent modifications or substitutions are included in the scope defined by the claims of the present application.

Claims

1. A weakly supervised semantic segmentation method for point clouds based on spherical projection, characterized in that, Includes the following steps: Acquire point cloud data from LiDAR; Map point cloud data to image space; The mapped data is input into the trained semantic segmentation model for processing to obtain dense and sparse prediction results, thereby achieving semantic segmentation of point cloud data. Point cloud data is mapped to image space in the following way: The point cloud data is projected onto the image space using a spherical projection algorithm; each projected point corresponds to a five-dimensional feature. , Represents coordinate values. Indicates depth, Indicates the intensity of reflection; The weakly supervised semantic segmentation method for point clouds further includes a step of training the semantic segmentation model, which includes: Neural networks are constructed using convolutional layers, batch normalization layers, linear rectified layers, and pooling layers; A contrastive loss function is inserted into the neural network, and the contrastive loss function, together with Multi-class focal loss and Lov'asz softmax loss, constitutes the target loss function. The neural network is trained according to the target loss function to obtain a semantic segmentation model; The method of constructing a neural network using convolutional layers, batch normalization layers, linear rectified layers, and pooling layers includes: A basic neural network is constructed using convolutional layers, batch normalization layers, linear rectified layers, and pooling layers; A contrastive learning module is inserted into the basic neural network to construct a weakly supervised neural network that utilizes both supervised and unsupervised learning methods, which serves as the final neural network.

2. The weakly supervised semantic segmentation method for point clouds based on spherical projection according to claim 1, characterized in that, The expression for the contrastive loss function is: In the contrastive learning module, the anchor point is denoted as... For each anchor point Select the generic type corresponding to the anchor point type, and denote them as positive class keypoints. Key points of negative classes ; It is a temperature hyperparameter; It refers to the number of anchor points.

3. The weakly supervised semantic segmentation method for point clouds based on spherical projection according to claim 2, characterized in that, The anchor points in the contrastive learning module are obtained in the following way: For a feature map projected into the image space, the quality of each pixel in the feature map is calculated using the following formula: in, It is the basic neural network for pixels The category prediction output; It is the number of categories; For pixels of the same category, calculate the sampling probability of the pixels. The formula for calculating the sampling probability is as follows: in, They are pixels of the same category. X It is a collection of pixels of the same category; For each category of pixels, weight sampling is used to obtain... Each pixel is used as the anchor point for this category, denoted as . .

4. The weakly supervised semantic segmentation method for point clouds based on spherical projection according to claim 2, characterized in that, Key points in the comparative learning module were obtained in the following ways: For each category of pixels, initialize A generic with dimension D is denoted as: During each round of training, for the category k Based on probability sampling The cost matrix is ​​obtained by calculating the cosine value between each pixel and the generic type. ; The transmission matrix is ​​obtained by calculating the cost matrix C using the Sinkhorn optimal transmission method. T ; Will Each pixel is updated via the transfer matrix. The calculation process for each generic type is as follows: in, .

5. The weakly supervised semantic segmentation method for point clouds based on spherical projection according to claim 1, characterized in that, The expression for the target loss function is: in, For multi-class focal loss, For Lov'asz softmax loss, To compare loss functions; , , All are weights. i Indicates the index of the pixel involved in the calculation; During network training, the stochastic gradient descent algorithm is used to bring the target loss function to converge.

6. A weakly supervised semantic segmentation device for point clouds based on spherical projection, used to implement the method described in any one of claims 1-5, characterized in that, include: The acquisition module is used to acquire point cloud data from the lidar. The mapping module is used to map point cloud data to image space; The segmentation module is used to input the mapped data into the trained semantic segmentation model for processing, to obtain dense and sparse prediction results, and to achieve semantic segmentation of point cloud data. Point cloud data is mapped to image space in the following way: The point cloud data is projected onto the image space using a spherical projection algorithm; each projected point corresponds to a five-dimensional feature. , Represents coordinate values. Indicates depth, Indicates the intensity of reflection.

7. A weakly supervised semantic segmentation device for point clouds based on spherical projection, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method of any one of claims 1-5.

8. A computer-readable storage medium storing a processor-executable program, characterized in that, The processor-executable program, when executed by the processor, is used to perform the method as described in any one of claims 1-5.

Images