Sparse point cloud semantic occupation prediction method and device based on spherical projection, equipment and medium
Through spherical projection and deep neural network technology, the problems of spatial information discontinuity and context content in sparse point cloud semantic occupation prediction are solved, the accuracy of semantic occupation prediction is improved, and efficient processing of sparse point cloud data is achieved.
Patent Information
- Application Number
- CN202411326217.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-23
- Publication Date
- 2025-01-10
AI Technical Summary
The prior art has problems such as discontinuity of spatial information and uncompact context content in sparse point cloud semantic occupation prediction, resulting in low prediction accuracy of semantic occupation and relying on continuous multi-frame point cloud data, making it difficult to effectively realize efficient semantic occupation prediction.
Map sparse point cloud data to spherical coordinate system through spherical projection, generate a continuous and compact range view representation, and construct a semantic occupation prediction model using convolutional layers, deformable attention modules and progressive decoders to extract multi-scale features. Voxel characteristics are aggregated and refined to generate voxel-by-voxel classification probability.
The accuracy of semantic occupancy prediction of sparse point clouds is improved, the dependence on continuous multi-frame point cloud data is alleviated, and a more efficient semantic occupancy prediction effect is achieved.
Smart Images
Figure CN119295809A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent recognition technology, and in particular to a sparse point cloud semantic occupancy prediction method, device, equipment and medium based on spherical projection. Background Art
[0002] In many applications such as autonomous driving and mobile robotics, accurate and comprehensive environmental perception is crucial, as it provides surrounding information for downstream modules such as path planning and motion control. Semantic occupancy prediction is an important task in achieving environmental perception, which divides the three-dimensional environment into regular voxel grids and requires deep neural networks to process input sensor data and determine the occupancy state and semantic category of each voxel grid.
[0003] Existing semantic occupancy prediction methods can be divided into: camera-based methods and lidar-based methods. Camera-based methods take RGB images as input and extract rich appearance information to generate three-dimensional scene representations, but their accuracy is bottlenecked due to the lack of depth information. LiDAR-based methods usually have better accuracy, thanks to the accurate spatial information of point clouds. Due to the inherent sparsity and irregular distribution of single-frame point clouds, existing methods often stack continuous multi-frame point clouds to construct dense spatial information to promote the learning of semantic representations of scenes, but they may encounter challenges in operating efficiency and robustness, hindering their application in practical scenarios. Although single-frame point cloud semantic occupancy prediction is a better choice, the spatial information and contextual content of sparse point clouds are discontinuous, which brings difficulties to semantic occupancy prediction. In order to alleviate the discontinuity problem of sparse point clouds, spherical projection has been studied and applied, which maps three-dimensional point clouds to a two-dimensional plane to obtain a range image with continuous spatial information and compact contextual content. However, there are two main problems in reconstructing 3D scenes from range images: 1) the geometric representation of the original point cloud is sparse and discrete, making it difficult to effectively preserve the semantic information of the range image; 2) the inherent "many-to-one" mapping relationship of spherical projection leads to semantic and geometric ambiguity in the reconstruction results. Therefore, how to achieve efficient semantic occupancy prediction of sparse point clouds based on spherical projection remains to be solved. Summary of the invention
[0004] In order to solve at least one of the technical problems existing in the prior art to a certain extent, the object of the present invention is to provide a sparse point cloud semantic occupancy prediction method, device, equipment and medium based on spherical projection.
[0005] The first technical solution adopted by the present invention is:
[0006] A sparse point cloud semantic occupancy prediction method based on spherical projection comprises the following steps:
[0007] Get the point cloud data of LiDAR;
[0008] Perform spherical projection on the acquired point cloud data to obtain a range view representation of the point cloud;
[0009] Input the range view representation of the point cloud into the trained semantic occupancy prediction model, extract features from the range view representation, and obtain multi-scale range view features;
[0010] Perform feature aggregation on multi-scale range view features to obtain voxel features of the three-dimensional scene;
[0011] The voxel features of the 3D scene are refined and enhanced, and voxel-by-voxel classification probabilities are generated to achieve semantic occupancy prediction of the 3D scene.
[0012] Furthermore, the step of performing spherical projection on the obtained point cloud data to obtain a range view representation of the point cloud includes:
[0013] The three-dimensional coordinates (x, y, z) of the point cloud data are mapped point by point through the spherical projection formula to obtain the pixel coordinates (u, v) on the range image; wherein the spherical projection formula is:
[0014]
[0015] Where W and H represent the width and height of the projected image, respectively. Represents the depth value of the projected point cloud, f v =|f up |+|f down | represents the vertical field of view of the laser radar, |f up | represents the field of view in the horizontal upward direction, |f down |Indicates the field of view angle in the horizontal downward direction.
[0016] Furthermore, the semantic occupancy prediction model is constructed in the following way:
[0017] A semantic occupancy prediction model is constructed by using a convolutional layer, a batch normalization layer, a linear rectification layer, a pooling layer, and a linear layer; wherein the constructed semantic occupancy prediction model includes a feature extractor, a deformable attention module, and a progressive decoder;
[0018] The feature extractor is used to extract features from the range view representation to obtain multi-scale range view features;
[0019] The deformable attention module is used to combine predefined voxel queries to perform feature aggregation on multi-scale range view features to obtain voxel features of the three-dimensional scene;
[0020] The progressive decoder is used to refine and enhance the voxel features of the three-dimensional scene and generate voxel-by-voxel classification probabilities.
[0021] Furthermore, the deformable attention module works as follows:
[0022] Voxel queries with voxel grid shapes are defined in voxel space, and each voxel query dynamically captures features of interest on multi-scale range view features through a deformable attention mechanism, thereby obtaining voxel features of the 3D scene.
[0023] Further, the progressive decoder includes a dense voxel feature extractor, a sparse voxel feature extractor, and an occupancy prediction classifier;
[0024] The dense voxel feature extractor is used to extract dense voxel features, and then input the extracted dense voxel features into the occupancy prediction classifier to generate an occupancy prediction result.
[0025] The sparse voxel feature extractor is used to extract sparse voxel features, and then input the extracted sparse voxel features into the occupancy prediction classifier to generate an occupancy prediction result.
[0026] Furthermore, the overall target loss for training the semantic occupancy prediction model is for:
[0027]
[0028] Where O represents the voxel label, and represents the prediction result of the progressive decoder; is the target loss function.
[0029] Furthermore, the objective loss function The expression is:
[0030]
[0031] In the formula, represents the weighted cross entropy loss, represents the lovasz-softmax loss, and represents the scene-category affinity loss.
[0032] The second technical solution adopted by the present invention is:
[0033] A sparse point cloud semantic occupancy prediction device based on spherical projection, comprising:
[0034] Acquisition module, used to obtain point cloud data of LiDAR;
[0035] A mapping module is used to perform spherical projection on the obtained point cloud data to obtain a range view representation of the point cloud;
[0036] The processing module is used to input the range view representation of the point cloud into the trained semantic occupancy prediction model, perform feature extraction on the range view representation to obtain multi-scale range view features; perform feature aggregation on the multi-scale range view features to obtain voxel features of the three-dimensional scene; refine and enhance the voxel features of the three-dimensional scene, and generate voxel-by-voxel classification probabilities to achieve semantic occupancy prediction of the three-dimensional scene.
[0037] The third technical solution adopted by the present invention is:
[0038] An electronic device comprises a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement a sparse point cloud semantic occupancy prediction method based on spherical projection as described above.
[0039] The fourth technical solution adopted by the present invention is:
[0040] A computer-readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, wherein the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement a sparse point cloud semantic occupancy prediction method based on spherical projection as described above.
[0041] The fifth technical solution adopted by the present invention is:
[0042] A computer program product or a computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above method.
[0043] Compared with the prior art, the present invention has the following beneficial effects: the present invention projects a sparse point cloud with discontinuous spatial information and non-compact contextual content into a spherical coordinate system to obtain a continuous and compact range view representation; extracts multi-scale range view features through a pre-trained feature extractor to capture object information in the scene from a global and local perspective; transfers two-dimensional range view semantic features to the voxel space through a deformable attention module to obtain a voxel representation of a three-dimensional scene; refines and enhances voxel features through a progressive decoder, and generates semantic occupancy prediction results. The present invention effectively improves the accuracy of semantic occupancy prediction of sparse point clouds and alleviates the dependence on continuous multi-frame point cloud data. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the embodiments of the present invention or the drawings of related technical solutions in the prior art are introduced below. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0045] Figure 1 is a flowchart of the steps of a sparse point cloud semantic occupancy prediction method based on spherical projection in an embodiment of the present invention;
[0046] Figure 2 is a flow chart of a sparse point cloud semantic occupancy prediction method based on spherical projection in an embodiment of the present invention;
[0047] Figure 3 Schematic diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION
[0048] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limitations of the present invention. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.
[0049] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.
[0050] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.
[0051] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.
[0052] Example 1
[0053] like Figure 1 and Figure 2 As shown, this embodiment provides a sparse point cloud semantic occupancy prediction method based on spherical projection, comprising the following steps:
[0054] Get the point cloud data of LiDAR;
[0055] S1. Perform spherical projection on the obtained point cloud data to obtain a range view representation of the point cloud.
[0056] Specifically, the point cloud data output by the lidar It is a set of N points, where each point p∈P contains three-dimensional coordinates (x, y, z) and reflection intensity i.
[0057] S2. Input the range view representation of the point cloud into the trained semantic occupancy prediction model, extract features from the range view representation, and obtain multi-scale range view features.
[0058] For point clouds, discontinuous spatial information and non-compact contextual content hinder the semantic representation learning of each object in the scene. Therefore, in some embodiments, the original point cloud is generated into a continuous and compact range image by spherical projection, that is, each three-dimensional coordinate (x, y, z) can be mapped to the pixel coordinate (u, v) on the range image by the following formula:
[0059]
[0060] Where W and H represent the width and height of the projected image, respectively. Represents the depth value of the projected point cloud, f v =|f up |+|f down | represents the vertical field of view of the laser radar. Range image obtained by spherical projection Each pixel has 5 channels (x, y, z, i, r). For pixels without projection points, the 5 channels are filled with 0.
[0061] S3. Input the range view representation of the point cloud into the trained semantic occupancy prediction model, extract features from the range view representation, and obtain multi-scale range view features.
[0062] In some embodiments, a semantic occupancy prediction model is constructed using convolutional layers, batch normalization layers, linear rectification layers, pooling layers, and linear layers, including a feature extractor, a deformable attention module, and a progressive decoder.
[0063] The range image obtained by spherical projection is input into the pre-trained feature extractor to obtain multi-scale range view features. Multi-scale semantic features can capture object information in the scene from both global and local perspectives for subsequent semantic occupancy prediction.
[0064] S4. Perform feature aggregation on multi-scale range view features to obtain voxel features of the three-dimensional scene.
[0065] Spherical projection will inevitably compress the spatial structure of the original point cloud, while the goal of the semantic occupancy prediction task is to infer the occupancy state and semantic category of each voxel grid in the voxel space. In addition, the geometric structure of the original point cloud is sparse and discrete, and it is difficult to preserve the semantic information of the range view. Therefore, as an optional implementation, a voxel query of a voxel grid shape is defined in the voxel space, and each voxel query dynamically captures the features of interest on the multi-scale range view features through a deformable attention mechanism, thereby obtaining the voxel features of the three-dimensional scene.
[0066] S5. Refine and enhance the voxel features of the three-dimensional scene and generate voxel-by-voxel classification probabilities to achieve semantic occupancy prediction of the three-dimensional scene.
[0067] There are a lot of blank areas in the scene, that is, the voxels are unoccupied, while the occupied areas are often more concerned. Therefore, as an optional implementation, a progressive decoder is introduced to refine and enhance the voxel features, thereby improving the accuracy of the prediction. The progressive decoder consists of several parts: a dense voxel feature extractor, a sparse voxel feature extractor, and an occupancy prediction classifier. The dense voxel feature extractor is implemented based on standard 3D convolution, which is used to refine the dense voxel features obtained in the previous step, and then input it into the classifier to generate the occupancy prediction result. According to the prediction results, empty voxels and occupied voxels can be filtered from the voxel grid by the confidence of the occupancy state. Since a large number of empty voxels are filtered, the current voxel features are sparse, corresponding to the occupied voxels. According to this feature, a sparse voxel feature extractor based on sparse convolution is proposed, which only focuses on the position of occupied voxels and can be used for the extraction of sparse voxel features. It is worth noting that if the current voxel grid scale is inconsistent with the target scale, the voxel features of the target scale can be obtained by coordinate mapping. Finally, the voxel features are input into the occupancy prediction classifier to obtain the semantic occupancy prediction results.
[0068] In some embodiments, during the training of the semantic occupancy prediction model, the loss is calculated by the target loss function, and the model parameters are optimized by the SGD algorithm. The target loss function includes the weighted cross entropy loss lovasz-softmax loss Scenario-Category Affinity Loss and Right now:
[0069]
[0070] Given a voxel label O, the overall objective loss is:
[0071]
[0072] The above method is explained in detail below in conjunction with specific embodiments.
[0073] The embodiment of the present invention applies the above-provided method to the commonly used large-scale outdoor scene dataset nuScenes-Occupancy, which has 16 semantic categories, including 700 training scenes and 150 verification scenes. The semantic occupancy prediction range of this dataset is: from -51.2m to 51.2m along the X-axis and Y-axis, respectively, and from -5m to 3m along the Z-axis, with a 512×512×40 voxel grid, and each voxel size is 0.2m×0.2m×0.2m. On the nuScenes-Occupancy dataset, the semantic occupancy prediction results are shown in Table 1 below.
[0074] Table 1 Experimental results of Example 2 on the nuScenes-Occupancy dataset
[0075]
[0076] In summary, the method of this embodiment projects a sparse point cloud with discontinuous spatial information and non-compact contextual content into a spherical coordinate system to obtain a continuous and compact range view representation; extracts multi-scale range view features through a pre-trained feature extractor to capture object information in the scene from a global and local perspective; transfers the two-dimensional range view semantic features to the voxel space through a deformable attention module to obtain a voxel representation of the three-dimensional scene; refines and enhances the voxel features through a progressive decoder, and generates a semantic occupancy prediction result. The method of this embodiment effectively improves the accuracy of semantic occupancy prediction of sparse point clouds and alleviates the dependence on multi-frame point cloud data.
[0077] Example 2
[0078] This embodiment provides a sparse point cloud semantic occupancy prediction device based on spherical projection, comprising:
[0079] Acquisition module, used to obtain point cloud data of LiDAR;
[0080] A mapping module is used to perform spherical projection on the obtained point cloud data to obtain a range view representation of the point cloud;
[0081] The processing module is used to input the range view representation of the point cloud into the trained semantic occupancy prediction model, perform feature extraction on the range view representation to obtain multi-scale range view features; perform feature aggregation on the multi-scale range view features to obtain voxel features of the three-dimensional scene; refine and enhance the voxel features of the three-dimensional scene, and generate voxel-by-voxel classification probabilities to achieve semantic occupancy prediction of the three-dimensional scene.
[0082] Since the device is a device corresponding to the sparse point cloud semantic occupancy prediction based on spherical projection of an embodiment of the present invention, and the principle of solving the problem by the device is similar to that of the method, the implementation of the device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.
[0083] Example 3
[0084] See also Figure 3 An embodiment of the present invention further provides an electronic device, the electronic device comprising a processor and a memory, the memory storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to implement the following Figure 1 and / or Figure 2 The shown method is an efficient semantic occupancy prediction method for sparse point clouds based on spherical projection.
[0085] It is understood that the memory may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory may be used to store instructions, programs, codes, code sets, or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data created according to the use of the server, etc.
[0086] The processor may include one or more processing cores. The processor uses various interfaces and lines to connect the various parts of the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory, and calling data stored in the memory. Optionally, the processor can be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor can integrate one or a combination of a central processing unit (CPU) and a modem. Among them, the CPU mainly processes the operating system and application programs; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor, but implemented separately through a chip.
[0087] Since the electronic device is an electronic device corresponding to a sparse point cloud semantic occupancy prediction method based on spherical projection in an embodiment of the present invention, and the principle of solving the problem by the electronic device is similar to that of the method, the implementation of the electronic device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.
[0088] Example 4
[0089] The embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, the at least one program, the code set or instruction set is loaded and executed by a processor to implement the following Figure 1 and / or Figure 2 The shown method is an efficient semantic occupancy prediction method for sparse point clouds based on spherical projection.
[0090] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.
[0091] Since the storage medium is a storage medium corresponding to a sparse point cloud semantic occupancy prediction method based on spherical projection in an embodiment of the present invention, and the principle of solving the problem by the storage medium is similar to that of the method, the implementation of the storage medium can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.
[0092] Example 5
[0093] In some possible implementations, various aspects of the method of the embodiment of the present invention may also be implemented in the form of a program product, which includes a program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of a sparse point cloud semantic occupancy prediction method based on spherical projection according to various exemplary embodiments of the present application described above in this specification. Among them, the executable computer program code or "code" for executing each embodiment can be written in a high-level programming language such as C, C++, C#, Smalltalk, Java, JavaScript, Visual Basic, structured query language (e.g., Transact-SQL), Perl, or in various other programming languages.
[0094] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0095] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0096] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable ordinary technicians in the field to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made based on the essence of the content of the present invention should be included in the protection scope of the present invention.
Claims
1. A sparse point cloud semantic occupancy prediction method based on spherical projection, characterized in that: The following steps are involved: Get the point cloud data of LiDAR; Perform spherical projection on the acquired point cloud data to obtain a range view representation of the point cloud; The range view representation of the point cloud is input into the semantic occupancy prediction model, and features are extracted from the range view representation to obtain multi-scale range view features; Perform feature aggregation on multi-scale range view features to obtain voxel features of the three-dimensional scene; The voxel features of the 3D scene are refined and enhanced, and voxel-by-voxel classification probabilities are generated to achieve semantic occupancy prediction of the 3D scene.
2. The sparse point cloud semantic occupancy prediction method based on spherical projection according to claim 1, characterized in that: The step of performing spherical projection on the obtained point cloud data to obtain a range view representation of the point cloud includes: The three-dimensional coordinates (x, y, z) of the point cloud data are mapped point by point through a spherical projection formula to obtain pixel coordinates (u, v); The spherical projection formula is: Where W and H represent the width and height of the projected image, respectively. Represents the depth value of the projected point cloud, f v =|f up |+|f down | represents the vertical field of view of the laser radar, |f up | represents the field of view in the horizontal upward direction, |f down |Indicates the field of view angle in the horizontal downward direction.
3. The sparse point cloud semantic occupancy prediction method based on spherical projection according to claim 1, characterized in that: The semantic occupancy prediction model is constructed in the following way: Construct a semantic occupancy prediction model using convolutional layers, batch normalization layers, linear rectification layers, pooling layers, and linear layers; Among them, the constructed semantic occupancy prediction model includes a feature extractor, a deformable attention module, and a progressive decoder; The feature extractor is used to extract features from the range view representation to obtain multi-scale range view features; The deformable attention module is used to combine predefined voxel queries to perform feature aggregation on multi-scale range view features to obtain voxel features of the three-dimensional scene; The progressive decoder is used to refine and enhance the voxel features of the three-dimensional scene and generate voxel-by-voxel classification probabilities.
4. The sparse point cloud semantic occupancy prediction method based on spherical projection according to claim 3, characterized in that: The deformable attention module works as follows: Voxel queries with voxel grid shapes are defined in voxel space, and each voxel query dynamically captures features of interest on multi-scale range view features through a deformable attention mechanism, thereby obtaining voxel features of the 3D scene.
5. The sparse point cloud semantic occupancy prediction method based on spherical projection according to claim 1, characterized in that: The progressive decoder includes a dense voxel feature extractor, a sparse voxel feature extractor and an occupancy prediction classifier; the dense voxel feature extractor is used to extract dense voxel features, and then the extracted dense voxel features are input into the occupancy prediction classifier to generate an occupancy prediction result. The sparse voxel feature extractor is used to extract sparse voxel features, and then input the extracted sparse voxel features into the occupancy prediction classifier to generate an occupancy prediction result.
6. The sparse point cloud semantic occupancy prediction method based on spherical projection according to claim 1 or 5, characterized in that: The overall objective loss used to train the semantic occupancy prediction model for: Where o represents the voxel label, and represents the prediction result of the progressive decoder; is the target loss function.
7. The method for predicting semantic occupancy of sparse point clouds based on spherical projection according to claim 6, characterized in that: The objective loss function The expression is: In the formula, represents the weighted cross entropy loss, represents the lovasz-softmax loss, and represents the scene-category affinity loss.
8. A sparse point cloud semantic occupancy prediction device based on spherical projection, characterized in that: include: Acquisition module, used to obtain point cloud data of LiDAR; A mapping module is used to perform spherical projection on the obtained point cloud data to obtain a range view representation of the point cloud; A processing module, used for inputting the range view representation of the point cloud into the semantic occupancy prediction model, performing feature extraction on the range view representation to obtain multi-scale range view features; performing feature aggregation on the multi-scale range view features to obtain voxel features of the three-dimensional scene; The voxel features of the 3D scene are refined and enhanced, and voxel-by-voxel classification probabilities are generated to achieve semantic occupancy prediction of the 3D scene.
9. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement a sparse point cloud semantic occupancy prediction method based on spherical projection as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement a sparse point cloud semantic occupancy prediction method based on spherical projection as described in any one of claims 1 to 7.