Logistics vehicle obstacle perception method and system based on sparse semantic occupancy network

Abstract

The application provides a logistics vehicle obstacle perception method and system based on a sparse semantic occupancy network. The method obtains perception information within a preset range of an unmanned logistics vehicle, fuses current image information and current laser point clouds, performs down-sampling on a bird's-eye view feature body based on a pre-constructed sparse occupancy prediction model, performs binary occupancy prediction on the down-sampled bird's-eye view feature body, extracts bird's-eye view features from the bird's-eye view feature body based on a binary occupancy mask, performs occupancy prediction on sparse features, performs geometric correction on occupancy probabilities and signed distance field values of the sparse features based on a pre-constructed double-flow network, performs semantic correction on semantic category probability distribution, performs time-series feature fusion on historical bird's-eye view features, corrected geometric features and corrected semantic features, and decodes the time-series fused features. The application reduces the number of voxels that need to calculate high-resolution features, and can run in real time on an unmanned logistics vehicle with low computing power.

Description

Technical Field

[0001] This application relates to the field of autonomous driving perception technology, and in particular to a method and system for obstacle perception of logistics vehicles based on sparse semantic occupancy networks. Background Technology

[0002] In the operation scenario of L4 level urban unmanned logistics vehicles, the environment is extremely complex, such as urban villages, narrow streets, and areas where people and vehicles mix. There are a large number of non-standard obstacles (fallen cones, scattered cardboard boxes, protruding tree branches, construction fences, etc.). These non-standard obstacles are usually not in the "whitelist" of conventional detection models, which makes it easy for bounding box-based perception methods to miss detection and cause safety accidents.

[0003] In related technologies, occupancy networks are used to divide 3D space into voxels and predict the probability of each voxel being occupied. This method can detect obstacles of arbitrary shapes without relying on preset categories. However, dense occupancy networks are computationally intensive, requiring high-resolution voxels to capture fine geometry, making it difficult to run in real time on the limited computing power of embedded chips (such as a single Orin-X or lower computing power platforms) typically equipped in logistics vehicles. In addition, pure occupancy networks lack semantic information, making it difficult for the planning module to distinguish between "pushable grass" and "non-collision stone blocks," resulting in imprecise decision-making.

[0004] Therefore, there is an urgent need for a comprehensive perception scheme that utilizes multimodal sensors (camera + low-cost LiDAR) for sparse occupancy perception, which can ensure computational efficiency, provide semantic information, and improve the accuracy of dynamic obstacle perception. Summary of the Invention

[0005] In view of the shortcomings of the prior art described above, this application provides a method and system for obstacle perception of logistics vehicles based on sparse semantic occupancy networks to solve the above-mentioned technical problems.

[0006] According to one aspect of the embodiments of this application, a method for obstacle perception of a logistics vehicle based on a sparse semantic occupancy network is provided, comprising: acquiring perception information within a preset range of an unmanned logistics vehicle; the perception information including: current image information, current laser point cloud, historical image information, and historical laser point cloud; fusing the current image information and the current laser point cloud to obtain a bird's-eye view feature volume; downsampling the bird's-eye view feature volume based on a pre-constructed sparse occupancy prediction model, and performing binarized occupancy prediction on the downsampled bird's-eye view feature volume to generate a binarized occupancy mask; extracting bird's-eye view features from the bird's-eye view feature volume based on the binarized occupancy mask to obtain sparse features; and further... The sparse features are used for occupancy prediction to obtain the occupancy probability, signed distance field value, semantic category probability distribution, and running speed of the sparse features. Based on a pre-constructed two-stream network, the occupancy probability and signed distance field value of the sparse features are geometrically corrected to obtain corrected geometric features, and the semantic category probability distribution is semantically corrected to obtain corrected semantic features. The historical bird's-eye view features, the corrected geometric features, and the corrected semantic features are fused in a temporal sequence to obtain temporal fused features. The temporal fused features are decoded to obtain obstacle perception results. The historical bird's-eye view features are obtained by fusing the historical image information and the historical laser point cloud.

[0007] According to another aspect of the embodiments of this application, a logistics vehicle obstacle perception system based on a sparse semantic occupancy network is also provided, comprising: an information acquisition module for acquiring perception information within a preset range of an unmanned logistics vehicle; the perception information including: current image information, current laser point cloud, historical image information, and historical laser point cloud; an information fusion module for fusing the current image information and the current laser point cloud to obtain a bird's-eye view feature volume; and an occupancy prediction module for downsampling the bird's-eye view feature volume based on a pre-constructed sparse occupancy prediction model, and performing binarized occupancy prediction on the downsampled bird's-eye view feature volume to generate a binarized occupancy mask; and extracting bird's-eye view features from the bird's-eye view feature volume based on the binarized occupancy mask to obtain... The system obtains sparse features; performs occupancy prediction on the sparse features to obtain the occupancy probability, signed distance field value, and semantic category probability distribution of the sparse features; a feature correction module is used to perform geometric correction on the occupancy probability and signed distance field value of the sparse features based on a pre-constructed two-stream network to obtain corrected geometric features, and performs semantic correction on the semantic category probability distribution to obtain corrected semantic features; a perception result determination module is used to perform temporal feature fusion on historical bird's-eye view features, the corrected geometric features, and the corrected semantic features to obtain temporal fusion features; decodes the temporal fusion features to obtain obstacle perception results; the historical bird's-eye view features are obtained by fusing historical image information and historical laser point clouds.

[0008] The beneficial effects of this application are as follows: This application acquires perception information within a preset range of an unmanned logistics vehicle, fuses current image information and current laser point cloud to obtain a bird's-eye view feature volume, downsamples the bird's-eye view feature volume based on a pre-built sparse occupancy prediction model, performs binarized occupancy prediction on the downsampled bird's-eye view feature volume, generates a binarized occupancy mask, extracts bird's-eye view features from the bird's-eye view feature volume based on the binarized occupancy mask, obtains sparse features, performs occupancy prediction on the sparse features, and obtains the occupancy probability, signed distance field value, semantic category probability distribution, and running speed of the sparse features. Based on the pre-built dual... The flow network performs geometric corrections on the occupancy probabilities and signed distance field values ​​of sparse features to obtain corrected geometric features, and semantic corrections on the semantic category probability distribution to obtain corrected semantic features. Temporal feature fusion is then performed on historical bird's-eye view features, corrected geometric features, and corrected semantic features to obtain temporal fused features. Decoding these temporal fused features yields the obstacle perception results. This process, by first downsampling and binarizing the bird's-eye view feature volume for occupancy prediction, quickly identifies sparse regions where obstacles may exist. High-precision attribute prediction is performed only on sparse features, significantly reducing the need to compute high-resolution features. The limited number of voxels (typically only 10%–20% of the total) enables this method to run in real-time on low-computing-power platforms for unmanned logistics vehicles, effectively solving the technical challenge of deploying high-order perception algorithms at the edge in complex scenarios. By using a pre-built dual-stream network to geometrically correct the occupancy probability of sparse features and the signed distance field value, and semantically correcting the semantic category probability distribution, the method further improves the reconstruction accuracy of obstacle edges and microstructures, enhancing the geometric perception capability for non-standard obstacles and effectively avoiding scratches or collisions caused by missed detection of irregularly shaped obstacles. Through the correction of geometric features and semantics… The fusion of temporal features provides a more interpretable environmental description for downstream planning and control modules, enabling logistics vehicles to make more human-like and reasonable driving decisions, significantly improving traffic efficiency and safety. The fusion of historical bird's-eye view features, corrected geometric features, and semantic features can effectively utilize historical information to supplement the observation gaps in the current frame (e.g., objects that are temporarily occluded), smooth the jitter of perception results, and accurately estimate the movement speed of obstacles. This is especially important for handling dynamic targets that suddenly appear in areas where people and vehicles mix, further improving the robustness of the logistics vehicle obstacle perception system in dynamic and complex environments.

[0009] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0010] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0011] Figure 1 This is a schematic diagram illustrating an exemplary system architecture as shown in an exemplary embodiment of this application;

[0012] Figure 2 This is a flowchart illustrating an exemplary embodiment of the present application of a logistics vehicle obstacle perception method based on a sparse semantic occupancy network;

[0013] Figure 3 This is a flowchart illustrating a logistics vehicle obstacle perception method based on a sparse semantic occupancy network, which is another exemplary embodiment of this application.

[0014] Figure 4 This is a flowchart illustrating the sparsity processing of an exemplary embodiment of this application;

[0015] Figure 5 This is a flowchart illustrating a logistics scenario involving fallen objects on the ground, as shown in an exemplary embodiment of this application.

[0016] Figure 6 This is a flowchart illustrating a logistics scenario with suspended obstacles, as shown in an exemplary embodiment of this application.

[0017] Figure 7 This is a block diagram illustrating an obstacle perception system for logistics vehicles based on a sparse semantic occupancy network, as shown in an exemplary embodiment of this application. Detailed Implementation

[0018] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0019] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. The drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the shape, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0020] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the present application. However, it will be apparent to those skilled in the art that embodiments of the present application may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the present application.

[0021] Figure 1 This is a schematic diagram illustrating an exemplary system architecture as shown in an exemplary embodiment of this application.

[0022] Reference Figure 1 As shown, the system architecture may include an acquisition device 101 and a processing device 102. The processing device 102 may be at least one of a desktop graphics processing unit (GPU) computer, a GPU computing cluster, a neural network computer, etc. Technical personnel can use the processing device 102 to acquire perception information within a preset range of the unmanned logistics vehicle, fuse the current image information and the current laser point cloud to obtain a bird's-eye view feature volume, downsample the bird's-eye view feature volume based on a pre-built sparse occupancy prediction model, and perform binarized occupancy prediction on the downsampled bird's-eye view feature volume to generate a binarized occupancy mask. Based on the binarized occupancy mask, bird's-eye view features are extracted from the bird's-eye view feature volume to obtain sparse features. Occupancy prediction is performed on the sparse features to obtain the occupancy probability, signed distance field value, semantic category probability distribution, and running speed of the sparse features. Based on a pre-built two-stream network, the occupancy probability and signed distance field value of the sparse features are geometrically corrected to obtain corrected geometric features, and the semantic category probability distribution is semantically corrected to obtain corrected semantic features. The historical bird's-eye view features, corrected geometric features, and corrected semantic features are fused temporally to obtain temporally fused features. The temporally fused features are decoded to obtain the obstacle perception result. The data acquisition device 101 is used to collect perception information within a preset range of the unmanned logistics vehicle and provide the perception information within the preset range of the unmanned logistics vehicle to the processing device 102 for processing.

[0023] The implementation details of the technical solutions in the embodiments of this application are described in detail below:

[0024] Figure 2This is a flowchart illustrating an exemplary embodiment of the obstacle perception method for logistics vehicles based on sparse semantic occupancy networks. (Refer to...) Figure 2 As shown, the obstacle perception method for logistics vehicles based on sparse semantic occupancy networks includes at least steps S210 to S250, which are described in detail below:

[0025] In step S210, perception information within a preset range of the unmanned logistics vehicle is acquired. In one embodiment of this application, the perception information includes: current image information, current laser point cloud, historical image information, and historical laser point cloud. The preset range is set according to actual conditions. The current image information is acquired by a camera or surround-view camera, the current laser point cloud is acquired by a 16-line or 32-line LiDAR, and the historical image information and historical laser point cloud come from historical frame acquisition data stored in the unmanned logistics vehicle perception system.

[0026] In step S220, the current image information and the current laser point cloud are fused to obtain a bird's-eye view feature. In one embodiment of this application, by fusing the current image information and the current laser point cloud, the obtained bird's-eye view feature retains the complete spatial geometric and semantic information in the perceived information, providing a reliable foundation for subsequent sparse extraction and high-precision prediction.

[0027] In step S230, based on the pre-built sparse occupancy prediction model, the bird's-eye view feature volume is downsampled, and the downsampled bird's-eye view feature volume is subjected to binarized occupancy prediction to generate a binarized occupancy mask. Based on the binarized occupancy mask, bird's-eye view features are extracted from the bird's-eye view feature volume to obtain sparse features. Occupancy prediction is performed on the sparse features to obtain the occupancy probability, signed distance field value, semantic category probability distribution, etc. of the sparse features. In one embodiment of this application, the number of voxels in the bird's-eye view feature volume is 200×200×8, and the number of voxels in the downsampled bird's-eye view feature volume is 50×50×4. The downsampling rate is kept at one-quarter, which effectively compresses the overall computational load without excessively losing the key information of the original features, adapting to the computing power limitations of the edge. By downsampling and binarized occupancy prediction of the bird's-eye view feature volume, sparse regions that may contain obstacles are quickly screened out, and high-precision attribute prediction is performed only for sparse features. This significantly reduces the number of voxels that need to be calculated for high-resolution features (usually only accounting for 10%~20% of the total).

[0028] In step S240, based on the pre-built two-stream network, the occupancy probability and signed distance field value of sparse features are geometrically corrected to obtain corrected geometric features, and the semantic category probability distribution is semantically corrected to obtain corrected semantic features. In one embodiment of this application, the pre-built two-stream network includes a geometric correction branch and a semantic correction branch. The geometric correction branch uses residual convolution to recover the occupancy probability and signed distance field value of sparse features for fine-tuning, restoring the fine spatial information lost during downsampling, making the output obstacle position and boundary information more consistent with the actual scene. The semantic correction branch addresses the semantic ambiguity problem caused by downsampling by recalibrating the semantic category probability distribution to improve the classification accuracy of small-volume obstacles and semantically similar obstacles. The geometric correction branch and the semantic correction branch share some basic feature extraction structures. This design effectively avoids redundant calculations while ensuring the effect of multi-dimensional feature correction, keeping the additional computational overhead within a very low range, and is highly adaptable to vehicle-side operating platforms with limited computing power.

[0029] In step S250, the historical bird's-eye view features, corrected geometric features, and corrected semantic features are fused temporally to obtain temporally fused features; the temporally fused features are then decoded to obtain the obstacle perception result. In one embodiment of this application, the historical bird's-eye view features are obtained by fusing historical image information and historical laser point clouds. The process of obtaining historical bird's-eye view features by fusing historical image information and historical laser point clouds is the same as the process of fusing current image information and current laser point clouds to obtain the bird's-eye view feature volume.

[0030] In one embodiment of this application, by first downsampling and binarizing the bird's-eye view feature volume for occupancy prediction, sparse regions where obstacles may exist are quickly screened out. High-precision attribute prediction is then performed only for sparse features, which significantly reduces the number of voxels that need to be computed for high-resolution features (typically only accounting for 10% to 20% of the total). This allows the method to run in real time on unmanned logistics vehicles with low computing power platforms, effectively solving the technical challenge of deploying high-order perception algorithms at the edge in complex scenarios. Furthermore, by using a pre-constructed two-stream network to geometrically correct the occupancy probability and signed distance field value of sparse features, and semantically correcting the semantic category probability distribution, the restoration accuracy of obstacle edges and microstructures is further improved, enhancing the geometric perception of non-standard obstacles. The system's obstacle perception capabilities effectively prevent scratches or collisions caused by missed detection of irregularly shaped obstacles. By fusing temporal features of corrected geometric and semantic features, it provides a more interpretable environmental description for downstream planning and control modules, enabling logistics vehicles to make more human-like and reasonable driving decisions, significantly improving traffic efficiency and safety. The fusion of historical bird's-eye view features, corrected geometric features, and semantic features can effectively utilize historical information to supplement the observation gaps in the current frame (e.g., objects that are temporarily occluded), smooth out jitter in perception results, and accurately estimate the speed of obstacles. This is particularly important for handling dynamic targets that suddenly appear in mixed pedestrian and vehicle areas, further enhancing the robustness of the logistics vehicle obstacle perception system in dynamic and complex environments.

[0031] In one embodiment of this application, the process of extracting bird's-eye view features from the bird's-eye view feature volume based on a binarized occupancy mask to obtain sparse features includes:

[0032] The occupied voxel grid is determined based on the binarized occupancy mask. In one embodiment of this application, the voxel grid with a value of 1 in the binarized occupancy mask is taken as the occupied voxel grid; the occupied voxel grid is derived from the grid space determined by the downsampled bird's-eye view feature volume, that is, the occupied voxel grid is determined in the low-resolution grid space.

[0033] The occupied voxel mesh is mapped to the mesh space defined by the bird's-eye view feature volume, resulting in a mapped voxel mesh in the mesh space defined by the bird's-eye view feature volume. Features from the mapped voxel mesh are then extracted as sparse features. In one embodiment of this application, the process of mapping the occupied voxel mesh to the mesh space defined by the bird's-eye view feature volume can be implemented using linear scaling or receptive field correspondence methods. The occupied voxel mesh is defined in a low-resolution mesh space, while the sparse features are extracted from a high-resolution bird's-eye view feature volume. This setup preserves the complete and detailed features of the high-resolution bird's-eye view feature volume while extracting features only from areas where obstacles may exist, avoiding invalid computation of a large number of empty voxels and effectively controlling the overall computational load while ensuring perception accuracy.

[0034] In one embodiment of this application, the process of pre-constructing a sparse occupancy prediction model includes:

[0035] This invention acquires historical video data and historical point cloud data of unmanned logistics vehicles in urban scenarios. In one embodiment of this application, historical video data is collected using cameras or surround-view cameras, and historical point cloud data is collected using 16-line or 32-line LiDAR.

[0036] Historical video data and historical point cloud data are fused to obtain 3D fused data. In one embodiment of this application, the process of fusing historical video data and historical point cloud data to obtain 3D fused data is the same as the process of fusing current image information and current laser point cloud to obtain bird's-eye view feature volume.

[0037] Using 3D fusion data as unlabeled sample data, a pre-defined feature extractor is trained under self-supervised conditions to obtain the trained feature extractor. In one embodiment of this application, the pre-defined feature extractor is used to extract semantic features, geometric features, etc., from the 3D fusion data. By using the 3D fusion data as unlabeled sample data and training the pre-defined feature extractor under self-supervised conditions, the trained feature extractor can automatically learn the potential feature distribution of different obstacles in urban scenes. It can obtain good general feature extraction capabilities without a large amount of manually labeled data, significantly reducing the labeling cost of training the pre-built sparse occupancy prediction model, and is also more adaptable to the fine-tuning needs of dynamic changes in logistics vehicle operation scenarios.

[0038] Information annotation is performed on the 3D fusion data to obtain sample data with annotation information. In one embodiment of this application, the annotation information includes occupancy status, semantic category, motion speed, and signed distance field value. The occupancy status is used to mark whether the voxel mesh is occupied by an obstacle, the semantic category is used to mark the specific category of the corresponding obstacle, the motion speed is used to mark the motion parameters of the dynamic obstacle, and the signed distance field value is used to describe the distance and direction from the voxel mesh to the nearest obstacle surface.

[0039] A pre-constructed sparse occupancy prediction model is obtained by training a pre-defined sparse occupancy prediction model using labeled sample data. In one embodiment of this application, the pre-defined sparse occupancy prediction model includes: a trained feature extractor, a pre-defined view transformation and sparsification module, a pre-defined sparse prediction head, and a pre-defined task output branch. The trained feature extractor is used to perform noise filtering, semantic feature extraction, and geometric feature extraction on the 3D fusion data. The pre-defined view transformation and sparsification module is used to downsample the 3D fusion data and perform binarized occupancy prediction on the downsampled 3D fusion data. Based on the occupancy prediction result, bird's-eye view features are extracted from the 3D fusion data to obtain sample sparse features. The pre-defined sparse prediction head performs occupancy prediction on the sample sparse features to obtain hidden feature representations of sample geometric features, sample semantic features, and sample occupancy probabilities. The pre-defined task output branch is used to decode the hidden feature representations of sample geometric features, sample semantic features, and sample occupancy probabilities to obtain sample geometric features, sample semantic features, and sample occupancy probabilities. The pre-built sparse occupancy prediction model consists of a feature extractor with adjusted parameters, a view transformation and sparsification module with adjusted parameters, a sparse prediction head with adjusted parameters, and a task output branch with adjusted parameters.

[0040] In one embodiment of this application, a preset feature extractor is trained in a self-supervised manner by using 3D fused data as unlabeled sample data. Then, a preset sparse occupancy prediction model is trained using labeled sample data. This approach leverages the general feature extraction capabilities obtained through self-supervised learning as a foundation, and fine-tuning with a small amount of labeled data can achieve the desired perception effect. This effectively reduces the dependence on large-scale labeled datasets while ensuring the perception accuracy of the trained preset sparse occupancy prediction model in urban unmanned logistics scenarios. It also reduces the dataset preparation cost and training time for training the preset sparse occupancy prediction model.

[0041] In one embodiment of this application, to meet the real-time control requirements of autonomous driving for output frequencies of 10Hz-20Hz, the method in this application performs multi-dimensional inference acceleration optimization on the preset sparse occupancy prediction model: at the underlying computational level, the preset sparse occupancy prediction model is quantized by INT8 and deployed using TensorRT or a dedicated inference engine, significantly improving basic computational efficiency; at the algorithm level, a sparse inference strategy is adopted, using sparse convolution (e.g., Minkowski Engine) or custom sparse operators to calculate only the activated voxels, effectively eliminating redundant computations on empty areas; at the scene adaptation level, the method in this application can dynamically adjust the BEV perception range according to vehicle speed and scene complexity (e.g., reducing the range at low speeds to improve resolution, and expanding the range at high speeds to ensure far-end perception), thereby achieving the best balance between perception performance and operating speed under limited computing power.

[0042] In one embodiment of this application, the process of using 3D fused data as unlabeled sample data to perform self-supervised training on a preset feature extractor includes:

[0043] According to the occlusion ratio, the unlabeled sample data is occluded to obtain occluded sample data; based on a preset feature extractor, the occluded sample data is reconstructed to obtain reconstructed data. In one embodiment of this application, the occlusion ratio is between 50% and 75%. The process of reconstructing the occluded sample data based on the preset feature extractor includes: extracting features from the occluded sample data to obtain a latent feature representation, and decoding the latent feature representation using a decoder to obtain the reconstructed data.

[0044] With the goal of minimizing the difference between reconstructed data and unlabeled sample data, the parameters in a preset feature extractor are adjusted to obtain a trained feature extractor. In one embodiment of this application, the process of adjusting the parameters in the preset feature extractor with the goal of minimizing the difference between reconstructed data and unlabeled sample data includes: calculating the losses of the reconstructed data and unlabeled sample data in terms of point cloud geometry and image semantic information, obtaining point cloud geometry loss and image semantic information loss; weighted summing of point cloud geometry loss and image semantic information loss to obtain total loss, and then backpropagating the total loss layer by layer through the backpropagation algorithm to update the trainable parameters of each layer in the preset feature extractor in sequence; repeating the iterative process of occlusion, feature extraction, reconstruction, and parameter update until the total loss converges or reaches the preset maximum number of iterations, stopping training and outputting the trained feature extractor.

[0045] In one embodiment of this application, the difference between the reconstructed data and the unlabeled sample data is characterized by a first total loss function, the expression of which is as follows:

[0046] Equation (1)

[0047] in, Denotes the first total loss function. The weights of the point cloud geometry loss term are represented. This represents the point cloud geometric structure loss. The weights represent the image semantic information loss terms. The image semantic information loss is represented by the sum of the weights of the point cloud geometric structure loss term and the image semantic information loss term, which equals 1.

[0048] The formula for calculating the point cloud geometric structure loss is as follows:

[0049] Equation (2)

[0050] in, This represents the point cloud geometric structure loss. The geometric properties of the reconstructed data after normalization are represented in vector form. The geometric attributes of the normalized, unlabeled sample data are represented as vectors. These attributes include the spatial coordinates of each sampling point in three-dimensional space and the voxel occupancy status.

[0051] The formula for calculating the loss of semantic information in an image is shown below:

[0052] Equation (3)

[0053] in, This represents the loss of semantic information in the image. This represents the semantic category probability distribution of unlabeled sample data. The semantic category probability distribution of the reconstructed data is obtained by classifying the reconstructed data using a point cloud data classification model (e.g., an MLP (Multilayer Perceptron)-based point cloud classification model) and then activating it using a softmax function. The semantic category probability distribution of unlabeled sample data is obtained by classifying unlabeled sample data using a point cloud data classification model (e.g., an MLP (Multilayer Perceptron)-based point cloud classification model) and then activating it using a softmax function.

[0054] In one embodiment of this application, the process of training a preset sparse occupancy prediction model using sample data with labeled information includes:

[0055] Using a sliding window mechanism, sample data with labeled information is selected to obtain selected sample data; the selected sample data is then input into a preset sparse occupancy prediction model to obtain predicted data. In one embodiment of this application, the occurrence time of the predicted data is after the occurrence time of the selected sample data; the size of the sliding window is set according to the actual situation.

[0056] With the objective of minimizing the differences between the occupancy state of the predicted data and the occupancy state of the selected sample data, the differences between the semantic categories of the predicted data and the selected sample data, the differences between the movement speed of the predicted data and the movement speed of the selected sample data, and the differences between the signed distance field values ​​of the predicted data and the selected sample data, the parameters in the preset sparse occupancy prediction model are adjusted to obtain the pre-constructed sparse occupancy prediction model. In one embodiment of this application, if the differences between the occupancy state of the predicted data and the occupancy state of the selected sample data, the differences between the semantic categories of the predicted data and the selected sample data, the differences between the movement speed of the predicted data and the movement speed of the selected sample data, and the differences between the signed distance field values ​​of the predicted data and the selected sample data are characterized by a second total loss function, the expression of which is as follows:

[0057] Equation (4)

[0058] in, This represents the second loss function. Indicates the weight of the occupied state loss. Indicates the loss of occupancy status. Represents the semantic category loss weights. Represents semantic category loss, This indicates the weight of the running speed loss. This indicates a loss in operating speed. This represents the signed distance field value loss weight. This indicates that the sum of the signed distance field value loss weight, the occupied state loss weight, the semantic category loss weight, the running speed loss weight, and the signed distance field value loss weight equals 1.

[0059] The formula for calculating the occupancy state loss is as follows:

[0060] Equation (5)

[0061] in, Indicates the loss of occupancy status. The weights represent the binary cross-entropy loss. This represents the occupancy state vector of the selected sample data. Represents the occupancy state vector of the predicted data. Indicates the focus loss weight, This represents the binary cross-entropy loss between the occupied state vector of the selected sample data and the occupied state vector of the predicted data. The focus loss represents the difference between the occupancy state vector of the selected sample data and the occupancy state vector of the predicted data. The sum of the weights of the binary cross-entropy loss and the focus loss equals 1. Formula (5) sets the binary cross-entropy loss as the basic loss to ensure that the preset sparse occupancy prediction model can quickly converge to a reasonable baseline level and prevent deviation. The focus loss performs very well when dealing with long-tailed distributions and difficult samples. In the later stages of training or for difficult regions (e.g., objects with severe occlusion), it can provide stronger gradient signals. The combination of binary cross-entropy loss and focus loss utilizes both the stability of binary cross-entropy loss and the specificity of focus loss, effectively improving the overall accuracy of the pre-built sparse occupancy prediction model in predicting occupancy state. In particular, it improves the recall rate of small obstacles and occluded obstacles and solves the extreme class imbalance problem.

[0062] The formula for calculating the binary cross-entropy loss is as follows:

[0063] Equation (6)

[0064] in, This represents the binary cross-entropy loss between the occupied state vector of the selected sample data and the occupied state vector of the predicted data. This represents the occupancy state vector of the selected sample data. This represents the occupancy state vector of the predicted data.

[0065] The formula for calculating focus loss is as follows:

[0066] Equation (7)

[0067] in, This represents the focus loss between the occupied state vector of the selected sample data and the occupied state vector of the predicted data. This represents the occupancy state vector of the selected sample data. Represents the occupancy state vector of the predicted data. Indicates the focus parameter, This represents the equilibrium parameter.

[0068] The formula for calculating semantic category loss is as follows:

[0069] Equation (8)

[0070] in, Represents semantic category loss, This represents the semantic category probability distribution of the selected sample data. This represents the semantic category probability distribution of the selected sample data. This represents the focus loss between the semantic category probability distribution of the selected sample data and the semantic category probability distribution of the selected sample data.

[0071] The formula for calculating the operating speed loss is as follows:

[0072] Equation (9)

[0073] in, This indicates a loss in operating speed. This indicates the motion velocity vector of the selected sample data. This represents the velocity vector of the predicted data. This represents the L1 norm.

[0074] The formula for calculating the signed distance field value loss is as follows:

[0075] Equation (10)

[0076] in, This represents the signed distance field value loss. This represents the signed distance field value vector of the selected sample data. This represents the signed distance field value vector of the predicted data. This represents the threshold hyperparameter.

[0077] In one embodiment of this application, if the amount of data used to train the detection task is much larger than the amount of data used to train the occupancy state, gradient accumulation is used to balance the gradients of each task to prevent the occupancy task from being overwhelmed.

[0078] In one embodiment of this application, the process of annotating 3D fused data to obtain sample data with annotated information includes:

[0079] Motion compensation and data fusion are performed on multi-frame 3D fusion data to form global point cloud data. This global point cloud data is then mapped onto a preset 3D voxel grid to obtain 3D voxel data. In one embodiment of this application, the process of performing motion compensation on multi-frame 3D fusion data to obtain motion-compensated multi-frame 3D fusion data can be accomplished using a pose estimation model based on Kalman filtering and sensor fusion, or a geometric alignment algorithm based on point cloud registration. The data fusion process of the motion-compensated multi-frame 3D fusion data is implemented using a probabilistic statistical and dynamic scene fusion model, or a deep learning and feature-level fusion network model. The process of mapping the global point cloud data onto a preset 3D voxel grid to obtain 3D voxel data can be implemented using spatial hash grid mapping methods, regular voxelization algorithms, octree partitioning algorithms, etc.

[0080] In one embodiment of this application, the voxel grid containing point cloud data in a preset 3D voxel grid is marked as occupied. When a voxel grid is marked as "0", it indicates that the voxel grid is not occupied.

[0081] Semantic segmentation is performed on 3D voxel data to obtain the semantic type of the voxel data; the semantic type of the voxel data is then marked in the corresponding voxel mesh to obtain the semantic type of the preset 3D voxel mesh. In one embodiment of this application, the process of semantic segmentation of 3D voxel data to obtain the semantic type of voxel data is implemented using models such as Sparse UNet. Since there is a one-to-one correspondence between voxel data and voxel mesh, the semantic type of the corresponding voxel mesh can be obtained after obtaining the semantic type of the voxel data.

[0082] Cross-frame matching of 3D voxel data is performed to obtain the displacement difference of voxel data between adjacent frames; based on the displacement difference of voxel data between adjacent frames, the motion velocity of the voxel data is determined; the motion velocity is marked in the corresponding voxel grid to obtain the motion velocity of the preset 3D voxel grid. In one embodiment of this application, the process of performing cross-frame matching of 3D voxel data to obtain the displacement difference of voxel data between adjacent frames can be implemented by a feature alignment model based on a temporal attention mechanism or an end-to-end scene flow estimation network model; wherein, the temporal attention mechanism is used to implicitly capture the temporal motion offset of voxel data in the feature space, and the scene flow estimation network is used to directly regress the explicit displacement vector of voxel data in 3D space. The motion velocity of voxel data is the ratio of the displacement difference of voxel data between adjacent frames to the time interval between adjacent frames. Since there is a one-to-one correspondence between voxel data and voxel grids, after obtaining the motion velocity of voxel data, the motion velocity of the corresponding voxel grid can be obtained.

[0083] The Euclidean distance from the voxel center to the smooth mesh surface in a preset 3D voxel mesh is calculated, and the positional relationship and distance sign between the voxel center and the smooth mesh surface are determined based on the ray crossing method. The Euclidean distance is associated with the corresponding distance sign to obtain the signed distance field of the preset 3D voxel mesh. In one embodiment of this application, the positional relationship between the voxel center and the smooth mesh surface includes: the voxel center is outside the smooth mesh surface, the voxel center is inside the smooth mesh surface, and the voxel center is on the smooth mesh surface. When the voxel center is outside the smooth mesh surface, the distance sign is positive; when the voxel center is inside the smooth mesh surface, the distance sign is negative; and when the voxel center is on the smooth mesh surface, the Euclidean distance from the voxel center to the smooth mesh surface is 0. There is a one-to-one correspondence between the Euclidean distance and the distance sign. After associating the Euclidean distance with the corresponding distance sign, the signed distance field of the preset 3D voxel mesh is obtained. The smooth mesh surface is constructed based on global point cloud data. The process of constructing the smooth mesh surface based on global point cloud data is implemented by algorithms such as Poisson Surface Reconstruction or Ball-Pivoting Algorithm (BPA).

[0084] The formula for calculating Euclidean distance is as follows:

[0085] Equation (11)

[0086] in, This represents the Euclidean distance from the center of a voxel in a predefined 3D voxel mesh to the surface of the smooth mesh. Represents the coordinates of the voxel center. This represents the coordinates of the point on the smooth mesh surface that is closest to the center of the voxel.

[0087] In one embodiment of this application, the process of decoding temporal fusion features to obtain obstacle perception results includes:

[0088] Velocity estimation is performed on the temporal fusion features to obtain the obstacle velocity field. In one embodiment of this application, the process of estimating the velocity of the temporal fusion features is implemented by a motion head, which is a convolutional neural network (CNN) model or an MLP (multilayer perceptron) network model, etc.

[0089] Geometric prediction is performed on the temporal fusion features to obtain the signed distance field values ​​of obstacles. In one embodiment of this application, the process of geometric prediction of the temporal fusion features is implemented through a geometric head, which consists of several convolutional layers.

[0090] Semantic prediction is performed on temporal fusion features to obtain the semantic category of obstacles. In one embodiment of this application, the process of semantic prediction on temporal fusion features is implemented through a semantic head, which includes a convolutional layer and a softmax activation function, and obtains the semantic category of obstacles by classifying the temporal fusion features.

[0091] Occupancy prediction is performed on the temporal fusion features to obtain the obstacle occupancy probability. In one embodiment of this application, the occupancy prediction process is implemented through an occupancy header, which consists of a convolutional layer and a sigmoid activation function, outputting a probability value between 0 and 1.

[0092] The obstacle's signed distance field value, obstacle semantic category, obstacle occupancy probability, and obstacle velocity field are used as the obstacle perception results. In one embodiment of this application, based on the obstacle occupancy probability, obstacle semantic category, obstacle velocity field, and obstacle signed distance field value, the spatial position and three-dimensional contour of the obstacle can be reconstructed to obtain the obstacle's precise spatial occupancy information. This directly provides interpretable perception basis for the path planning and obstacle avoidance decisions of logistics vehicles, further improving the accuracy and robustness of obstacle perception, especially suitable for complex perception environments with multiple occlusions and dense distribution of small obstacles in logistics city scenarios.

[0093] In one embodiment of this application, the process of fusing historical bird's-eye view features, modified geometric features, and modified semantic features in a temporal sequence to obtain temporal fused features includes:

[0094] The historical bird's-eye view features are transformed and aligned to the current frame's bird's-eye view features. In one embodiment of this application, the process of transforming and aligning historical bird's-eye view features to the current frame's bird's-eye view features can be implemented through deformable convolution or flow-based transformation.

[0095] The historical bird's-eye view features, corrected geometric features, and corrected semantic features aligned to the current frame are fused temporally to obtain temporally fused features. In one embodiment of this application, the historical bird's-eye view features, corrected geometric features, and corrected semantic features aligned to the current frame can be fused by concatenating them along the channel dimension, or by weighted fusion based on temporal attention to achieve feature aggregation. The fused result is used as the temporally fused feature. This process can effectively utilize the complete geometric and semantic information of occluded objects in historical frames, make up for the lack of perception information in the current frame, and further improve the stability of the perception results.

[0096] In one embodiment of this application, the process of fusing current image information and current laser point cloud to obtain a bird's-eye view feature volume includes:

[0097] Multi-scale feature extraction is performed on the current image information to obtain two-dimensional multi-scale features. In one embodiment of this application, multi-scale feature extraction is performed on the current image information using a residual network (e.g., ResNet-50) model or a Swin Transformer Tiny model.

[0098] Point cloud features are extracted from the current laser point cloud. In one embodiment of this application, point cloud features are extracted from the current laser point cloud using voxelization or PointPillar methods.

[0099] Two-dimensional multi-scale features are mapped onto the bird's-eye view space to obtain mapped features; these mapped features are then fused with point cloud features to obtain the bird's-eye view feature volume. In one embodiment of this application, the process of mapping two-dimensional multi-scale features onto the bird's-eye view space is implemented through LSS (Lift-Splat-Shoot) or a Transformer-based mechanism (e.g., BEVFormer (LearningBird's-Eye-View Representation from Multi-Camera Images via SpatiotemporalTransformers)). The size of the bird's-eye view feature volume is X×Y×Z×C, where X×Y represents the horizontal resolution of the bird's-eye view feature volume, Z represents the number of voxel layers in the height direction (e.g., 8 layers) to enable the bird's-eye view feature volume to retain the vertical spatial structure information of the scene, thereby improving the perception ability of objects with complex height features, and C represents the number of feature channels of the bird's-eye view feature volume. The process of fusing mapping features and point cloud features to obtain a bird's-eye view feature volume includes: concatenating the mapping features and point cloud features along the channel dimension to obtain concatenated features along different channel dimensions; and inputting the concatenated features along different channel dimensions into a fully convolutional fusion module (including: 2D convolutional layers, batch normalization layers, and linear rectified units) to obtain the bird's-eye view feature volume.

[0100] Figure 3 This is a flowchart illustrating a logistics vehicle obstacle perception method based on a sparse semantic occupancy network, as shown in another exemplary embodiment of this application. Figure 3In this paper, the process of the obstacle perception method for logistics vehicles based on sparse semantic occupancy network includes: (1) Input data: input the current image information collected by the surround-view camera and the current laser point cloud collected by the low-beam lidar; (2) Feature extraction: through image backbone (ResNet-50 model or Swin Transformer) Tiny model) performs multi-scale feature extraction on the current image information to obtain two-dimensional multi-scale features; performs point cloud feature extraction on the current laser point cloud to obtain point cloud features; (3) Feature projection: projects the two-dimensional multi-scale features onto the bird's-eye view space to obtain the mapped features; (4) Feature fusion: fuses the mapped features with the point cloud features to obtain the bird's-eye view feature volume; (5) Coarse-grained occupancy prediction: downsamples the bird's-eye view feature volume and performs binarized occupancy prediction on the downsampled bird's-eye view feature volume to generate a binarized occupancy mask; (6) Determine sparse index: determine the occupied voxel grid (the occupied voxel grid index is the sparse index) based on the binarized occupancy mask; (7) Fine-grained feature extraction: maps the occupied voxel grid to the grid space determined by the bird's-eye view feature volume to obtain the mapped voxel grid in the grid space determined by the bird's-eye view feature volume; and extracts the features in the mapped voxel grid as sparse features. (i.e., sparse voxel features); (8) Fine-grained occupancy prediction: perform occupancy prediction on sparse features to obtain the occupancy probability, signed distance field value, and semantic category probability distribution of sparse features; (9) Feature correction: input the occupancy probability, signed distance field value, and semantic category probability distribution of sparse features into the dual-stream network, use the occupancy probability and signed distance field value as geometric flow features, correct the geometric flow features to obtain corrected geometric features; use the semantic category probability distribution as semantic flow features, correct the semantic flow features to obtain corrected semantic features; (10) Temporal fusion: perform temporal feature fusion on historical bird's-eye view features, corrected geometric features, and corrected semantic features to obtain temporal fusion features; (11) Feature decoding: decode the temporal fusion features to obtain obstacle perception results (including: obstacle signed distance field value, obstacle semantic category, obstacle occupancy probability, and obstacle velocity field, etc.).

[0101] Figure 4 This is a flowchart illustrating a sparsity process in an exemplary embodiment of this application. Figure 4 In the process of sparsification, the steps include: extracting the foreground region from the full set of voxels, and calculating the features of only the foreground region. The process of extracting the foreground region from the full set of voxels is achieved through... Figure 3 The process involves inputting data, feature extraction, feature projection, feature fusion, coarse-grained occupancy prediction, determining sparse indexes, and fine-grained feature extraction. The calculation of features for the foreground region is achieved through... Figure 3 The steps of fine-grained occupancy prediction, feature correction, temporal fusion, and feature decoding are implemented.

[0102] Figure 5 This is a flowchart illustrating an exemplary embodiment of the present application, showing the processing of a logistics scenario involving fallen objects on the ground. Figure 5 In the process of handling logistics scenarios involving fallen objects (e.g., cardboard boxes on the ground), the steps include: taking the current image information as the input source, extracting features from the current image information, mapping the extracted image features to the bird's-eye view space to obtain a voxel representation of the extracted image features, and after occupancy prediction through a sparse semantic occupancy network model and geometric and semantic correction through a two-stream network, the spatial height position (ground) and semantic type (unknown obstacle) in the current image information are obtained by temporal feature fusion and decoding of historical bird's-eye view features, corrected geometric features, and corrected semantic features.

[0103] Figure 6 This is a flowchart illustrating a logistics scenario involving suspended obstacles, as shown in an exemplary embodiment of this application. Figure 6 In the process of handling logistics scenarios with suspended obstacles (e.g., height restriction poles), the steps include: taking the current image information as the input source, extracting features from the current image information, mapping the extracted image features to the bird's-eye view space to obtain the voxel representation of the extracted image features, and after occupancy prediction through the sparse semantic occupancy network model and geometric and semantic correction through the dual-stream network, the spatial height position (suspended) and semantic type (unknown obstacle) in the current image information are obtained by temporal feature fusion and decoding of historical bird's-eye view features, corrected geometric features and corrected semantic features.

[0104] In one embodiment of this application, the method of this application was deployed in a real vehicle and tested in a logistics park and on open urban roads. The test results show that in long-tail obstacle scenarios (e.g., sudden appearance of cardboard boxes or fallen traffic cones), the method of this application can stably detect the occupied area and trigger safe avoidance to prevent collisions; in complex dynamic scenarios (e.g., pedestrian crossings), the temporal fusion module accurately predicts the speed of pedestrians, thereby enabling the pre-deployed planning module on the unmanned logistics vehicle to issue deceleration and yield control commands in advance; on the Jetson Orin platform, the inference time of the method of this application is approximately 65ms (including pre- and post-processing of perception inference), meeting the 15Hz operation requirement.

[0105] In one embodiment of this application, the method introduces a coarse-to-fine sparsity mechanism, performing high-resolution calculations only on the foreground region. Compared to a fully dense network, this reduces computational cost by more than 70%, significantly improving computational efficiency and adapting to the hardware cost constraints of logistics vehicles. Even with occupancy probability and signed distance field values, it generates physical obstruction constraints for unseen obstacles without relying on specific class samples, fundamentally avoiding collisions and improving robustness in long-tail scenarios. Furthermore, semantic information is added on top of the occupancy probability, enabling the planning module to distinguish the passability of different obstacles and optimize decision quality. By combining temporal feature fusion and velocity estimation of historical bird's-eye view features, corrected geometric features, and corrected semantic features, it can effectively memorize temporarily occluded obstacles and predict their movement trends, improving stability in congested road segment games. Finally, by fusing image data and point cloud data acquired by low-cost LiDAR, it combines rich visual semantics with the geometric accuracy of LiDAR, enhancing overall perception reliability.

[0106] The following describes an embodiment of the apparatus described in this application, which can be used to execute the obstacle perception system for logistics vehicles based on sparse semantic occupancy networks described in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the obstacle perception method for logistics vehicles based on sparse semantic occupancy networks described above in this application.

[0107] Figure 7 This is a block diagram illustrating an obstacle perception system for logistics vehicles based on a sparse semantic occupancy network, as shown in an exemplary embodiment of this application.

[0108] like Figure 7 As shown, the exemplary logistics vehicle obstacle perception system 700 based on a sparse semantic occupancy network includes:

[0109] The information acquisition module 701 is used to acquire perception information within the preset range of the unmanned logistics vehicle.

[0110] The information fusion module 702 is used to fuse the current image information and the current laser point cloud to obtain the bird's-eye view feature volume.

[0111] The occupancy prediction module 703 is used to downsample the bird's-eye view feature volume based on a pre-built sparse occupancy prediction model, and perform binarized occupancy prediction on the downsampled bird's-eye view feature volume to generate a binarized occupancy mask; based on the binarized occupancy mask, bird's-eye view features are extracted from the bird's-eye view feature volume to obtain sparse features; occupancy prediction is performed on the sparse features to obtain the occupancy probability, signed distance field value and semantic category probability distribution of the sparse features.

[0112] The feature correction module 704 is used to perform geometric correction on the occupancy probability and signed distance field value of sparse features based on a pre-built two-stream network to obtain corrected geometric features, and to perform semantic correction on the semantic category probability distribution to obtain corrected semantic features.

[0113] The perception result determination module 705 is used to perform temporal feature fusion on historical bird's-eye view features, corrected geometric features and corrected semantic features to obtain temporal fusion features; and to decode the temporal fusion features to obtain obstacle perception results.

[0114] In one embodiment of this application, the sensing information includes: current image information, current laser point cloud, historical image information, and historical laser point cloud. The preset range is set according to the actual situation. The current image information is acquired by a camera or surround-view camera, the current laser point cloud is acquired by a 16-line or 32-line LiDAR, and the historical image information and historical laser point cloud come from historical frame acquisition data stored in the unmanned logistics vehicle's sensing system.

[0115] In one embodiment of this application, by fusing the current image information and the current laser point cloud, the resulting bird's-eye view feature retains the complete spatial geometric and semantic information in the perceived information, providing a reliable foundation for subsequent sparse extraction and high-precision prediction.

[0116] In one embodiment of this application, the number of voxels in the bird's-eye view feature is 200×200×8, and the number of voxels in the downsampled bird's-eye view feature is 50×50×4. The downsampling rate is kept at one-quarter, which effectively compresses the overall computational load without excessively losing key information of the original features, adapting to the computing power limitations of the edge. By downsampling and binarizing the bird's-eye view feature for occupancy prediction, sparse regions that may contain obstacles are quickly screened out, and high-precision attribute prediction is performed only for sparse features. This significantly reduces the number of voxels that need to be calculated for high-resolution features (usually only accounting for 10%~20% of the total).

[0117] In one embodiment of this application, the pre-constructed dual-stream network includes a geometric correction branch and a semantic correction branch. The geometric correction branch refines the occupancy probability of sparse features and the signed distance field value through residual convolution, restoring the fine spatial information lost during downsampling and making the output obstacle positions and boundary information more consistent with the actual scene. The semantic correction branch addresses the semantic ambiguity caused by downsampling by recalibrating the semantic category probability distribution, improving the classification accuracy of small-volume obstacles and semantically similar obstacles. The geometric correction branch and the semantic correction branch share some basic feature extraction structures. This design effectively avoids redundant computation while ensuring the effect of multi-dimensional feature correction, keeping the additional computational overhead to a very low level, and is highly adaptable to vehicle-mounted operating platforms with limited computing power.

[0118] In one embodiment of this application, historical bird's-eye view features are obtained by fusing historical image information and historical laser point clouds. The process of obtaining historical bird's-eye view features by fusing historical image information and historical laser point clouds is the same as the process of fusing current image information and current laser point clouds to obtain bird's-eye view feature volumes.

[0119] In one embodiment of this application, by first downsampling and binarizing the bird's-eye view feature volume for occupancy prediction, sparse regions where obstacles may exist are quickly screened out. High-precision attribute prediction is then performed only for sparse features, which significantly reduces the number of voxels that need to be computed for high-resolution features (typically only accounting for 10% to 20% of the total). This allows the method to run in real time on unmanned logistics vehicles with low computing power platforms, effectively solving the technical challenge of deploying high-order perception algorithms at the edge in complex scenarios. Furthermore, by using a pre-constructed two-stream network to geometrically correct the occupancy probability and signed distance field value of sparse features, and semantically correcting the semantic category probability distribution, the restoration accuracy of obstacle edges and microstructures is further improved, enhancing the geometric perception of non-standard obstacles. The system's obstacle perception capabilities effectively prevent scratches or collisions caused by missed detection of irregularly shaped obstacles. By fusing temporal features of corrected geometric and semantic features, it provides a more interpretable environmental description for downstream planning and control modules, enabling logistics vehicles to make more human-like and reasonable driving decisions, significantly improving traffic efficiency and safety. The fusion of historical bird's-eye view features, corrected geometric features, and semantic features can effectively utilize historical information to supplement the observation gaps in the current frame (e.g., objects that are temporarily occluded), smooth out jitter in perception results, and accurately estimate the speed of obstacles. This is particularly important for handling dynamic targets that suddenly appear in mixed pedestrian and vehicle areas, further enhancing the robustness of the logistics vehicle obstacle perception system in dynamic and complex environments.

[0120] It should be noted that the obstacle perception system for logistics vehicles based on sparse semantic occupancy networks provided in the above embodiments and the obstacle perception method for logistics vehicles based on sparse semantic occupancy networks provided in the above embodiments belong to the same concept. The specific ways in which each module and unit performs its operations have been described in detail in the method embodiments, and will not be repeated here. In practical applications, the obstacle perception system for logistics vehicles based on sparse semantic occupancy networks provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above, and this is not a limitation here.

[0121] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A method for obstacle perception in logistics vehicles based on sparse semantic occupancy networks, characterized in that, The method includes: Acquire perception information within a preset range of the unmanned logistics vehicle; the perception information includes: current image information, current laser point cloud, historical image information, and historical laser point cloud; The current image information and the current laser point cloud are fused to obtain a bird's-eye view feature volume; Based on a pre-built sparse occupancy prediction model, the bird's-eye view feature volume is downsampled, and the downsampled bird's-eye view feature volume is subjected to binarized occupancy prediction to generate a binarized occupancy mask; based on the binarized occupancy mask, bird's-eye view features are extracted from the bird's-eye view feature volume to obtain sparse features; occupancy prediction is performed on the sparse features to obtain the occupancy probability, signed distance field value, and semantic category probability distribution of the sparse features; Based on a pre-constructed two-stream network, the occupancy probability of the sparse features and the signed distance field value are geometrically corrected to obtain corrected geometric features, and the semantic category probability distribution is semantically corrected to obtain corrected semantic features. The historical bird's-eye view features, the modified geometric features, and the modified semantic features are fused in a temporal sequence to obtain temporal fused features; the temporal fused features are decoded to obtain obstacle perception results; the historical bird's-eye view features are obtained by fusing the historical image information and the historical laser point cloud.

2. The obstacle perception method for logistics vehicles based on sparse semantic occupancy networks according to claim 1, characterized in that, Based on the binarized occupancy mask, the process of extracting bird's-eye view features from the bird's-eye view feature volume to obtain sparse features includes: The occupied voxel grid is determined based on the binarized occupancy mask; the occupied voxel grid is derived from the grid space determined by the downsampled bird's-eye view feature volume. The occupied voxel grid is mapped to the grid space defined by the bird's-eye view feature volume to obtain the mapped voxel grid in the grid space defined by the bird's-eye view feature volume; and the features in the mapped voxel grid are extracted as the sparse features.

3. The obstacle perception method for logistics vehicles based on sparse semantic occupancy networks according to claim 1 or 2, characterized in that, The process of pre-constructing a sparse occupancy prediction model includes: Acquire historical video data and historical point cloud data of the unmanned logistics vehicle in urban scenarios; The historical video data and the historical point cloud data are fused to obtain three-dimensional fused data; The three-dimensional fused data is used as unlabeled sample data to perform self-supervised training on the preset feature extractor, resulting in the trained feature extractor. The 3D fusion data is annotated to obtain sample data with annotation information; the annotation information includes: occupancy status, semantic category, motion speed, and signed distance field value; The preset sparse occupancy prediction model is trained using the labeled sample data to obtain the pre-constructed sparse occupancy prediction model. The preset sparse occupancy prediction model includes: the trained feature extractor, the preset view transformation and sparsification module, the preset sparse prediction head, and the preset task output branch.

4. The obstacle perception method for logistics vehicles based on sparse semantic occupancy networks according to claim 3, characterized in that, The process of using the 3D fused data as unlabeled sample data to perform self-supervised training on a preset feature extractor includes: According to the occlusion ratio, the unlabeled sample data is occluded to obtain occluded sample data; based on the preset feature extractor, the occluded sample data is reconstructed to obtain reconstructed data; With the goal of minimizing the difference between the reconstructed data and the unlabeled sample data, the parameters in the preset feature extractor are adjusted to obtain the trained feature extractor.

5. The obstacle perception method for logistics vehicles based on sparse semantic occupancy networks according to claim 3, characterized in that, The process of training the preset sparse occupancy prediction model using the labeled sample data includes: According to the sliding window mechanism, the sample data with labeled information is selected to obtain selected sample data; the selected sample data is input into the preset sparse occupancy prediction model to obtain predicted data; the occurrence time of the predicted data is after the occurrence time of the selected sample data; With the goal of minimizing the differences between the occupancy status of the predicted data and the occupancy status of the selected sample data, the differences between the semantic categories of the predicted data and the selected sample data, the differences between the movement speed of the predicted data and the movement speed of the selected sample data, and the differences between the signed distance field value of the predicted data and the signed distance field value of the selected sample data, the parameters in the preset sparse occupancy prediction model are adjusted to obtain the pre-constructed sparse occupancy prediction model.

6. The obstacle perception method for logistics vehicles based on sparse semantic occupancy networks according to claim 3, characterized in that, The process of annotating the 3D fused data to obtain sample data with annotated information includes: Motion compensation and data fusion are performed on multi-frame 3D fusion data to form global point cloud data; the global point cloud data is then mapped onto a preset 3D voxel grid to obtain 3D voxel data. Mark the voxel grid containing point cloud data in the preset three-dimensional voxel grid as occupied; The three-dimensional voxel data is semantically segmented to obtain the semantic type of the voxel data; the semantic type of the voxel data is marked in the corresponding voxel grid to obtain the semantic type of the preset three-dimensional voxel grid. Cross-frame matching is performed on the three-dimensional voxel data to obtain the displacement difference of the voxel data between adjacent frames; the motion speed of the voxel data is determined based on the displacement difference of the voxel data between adjacent frames; the motion speed is marked in the corresponding voxel grid to obtain the motion speed of the preset three-dimensional voxel grid. Calculate the Euclidean distance from the voxel center to the smooth mesh surface in the preset three-dimensional voxel mesh, and determine the positional relationship and distance sign between the voxel center and the smooth mesh surface based on the ray crossing method; associate the Euclidean distance with the corresponding distance sign to obtain the signed distance field of the preset three-dimensional voxel mesh; the smooth mesh surface is constructed based on the global point cloud data.

7. The obstacle perception method for logistics vehicles based on sparse semantic occupancy networks according to claim 1 or 2, characterized in that, The process of decoding the temporal fusion features to obtain obstacle perception results includes: Velocity estimation is performed on the temporal fusion features to obtain the obstacle velocity field; Geometric prediction is performed on the temporal fusion features to obtain the signed distance field value of the obstacle; Semantic prediction is performed on the temporal fusion features to obtain the semantic category of the obstacle; The occupancy probability of obstacles is obtained by performing occupancy prediction on the temporal fusion features. The signed distance field value of the obstacle, the semantic category of the obstacle, the occupancy probability of the obstacle, and the velocity field of the obstacle are used as the obstacle perception results.

8. The obstacle perception method for logistics vehicles based on sparse semantic occupancy networks according to claim 1 or 2, characterized in that, The process of fusing historical bird's-eye view features, the modified geometric features, and the modified semantic features in a temporal sequence to obtain temporal fused features includes: The historical bird's-eye view features are transformed and aligned to the current frame bird's-eye view features; The historical bird's-eye view features aligned to the current frame, the corrected geometric features, and the corrected semantic features are fused temporally to obtain the temporally fused features.

9. The obstacle perception method for logistics vehicles based on sparse semantic occupancy networks according to claim 1 or 2, characterized in that, The process of fusing the current image information and the current laser point cloud to obtain the bird's-eye view feature volume includes: Multi-scale feature extraction is performed on the current image information to obtain two-dimensional multi-scale features; Point cloud features are extracted from the current laser point cloud to obtain point cloud features; The two-dimensional multi-scale features are mapped onto the bird's-eye view space to obtain the mapped features; the mapped features are fused with the point cloud features to obtain the bird's-eye view feature volume.

10. A logistics vehicle obstacle perception system based on a sparse semantic occupancy network, characterized in that, include: The information acquisition module is used to acquire perception information within the preset range of the unmanned logistics vehicle; The perceived information includes: current image information, current laser point cloud, historical image information, and historical laser point cloud; The information fusion module is used to fuse the current image information and the current laser point cloud to obtain a bird's-eye view feature volume; The occupancy prediction module is used to downsample the bird's-eye view feature volume based on a pre-built sparse occupancy prediction model, and perform binarized occupancy prediction on the downsampled bird's-eye view feature volume to generate a binarized occupancy mask; based on the binarized occupancy mask, extract bird's-eye view features from the bird's-eye view feature volume to obtain sparse features; and perform occupancy prediction on the sparse features to obtain the occupancy probability, signed distance field value, and semantic category probability distribution of the sparse features. The feature correction module is used to perform geometric correction on the occupancy probability of the sparse features and the signed distance field value based on a pre-built two-stream network to obtain corrected geometric features, and to perform semantic correction on the semantic category probability distribution to obtain corrected semantic features. The perception result determination module is used to perform temporal feature fusion on historical bird's-eye view features, the corrected geometric features, and the corrected semantic features to obtain temporal fusion features; decode the temporal fusion features to obtain obstacle perception results; the historical bird's-eye view features are obtained by fusing the historical image information and the historical laser point cloud.

Images