Space occupation prediction method and device, electronic equipment and vehicle

Abstract

The invention provides a space occupation prediction method and device, electronic equipment and a vehicle, and relates to the technical field of automatic driving. The method comprises the following steps: acquiring multi-view image data around a vehicle and extracting multi-scale image features; constructing a plurality of mutually orthogonal feature planes; constructing a three-dimensional feature body based on the feature plane; determining a first feature part corresponding to a space within a preset range of the center of the vehicle and a second feature part corresponding to a space outside the preset range from the three-dimensional feature body; performing occupancy prediction on the first feature part by using a first spatial resolution, and performing occupancy prediction on the second feature part by using a second spatial resolution, the first spatial resolution being higher than the second spatial resolution; and finally, outputting the space occupation distribution of the surrounding environment of the vehicle. According to the method, the orthogonal feature plane is constructed to replace complex three-dimensional convolution, and a double-resolution strategy is adopted, so that the calculation overhead of long-distance sensing is remarkably reduced.

Description

Technical Field

[0001] This application relates to the field of autonomous driving perception technology, and in particular to a space occupancy prediction method, device, electronic device, and vehicle. Background Technology

[0002] In the environmental perception task of autonomous driving systems, the visual spatial occupancy algorithm provides key three-dimensional environmental information for downstream decision-making and control by estimating the occupancy status and semantic labels of each voxel in the scene from multi-view image sequences.

[0003] Existing technologies, such as the VoxFormer model, typically employ 3D initialization feature queries or dense 3D convolutional networks to construct voxel features. However, this approach consumes significant GPU computing and memory resources. Especially during vehicle operation, the use of uniform high-resolution processing for both near and far environmental perception results in a substantial waste of computing resources on distant background areas with minimal impact on safety, failing to meet the stringent requirements of automotive-grade chips for real-time performance and efficient resource utilization. Summary of the Invention

[0004] This application provides a space occupancy prediction method, device, electronic device, and vehicle, aiming to improve the technical problems of existing space occupancy prediction algorithms having high computational resource consumption and being difficult to deploy in real time on the vehicle.

[0005] In a first aspect, embodiments of this application provide a space occupancy prediction method, including: Acquire multi-view image data of the area surrounding the vehicle; extract features from the multi-view image data to obtain multi-scale image features; Based on the multi-scale image features, multiple mutually orthogonal feature planes are constructed; based on the multiple mutually orthogonal feature planes, a three-dimensional feature volume is constructed. From the three-dimensional feature body, determine a first feature portion corresponding to the space within a preset range of the vehicle center, and a second feature portion corresponding to the space outside the preset range; Occupancy prediction is performed on the first feature portion using a first spatial resolution, and occupancy prediction is performed on the second feature portion using a second spatial resolution, wherein the first spatial resolution is higher than the second spatial resolution; Based on the prediction results at different resolutions, the spatial occupancy distribution of the vehicle's surrounding environment is output.

[0006] The aforementioned technical solution avoids the massive computational burden of full-space 3D convolution by decoupling 3D spatial features into multiple mutually orthogonal feature planes (e.g., three-view TPV representation). Furthermore, by distinguishing between the first feature portion near the vehicle's center and the second feature portion at a distance, and employing differentiated spatial resolutions for prediction, the computational load on edge regions is significantly reduced while ensuring perception accuracy in core areas (such as nearby obstacles). This strategy significantly improves the model's inference speed and resource utilization, making it more suitable for deployment on resource-constrained automotive embedded platforms.

[0007] In one embodiment, constructing multiple mutually orthogonal feature planes based on the multi-scale image features includes: Initialize reference points on the plurality of mutually orthogonal feature planes respectively; Using the intrinsic and extrinsic parameters of the vehicle camera, a coordinate mapping relationship between the reference point and the multi-scale image features is established; Based on the coordinate mapping relationship, the multi-scale image features are sampled and fused to generate the multiple mutually orthogonal feature planes.

[0008] In this embodiment, by using this reference point initialization + geometric projection + fusion method, there is no need to blindly search for relevant features across the entire image. Instead, features are directly extracted from specific locations in the image based on geometric priors. This not only significantly reduces the computational load but also ensures that the generated feature planes have clear physical geometric meaning.

[0009] In one embodiment, the plurality of mutually orthogonal feature planes include a bird's-eye view plane corresponding to a bird's-eye view, a front view plane corresponding to a frontal view, and a side view plane corresponding to a side view; the construction of a three-dimensional feature volume based on the plurality of mutually orthogonal feature planes includes: The features of the bird's-eye view plane, the front view plane, and the side view plane are extended to the three-dimensional space dimension along their respective normal directions; The features of the three expanded planes are aggregated to generate the three-dimensional feature volume.

[0010] In this embodiment, the application only needs to maintain three planar features, which reduces the memory usage complexity from cubic to quadratic, thereby allowing for the processing of larger-scale or higher-resolution scenes. Furthermore, by aggregating features from three orthogonal viewpoints, the lack of height information in a single viewpoint is compensated for, enabling a more complete recovery of the object's three-dimensional geometry, thus ensuring the accuracy and precision of subsequent space occupancy prediction processing.

[0011] In one embodiment, after constructing multiple mutually orthogonal feature planes, the method further includes: obtaining the historical feature plane generated in the previous frame, and using a temporal attention mechanism to interact the feature plane of the current frame with the historical feature plane to obtain the temporally updated feature plane; The step of constructing a three-dimensional feature volume based on the plurality of mutually orthogonal feature planes includes: constructing the three-dimensional feature volume based on the time-updated feature planes.

[0012] In this embodiment, a temporal self-attention module is introduced. It should be understood that in autonomous driving scenarios, the vehicle is in motion. Therefore, before fusing features from historical frames (time t-1), it is necessary to first align the feature planes from historical moments to the coordinate system of the current moment (t$) based on the vehicle's motion information (such as speed and angular velocity). After alignment, an attention mechanism is used to interact with historical features as keys and values, and current features as queries, to obtain a temporally updated feature plane. Then, the 3D feature volume is constructed based on the temporally updated feature plane. In this way, if a region is occluded at the current moment (e.g., by a vehicle in front), the model can obtain information from the features of historical moments (when the region was not occluded) to complete the model.

[0013] In one embodiment, after generating the plurality of mutually orthogonal feature planes, the method further includes: using a self-attention mechanism to perform inter-plane feature interaction on the plurality of mutually orthogonal feature planes to obtain spatially interactive feature planes; The construction of a three-dimensional feature volume based on the plurality of mutually orthogonal feature planes includes: constructing the three-dimensional feature volume based on the feature planes after spatial interaction.

[0014] In this embodiment, through feature interactions between feature planes, the model can automatically correct feature conflicts that may arise from projections from different viewpoints. For example, it confirms whether the position of an object on the bird's-eye view matches its height on the forward-looking plane, thereby generating more geometrically reasonable and rigorous 3D features. Furthermore, this embodiment fuses information from the three planes into each plane, making each plane implicitly contain global spatial context, further improving the semantic classification accuracy of subsequent occupancy prediction.

[0015] In one embodiment, the step of predicting the occupancy of the first feature portion using a first spatial resolution includes: Perform an upsampling operation on the first feature portion; The upsampled features are input into the decoder for prediction to obtain the occupancy prediction result for the first spatial resolution.

[0016] In this embodiment, by physically upsampling and finely decoding the core area, the vehicle's ability to perceive the environment nearby can be significantly improved, effectively solving the problem of missed or blurred detection of small objects at close range, and providing high-precision input for the refined obstacle avoidance control of autonomous driving.

[0017] In one embodiment, determining the second feature portion corresponding to the space outside the preset range from the three-dimensional feature body includes: determining the features in the three-dimensional feature body other than the first feature portion as the second feature portion, or determining the untrimmed three-dimensional feature body as the second feature portion; The step of predicting occupancy of the second feature portion using the second spatial resolution includes: Maintain the original resolution of the second feature portion, or downsample the second feature portion; input the feature after maintaining or downsampling into the decoder for prediction to obtain the occupancy prediction result of the second spatial resolution.

[0018] In this embodiment, in conjunction with the aforementioned embodiments, a complete dual-resolution processing strategy is formed. It is evident that for distant areas with minimal impact on security, maintaining low resolution avoids significant unnecessary computational overhead and memory usage. This allows the model to cover a greater sensing distance (although the distant area is coarser) under the same computing power conditions, ensuring both near-field security accuracy and providing information about distant areas.

[0019] Secondly, embodiments of this application provide a space occupancy prediction device, comprising: The data acquisition module is used to acquire multi-view image data of the vehicle's surroundings; The feature construction module is used to extract multi-scale image features and construct multiple mutually orthogonal feature planes based on these features, thereby constructing a three-dimensional feature volume; A dual-resolution prediction module is used to determine a first feature portion corresponding to a space within a preset range of the vehicle center and a second feature portion corresponding to a space outside the preset range from the three-dimensional feature body; to predict the occupancy of the first feature portion using a first spatial resolution and to predict the occupancy of the second feature portion using a second spatial resolution, wherein the first spatial resolution is higher than the second spatial resolution; The results output module is used to output the spatial occupancy distribution of prediction results based on different resolutions. Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any of the above-mentioned embodiments.

[0020] Fourthly, the storage medium described in the embodiments of this application stores a computer program, which, when executed by a processor, implements the method described in any of the above-mentioned embodiments.

[0021] Fifthly, embodiments of this application provide a vehicle that includes the electronic equipment described above. Attached Figure Description

[0022] Figure 1 This is a schematic flowchart of a space occupancy prediction method provided in an embodiment of this application; Figure 2 This is a schematic diagram of a processing architecture for a space occupancy prediction model provided in one embodiment of this application; Figure 3 This is a schematic diagram of a process for generating supervisory data for a space occupancy prediction model, provided in an embodiment of this application. Figure 4 This is a schematic diagram of a space occupancy prediction device provided in one embodiment of this application; Figure 5 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0023] To make the technical problems, technical solutions, and beneficial effects solved by this application clearer, the following detailed description is provided in conjunction with embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0024] In the field of autonomous driving, accurately perceiving the 3D environment surrounding the vehicle is fundamental to safe driving. Traditional monocular or binocular depth estimation methods often struggle to provide dense geometric information in complex scenes. While emerging visual occupancy networks can predict voxel-level occupancy states, existing methods typically rely on global 3D queries or dense 3D convolutions, leading to an explosive increase in computational cost. Especially in real-world driving scenarios, vehicles require extremely high resolution for nearby obstacles (such as curbs, pedestrians, and nearby vehicles) to ensure obstacle avoidance safety, while only a rough estimate of the distant background (such as distant buildings and trees) is needed. Existing uniform resolution processing methods clearly ignore this difference, resulting in wasted computational resources. Therefore, this application proposes a novel spatial occupancy prediction scheme to address the aforementioned technical problems.

[0025] Terminology Explanation Occupancy refers to dividing a three-dimensional space into regular voxels and predicting the probability that each voxel is occupied by an object and the semantic category of that object (such as vehicle, pedestrian, road, etc.).

[0026] TPV (Tri-Perspective View): A sparse representation method for three-dimensional space, which approximates three-dimensional features by using three orthogonal planes (usually XY, XZ, and YZ planes). Compared with full 3D voxel representation, it can significantly reduce video memory usage.

[0027] Knowledge distillation: A model compression technique that reduces the number of parameters while maintaining high accuracy by having a lightweight model (student network) mimic the output of a complex model (teacher network).

[0028] This application primarily provides a space occupancy prediction method. The method involves acquiring multi-view image data of the area surrounding a vehicle; extracting features from the multi-view image data to obtain multi-scale image features; constructing multiple mutually orthogonal feature planes based on the multi-scale image features; constructing a three-dimensional feature volume based on the multiple mutually orthogonal feature planes; determining a first feature portion corresponding to a space within a preset range around the vehicle's center and a second feature portion corresponding to a space outside the preset range from the three-dimensional feature volume; predicting the occupancy of the first feature portion using a first spatial resolution and predicting the occupancy of the second feature portion using a second spatial resolution, where the first spatial resolution is higher than the second spatial resolution; and finally, outputting the space occupancy distribution of the environment surrounding the vehicle based on the prediction results at different resolutions.

[0029] This scheme reduces the dimensionality of 3D convolution operations to 2D planar operations by constructing orthogonal feature planes, fundamentally reducing the computational complexity of feature extraction. At the same time, by combining a dual-resolution prediction strategy, it utilizes the spatial characteristics of autonomous driving scenarios where near objects require fine detail while distant objects can be coarsely approximated. This concentrates valuable computing resources on the first feature part (near objects), which is most critical to safety. While ensuring the perception accuracy of the core area, it significantly reduces redundant calculations for distant background areas, achieving the best balance between model performance and resource consumption. This effectively solves the technical challenge of deploying high-precision space-occupying algorithms in real time on vehicle-grade embedded devices.

[0030] The present application will now be described in detail through various embodiments.

[0031] Example 1 This application provides a space occupancy prediction method, please refer to... Figure 1 and combined Figure 2 Understandably, this method includes the following steps: S10: Acquire multi-view image data around the vehicle; extract features from the multi-view image data to obtain multi-scale image features.

[0032] In this step, multi-view image data is typically acquired by surround-view cameras (e.g., six or more cameras for front, rear, left, and right views) mounted around the vehicle. For example, to fully understand the image content, the image is first fed into a backbone network (e.g., ResNet-50) for feature extraction. The backbone is responsible for capturing texture details and shallow semantics in the image. Further, to handle both small objects (such as distant cones) and large objects (such as nearby buses), a neck network (e.g., FPN) is usually connected. The neck is responsible for generating multi-scale image features. This multi-scale feature integration of information from different resolutions allows the model to more comprehensively understand complex scenes.

[0033] S20: Based on the multi-scale image features, construct multiple mutually orthogonal feature planes.

[0034] In this step, to avoid the enormous computational overhead of directly constructing high-dimensional 3D features, this application adopts a novel processing strategy: constructing Tri-Perspective View (TPV) features. Specifically, three mutually orthogonal feature planes are constructed, i.e., bird's-eye view features. For example, the mutually orthogonal feature planes correspond to the bird's-eye view plane (Topview, XY plane), the front view plane (Frontview, XZ plane), and the side view plane (Sideview, YZ plane). These three feature planes capture the features of the scene in different projection directions. Through this orthogonal decomposition, the originally complex 3D spatial computation is reduced, greatly improving efficiency.

[0035] S30: Construct a three-dimensional feature volume based on the plurality of mutually orthogonal feature planes.

[0036] In this step, although the features are compressed onto three planes, the final prediction task is still performed in three-dimensional space. Therefore, it is necessary to restore these three planes back to three-dimensional space. For example, this is usually achieved through feature aggregation, which involves backprojecting the planar features back into three-dimensional space and fusing them to generate a full-space three-dimensional feature volume containing rich semantic and geometric information.

[0037] S40: Determine a first feature portion corresponding to the space within a preset range of the vehicle center from the three-dimensional feature body, and a second feature portion corresponding to the space outside the preset range.

[0038] In this step, considering that the safety requirements of autonomous driving decrease with distance, this application divides the three-dimensional feature volume into a core area of ​​interest and a peripheral area of ​​interest. Specifically, a preset range is set with the center of the vehicle as the origin (for example, a rectangular area with length and width of [-20m, 20m]). The three-dimensional features located within this preset range are marked as the first feature part, which corresponds to the environment near the vehicle and generally includes key obstacle avoidance objects such as roadside details and nearby pedestrians. Features located outside the preset range are marked as the second feature part, which corresponds to the road topology and large obstacles in the distance.

[0039] S50: Use a first spatial resolution to predict the occupancy of the first feature portion and use a second spatial resolution to predict the occupancy of the second feature portion, wherein the first spatial resolution is higher than the second spatial resolution.

[0040] This step employs a dual-head decoder mechanism, including a high-resolution branch and a low-resolution branch. The high-resolution branch processes the first feature portion (nearby), performing fine-grained processing and predicting occupancy at a first spatial resolution (e.g., 0.2m × 0.2m × 0.2m). The low-resolution branch processes the second feature portion (far away), predicting occupancy at a second spatial resolution (e.g., 0.4m × 0.4m × 0.4m or a coarser size). This approach significantly reduces the computational load of far-field perception while maintaining near-field perception accuracy.

[0041] S60: Based on the prediction results at different resolutions, output the spatial occupancy distribution of the environment around the vehicle.

[0042] In this step, the prediction results from the high-resolution and low-resolution branches are integrated to output the final spatial occupancy distribution. For example, this spatial occupancy distribution is typically presented as a semantic occupancy grid, where each grid contains not only geometric information indicating whether it is occupied, but also semantic label information indicating what kind of object occupies it (e.g., vehicle, pedestrian, road, vegetation, etc.). In one application scenario, for the downstream planning and control module of the vehicle, the nearby high-precision grid can be directly used for fine-grained obstacle avoidance path planning, while the distant large-scale grid can be used for global path navigation and situational awareness.

[0043] As can be seen, existing technologies typically rely on the construction of 3D voxel features with a globally uniform resolution, which leads to huge memory consumption and computational latency, making it difficult to run in real time on automotive chips with limited computing power. This application's embodiment reduces the dimensionality of 3D convolution operations to 2D planar operations by constructing orthogonal feature planes, fundamentally reducing the computational complexity of feature extraction. Simultaneously, by combining a dual-resolution prediction strategy, valuable computing resources are concentrated on the first feature part (nearby), which is most critical to safety. This ensures the perception accuracy of core areas (such as accurate identification of road edges and nearby obstacles) while significantly reducing redundant calculations for distant background areas. In summary, this solution achieves the optimal balance between model performance and resource consumption, effectively solving the technical challenge of deploying high-precision, space-consuming algorithms in real time on automotive-grade embedded devices.

[0044] In one embodiment, step S20, namely, constructing multiple mutually orthogonal feature planes based on the multi-scale image features, specifically includes the following process: S21: Initialize reference points on the plurality of mutually orthogonal feature planes respectively.

[0045] This step corresponds to the initialization phase of the TPV encoder. Unlike traditional convolutional networks that process the entire image directly, this application employs a query-based mechanism. Specifically, for each feature plane (e.g., an H×W bird's-eye view plane), a set of fixed 3D reference points is predefined at each grid position (h,w). For example, for a feature point on the bird's-eye view plane (XY plane), its corresponding reference points are not merely two-dimensional coordinates, but a set of three-dimensional coordinate points (x,y,z_1), (x,y,z_2),...,(x,y,z_k) distributed along the Z-axis. These reference points act as probes, used to subsequently retrieve corresponding semantic information in the image space.

[0046] S22: Using the intrinsic and extrinsic parameters of the vehicle camera, establish the coordinate mapping relationship between the reference point and the multi-scale image features.

[0047] This step corresponds to the geometric projection process. To extract features belonging to a specific location in 3D space from a 2D image, the imaging geometry of the camera must be utilized. Specifically: for each 3D reference point generated in step S21, it is first transformed to the camera coordinate system using an extrinsic matrix, and then projected onto the 2D image plane using an intrinsic matrix to obtain projected coordinates. This process establishes an index relationship between the 3D space (feature plane) and the 2D observation data (image features), that is, it establishes a coordinate mapping relationship between the reference point and the multi-scale image features.

[0048] S23: Based on the coordinate mapping relationship, the multi-scale image features are sampled and fused to generate the multiple mutually orthogonal feature planes.

[0049] This step is similar to spatial cross attention or deformable attention mechanisms. Since the projection coordinates (i.e., the projection point) typically fall at non-integer sub-pixel locations, and a single pixel may not contain a sufficient receptive field, the pixel value at that point cannot be simply taken. In this embodiment, bilinear interpolation or multi-point sampling is performed on the multi-scale image feature map centered on the projection point to extract the image feature vector near that location. During this process, a set of attention weights is learned through a fully connected linear layer. The sampled image features from multiple perspectives are weighted and summed with these weights, and the fused features are finally filled into the grid positions (h, w) corresponding to the feature plane. Through this process, the three originally empty orthogonal planes (Query) successfully extract texture and semantic information from the image, forming a feature plane with rich environmental representation capabilities.

[0050] In this embodiment, by using this reference point initialization + geometric projection + fusion method, there is no need to blindly search for relevant features across the entire image. Instead, features are directly extracted from specific locations in the image based on geometric priors. This not only significantly reduces the computational load but also ensures that the generated feature planes have clear physical geometric meaning.

[0051] In one embodiment, the plurality of mutually orthogonal feature planes include a bird's-eye view plane corresponding to a bird's-eye view, a front view plane corresponding to a frontal view, and a side view plane corresponding to a side view; in step S30, the construction of a three-dimensional feature volume based on the plurality of mutually orthogonal feature planes specifically includes the following steps: S31: Extend the features of the bird's-eye view plane, the front view plane, and the side view plane to the three-dimensional space dimension along their respective normal directions.

[0052] This step corresponds to feature broadcasting or dimensionality upscaling. Since TPV features are essentially three 2D feature maps (e.g., each with dimensions H×W×C), and the final occupancy prediction needs to be performed in 3D space, the 2D information needs to be stretched along its missing dimensions.

[0053] Specifically, for the bird's-eye view plane (XY plane): its normal direction is the Z-axis. The features of this plane are copied along the Z-axis to generate a feature volume with dimensions H×W×Z×C; for the front view plane (XZ plane): its normal direction is the Y-axis. The features of this plane are expanded along the Y-axis to generate a feature volume of the same size; for the side view plane (YZ plane): its normal direction is the X-axis. The system expands the features of this plane along the X-axis to generate a feature volume of the same size. In other words, the 2D planar features are projected back into the view frustum in 3D space.

[0054] S32: Aggregate the features of the three expanded planes to generate the three-dimensional feature body.

[0055] This step is equivalent to feature fusion. At this point, the three expanded feature vectors are aligned one-to-one in spatial coordinates and need to be merged into a complete representation. For example, aggregation can be performed by element-wise addition or concatenation followed by convolution. For instance, for any coordinate point (x, y, z) in 3D space, its final feature vector V(x, y, z) is obtained by adding the features of the three planes at their corresponding projection points, as shown below: V(x,y,z)=F xy (x,y)+F xz (x,z)+F yz (y,z) The resulting V is a three-dimensional feature volume containing information from all omnidirectional perspectives.

[0056] In this embodiment, the application only needs to maintain three planar features, which reduces the memory usage complexity from cubic to quadratic, thus allowing for the processing of larger-scale or higher-resolution scenes. Furthermore, by fusing features from three orthogonal viewpoints, the lack of height information in a single viewpoint (such as only the BEV viewpoint) is compensated for, enabling a more complete recovery of the object's three-dimensional geometry (such as suspended objects or structures with complex height variations), thereby ensuring the accuracy and precision of subsequent space occupancy prediction processing.

[0057] In one embodiment, after constructing multiple mutually orthogonal feature planes, this application embodiment further includes a temporal fusion step: obtaining the historical feature plane generated at the previous frame time, and using a temporal attention mechanism to interact the feature plane of the current frame with the historical feature plane to obtain the temporally updated feature plane; The step of constructing a three-dimensional feature volume based on the plurality of mutually orthogonal feature planes includes: constructing the three-dimensional feature volume based on the time-updated feature planes.

[0058] In this embodiment, a temporal self-attention module is introduced. It should be understood that in autonomous driving scenarios, the vehicle is in motion. Therefore, before fusing features from historical frames (time t-1), it is necessary to first align the feature planes from historical moments to the coordinate system of the current moment (t) based on the vehicle's motion information (such as speed and angular velocity). After alignment, an attention mechanism is used to interact with historical features as keys and values, and current features as queries, to obtain a temporally updated feature plane. The 3D feature volume is then constructed based on this temporally updated feature plane.

[0059] The process of constructing the three-dimensional feature volume can be seen in the description of the above embodiments, and will not be repeated here. The main difference is that this embodiment uses a temporally updated feature plane. In this way, if a certain area is occluded at the current time (e.g., by a vehicle in front), the model can obtain information from the features of historical times (when the area was not occluded) to complete the model.

[0060] As can be seen, in this embodiment, by fusing temporal information, dynamic changes in the scene can be captured, solving the problems of occlusion and blind spots in single-frame images. Simultaneously, temporal fusion can maintain the continuity of perception, reduce flickering in prediction results, and improve the stability of dynamic obstacle detection.

[0061] In one embodiment, after generating the plurality of mutually orthogonal feature planes, the method further includes: using a self-attention mechanism to perform inter-plane feature interaction on the plurality of mutually orthogonal feature planes to obtain spatially interactive feature planes; the method of constructing a three-dimensional feature volume based on the plurality of mutually orthogonal feature planes includes: constructing the three-dimensional feature volume based on the spatially interactive feature planes.

[0062] In this embodiment, this step corresponds to Cross-Plane Attention HybridAttention. It's important to understand that although the three feature planes are orthogonally decoupled, they describe the same physical environment and therefore should be logically consistent. Therefore, in this embodiment, an interaction layer is introduced between the three initial feature planes (bird's-eye view, front view, and side view). For example, the features of the bird's-eye view plane are cross-attentioned with the features of the other two planes, or the features of the three planes are concatenated and then self-attention is performed. This means that a point on the bird's-eye view plane not only aggregates image information but also reflects information from corresponding positions on the front and side view planes, thus enabling the flow of contextual information between different views.

[0063] In this embodiment, through feature interactions between feature planes, the model can automatically correct feature conflicts that may arise from projections from different viewpoints. For example, it confirms whether the position of an object on the bird's-eye view matches its height on the forward-looking plane, thereby generating more geometrically reasonable and rigorous 3D features. Furthermore, this embodiment fuses information from the three planes into each plane, making each plane implicitly contain global spatial context, further improving the semantic classification accuracy of subsequent occupancy prediction.

[0064] In one embodiment, the occupancy prediction of the first feature portion using the first spatial resolution specifically includes the following steps: S51: Perform an upsampling operation on the first feature portion.

[0065] S52: Input the upsampled features into the decoder for prediction to obtain the occupancy prediction result of the first spatial resolution.

[0066] Step S51 aims to improve the spatial granularity of the vehicle's near-field region. The first feature portion corresponds to the core safety area around the vehicle (e.g., within 20 meters), requiring extremely high resolution to identify curbs and small obstacles. Therefore, in this embodiment, the central region feature V corresponding to the first feature portion is cropped from the aggregated three-dimensional feature volume. center Then, trilinear interpolation or transposed convolution can be used to enlarge the spatial size of the feature volume, thereby increasing the feature dimension and obtaining a high-dimensional feature.

[0067] Step S52 is high-resolution decoding. In this step, the upsampled high-dimensional features are input into a lightweight decoder (exemplary, such as a multilayer perceptron MLP). The decoder maps the feature vectors to an N+1-dimensional probability distribution (N semantic categories + 1 idle category), thereby outputting a refined voxel-level occupancy status.

[0068] As can be seen, in this embodiment, by physically upsampling and finely decoding the core area, the vehicle's ability to perceive its surroundings can be significantly improved. Specifically, it can accurately delineate the edge shape of the roadside, clearly distinguish adjacent pedestrians and vehicles, and effectively solve the problem of missed or blurred detection of small objects at close range, providing high-precision input for refined obstacle avoidance control in autonomous driving.

[0069] In one embodiment, step S50, which involves predicting the occupancy of the second feature portion using a second spatial resolution, specifically includes the following steps: S53: Determine the features in the three-dimensional feature body other than the first feature part as the second feature part, or determine the untrimmed three-dimensional feature body as the second feature part; In this embodiment, step S53 defines the feature range of the far field or background region, and determines the features in the three-dimensional feature volume other than the first feature portion as the second feature portion, or determines the untrimmed three-dimensional feature volume as the second feature portion. That is, it is possible to choose to retain the entire large-scale coarse feature volume as the second feature portion, or only truncate the outer area.

[0070] S54: Maintain the original resolution of the second feature portion, or downsample the second feature portion; input the feature after maintaining or downsampling into the decoder for prediction to obtain the occupancy prediction result of the second spatial resolution.

[0071] This step S54 is low-resolution decoding. For distant environments, high geometric precision is not required. Therefore, the unsampled raw feature volume is directly used (or, to further save bandwidth, pooling is used for downsampling) and input into another decoder, which also includes a multilayer perceptron (MLP). The spatial resolution of the decoder output is lower than that of the decoder output in steps S51-S52, and it is mainly responsible for predicting the direction of distant roads, the outlines of large buildings, etc.

[0072] In this embodiment, in conjunction with the aforementioned embodiments, a complete dual-resolution processing strategy is formed. It is evident that for distant areas with minimal impact on security, maintaining low resolution avoids significant unnecessary computational overhead and memory usage. This allows the model to cover a greater sensing distance (although the distant area is coarser) under the same computing power conditions, ensuring both near-field security accuracy and providing global navigation information for distant areas, making it extremely suitable for the actual deployment needs of automotive embedded platforms.

[0073] Example 2 In one embodiment, such as Figure 4 As shown, a space occupancy prediction method is provided, including the following steps: S101: Acquire multi-view image data around the vehicle; extract features from the multi-view image data to obtain multi-scale image features.

[0074] S102: Initialize reference points on the multiple mutually orthogonal feature planes respectively; establish a coordinate mapping relationship between the reference points and the multi-scale image features using the intrinsic and extrinsic parameters of the vehicle camera; sample and fuse the multi-scale image features based on the coordinate mapping relationship, corresponding to the bird's-eye view plane, the front view plane, and the side view plane. S103: Extend the features of the bird's-eye view plane, the front view plane, and the side view plane to the three-dimensional space dimension along their respective normal directions; aggregate the features of the three extended planes to generate the three-dimensional feature volume. S104: Determine a first feature portion corresponding to the space within a preset range of the vehicle center from the three-dimensional feature body, and a second feature portion corresponding to the space outside the preset range; S105: Perform an upsampling operation on the first feature portion; input the upsampled feature into the decoder for prediction to obtain the occupancy prediction result of the first spatial resolution; S106: Maintain the original resolution of the second feature portion, or downsample the second feature portion; input the feature after maintaining or downsampling into the decoder for prediction to obtain the occupancy prediction result of the second spatial resolution; S107: Based on the prediction results at different resolutions, output the spatial occupancy distribution of the environment around the vehicle.

[0075] For steps S101-S107 of this embodiment, please refer to the relevant description in the aforementioned Embodiment 1, which will not be repeated here.

[0076] It should be noted that, in order to implement the above-mentioned space occupancy prediction method, a corresponding space occupancy prediction model can be pre-trained. After inputting multi-view image data around the vehicle into the space occupancy prediction model, the space occupancy prediction model can implement the above-mentioned space occupancy prediction method and output the space occupancy distribution of the environment around the vehicle. The space occupancy prediction model can be directly deployed in vehicle equipment.

[0077] like Figure 2 As shown, the Figure 2 The architecture diagram of the space occupancy prediction model is shown, and the... Figure 2 This paper demonstrates the complete data flow process of the space occupancy prediction model provided in the embodiments of this application, mainly including four modules: image feature extraction, orthogonal plane construction, 3D feature reconstruction, and dual-resolution prediction. These modules correspond to the relevant processing in the above method embodiments. The following describes the working process of the space occupancy prediction model provided in the embodiments of this application from an architectural perspective, wherein: The image feature extraction module takes multi-view image data (e.g., surround view images) of the vehicle's surroundings as input. The data is sequentially processed by the backbone network to extract basic features, and then by the neck network for multi-scale fusion, ultimately outputting 2D multi-scale image features containing rich texture and semantics.

[0078] The Orthogonal Feature Plane Construction Module (View Transformer) transforms 2D multi-scale image features into a sparse 3D representation using a spatial cross attention mechanism. This is achieved through three mutually orthogonal feature planes: the Bird's Eye View Feature (BEV Feature), which includes the bird's eye view plane, the front view plane, and the side view plane. Specifically, 3D reference points are initialized on these three planes and projected onto the multi-scale image features using camera intrinsic and extrinsic parameters for sampling and weighted fusion. Furthermore, this module includes a temporal self-attention unit to incorporate historical feature planes from the previous frame to address dynamic occlusion issues.

[0079] Spatial CrossAttention: The first module in the View Transformer is Spatial CrossAttention. It initializes 3D reference points, projects intrinsic and extrinsic parameters onto the image to find features, predicts a set of attention weights, weights these weights with the image features, and finally outputs a bird's-eye view feature that integrates image information. This process effectively combines 2D image features with 3D spatial information, providing a foundation for subsequent 3D scene understanding.

[0080] Temporal Self Attention: The second module in the View Transformer is Temporal Self Attention. This module is responsible for fusing temporal information, interacting with the features of the bird's-eye view generated in the previous frame and the features of the current frame. In this way, the model can capture dynamic changes in the scene, further improving its ability to perceive 3D space.

[0081] The 3D feature volume reconstruction module: Broadcasts or expands the three orthogonal feature planes along their normal directions (i.e., the missing dimension) to restore them to a 3D tensor form. Through aggregation operations (such as addition or splicing), it generates a 3D feature volume capable of representing the geometric information of the entire scene. For example, Figure 2 The 200×200×16 three-dimensional feature volume.

[0082] The dual-head decoder module employs a dual-head decoding mechanism, comprising a main task head and an auxiliary task head. The main task head corresponds to the high-resolution branch, while the auxiliary task head corresponds to the low-resolution branch. It is used to divide the 3D feature volume into a first feature portion corresponding to the near area and a second feature portion corresponding to the distant area. Specifically: High-resolution branch: Upsamples and finely decodes the first feature part to output high-precision near-field occupancy prediction, so as to meet the vehicle's need for accurate perception of nearby curbs and obstacles.

[0083] Low-resolution branch: Decode the second feature part at the original resolution or after downsampling, and output a coarse far-field occupancy prediction to reduce computational resource consumption.

[0084] After constructing the complete 3D feature volume, the model enters a dual-resolution processing flow. To save computational resources, the model does not perform high-resolution reconstruction of the entire massive 3D space. Instead, it uses a cutout operation to cut out the core feature region corresponding to the vicinity of the vehicle's center (i.e., the first feature part mentioned earlier, such as a 20-meter radius around the vehicle) from the 3D feature volume of the entire scene. This cutout feature is then fed into the high-resolution branch for upsampling, enlarging the spatial size of the feature volume and correspondingly increasing the feature dimension to obtain high-dimensional features, such as a feature dimension of 200×200×32, to achieve fine perception of nearby obstacles. The remaining part (or the whole) that was not cut out, corresponding to the second feature part, remains processed at low resolution. In the high-resolution and low-resolution branches, prediction processing is performed by the decoder to obtain prediction results corresponding to the two resolutions. The decoder includes a feature decoder module and a multilayer perceptron (MLP).

[0085] In the high-resolution branch, the Feature Decoder module, located between the feature input and the MLP (Classification Head), plays a crucial role, primarily responsible for the computational unit that adjusts the feature resolution and feature space mapping. Essentially, the Feature Decoder is an upsampling module or feature reconstruction module. It performs upsampling—using trilinear interpolation or deconvolution operations—to enlarge the spatial size of the feature volume.

[0086] The feature decoder module in the low-resolution branch does not upsample the input features; it may downsample or perform identity mapping. In some implementations, to further save computational power, it may even include pooling layers or strided convolutions to actively reduce the spatial resolution of the features.

[0087] It should be noted that the decoder in this application embodiment can have multiple implementation forms, and this application embodiment does not limit the specific implementation. For example, a feature decoder + multilayer perceptron (MLP) is one example network architecture, and it is not limited to any particular implementation. It should also be noted that... Figure 2 For details on the processing of each module, please refer to the relevant descriptions in the foregoing method embodiments. To avoid redundancy, they will not be repeated here.

[0088] Example 3 Next, from the perspective of the space occupancy prediction model, we will explain the training process of the space occupancy prediction model.

[0089] In one embodiment, to construct the aforementioned high-precision space occupancy prediction model, this application also employs a complete space occupancy prediction model training and optimization process. This process covers the entire lifecycle from ground truth construction and iterative training to final deployment and optimization, specifically including: a supervised data construction phase, a model training iteration phase, a model lightweighting phase, and a verification mechanism, which are described below, including: Supervising the data construction phase To enable the space occupancy prediction model to predict accurate space occupancy distribution based solely on image input, one embodiment of this application constructs high-quality 3D semantic occupancy ground truth as the standard answer for training. To address the problem of sparse point clouds lacking semantic information in single frames, this process mainly includes the following three core processing stages: multi-frame point cloud temporal fusion stage, road surface ground truth enhancement stage, and semantic category assignment stage. This process rigorously handles static backgrounds, dynamic objects, and road surface details, including: Multi-frame point cloud temporal fusion stage Multi-frame point cloud fusion: solving sparsity and occlusion; Static scene fusion: Using the global coordinate system provided by the vehicle positioning system as a medium, the point clouds of static objects (such as buildings, bridges, and traffic facilities) scanned at different times are uniformly aligned to the same coordinate system to construct a high-density static environment model.

[0090] Dynamic target fusion: For dynamic objects such as vehicles, pedestrians, and bicycles, 3D bounding boxes labeled manually or automatically are used for capture and tracking. During fusion, point cloud compensation can be performed based on the motion trajectory of the bounding boxes, thereby improving point cloud density while effectively eliminating the ghosting phenomenon caused by object movement and reducing the ghosting problem of static objects caused by positioning jitter.

[0091] Road surface true value enhancement stage Road surface ground truth enhancement (using flooding algorithm): To address the issue of missing scanning data from lidar on nearby road surfaces (especially in the blind spot), this application also designs a road surface completion mechanism based on the flooding algorithm, including: Search logic: Starting from the center point of the vehicle, search for road points radially in all directions.

[0092] Stop conditions: Set a height difference threshold (such as stopping the search when encountering obstacles with obvious height changes, such as curbs or walls) and a specific perimeter limit (similar to the working boundary logic of a robot vacuum cleaner) to prevent the algorithm from incorrectly including non-road areas.

[0093] The above logic ensures that the generated true road surface values ​​cover the nearby blind spots without causing multiple marking errors.

[0094] Semantic category assignment phase Semantic category assignment (2D to 3D): A high-performance 2D semantic segmentation model is used to infer meaning from the panoramic image. Then, strictly utilizing the camera's intrinsic and extrinsic parameters, the fused 3D point cloud is projected back onto the 2D segmentation image. Based on the pixel category where the projected points fall, specific semantic labels are assigned to the 3D point cloud (e.g., vegetation, vehicles), completing the upgrade from geometric truth to semantic truth. These labels are then used to calculate the cross-entropy loss function to guide model training.

[0095] Ground value quality verification mechanism: After ground values ​​are generated, manual or automatic spot checks are performed using point cloud visualization tools. The focus is on checking for motion blur or ghosting caused by multi-frame fusion, missing or extra road surface ground values, and accurate semantic category projection. Only high-quality ground values ​​that pass verification are included in the training set.

[0096] Example lines, such as Figure 3 As shown, Figure 3 This is a process for generating ground truth values. Figure 3 This application illustrates the entire process of constructing supervised data (ground values) for model training in its embodiments, including the following input stage, preprocessing stage, road surface generation stage, and output stage, wherein: Input stage. The input data is point cloud data containing road edges. That is, point cloud data after multi-frame temporal fusion. Although the data at this stage is denser than that of a single frame, there may still be scanning holes in the road surface area, and clear semantic labels are lacking.

[0097] Preprocessing stage. To enable the computer to identify the road surface area, the 3D data first needs to be reduced in dimensionality: Original road surface flattening for masking: The 3D point cloud is projected along the Z-axis onto a 2D plane (bird's-eye view features) to generate an initial binary mask. In this initial mask, areas hit by the LiDAR are white, and areas not hit (holes) are black. Next, dilation is used to fill the holes: Utilizing the dilation operation in image morphology, sparse LiDAR points are connected to fill the small scanning holes inside the road surface, making the road surface area more connected. Curb flattening for masking: Similarly, the detected curb point cloud is projected into a 2D mask.

[0098] Road surface generation stage. This is the core step of the process. Figure 3 The algorithm for the robot vacuum cleaner is the same as the flooding algorithm mentioned earlier.

[0099] The robot vacuum cleaner's algorithm generates the road surface and curbs: Starting from the center of the vehicle (seed point), the algorithm expands outwards from the original road surface mask (similar to the coverage path of a robot vacuum cleaner). The expansion process is limited by the curb mask. The algorithm stops expanding when it reaches the curb mask. This ensures that the generated road surface values ​​do not overflow onto sidewalks or flower beds. In other words, it stops searching when encountering obstacles with varying elevations.

[0100] Mapping back to point cloud space to generate road surface: The complete, hole-free road surface area generated on the 2D plane is mapped back to the 3D point cloud space using extrinsic parameters from the camera or radar. All 3D points falling within this area are labeled as road surface categories.

[0101] Output stage. The output data is point cloud data containing ground truth road surface data. Finally, a dense, well-defined 3D ground truth road surface data with missing blind spots (holes) is obtained, which is used to supervise the model's learning of the first feature part (nearby road surface).

[0102] Model training iteration phase This application employs an iterative process of baseline model - main line improvement - subdivision tuning to ensure the optimal performance of the space occupancy prediction model.

[0103] Initially, a BEVFormer + single decoder architecture was used as a benchmark to quickly verify the data path and basic effects. This was subsequently evolved into the TPVFormer architecture described in this application, which introduces three orthogonal planes (bird's-eye view, front view, and side view). In this process, not only are reference points used to interact with image features, but self-attention interaction between features of the three planes is also introduced, enabling the information from the three planes to be fused together for a more comprehensive learning of 3D spatial features.

[0104] Training strategy and hyperparameter tuning: The main task is to improve the architecture (such as introducing a dual-head decoder to achieve multi-resolution recognition). Under the main task, several smaller tasks are subdivided for fine-tuning. A fine search is conducted to find the optimal solution for optimizer hyperparameters (such as learning rate and weight decay); simultaneously, data augmentation parameters are optimized, such as adjusting the random rotation angle and cropping ratio of training images, to improve the model's robustness and adaptability to different viewpoints.

[0105] During training, the IoU (Intersection over Union) metric is monitored in real time to compare the overlap between the model's predictions and the ground truth. Simultaneously, point cloud visualization tools are used to visually examine the model output, analyze for specific issues such as missed detection of nearby large vehicles or object fragmentation, and incorporate these issues into the next iteration for targeted optimization.

[0106] Model lightweighting stage To address the challenge of running complex models in real-time on automotive-grade embedded devices, embodiments of this application employ deep lightweighting techniques, including: The high-performance, convergent heavyweight model is used as the teacher network, while the streamlined, lightweight model is used as the student network. A distillation technique is employed to transfer the feature extraction capabilities of the complex model to the student network, making its accuracy as close as possible to that of the teacher model, thereby reducing computational cost while maintaining performance.

[0107] The trained model is then quantized using Int8, converting the original Float32 format weights and activation values ​​into 8-bit integers. This step aims to reduce the model's storage footprint and memory access bandwidth consumption.

[0108] After quantization, rigorous consistency verification is performed. Performance metrics (such as mIoU) and visualization tools are used to compare the outputs of the original floating-point model and the quantized model. The verification goal is to ensure that the decrease in key metrics after quantization is within an acceptable range (e.g., accuracy loss less than 1%) and there are no obvious visual artifacts, thereby confirming that the model is ready for deployment.

[0109] This application also provides a space occupancy prediction device 40, please refer to... Figure 4 ,include: Data acquisition module 410 is used to acquire multi-view image data around the vehicle; The feature construction module 420 is used to extract multi-scale image features and construct multiple mutually orthogonal feature planes based on these features, thereby constructing a three-dimensional feature volume; The dual-resolution prediction module 430 is used to determine a first feature portion corresponding to a space within a preset range of the vehicle center and a second feature portion corresponding to a space outside the preset range from the three-dimensional feature body; to predict the occupancy of the first feature portion using a first spatial resolution and to predict the occupancy of the second feature portion using a second spatial resolution, wherein the first spatial resolution is higher than the second spatial resolution. The result output module 450 is used to output the spatial occupancy distribution of prediction results based on different resolutions.

[0110] For details regarding the space occupancy prediction device 40, please refer to the description of the aforementioned method embodiments. This application does not limit the scope of the embodiments and will not describe them one by one.

[0111] This application also provides an electronic device 50, please refer to... Figure 5It includes a memory 510 and a processor 520, wherein the memory 510 is used to store computer programs; and the processor 520 is used to execute the programs stored in the memory 510 to implement the space occupancy prediction method described in any embodiment of this application.

[0112] This application also provides a vehicle including the electronic device 50 described above.

[0113] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the space occupancy prediction method described in any embodiment of this application.

[0114] In this application, "multiple" refers to two or more.

[0115] In this application, unless otherwise expressly defined, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0116] The terms “first,” “second,” “third,” “fourth,” etc., in this application (if any) are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0117] In this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, in this application, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0118] Unless otherwise specified, all steps in this application may be performed sequentially or randomly. For example, if the method includes steps A and B, it means that the method may include steps A and B performed sequentially, or it may include steps B and A performed sequentially. For example, if the method may also include step C, it means that step C may be added to the method in any order. For example, the method may include steps A, B, and C, or it may include steps A, C, and B, or it may include steps C, A, and B, etc.

[0119] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for predicting space occupancy, characterized in that, include: Acquire multi-view image data of the area surrounding the vehicle; Feature extraction is performed on the multi-view image data to obtain multi-scale image features; Based on the multi-scale image features, multiple mutually orthogonal feature planes are constructed; A three-dimensional feature volume is constructed based on the multiple mutually orthogonal feature planes. From the three-dimensional feature body, determine a first feature portion corresponding to the space within a preset range of the vehicle center, and a second feature portion corresponding to the space outside the preset range; Occupancy prediction is performed on the first feature portion using a first spatial resolution, and occupancy prediction is performed on the second feature portion using a second spatial resolution, wherein the first spatial resolution is higher than the second spatial resolution; Based on the prediction results at different resolutions, the spatial occupancy distribution of the vehicle's surrounding environment is output.

2. The method according to claim 1, characterized in that, The construction of multiple mutually orthogonal feature planes based on the multi-scale image features includes: Initialize reference points on the plurality of mutually orthogonal feature planes respectively; Using the intrinsic and extrinsic parameters of the vehicle camera, a coordinate mapping relationship between the reference point and the multi-scale image features is established; Based on the coordinate mapping relationship, the multi-scale image features are sampled and fused to generate the multiple mutually orthogonal feature planes.

3. The method according to claim 1, characterized in that, The plurality of mutually orthogonal feature planes include a bird's-eye view plane corresponding to a bird's-eye view, a front view plane corresponding to a frontal view, and a side view plane corresponding to a side view; the construction of a three-dimensional feature volume based on the plurality of mutually orthogonal feature planes includes: The features of the bird's-eye view plane, the front view plane, and the side view plane are extended to the three-dimensional space dimension along their respective normal directions; The features of the three expanded planes are aggregated to generate the three-dimensional feature volume.

4. The method according to claim 1, characterized in that, After constructing multiple mutually orthogonal feature planes, the method further includes: obtaining the historical feature plane generated in the previous frame, and using a temporal attention mechanism to interact the feature plane of the current frame with the historical feature plane to obtain the temporally updated feature plane. The step of constructing a three-dimensional feature volume based on the plurality of mutually orthogonal feature planes includes: constructing the three-dimensional feature volume based on the time-updated feature planes.

5. The method according to claim 2, characterized in that, After generating the plurality of mutually orthogonal feature planes, the method further includes: using a self-attention mechanism to perform inter-plane feature interaction on the plurality of mutually orthogonal feature planes to obtain spatially interactive feature planes. The construction of a three-dimensional feature volume based on the plurality of mutually orthogonal feature planes includes: constructing the three-dimensional feature volume based on the feature planes after spatial interaction.

6. The method according to claim 1, characterized in that, The step of predicting occupancy of the first feature portion using the first spatial resolution includes: Perform an upsampling operation on the first feature portion; The upsampled features are input into the decoder for prediction to obtain the occupancy prediction result for the first spatial resolution.

7. The method according to any one of claims 1-6, characterized in that, Determining the second feature portion corresponding to the space outside the preset range from the three-dimensional feature body includes: determining the features in the three-dimensional feature body other than the first feature portion as the second feature portion, or determining the untrimmed three-dimensional feature body as the second feature portion; The step of predicting occupancy of the second feature portion using the second spatial resolution includes: Maintain the original resolution of the second feature portion, or downsample the second feature portion; input the feature after maintaining or downsampling into the decoder for prediction to obtain the occupancy prediction result of the second spatial resolution.

8. A space occupancy prediction device, characterized in that, include: The data acquisition module is used to acquire multi-view image data of the vehicle's surroundings; The feature construction module is used to extract multi-scale image features and construct multiple mutually orthogonal feature planes based on these features, thereby constructing a three-dimensional feature volume; A dual-resolution prediction module is used to determine, from the three-dimensional feature body, a first feature portion corresponding to a space within a preset range of the vehicle center and a second feature portion corresponding to a space outside the preset range; Occupancy prediction is performed on the first feature portion using a first spatial resolution, and occupancy prediction is performed on the second feature portion using a second spatial resolution, wherein the first spatial resolution is higher than the second spatial resolution; The results output module is used to output the spatial occupancy distribution of prediction results based on different resolutions.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 7.

10. A vehicle, characterized in that, The vehicle includes the electronic equipment as described in claim 9.

Images