A BEV Viewpoint Transformation Method and System Based on 3D Polar Coordinates and Perception Range Priors
By introducing three-dimensional polar coordinates and prior perception range into the BEV perspective transformation, the problem of ignoring height information in existing technologies is solved, achieving higher accuracy and efficiency in BEV feature generation, which is applicable to autonomous driving environment perception and occupancy detection.
Patent Information
- Application Number
- CN202610929064.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-25
- Publication Date
- 2026-11-17
AI Technical Summary
Existing BEV perspective conversion methods ignore height information, resulting in insufficient accuracy for 3D perception tasks. Furthermore, they include invalid spatial locations outside the perception range in the calculation, leading to low efficiency.
We employ a method that combines 3D polar coordinates with prior perception range. By introducing 3D polar coordinate position encoding with height information and combining it with an effective perception range mask, we select effective coordinate points for encoding and generate BEV feature maps using a cross-attention mechanism.
It improves the accuracy and robustness of BEV features, reduces noise interference and redundant calculations, and enhances the accuracy and efficiency of ocupancy detection and 3D scene understanding.
Smart Images

Figure CN123118754A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and deep learning technology, and in particular to a BEV viewpoint conversion method and system based on three-dimensional polar coordinates and prior perception range. Background Technology
[0002] In vehicle autonomous driving perception systems, converting 2D image features from a camera perspective into bird's-eye view (BEV) features is a crucial technology for achieving accurate perception of the surrounding environment. Existing BEV view conversion methods rely on deep neural networks to extract image features, using position encoding and attention mechanisms to convert image features into BEV features. Currently, 2D polar coordinates are commonly used... Position encoding of image features, radial coordinates and angular coordinates It can depict the relative distance and direction between objects and vehicles, and is more consistent with the geometric characteristics of the driving environment than Cartesian coordinates, but two-dimensional polar coordinates ignore height. z Information, compressed 3D spatial details, makes it difficult to distinguish targets at different heights, resulting in insufficient accuracy for 3D perception tasks. Furthermore, it encodes the entire BEV space uniformly before subsequent calculations, and the full-range spatial position is prone to noise interference and redundant calculations, reducing the efficiency and accuracy of feature conversion. Summary of the Invention
[0003] To address the shortcomings of the existing technologies, this invention provides a BEV perspective conversion method and system based on three-dimensional polar coordinates and prior perception range, solving the problems of insufficient accuracy and low efficiency caused by the existing BEV perspective conversion methods ignoring height information and including invalid spatial positions outside the perception range in the calculation.
[0004] In a first aspect, the present invention provides a BEV perspective conversion method based on three-dimensional polar coordinates and prior perception range.
[0005] A BEV viewpoint transformation method based on three-dimensional polar coordinates and prior perception range includes: Real-time acquisition of images of the vehicle's surrounding environment, and extraction of image features from the environmental images; Based on the camera imaging model and predicted depth information, obtain the three-dimensional spatial coordinates corresponding to each pixel of the image features; Convert three-dimensional spatial coordinates into three-dimensional polar coordinates that include radial coordinates, angular coordinates, and height coordinates; Based on the task requirements, an effective perception range mask is pre-constructed, effective coordinate points are filtered using the mask, and then the three-dimensional polar coordinates of the effective coordinate points are position-encoded. The three-dimensional polar coordinate position encoding is fused with image features to obtain a fused feature vector with spatial position information. The fused feature vector is then mapped to the BEV plane through a BEV transformation framework based on cross attention to obtain a BEV feature map.
[0006] A further technical solution involves converting the three-dimensional spatial coordinates of each pixel in the image features into three-dimensional spatial coordinates in the Cartesian coordinate system. Convert to include radial coordinates r、 Angular coordinates and height coordinates z The three-dimensional polar coordinates, and the radial coordinates r Angular coordinates and height coordinates z Normalization is performed; the formula for coordinate transformation is as follows: ; ; ; ; In the above formula, the height coordinates z It remains unchanged.
[0007] Further technical solutions include pre-constructing an effective perception range mask based on task requirements, including: Based on preset parameters for the actual task of the driving vehicle, a three-dimensional effective perception range is generated; the preset parameters include the maximum perception radius. and height range The maximum sensing radius is based on the forward sensing distance. Backward sensing distance 1. Perceive distance from left to right Determine the maximum value; Based on the effective 3D sensing range, the 3D polar coordinates corresponding to each image feature pixel are judged point by point. If the radial coordinate does not exceed the preset maximum sensing radius... And the height coordinates are within the preset height range If the value of a point is within the specified range, the mask value of that point is recorded as 1 to indicate validity; otherwise, it is recorded as 0 to indicate invalidity, thus generating a point-level mask vector.
[0008] A further technical solution, combining an effective perception range mask, encodes the position of the three-dimensional polar coordinates of image feature pixels, including: Before encoding, the generated point-level mask vector is used to index and filter the three-dimensional polar coordinate points, and the valid coordinate points with mask=1 are selected for encoding. For each valid coordinate point, the normalized radial coordinates are... r Angular coordinates and height coordinates zSine and cosine functions are used for encoding separately, and the encoded vectors of multiple components are concatenated to form a three-dimensional polar coordinate position encoded vector.
[0009] Further technical solutions involve using any one of the following methods: vector concatenation, vector addition, or attention fusion, to fuse the three-dimensional polar coordinate position encoding with image features. The vector concatenation method is as follows: the image feature vector is directly concatenated with the three-dimensional polar coordinate position encoding vector to form an expanded feature vector. The vector addition method is as follows: the three-dimensional polar coordinate position encoding vector is projected to the same dimension as the image features through a trainable mapping matrix, and the projected position encoding vector is added to the image feature vector element by element to obtain the fused feature vector with spatial position information. The attention fusion method is as follows: the three-dimensional polar coordinate position encoding vector is used as the input for attention generation. A lightweight attention convolutional network is used to generate attention weight vectors in the channel dimension. The attention weight vector is multiplied with the image feature vector channel by channel to obtain the position-aware weighted feature vector. The weighted feature vector is added to the original image feature vector to obtain the final fused feature vector.
[0010] A further technical solution employs a masked cross-attention mechanism to map the fused feature vectors onto the BEV plane, resulting in a grid-arranged BEV feature map, including: Initialize BEV query features; Using the BEV query feature as the Q value and the fused feature vector as the K and V values, a two-dimensional attention mask matrix is introduced for cross-attention calculation to obtain a masked attention score matrix; wherein, valid point positions in the point-level mask vector are mapped to 0 and invalid point positions are mapped to -. This is extended to obtain a two-dimensional attention mask matrix, the dimension of which is the number of BEV query features × the number of image feature points; Perform Softmax normalization on the masked attention score matrix to obtain the attention weight matrix; The fused feature vectors are weighted and summed using the attention weight matrix to obtain the final BEV feature map arranged in a grid.
[0011] Further technical solutions include initializing BEV query features, including: The two-dimensional Cartesian coordinates of each grid in the BEV plane are respectively sine and cosine encoded, and the two sets of encoded vectors are concatenated to obtain the initial position encoded vector; The initial position encoding vector is mapped to the query feature space through a learnable linear layer, and then added to the learnable embedding bias to obtain the initialized BEV query features.
[0012] Secondly, the present invention provides a BEV perspective conversion system based on three-dimensional polar coordinates and prior perception range.
[0013] A BEV perspective transformation system based on three-dimensional polar coordinates and prior perception range includes: The image acquisition and feature extraction module is used to acquire images of the environment surrounding the vehicle in real time and extract image features from the environmental images. The pixel coordinate extraction module is used to obtain the three-dimensional spatial coordinates of each pixel in the image feature based on the camera imaging model and predicted depth information. The coordinate transformation module is used to convert three-dimensional spatial coordinates into three-dimensional polar coordinates that include radial coordinates, angular coordinates, and height coordinates; The coordinate position encoding module is used to pre-build an effective perception range mask according to task requirements, use the mask to filter effective coordinate points, and then encode the position of the three-dimensional polar coordinates of the effective coordinate points. The BEV feature generation module is used to fuse three-dimensional polar coordinate position encoding with image features to obtain a fused feature vector with spatial position information. Through a BEV transformation framework based on cross attention, the fused feature representation is mapped to the BEV plane to obtain a BEV feature map.
[0014] Thirdly, the present invention also provides a vehicle, including the above-described BEV perspective conversion system based on three-dimensional polar coordinates and prior perception range, or performing the above-described BEV perspective conversion method based on three-dimensional polar coordinates and prior perception range.
[0015] Fourthly, the present invention also provides an electronic device, comprising: a memory for storing executable instructions; and a processor for implementing the above-described BEV perspective conversion method based on three-dimensional polar coordinates and prior perception range when executing the executable instructions stored in the memory.
[0016] Fifthly, the present invention also provides a computer-readable storage medium storing executable instructions for causing a processor to execute the executable instructions to implement the above-described BEV perspective conversion method based on three-dimensional polar coordinates and prior perception range.
[0017] The above one or more technical solutions have the following beneficial effects: This invention provides a BEV (Browser-Electric Vehicle) viewpoint conversion method and system based on three-dimensional polar coordinates and perception range prior. By introducing z-direction information on top of polar coordinate position encoding, the two-dimensional polar coordinates are extended into three-dimensional polar coordinate position encoding. This allows BEV features to not only include planar positional relationships but also accurately express spatial height distribution, improving the accuracy of ocupancy detection and three-dimensional scene understanding. Simultaneously, by utilizing effective perception range prior, a mask matrix is injected before the Softmax calculation of cross-attention through a masking mechanism. This prevents invalid spatial positions from participating in the generation of BEV features, effectively reducing noise interference and redundant computation. This further improves inference speed and overall robustness, solving the problems of insufficient accuracy and low efficiency caused by existing BEV viewpoint conversion methods ignoring height information and including invalid spatial positions outside the perception range in the calculation.
[0018] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description
[0019] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.
[0020] Figure 1 This is an overall flowchart of the BEV perspective conversion method based on three-dimensional polar coordinates and prior perception range proposed in this embodiment of the invention. Detailed Implementation
[0021] It should be noted that the following detailed descriptions are exemplary and are intended only to describe specific embodiments and to provide further explanation of the invention, and are not intended to limit the scope of exemplary embodiments of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.
[0022] Example 1 As noted in the background section, camera-based environmental perception solutions are widely used in autonomous driving perception systems due to their low cost and ease of deployment. To achieve accurate perception of roads, traffic participants, and the surrounding environment, it is typically necessary to convert 2D image features from the camera's perspective into Bird's-Eye (BEV) view features. BEV representations can integrate multi-view image information in a unified spatial coordinate system, facilitating downstream tasks such as object detection, semantic segmentation, and path prediction. Existing BEV feature acquisition methods usually rely on deep neural networks to extract features from camera images and provide spatial location information for the features through positional encoding. Subsequently, a cross-attention mechanism is used to establish a correlation between camera view features and BEV query features, thereby realizing the conversion of camera image features into BEV view features. Specifically: first, a 3D view frustum is constructed based on the image plane coordinates combined with depth sampling; then, the view frustum coordinates are projected onto the vehicle coordinate system using the camera's intrinsic and extrinsic parameters to establish a geometric mapping relationship between image features and BEV space; subsequently, the 3D coordinate information in the view frustum is superimposed onto the corresponding pixel features through positional encoding, enabling the neural network to use this spatial location information to complete feature alignment and fusion in the subsequent cross-attention process.
[0023] Existing camera-image-based BEV viewpoint transformation methods often use polar coordinates to encode spatial positions, that is, using radial and angular coordinates to provide spatial position information for image features. However, such methods usually ignore height information, i.e., z-direction information, resulting in insufficient expression of the three-dimensional spatial structure in BEV features, making it difficult to meet the needs of tasks such as occupancy detection for three-dimensional occupancy state modeling. In addition, when performing position encoding, existing methods often uniformly encode the entire BEV space range, failing to combine the effective perception range in actual intelligent driving scenarios. As a result, redundant calculations and invalid feature introductions are caused, reducing the efficiency and accuracy of feature transformation.
[0024] To address the aforementioned issues, this embodiment proposes a BEV perspective transformation method based on three-dimensional polar coordinates and perception range priors. By constructing a three-dimensional polar coordinate position code that incorporates height information during the BEV perspective transformation process, the three-dimensional spatial representation capability is enhanced. Furthermore, the position code is optimized by combining an effective perception range prior, thereby improving the accuracy and robustness of BEV features while ensuring computational efficiency.
[0025] This embodiment constructs a three-dimensional polar coordinate position encoding, namely: in the process of generating BEV features from camera image features, firstly, based on the camera imaging model and depth information, the three-dimensional spatial coordinates (x, y, z) corresponding to the image pixels are obtained. Then, these coordinates are converted into a system containing radial coordinates r and angular coordinates. It also provides a three-dimensional polar coordinate representation of the height coordinate z, and fuses this three-dimensional polar coordinate encoding with the corresponding image features, thereby preserving complete spatial geometric information during BEV perspective transformation.
[0026] Specifically, BEV (Battery Electric Vehicle) is essentially a two-dimensional plane. The z-coordinate is flattened when projected onto the BEV. Existing technologies generally consider introducing z-information during the position encoding stage redundant since the final output is a two-dimensional BEV feature. However, the innovation of this solution lies in retaining z-information during the position encoding stage before BEV transformation. This allows the cross-attention mechanism to distinguish targets at different height levels (such as ground vehicles and aerial obstacles, vehicles under bridges and vehicles on bridges) by utilizing height information when establishing the association between image features and BEV query features. Although the final BEV feature map is two-dimensional, the feature formation process utilizes three-dimensional spatial information, enabling objects at different heights at the same horizontal position to be correctly associated with their corresponding BEV grids, thus supporting ocupancy-based three-dimensional stereo perception tasks. This strategy of three-dimensionalizing the intermediate process and two-dimensionalizing the final output is the key difference between this solution and existing technologies.
[0027] Meanwhile, this embodiment also introduces an effective perception range prior, that is: when constructing the location code, based on the actual needs of the autonomous driving scenario, the effective perception range of the BEV space (e.g., a rectangular or circular area centered on the vehicle) is pre-set. In the process of generating the location code, for spatial locations outside the effective range, their location codes are masked by setting a mask to avoid invalid features from participating in the generation and fusion of BEV features.
[0028] The introduction of the aforementioned effective sensing range prior is coordinated and related to the three-dimensional polar coordinate position encoding. Because of the introduction of three-dimensional polar coordinates (including z), the mask can accurately define the effective range in three-dimensional space (limiting not only the horizontal distance r but also the height z), thereby filtering out noise points that are within the horizontal distance range but have abnormal heights (such as birds flying high in the sky, the top of tunnels, etc.). If only two-dimensional polar coordinates are used, the mask can only define the range in the horizontal plane and cannot perform fine filtering in the height direction, causing noise features in the height direction to still participate in BEV feature generation. Correspondingly, the mask mechanism also enhances the effectiveness of three-dimensional polar coordinate encoding: by shielding invalid areas outside the sensing range, the three-dimensional polar coordinate encoding only acts on points within the effective sensing area, avoiding the waste of encoding resources in irrelevant areas, while also improving the normalization process. , , By keeping the parameters consistent with the mask's receptive range parameters, the encoding space and mask space are unified, allowing the normalized coordinate values to be directly compared with the mask threshold, simplifying the judgment logic. Therefore, the above schemes are interdependent and synergistic. By introducing height information on top of polar coordinate position encoding, the ability of BEV features to express three-dimensional spatial relationships is enhanced. Simultaneously, by combining effective receptive range priors, interference and computational overhead caused by invalid features are reduced. Ultimately, this effectively improves the accuracy and efficiency of BEV feature transformation in occupancy detection and environmental perception tasks.
[0029] As one implementation method, an example application environment design is provided, in which this embodiment can be implemented. The example application environment includes an autonomous vehicle, an onboard domain controller, and a data acquisition module. The autonomous vehicle is equipped with a data acquisition module, which includes a camera, LiDAR, and GPS module, for acquiring road images, point cloud images, and vehicle location data. The onboard domain controller is used to receive the data acquired by the data acquisition module and execute the perspective conversion method proposed in this embodiment. The data acquisition module and the onboard domain controller communicate via onboard Ethernet, Wi-Fi, or Bluetooth.
[0030] In this embodiment, data collection and processing strictly comply with relevant laws and regulations. The collection of data such as vehicle location and road images requires authorization and is used solely for BEV perspective conversion, not for any other unauthorized purposes. It should be understood that the example application environment is described for illustrative purposes only and does not imply any limitation on the scope of this application. This application can also be applied to other scenarios such as intelligent transportation and 3D reconstruction, and the environment may also include components such as sensors and communication modules not shown.
[0031] like Figure 1 As shown, the BEV perspective transformation method proposed in this embodiment, based on three-dimensional polar coordinates and prior perception range, is applicable to scenarios such as autonomous driving environment perception and occupancy detection. The method specifically includes the following steps: Step S1: Acquire images of the vehicle's surrounding environment in real time and extract image features from the environmental images.
[0032] Specifically, during autonomous driving, images of the vehicle's surrounding environment are captured in real time using onboard cameras or other cameras. High-dimensional spatial features are extracted from these images using convolutional neural networks (CNNs) or Transformers, serving as input for subsequent BEV (Battery-to-Electronic Vehicle) conversion. The extracted image features can be multi-scale feature pyramids or single-scale feature maps. CNNs progressively extract semantic features from the images through multi-layer convolution and pooling operations, while Transformers capture global dependencies in the images through a self-attention mechanism. These can be implemented using conventional existing technologies and will not be elaborated upon further here.
[0033] The image features extracted in this way carry semantic information about targets such as roads, vehicles, and obstacles, providing a feature foundation for subsequent coordinate transformation and feature fusion.
[0034] Step S2: Based on the camera imaging model and predicted depth information, obtain the three-dimensional spatial coordinates corresponding to each pixel of the image features.
[0035] In this embodiment, pixel coordinates (u,v) are converted into three-dimensional spatial coordinates (x,y,z) based on the camera imaging model and depth information. The camera imaging model can be pre-calibrated; the depth information can be obtained through a depth prediction network or by depth sampling from actual distances. The homogeneous coordinates of image feature pixels are represented as follows: , The depth corresponding to this pixel can be represented by the coordinate transformation as follows: ; In the above formula, K is the camera intrinsic parameter matrix, R is the rotation matrix from the camera to the vehicle body, and t is the translation matrix from the camera to the vehicle body.
[0036] This step utilizes camera intrinsic and extrinsic parameters and depth information to convert pixel coordinates (u,v) into three-dimensional coordinates (x,y,z) in the vehicle coordinate system, establishing a geometric mapping relationship between image features and three-dimensional space. Then, the camera intrinsic parameter matrix K is used to convert the pixel coordinates into three-dimensional coordinates in the camera coordinate system. Finally, the camera coordinate system is converted into the vehicle coordinate system through the rotation matrix R and the translation matrix t. This achieves the mapping from a two-dimensional image plane to three-dimensional space, providing an accurate spatial coordinate basis for subsequent three-dimensional polar coordinate transformation.
[0037] Step S3: Convert the three-dimensional spatial coordinates into three-dimensional polar coordinates that include radial coordinates, angular coordinates, and height coordinates.
[0038] In this embodiment, the three-dimensional spatial coordinates of the three-dimensional spatial points obtained above are converted into three-dimensional polar coordinates, which include radial coordinates, angular coordinates and height coordinates, and then the three-dimensional polar coordinates are normalized.
[0039] Specifically, corresponding to BEV space, the three-dimensional spatial coordinates in the Cartesian coordinate system in three-dimensional space are... Convert to three-dimensional polar coordinates containing radial coordinate r, angular coordinate θ, and height coordinate z, and then adjust the radial coordinate r and angular coordinate z. The coordinates are normalized to the height coordinate z; the formula for coordinate transformation is as follows: ; ; ; ; In the above formula, the height coordinate z remains unchanged.
[0040] The two-dimensional polar coordinates in existing BEV perspective transformation methods only include ( Unlike other embodiments, this embodiment explicitly introduces z-direction information in position encoding to enhance the representation of three-dimensional spatial structures.
[0041] Furthermore, considering r (distance, on the order of 0 to 200 meters), The numerical ranges of the three components (angle, -π to π), z (height, possibly -5 to 10 meters) differ greatly. Without normalization, direct encoding makes it difficult for the network to learn information from all three dimensions in a balanced way. Therefore, the components of the three-dimensional polar coordinates are normalized to map them to a uniform numerical range. ; ; ; in, For the maximum sensing radius (e.g., 60m), and The lower and upper limits of the height range (e.g., -2m to 8m, covering below ground and viaduct height).
[0042] The normalized parameters are consistent with the sensing range parameters in subsequent steps, thus achieving the unification of the coding space and the mask space.
[0043] Through the above steps, Cartesian coordinates are converted into three-dimensional polar coordinates. Where the radial coordinate r reflects the distance between the target and the vehicle, and the angular coordinate... Reflecting the target's direction, the height coordinate z preserves height information, and normalization eliminates the dimensional differences between different components, enabling the network to learn information in all three dimensions in a balanced manner. Compared to the two-dimensional polar coordinates in existing technologies (…), this approach… In this embodiment, the z-coordinate is explicitly introduced, which can distinguish between ground vehicles and high-altitude obstacles, support three-dimensional perception tasks such as occupancy detection, and improve the ability to represent three-dimensional spatial structures.
[0044] Step S4: Construct an effective perception range mask in advance according to task requirements, use the mask to filter effective coordinate points, and then encode the position of the three-dimensional polar coordinates of the effective coordinate points.
[0045] In this embodiment, a three-dimensional effective perception range is generated based on preset parameters of the actual driving task (such as an autonomous driving task). This preset effective perception range parameter, centered on the vehicle, includes the maximum perception radius. Minimum perceived height Maximum sensing height The maximum sensing radius is based on the forward sensing distance. Backward sensing distance 1. Perceive distance from left to right The value can be determined, for example, by taking the maximum value or by calculating the distance to the farthest corner point. The calculation formula is as follows: ; (Take the distance from the farthest corner point).
[0046] Next, for each image feature point, the corresponding three-dimensional polar coordinate points Perform point-by-point judgment: if r does not exceed And z is arrive If the values are within a certain range, then the mask value for that point is 1 (valid); otherwise, the mask value is 0 (invalid). This condition can be expressed as: ; Through the above method, a length of A one-dimensional point-level mask vector, where This represents the total number of image feature points. Furthermore, since mask determination is performed directly in polar coordinate space, only two scalars, r and z, need to be compared, compared to calculating Euclidean distance in Cartesian coordinate space. More efficient, this efficiency advantage can significantly reduce computation time when dealing with scenes containing hundreds of thousands of feature points.
[0047] As a further implementation method, in the subsequent cross-attention mechanism, the BEV query features have Each cell (corresponding to a BEV grid cell) has the following image features: The shape of the attention weight matrix is... OK The column represents the level of attention each BEV query point gives to each image feature point. Therefore, the one-dimensional point-level mask vector from the previous step needs to be expanded into a two-dimensional attention mask matrix M, with the shape of... OK The column's expansion rules are as follows: For the element in the i-th row and j-th column of the mask matrix M If the mask value of image feature point j If the value equals 1 (within the valid range), then =0 (indicates no occlusion, the image feature point is visible to BEV query point i); if If the value is 0 (outside the valid range), then Equal to negative infinity (indicating complete occlusion, meaning the image feature point is not visible to BEV query point i).
[0048] It should be noted that since the mask is determined based on whether the spatial position of the image feature point itself is within the effective perception range, and is unrelated to the specific BEV cell being queried, for the same image feature point j, its mask value is the same for all BEV query points i. That is, a certain image feature point is visible to all BEV queries (the j-th column of M is all 0), or conversely, it is not visible to all BEV queries (the j-th column of M is all negative infinity).
[0049] Through the above steps, based on the actual perception range of autonomous driving (e.g., 60 meters forward, -2 meters to 8 meters in height), the system determines whether each image feature point is within the effective range, generating a point-level mask vector. This vector is then expanded into a two-dimensional attention mask matrix M (where elements at effective positions have a value of 0, and elements at invalid positions have a value of negative infinity), preparing for the subsequent cross-attention calculation. This two-step construction method allows the masking mechanism to shield image feature points outside the effective range during subsequent attention calculations, making their corresponding attention weights approach zero.
[0050] As another implementation method, the shielding method for the effective sensing range is not limited to a fixed geometry, and can also employ: (1) Dynamic range: The perception area is dynamically adjusted by the scene perception module (such as detection results or map priors). For example, the forward perception distance is increased in highway scenes, and the lateral perception range is increased in urban intersection scenes.
[0051] (2) Layered range: Set a high-resolution effective range for near objects and a low-resolution range for distant objects to balance accuracy and efficiency. For example, use a 0.5m grid resolution in the 0-30m range and a 1m grid resolution in the 30-60m range.
[0052] (3) Probability mask: Instead of completely masking, it assigns lower weights to positions outside the effective range to flexibly reduce their impact. For example, the mask value is expanded from a binary value {0,1} to a continuous value [0,1], and the weights are gradually reduced based on the distance from the effective range.
[0053] Based on obtaining the effective sensing range mask, the normalized three-dimensional polar coordinates ( Perform fixed-position encoding to generate a high-dimensional positional encoding vector. In this embodiment, a method of independent sine and cosine encoding followed by concatenation is adopted. That is, the four components are independently encoded with sine and cosine, and then the encoded vectors are concatenated to form a complete three-dimensional polar coordinate position encoded vector. .
[0054] As a further implementation, before location encoding and image feature fusion, the mask is used to determine whether the three-dimensional polar coordinate points are within the valid range. Points within the range (mask=1) retain their encoding and participate in the fusion normally; points outside the range (mask=0) are masked and do not participate in subsequent calculations. In this embodiment, before encoding, the generated point-level mask vector is used to index and filter the three-dimensional polar coordinate points, selecting valid coordinate points with mask=1 for subsequent encoding. This masks the encoding of invalid positions, thereby avoiding interference from irrelevant features and reducing computational overhead.
[0055] Preferably, the masking method can also adopt various implementation forms such as setting to zero, weight suppression or direct elimination, and there are no specific restrictions here.
[0056] Next, regarding the specific encoding method of the z-coordinate, this embodiment adopts independent encoding followed by concatenation, that is: first, the normalized (r, θ) is encoded using sine and cosine positions: ; ; Where i is the encoding dimension index and d is the total dimension of the encoding vector.
[0057] Then, the normalized z is independently encoded by sine and cosine: ; ; Where j is the dimension index encoded by z. Encode the dimension of the z-vector.
[0058] Finally, the two encoded vectors are concatenated to obtain the three-dimensional polar coordinate position code: ; The above-described method of independently encoding and then concatenating maintains the independence of the horizontal and height components, facilitating separate adjustment of encoding parameters. For example, a lower frequency can be set in the z-direction to represent the height level, as shown in the example (…). Set a higher frequency to capture fine positional changes.
[0059] As another implementation method, three-dimensional polar coordinate position encoding can be implemented in a variety of existing methods, or it can be implemented using an embedding layer with learnable parameters. This embodiment does not limit this.
[0060] Step S5: Fuse the three-dimensional polar coordinate position encoding with the image features to obtain a fused feature vector with spatial position information. Then, map the fused feature vector onto the BEV plane using a BEV transformation framework based on cross-attention to obtain a BEV feature map.
[0061] In this embodiment, the encoded three-dimensional polar coordinate position encoding vector is... The image features F' are projected onto the same dimension as the image features F using a trainable mapping matrix W, and then added element-wise to the image features F to obtain the fused image features F'. In this embodiment, the fusion is achieved by vector addition, and the fusion formula is as follows: ; in, Let F' be a 3D polar coordinate position encoding vector, and W be a trainable linear mapping matrix used to transform the position encoding to the same dimension as the image features. The fused feature F' carries both image semantic information (from the original image features F) and 3D spatial geometric information (from the 3D polar coordinate position encoding). This provides rich input for subsequent cross-attention calculations. This additive method can directly enhance the spatial expressiveness of image features while keeping the feature dimensions unchanged.
[0062] As another implementation method, the above fusion method can also employ vector concatenation, attention fusion, etc., and there are no specific limitations here. The vector concatenation method is as follows: The image feature vector F is compared with the three-dimensional polar coordinate position encoding vector. The features are concatenated to form the expanded feature vector: ; Here, [;] represents a concatenation operation. The concatenated feature vector can be input into a linear mapping layer or subsequent neural network modules to maintain dimensionality consistency.
[0063] In addition, the attention fusion method is: to As a guiding signal, the attention mechanism is introduced to weight image features, that is, to use position encoding as query or key to interact with image features for calculation, thereby highlighting effective features within the perception range.
[0064] Specifically, the three-dimensional polar coordinate position encoding vector is used as the input for attention generation. A lightweight attention convolutional network is used to generate attention weight vectors in the channel dimension. The attention weight vectors are multiplied with the image feature vectors channel by channel to obtain the position-aware weighted feature vector. Finally, the weighted feature vector is added to the original image feature vector to obtain the final fused feature vector.
[0065] By employing the above methods, BEV features can retain semantic information from images while explicitly combining three-dimensional spatial geometric information, thereby enhancing scene understanding capabilities.
[0066] Furthermore, initialize the BEV query feature Q. Specifically, initialize the two-dimensional Cartesian coordinates of each cell in the BEV grid. Perform sine and cosine position encoding respectively, for The components are encoded using a set of sine and cosine functions of different frequencies. The components are also encoded using a set of sine and cosine functions of different frequencies. The two sets of encoded vectors are then concatenated to obtain the initial position encoded vector for each BEV grid. Subsequently, this vector is mapped to the query feature space through a learnable linear layer and added to a learnable embedding bias to obtain the final BEV query feature Q. This approach allows the query features to possess a priori geometrical knowledge of the BEV grid during the initial training phase, which helps accelerate convergence and improve perception accuracy.
[0067] Next, the three-dimensional polar coordinate position is encoded using a trainable mapping matrix W. After projecting onto the same dimension as the image features, the features are added element-wise, allowing the fused features to carry both semantic and 3D spatial geometric information. The BEV query feature Q is obtained by initializing it with BEV grid coordinate position encoding and then using linear layer mapping, giving the query features geometric priors, accelerating model convergence, and improving perceptual accuracy.
[0068] Then, a masked cross-attention computation is performed using the BEV query features and the fused image features. A two-dimensional attention mask matrix is injected into the cross-attention computation to generate the BEV feature map. That is, using the BEV query features Q( The fused image features F' are used as the Key and Value. The process involves performing masked cross-attention calculation, calculating the attention score matrix, and then superimposing it with a mask matrix M to obtain the masked attention score matrix. Next, Softmax normalization is performed on the masked attention score matrix, specifically along the Key dimension (column direction), to obtain the attention weight matrix. Finally, the fused image features are weighted and summed using the attention weight matrix to obtain the final BEV feature map. The entire cross-attention calculation process described above can be represented as follows: ; Through the above mechanism, each BEV grid location can aggregate image features from the corresponding 3D spatial region, while using height information z to distinguish targets at different height levels. For example, for two objects at the same horizontal position but different heights (such as a vehicle on the ground and a vehicle on a bridge), their z coordinates are different, and their corresponding position codes are also different. This allows the attention mechanism to correctly associate them with different BEV grids (if BEV indicates support for multiple heights) or selectively focus on specific height levels according to task requirements.
[0069] Ultimately, the output It is a shape of The feature map can be reshaped into The 3D tensor is directly used for downstream tasks. Specifically, the feature vectors of all BEV grid cells are arranged according to the spatial layout of the BEV grid to obtain a complete BEV feature map. This BEV feature map has both 3D spatial semantics (from 3D polar coordinate encoding) and effective region focusing (from the masking mechanism), and can be directly applied to downstream tasks such as ocupancy detection, 3D object detection, and environmental perception.
[0070] The above steps are the core of the entire method. The constructed mask matrix M is injected into the cross-attention calculation to achieve the aggregation of effective features. By injecting the mask matrix M (negative infinity makes the Softmax weight of points outside the range approach zero) before Softmax, the aggregation of invalid features is reduced from the source of calculation, thereby reducing redundant calculations, improving inference speed, and making the feature extraction of the model more targeted in complex environments, thus improving the overall robustness.
[0071] Through the above methods, this embodiment demonstrates improved 3D spatial representation capabilities. By introducing the z-coordinate, the model can distinguish targets at different height levels. For example, when processing elevated bridge scenes, although vehicles on and under the bridge have similar projected positions on the BEV plane, their z-coordinates differ significantly. This difference in position encoding allows the attention mechanism to correctly distinguish them, avoiding feature confusion. Regarding computational efficiency, the mask mechanism significantly reduces computation by shielding invalid regions before attention calculation. Assuming the original image has a feature point count of [missing information], [missing information]. BEV query points are The computational complexity of attention without using a mask is O(n). However, after using a mask, assuming the number of image feature points within the effective perception range is reduced to... The computational complexity is reduced to This saves approximately 30% of computation. Regarding feature quality, by masking noise features outside the perception range (such as distant sky and irrelevant objects at the road edge), the signal-to-noise ratio of BEV features is improved. In terms of synergistic effects, the combination of 3D polar coordinate encoding and the mask mechanism produces superior results. If only 3D polar coordinate encoding is used without a mask, noise points such as birds flying high in the sky and tunnel tops, although having correct z-coding, will still participate in BEV feature generation, introducing interference. If only a mask is used without introducing z-coding, the mask can only define a range within the horizontal plane and cannot filter noise points that are within the horizontal distance but have abnormal heights. Only by combining both can accurate filtering in three-dimensional space be achieved.
[0072] By introducing z-direction information on the basis of polar coordinate position encoding, this embodiment expands the two-dimensional polar coordinates into three-dimensional polar coordinate position encoding. This allows BEV features to not only include planar positional relationships but also accurately express spatial height distribution, improving the accuracy of ocupancy detection and 3D scene understanding. Simultaneously, by utilizing the effective perception range prior and injecting a mask matrix before the Softmax calculation in the cross-attention mechanism, invalid spatial locations are excluded from BEV feature generation. This effectively reduces noise interference and redundant computation, further improving inference speed and overall robustness. It solves the problems of insufficient accuracy and low efficiency caused by existing BEV viewpoint conversion methods ignoring height information and including invalid spatial locations outside the perception range in calculations.
[0073] Example 2 This embodiment provides a BEV perspective conversion system based on three-dimensional polar coordinates and prior perception range, including: The image acquisition and feature extraction module is used to acquire images of the environment surrounding the vehicle in real time and extract image features from the environmental images. The pixel coordinate extraction module is used to obtain the three-dimensional spatial coordinates of each pixel in the image feature based on the camera imaging model and predicted depth information. The coordinate transformation module is used to convert three-dimensional spatial coordinates into three-dimensional polar coordinates that include radial coordinates, angular coordinates, and height coordinates; The coordinate position encoding module is used to pre-build an effective perception range mask according to task requirements, use the mask to filter effective coordinate points, and then encode the position of the three-dimensional polar coordinates of the effective coordinate points. The BEV feature generation module is used to fuse three-dimensional polar coordinate position encoding with image features to obtain a fused feature vector with spatial position information. Through a BEV transformation framework based on cross attention, the fused feature representation is mapped to the BEV plane to obtain a BEV feature map.
[0074] Based on the above system, this embodiment constructs a three-dimensional polar coordinate position code that incorporates height information during the BEV perspective transformation process to enhance the three-dimensional spatial representation capability. It also optimizes the position code by combining the effective perception range prior, thereby improving the accuracy and robustness of BEV features while ensuring computational efficiency.
[0075] Specifically, this embodiment constructs a three-dimensional polar coordinate position encoding, that is: in the process of generating BEV features from camera image features, firstly, based on the camera imaging model and depth information, the three-dimensional spatial coordinates (x, y, z) corresponding to the image pixels are obtained. On this basis, the coordinates are converted into a system containing radial coordinates r and angular coordinates. It also provides a three-dimensional polar coordinate representation of the height coordinate z, and fuses this three-dimensional polar coordinate encoding with the corresponding image features, thereby preserving complete spatial geometric information during BEV perspective transformation.
[0076] BEV (Battery Electric Vehicle) is essentially a two-dimensional plane. The z-coordinate is flattened when projected onto the BEV. Existing technologies generally consider introducing z-information during the position encoding stage redundant, given that the final output is a two-dimensional BEV feature. However, the innovation of this solution lies in retaining z-information during the position encoding stage before BEV transformation. This allows the cross-attention mechanism to distinguish targets at different heights (e.g., ground vehicles versus airborne obstacles, vehicles under bridges versus vehicles on bridges) by utilizing height information when establishing the association between image features and BEV query features. Although the final BEV feature map is two-dimensional, the feature formation process utilizes three-dimensional spatial information, enabling objects at different heights within the same horizontal position to be correctly associated with their corresponding BEV grids, thus supporting ocupancy-based three-dimensional stereo perception tasks. This strategy of three-dimensionalizing the intermediate process and two-dimensionalizing the final output is the key difference between this solution and existing technologies.
[0077] Meanwhile, this embodiment also introduces an effective perception range prior, that is: when constructing the location code, based on the actual needs of the autonomous driving scenario, the effective perception range of the BEV space (e.g., a rectangular or circular area centered on the vehicle) is pre-set. In the process of generating the location code, for spatial locations outside the effective range, their location codes are masked by setting a mask to avoid invalid features from participating in the generation and fusion of BEV features.
[0078] The introduction of the aforementioned effective sensing range prior is coordinated and related to the three-dimensional polar coordinate position encoding. Because of the introduction of three-dimensional polar coordinates (including z), the mask can accurately define the effective range in three-dimensional space (limiting not only the horizontal distance r but also the height z), thereby filtering out noise points that are within the horizontal distance range but have abnormal heights (such as birds flying high in the sky, the top of tunnels, etc.). If only two-dimensional polar coordinates are used, the mask can only define the range in the horizontal plane and cannot perform fine filtering in the height direction, causing noise features in the height direction to still participate in BEV feature generation. Correspondingly, the mask mechanism also enhances the effectiveness of three-dimensional polar coordinate encoding: by shielding invalid areas outside the sensing range, the three-dimensional polar coordinate encoding only acts on points within the effective sensing area, avoiding the waste of encoding resources in irrelevant areas, while also improving the normalization process. , , By keeping the parameters consistent with the mask's receptive range parameters, the encoding space and mask space are unified, allowing the normalized coordinate values to be directly compared with the mask threshold, simplifying the judgment logic. Therefore, the above schemes are interdependent and synergistic. By introducing height information on top of polar coordinate position encoding, the ability of BEV features to express three-dimensional spatial relationships is enhanced. Simultaneously, by combining effective receptive range priors, interference and computational overhead caused by invalid features are reduced. Ultimately, this effectively improves the accuracy and efficiency of BEV feature transformation in occupancy detection and environmental perception tasks.
[0079] Example 3 This embodiment also provides a vehicle, including the above-described BEV perspective conversion system based on three-dimensional polar coordinates and perception range prior, or performing the above-described BEV perspective conversion method based on three-dimensional polar coordinates and perception range prior.
[0080] Example 4 This embodiment provides an electronic device, including: a memory for storing executable instructions; and a processor for executing the executable instructions stored in the memory to implement the method provided in this embodiment.
[0081] Example 5 This embodiment also provides a computer-readable storage medium storing executable instructions, which, when executed by a processor, will cause the processor to execute the method described above in this embodiment.
[0082] The steps and methods involved in Embodiments 2 to 5 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.
[0083] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.
[0084] The above description is only a preferred embodiment of the present invention. Although the specific implementation of the present invention has been described in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that, based on the technical solution of the present invention, various modifications or variations that can be made by those skilled in the art without creative effort are still within the scope of protection of the present invention.
Claims
1. A BEV view transformation method based on three-dimensional polar coordinates and perception range prior, characterized in that, include: Real-time acquisition of images of the vehicle's surrounding environment, and extraction of image features from the environmental images; Based on the camera imaging model and predicted depth information, obtain the three-dimensional spatial coordinates corresponding to each pixel of the image features; Convert three-dimensional spatial coordinates into three-dimensional polar coordinates that include radial coordinates, angular coordinates, and height coordinates; Based on the task requirements, an effective perception range mask is pre-constructed, effective coordinate points are filtered using the mask, and then the three-dimensional polar coordinates of the effective coordinate points are position-encoded. The three-dimensional polar coordinate position encoding is fused with image features to obtain a fused feature vector with spatial position information. The fused feature vector is then mapped to the BEV plane through a BEV transformation framework based on cross attention to obtain a BEV feature map.
2. The BEV perspective conversion method based on three-dimensional polar coordinates and prior perception range as described in claim 1, characterized in that, For the three-dimensional space coordinates of each pixel point in the image features, the three-dimensional space coordinates in the Cartesian coordinate system are converted into three-dimensional polar coordinates including a radial coordinate r, an angle coordinate and a height coordinate z, and the radial coordinate r, the angle coordinate and the height coordinate z are normalized; wherein the calculation formula of the coordinate conversion is: r = x / sqrt(x^2 + y^2) and z = z / sqrt(x^2 + y^2) ; ; ; ; In the above formula, the height coordinate z remains unchanged.
3. The BEV view conversion method based on three-dimensional polar coordinates and perception range prior of claim 1, wherein, Pre-construct an effective perception range mask based on task requirements, including: According to the actual task of driving the vehicle, preset parameters are generated to generate a three-dimensional effective sensing range; the preset parameters include a maximum sensing radius and a height range , wherein the maximum sensing radius is determined by taking the maximum value based on a forward sensing distance , a rear sensing distance , and left and right sensing distances . Based on the effective 3D sensing range, the 3D polar coordinates corresponding to each image feature pixel are judged point by point. If the radial coordinate does not exceed the preset maximum sensing radius... And the height coordinates are within the preset height range If the value of a point is within the specified range, the mask value of that point is recorded as 1 to indicate validity; otherwise, it is recorded as 0 to indicate invalidity, thus generating a point-level mask vector.
4. The BEV perspective conversion method based on three-dimensional polar coordinates and prior perception range as described in claim 3, characterized in that, By combining an effective perceptual range mask, the three-dimensional polar coordinates of image feature pixels are encoded for their positions, including: Before encoding, the generated point-level mask vector is used to index and filter the three-dimensional polar coordinate points, and the valid coordinate points with mask=1 are selected for encoding. For each valid coordinate point, the normalized radial coordinate r and angular coordinate... The z-coordinate and height coordinate are encoded using sine and cosine functions respectively, and the encoded vectors of multiple components are concatenated to form a three-dimensional polar coordinate position encoded vector.
5. The BEV perspective conversion method based on three-dimensional polar coordinates and prior perception range as described in claim 1, characterized in that, The three-dimensional polar coordinate position code and image features are fused using any one of the following methods: vector concatenation, vector addition, or attention fusion. The vector concatenation method is as follows: the image feature vector is directly concatenated with the three-dimensional polar coordinate position encoding vector to form an expanded feature vector. The vector addition method is as follows: the three-dimensional polar coordinate position encoding vector is projected to the same dimension as the image features through a trainable mapping matrix, and the projected position encoding vector is added to the image feature vector element by element to obtain the fused feature vector with spatial position information. The attention fusion method is as follows: the three-dimensional polar coordinate position encoding vector is used as the input for attention generation. A lightweight attention convolutional network is used to generate attention weight vectors in the channel dimension. The attention weight vector is multiplied with the image feature vector channel by channel to obtain the position-aware weighted feature vector. The weighted feature vector is added to the original image feature vector to obtain the final fused feature vector.
6. The BEV perspective conversion method based on three-dimensional polar coordinates and prior perception range as described in claim 1, characterized in that, A masked cross-attention mechanism is used to map the fused feature vectors onto the BEV plane, resulting in a grid-arranged BEV feature map, including: Initialize BEV query features; Using the BEV query feature as the Q value and the fused feature vector as the K and V values, a two-dimensional attention mask matrix is introduced for cross-attention calculation to obtain a masked attention score matrix; wherein, valid point positions in the point-level mask vector are mapped to 0 and invalid point positions are mapped to -. This is extended to obtain a two-dimensional attention mask matrix, the dimension of which is the number of BEV query features × the number of image feature points; Perform Softmax normalization on the masked attention score matrix to obtain the attention weight matrix; The fused feature vectors are weighted and summed using the attention weight matrix to obtain the final BEV feature map arranged in a grid.
7. The BEV perspective conversion method based on three-dimensional polar coordinates and prior perception range as described in claim 6, characterized in that, Initialize BEV query features, including: The two-dimensional Cartesian coordinates of each grid in the BEV plane are respectively sine and cosine encoded, and the two sets of encoded vectors are concatenated to obtain the initial position encoded vector; The initial position encoding vector is mapped to the query feature space through a learnable linear layer, and then added to the learnable embedding bias to obtain the initialized BEV query features.
8. A BEV perspective conversion system based on three-dimensional polar coordinates and prior perception range, characterized in that, include: The image acquisition and feature extraction module is used to acquire images of the environment surrounding the vehicle in real time and extract image features from the environmental images. The pixel coordinate extraction module is used to obtain the three-dimensional spatial coordinates of each pixel in the image feature based on the camera imaging model and predicted depth information. The coordinate transformation module is used to convert three-dimensional spatial coordinates into three-dimensional polar coordinates that include radial coordinates, angular coordinates, and height coordinates; The coordinate position encoding module is used to pre-build an effective perception range mask according to task requirements, use the mask to filter effective coordinate points, and then encode the position of the three-dimensional polar coordinates of the effective coordinate points. The BEV feature generation module is used to fuse three-dimensional polar coordinate position encoding with image features to obtain a fused feature vector with spatial position information. Through a BEV transformation framework based on cross attention, the fused feature representation is mapped to the BEV plane to obtain a BEV feature map.
9. A vehicle, characterized in that, Includes the BEV perspective conversion system based on three-dimensional polar coordinates and perception range prior as described in claim 8, or performs the BEV perspective conversion method based on three-dimensional polar coordinates and perception range prior as described in any one of claims 1-7.
10. An electronic device, characterized in that, include: Memory, used to store executable instructions; The processor, when executing executable instructions stored in the memory, implements the BEV perspective conversion method based on three-dimensional polar coordinates and prior perception range as described in any one of claims 1-7.
11. A computer-readable storage medium, characterized in that, The device stores executable instructions that, when executed by a processor, implement the BEV viewpoint conversion method based on three-dimensional polar coordinates and prior perception range as described in any one of claims 1-7.
