Environment sensing method and apparatus, and device, medium and product

Abstract

An environment sensing method and apparatus, and a device, a medium and a product. The method comprises: acquiring surrounding environment data of a vehicle, and on the basis of the environment data, generating environmental features of different actual areas in an environment where the vehicle is located (101); mapping the environmental features of different actual areas to corresponding grid cells in a BEV grid map, so as to obtain a BEV feature map, wherein the resolutions of layers of grid cells in the BEV grid map, which extend outwards from the center, progressively decrease (102); and on the basis of the BEV feature map, performing an environment sensing task (103).

Description

An environmental sensing method, device, equipment, medium, and product Cross-references to related applications

[0001] This application claims priority to Chinese Patent Application No. 2025104876747, filed with the Chinese Patent Office on April 18, 2025, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The embodiments of this application relate to, but are not limited to, the field of autonomous driving technology, and particularly to, but are not limited to, an environmental perception method, device, equipment, medium, and product. Background Technology

[0003] With the rapid development of autonomous driving technology, environmental perception, as one of the core components of autonomous driving systems, plays a crucial role in achieving safe and reliable autonomous driving. Bird's Eye View (BEV) technology, due to its ability to intuitively present environmental information, has become the mainstream method in the field of environmental perception. Summary of the Invention

[0004] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0005] According to a first aspect of any embodiment of this application, an environmental perception method is provided, the method comprising:

[0006] Generate environmental characteristics of different actual areas in the vehicle's environment based on the environmental data surrounding the vehicle;

[0007] The environmental features of the different regions are mapped to the corresponding grid cells in the BEV grid map to obtain the BEV feature map; wherein, the resolution of each layer of grid cells in the BEV grid map decreases sequentially from the center outwards.

[0008] Environmental perception tasks are performed based on the BEV feature map.

[0009] According to a second aspect of any embodiment of this application, an environmental sensing device is provided, the device comprising:

[0010] The generation module is configured to generate environmental characteristics of different actual areas in the environment where the vehicle is located, based on environmental data around the vehicle.

[0011] The mapping module is configured to map the environmental features of the different regions to corresponding grid cells in the BEV grid map to obtain a BEV feature map; wherein the resolution of each layer of grid cells in the BEV grid map decreases sequentially from the center outwards.

[0012] The perception module is configured to perform environmental perception tasks based on the BEV feature map.

[0013] According to a third aspect of any embodiment of this application, an electronic device is provided, comprising:

[0014] processor;

[0015] Memory used to store processor-executable instructions;

[0016] The processor executes the executable instructions to implement the method described in any embodiment of this application.

[0017] According to a fourth aspect of any embodiment of the present application, a computer-readable storage medium is provided having computer instructions stored thereon that, when executed by a processor, implement the method described in any of the embodiments of the present application described above.

[0018] According to a fifth aspect of any embodiment of this application, a computer program product is provided, having a computer program / instructions stored thereon, which, when executed by a processor, implement the method described in any of the embodiments of this application described above.

[0019] The technical solution provided in this application may include the following beneficial effects:

[0020] As can be seen from the above embodiments, environmental features of different actual areas in the vehicle's environment are generated based on the environmental data around the vehicle. The environmental features of different areas are mapped to the corresponding grid cells in the BEV grid map to obtain the BEV feature map. The environmental perception task is performed based on the BEV feature map. By mapping the environmental features to the BEV grid map with decreasing resolution from the center to the outside, it is possible to reduce the amount of computation in the far-distance area while ensuring high-precision perception of the near-distance area around the vehicle, thereby achieving a balance between computing resources and perception accuracy.

[0021] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this specification. Other aspects will become clear after reading and understanding the accompanying drawings and detailed description. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application, but do not constitute a limitation on the technical solutions of this application.

[0023] Figure 1 is a flowchart illustrating an environmental perception method according to an exemplary embodiment of this application.

[0024] Figure 2 is a schematic diagram of a vehicle coordinate system according to an exemplary embodiment of this application.

[0025] Figure 3 is a schematic diagram of a field-of-view grid diagram according to an exemplary embodiment of this application.

[0026] Figure 4 is a schematic diagram of a BEV mesh diagram according to an exemplary embodiment of this application.

[0027] Figure 5 is a schematic diagram illustrating the division effect of a field-of-view grid map according to an exemplary embodiment of this application.

[0028] Figure 6 is a flowchart illustrating another environmental perception method according to an exemplary embodiment of this application.

[0029] Figure 7 is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of this application.

[0030] Figure 8 is a block diagram of an environmental sensing device according to an exemplary embodiment of this application. Detailed Implementation

[0031] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0032] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0033] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0034] Environmental perception methods based on BEV (Battery Electric Vehicle) convert environmental data collected by multiple cameras into a top-down view to achieve overall perception of the surrounding environment. In some embodiments, the BEV feature map uses a fixed-resolution feature representation, which presents a challenge in balancing computational resources and perception accuracy.

[0035] In autonomous driving scenarios, closer areas typically require more refined perception to ensure safety, while farther areas have relatively lower accuracy requirements. Fixed-resolution BEV feature maps waste computational resources for farther areas and result in insufficient perception accuracy for closer areas.

[0036] Please refer to Figure 1, which is a flowchart illustrating an environmental perception method according to an exemplary embodiment of this application. This environmental perception method can be executed by a perception system, which can be applied to vehicles, single servers, cluster servers, cloud servers, or other server-side applications. This environmental perception method can also be executed by other systems or devices in different application scenarios; this application embodiment does not impose any limitations on this.

[0037] As shown in Figure 1, the environmental perception method may include the following steps 101-103.

[0038] Step 101: Obtain environmental data around the vehicle and generate environmental characteristics of different actual areas in the vehicle's environment based on the environmental data.

[0039] In this step, the perception system can acquire environmental data around the vehicle based on onboard cameras, radar, or other sensor devices. This environmental data is raw information about the vehicle's surroundings obtained from various sensors, providing foundational information for subsequent environmental perception.

[0040] In one embodiment, environmental data may include visual data acquired by a visual sensor and / or point cloud data acquired by radar. The visual sensor may be a panoramic camera located at various points on the vehicle, covering a 360° field of view. The visual data is a sequence of raw images acquired by the camera, which may contain RGB or grayscale pixel information. For example, the visual data may be acquired by six panoramic cameras located at the front, rear, left, right, front left, and front right of the vehicle.

[0041] As mentioned above, by fusing multimodal data from visual data and point cloud data, the advantages of different sensors can be complemented, enhancing the generalization ability of environmental perception in scenarios with changes in lighting or occlusion, and improving the accuracy of environmental perception.

[0042] Feature extraction can be performed on the environmental data surrounding the vehicle to generate environmental features for different actual areas within the vehicle's environment. These environmental features are information extracted from the environmental data to describe specific attributes of the environment surrounding the vehicle; they can be two-dimensional (2D) features, three-dimensional (3D) features, etc. Different actual areas are the different spatial regions that divide the vehicle's environment.

[0043] For example, a progressively expanding geometric partitioning method can be used to divide the vehicle's environment into different actual regions, forming an outwardly expanding nested structure. For instance, a nested square region can be expanded outward from the origin of the vehicle coordinate system, dividing it into multiple concentric square regions, resulting in different actual regions, each corresponding to a different distance range. Alternatively, a nested circular region can be expanded outward from the origin of the vehicle coordinate system, dividing it into multiple concentric annular regions, resulting in different actual regions, each corresponding to a different distance range. Yet another example is that a nested polygonal region can be expanded outward from the origin of the vehicle coordinate system, dividing it into multiple concentric polygonal regions, resulting in different actual regions, each corresponding to a different distance range.

[0044] It is understandable that other partitioning methods such as polar coordinates and dynamic topology can also be used to divide the different actual areas in the vehicle's environment, as long as the resolution of each layer of grid cells from the center outwards in the mapped BEV grid map can be successively reduced. This application does not limit this.

[0045] In one embodiment, environmental data may include visual data collected by a visual sensor, and environmental features may include visual features, which are two-dimensional features extracted from the visual data and used to capture abstract information such as target contours and textures.

[0046] Visual data acquired through vision sensors can be obtained. This visual data is then input into a feature extraction model to obtain the visual features output by the model. The feature extraction model can include the ResNet50 model, which has a deep residual structure. Visual features can be extracted from the visual data based on the ResNet50 model.

[0047] As mentioned above, by extracting visual features from visual data based on the ResNet50 model and utilizing the deep residual structure of ResNet50 itself, the ability to express complex environmental features can be enhanced, thereby improving the robustness of subsequent environmental perception tasks.

[0048] Step 102: Map the environmental features of different real areas to the corresponding grid cells in the BEV grid map to obtain the BEV feature map; wherein, the resolution of each layer of grid cells in the BEV grid map decreases sequentially from the center outwards.

[0049] In this step, the perception system can map the environmental features of different real areas to the corresponding grid cells in the BEV grid map. Each real area corresponds to each layer of grid cells, and the environmental features of each real area can be mapped to the corresponding layer of grid cells in the BEV grid map to obtain the BEV feature map.

[0050] Among them, the BEV mesh map is a structured spatial representation that describes the surrounding environment from a top-down (bird's-eye view) perspective, centered on the vehicle. It is a two-dimensional planar map composed of multiple layers of mesh units, with each mesh unit corresponding to a region in the physical world.

[0051] The BEV mesh diagram includes multiple layers of mesh cells from the center outwards, with the resolution (i.e., the physical size represented by each mesh cell) decreasing sequentially from the center outwards. The number of different actual regions is the same as the number of mesh cell layers in the BEV mesh diagram.

[0052] Step 103: Perform environmental perception task based on BEV feature map.

[0053] In this step, the perception system can perform environmental perception tasks based on the BEV feature map. Environmental perception tasks can include object detection, space occupancy prediction, etc.

[0054] For example, the BEV feature map can be input into an object detection model to obtain the object detection result output by the object detection model; similarly, the BEV feature map can be input into a space occupancy prediction model to obtain the space occupancy prediction result output by the space occupancy prediction model, and so on. The space occupancy prediction model can be a multilayer perceptron model, and the space occupancy prediction result can include the probability of each occupancy grid being occupied or unoccupied.

[0055] The environmental perception method in this embodiment acquires environmental data around the vehicle and generates environmental features for different actual areas in the vehicle's environment based on the environmental data. The environmental features of different areas are then mapped to corresponding grid cells in the BEV grid map to obtain the BEV feature map. The environmental perception task is performed based on the BEV feature map. By mapping the environmental features to the BEV grid map with decreasing resolution from the center outwards, the method can ensure high-precision perception of the near-field area around the vehicle while reducing the computational load of the far-field area. This achieves a balance between computational resources and perception accuracy, making it more suitable for deployment and application in actual autonomous driving scenarios.

[0056] Furthermore, by using a multi-resolution BEV mapping mechanism based on spatial representation angles, a more intuitive and stable resolution adjustment mechanism can be achieved without dynamically switching network depth according to resolution, thereby simplifying the system structure and improving the robustness of environmental perception.

[0057] In the foregoing embodiments, it was described that by mapping the environmental features of different real-world regions to each layer of grid cells in the BEV mesh map with progressively decreasing resolution from the center outwards, high-precision perception of near-field regions is maintained while reducing computational resource consumption for distant regions. The following embodiments will provide a more detailed explanation of the environmental feature mapping process, which can be applied to any of the embodiments described above.

[0058] In one embodiment, a field-of-view grid map can be set under a preset coordinate system. The field-of-view grid map is a two-dimensional grid structure established under the preset coordinate system, used to divide the physical space into multi-level regions. It can serve as an intermediate mapping carrier from environmental features to the BEV space, and is used to establish the spatial correspondence between environmental data and BEV feature maps. The preset coordinate system can be a world coordinate system, a vehicle coordinate system, etc.

[0059] The field-of-view (POV) grid is divided into n PAV regions from the center outwards, with the area of ​​each PAV region gradually increasing, and n being no less than 2. The number of n can be equal to the number of grid cell layers in the BEV grid. A PAV region is a continuous spatial range defined from the center outwards in the PAV grid. Each PAV region corresponds to different distance levels and resolution requirements, used for progressive partitioning management of physical space.

[0060] In one embodiment, the field-of-view grid map may include n nested square borders extending outwards from the center. The outermost square border has a side length greater than or equal to the side length of the field-of-view grid map, and the difference in side length between any two adjacent square borders increases sequentially outwards from the center. If the side length of a square border exceeds the side length of the field-of-view grid map, the corresponding area exceeding this length can be filled with zero values, considered an invalid area.

[0061] The n visual field regions can include: for the first visual field region, the visual field region is the area of ​​the first square; for the ith visual field region, the ith visual field region is the area between the ith square and the (i-1)th square, where i is not less than 2 and i is not greater than n.

[0062] Please refer to Figure 2, which illustrates a schematic diagram of a vehicle coordinate system. The origin of the vehicle coordinate system is located at the vehicle's center of mass, i.e., the vehicle's geometric center. The x-axis, y-axis, and z-axis correspond to the vehicle's front-to-back, left-to-right, and vertical directions, respectively. The x-axis points in the vehicle's forward direction, the y-axis points to the driver's left, and the z-axis is perpendicular to the vehicle and points upwards. For example, the environmental perception range can be defined as a radius of 10m in each direction (front, back, left, right), 2m above, and 1m below, centered on the origin of the vehicle coordinate system.

[0063] For example, the resolution of each layer of grid cells from the center outwards in the BEV mesh diagram can be determined according to the following formula 1:

[0064] Where, d i S represents the side length of the border of the i-th square in the field-of-view grid. i This represents the real-world side length corresponding to the i-th layer mesh cell in the BEV mesh diagram. This represents the progressive decay of resolution in each grid cell from the center outwards.

[0065] For example, the side length of the i-th square's border can be calculated using the following formula 2: d i =d i-1 +2×i Formula 2

[0066] Where, d i This represents the side length of the border of the i-th square.

[0067] Please refer to Figure 3, which shows a schematic diagram of a field of view grid. A field of view grid with a side length of 20 meters can be established using the origin of the vehicle coordinate system as a reference. Each grid cell in the field of view grid has a side length of 1 meter. The field of view grid is a top-down view of the grid in the world coordinate system. The field of view grid can include four nested square borders (black square borders) from the center outwards. Taking d1 = 2 meters as an example, the first square border (i = 1) has a side length of 2 meters, the second square border (i = 2) has a side length of 6 meters, the third square border (i = 3) has a side length of 12 meters, and the fourth square border (i = 4) has a side length of 20 meters.

[0068] Using nested square borders, the visual field grid is divided into four visual field regions (i = 1, 2, 3, 4) from the center outwards, with the area of ​​each region gradually increasing. The first visual field region is the area of ​​the first square, the second visual field region is the area between the second and first squares, the third visual field region is the area between the third and second squares, and the fourth visual field region is the area between the fourth and third squares.

[0069] As described above, by determining n nested square borders from the center outward in the field of view grid diagram set under the preset coordinate system, and using nested squares to divide the field of view region, the feature space mapping logic can be simplified through geometric symmetry and hierarchical progression, thereby improving the computational efficiency of grid division and feature mapping.

[0070] For the environmental features of each actual area, the environmental features can be mapped to the position of the corresponding grid cell in the BEV grid map according to the coordinate mapping relationship between the i-th field of view in the field of view grid map and the i-th layer in the BEV grid map to obtain the BEV feature map.

[0071] The coordinate mapping relationship describes the coordinate transformation relationship between the coordinate position of a certain grid cell in the field of view grid and the coordinate position of the corresponding grid cell in the BEV grid. This can ensure the geometric consistency of environmental features in spatial transformation and realize the lossless transfer of multi-resolution features.

[0072] Please refer to Figure 4, which illustrates a schematic diagram of a BEV mesh graph. For example, a BEV mesh graph may include four layers of mesh cells. The feature dimensions of the BEV features in the BEV mesh are B×C×H×W, where B is the number of image frames of the environmental data, C is the number of channels, W is the width of the BEV feature, and H is the height of the BEV feature. For example, the feature dimensions of the BEV mesh graph shown in Figure 4 are B=1, C=256, W=H=10.

[0073] Please refer to Figure 3. The actual areas in the vehicle's environment can be divided into four actual areas. For the environmental features of each actual area, according to the coordinate mapping relationship between the i-th field of view area in the field of view grid diagram shown in Figure 3 and the i-th layer in the BEV grid diagram shown in Figure 4, the environmental features can be mapped to the position of the corresponding grid cell in the BEV grid diagram to obtain the BEV feature map. This makes the resolution of BEV features higher in the near range at the center of the field of view than in the far range.

[0074] As described above, for each actual area, based on the coordinate mapping relationship between the i-th field of view in the field of view grid map and the i-th layer in the BEV grid map, the environmental features are mapped to the positions of the corresponding grid cells in the BEV grid map to obtain the BEV feature map. By utilizing the coordinate mapping relationship, the BEV mapping of environmental features can be quickly achieved without relying on complex models or algorithms, thereby further reducing computational overhead.

[0075] Please refer to Figure 5, which illustrates a schematic diagram of the grid division effect within a field of view. For example, the side length of the grid cell in the i-th square border can be calculated using the following formula 3: S i =i Formula 3

[0076] Taking S1 = 1 meter as an example, in the first field of view (i = 1), the side length of each grid cell is 1 meter; in the second field of view (i = 2), the side length of each grid cell is 2 meters; in the third field of view (i = 3), the side length of each grid cell is 3 meters; and in the fourth field of view (i = 4), the side length of each grid cell is 4 meters.

[0077] In the first layer of grid cells, each grid cell corresponds to a 1m×1m grid in the first field of view region of the field of view grid diagram. It covers a smaller physical area and has a higher resolution. In the fourth layer of grid cells, each grid cell corresponds to a 4m×4m grid in the fourth field of view region of the field of view grid diagram. It covers a larger physical area and has a lower resolution.

[0078] It is understood that the above-described method of dividing the field of view in the field of view grid diagram is only an example, and other methods of division can also be used. This application does not limit this.

[0079] In one embodiment, the coordinate position P of the grid cell in the field-of-view grid map can be obtained by means of camera parameter conversion, etc. 2d (u,v) and the real-world coordinates P in the vehicle coordinate system W Projection mapping relationship between (x,y).

[0080] The coordinate mapping relationship between the i-th view region in the view region grid map and the i-th layer in the BEV grid map can be determined using a linear interpolation algorithm. For example, the coordinate position P of the grid cell in the BEV grid map can be used as an example. bev (u,v) is used to calculate its real-world coordinate position P in the vehicle coordinate system using linear interpolation. W (x,y,z) determines the coordinate position P of the mesh element in the BEV mesh diagram. bev (u,v) and the real-world coordinates P in the vehicle coordinate system W Interpolation mapping relationship between (x,y,z).

[0081] By combining interpolation mapping and projection mapping, and based on the real-world coordinates in the vehicle coordinate system, the coordinates of the grid cells in the BEV mesh diagram are associated with the coordinates of the grid cells in the view mesh diagram. For each grid cell in the BEV mesh diagram corresponding to the real-world coordinates in the vehicle coordinate system, the coordinates of the grid cells in the view mesh diagram corresponding to that coordinate position can be determined, thus obtaining the coordinate mapping relationship between the i-th view region in the view mesh diagram and the i-th layer in the BEV mesh diagram.

[0082] As described above, by obtaining the projection mapping relationship between the coordinate positions of the grid cells in the field of view grid map and the coordinate positions in the real world under the vehicle coordinate system, and the interpolation mapping relationship between the coordinate positions of the grid cells in the BEV grid map and the coordinate positions in the real world under the vehicle coordinate system, the coordinate mapping relationship between the i-th field of view region in the field of view grid map and the i-th layer in the BEV grid map can be determined based on the projection mapping relationship and the interpolation mapping relationship. This can avoid the feature misalignment problem caused by coordinate discretization and improve the spatial continuity of the BEV feature map.

[0083] In one embodiment, the projection mapping relationship can be determined based on the camera parameters of the camera used to collect environmental data. For example, the camera parameters may include camera intrinsic parameters and camera extrinsic parameters. Based on the camera's intrinsic and extrinsic parameters, the coordinate position of the grid cell in the i-th field of view in the field of view grid map corresponding to the real-world coordinate position in the vehicle coordinate system can be determined, thus obtaining the projection mapping relationship between the coordinate position of the grid cell in the field of view grid map and the real-world coordinate position in the vehicle coordinate system.

[0084] Based on the determined coordinate mapping relationship, environmental features can be extracted from the coordinate position of the i-th visual region in the visual field grid map, and the environmental features can be assigned to the corresponding grid cell in the BEV grid map to obtain the BEV feature map.

[0085] Camera intrinsic parameters are matrices that describe the internal optical characteristics of the camera, and may include focal length, distortion coefficients, etc. Camera extrinsic parameters describe the camera's pose in the world coordinate system, and may include rotation matrices, translation vectors, etc.

[0086] For example, the coordinate position of the grid cell of the i-th field of view region in the field of view grid diagram can be determined according to the following formula 4: P 2d (u,v)=KTP W (x,y,z) Formula 4

[0087] Among them, P 2d (u,v) represents the coordinate position of the grid cell in the i-th field of view region in the field of view grid map, K represents the camera intrinsic parameters, T represents the camera extrinsic parameters, and P... W (x,y,z) represents the real-world coordinates in the vehicle coordinate system.

[0088] As described above, by determining the projection mapping relationship based on the camera parameters of the camera used to collect environmental data, environmental features are extracted from the coordinate position of the i-th field of view region in the field of view grid map, and the environmental features are assigned to the corresponding grid cells in the BEV grid map to obtain the BEV feature map. By using camera parameters to determine the coordinate position and directly assigning features, coordinate transformation errors can be reduced and the calculation process can be simplified, thereby improving the reliability of BEV feature map generation.

[0089] To further illustrate the environmental perception process, Figure 6 shows a flowchart of another environmental perception method. This environmental perception method may include the following steps:

[0090] Step 601: Acquire visual data about the area around the vehicle.

[0091] In this step, the perception system can simultaneously acquire images with dimensions of 1920×1080 using six panoramic cameras located at the front / rear / left / right / left front / right front of the vehicle to obtain visual data about the vehicle's surroundings.

[0092] Step 602: Extract visual features from visual data based on the ResNet50 model.

[0093] In this step, environmental data can be input into the ResNet50 model to obtain the visual features output by the ResNet50 model.

[0094] Step 603: In the field of view grid diagram, determine n nested square borders from the center outwards.

[0095] In this step, n nested square borders can be determined from the center outwards in the field of view grid diagram set under the preset coordinate system, resulting in n field of view regions divided from the center outwards in the field of view grid diagram.

[0096] Step 604: Based on the camera parameters, determine the real-world coordinates of the grid cells in the field of view grid map corresponding to the vehicle coordinate system coordinates, and obtain the projection mapping relationship.

[0097] In this step, for the visual features of each actual area, the coordinates of the grid cells in the field of view grid map are determined according to the camera intrinsic and extrinsic parameters, and the coordinates of the grid cells in the field of view grid map are determined in the real world in the vehicle coordinate system. The projection mapping relationship between the coordinates of the grid cells in the field of view grid map and the coordinates of the grid cells in the real world in the vehicle coordinate system is obtained.

[0098] Step 605: Based on the linear interpolation algorithm, determine the real-world coordinates of the grid cells in the BEV grid diagram in the vehicle coordinate system, and obtain the interpolation mapping relationship.

[0099] In this step, the coordinate positions of the grid cells in the BEV grid diagram are determined by linear interpolation to correspond to the real-world coordinate positions in the vehicle coordinate system, thus obtaining the interpolation mapping relationship between the coordinate positions of the grid cells in the BEV grid diagram and the real-world coordinate positions in the vehicle coordinate system.

[0100] Step 606: Based on the projection mapping relationship and the interpolation mapping relationship, determine the coordinate mapping relationship between the i-th field of view region in the field of view grid map and the i-th layer in the BEV grid map.

[0101] In this step, by combining interpolation mapping and projection mapping, and based on the real-world coordinates in the vehicle coordinate system, the coordinates of the grid cells in the BEV mesh map are correlated with the coordinates of the grid cells in the field of view mesh map, thus obtaining the coordinate mapping relationship between the i-th field of view region in the field of view mesh map and the i-th layer in the BEV mesh map.

[0102] Step 607: Based on the coordinate mapping relationship, assign the visual features to the corresponding grid cells in the BEV grid map to obtain the BEV feature map.

[0103] In this step, based on the coordinate mapping relationship, visual features can be extracted from the coordinate positions of the corresponding grid cells in the field of view grid map for each grid cell in the BEV grid map. The visual features are then assigned to the corresponding grid cells in the BEV grid map to obtain a BEV feature map with B×C×H×W feature dimensions.

[0104] By using coordinate mapping, two-dimensional visual features can be directly assigned to the BEV mesh graph, which can quickly convert two-dimensional features into BEV features of different resolutions, further reducing computational overhead and making it more suitable for in-vehicle platforms with limited computing resources.

[0105] Step 608: Perform environmental perception task based on BEV feature map.

[0106] In this step, environmental perception tasks can be performed based on the BEV feature map. For example, the BEV feature map can be input into a space occupancy prediction model, which predicts the space occupancy status of the vehicle's environment, resulting in an occupancy probability map output by the space occupancy prediction model. This facilitates subsequent support for tasks such as autonomous driving decision-making.

[0107] The dimensions of the occupancy probability map can be B×Z×H×W. For example, the dimensions of the occupancy probability map can be B=1, H=W=8, Z=6. The occupancy probability map includes multiple occupancy grids, each with a height value of 0.5m in the vertical direction, covering a height range of -1m to 2m. The grid value of each occupancy grid represents the occupancy probability (0~1) of the corresponding cube space.

[0108] Understandably, steps 602 to 606 above can be implemented using a zoom-based BEV model. This involves acquiring visual data of the vehicle's surroundings, inputting this visual data into the zoom-based BEV model, and obtaining the BEV feature map output by the model. The BEV feature map is then input into downstream target detection models, space occupancy prediction models, etc., to perform environmental perception tasks.

[0109] Figure 7 is a schematic diagram of an electronic device according to an exemplary embodiment of this application. This electronic device may be, for example, a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, personal digital assistant, server, smart home appliance, in-vehicle system, etc. Referring to Figure 7, at the hardware level, the electronic device includes a processor 701, an internal bus 702, a network interface 703, memory 704, and non-volatile memory 705, and may also include other hardware required for business operations. The processor 701 reads the corresponding computer program from the non-volatile memory 705 into the memory 704 and then runs it, forming an environmental perception device at the logical level. Of course, besides software implementation, this application does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to individual logic units, but may also be hardware or logic devices.

[0110] Figure 8 is a block diagram illustrating an environmental sensing device according to an exemplary embodiment of this application. Referring to Figure 8, the device may include: a generation module 801, a mapping module 802, and a sensing module 803, wherein:

[0111] The generation module 801 is configured to generate environmental characteristics of different actual areas in the environment where the vehicle is located based on the environmental data around the vehicle.

[0112] The mapping module 802 is configured to map the environmental features of the different actual areas to the corresponding grid cells in the BEV grid map to obtain the BEV feature map; wherein the resolution of each layer of grid cells in the BEV grid map decreases sequentially from the center outwards.

[0113] The perception module 803 is configured to perform an environmental perception task based on the BEV feature map.

[0114] In one example, the field-of-view grid map set in a preset coordinate system is divided into n field-of-view regions from the center outwards, and the area of ​​the field-of-view regions gradually increases, where n is not less than 2; the mapping module 802, when configured to map the environmental features of the different actual regions to the corresponding grid cells in the BEV grid map to obtain the BEV feature map, includes: for the environmental features of each actual region, according to the coordinate mapping relationship between the i-th field-of-view region in the field-of-view grid map and the i-th layer in the BEV grid map, mapping the environmental features to the position of the corresponding grid cell in the BEV grid map to obtain the BEV feature map.

[0115] In one example, the field of view grid diagram includes: n nested square borders extending outward from the center; wherein the side length of the outermost square border is greater than or equal to the side length of the field of view grid diagram; the n field of view regions include: for the first field of view region, the field of view region is the region of the first square; for the i-th field of view region, the i-th field of view region is the region between the i-th square and the (i-1)-th square, where i is not less than 2 and i is not greater than n.

[0116] In one example, the mapping module 802 is further configured to obtain the projection mapping relationship between the coordinate positions of the grid cells in the view grid map and the coordinate positions in the real world under the vehicle coordinate system; determine the interpolation mapping relationship between the coordinate positions of the grid cells in the BEV grid map and the coordinate positions in the real world under the vehicle coordinate system according to a linear interpolation algorithm; and determine the coordinate mapping relationship between the i-th view region in the view grid map and the i-th layer in the BEV grid map based on the projection mapping relationship and the interpolation mapping relationship.

[0117] In one example, when the mapping module 802 is configured to obtain the projection mapping relationship between the coordinate positions of the grid cells in the field of view grid map and the coordinate positions in the real world under the vehicle coordinate system, it includes: determining the projection mapping relationship based on the camera parameters of the camera used to collect the environmental data; when the mapping module 802 is configured to map the environmental features to the corresponding grid cells in the BEV grid map to obtain the BEV feature map, it includes: extracting the environmental features from the coordinate positions of the i-th field of view region in the field of view grid map, and assigning the environmental features to the corresponding grid cells in the BEV grid map to obtain the BEV feature map.

[0118] In one example, the environmental data includes visual data collected by a visual sensor; the generation module 801, when configured to generate environmental features of different actual areas in the environment where the vehicle is located, includes: extracting visual features from the visual data based on the ResNet50 model.

[0119] In one example, the environmental data includes visual data acquired by a visual sensor and / or point cloud data acquired by radar.

[0120] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0121] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0122] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, such as a memory including instructions, is also provided, which can be executed by a processor of an environmental sensing device to implement the method as described in any of the above embodiments.

[0123] The non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc., and this application does not limit it.

[0124] In an exemplary embodiment, a computer program product including a computer program / instructions is also provided, which can be executed by a processor of an environmental sensing device to implement the method described in any of the above embodiments.

[0125] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0126] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention filed herein. This application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and alterations can be made without departing from its scope. The scope of this application is limited only by the appended claims.

[0127] The above description is merely some embodiments of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An environmental perception method, comprising: Based on the environmental data surrounding the vehicle, environmental characteristics of different actual areas in the vehicle's environment are generated; The environmental features of the different actual areas are mapped to the corresponding grid cells in the BEV grid map to obtain the BEV feature map; wherein, the resolution of each layer of grid cells in the BEV grid map decreases sequentially from the center outwards. Environmental perception tasks are performed based on the BEV feature map.

2. The method according to claim 1, wherein, The field of view grid map set in the preset coordinate system is divided into n field of view regions from the center outwards, and the area of ​​the field of view regions gradually increases, and n is not less than 2. The step of mapping the environmental features of the different actual areas to the positions of corresponding grid cells in the BEV grid map to obtain the BEV feature map includes: For the environmental features of each actual area, based on the coordinate mapping relationship between the i-th field of view in the field of view grid map and the i-th layer in the BEV grid map, the environmental features of the actual area are mapped to the position of the corresponding grid cell in the BEV grid map to obtain the BEV feature map.

3. The method according to claim 2, wherein, The field of view grid diagram includes: n nested square borders from the center outwards; wherein the side length of the outermost square border is greater than or equal to the side length of the field of view grid diagram; The n visual regions include: For the first visual field region, this visual field region is the area of ​​the first square; For the i-th visual field region, the i-th visual field region is the region between the i-th square and the (i-1)-th square, where i is not less than 2 and i is not greater than n.

4. The method according to claim 2, wherein, The method further includes: Obtain the projection mapping relationship between the coordinate positions of the grid cells in the field of view grid map and the coordinate positions in the real world under the vehicle coordinate system; Based on the linear interpolation algorithm, the interpolation mapping relationship between the coordinate positions of the grid cells in the BEV grid diagram and the coordinate positions in the real world under the vehicle coordinate system is determined; Based on the projection mapping relationship and the interpolation mapping relationship, the coordinate mapping relationship between the i-th visual region in the visual field grid map and the i-th layer in the BEV grid map is determined.

5. The method according to claim 4, wherein, The step of obtaining the projection mapping relationship between the coordinate positions of the grid cells in the field of view grid map and the coordinate positions in the real world under the vehicle coordinate system includes: The projection mapping relationship is determined based on the camera parameters of the camera used to collect the environmental data; The step of mapping the environmental features of the different actual areas to corresponding grid cells in the BEV grid map to obtain the BEV feature map includes: The environmental features are extracted from the coordinates of the i-th visual region in the visual field grid map, and the environmental features are assigned to the corresponding grid cells in the BEV grid map to obtain the BEV feature map.

6. The method according to claim 1, wherein, The environmental data includes visual data collected by visual sensors; The step of generating environmental characteristics for different actual areas in the vehicle's environment based on environmental data surrounding the vehicle includes: Visual features of the visual data were extracted based on the ResNet50 model.

7. The method according to claim 1, wherein, The environmental data includes visual data collected by visual sensors and / or point cloud data collected by radar.

8. An environmental sensing device, comprising: The generation module is configured to generate environmental features of different actual areas in the environment where the vehicle is located, based on environmental data around the vehicle. The mapping module is configured to map the environmental features of the different actual areas to the corresponding grid cells in the BEV grid map to obtain the BEV feature map; wherein the resolution of each layer of grid cells in the BEV grid map decreases sequentially from the center outwards. The perception module is configured to perform environmental perception tasks based on the BEV feature map.

9. An electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor implements the method as described in any one of claims 1-7 by executing the executable instructions.

10. A computer-readable storage medium having stored thereon computer instructions that, when executed by a processor, implement the method as described in any one of claims 1-7.

11. A computer program product having a computer program / instructions stored thereon, which, when executed by a processor, implement the method as described in any one of claims 1-7.

Images