Environmental perception method, device, equipment, medium and product

Abstract

The present application provides an environmental perception method, apparatus, device, medium, and product. According to an example of the present application, the method may include: obtaining environmental data surrounding a vehicle, and generating environmental features of different actual areas in the vehicle's environment based on the environmental data; mapping the environmental features of different areas to corresponding grid cells in a BEV grid map to obtain a BEV feature map; wherein the resolution of each layer of grid cells in the BEV grid map decreases from the center outward; performing an environmental perception task based on the BEV feature map, and by mapping the environmental features to a BEV grid map with decreasing resolution from the center outward, it is possible to ensure high-precision perception of the close-range area around the vehicle while reducing the amount of computation in the long-range area, thereby achieving a balance between computing resources and perception accuracy.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to an environment perception method, device, equipment, medium and product. Background Art

[0002] With the rapid development of autonomous driving technology, environmental perception, as one of the core components of autonomous driving systems, plays a vital role in achieving safe and reliable autonomous driving. Bird's Eye View (BEV) technology has become a mainstream method in the field of environmental perception due to its ability to intuitively present environmental information.

[0003] Currently, BEV-based environmental perception methods usually use BEV feature maps with fixed resolution, which has the problem of difficult balance between computing resources and perception accuracy. Summary of the Invention

[0004] To overcome the problems existing in the related art, this specification provides an environmental perception method, device, equipment, medium and product.

[0005] According to a first aspect of any embodiment of the present application, there is provided an environment perception method, the method comprising:

[0006] Acquiring environmental data surrounding the vehicle, and generating environmental features of different actual areas in the vehicle's environment based on the environmental data;

[0007] Mapping the environmental characteristics of the different areas to corresponding grid cells in the BEV grid map to obtain a BEV feature map; wherein the resolution of each layer of grid cells in the BEV grid map decreases from the center to the outside;

[0008] An environmental perception task is performed according to the BEV characteristic map.

[0009] According to a second aspect of any embodiment of the present application, there is provided an environment perception device, the device comprising:

[0010] A generation module, configured to obtain environmental data surrounding the vehicle and generate environmental features of different actual areas in the vehicle's environment based on the environmental data;

[0011] A mapping module, configured to map the environmental features of the different regions to corresponding grid cells in a BEV grid map to obtain a BEV feature map; wherein the resolution of each grid cell layer in the BEV grid map decreases from the center outward;

[0012] A perception module is used to perform environmental perception tasks based on the BEV characteristic map.

[0013] According to a third aspect of any embodiment of the present application, an electronic device is provided, including:

[0014] processor;

[0015] a memory for storing processor-executable instructions;

[0016] The processor implements the method described in any embodiment of the present application by running the executable instructions.

[0017] According to a fourth aspect of any embodiment of the present application, a computer-readable storage medium is provided, on which computer instructions are stored. When the instructions are executed by a processor, the method described in any embodiment of the present application is implemented.

[0018] According to a fifth aspect of any embodiment of the present application, a computer program product is provided, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the method described in any embodiment of the present application is implemented.

[0019] The technical solution provided by this application may have the following beneficial effects:

[0020] According to the above embodiments, by acquiring environmental data around the vehicle and generating environmental features of different actual areas in the vehicle's environment based on the environmental data, the environmental features of different areas are respectively mapped to corresponding grid cells in the BEV grid map to obtain a BEV feature map, and the environmental perception task is performed based on the BEV feature map. By mapping the environmental features to a BEV grid map with decreasing resolution from the center to the outside, it is possible to ensure high-precision perception of the close-range area around the vehicle while reducing the amount of calculation in the long-range area, thereby achieving a balance between computing resources and perception accuracy.

[0021] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0023] Figure 1 This is a flow chart of an environment perception method according to an exemplary embodiment of the present application;

[0024] Figure 2 is a schematic diagram of a vehicle body coordinate system according to an exemplary embodiment of the present application;

[0025] Figure 3 is a schematic diagram of a visual field grid diagram according to an exemplary embodiment of the present application;

[0026] Figure 4 is a schematic diagram of a BEV grid diagram according to an exemplary embodiment of the present application;

[0027] Figure 5 is a schematic diagram illustrating a division effect of a visual field grid map according to an exemplary embodiment of the present application;

[0028] Figure 6 is a flow chart of another environment perception method according to an exemplary embodiment of the present application;

[0029] Figure 7 is a structural diagram of an electronic device according to an exemplary embodiment of the present application;

[0030] Figure 8 This is a block diagram of an environment perception device according to an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0031] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0032] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0033] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0034] BEV-based environmental perception methods convert environmental data collected by multiple cameras into a bird's-eye view to achieve a holistic perception of the surrounding environment. However, current BEV feature maps use a fixed-resolution feature representation, which makes it difficult to balance computing resources and perception accuracy.

[0035] In autonomous driving scenarios, close-range areas typically require more refined perception to ensure safety, while long-range areas require relatively lower accuracy. Fixed-resolution BEV feature maps waste computing resources in long-range areas and result in insufficient perception accuracy in close-range areas.

[0036] In order to solve the above problems, this application proposes an environment perception method. To further illustrate this application, the following embodiments are provided:

[0037] See also Figure 1 , Figure 1 This is a flowchart illustrating an environmental perception method according to an exemplary embodiment of the present application. This environmental perception method can be executed by a perception system, which can be applied to a vehicle or to a server such as a single server, a cluster server, or a cloud server. This environmental perception method can also be executed by other systems or devices in different application scenarios, and this embodiment of the present application does not limit this.

[0038] like Figure 1 As shown, the environment perception method may include the following steps:

[0039] Step 101: Acquire environmental data surrounding the vehicle, and generate environmental features of different actual areas in the vehicle's environment based on the environmental data.

[0040] In this step, the perception system can acquire environmental data around the vehicle using onboard cameras, radar, or other sensor devices. Environmental data is the raw information about the vehicle's surroundings acquired by various sensors, providing the foundation for subsequent environmental perception.

[0041] In one embodiment, environmental data may include visual data collected by visual sensors and / or point cloud data collected by radar. The visual sensors may be panoramic cameras located at various locations on the vehicle, providing a 360° field of view. The visual data is a sequence of raw images captured by the cameras and may include RGB or grayscale pixel information. For example, the visual data may be collected by six panoramic cameras located at the front, rear, left, right, left front, and right front of the vehicle, respectively.

[0042] As mentioned above, by fusing multimodal data of visual data and point cloud data, the advantages of different sensors can be complemented, the generalization ability of environmental perception in scenes with changing lighting or occlusion can be enhanced, and the accuracy of environmental perception can be improved.

[0043] Feature extraction can be performed on the environmental data surrounding the vehicle to generate environmental features for different actual regions within the vehicle's environment. Environmental features are information extracted from environmental data that describe specific properties of the vehicle's surroundings. These can be two-dimensional (2D) or three-dimensional (3D) features. Different actual regions are spatial regions that divide the vehicle's environment.

[0044] For example, a progressively expanding geometric partitioning approach can be used to delineate different actual regions within the vehicle's environment, forming an outwardly expanding nested structure. For example, a nested square region can be expanded outward from the origin of the vehicle coordinate system to form multiple concentric square regions, resulting in different actual regions, each corresponding to a different distance range.

[0045] It is understandable that other division methods such as polar coordinates and dynamic topology can also be used to divide different actual areas in the vehicle's environment, as long as the resolution of each layer of grid cells in the mapped BEV grid map decreases successively from the center to the outside. The embodiments of the present application do not limit this.

[0046] In one embodiment, the environmental data may include visual data collected by a visual sensor, and the environmental features may include visual features. Visual features are two-dimensional features extracted from the visual data and are used to capture abstract information such as target contours and textures.

[0047] Visual data collected by a visual sensor can be obtained. The visual data is input into a feature extraction model to obtain visual features output by the feature extraction model. The feature extraction model may include a ResNet50 model, which has a deep residual structure. Visual features of the visual data can be extracted based on the ResNet50 model.

[0048] As mentioned above, by extracting visual features of visual data based on the ResNet50 model and utilizing the deep residual structure of ResNet50 itself, the expression ability of complex environmental features can be enhanced and the robustness of subsequent environmental perception tasks can be improved.

[0049] Step 102: Mapping the environmental features of different actual areas to corresponding grid cells in the BEV grid map to obtain a BEV feature map; wherein the resolution of each layer of grid cells in the BEV grid map decreases from the center to the outside.

[0050] In this step, the perception system can map the environmental characteristics of different actual areas to the corresponding grid cells in the BEV grid map. Each actual area corresponds to each layer of grid cells. The environmental characteristics of each actual area can be mapped to the grid cells of the corresponding layer in the BEV grid map to obtain a BEV feature map.

[0051] Among them, the BEV grid map is a structured spatial representation centered on the vehicle, describing the surrounding environment from a bird's-eye view. It is a two-dimensional plane map composed of multiple layers of grid units, each of which corresponds to an area in the physical world.

[0052] The BEV grid map consists of multiple layers of grid cells extending outward from the center. The resolution of each layer (i.e., the physical size represented by each grid cell) decreases from the center outward. The number of different actual regions is the same as the number of grid cell layers in the BEV grid map.

[0053] Step 103: Perform an environmental perception task based on the BEV feature map.

[0054] In this step, the perception system can perform environmental perception tasks based on the BEV feature map. Environmental perception tasks can include target detection tasks, space occupancy prediction tasks, etc.

[0055] For example, the BEV feature map can be input into a target detection model to obtain a target detection result output by the target detection model; for another example, the BEV feature map can be input into a space occupancy prediction model to obtain a space occupancy prediction result output by the space occupancy prediction model, etc. The space occupancy prediction model can be a multi-layer perceptron model, and the space occupancy prediction result can include the probability of each occupied grid being occupied or unoccupied.

[0056] The environmental perception method of this embodiment obtains environmental data around the vehicle, generates environmental features of different actual areas in the vehicle's environment based on the environmental data, maps the environmental features of different areas to corresponding grid cells in the BEV grid map to obtain a BEV feature map, and performs environmental perception tasks based on the BEV feature map. By mapping the environmental features to the BEV grid map with decreasing resolution from the center to the outside, it can ensure high-precision perception of the close-range area around the vehicle while reducing the amount of calculation in the long-range area, thereby achieving a balance between computing resources and perception accuracy, and is more suitable for deployment and application in actual autonomous driving scenarios.

[0057] Moreover, through the multi-resolution BEV mapping mechanism based on the spatial representation angle, a more intuitive and stable resolution adjustment mechanism can be achieved, without the need to dynamically switch the network depth according to the resolution, thereby simplifying the system structure and improving the robustness of environmental perception.

[0058] In the aforementioned embodiments, the environmental characteristics of different actual areas are mapped to each layer of grid cells in the BEV grid map, with the resolution decreasing from the center outward. This ensures high-precision perception of nearby areas while reducing computing resource consumption for distant areas. The following embodiments provide a more detailed description of the environmental characteristics mapping process, which can be applied to any of the above embodiments.

[0059] In one embodiment, a field of view grid map can be set within a preset coordinate system. The field of view grid map is a two-dimensional grid structure established within the preset coordinate system. It is used to divide the physical space into multiple hierarchical regions. It can serve as an intermediate mapping carrier from environmental features to the BEV space, and is used to establish a spatial correspondence between environmental data and the BEV feature map. The preset coordinate system can be a world coordinate system, a vehicle coordinate system, or the like.

[0060] The field of view grid is divided into n fields of view (FOVs) from the center outward, with the area of the FOVs increasing gradually, with n being no less than 2. The number of n can be equal to the number of grid cell layers in the BEV grid. The FOV is a continuous spatial range divided from the center outward in the FOV grid. Each FOV corresponds to a different distance level and resolution requirement, allowing for progressive zoning management of the physical space.

[0061] In one embodiment, the visual field grid may include: n nested square borders extending from the center outward. The side length of the outermost square border is equal to or greater than the side length of the visual field grid, and the difference between the side lengths of two adjacent square borders increases from the center outward. If the side length of a square border is greater than the side length of the visual field grid, the corresponding area in the visual field grid that exceeds the side length may be filled with zeros and deemed invalid.

[0062] The n viewing areas may include: for the 1st viewing area, the viewing area is the area of the 1st square; for the i-th viewing area, the i-th viewing area is the area between the i-th square and the i-1-th square, i is not less than 2 and i is not greater than n.

[0063] See also Figure 2 , Figure 2 A schematic diagram of a vehicle coordinate system is shown. The origin of the vehicle coordinate system is located at the vehicle's center of mass, i.e., its geometric center. The x-axis, y-axis, and z-axis correspond to the vehicle's front-to-back, left-to-right, and vertical directions, respectively. The x-axis points in the vehicle's forward direction, the y-axis points to the driver's left, and the z-axis points perpendicular to the vehicle and upward. For example, the environmental perception range can be 10 meters in front, behind, left, and right, 2 meters in the top, and 1 meter in the bottom, centered at the vehicle coordinate system origin.

[0064] For example, the resolution of each layer of grid cells from the center outward in the BEV grid map can be determined according to the following formula 1:

[0065] Formula 1

[0066] Where d represents the side length of the i-th square border in the visual field grid map, S represents the real-world side length corresponding to the i-th grid unit in the BEV grid map, Indicates the gradual attenuation of the resolution of each layer of grid cells from the center outward.

[0067] For example, the side length of the i-th square border can be calculated according to the following formula 2:

[0068] Formula 2

[0069] in, Represents the side length of the border of the i-th square.

[0070] See also Figure 3 , Figure 3 A schematic diagram of a field of view grid is shown. A field of view grid with a side length of 20 meters can be established based on the origin of the vehicle coordinate system. The side length of each grid unit in the field of view grid is 1 meter. The field of view grid is a grid top view in the world coordinate system. The field of view grid can include four nested square borders (red square borders) from the center outward. =2 meters as an example, the side length of the first square border (i=1) is 2 meters, the side length of the second square border (i=2) is 6 meters, the side length of the third square border (i=3) is 12 meters, and the side length of the fourth square border (i=4) is 20 meters.

[0071] Using nested square borders, the visual field grid is divided from the center outward into four visual field regions (i=1, 2, 3, 4), with the area of each of the four visual field regions gradually increasing. The first visual field region is the area of the first square, the second visual field region is the area between the second and first squares, the third visual field region is the area between the third and second squares, and the fourth visual field region is the area between the fourth and third squares.

[0072] As described above, by determining n nested square borders from the center to the outside in the field of view grid diagram set under a preset coordinate system, using nested squares to divide the field of view area, and simplifying the feature space mapping logic through geometric symmetry and hierarchical progressive relationship, the computational efficiency of grid division and feature mapping can be improved.

[0073] For the environmental features of each actual area, the environmental features can be mapped to the position of the corresponding grid unit in the BEV grid map according to the coordinate mapping relationship between the i-th area in the field of view grid map and the i-th layer in the BEV grid map to obtain a BEV feature map.

[0074] Among them, the coordinate mapping relationship can describe the coordinate transformation relationship between the coordinate position of a grid cell in the field of view grid map and the coordinate position of the corresponding grid cell in the BEV grid map, which can ensure the geometric consistency of environmental features in spatial transformation and realize lossless transmission of multi-resolution features.

[0075] See also Figure 4 , Figure 4 A schematic diagram of a BEV grid diagram is shown. Exemplarily, the BEV grid diagram may include four layers of grid cells. The feature dimensions of the BEV feature in the BEV grid are B×C×H×W, where B is the number of image frames of the environmental data, C is the number of channels, W is the width of the BEV feature, and H is the height of the BEV feature. Exemplarily, Figure 4 The feature dimensions of the BEV grid map shown are B=1, C=256, W=H=10.

[0076] Please continue reading Figure 3 , the different actual areas in the vehicle environment can be divided into four actual areas, and the environmental characteristics of each actual area can be calculated according to Figure 3 The i-th region in the visual field grid diagram is Figure 4 The coordinate mapping relationship of the i-th layer in the BEV grid map shown maps the environmental features to the positions of the corresponding grid cells in the BEV grid map to obtain a BEV feature map, so that the resolution of the BEV features in the close range of the center of the field of view is higher than that in the long range.

[0077] As described above, for the environmental features of each actual area, according to the coordinate mapping relationship between the i-th area in the field of view grid map and the i-th layer in the BEV grid map, the environmental features are mapped to the positions of the corresponding grid cells in the BEV grid map to obtain a BEV feature map. By utilizing the coordinate mapping relationship, the BEV mapping of environmental features can be quickly realized without relying on complex models or algorithms, thereby further reducing the computational overhead.

[0078] See also Figure 5 , Figure 5 A schematic diagram of a grid division effect of a field of view is shown. For example, the side length of the grid unit in the i-th square border can be calculated according to the following formula 3:

[0079] Formula 3

[0080] by =1 meter as an example, in the first visual field (i=1), the side length of each grid cell is 1 meter, in the second visual field (i=2), the side length of each grid cell is 2 meters, in the third visual field (i=3), the side length of each grid cell is 3 meters, and in the fourth visual field (i=4), the side length of each grid cell is 4 meters.

[0081] Each grid cell in the first layer corresponds to a 1m×1m grid in the first field of view area in the field of view grid map, covering a smaller physical range and having a higher resolution. Each grid cell in the fourth layer corresponds to a 4m×4m grid in the fourth field of view area in the field of view grid map, covering a larger physical range and having a lower resolution.

[0082] It can be understood that the above-mentioned method of dividing the field of view range in the field of view grid diagram is only an example, and other division methods can also be used. The embodiments of the present application do not limit this.

[0083] In one embodiment, the coordinate position of the grid unit in the field of view grid map can be obtained by camera parameter conversion or other methods. The coordinate position of the real world in the vehicle coordinate system The projection mapping relationship between them.

[0084] The coordinate mapping relationship between the ith region in the visual field grid map and the ith layer in the BEV grid map can be determined based on the linear interpolation algorithm. For example, the coordinate position of the grid cell in the BEV grid map can be , calculate the real-world coordinate position of the corresponding vehicle coordinate system by linear interpolation , determine the coordinate position of the grid cell in the BEV grid map The coordinate position of the real world in the vehicle coordinate system The interpolation mapping relationship between .

[0085] Combining the interpolation mapping relationship and the projection mapping relationship, the coordinate positions of the grid cells in the BEV grid map are associated with the coordinate positions of the grid cells in the field of view grid map according to the real-world coordinate positions in the vehicle body coordinate system. For each grid cell in the BEV grid map corresponding to the real-world coordinate position in the vehicle body coordinate system, the coordinate position of the grid cell in the field of view grid map corresponding to the coordinate position can be determined, and the coordinate mapping relationship between the i-th area in the field of view grid map and the i-th layer in the BEV grid map is obtained.

[0086] As described above, by respectively obtaining the projection mapping relationship between the coordinate positions of the grid cells in the field of view grid map and the coordinate positions of the real world in the vehicle body coordinate system, and the interpolation mapping relationship between the coordinate positions of the grid cells in the BEV grid map and the coordinate positions of the real world in the vehicle body coordinate system, based on the projection mapping relationship and the interpolation mapping relationship, the coordinate mapping relationship between the i-th region in the field of view grid map and the i-th layer in the BEV grid map is determined, which can avoid the feature dislocation problem caused by coordinate discretization and improve the spatial continuity of the BEV feature map.

[0087] In one embodiment, the projection mapping relationship can be determined based on camera parameters of a camera used to collect environmental data. For example, the camera parameters may include camera intrinsic parameters and camera extrinsic parameters. Based on the camera intrinsic parameters and extrinsic parameters, the coordinate position of the grid cell in the i-th region in the field of view grid map corresponding to the real-world coordinate position in the vehicle coordinate system can be determined, thereby obtaining a projection mapping relationship between the coordinate position of the grid cell in the field of view grid map and the real-world coordinate position in the vehicle coordinate system.

[0088] According to the determined coordinate mapping relationship, environmental features can be extracted from the coordinate position of the i-th region in the field of view grid map, and the environmental features can be assigned to corresponding grid cells in the BEV grid map to obtain a BEV feature map.

[0089] The camera intrinsic parameters can describe the matrix of the camera's internal optical properties, including focal length, distortion coefficients, etc. The camera extrinsic parameters can describe the camera's position in the world coordinate system, including rotation matrices, translation vectors, etc.

[0090] For example, the coordinate position of the grid cell of the i-th region of the environmental feature in the field of view grid map can be determined according to the following formula 4:

[0091] Formula 4

[0092] in, Represents the coordinate position of the grid unit in the i-th area of the field of view grid map, K represents the camera intrinsic parameter, T represents the camera extrinsic parameter, Represents the real-world coordinate position in the vehicle coordinate system.

[0093] As described above, by determining the projection mapping relationship based on the camera parameters of the camera used to collect environmental data, extracting environmental features from the coordinate position of the i-th area in the field of view grid map, and assigning the environmental features to the corresponding grid cells in the BEV grid map to obtain a BEV feature map, and using the camera parameters to determine the coordinate position and directly assign features, the coordinate conversion error can be reduced, the calculation process can be simplified, and the reliability of the BEV feature map generation can be improved.

[0094] To further introduce the environmental perception process, Figure 6 A flow chart of another environment perception method is shown. The environment perception method may include the following steps:

[0095] Step 601: Acquire visual data around the vehicle.

[0096] In this step, the perception system can obtain visual data around the vehicle by using six panoramic cameras located in front / rear / left / right / left front / right front of the vehicle to synchronously capture images with a length and width size of 1920×1080.

[0097] Step 602: Extract visual features of visual data based on the resnet50 model.

[0098] In this step, the environmental data can be input into the resnet50 model to obtain the visual features output by the resnet50 model.

[0099] Step 603: Determine n nested square frames from the center outward in the visual field grid.

[0100] In this step, n sequentially nested square frames from the center outward may be determined in the visual field grid map set in the preset coordinate system to obtain n visual field areas sequentially divided from the center outward in the visual field grid map.

[0101] Step 604: Determine the coordinate position of the grid unit in the field of view grid map corresponding to the real-world coordinate position in the vehicle coordinate system according to the camera parameters, and obtain a projection mapping relationship.

[0102] In this step, for the visual features of each actual area, the coordinate positions of the grid cells in the field of view grid map corresponding to the real-world coordinate positions in the vehicle coordinate system are determined based on the camera intrinsic parameters and camera extrinsic parameters, and the projection mapping relationship between the coordinate positions of the grid cells in the field of view grid map and the real-world coordinate positions in the vehicle coordinate system is obtained.

[0103] Step 605: Determine the coordinate position of the grid unit in the BEV grid map corresponding to the real-world coordinate position in the vehicle body coordinate system according to the linear interpolation algorithm, and obtain an interpolation mapping relationship.

[0104] In this step, the coordinate positions of the grid cells in the BEV grid map corresponding to the real-world coordinate positions in the vehicle body coordinate system are determined by linear interpolation, and the interpolation mapping relationship between the coordinate positions of the grid cells in the BEV grid map and the real-world coordinate positions in the vehicle body coordinate system is obtained.

[0105] Step 606: Based on the projection mapping relationship and the interpolation mapping relationship, determine the coordinate mapping relationship between the i-th region in the field of view grid map and the i-th layer in the BEV grid map.

[0106] In this step, the interpolation mapping relationship and the projection mapping relationship are combined, and according to the coordinate position of the real world in the vehicle body coordinate system, the coordinate position of the grid unit in the BEV grid map is associated with the coordinate position of the grid unit in the field of view grid map, and the coordinate mapping relationship between the i-th area in the field of view grid map and the i-th layer in the BEV grid map is obtained.

[0107] Step 607: Based on the coordinate mapping relationship, the visual features are assigned to the corresponding grid cells in the BEV grid map to obtain a BEV feature map.

[0108] In this step, based on the coordinate mapping relationship, the coordinate position of each grid cell in the BEV grid map can be used to extract visual features from the coordinate position of the grid cell in its corresponding field of view grid map, and the visual features can be assigned to the corresponding grid cell in the BEV grid map to obtain a BEV feature map with a BEV feature dimension of B×C×H×W.

[0109] By directly assigning two-dimensional visual features to the BEV grid map through coordinate mapping, the two-dimensional features can be quickly converted into BEV features of different resolutions, further reducing the computational overhead and making it more suitable for vehicle platforms with limited computing resources.

[0110] Step 607: Perform an environment perception task according to the BEV characteristic map.

[0111] In this step, environmental perception tasks can be performed based on the BEV feature map. For example, the BEV feature map can be input into a space occupancy prediction model, which predicts the space occupancy state of the vehicle's environment. The resulting occupancy probability map is then output by the model to facilitate subsequent tasks such as autonomous driving decision-making.

[0112] The dimensions of the occupancy probability map can be B × Z × H × W. For example, the dimensions of the occupancy probability map can be B = 1, H = W = 8, and Z = 6. The occupancy probability map includes multiple occupancy grids, each with a height value of 0.5 meters in the height direction, covering a height range of -1 meters to 2 meters. The grid value of each occupancy grid represents the occupancy probability of the corresponding cubic space (0 to 1).

[0113] It is understood that the zoom BEV model can be used to implement steps 602 to 606 above. Visual data surrounding the vehicle can be acquired and input into the zoom BEV model to obtain a BEV feature map output by the zoom BEV model. The BEV feature map is then input into downstream object detection models, space occupancy prediction models, and other models to perform environmental perception tasks.

[0114] Figure 7 This is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present application. The electronic device may be, for example, a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a personal digital assistant, a server, a smart home appliance, a car computer, etc. Figure 7 At the hardware level, the electronic device includes a processor 701, an internal bus 702, a network interface 703, a memory 704, and a non-volatile memory 705. Of course, it may also include hardware required for other services. The processor 701 reads the corresponding computer program from the non-volatile memory 705 into the memory 704 and then runs it, forming an environmental perception device at the logical level. Of course, in addition to software implementation, this application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0115] Figure 8 This is a block diagram of an environment perception device according to an exemplary embodiment of the present application. Figure 8 The apparatus may include: a generating module 801, a mapping module 802 and a sensing module 803, wherein:

[0116] The generating module 801 is used to obtain environmental data around the vehicle and generate environmental features of different actual areas in the vehicle's environment based on the environmental data;

[0117] The mapping module 802 is configured to map the environmental features of the different actual areas to corresponding grid cells in the BEV grid map to obtain a BEV feature map; wherein the resolution of each layer of grid cells in the BEV grid map decreases from the center to the outside;

[0118] The perception module 803 is configured to perform an environmental perception task according to the BEV characteristic map.

[0119] In one example, a field of view grid map set in a preset coordinate system is divided into n field of view areas from the center outward, and the area of the field of view areas gradually increases, and n is not less than 2; the mapping module 802, when used to map the environmental characteristics of the different actual areas to the corresponding grid cells in the BEV grid map to obtain a BEV feature map, includes: for the environmental characteristics of each actual area, according to the coordinate mapping relationship between the i-th area in the field of view grid map and the i-th layer in the BEV grid map, mapping the environmental characteristics to the position of the corresponding grid cell in the BEV grid map to obtain a BEV feature map.

[0120] In one example, the field of view grid diagram includes: n square borders nested in sequence from the center outward; wherein the side length of the outermost square border is greater than or equal to the side length of the field of view grid diagram; the n field of view areas include: for the first field of view area, the field of view area is the area of the first square; for the i-th field of view area, the i-th field of view area is the area between the i-th square and the i-1-th square, i is not less than 2 and i is not greater than n.

[0121] In one example, the mapping module 802 is further used to obtain the projection mapping relationship between the coordinate position of the grid unit in the field of view grid map and the coordinate position of the real world in the vehicle body coordinate system; determine the interpolation mapping relationship between the coordinate position of the grid unit in the BEV grid map and the coordinate position of the real world in the vehicle body coordinate system according to a linear interpolation algorithm; and determine the coordinate mapping relationship between the i-th area in the field of view grid map and the i-th layer in the BEV grid map based on the projection mapping relationship and the interpolation mapping relationship.

[0122] In one example, the mapping module 802, when used to obtain the projection mapping relationship between the coordinate position of the grid cell in the field of view grid map and the coordinate position of the real world in the vehicle body coordinate system, includes: determining the projection mapping relationship based on the camera parameters of the camera used to collect the environmental data; the mapping module 802, when used to map the environmental features to the corresponding grid cells in the BEV grid map to obtain a BEV feature map, includes: extracting the environmental features from the coordinate position of the i-th area in the field of view grid map, and assigning the environmental features to the corresponding grid cells in the BEV grid map to obtain a BEV feature map.

[0123] In one example, the environmental data includes visual data collected by a visual sensor; the generation module 801, when used to generate environmental features of different actual areas in the vehicle's environment, includes: extracting visual features of the visual data based on a resnet50 model.

[0124] In one example, the environmental data includes visual data collected by a visual sensor and / or point cloud data collected by a radar.

[0125] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0126] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is merely illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present application scheme. Those of ordinary skill in the art can understand and implement it without paying any creative work.

[0127] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory including instructions, and the above instructions can be executed by a processor of the environment perception device to implement any method as described in the above embodiments.

[0128] The non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc., and this application does not limit this.

[0129] In an exemplary embodiment, a computer program product including a computer program / instruction is further provided. The computer program / instruction can be executed by a processor of an environment perception device to implement any of the methods described in the above embodiments.

[0130] The foregoing description describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0131] Other embodiments of the present invention will readily occur to those skilled in the art after consideration of the specification and practice of the invention claimed herein. The present application is not limited to the precise construction described above and illustrated in the accompanying drawings, and various modifications and variations may be made without departing from the scope thereof. The scope of the present application is limited solely by the appended claims.

[0132] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for environmental perception, characterized in that: A visual field grid diagram set in a preset coordinate system is divided into n visual field regions from the center outward, and the areas of the visual field regions gradually increase, and n is not less than 2; the method includes: Acquire environmental data surrounding the vehicle, and generate environmental features of different actual areas in the vehicle's environment based on the environmental data, wherein the environmental features include two-dimensional features; For each actual area's environmental features, extract the environmental features from the coordinate position of the i-th area in the field of view grid map based on a coordinate mapping relationship between the i-th area in the field of view grid map and the i-th layer in the BEV grid map, and assign the environmental features to corresponding grid cells in the BEV grid map to obtain a BEV feature map; wherein the resolution of each layer of grid cells in the BEV grid map decreases from the center outward, and the coordinate mapping relationship is obtained by associating the coordinate positions of the grid cells in the BEV grid map with the coordinate positions of the grid cells in the field of view grid map based on the real-world coordinate positions in the vehicle body coordinate system; Performing an environmental perception task based on the BEV characteristic map; The method further comprises: Obtaining a projection mapping relationship between the coordinate positions of the grid cells in the field of view grid map and the coordinate positions of the real world in the vehicle body coordinate system; Determining, based on a linear interpolation algorithm, an interpolation mapping relationship between coordinate positions of grid cells in the BEV grid map and coordinate positions in the real world in the vehicle body coordinate system; Based on the projection mapping relationship and the interpolation mapping relationship, a coordinate mapping relationship between the i-th region in the field of view grid map and the i-th layer in the BEV grid map is determined.

2. The method according to claim 1, characterized in that The visual field grid diagram includes: n square frames nested sequentially from the center outward; wherein the side length of the outermost square frame is greater than or equal to the side length of the visual field grid diagram; The n viewing areas include: For the first visual field area, the visual field area is the area of the first square; For the i-th visual field area, the i-th visual field area is an area between the i-th square and the i-1-th square, where i is not less than 2 and i is not greater than n.

3. The method according to claim 1, characterized in that The obtaining of a projection mapping relationship between a coordinate position of a grid cell in the field of view grid map and a coordinate position of a real world in a vehicle body coordinate system includes: The projection mapping relationship is determined according to camera parameters of a camera used to collect the environmental data.

4. The method according to claim 1, wherein The environmental data includes visual data collected by a visual sensor; The generating of environmental features of different actual areas in the vehicle environment includes: The visual features of the visual data are extracted based on the resnet50 model.

5. The method according to claim 1, wherein The environmental data includes visual data collected by a visual sensor and / or point cloud data collected by a radar.

6. An environmental sensing device, characterized in that: The visual field grid diagram set in a preset coordinate system is divided into n visual field regions from the center outward in sequence, and the areas of the visual field regions gradually increase, and n is not less than 2; the device includes: A generating module, configured to obtain environmental data surrounding the vehicle and generate environmental features of different actual areas in the vehicle's environment based on the environmental data, wherein the environmental features include two-dimensional features; a mapping module for extracting, for each actual area, environmental features from the coordinate position of the i-th area in the field of view grid map based on a coordinate mapping relationship between the i-th area in the field of view grid map and the i-th layer in the BEV grid map, and assigning the environmental features to corresponding grid cells in the BEV grid map to obtain a BEV feature map; wherein the resolution of each layer of grid cells in the BEV grid map decreases from the center outward, and the coordinate mapping relationship is obtained by associating the coordinate positions of the grid cells in the BEV grid map with the coordinate positions of the grid cells in the field of view grid map based on the real-world coordinate positions in the vehicle body coordinate system; a perception module, configured to perform an environmental perception task based on the BEV characteristic map; The mapping module is further used to obtain the projection mapping relationship between the coordinate positions of the grid cells in the field of view grid map and the coordinate positions in the real world under the vehicle body coordinate system; determine the interpolation mapping relationship between the coordinate positions of the grid cells in the BEV grid map and the coordinate positions in the real world under the vehicle body coordinate system based on the linear interpolation algorithm; and determine the coordinate mapping relationship between the i-th area in the field of view grid map and the i-th layer in the BEV grid map based on the projection mapping relationship and the interpolation mapping relationship.

7. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor implements the method according to any one of claims 1 to 5 by running the executable instructions.

8. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

9. A computer program product having a computer program / instructions stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images