Parking space detection method, device, vehicle and storage medium

Abstract

The present application relates to a parking space detection method, device, vehicle and storage medium. The method includes: extracting features from the vehicle's exterior environment images from different perspectives to obtain feature maps from different perspectives; obtaining a preset bird's-eye view grid, and constructing multiple three-dimensional space points along the height direction of the preset bird's-eye view grid; collecting multiple feature points associated with each three-dimensional space point from the feature map of the corresponding perspective; fusing multiple feature points to obtain three-dimensional feature information, and filling the three-dimensional feature information into the preset bird's-eye view grid to obtain a three-dimensional bird's-eye view; detecting parking space information in the three-dimensional bird's-eye view to plan the vehicle's parking route based on the parking space information. The solution provided by the present application can effectively extract features of objects with height, improve the accuracy of parking space detection, and improve the safety of automatic parking.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a parking space detection method, device, vehicle and storage medium. Background Art

[0002] Automatic parking is an important function of self-driving vehicles. Compared with manual parking, automatic parking can provide more accurate parking paths and safer control, thereby reducing vehicle scratches and collisions caused by manual operation errors of the driver. Parking space detection is a key part of automatic parking, which directly affects the parking accuracy of self-driving vehicles. Therefore, parking space detection plays a vital role in automatic parking.

[0003] In the related art, images are captured by multiple fisheye cameras on the vehicle, and the captured images are stitched together by a stitching algorithm to form a bird's-eye view of the vehicle's surroundings, so as to detect parking spaces based on the bird's-eye view. However, one of the premises relied on by the stitching algorithm is that all objects are above the ground. However, objects above the ground in the actual scene do not conform to the assumption. Therefore, objects with height (such as surrounding vehicles, ground locks, parking space rear gear levers, etc.) will be severely distorted after projection, which will affect the model's judgment of the parking space's parking properties to a certain extent. Summary of the invention

[0004] In order to solve or partially solve the problems existing in the related art, the present application provides a parking space detection method, device, vehicle and storage medium, which can effectively extract the features of objects with height, improve the accuracy of parking space detection, and improve the safety of automatic parking.

[0005] The first aspect of the present application provides a parking space detection method, comprising:

[0006] Feature extraction is performed on the vehicle's exterior environment images at different perspectives to obtain feature maps at different perspectives;

[0007] Acquire a preset bird's-eye view grid, and construct a plurality of three-dimensional space points along the height direction of the preset bird's-eye view grid;

[0008] Collect multiple feature points associated with each three-dimensional space point from the feature map of the corresponding perspective;

[0009] Fusion of the plurality of feature points to obtain three-dimensional feature information, and filling the three-dimensional feature information into the preset bird's-eye view grid to obtain a three-dimensional bird's-eye view;

[0010] The parking space information in the three-dimensional bird's-eye view is detected to plan a parking route for the vehicle according to the parking space information.

[0011] In one embodiment, the vehicle is equipped with a plurality of surround view cameras; the feature extraction of the vehicle exterior environment images at different perspectives of the vehicle is performed to obtain feature maps at different perspectives, including:

[0012] Acquire images of the vehicle's exterior environment from different perspectives captured by the multiple surround view cameras at the same time;

[0013] The convolutional network is used to extract features of the input vehicle exterior environment images at different perspectives to obtain feature maps at different perspectives.

[0014] In one embodiment, constructing a plurality of three-dimensional space points along the height direction of the preset bird's-eye view grid includes:

[0015] Each grid point on the preset bird's-eye view grid is used as a query vector;

[0016] At least one three-dimensional space point is constructed at different heights of corresponding grid points through each query vector to obtain multiple three-dimensional space points.

[0017] In one embodiment, collecting multiple feature points associated with each three-dimensional space point from the feature map of the corresponding viewing angle includes:

[0018] For a query vector located within the acquisition range of two surround view cameras, a plurality of feature points within the domain of each three-dimensional space point corresponding to the query vector are collected from feature maps of corresponding view angles of the two surround view cameras; or,

[0019] For a query vector located within the acquisition range of a surround-view camera, a plurality of feature points within the range of each three-dimensional space point corresponding to the query vector are acquired from a feature map of a corresponding viewing angle of the surround-view camera.

[0020] In one embodiment, the fusing of the plurality of feature points to obtain three-dimensional feature information, and filling the three-dimensional feature information into the preset bird's-eye view grid to obtain the three-dimensional bird's-eye view, comprises:

[0021] Fusing a plurality of feature points within the domain of each three-dimensional space point to obtain three-dimensional feature information of each three-dimensional space point;

[0022] Fusing the three-dimensional feature information of the multiple three-dimensional space points corresponding to the query vectors to obtain the three-dimensional feature information of the query vectors;

[0023] The three-dimensional feature information of each query vector is respectively filled into corresponding grid points on the preset bird's-eye view grid to obtain a three-dimensional bird's-eye view.

[0024] In one embodiment, the detecting the parking space information in the three-dimensional bird's-eye view to plan the parking route of the vehicle according to the parking space information includes:

[0025] Detecting a center point of a parking space in the three-dimensional bird's-eye view;

[0026] Determine each corner point of the parking space according to the center point of the parking space and the adaptively learned offset; wherein the offset is the offset of the corner point of the parking space relative to the center point of the parking space;

[0027] Determining whether the parking space is a parking space;

[0028] If the parking space is a parking space, a parking route for the vehicle is planned according to the coordinate information of each corner point.

[0029] In one embodiment, the determining whether the parking space is a parking space includes:

[0030] Based on the center point of the parking space, obtaining a target probability value of the parking space being occupied;

[0031] Determining whether the target probability value is less than a first threshold;

[0032] If the target probability value is less than a first threshold, the parking space is determined to be a parking space.

[0033] A second aspect of the present application provides a parking space detection device, comprising:

[0034] A feature extraction module is used to extract features from the vehicle's exterior environment images at different viewing angles to obtain feature maps at different viewing angles;

[0035] A three-dimensional space point construction module, used to obtain a preset bird's-eye view grid and construct a plurality of three-dimensional space points along the height direction of the preset bird's-eye view grid;

[0036] A feature point acquisition module is used to acquire multiple feature points associated with each three-dimensional space point from a feature map of a corresponding viewing angle;

[0037] A feature fusion module, used for fusing the plurality of feature points to obtain three-dimensional feature information, and filling the three-dimensional feature information into the preset bird's-eye view grid to obtain a three-dimensional bird's-eye view;

[0038] The parking space information detection module is used to detect the parking space information in the three-dimensional bird's-eye view to plan the parking route of the vehicle according to the parking space information.

[0039] A third aspect of the present application provides a vehicle, comprising:

[0040] Processor; and

[0041] The memory stores executable codes thereon, and when the executable codes are executed by the processor, the processor is caused to execute the method as described above.

[0042] A fourth aspect of the present application provides a computer-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of a vehicle, the processor is caused to execute the method as described above.

[0043] The technical solution provided by this application may have the following beneficial effects:

[0044] The solution provided by the present application performs feature extraction on the images of the vehicle's exterior environment from different perspectives to obtain feature maps from different perspectives; obtains a preset bird's-eye view grid, and constructs multiple three-dimensional space points along the height direction of the preset bird's-eye view grid; collects multiple feature points associated with each three-dimensional space point from the feature map of the corresponding perspective; fuses multiple feature points to obtain three-dimensional feature information, and fills the three-dimensional feature information into the preset bird's-eye view grid to obtain a three-dimensional bird's-eye view; detects parking space information in the three-dimensional bird's-eye view to plan the vehicle's parking route based on the parking space information. The present application constructs multiple three-dimensional space points in the height direction of the preset bird's-eye view grid, and then collects multiple feature points associated with each three-dimensional space point in the feature map of the corresponding perspective, thereby not relying on the assumption of the same plane, and can effectively extract features of objects with height, improve the accuracy of parking space detection, and improve the safety of automatic parking.

[0045] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail the exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0047] Figure 1 is a schematic diagram of a bird's-eye view formed based on a stitching algorithm in the related art;

[0048] Figure 2 is a flow chart of a parking space detection method shown in an embodiment of the present application;

[0049] Figure 3 is a schematic diagram of constructing a three-dimensional space point shown in an embodiment of the present application;

[0050] Figure 4 is a schematic diagram of a three-dimensional bird's-eye view shown in an embodiment of the present application;

[0051] Figure 5is another flow chart of the parking space detection method shown in the embodiment of the present application;

[0052] Figure 6 is a schematic diagram of an image of an external environment of a vehicle captured by a surround-view camera shown in an embodiment of the present application;

[0053] Figure 7 is a schematic diagram of feature fusion based on BEVFormer shown in an embodiment of the present application;

[0054] Figure 8 is a schematic diagram of parking space information in a three-dimensional bird's-eye view shown in an embodiment of the present application;

[0055] Fig. 9 It is a structural schematic diagram of a parking space detection device shown in the present application;

[0056] Fig.10 It is a schematic diagram of the structure of the vehicle shown in this application. DETAILED DESCRIPTION

[0057] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0058] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0059] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0060] In the related art, multiple fisheye cameras on a vehicle take images, and the images are stitched together using a stitching algorithm to form a bird's-eye view of the vehicle's surroundings, so as to detect parking spaces based on the bird's-eye view. The specific operation of the stitching algorithm is to assume that each grid point in the bird's-eye view is on the ground, and project the grid points in the bird's-eye view onto the image taken by the fisheye camera, thereby obtaining the features of the grid points in the bird's-eye view. Figure 1 As shown in the figure, the four images on the left are all taken by the fisheye camera of the vehicle, and the bird's-eye view on the right is formed by the stitching algorithm. The bird's-eye view is a two-dimensional image. It can be seen that one of the premises relied on by the stitching algorithm is to assume that all objects are above the ground. However, in the actual scene, objects above the ground do not conform to the assumption. Therefore, for objects with height (such as surrounding vehicles, ground locks, parking space rear gear levers, etc.), serious distortion will occur after projection, which will affect the model's judgment of the parking space's parking properties to a certain extent.

[0061] In response to the above problems, the present application provides a parking space detection method, which constructs multiple three-dimensional space points in the height direction of a preset bird's-eye view grid, and then collects multiple feature points associated with each three-dimensional space point in the feature map of the corresponding perspective. This method does not rely on the assumption of the same plane, and can effectively extract the features of objects with height, thereby improving the accuracy of parking space detection and the safety of automatic parking.

[0062] The technical solution of the present application is described in detail below with reference to the accompanying drawings.

[0063] Figure 2 It is a flow chart of the parking space detection method shown in the embodiment of the present application.

[0064] See also Figure 2 , a parking space detection method shown in the present application includes:

[0065] S110, extracting features from the vehicle's exterior environment images at different viewing angles to obtain feature maps at different viewing angles.

[0066] Among them, the execution subject of the parking space detection process can be an automatic parking system installed on the vehicle. When the driver turns on the automatic parking function, the automatic parking system can obtain the images of the external environment of the vehicle from different perspectives collected by the various cameras configured on the vehicle. For example, assuming that the front position, left position, right position and rear position of the vehicle are all equipped with cameras, the automatic parking system can obtain the external environment images of the front side perspective, left side perspective, right side perspective and rear side perspective of the vehicle respectively. Among them, the external environment image can be an image in RGB format.

[0067] The automatic parking system can extract parking space features around the vehicle from the external environment images at different angles, wherein the parking space features can include at least one of the corner points of the parking space, the ground lock on the parking space, other vehicles parked in the parking space, and the rear gear lever of the parking space. The automatic parking system can represent the extracted parking space features at different angles in the form of feature maps, thereby obtaining feature maps at different angles. Exemplarily, the automatic parking system can extract the parking space features of the front side perspective from the external environment images of the front side perspective, and then represent the parking space features of the front side perspective in the form of a feature map, thereby obtaining a feature map of the front side perspective. In this way, feature maps corresponding to the front side perspective, the left side perspective, the right side perspective, and the rear side perspective are obtained respectively.

[0068] S120, obtaining a preset bird's-eye view grid, and constructing a plurality of three-dimensional space points along a height direction of the preset bird's-eye view grid.

[0069] The embodiment of the present application can pre-set a bird's-eye view grid of M*N grid points, where M and N are both positive integers. For example, a bird's-eye view grid of 272*320 grid points can be pre-set to obtain a preset bird's-eye view grid, so that in actual applications, the automatic parking system can directly obtain the preset bird's-eye view grid, thereby saving the setting time of the bird's-eye view grid.

[0070] The preset bird's eye view grid is a BEV (Bird's Eye View) plane, that is, the preset bird's eye view grid is a two-dimensional bird's eye view, so the coordinate position of each grid point on the preset bird's eye view grid is expressed in the form of (x ’ ,y ’ ). In order to extract the features of objects with height, the embodiment of the present application can construct multiple three-dimensional space points (voxels) along the height direction of the preset bird's-eye view grid, where the height direction can refer to the direction perpendicular to the preset bird's-eye view grid, so the coordinate position of each three-dimensional space point is expressed in the form of (x ’ ,y ’ ,z ’ ),like Figure 3 As shown, on the preset bird's-eye view grid, the ’ ,y ’ ) are constructed in the height direction of the grid point at , and the coordinate positions of these four three-dimensional space points are expressed in the form of (x ’ ,y ’ ,z ’ ), so that the information in the height direction can be taken into account.

[0071] S130, collecting multiple feature points associated with each three-dimensional space point from the feature map of the corresponding perspective.

[0072] After constructing multiple three-dimensional space points in the height direction of the preset bird's-eye view grid, the multiple three-dimensional space points can be projected into feature maps of different perspectives respectively, so that multiple feature points associated with each three-dimensional space point can be collected in the feature maps of the corresponding perspectives, wherein the multiple feature points within the domain of the three-dimensional space point can be used to characterize the characteristics of the three-dimensional space point. Therefore, multiple feature points can be correspondingly collected within the domain of each three-dimensional space point, thereby obtaining multiple feature points associated with each three-dimensional space point.

[0073] S140, fusing multiple feature points to obtain three-dimensional feature information, and filling the three-dimensional feature information into a preset bird's-eye view grid to obtain a three-dimensional bird's-eye view.

[0074] The automatic parking system can fuse multiple feature points collected, for example, by calculating the weighted average of multiple feature points to achieve the fusion of multiple feature points, thereby obtaining three-dimensional feature information, and then the three-dimensional feature information can be filled into a preset bird's-eye view grid to obtain a three-dimensional bird's-eye view.

[0075] S150, detecting parking space information in the three-dimensional bird's-eye view to plan a parking route for the vehicle according to the parking space information.

[0076] Figure 4 It is a schematic diagram of the three-dimensional bird's-eye view shown in the embodiment of the present application. It can be seen that the three-dimensional bird's-eye view includes both features on the ground and features of objects with height, wherein the features on the ground may be, for example, the corner points of a parking space, and the features of objects with height may be, for example, the ground lock on the parking space, other vehicles parked in the parking space, the rear gear lever of the parking space, etc. Therefore, compared with the parking space detection based on the two-dimensional bird's-eye view in the related art, the parking space information obtained based on the three-dimensional bird's-eye view detection in the embodiment of the present application will be more accurate, so that automatic parking can be performed based on the parking space information, which can improve the safety of automatic parking.

[0077] In an embodiment of the present application, the parking space information may include at least one of the coordinate information of the center point of the parking space, the coordinate information of the corner points of the parking space, and the parking attribute information of the parking space. Therefore, the parking route of the vehicle can be planned according to the parking space information, so that the vehicle can automatically park in the available parking space along the parking route.

[0078] As can be seen from this example, the solution provided by the present application performs feature extraction on the images of the vehicle's exterior environment from different perspectives to obtain feature maps from different perspectives; obtains a preset bird's-eye view grid, and constructs multiple three-dimensional space points along the height direction of the preset bird's-eye view grid; collects multiple feature points associated with each three-dimensional space point from the feature map of the corresponding perspective; fuses multiple feature points to obtain three-dimensional feature information, and fills the three-dimensional feature information into the preset bird's-eye view grid to obtain a three-dimensional bird's-eye view; detects parking space information in the three-dimensional bird's-eye view to plan the vehicle's parking route based on the parking space information. The present application constructs multiple three-dimensional space points in the height direction of the preset bird's-eye view grid, and then collects multiple feature points associated with each three-dimensional space point in the feature map of the corresponding perspective, thereby not relying on the assumption of the same plane, and can effectively extract features of objects with height, improve the accuracy of parking space detection, and improve the safety of automatic parking.

[0079] Figure 5 It is another flow chart of the parking space detection method shown in the embodiment of the present application.

[0080] See also Figure 5 , the parking space detection method of the present application comprises:

[0081] S210 , extracting features from the vehicle's exterior environment images at different viewing angles to obtain feature maps at different viewing angles.

[0082] For this step, please refer to the description in S110 and will not be repeated here.

[0083] In one embodiment, the vehicle is equipped with a plurality of surround view cameras; feature extraction is performed on the vehicle's external environment images at different viewing angles to obtain feature maps at different viewing angles, which may include:

[0084] Acquire images of the vehicle's exterior environment at different perspectives collected by multiple surround-view cameras at the same time; perform feature extraction on the input vehicle's exterior environment images at different perspectives through a convolutional network to obtain feature maps at different perspectives.

[0085] Compared with ordinary cameras, the field of view of surround-view cameras is relatively large. For example, during automatic parking, when the vehicle is close to the parking space, the corners of the parking space can be clearly seen from the image of the outside environment captured by the surround-view camera, while other ordinary cameras cannot clearly see it. Therefore, the embodiment of the present application can select the surround-view camera as the device for collecting the image of the outside environment.

[0086] The vehicle can be equipped with at least four surround-view cameras, through which the vehicle's surrounding conditions can be known in 360 degrees without blind spots. For example, a surround-view camera can be configured at the front, left, right and rear positions of the vehicle, respectively, and each surround-view camera can collect the corresponding perspective of the vehicle's external environment image in real time, so the automatic parking system can obtain the vehicle's external environment images at the front, left, right and rear perspectives of the vehicle at the same time. The automatic parking system can input the vehicle's external environment images at different angles into the convolutional network, so that the convolutional network can extract features from the input vehicle's external environment images at different perspectives, thereby obtaining feature maps of different perspectives.

[0087] In one example, if Figure 6 As shown, the surround view camera at the front side of the vehicle can obtain the image of the outside environment at the front side of the vehicle, and the surround view camera at the left side of the vehicle can obtain the image of the outside environment at the left side of the vehicle, and the surround view camera at the right side of the vehicle can obtain the image of the outside environment at the right side of the vehicle, and the surround view camera at the rear side of the vehicle can obtain the image of the outside environment at the rear side of the vehicle. The automatic parking system can input the images of the outside environment at the front side, left side, right side and rear side into the convolution network respectively, and the convolution network can extract features of the input images of the outside environment at the front side, left side, right side and rear side respectively, so as to output feature maps of the front side, left side, right side and rear side. It can be seen that the images of the outside environment at the front side, left side, right side and rear side can comprehensively reflect the situation around the vehicle, so the feature maps of the front side, left side, right side and rear side output by the convolution network can comprehensively cover the parking space features around the vehicle.

[0088] Among them, the feature map and the vehicle exterior environment image can have the same size, for example, the size of the feature map and the vehicle exterior environment image can be set to H*W, where H is the image height and W is the image width.

[0089] S220, obtaining a preset bird's-eye view grid, and using each grid point on the preset bird's-eye view grid as a query vector, so as to construct at least one three-dimensional space point at different heights of the corresponding grid point through each query vector, so as to obtain multiple three-dimensional space points.

[0090] The automatic parking system can fuse the features extracted by the convolutional network into the bird's-eye view through BEVFormer (bird's-eye view model). Among them, BEVFormer is a BEV encoder based on Transformer (a model that combines attention mechanism and temporal information), so BEVForme can aggregate temporal information and spatial information to realize the conversion of multi-view images to BEV views.

[0091] In practical applications, the automatic parking system can obtain a preset bird's-eye view grid, which is a BEV plane, that is, the preset bird's-eye view grid is a two-dimensional bird's-eye view, so the coordinate position of each grid point on the preset bird's-eye view grid is expressed in the form of (x ’ ,y ’ ). If for (x ’ ,y ’ ) is a vehicle, and its corresponding features should be the convergence of features at the bottom, body, and roof of the vehicle. However, the heights corresponding to these positions are inconsistent. In order to better obtain the (x ’ ,y ’ ) at the feature, the embodiment of the present application can use BEVFormer to convert each (x ’ ,y ’ ) coordinates to perform a lift operation to obtain multiple three-dimensional space points (voxels) on the z-axis, that is, by taking each grid point on the preset bird's-eye view grid as a query vector (Query), so that at least one three-dimensional space point (voxel) is constructed at different heights of the corresponding grid point through each query vector, wherein at least one three-dimensional space point may include one or two three-dimensional space points, thereby obtaining multiple three-dimensional space points. Let different heights be d0~d n , where n is a positive integer, and the distance between each two adjacent heights is equal, for example, the distance between d0 and d1 is equal to the distance between d1 and d2. Figure 7 As shown, located at (x ’ ,y ’ ) constructs three-dimensional space points at the heights of the corresponding grid points d0, d1, d2, and d3. The coordinate position of each three-dimensional space point is represented by (x ’ ,y ’ ,z ’ ), so that the information in the height direction can be taken into account.

[0092] Among them, different heights (d0~d n) can be divided according to the height of the parking space features. For example, the height of the corner point of the parking space is generally 0m, the ground lock on the parking space is generally 0.3m, the height of the rear gear lever of the parking space is generally 1.2m, and the height of other vehicles is generally 1.5m. Therefore, the heights of 0m, 0.5m, 1m and 1.5m can be divided along the height direction of the preset bird's-eye view grid. It should be noted that the embodiment of the present application can also be divided into more or fewer heights, and the embodiment of the present application is not limited to this.

[0093] Among them, the number of three-dimensional space points constructed by different query vectors can be determined according to the position of the query vector on the preset bird's-eye view grid. For example, if the position of a query vector on the preset bird's-eye view grid belongs to the acquisition range of two surround-view cameras, it means that the query vector can be projected into the feature map of the corresponding perspectives of the two surround-view cameras respectively, so the query vector can construct two three-dimensional space points at different heights of the corresponding grid points; if the position of a query vector on the preset bird's-eye view grid belongs to the acquisition range of one surround-view camera, it means that the query vector can only be projected into the feature map of the corresponding perspective of this one surround-view camera, so the query vector can construct a three-dimensional space point at different heights of the corresponding grid points.

[0094] In one example, suppose that i ’ ,y i ’ ) on the preset bird's-eye view grid belongs to the acquisition range of the front surround view camera and the left surround view camera, which means that the query vector i can be projected into the feature maps of the front view and the left view respectively. Therefore, the query vector i can construct two three-dimensional space points at different heights of the corresponding grid points. For example, the query vector i constructs two three-dimensional space points at the height d0 and also constructs two three-dimensional space points at the height d1.

[0095] In another example, suppose that j ’ ,y j ’ ) on the preset bird's-eye view grid belongs to the acquisition range of the front surround view camera, which means that the query vector j can only be projected into the feature map of the front view. Therefore, the query vector j can construct a three-dimensional space point at different heights of the corresponding grid points. For example, the query vector j constructs a three-dimensional space point at the height d0 and also constructs a three-dimensional space point at the height d1.

[0096] After each query vector is constructed to obtain the corresponding multiple three-dimensional space points, it can be determined to enter step S230 or step S240 according to the position of the query vector on the preset bird's-eye view grid. Exemplarily, if the position of a query vector on the preset bird's-eye view grid belongs to the acquisition range of two surround-view cameras, it is determined to enter step S230 for the query vector; if the position of a query vector on the preset bird's-eye view grid belongs to the acquisition range of one surround-view camera, it is determined to enter step S240 for the query vector.

[0097] S230, for the query vector located within the acquisition range of the two surround view cameras, a plurality of feature points within the range of each three-dimensional space point corresponding to the query vector are respectively acquired from the feature maps of the corresponding view angles of the two surround view cameras.

[0098] BEVFormer can include Deformable AttentionModule, which is a module that transforms the global attention of Transformer into local attention. Global attention means that each query vector collects feature points from feature maps of all perspectives. However, each query vector is generally projected to feature maps of certain perspectives / certain perspectives. Therefore, DeformableAttentionModule can be used to transform global attention into local attention. Local attention means that each query vector only collects feature points from feature maps that can be projected, thereby improving the efficiency of feature point collection.

[0099] In one implementation, for query vectors located within the acquisition range of the two surround view cameras, they can be projected into the feature maps of the corresponding viewpoints of the two surround view cameras, respectively. Therefore, the DeformableAttentionModule can be used to collect multiple feature points within the domain of each 3D space point corresponding to the query vector from the feature maps of the corresponding viewpoints of the two surround view cameras, respectively. Exemplarily, for query vector i located within the acquisition range of the front surround view camera and the left surround view camera, they can be projected into the feature maps of the front view and the left view, respectively. Therefore, the Deformable AttentionModule can be used to collect multiple feature points within the domain of each 3D space point corresponding to the query vector i from the feature map of the front view, and collect multiple feature points within the domain of each 3D space point corresponding to the query vector i from the feature map of the left view. Each 3D space point can correspond to 4 feature points. For example, assuming that the query vector i constructs two 3D space points at height d0: voxel i1 and voxel i2, and construct two 3D points at height d1: voxel i3 and voxel i4 If voxel i1 and voxel i3 can be projected into the feature map of the front view, then the Deformable AttentionModule can be used to collect voxel from the feature map of the front view i1 4 feature points within the scope of the field, and the collection voxel i3 Similarly, if voxel i2 and voxel i4 can be projected into the feature map of the left view, then the DeformableAttentionModule can be used to collect voxel from the feature map of the left view i2 4 feature points within the scope of the field, and the collection voxel i4 4 feature points within the scope of the field.

[0100] S240, for a query vector located within the acquisition range of a surround view camera, a plurality of feature points within the range of each three-dimensional space point corresponding to the query vector are acquired from a feature map of a corresponding viewing angle of a surround view camera.

[0101] In another implementation, for a query vector located within the acquisition range of a surround-view camera, it can only be projected into the feature map of the corresponding perspective of this surround-view camera. Therefore, the DeformableAttentionModule can be used to collect multiple feature points within the domain of each 3D space point corresponding to the query vector from the feature map of the corresponding perspective of this surround-view camera. Exemplarily, for a query vector j located within the acquisition range of the front surround-view camera, it can only be projected into the feature map of the front perspective. Therefore, the DeformableAttentionModule can be used to collect multiple feature points within the domain of each 3D space point corresponding to the query vector j from the feature map of the front perspective. Each 3D space point can correspond to 4 feature points. For example, assuming that the query vector j constructs a 3D space point at height d0: voxel j1 , and construct a three-dimensional space point at height d1: voxel j2 , due to voxel j1 and voxel j2 can be projected into the feature map of the front view, then the DeformableAttentionModule can be used to collect voxel from the feature map of the front view j14 feature points within the scope of the field, and the collection voxel j2 4 feature points within the scope of the field.

[0102] It can be seen that compared with the stitching algorithm used in the related art, which is highly dependent on the accuracy of the extrinsic parameter calibration of the fisheye camera, inaccurate extrinsic parameter calibration leads to misalignment of the stitched image, which directly affects the accuracy of parking corner point detection. The embodiment of the present application uses the Deformable Attention Module to adaptively collect the four most representative feature points around each voxel (three-dimensional space point). Therefore, the embodiment of the present application does not rely on the accuracy of the extrinsic parameter calibration, and can improve the accuracy of parking space detection, thereby improving the safety of automatic parking.

[0103] S250, fusing multiple feature points to obtain three-dimensional feature information, and filling the three-dimensional feature information into a preset bird's-eye view grid to obtain a three-dimensional bird's-eye view.

[0104] For this step, please refer to the description in S140 and will not be repeated here.

[0105] In one embodiment, fusing multiple feature points to obtain three-dimensional feature information, and filling the three-dimensional feature information into a preset bird's-eye view grid to obtain a three-dimensional bird's-eye view may include:

[0106] Multiple feature points within the domain of each three-dimensional space point are fused to obtain three-dimensional feature information of each three-dimensional space point; the three-dimensional feature information of multiple three-dimensional space points corresponding to each query vector is fused to obtain three-dimensional feature information of each query vector; the three-dimensional feature information of each query vector is filled into the corresponding grid points on the preset bird's-eye view grid to obtain a three-dimensional bird's-eye view.

[0107] Among them, each 3D space point can correspond to multiple feature points. Therefore, after obtaining multiple feature points within the domain of each 3D space point, the multiple feature points within the domain of each 3D space point can be adaptively fused respectively. For example, the weighted average of the multiple feature points within the domain of each 3D space point can be calculated to achieve the fusion of the multiple feature points within each domain, thereby obtaining the 3D feature information of each 3D space point respectively. The 3D feature information of the 3D space point can be used as the final feature of the 3D space point. For example, assuming that voxel i1 The four feature points within the domain are feature i11 、feature i12 、feature i13 and feature i14 , then by calculating the feature i11 、feature i12、feature i13 and feature i14 The weighted average of voxel i1 3D feature information.

[0108] Each query vector may correspond to multiple 3D space points. Therefore, after obtaining the 3D feature information of each 3D space point, the 3D feature information of the multiple 3D space points corresponding to each query vector may be adaptively fused. For example, the weighted average of the 3D feature information of the multiple 3D space points corresponding to each query vector may be calculated to achieve the fusion of the 3D feature information of the corresponding multiple 3D space points, thereby obtaining the 3D feature information of each query vector. The 3D feature information of the query vector may be used as the final feature of the query vector. For example, assuming that the query vector i corresponds to voxel i1 、voxel i2 、voxel i3 and voxel i4 , then by calculating voxel i1 、voxel i2 、voxel i3 and voxel i4 The weighted average of the three-dimensional feature information of can be used to obtain the three-dimensional feature information of the query vector i.

[0109] Each grid point on the preset bird's-eye view grid can be used as a query vector. Therefore, after obtaining the three-dimensional feature information of each query vector, the three-dimensional feature information of each query vector can be filled into the corresponding grid point on the preset bird's-eye view grid, thereby obtaining a three-dimensional bird's-eye view. For example, assuming that the grid point (x i ’ ,y i ’ ) as the query vector i, and the grid point at (x j ’ ,y j ’ ) as the query vector j, then the three-dimensional feature information of the query vector i can be filled into the grid point at (x i ’ ,y i ’ ), and the three-dimensional feature information of the query vector j can be filled into the grid point located at (x j ’ ,y j ’ ) at the grid points, traversing the entire preset bird's-eye view grid can obtain the three-dimensional feature information of all grid points, thereby obtaining a three-dimensional bird's-eye view.

[0110] It can be seen that the direct puzzle in the related art is a bird's-eye view grid point that collects a feature point on the ground, while the BEV Former provided in the embodiment of the present application is a bird's-eye view grid point that collects a large number of feature points at different heights and within a certain range. Therefore, the embodiment of the present application can ensure more sufficient feature points and feature fusion, so that BEV Former can achieve better results than direct puzzle.

[0111] It should be noted that the weight corresponding to the fusion of multiple feature points and the weight corresponding to the fusion of three-dimensional feature information of multiple three-dimensional space points may be the same or different, and the embodiments of the present application do not limit this.

[0112] S260, detecting parking space information in the three-dimensional bird's-eye view to plan a parking route for the vehicle according to the parking space information.

[0113] Figure 8 It is a schematic diagram of parking space information in a three-dimensional bird's-eye view shown in an embodiment of the present application. It can be seen that the three-dimensional bird's-eye view includes both features on the ground and features of objects with height, wherein the features on the ground may be, for example, corner points of parking spaces, and the features of objects with height may be, for example, ground locks on parking spaces, other vehicles parked in parking spaces, rear gear levers of parking spaces, etc. Therefore, compared with the related art in which parking space detection is performed based on a two-dimensional bird's-eye view, the parking space information obtained by the embodiment of the present application based on the three-dimensional bird's-eye view detection will be more accurate, so that automatic parking can be performed based on the parking space information, which can improve the safety of automatic parking.

[0114] In an embodiment of the present application, the parking space information may include at least one of the coordinate information of the center point of the parking space, the coordinate information of the corner points of the parking space, and the parking attribute information of the parking space. Therefore, the parking route of the vehicle can be planned according to the parking space information, so that the vehicle can automatically park in the available parking space along the parking route.

[0115] It should be noted that Figure 4 and Figure 8 All are three-dimensional bird's-eye views of the embodiments of the present application, wherein: Figure 8 Is Figure 6 The image of the vehicle's exterior environment is formed through a series of steps in the embodiment of the present application.

[0116] In one embodiment, detecting parking space information in a three-dimensional bird's-eye view to plan a parking route for a vehicle according to the parking space information may include:

[0117] Detect the center point of the parking space in the three-dimensional bird's-eye view; determine the corner points of the parking space based on the center point of the parking space and the offset of adaptive learning; wherein the offset is the offset of the corner point of the parking space relative to the center point of the parking space; determine whether the parking space is a parkable space; if the parking space is a parkable space, plan a parking route for the vehicle based on the coordinate information of each corner point.

[0118] When training BEV Former, the number of network channels of the three-dimensional bird's-eye view can be configured as C, where C ≥ 0 and C is an integer. Assuming that C is configured as 10, the three-dimensional bird's-eye view contains the 0th to 9th network channels, and then the role of each network channel can be configured separately. Exemplarily, the 0th network channel can be configured to detect the center point of the parking space in the three-dimensional bird's-eye view, the 1st to 8th network channels can be configured to adaptively learn the offset (offset) of each corner point relative to the center point, and the 9th network channel can be configured to determine the parking attribute of the parking space. Among them, for the 1st to 8th network channels, the 1st to 2nd network channels can be configured as the offset of the first corner point of the adaptive learning parking space relative to the center point of the parking space, and the 3rd to 4th network channels can be configured as the offset of the second corner point of the adaptive learning parking space relative to the center point of the parking space, and the 5th to 6th network channels can be configured as the offset of the third corner point of the adaptive learning parking space relative to the center point of the parking space, and the 7th to 8th network channels can be configured as the offset of the fourth corner point of the adaptive learning parking space relative to the center point of the parking space.

[0119] In practical applications, the three-dimensional bird's-eye view can be set to the 0th network channel first, wherein the three-dimensional bird's-eye view of the 0th network channel is composed of probability values ​​(value) of 0 to 1. Since the 0th network channel is used to detect the center point of the parking space in the three-dimensional bird's-eye view, the three-dimensional bird's-eye view of the 0th network channel only contains information about the center point of the parking space. For the 0th network channel, the embodiment of the present application can traverse each probability value of the three-dimensional bird's-eye view of the 0th network channel to detect a probability value greater than the second threshold, and then determine the coordinate position of the probability value as the coordinate position of the center point of the parking space in the three-dimensional bird's-eye view. Among them, the probability value of the center point of the parking space can reach above 0.8, so the second threshold can be set to 0.8.

[0120] like Figure 8 As shown, point O is the center point of the parking space. Assuming that points A to D are the first to fourth corner points of the parking space, then the coordinate position (x O ’ ,y O ’), the three-dimensional bird's-eye view can be set to be in the first network channel. In the embodiment of the present application, the coordinate position (x O ’ ,y O ’ ), the first network channel can output the offset offset1 of the adaptively learned point A (the first corner point of the parking space) relative to point O in the X direction, and then according to the coordinate position of point O (x O ’ ,y O ’ ) and the adaptively learned offset offset1 to determine the x coordinate of point A. Then, the 3D bird's-eye view can be set to the second network channel. In the embodiment of the present application, the coordinate position (x O ’ ,y O ’ ), the second network channel can output the adaptively learned offset offset2 of point A relative to point O in the Y direction, and then according to the coordinate position of point O (x O ’ ,y O ’ ) and the adaptively learned offset offset2 to determine the y coordinate of point A, so that the coordinate position of point A (x A ’ ,y A ’ ), that is, the coordinate position of the first corner point of the parking space can be obtained. In this way, the coordinate positions corresponding to the first corner point (point A), the second corner point (point B), the third corner point (point C), and the fourth corner point (point D) of the parking space are obtained in sequence.

[0121] After obtaining the center point and four corner points of the parking space, the embodiment of the present application can determine whether the parking space is a parking space. If the parking space is a parking space, the parking route of the vehicle can be planned according to the coordinate position of each corner point, so that the vehicle can automatically park in the parking space along the parking route. Among them, when planning the parking route, since the three-dimensional bird's-eye view is already an image in the world coordinate system, the embodiment of the present application only needs to convert the image origin to the center point of the rear wheel axle of the vehicle. For example, the image origin is generally located in the upper left corner of the three-dimensional bird's-eye view, and the image origin in the upper left corner of the three-dimensional bird's-eye view can be translated to the center point of the rear wheel axle of the vehicle.

[0122] It can be seen that compared with the parking space detection algorithm in the related art, which usually adopts artificially designed geometric rules for post-processing to realize parking space detection, such algorithms can only predict relatively regular parking spaces and require more complex post-processing, which greatly limits the robustness and real-time performance of parking space detection. In this regard, the embodiment of the present application uses BEV Former to adaptively learn the offset of each corner point of the parking space relative to the center point, which can reduce the dependence on artificially designed geometric rules, thereby detecting more types of parking spaces and improving the robustness of parking space detection. In addition, the embodiment of the present application does not require complex post-processing, thereby improving the real-time performance of parking space detection.

[0123] In one implementation, determining whether a parking space is a parking space may include:

[0124] Based on the center point of the parking space, a target probability value of the parking space being occupied is obtained; and it is determined whether the target probability value is less than a first threshold value; if the target probability value is less than the first threshold value, the parking space is determined to be a parking space.

[0125] The ninth network channel can be used to determine the parking properties of the parking space, so the three-dimensional bird's-eye view can be set in the ninth network channel. In the embodiment of the present application, the coordinate position (x ) of the center point (point O) of the parking space can be input into the ninth network channel. O ’ ,y O ’ ), the 9th network channel is based on the coordinate position of the center point of the parking space (x O ’ ,y O ’ ) Output the corresponding value ’ , the value ’ It refers to the target probability value of the parking space being occupied. In this embodiment of the application, the target probability value (value ’ ) is compared with a first threshold. If the target probability value is greater than or equal to the first threshold, it means that the parking space has been occupied, so the parking space can be determined as a non-parking space; if the target probability value is less than the first threshold, it means that the parking space is not occupied, so the parking space can be determined as a parking space.

[0126] It can be seen from this example that the solution provided by the present application constructs multiple three-dimensional space points at different heights of each bird's-eye view grid point, and then collects multiple feature points within the field of each three-dimensional space point in the feature map of the corresponding perspective. It does not rely on the assumption of the same plane, and can effectively extract the features of objects with height. At the same time, it also considers the features within the field of the three-dimensional space point, thereby greatly improving the accuracy of parking space detection and improving the safety of automatic parking.

[0127] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides a parking space detection device, a vehicle, a computer-readable storage medium and corresponding embodiments.

[0128] Fig. 9 It is a schematic diagram of the structure of the parking space detection device shown in an embodiment of the present application.

[0129] See also Fig. 9 The present application provides a parking space detection device, which may include a feature extraction module 910, a three-dimensional space point construction module 920, a feature point acquisition module 930, a feature fusion module 940 and a parking space information detection module 950; wherein:

[0130] The feature extraction module 910 is used to extract features from the vehicle's exterior environment images at different viewing angles to obtain feature maps at different viewing angles.

[0131] The three-dimensional space point construction module 920 is used to obtain a preset bird's-eye view grid and construct a plurality of three-dimensional space points along the height direction of the preset bird's-eye view grid.

[0132] The feature point acquisition module 930 is used to acquire multiple feature points associated with each three-dimensional space point from the feature map of the corresponding viewing angle.

[0133] The feature fusion module 940 is used to fuse multiple feature points to obtain three-dimensional feature information, and fill the three-dimensional feature information into a preset bird's-eye view grid to obtain a three-dimensional bird's-eye view.

[0134] The parking space information detection module 950 is used to detect the parking space information in the three-dimensional bird's-eye view to plan the parking route of the vehicle according to the parking space information.

[0135] In one embodiment, the vehicle is equipped with multiple surround view cameras; the feature extraction module 910 may include an external environment image acquisition submodule and a feature extraction submodule; wherein:

[0136] The vehicle exterior environment image acquisition submodule is used to acquire vehicle exterior environment images of different perspectives collected by multiple surround view cameras at the same time.

[0137] The feature extraction submodule is used to extract features from the input vehicle exterior environment images of different perspectives through a convolutional network to obtain feature maps of different perspectives.

[0138] In one embodiment, the three-dimensional space point construction module 920 may include a query vector definition submodule and a three-dimensional space point construction submodule; wherein:

[0139] The query vector definition submodule is used to use each grid point on the preset bird's-eye view grid as a query vector.

[0140] The three-dimensional space point construction submodule is used to construct at least one three-dimensional space point at different heights of the corresponding grid points through each query vector to obtain multiple three-dimensional space points.

[0141] In one embodiment, the feature point acquisition module 930 may include a feature point acquisition submodule; wherein:

[0142] The feature point acquisition submodule is used to collect multiple feature points within the range of each three-dimensional space point corresponding to the query vector from the feature maps of the corresponding perspectives of the two surround-view cameras for the query vector located within the acquisition range of the two surround-view cameras; or, for the query vector located within the acquisition range of one surround-view camera, collect multiple feature points within the range of each three-dimensional space point corresponding to the query vector from the feature map of the corresponding perspective of one surround-view camera.

[0143] In one implementation, the feature fusion module 940 may include a first fusion submodule, a second fusion submodule, and a filling submodule; wherein:

[0144] The first fusion submodule is used to fuse multiple feature points within the domain of each three-dimensional space point to obtain three-dimensional feature information of each three-dimensional space point.

[0145] The second fusion submodule is used to fuse the three-dimensional feature information of multiple three-dimensional space points corresponding to each query vector to obtain the three-dimensional feature information of each query vector.

[0146] The filling submodule is used to fill the three-dimensional feature information of each query vector into the corresponding grid points on the preset bird's-eye view grid to obtain a three-dimensional bird's-eye view.

[0147] In one embodiment, the parking space information detection module 950 may include a center point detection submodule, a corner point determination submodule, a parking attribute judgment submodule, and a parking route planning submodule; wherein:

[0148] The center point detection submodule is used to detect the center point of the parking space in the three-dimensional bird's-eye view;

[0149] The corner point determination submodule is used to determine each corner point of the parking space according to the center point of the parking space and the offset learned adaptively; wherein the offset is the offset of the corner point of the parking space relative to the center point of the parking space.

[0150] The parking attribute judgment submodule is used to judge whether a parking space is a parking space.

[0151] The parking route planning submodule is used to plan the parking route of the vehicle according to the coordinate information of each corner point if the parking space is a parking space.

[0152] In one embodiment, the parking attribute determination submodule may include a target probability value acquisition unit, a target probability value determination unit, and a parking space determination unit; wherein:

[0153] The target probability value determination unit is used to obtain a target probability value of the parking space being occupied based on the center point of the parking space.

[0154] The target probability value judgment unit is used to judge whether the target probability value is less than a first threshold.

[0155] The parking space available determination unit is configured to determine that the parking space is a parking space available if the target probability value is less than a first threshold value.

[0156] As can be seen from this example, the solution provided by the present application performs feature extraction on the images of the vehicle's exterior environment from different perspectives to obtain feature maps from different perspectives; obtains a preset bird's-eye view grid, and constructs multiple three-dimensional space points along the height direction of the preset bird's-eye view grid; collects multiple feature points associated with each three-dimensional space point from the feature map of the corresponding perspective; fuses multiple feature points to obtain three-dimensional feature information, and fills the three-dimensional feature information into the preset bird's-eye view grid to obtain a three-dimensional bird's-eye view; detects parking space information in the three-dimensional bird's-eye view to plan the vehicle's parking route based on the parking space information. The present application constructs multiple three-dimensional space points in the height direction of the preset bird's-eye view grid, and then collects multiple feature points associated with each three-dimensional space point in the feature map of the corresponding perspective, thereby not relying on the assumption of the same plane, and can effectively extract features of objects with height, improve the accuracy of parking space detection, and improve the safety of automatic parking.

[0157] Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment of the method, and will not be elaborated again here.

[0158] Fig.10 It is a schematic diagram of the structure of a vehicle shown in an embodiment of the present application.

[0159] See also Fig.10 , vehicle 1000 includes a memory 1010 and a processor 1020 .

[0160] The processor 1020 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0161] The memory 1010 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, ROM can store static data or instructions required by the processor 1020 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 1010 may include any combination of computer-readable storage media, including various types of semiconductor storage chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 1010 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (such as a DVD-ROM, a double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a mini SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0162] The memory 1010 stores executable codes, and when the executable codes are processed by the processor 1020 , the processor 1020 can execute part or all of the methods described above.

[0163] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0164] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-temporary machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by the vehicle's processor, the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0165] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A parking space detection method, characterized in that: include: Feature extraction is performed on the vehicle's exterior environment images at different perspectives to obtain feature maps at different perspectives; Acquire a preset bird's-eye view grid, and construct a plurality of three-dimensional space points along the height direction of the preset bird's-eye view grid; Collect multiple feature points associated with each three-dimensional space point from the feature map of the corresponding perspective; Fusion of the plurality of feature points to obtain three-dimensional feature information, and filling the three-dimensional feature information into the preset bird's-eye view grid to obtain a three-dimensional bird's-eye view; Detecting parking space information in the three-dimensional bird's-eye view to plan a parking route for the vehicle according to the parking space information; The vehicle is equipped with a plurality of surround-view cameras, each surround-view camera is used to collect an image of the vehicle's external environment at a corresponding viewing angle; The collecting of multiple feature points associated with each three-dimensional space point from the feature map of the corresponding perspective includes: For a query vector located within the acquisition range of two surround view cameras, a plurality of feature points within the domain of each three-dimensional space point corresponding to the query vector are collected from feature maps of corresponding view angles of the two surround view cameras; or, For a query vector located within a collection range of a surround view camera, a plurality of feature points within a range of each three-dimensional space point corresponding to the query vector are collected from a feature map of a corresponding viewing angle of the surround view camera; Wherein, each query vector is a grid point on the preset bird's-eye view grid; The fusing the multiple feature points to obtain three-dimensional feature information includes: A plurality of feature points within the domain of each three-dimensional space point corresponding to each query vector are fused to obtain three-dimensional feature information of each query vector.

2. The method according to claim 1, characterized in that The feature extraction of the vehicle exterior environment images at different perspectives of the vehicle is performed to obtain feature maps at different perspectives, including: Acquire images of the vehicle's exterior environment from different perspectives captured by the multiple surround view cameras at the same time; The features of the input vehicle exterior environment images at different perspectives are extracted respectively through a convolutional network to obtain feature maps at different perspectives.

3. The method according to claim 2, characterized in that The step of constructing a plurality of three-dimensional space points along the height direction of the preset bird's-eye view grid includes: Each grid point on the preset bird's-eye view grid is used as a query vector; At least one three-dimensional space point is constructed at different heights of corresponding grid points through each query vector to obtain multiple three-dimensional space points.

4. The method according to claim 1, characterized in that: The step of fusing a plurality of feature points within the domain of each three-dimensional space point corresponding to each query vector to obtain three-dimensional feature information of each query vector includes: Fusing a plurality of feature points within the domain of each three-dimensional space point to obtain three-dimensional feature information of each three-dimensional space point; Fusing the three-dimensional feature information of the multiple three-dimensional space points corresponding to the query vectors to obtain the three-dimensional feature information of the query vectors; The step of filling the three-dimensional feature information into the preset bird's-eye view grid to obtain a three-dimensional bird's-eye view includes: The three-dimensional feature information of each query vector is respectively filled into corresponding grid points on the preset bird's-eye view grid to obtain a three-dimensional bird's-eye view.

5. The method according to claim 4, characterized in that The fusing of multiple feature points within the domain of each three-dimensional space point to obtain three-dimensional feature information of each three-dimensional space point includes: Calculating the weighted average of multiple feature points within the domain of each three-dimensional space point to obtain three-dimensional feature information of each three-dimensional space point; The fusing the three-dimensional feature information of the multiple three-dimensional space points corresponding to the respective query vectors to obtain the three-dimensional feature information of the respective query vectors includes: The weighted average of the three-dimensional feature information of the multiple three-dimensional space points corresponding to each query vector is calculated respectively to obtain the three-dimensional feature information of each query vector.

6. The method according to claim 1, characterized in that The detecting the parking space information in the three-dimensional bird's-eye view to plan the parking route of the vehicle according to the parking space information includes: Detecting a center point of a parking space in the three-dimensional bird's-eye view; Determine each corner point of the parking space according to the center point of the parking space and the adaptively learned offset; wherein the offset is the offset of the corner point of the parking space relative to the center point of the parking space; Determining whether the parking space is a parking space; If the parking space is a parking space, a parking route for the vehicle is planned according to the coordinate information of each corner point.

7. The method according to claim 6, characterized in that The determining whether the parking space is a parking space includes: Based on the center point of the parking space, obtaining a target probability value of the parking space being occupied; Determining whether the target probability value is less than a first threshold; If the target probability value is less than a first threshold, the parking space is determined to be a parking space.

8. A parking space detection device, characterized in that: include: A feature extraction module is used to extract features from the vehicle's exterior environment images at different viewing angles to obtain feature maps at different viewing angles; A three-dimensional space point construction module, used to obtain a preset bird's-eye view grid and construct a plurality of three-dimensional space points along the height direction of the preset bird's-eye view grid; A feature point acquisition module is used to acquire multiple feature points associated with each three-dimensional space point from a feature map of a corresponding viewing angle; A feature fusion module, used for fusing the plurality of feature points to obtain three-dimensional feature information, and filling the three-dimensional feature information into the preset bird's-eye view grid to obtain a three-dimensional bird's-eye view; A parking space information detection module, used to detect the parking space information in the three-dimensional bird's-eye view, so as to plan the parking route of the vehicle according to the parking space information; The vehicle is equipped with a plurality of surround-view cameras, each surround-view camera is used to collect an image of the vehicle's external environment at a corresponding viewing angle; The feature point acquisition module comprises: A feature point acquisition submodule is used to acquire, for a query vector located within the acquisition range of two surround view cameras, a plurality of feature points within the domain of each three-dimensional space point corresponding to the query vector from the feature maps of the corresponding view angles of the two surround view cameras; or, for a query vector located within the acquisition range of one surround view camera, a plurality of feature points within the domain of each three-dimensional space point corresponding to the query vector from the feature map of the corresponding view angle of one surround view camera; Wherein, each query vector is a grid point on the preset bird's-eye view grid; The feature fusion module is used to fuse multiple feature points within the domain of each three-dimensional space point corresponding to each query vector to obtain three-dimensional feature information of each query vector.

9. A vehicle, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having executable codes stored thereon, which, when executed by a processor of a vehicle, causes the processor to execute the method according to any one of claims 1 to 7.

Images