Parking environment sensing method and apparatus in absence of parking space lines, and vehicle and computer-readable storage medium

Abstract

Disclosed in the present application are a parking environment sensing method and apparatus in the absence of parking space lines, and a vehicle and a computer-readable storage medium. The method comprises: acquiring a first surround-view image corresponding to each surround-view camera of a first vehicle; determining a drivable area boundary corresponding to each first surround-view image; and projecting each drivable area boundary as a bird's eye view, so as to obtain a drivable area. In the present application, when roadside parking in the absence of parking space lines is performed, a vehicle performs environment sensing by means of drivable area boundaries corresponding to a plurality of surround-view images, so as to accurately obtain a drivable area; compared with environment sensing performed by means of LiDAR, the present application can effectively reduce the costs and also reduce the amount of computation for point cloud data, and thus can improve efficiency and accuracy of detection for the drivable area; and when roadside parking in the absence of parking space lines is performed, parking assistance is provided by means of the drivable area, such that the accuracy and efficiency of parking of the vehicle are improved.

Description

Parking environment perception method and device without parking space line, vehicle and computer readable storage medium

[0001] The present application claims priority to the Chinese patent application No. 2024104265698, filed on April 9, 2024, and entitled "Parking environment perception method and device without parking space line, vehicle and computer readable storage medium", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the field of automobile technology, in particular to a parking environment perception method and device without parking space line, vehicle and computer readable storage medium. BACKGROUND

[0003] With the continuous growth of the number of motor vehicles, the parking lot at peak hours is not enough to meet the parking demand, and sometimes vehicles need to be temporarily parked on the roadside. When there is no obvious parking space on the roadside, or the parking line is blurred, the driver often chooses the position of the parked vehicle to park the car.

[0004] Currently, in the case of parking space line, the target position of the car is determined by recognizing the parking space line, so as to plan the trajectory. In the case of no parking space line, the positions of the front and rear vehicles and the roadside need to be accurately delineated to confirm the drivable area and the target parking position, and then the trajectory is planned. When identifying the drivable area in the case of no parking space line, the point cloud data in the surrounding area of the vehicle is generally obtained, the category of each point in the point cloud data is judged, and then the drivable area in the surrounding area is determined according to the category judgment result of each point. This method needs to install expensive laser radar sensors on the vehicle to collect point cloud data; at the same time, the calculation amount of the category judgment of each point in the point cloud data is also very large, so the calculation efficiency of the drivable area in the surrounding area of the vehicle is also poor. SUMMARY

[0005] To solve or partially solve the problems in the related art, the present application provides a parking environment perception method and device without parking space line, vehicle and computer readable storage medium, which can improve the detection efficiency of the drivable area.

[0006] To achieve the above-mentioned purpose, the present application provides a parking environment perception method without parking space line, comprising:

[0007] obtaining a first surround view image corresponding to each surround camera of a first vehicle;

[0008] determining a drivable area boundary corresponding to each first surround view image;

[0009] projecting each drivable area boundary to a bird's eye view angle respectively to obtain a drivable area.

[0010] In an embodiment, the step of determining the drivable area boundary corresponding to each first surround view image comprises:

[0011] respectively inputting each first surround view image into a deep neural network, and determining a column boundary point of each column of pixels in each first surround view image through the deep neural network;

[0012] respectively determining a boundary line of the drivable area boundary corresponding to each first surround view image based on the column boundary point.

[0013] In an embodiment, after the step of obtaining the first surround view image corresponding to each surround view camera, the method further comprises:

[0014] inputting each first surround view image into a target bird's eye view network model for model prediction to obtain first occupancy grid data corresponding to the drivable area.

[0015] In an embodiment, before the step of inputting each first surround view image into a target bird's eye view network model for model prediction to obtain first occupancy grid data, the method further comprises:

[0016] obtaining second surround view images corresponding to a plurality of surround view cameras of a second vehicle;

[0017] respectively performing annotation operations on the second surround view images to obtain first occupancy grid annotation information;

[0018] inputting the first occupancy grid annotation information and the second surround view images into a bird's eye view network model to be trained for model training to obtain the target bird's eye view network model.

[0019] In an embodiment, the step of respectively performing annotation operations on the second surround view images to obtain first occupancy grid annotation information comprises:

[0020] obtaining first point cloud data corresponding to a laser radar of the second vehicle;

[0021] performing gridding processing on a three-dimensional space corresponding to the point cloud data to obtain first gridding data;

[0022] determining occupancy grid information corresponding to the first point cloud data in the first gridding data based on the first point cloud data;

[0023] determining the first occupancy grid annotation information based on the occupancy grid information corresponding to the first point cloud data.

[0024] In addition, to achieve the above-mentioned purpose, the application also provides a parking environment perception method without parking space lines, comprising:

[0025] obtain third surround view images corresponding to each surround view camera of the first vehicle;

[0026] input each third surround view image into a target bird's eye view network model for model prediction to obtain second occupancy grid data corresponding to the drivable area.

[0027] In an embodiment, before the step of inputting each third surround view image into a target bird's eye view network model for model prediction to obtain second occupancy grid data corresponding to the drivable area, the method further comprises:

[0028] obtain fourth surround view images corresponding to a plurality of surround view cameras of a second vehicle;

[0029] respectively perform annotation operations on the fourth surround view images to obtain second occupancy grid annotation information;

[0030] input the second occupancy grid annotation information and the fourth surround view images into a bird's eye view network model to be trained for model training to obtain the target bird's eye view network model.

[0031] In addition, to achieve the above object, the application further provides a vehicle, which comprises:

[0032] an acquisition module configured to acquire first surround view images corresponding to each surround view camera of the vehicle;

[0033] a determination module configured to determine a drivable area boundary corresponding to each first surround view image;

[0034] a projection module configured to respectively project each drivable area boundary to a bird's eye view angle to obtain a drivable area.

[0035] In an embodiment, the determination module is configured to input each first surround view image into a deep neural network to determine a column boundary point of each column of pixels in each first surround view image through the deep neural network.

[0036] based on the column boundary point, respectively determine a boundary line of the drivable area boundary corresponding to each first surround view image.

[0037] In an embodiment, the acquisition module is further configured to input each first surround view image into a target bird's eye view network model for model prediction to obtain first occupancy grid data corresponding to the drivable area.

[0038] In addition, to achieve the above object, the application further provides a parking environment sensing device without parking lines, which comprises a memory, a processor and a parking environment sensing program without parking lines stored in the memory and capable of running on the processor, and the parking environment sensing program without parking lines implements the steps of the above parking environment sensing method without parking lines when executed by the processor.

[0039] In addition, to achieve the above object, the application further provides a computer readable storage medium, which stores a parking environment sensing program without parking lines, and the parking environment sensing program without parking lines implements the steps of the above parking environment sensing method without parking lines when executed by a processor.

[0040] In addition, to achieve the above object, the application further provides a computer program product, which comprises a parking environment sensing program without parking lines, and the parking environment sensing program without parking lines implements the steps of the above parking environment sensing method without parking lines when executed by a processor.

[0041] The application obtains first surround view images corresponding to each surround camera of a first vehicle, then determines drivable area boundaries corresponding to each first surround view image, and then projects each drivable area boundary to a bird's eye view angle to obtain a drivable area. When parking on a roadside without parking lines, the vehicle can accurately obtain the drivable area through the environment sensing of the drivable area boundaries corresponding to the surround view images. Compared with the environment sensing by a laser radar, the application can effectively reduce the cost and the calculation amount of point cloud data, and thus can improve the efficiency and accuracy of drivable area detection, and the drivable area can assist parking when parking on a roadside without parking lines, so as to improve the accuracy and efficiency of vehicle parking.

[0042] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and are not limiting to the application. BRIEF DESCRIPTION OF DRAWINGS

[0043] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings, in which like reference characters refer to like parts throughout the figures, and in which:

[0044] Fig. 1 is a structural schematic diagram of a parking environment sensing device without parking lines in a hardware running environment related to an embodiment of the application;

[0045] Fig. 2 is a flow schematic diagram of a first embodiment of a parking environment sensing method without parking lines of the application;

[0046] Fig. 3 is a schematic diagram of one possibility of a drivable area in an embodiment of the present application;

[0047] Fig. 4 is a schematic diagram of one possibility of a boundary point in an embodiment of the present application;

[0048] Fig. 5 is a flowchart of a fourth embodiment of a parking environment perception method without parking space lines in the present application;

[0049] Fig. 6 is a functional module diagram of an embodiment of a vehicle in the present application. DETAILED DESCRIPTION

[0050] Embodiments of the present application will be described in more detail by referring to the drawings. Although embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided so that the present application is more thoroughly and completely conveyed to those skilled in the art, and the scope of the present application is fully conveyed to those skilled in the art.

[0051] The terminology used in the present application is merely for the purpose of describing specific embodiments and is not intended to limit the present application. The singular forms "a", "an", and "the" used in the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0052] It should be understood that although the terms "first", "second", "third", etc. can be used in the present application to describe various information, these information should not be limited by these terms. These terms are only used to distinguish the same type of information from each other. For example, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information without departing from the scope of the present application. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0053] As shown in Fig. 1, Fig. 1 is a structural diagram of a parking environment perception device without parking space lines in a hardware operating environment related to an embodiment of the present application.

[0054] The parking environment perception device without parking space lines in the embodiments of the present application can be a vehicle. As shown in FIG. 1, the parking environment perception device without parking space lines can include a processor 1001 such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to realize the connection and communication between the components. The user interface 1003 can include a display screen (Display) and an input unit such as a keyboard (Keyboard). Optionally, the user interface 1003 can also include a standard wired interface and a wireless interface. The network interface 1004 can optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 can be a high-speed RAM memory or a stable memory (non-volatile memory) such as a disk memory. Optionally, the memory 1005 can also be a storage device independent of the aforementioned processor 1001.

[0055] Optionally, the parking environment perception device without parking space lines can also include a camera, an RF (Radio Frequency, radio frequency) circuit, a sensor, an audio circuit, a WiFi module, and the like. The sensor can be, for example, a light sensor, a motion sensor, and other sensors. Of course, the parking environment perception device without parking space lines can also be configured with a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and other sensors, which will not be described here.

[0056] Those skilled in the art can understand that the terminal structure shown in FIG. 1 does not constitute a limitation on the parking environment perception device without parking space lines, and can include more or fewer components than those shown, or combine certain components, or different component arrangements.

[0057] As shown in FIG. 1, the memory 1005 as a computer storage medium can include an operating system, a network communication module, a user interface module, and a parking environment perception program without parking space lines.

[0058] In the parking environment perception device without parking space lines shown in FIG. 1, the network interface 1004 is mainly used to connect to a background server and communicate data with the background server; the user interface 1003 is mainly used to connect to a client (user end) and communicate data with the client; and the processor 1001 can be used to call the parking environment perception program without parking space lines stored in the memory 1005.

[0059] In the embodiment, the parking environment perception device without parking lines comprises a memory 1005, a processor 1001, and a parking environment perception program without parking lines stored in the memory 1005 and capable of running on the processor 1001. When the processor 1001 invokes the parking environment perception program without parking lines stored in the memory 1005, the steps of the parking environment perception method without parking lines in each of the embodiments are performed.

[0060] The application also provides a parking environment perception method without parking lines. Referring to FIG. 2, FIG. 2 is a flowchart of a first embodiment of the parking environment perception method without parking lines.

[0061] The first vehicle is provided with a plurality of surround-view cameras, wherein the surround-view cameras can comprise fisheye cameras, and the number of the surround-view cameras can be reasonably set. The 360° environment around the first vehicle can also be imaged by the plurality of surround-view cameras. For example, the surround-view cameras can be set to four, and are respectively arranged at the left and right sides of the front end of the first vehicle and the left and right sides of the rear end of the first vehicle.

[0062] The parking environment perception method without parking lines comprises the following steps.

[0063] In step S101, a first surround-view image corresponding to each surround-view camera of the first vehicle is acquired.

[0064] In step S102, a drivable area boundary corresponding to each first surround-view image is determined.

[0065] In step S103, each drivable area boundary is projected to a bird's-eye view angle to obtain a drivable area.

[0066] When the vehicle is parked on the roadside, for example, when the vehicle is parked on the roadside without parking lines, a first surround-view image is acquired by each surround-view camera of the first vehicle. After the first surround-view image is acquired, a drivable area boundary corresponding to each first surround-view image is determined. In an embodiment, the drivable area boundary corresponding to each first surround-view image can be determined by a trained deep neural network, that is, each first surround-view image is input into the deep neural network for prediction, and the drivable area boundary in each first surround-view image is determined by the deep neural network. For example, each column boundary point in each column of pixels in the first surround-view image is predicted by the deep neural network, and the connecting line of each column boundary point is taken as the drivable area boundary of the corresponding surround-view image, wherein one side of the drivable area boundary is a drivable area, and the other side is a non-drivable area (for example, a parking area of other vehicles or an outer side of a road, etc.).

[0067] After obtaining the drivable area boundaries corresponding to each first surround view image, each drivable area boundary is projected to a bird's eye view (BEV) respectively to obtain each drivable area boundary in the bird's eye view, and a region surrounded by each drivable area boundary in the bird's eye view is taken as a drivable area, that is, the drivable area is a region surrounded by each drivable area boundary in the bird's eye view. Referring to FIG. 3, FIG. 3 is a schematic diagram of a drivable area in an embodiment of the present application. The drivable area is a closed region surrounded by each drivable area boundary in the bird's eye view. If the region surrounded by each drivable area boundary in the bird's eye view is a non-closed region, each drivable area boundary in the bird's eye view can be extended so that the region surrounded by each drivable area boundary in the bird's eye view after extension is a closed region, and the drivable area is obtained.

[0068] Further, in a possible implementation, after step S101, the parking environment perception method without parking space can further include:

[0069] In step S104, each first surround view image is input into a target bird's eye network model for model prediction to obtain first occupancy grid data corresponding to the drivable area.

[0070] It should be noted that, before the prediction of the drivable area, model training can be performed to obtain a target bird's eye (BEV) network model, and the target bird's eye network model is loaded to the first vehicle.

[0071] After obtaining the first surround view image, the first surround view image is input into the target bird's eye network model for model prediction to obtain first occupancy grid data corresponding to the drivable area, wherein the first occupancy grid data is grid information occupied by the drivable area in a parking area corresponding to the first vehicle.

[0072] In an embodiment, a convolutional neural network in the target bird's eye network model is used to extract features of each input (first surround view image), and then a transformer in the target bird's eye network model is used to convert the features from a camera view to a BEV view to obtain a bird's eye view, and the target bird's eye network model performs prediction through the bird's eye view to obtain the first occupancy grid data.

[0073] In the roadside parking without parking lines, the vehicle obtains the drivable area and the corresponding occupancy grid data through the environment perception of the multiple surround view images. Compared with the environment perception through the laser radar, the application can effectively reduce the cost and the calculation amount of the point cloud data, and thus can improve the efficiency and the accuracy of the drivable area detection. In the roadside parking without parking lines, the drivable area can be displayed to assist the driver to park or assist the autonomous vehicle to park, and thus the drivable area can improve the accuracy and the efficiency of the vehicle parking.

[0074] The first surround view image corresponding to each surround view camera of the first vehicle is obtained, the drivable area boundary corresponding to each first surround view image is determined, each drivable area boundary is projected to the bird's eye view, and the drivable area is obtained. In the roadside parking without parking lines, the vehicle accurately obtains the drivable area through the environment perception of the drivable area boundary corresponding to the multiple surround view images. Compared with the environment perception through the laser radar, the application can effectively reduce the cost and the calculation amount of the point cloud data, and thus can improve the efficiency and the accuracy of the drivable area detection. In the roadside parking without parking lines, the drivable area can be used to assist the parking, and thus the accuracy and the efficiency of the vehicle parking can be improved.

[0075] Based on the first embodiment, the second embodiment of the parking environment perception method without parking lines is provided, wherein the step S102 comprises:

[0076] In step S201, each first surround view image is input into a deep neural network, and the column boundary point of each column pixel in each first surround view image is determined through the deep neural network.

[0077] In step S202, based on the column boundary point, the boundary line of the drivable area boundary corresponding to each first surround view image is determined.

[0078] After obtaining each first surround view image, each first surround view image is input into a deep neural network, and the column boundary point of each column pixel in each first surround view image is determined through the deep neural network. As shown in FIG. 4, one side of the column boundary point is the drivable area and the other side is the non-drivable area.

[0079] After the column boundary points of each column of pixels in each first surround view image are obtained, the boundary points corresponding to each first surround view image are obtained based on the column boundary points. In an embodiment, for each first surround view image, the set of all column boundary points of the first surround view image is the boundary point of the first surround view image. The boundary line of the drivable area boundary corresponding to each first surround view image is determined based on the boundary points. In an embodiment, for each first surround view image, the column boundary points of each column of pixels in the first surround view image are sequentially connected, and the connecting line is taken as the boundary line of the drivable area boundary corresponding to the first surround view image.

[0080] wherein the deep neural network is a pre-trained neural network. When training the deep neural network, the boundary point labels of the column boundary points of each column of pixels in each training image can be obtained, each training image is input into the initial deep neural network for model training to obtain the predicted column boundary points of each column of pixels in each training image, the accuracy of the predicted column boundary points is determined according to the predicted column boundary points and the corresponding boundary point labels, when the accuracy is greater than a preset accuracy, the trained initial deep neural network is taken as the deep neural network, otherwise the trained initial deep neural network is taken as the initial deep neural network, and the initial deep neural network is trained again according to the training images until the accuracy is greater than the preset accuracy, or the training of the initial deep neural network reaches a preset number of times, the trained initial deep neural network is taken as the deep neural network, and the trained deep neural network is loaded into the first vehicle.

[0081] By inputting each first surround view image into the deep neural network respectively, the column boundary points of each column of pixels in each first surround view image are determined by the deep neural network, and then the boundary line of the drivable area boundary corresponding to each first surround view image is determined based on the column boundary points. The column boundary points in each first surround view image can be accurately obtained by the deep neural network, and then the boundary line of the drivable area boundary can be accurately obtained according to the column boundary points, so as to obtain the drivable area through the boundary line, and further improve the efficiency and accuracy of drivable area detection. The accuracy and efficiency of vehicle parking are improved by assisting parking in the drivable area in the roadside parking without a parking line.

[0082] Based on the first embodiment, a third embodiment of the parking environment perception method without a parking line is provided, wherein before step S104, the parking environment perception method without a parking line further comprises:

[0083] In step S301, a plurality of surround view images corresponding to the second vehicle are obtained.

[0084] In step S302, the second surround view image is labeled respectively to obtain first occupancy grid label information.

[0085] In step S303, the first occupancy grid label information and the second surround view image are input into a bird's eye network model to be trained to perform model training, and the target bird's eye network model is obtained.

[0086] The second vehicle can be a data collection vehicle.

[0087] In the model training of the target bird's eye network model, the second surround view image corresponding to the plurality of surround view cameras of the second vehicle is obtained, and the second surround view image can be images at multiple time points to obtain multiple groups of surround view images for model training, and each group of images includes surround view images captured by the plurality of surround view cameras at the same time.

[0088] After obtaining the second surround view image, each second surround view image is labeled respectively to obtain first occupancy grid label information, and the first occupancy grid label information is a label of an occupancy grid of a drivable area in a parking area corresponding to the second vehicle.

[0089] In step S3021, the first point cloud data corresponding to the laser radar of the second vehicle is obtained.

[0090] In step S3022, the three-dimensional space corresponding to the point cloud data is gridded to obtain first grid data.

[0091] In step S3023, based on the first point cloud data, the occupancy grid information corresponding to the first point cloud data in the first grid data is determined.

[0092] In step S3024, based on the occupancy grid information corresponding to the first point cloud data, the first occupancy grid label information is determined.

[0093] The second vehicle is provided with a laser radar, and after obtaining the second surround view image, the first point cloud data corresponding to the laser radar of the second vehicle is obtained, and the first point cloud data can be collection data of the laser radar at multiple time points, and the time of each collection data in the first point cloud data is the same as the shooting time of the image in the second surround view image.

[0094] After the first point cloud data is acquired, the three-dimensional space corresponding to the first point cloud data is gridded to obtain first grid data. For example, the three-dimensional space corresponding to the first point cloud data is a three-dimensional space of 20m*20m*5m, and the three-dimensional space corresponding to the first point cloud data is gridded according to a preset resolution (such as 0.1m) to obtain the first grid data. Then, the occupancy grid information corresponding to the first point cloud data is determined in the first grid data based on the first point cloud data, that is, the grid with the point cloud in the first point cloud data in the first grid data is taken as the occupancy grid in the occupancy grid information, and of course the intensity (number of point cloud data) of each occupancy grid can also be counted.

[0095] After the occupancy grid information is acquired, the first occupancy grid labeling information is determined based on the occupancy grid information corresponding to the first point cloud data. Specifically, the occupancy grid information corresponding to the first point cloud data can be converted into the first occupancy grid labeling information according to the relationship between the point cloud data and the second surround view image.

[0096] After the first occupancy grid labeling information is acquired, the first occupancy grid labeling information and the second surround view image are input into a to-be-trained bird's eye view network model for model training to obtain the target bird's eye view network model. In an embodiment, the model is trained according to each group of images in the second surround view image corresponding to the plurality of groups of images, the loss function of the trained model is calculated through the prediction result of each group of images and the first occupancy grid labeling information, and when the loss function is less than a preset loss function, the to-be-trained bird's eye view network model after the current training is taken as the target bird's eye view network model, otherwise, the to-be-trained bird's eye view network model after the current training is taken as the to-be-trained bird's eye view network model, and the step S303 is returned.

[0097] It should be noted that the training process of the bird's eye view network model can be applied to a server in communication connection with the first vehicle. After the target bird's eye view network model is obtained, the server sends the target bird's eye view network model to the first vehicle, and the first vehicle loads the target bird's eye view network model.

[0098] The second surround view image corresponding to the plurality of surround view cameras of the second vehicle is acquired, then the first occupancy grid labeling information is obtained by performing labeling operation on the second surround view image respectively, and then the first occupancy grid labeling information and the second surround view image are input into a to-be-trained bird's eye view network model for model training to obtain the target bird's eye view network model. The target bird's eye view network model is obtained by model training through the second surround view image and the labeling data, which improves the accuracy of the target bird's eye view network model, and further improves the accuracy and precision of the drivable area detection through the target bird's eye view network model, and further improves the accuracy of the roadside parking without parking lines.

[0099] The application also provides a parking environment perception method without parking lines. Referring to FIG. 5, FIG. 5 is a flowchart of a fourth embodiment of the parking environment perception method without parking lines according to the application.

[0100] The first vehicle is provided with multiple surround-view cameras, wherein the surround-view cameras can include fisheye cameras, and the number of surround-view cameras can be reasonably set. The 360° environment around the first vehicle can also be imaged by the multiple surround-view cameras. For example, the surround-view cameras can be set to 4, and are respectively arranged on the left and right sides of the front end of the first vehicle and the left and right sides of the rear end of the first vehicle.

[0101] The parking environment perception method without parking lines includes:

[0102] In step S401, third surround-view images corresponding to each surround-view camera of the first vehicle are obtained.

[0103] In step S402, each third surround-view image is input into a target bird's eye view network model for model prediction to obtain second occupancy grid data corresponding to a drivable area.

[0104] It should be noted that before predicting the drivable area, the target bird's eye view (BEV) network model can be obtained by model training, and the target bird's eye view network model is loaded into the first vehicle.

[0105] When the vehicle is performing roadside parking, for example, roadside parking without parking lines, the third surround-view image is obtained by each surround-view camera of the first vehicle. After obtaining the third surround-view image, the third surround-view image is input into the target bird's eye view network model for model prediction to obtain second occupancy grid data corresponding to a drivable area, wherein the second occupancy grid data is the grid information occupied by the drivable area in the parking area corresponding to the first vehicle.

[0106] In an embodiment, a convolutional neural network in the target bird's eye view network model is used to extract features from each input (second surround-view image), and then a transformer in the target bird's eye view network model is used to convert the features from a camera perspective to a BEV perspective to obtain a bird's eye view. The target bird's eye view network model predicts the second occupancy grid data through the bird's eye view.

[0107] The third bird's-eye view image corresponding to each surround view camera of the first vehicle is acquired; then each third bird's-eye view image is input into a target bird's-eye network model for model prediction to obtain second occupancy grid data corresponding to a drivable area. When roadside parking without a parking line is performed, the second occupancy grid data of the drivable area can be obtained through model prediction of multiple surround view images. Compared with environment perception through a laser radar, the application can effectively reduce the cost and the amount of calculation of point cloud data, thereby improving the efficiency and accuracy of drivable area detection, and the occupancy grid data of the drivable area is used to assist parking when roadside parking without a parking line is performed to improve the accuracy and efficiency of vehicle parking.

[0108] Based on the first embodiment, a fifth embodiment of the parking environment perception method without a parking line is provided, wherein before step S402, the parking environment perception method without a parking line can further include:

[0109] In step S501, fourth surround view images corresponding to multiple surround view cameras of a second vehicle are acquired.

[0110] In step S502, a second occupancy grid annotation information is obtained by performing an annotation operation on each fourth surround view image.

[0111] In step S503, the second occupancy grid annotation information and the fourth surround view image are input into a bird's-eye network model to be trained for model training to obtain the target bird's-eye network model.

[0112] The arrangement mode of each surround view camera in the second vehicle can be the same as that of each surround view camera in the first vehicle, and the second vehicle can be a data collection vehicle.

[0113] When the model training of the target bird's-eye network model is performed, the fourth surround view images corresponding to the multiple surround view cameras of the second vehicle are acquired, wherein the fourth surround view images can be images at multiple time points to obtain multiple groups of surround view images for model training, and each group of images includes surround view images captured by multiple surround view cameras at the same time point.

[0114] After the fourth surround view images are acquired, an annotation operation is performed on each fourth surround view image to obtain second occupancy grid annotation information, which is an annotation of the occupied grid of the drivable area in the parking area corresponding to the second vehicle. In a possible implementation, step S502 includes:

[0115] In step S5021, second point cloud data corresponding to a laser radar of the third vehicle is acquired.

[0116] In step S5022, occupancy grid information corresponding to the second point cloud data is acquired.

[0117] In step S5023, the second occupancy grid label information is determined based on the occupancy grid information corresponding to the second point cloud data.

[0118] The second vehicle is provided with a laser radar, and after the fourth surround view image is acquired, second point cloud data corresponding to the laser radar of the second vehicle is acquired. The second point cloud data can be acquisition data of the laser radar at multiple time points, and the time point of each acquisition data in the second point cloud data is the same as the shooting time point of the image in the fourth surround view image.

[0119] After the second point cloud data is acquired, the occupancy grid information corresponding to the second point cloud data is acquired. In a possible implementation, the step S5022 includes:

[0120] In step d, the three-dimensional space corresponding to the second point cloud data is gridded to obtain second grid data.

[0121] In step e, the occupancy grid information corresponding to the second point cloud data is determined in the second grid data based on the second point cloud data.

[0122] After the second point cloud data is acquired, the three-dimensional space corresponding to the second point cloud data is gridded to obtain second grid data. For example, the three-dimensional space corresponding to the second point cloud data is a three-dimensional space of 20m*20m*5m, and the three-dimensional space corresponding to the second point cloud data is gridded according to a preset resolution (such as 0.1m) to obtain the second grid data. Then, the occupancy grid information corresponding to the second point cloud data is determined in the second grid data based on the second point cloud data, that is, the grid in the second grid data that exists in the point cloud in the second point cloud data is taken as the occupancy grid in the occupancy grid information. Of course, the intensity (number of point cloud data) of each occupancy grid can also be counted.

[0123] After the occupancy grid information is acquired, the second occupancy grid label information is determined based on the occupancy grid information corresponding to the second point cloud data. The occupancy grid information corresponding to the second point cloud data can be converted into the second occupancy grid label information according to the relationship between the point cloud data and the fourth surround view image.

[0124] After the second occupancy grid label information is acquired, the second occupancy grid label information and the third surround view image are input into the to-be-trained bird's eye view network model for model training to obtain the target bird's eye view network model. In an embodiment, model training is performed according to each group of images corresponding to the third surround view image, a loss function of the trained model is calculated through a prediction result of each group of images and the second occupancy grid label information, when the loss function is less than a preset loss function, the to-be-trained bird's eye view network model after current training is taken as the target bird's eye view network model, otherwise, the to-be-trained bird's eye view network model after current training is taken as the to-be-trained bird's eye view network model, and the step S503 is executed again.

[0125] The fourth surround view image corresponding to the plurality of surround view cameras of the second vehicle is acquired, then the fourth surround view image is labeled to obtain the second occupancy grid label information, and then the second occupancy grid label information and the fourth surround view image are input into the to-be-trained bird's eye view network model for model training to obtain the target bird's eye view network model. The target bird's eye view network model is obtained through model training based on the fourth surround view image and the labeled data, the accuracy of the target bird's eye view network model is improved, and the accuracy and precision of the drivable area detection based on the target bird's eye view network model are improved, and the accuracy of the roadside parking without a parking line is further improved.

[0126] In addition, the application also provides a vehicle. Referring to FIG. 6, the vehicle comprises:

[0127] The acquisition module 10 is configured to acquire the first surround view image corresponding to each surround view camera of the vehicle.

[0128] The determination module 20 is configured to determine the drivable area boundary corresponding to each first surround view image.

[0129] The projection module 30 is configured to project each drivable area boundary to a bird's eye view angle to obtain a drivable area.

[0130] The determination module 20 is configured to input each first surround view image into a deep neural network, and determine the column boundary point of each column of pixels in each first surround view image through the deep neural network.

[0131] Based on the column boundary point, the boundary line of the drivable area boundary corresponding to each first surround view image is determined.

[0132] The acquisition module 10 is further configured to input each first surround view image into the target bird's eye view network model for model prediction to obtain the first occupancy grid data corresponding to the drivable area.

[0133] The method performed by each of the above program units can refer to each embodiment of the no-parking-space-line parking environment perception method of the present application, and will not be described here again.

[0134] In addition, the embodiments of the present application also propose a computer readable storage medium, the computer readable storage medium stores a no-parking-space-line parking environment perception program, and the no-parking-space-line parking environment perception program implements the steps of the no-parking-space-line parking environment perception method described above when executed by a processor.

[0135] In addition, the embodiments of the present application also propose a computer program product, the computer program product includes a no-parking-space-line parking environment perception program, and the no-parking-space-line parking environment perception program implements the steps of the no-parking-space-line parking environment perception method described above when executed by a processor.

[0136] It should be noted that in this paper, the term "comprise", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or system. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of another identical element in the process, method, article or system including the element.

[0137] The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0138] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment method can be realized by means of software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) execute the method described in each embodiment of the present application.

[0139] Having described various embodiments of the application, it is to be understood that the above description is meant not to limit and not to encompass all of the possible embodiments. Many modifications and variations of this application can be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. It is intended that the scope of the application be defined by the scope of the patent and by the claims as allowed by the patent office, which can include adaptations based on the description, equivalents, and / or substitutions of elements individually or collectively to the entire disclosure.

Claims

1. A parking environment perception method without parking space lines, characterized in that: include: Acquire first surround view images corresponding to each surround view camera of the first vehicle; Determining the boundaries of the drivable areas corresponding to the first surround view images; The boundaries of each drivable area are projected onto a bird's-eye view to obtain a drivable area.

2. The parking environment perception method without parking space lines according to claim 1, characterized in that: Determining the boundaries of the drivable areas corresponding to the first surround view images includes: Inputting each first surround view image into a deep neural network, and determining a column boundary point of each column of pixels in each first surround view image through the deep neural network; Based on the column boundary points, boundary lines of the drivable area boundaries corresponding to the first surround view images are respectively determined.

3. The parking environment perception method without parking space lines according to claim 1 or 2, characterized in that: After obtaining the first surround view images corresponding to the surround view cameras, the method further includes: Each first surround view image is input into the target bird's-eye view network model for model prediction to obtain the first occupancy grid data corresponding to the drivable area.

4. The parking environment perception method without parking space lines according to claim 3, characterized in that: Before inputting each first surround view image into the target bird's-eye view network model for model prediction to obtain first occupied grid data, the method further includes: Acquire second surround view images corresponding to multiple surround view cameras of a second vehicle; performing a labeling operation on each of the second surround view images to obtain first occupied grid labeling information; The first occupied grid annotation information and the second surround view image are input into the to-be-trained bird's-eye view network model for model training to obtain the target bird's-eye view network model.

5. The parking environment perception method without parking space lines according to claim 4, characterized in that: The performing the labeling operation on the second surround view images respectively to obtain the first occupied grid labeling information includes: Obtaining first point cloud data corresponding to the laser radar of the second vehicle; Performing gridding processing on the three-dimensional space corresponding to the point cloud data to obtain first grid data; Based on the first point cloud data, determining, in the first grid data, occupied grid information corresponding to the first point cloud data; The first occupied grid annotation information is determined based on the occupied grid information corresponding to the first point cloud data.

6. A parking environment perception method without parking space lines, characterized in that: include: Acquire a third surround view image corresponding to each surround view camera of the first vehicle; Each third surround view image is input into the target bird's-eye view network model for model prediction to obtain the second occupancy grid data corresponding to the drivable area.

7. The parking environment perception method without parking space lines according to claim 6, characterized in that: Before inputting each third surround view image into the target bird's-eye view network model for model prediction to obtain the second occupied grid data corresponding to the drivable area, the method further includes: Acquire a fourth surround view image corresponding to the plurality of surround view cameras of the second vehicle; performing a labeling operation on each of the fourth surround view images to obtain second occupied grid labeling information; The second occupied grid annotation information and the fourth surround view image are input into the to-be-trained bird's-eye view network model for model training to obtain the target bird's-eye view network model.

8. A vehicle, characterized in that: The vehicle comprises: An acquisition module, configured to acquire a first surround view image corresponding to each surround view camera of the vehicle; a determination module, configured to determine a drivable area boundary corresponding to each first surround view image; The projection module is used to project the boundaries of each drivable area to a bird's-eye view to obtain a drivable area.

9. The vehicle according to claim 8, wherein: The determination module is used to input each first surround view image into a deep neural network, and determine the column boundary points of each column of pixels in each first surround view image through the deep neural network; Based on the column boundary points, boundary lines of the drivable area boundaries corresponding to the respective first surround view images are determined respectively.

10. The vehicle according to claim 8 or 9, characterized in that: The acquisition module is further configured to input each first surround view image into a target bird's-eye view network model for model prediction, thereby obtaining first occupancy grid data corresponding to a drivable area.

11. A parking environment sensing device without parking space lines, characterized in that: The parking environment perception device without parking space lines includes: a memory, a processor, and a parking environment perception program without parking space lines stored in the memory and executable on the processor. When the parking environment perception program without parking space lines is executed by the processor, the steps of the parking environment perception method without parking space lines as described in any one of claims 1 to 5 or 6-7 are implemented.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a parking environment perception program without parking space lines. When the parking environment perception program without parking space lines is executed by a processor, the steps of the parking environment perception method without parking space lines according to any one of claims 1 to 5 or 6-7 are implemented.

13. A computer program product, characterized in that The computer program product includes a parking environment perception program without parking space lines, and when the parking environment perception program without parking space lines is executed by a processor, the steps of the parking environment perception method without parking space lines are implemented as described in any one of claims 1 to 5 or 6-7.

Images