Parking environment sensing method and device without parking space line, vehicle and computer readable storage medium

Abstract

The application discloses a parking environment sensing method and device without parking lines, a vehicle and a computer readable storage medium. The method comprises the following steps: acquiring first surround view images corresponding to each surround camera of a first vehicle; determining a drivable area boundary corresponding to each first surround view image; and projecting each drivable area boundary to a bird's eye view angle to obtain a drivable area. When roadside parking without parking lines is performed, the vehicle can accurately obtain the drivable area through environment sensing of the drivable area boundary corresponding to multiple surround view images. Compared with environment sensing through a laser radar, the cost can be effectively reduced and the calculation amount of point cloud data can be reduced, so that the efficiency and accuracy of drivable area detection can be improved. When roadside parking without parking lines is performed, the drivable area is used for assisting parking, so that the accuracy and efficiency of vehicle parking can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automobiles, in particular to a parking environment perception method and device without parking lines, a vehicle and a computer readable storage medium. BACKGROUND

[0002] With the continuous growth of the number of motor vehicles, the parking lot is not enough to meet the parking demand during peak hours, and sometimes vehicles need to be temporarily parked on the roadside. When there is no obvious parking space on the roadside parking space, or the parking line is blurred, the driver often chooses the position of the parked vehicle for parking.

[0003] At present, in the case of parking lines, the target position of the vehicle is determined by recognizing the parking lines, and then the trajectory planning is performed. In the case of no parking lines, the positions of the front and rear vehicles and the roadside need to be accurately delineated to confirm the drivable area and the target parking position, and then the trajectory planning is performed. When identifying the drivable area in the case of no parking lines, the point cloud data in the surrounding area of the vehicle is generally obtained, the category of each point in the point cloud data is judged, and then the drivable area in the surrounding area is determined according to the category judgment result of each point. This method needs to install expensive laser radar sensors on the vehicle to collect point cloud data, and the calculation amount of the category judgment of each point in the point cloud data is also large, so the calculation efficiency of the drivable area in the surrounding area of the vehicle is also poor. SUMMARY

[0004] The main purpose of the present application is to provide a parking environment perception method and device without parking lines, a vehicle and a computer readable storage medium, which aims to solve the technical problem of poor detection efficiency of the drivable area in the related art.

[0005] To achieve the above purpose, the present application provides a parking environment perception method without parking lines, comprising:

[0006] obtaining a first surround view image corresponding to each surround camera of a first vehicle;

[0007] determining a drivable area boundary corresponding to each first surround view image;

[0008] projecting each drivable area boundary to a bird's eye view angle respectively to obtain a drivable area.

[0009] Further, the step of determining the drivable area boundary corresponding to each first surround view image comprises:

[0010] inputting each first surround view image into a deep neural network respectively to determine a column boundary point of each column of pixels in each first surround view image through the deep neural network;

[0011] Based on the column boundary points, a boundary line corresponding to a drivable area boundary of each first surround view image is determined respectively.

[0012] Further, after the step of acquiring the first surround view image corresponding to each surround view camera of the second vehicle, the method further comprises:

[0013] Each first surround view image is input into a target bird's eye view network model for model prediction to obtain first occupancy grid data corresponding to the drivable area.

[0014] Further, before the step of inputting each first surround view image into a target bird's eye view network model for model prediction to obtain first occupancy grid data, the method further comprises:

[0015] Acquiring second surround view images corresponding to a plurality of surround view cameras of a second vehicle;

[0016] Respectively, the second surround view images are labeled to obtain first occupancy grid label information;

[0017] The first occupancy grid label information and the second surround view images are input into a to-be-trained bird's eye view network model for model training to obtain the target bird's eye view network model.

[0018] Further, the step of respectively labeling the second surround view images to obtain first occupancy grid label information comprises:

[0019] Acquiring first point cloud data corresponding to a laser radar of the second vehicle;

[0020] Grid processing is performed on a three-dimensional space corresponding to the point cloud data to obtain first grid data;

[0021] Based on the first point cloud data, the occupancy grid information corresponding to the first point cloud data is determined in the first grid data;

[0022] Based on the occupancy grid information corresponding to the first point cloud data, the first occupancy grid label information is determined.

[0023] In addition, to achieve the above-mentioned purpose, the application also provides a parking environment perception method without parking space lines, comprising:

[0024] Acquiring third surround view images corresponding to each surround view camera of a first vehicle;

[0025] Each third surround view image is input into a target bird's eye view network model for model prediction to obtain second occupancy grid data corresponding to the drivable area.

[0026] Further, before the step of inputting each third ring view image into a target bird's eye view network model for model prediction to obtain second occupancy grid data corresponding to the drivable area, the method further comprises:

[0027] obtaining fourth ring view images corresponding to the plurality of ring view cameras of the second vehicle;

[0028] respectively performing labeling operations on the fourth ring view images to obtain second occupancy grid labeling information;

[0029] inputting the second occupancy grid labeling information and the fourth ring view images into a bird's eye view network model to be trained for model training to obtain the target bird's eye view network model.

[0030] In addition, to achieve the above object, the application further provides a vehicle, which comprises:

[0031] an acquisition module, configured to acquire first ring view images corresponding to each ring view camera of the vehicle;

[0032] a determination module, configured to determine a drivable area boundary corresponding to each first ring view image;

[0033] a projection module, configured to respectively project each drivable area boundary to a bird's eye view angle to obtain a drivable area.

[0034] In addition, to achieve the above object, the application further provides a parking environment perception device without parking lines, which comprises a memory, a processor, and a parking environment perception program without parking lines stored in the memory and executable on the processor, and the parking environment perception program without parking lines, when executed by the processor, implements the steps of the aforementioned parking environment perception method without parking lines.

[0035] In addition, to achieve the above object, the application further provides a computer readable storage medium, which has a parking environment perception program without parking lines stored thereon, and the parking environment perception program without parking lines, when executed by a processor, implements the steps of the aforementioned parking environment perception method without parking lines.

[0036] This application acquires first surround view images corresponding to each surround view camera of a first vehicle; then determines the drivable area boundary corresponding to each first surround view image; and then projects each drivable area boundary onto a bird's-eye view to obtain the drivable area. When parking on the side of the road without parking lines, the vehicle accurately obtains the drivable area by perceiving the environment through the drivable area boundaries corresponding to multiple surround view images. Compared with environmental perception through LiDAR, this can effectively reduce costs and the amount of point cloud data computation, thereby improving the efficiency and accuracy of drivable area detection. When parking on the side of the road without parking lines, drivable area-assisted parking improves the accuracy and efficiency of vehicle parking. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the structure of a parking environment sensing device without parking lines in the hardware operating environment involved in the embodiments of this application;

[0038] Figure 2 This is a flowchart illustrating the first embodiment of the parking environment perception method without parking lines in this application.

[0039] Figure 3 This is a schematic diagram illustrating one possible drivable area in one embodiment of this application;

[0040] Figure 4 This is a schematic diagram illustrating one possible boundary point in one embodiment of this application;

[0041] Figure 5 This is a flowchart illustrating the fourth embodiment of the parking environment perception method without parking lines in this application.

[0042] Figure 6 This is a schematic diagram of the functional modules of an embodiment of the vehicle of this application.

[0043] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0044] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0045] like Figure 1 As shown, Figure 1 This is a schematic diagram of the parking environment sensing device without parking space lines in the hardware operating environment involved in the embodiments of this application.

[0046] The parking environment sensing device without parking space lines in this embodiment can be a vehicle. For example... Figure 1As shown, the parking environment perception device without parking space line can include a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to realize the connection communication between the components. The user interface 1003 can include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 can also include a standard wired interface, a wireless interface. The network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a WI-FI interface). The memory 1005 can be a high-speed RAM memory, or a stable memory (non-volatile memory) such as a disk memory. The memory 1005 can also be an independent storage device from the aforementioned processor 1001.

[0047] Optionally, the parking environment perception device without parking space line can also include a camera, an RF (Radio Frequency, radio frequency) circuit, a sensor, an audio circuit, a WiFi module, etc. The sensor can be a light sensor, a motion sensor, and other sensors. Of course, the parking environment perception device without parking space line can also be configured with a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and other sensors, which will not be described here.

[0048] Those skilled in the art can understand that Figure 1 The terminal structure shown in the figure does not constitute a limitation on the parking environment perception device without parking space line, and can include more or fewer components than the figure, or combine certain components, or different component arrangements.

[0049] As Figure 1 As shown, the memory 1005 as a computer storage medium can include an operating system, a network communication module, a user interface module, and a parking environment perception program without parking space line.

[0050] In Figure 1 In the parking environment perception device without parking space line shown in the figure, the network interface 1004 is mainly used to connect the background server and communicate data with the background server; the user interface 1003 is mainly used to connect the client (user end) and communicate data with the client; and the processor 1001 can be used to call the parking environment perception program without parking space line stored in the memory 1005.

[0051] In the embodiment, the parking environment perception device without parking lines comprises a memory 1005, a processor 1001, and a parking environment perception program without parking lines stored in the memory 1005 and executable on the processor 1001. When the processor 1001 invokes the parking environment perception program without parking lines stored in the memory 1005, the steps of the parking environment perception method without parking lines in each of the following embodiments are performed.

[0052] The application also provides a parking environment perception method without parking lines, which refers to Figure 2 , Figure 2 FIG. 1 is a flowchart of a first embodiment of the parking environment perception method without parking lines of the application.

[0053] The first vehicle is provided with a plurality of surround-view cameras, which can be fisheye cameras. The number of surround-view cameras can be reasonably set, and the 360° environment around the first vehicle can also be imaged by the plurality of surround-view cameras. For example, the surround-view cameras can be set to four, which are respectively arranged on the left and right sides of the front end of the first vehicle and the left and right sides of the rear end of the first vehicle.

[0054] The parking environment perception method without parking lines comprises the following steps.

[0055] In step S101, first surround-view images corresponding to each surround-view camera of the first vehicle are acquired.

[0056] In step S102, drivable area boundaries corresponding to each first surround-view image are determined.

[0057] In step S103, each drivable area boundary is projected to a bird's-eye perspective to obtain a drivable area.

[0058] When the vehicle is parked on the roadside, for example, when the vehicle is parked on the roadside without parking lines, the first surround-view images are acquired by the surround-view cameras of the first vehicle. After the first surround-view images are acquired, the drivable area boundaries corresponding to each first surround-view image are determined. Specifically, the drivable area boundaries corresponding to each first surround-view image can be determined by a trained deep neural network, that is, each first surround-view image is input into the deep neural network for prediction, and the drivable area boundaries in each first surround-view image are determined by the deep neural network. For example, the column boundary points in each column of pixels of the first surround-view image are predicted by the deep neural network, and the lines connecting each column boundary point are used as the drivable area boundary of the corresponding surround-view image, wherein one side of the drivable area boundary is a drivable area and the other side is a non-drivable area (such as a parking area of other vehicles or an outer side of the road, etc.).

[0059] After obtaining the drivable area boundaries corresponding to each first surround view image, respectively project each of the drivable area boundaries to a bird's eye view (BEV), obtain each drivable area boundary in the bird's eye view, and take the area surrounded by each drivable area boundary in the bird's eye view as a drivable area, i.e., the drivable area is the area surrounded by each drivable area boundary in the bird's eye view, with reference to Figure 3 , Figure 3 FIG. 1 is a schematic diagram of a drivable area according to an embodiment of the present application, wherein the drivable area is a closed area surrounded by each drivable area boundary in the bird's eye view, and if the area surrounded by each drivable area boundary in the bird's eye view is an open area, each drivable area boundary in the bird's eye view can be extended so that the area surrounded by each drivable area boundary in the bird's eye view after extension is a closed area, obtaining the drivable area.

[0060] Further, in a possible implementation, after step S101, the parking environment perception method without parking space further includes:

[0061] Step S104: input each first surround view image into a target bird's eye network model for model prediction to obtain first occupancy grid data corresponding to the drivable area.

[0062] It should be noted that before predicting the drivable area, a target bird's eye (BEV) network model is obtained through model training, and the target bird's eye network model is loaded to the first vehicle.

[0063] After obtaining the first surround view image, input the first surround view image into the target bird's eye network model for model prediction to obtain first occupancy grid data corresponding to the drivable area, wherein the first occupancy grid data is the grid information occupied by the drivable area in the parking area corresponding to the first vehicle.

[0064] Specifically, the convolutional neural network in the target bird's eye network model is used to extract features from each input (first surround view image), and then the transformer in the target bird's eye network model is used to convert the features from the camera view to the BEV view to obtain a bird's eye view, and the target bird's eye network model predicts the first occupancy grid data through the bird's eye view.

[0065] In the roadside parking without parking lines, the vehicle obtains the drivable area and the corresponding occupancy grid data through the environment perception of the multiple surround view images, compared with the environment perception through the laser radar, the cost can be effectively reduced and the calculation amount of the point cloud data can be reduced, and then the efficiency and the accuracy of the drivable area detection can be improved, in the roadside parking without parking lines, the drivable area can be displayed to assist the driver to park, or the drivable area can be used to assist the autonomous vehicle to park, and then the accuracy and the efficiency of the vehicle parking can be improved through the drivable area.

[0066] The first surround view image corresponding to each surround view camera of the first vehicle is obtained, then the drivable area boundary corresponding to each first surround view image is determined, and then each drivable area boundary is projected to the bird's eye view angle to obtain the drivable area, in the roadside parking without parking lines, the vehicle obtains the drivable area through the environment perception of the drivable area boundary corresponding to the multiple surround view images, compared with the environment perception through the laser radar, the cost can be effectively reduced and the calculation amount of the point cloud data can be reduced, and then the efficiency and the accuracy of the drivable area detection can be improved, in the roadside parking without parking lines, the drivable area is used to assist the parking to improve the accuracy and the efficiency of the vehicle parking.

[0067] Based on the first embodiment, the second embodiment of the parking environment perception method without parking lines is provided, wherein the step S102 comprises:

[0068] In step S201, each first surround view image is input into a deep neural network, and the column boundary point of each column pixel in each first surround view image is determined through the deep neural network.

[0069] In step S202, based on the column boundary point, the boundary line of the drivable area boundary corresponding to each first surround view image is determined.

[0070] After obtaining each first surround view image, each first surround view image is input into a deep neural network, and the column boundary point of each column pixel in each first surround view image is determined through the deep neural network, as shown in Figure 4 The one side of the column boundary point is the drivable area and the other side is the non-drivable area.

[0071] After obtaining the column boundary points of each column of pixels in each first surround view image, the boundary points corresponding to each first surround view image are obtained based on the column boundary points. Specifically, for each first surround view image, the set of all column boundary points of the first surround view image is the boundary point of the first surround view image. And based on the boundary points, the boundary lines of the drivable area boundary corresponding to each first surround view image are determined. Specifically, for each first surround view image, the column boundary points of each column of pixels in the first surround view image are connected in turn, and the connecting lines are taken as the boundary lines of the drivable area boundary corresponding to the first surround view image.

[0072] The depth neural network is a pre-trained neural network. In the training of the depth neural network, the training image and the boundary point label of each column boundary point of each column of pixels in each training image can be obtained. Each training image is input into the initial depth neural network for model training to obtain the predicted column boundary point of each column of pixels in each training image. The accuracy of the predicted column boundary point is determined according to the predicted column boundary point and the corresponding boundary point label. When the accuracy is greater than a preset accuracy, the trained initial depth neural network is taken as the depth neural network. Otherwise, the trained initial depth neural network is taken as the initial depth neural network. The initial depth neural network is trained again according to the training image until the accuracy is greater than the preset accuracy, or the training of the initial depth neural network reaches a preset number of times. The trained initial depth neural network is taken as the depth neural network. The trained depth neural network is loaded into the first vehicle.

[0073] By inputting each first surround view image into the depth neural network respectively, the column boundary points of each column of pixels in each first surround view image are determined by the depth neural network. Then, based on the column boundary points, the boundary lines of the drivable area boundary corresponding to each first surround view image are determined respectively. The column boundary points in each first surround view image can be accurately obtained by the depth neural network. The boundary lines of the drivable area boundary can be accurately obtained according to the column boundary points. The drivable area can be obtained by the boundary lines. When parking on the roadside without a parking line, the vehicle can accurately obtain the drivable area by environmental perception through multiple surround view images, which further improves the efficiency and accuracy of drivable area detection. The accuracy and efficiency of vehicle parking are improved by assisting parking in the drivable area on the roadside without a parking line.

[0074] Based on the first embodiment, a third embodiment of the parking environment perception method without a parking line is provided. Before step S104, the parking environment perception method without a parking line further includes:

[0075] In step S301, a plurality of surround view cameras of a second vehicle are obtained.

[0076] In step S302, the second ring view image is labeled respectively to obtain first occupancy grid label information.

[0077] In step S303, the first occupancy grid label information and the second ring view image are input into a bird's eye network model to be trained to perform model training, and the target bird's eye network model is obtained.

[0078] The second vehicle can be a data collection vehicle.

[0079] In the model training of the target bird's eye network model, the second ring view image corresponding to the plurality of ring view cameras of the second vehicle is obtained, and the second ring view image can be an image at a plurality of time points to obtain a plurality of groups of ring view images for model training, and each group of images includes a plurality of ring view images taken by the ring view cameras at the same time point.

[0080] After the second ring view image is obtained, each second ring view image is labeled respectively to obtain first occupancy grid label information, and the first occupancy grid label information is a label of an occupied grid of a drivable area in a parking area corresponding to the second vehicle.

[0081] In step S3021, the first point cloud data corresponding to the laser radar of the second vehicle is obtained.

[0082] In step S3022, the three-dimensional space corresponding to the point cloud data is gridded to obtain first grid data.

[0083] In step S3023, based on the first point cloud data, the occupancy grid information corresponding to the first point cloud data in the first grid data is determined.

[0084] In step S3024, based on the occupancy grid information corresponding to the first point cloud data, the first occupancy grid label information is determined.

[0085] The second vehicle is provided with a laser radar, and after the second ring view image is obtained, the first point cloud data corresponding to the laser radar of the second vehicle is obtained, and the first point cloud data can be a plurality of time points of laser radar collection data. The time point of each collection data in the first point cloud data is the same as the shooting time point of the image in the second ring view image.

[0086] After the first point cloud data is acquired, the three-dimensional space corresponding to the first point cloud data is gridded to obtain first grid data. For example, the three-dimensional space corresponding to the first point cloud data is a three-dimensional space of 20m*20m*5m, and the three-dimensional space corresponding to the first point cloud data is gridded according to a preset resolution (such as 0.1m) to obtain the first grid data. Then, the occupancy grid information corresponding to the first point cloud data is determined in the first grid data based on the first point cloud data, that is, the grid with the point cloud in the first point cloud data in the first grid data is taken as the occupancy grid in the occupancy grid information. Of course, the intensity (number of point cloud data) of each occupancy grid can also be counted.

[0087] After the occupancy grid information is acquired, the first occupancy grid labeling information is determined based on the occupancy grid information corresponding to the first point cloud data. Specifically, the occupancy grid information corresponding to the first point cloud data can be converted into the first occupancy grid labeling information according to the relationship between the point cloud data and the second surround view image.

[0088] After the first occupancy grid labeling information is acquired, the first occupancy grid labeling information and the second surround view image are input into a to-be-trained bird's eye view network model for model training to obtain the target bird's eye view network model. Specifically, each image in the plurality of images corresponding to the second surround view image is used for model training, and a loss function of the trained model is calculated through the prediction result of each image and the first occupancy grid labeling information. When the loss function is less than a preset loss function, the to-be-trained bird's eye view network model after the current training is taken as the target bird's eye view network model, otherwise, the to-be-trained bird's eye view network model after the current training is taken as the to-be-trained bird's eye view network model, and the step S303 is returned.

[0089] It should be noted that the training process of the bird's eye view network model can be applied to a server in communication connection with the first vehicle. After the target bird's eye view network model is obtained, the server sends the target bird's eye view network model to the first vehicle, and the first vehicle loads the target bird's eye view network model.

[0090] The second surround view image corresponding to the plurality of surround view cameras of the second vehicle is acquired. Then, the second surround view image is labeled to obtain the first occupancy grid labeling information. Then, the first occupancy grid labeling information and the second surround view image are input into a to-be-trained bird's eye view network model for model training to obtain the target bird's eye view network model. The target bird's eye view network model is obtained by model training based on the second surround view image and the labeled data, which improves the accuracy of the target bird's eye view network model, and further improves the accuracy and precision of the drivable area detection based on the target bird's eye view network model, and further improves the accuracy of the roadside parking without parking lines.

[0091] The application also provides a parking environment perception method without a parking space line, referring to Figure 5 , Figure 5 FIG. 4 is a flowchart of a fourth embodiment of the parking environment perception method without a parking space line of the application.

[0092] The first vehicle is provided with multiple surround-view cameras, wherein the surround-view cameras can be fisheye cameras, and the number of surround-view cameras can be reasonably set. The 360° environment around the first vehicle can also be imaged by multiple surround-view cameras. For example, the surround-view cameras can be set to 4, and are respectively arranged on the left and right sides of the front end of the first vehicle and the left and right sides of the rear end of the first vehicle.

[0093] The parking environment perception method without a parking space line comprises the following steps.

[0094] In step S401, third surround-view images corresponding to each surround-view camera of the first vehicle are acquired.

[0095] In step S402, each third surround-view image is input into a target bird's eye view network model for model prediction to obtain second occupancy grid data corresponding to a drivable area.

[0096] It should be noted that before predicting the drivable area, the target bird's eye view (BEV) network model is obtained through model training, and the target bird's eye view network model is loaded into the first vehicle.

[0097] When the vehicle is parked on the roadside, for example, when the vehicle is parked on the roadside without a parking space line, the third surround-view image is acquired through each surround-view camera of the first vehicle. After the third surround-view image is acquired, the third surround-view image is input into the target bird's eye view network model for model prediction to obtain second occupancy grid data corresponding to a drivable area, wherein the second occupancy grid data is the grid information occupied by the drivable area in the parking area corresponding to the first vehicle.

[0098] Specifically, the convolutional neural network in the target bird's eye view network model extracts features from each input (second surround-view image), and then the transformer in the target bird's eye view network model converts the features from the camera perspective to the BEV perspective to obtain a bird's eye view. The target bird's eye view network model predicts the second occupancy grid data through the bird's eye view.

[0099] The third bird's eye view image corresponding to each surround view camera of the first vehicle is acquired; then each third bird's eye view image is input into a target bird's eye network model for model prediction to obtain second occupancy grid data corresponding to a drivable area. When roadside parking without a parking line is performed, the second occupancy grid data of the drivable area can be obtained through model prediction of multiple surround view images. Compared with environment perception through a laser radar, the cost can be effectively reduced and the calculation amount of point cloud data can be reduced, and thus the efficiency and accuracy of drivable area detection can be improved. The occupancy grid data of the drivable area is used to assist parking when roadside parking without a parking line is performed, so as to improve the accuracy and efficiency of vehicle parking.

[0100] Based on the first embodiment, a fifth embodiment of the parking environment perception method without a parking line is provided, wherein before step S402, the parking environment perception method without a parking line further comprises:

[0101] In step S501, fourth surround view images corresponding to multiple surround view cameras of a second vehicle are acquired.

[0102] In step S502, a second occupancy grid annotation information is obtained by performing an annotation operation on each fourth surround view image.

[0103] In step S503, the second occupancy grid annotation information and the fourth surround view image are input into a bird's eye network model to be trained for model training to obtain the target bird's eye network model.

[0104] The setting mode of each surround view camera in the second vehicle is the same as that of each surround view camera in the first vehicle. The second vehicle can be a data collection vehicle.

[0105] When the model training of the target bird's eye network model is performed, the fourth surround view images corresponding to the multiple surround view cameras of the second vehicle are acquired. The fourth surround view images can be images at multiple time points, so as to obtain multiple groups of surround view images for model training. Each group of images includes surround view images captured by multiple surround view cameras at the same time.

[0106] After the fourth surround view images are acquired, an annotation operation is performed on each fourth surround view image to obtain second occupancy grid annotation information. The second occupancy grid annotation information is an annotation of a grid occupied by a drivable area in a parking area corresponding to the second vehicle. Specifically, in a possible implementation, step S502 includes:

[0107] In step S5021, second point cloud data corresponding to a laser radar of the third vehicle is acquired.

[0108] In step S5022, occupancy grid information corresponding to the second point cloud data is acquired.

[0109] In step S5023, the second occupancy grid label information is determined based on the occupancy grid information corresponding to the second point cloud data.

[0110] The second vehicle is provided with a laser radar, and after the fourth surround view image is acquired, second point cloud data corresponding to the laser radar of the second vehicle is acquired. The second point cloud data can be acquisition data of the laser radar at multiple time points, and the time point of each acquisition data in the second point cloud data is the same as the shooting time point of the image in the fourth surround view image.

[0111] After the second point cloud data is acquired, the occupancy grid information corresponding to the second point cloud data is acquired. Specifically, in a possible implementation, the step S5022 includes:

[0112] In step d, the three-dimensional space corresponding to the second point cloud data is gridded to obtain second grid data.

[0113] In step e, the occupancy grid information corresponding to the second point cloud data is determined in the second grid data based on the second point cloud data.

[0114] After the second point cloud data is acquired, the three-dimensional space corresponding to the second point cloud data is gridded to obtain second grid data. For example, the three-dimensional space corresponding to the second point cloud data is a three-dimensional space of 20m*20m*5m, and the three-dimensional space corresponding to the second point cloud data is gridded according to a preset resolution (such as 0.1m) to obtain the second grid data. Then, the occupancy grid information corresponding to the second point cloud data is determined in the second grid data based on the second point cloud data, that is, the grid in the second grid data that exists in the point cloud in the second point cloud data is taken as the occupancy grid in the occupancy grid information. Of course, the intensity (number of point cloud data) of each occupancy grid can also be counted.

[0115] After the occupancy grid information is acquired, the second occupancy grid label information is determined based on the occupancy grid information corresponding to the second point cloud data. Specifically, the occupancy grid information corresponding to the second point cloud data can be converted into the second occupancy grid label information according to the relationship between the point cloud data and the fourth surround view image.

[0116] After the second occupancy grid label information is acquired, the second occupancy grid label information and the third surround view image are input into the to-be-trained bird's eye view network model for model training to obtain the target bird's eye view network model. Specifically, model training is performed according to each group of images corresponding to the third surround view image, a loss function of the trained model is calculated through a prediction result of each group of images and the second occupancy grid label information, when the loss function is less than a preset loss function, the to-be-trained bird's eye view network model after current training is taken as the target bird's eye view network model, otherwise, the to-be-trained bird's eye view network model after current training is taken as the to-be-trained bird's eye view network model, and the step S503 is executed.

[0117] The fourth surround view image corresponding to the plurality of surround view cameras of the second vehicle is acquired, then the fourth surround view image is labeled to obtain the second occupancy grid label information, and then the second occupancy grid label information and the fourth surround view image are input into the to-be-trained bird's eye view network model for model training to obtain the target bird's eye view network model. The target bird's eye view network model is obtained through model training of the fourth surround view image and the labeled data, the accuracy of the target bird's eye view network model is improved, and the accuracy and precision of the drivable area detection through the target bird's eye view network model are improved, and the accuracy of the roadside parking without a parking line is further improved.

[0118] In addition, the present application also provides a vehicle, referring to Figure 6 The vehicle comprises:

[0119] The acquisition module 10 is configured to acquire a first surround view image corresponding to each surround view camera of the vehicle.

[0120] The determination module 20 is configured to determine a drivable area boundary corresponding to each first surround view image.

[0121] The projection module 30 is configured to project each drivable area boundary to a bird's eye view angle to obtain a drivable area.

[0122] The method performed by each program unit can refer to each embodiment of the parking environment perception method without a parking line, which will not be described here.

[0123] In addition, the present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a parking environment perception program without a parking line, and the parking environment perception program without a parking line is executed by a processor to implement the steps of the parking environment perception method without a parking line.

[0124] In addition, the embodiment of the present application further provides a computer program product, which comprises a parking environment sensing program without parking lines, and the parking environment sensing program without parking lines realizes the steps of the parking environment sensing method without parking lines when executed by a processor.

[0125] It should be noted that, in this document, the terms "comprising", "containing", or any other variant thereof are intended to cover a non-exclusive inclusion, so that a process, method, article, or system that comprises a list of elements not only includes those elements, but also includes other elements not expressly listed or inherent to such process, method, article, or system. Without more limitations, an element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or system that includes the element.

[0126] The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0127] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and necessary general hardware platforms, and of course, they can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disk) as described above, and includes a plurality of instructions for making a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device) execute the methods described in the various embodiments of the present application.

[0128] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation using the content of the specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A parking environment sensing method without a parking space line, characterized by, The method comprises the following steps: obtaining a first surround view image corresponding to each surround view camera of a first vehicle; determining a drivable area boundary corresponding to each first surround view image; projecting each drivable area boundary to a bird's eye view, obtaining each drivable area boundary in the bird's eye view, and taking an area surrounded by each drivable area boundary in the bird's eye view as a drivable area; the step of determining the drivable area boundary corresponding to each first surround view image comprises: inputting each first surround view image into a deep neural network respectively, determining a column boundary point of each column of pixels in each first surround view image through the deep neural network, obtaining a training image and a boundary point label of each column of pixels in each training image, inputting each training image into an initial deep neural network for model training to obtain a predicted column boundary point of each column of pixels in each training image, determining an accuracy of the predicted column boundary point according to the predicted column boundary point and the corresponding boundary point label, and taking the trained initial deep neural network as the deep neural network when the accuracy is greater than a preset accuracy; based on the column boundary point, determining a boundary line of the drivable area boundary corresponding to each first surround view image respectively, wherein the drivable area is an area surrounded by each drivable area boundary in the bird's eye view, and if the area surrounded by the drivable area is an open area, the drivable area boundary in the bird's eye view is extended so that the area surrounded by each drivable area boundary in the bird's eye view after the extension is a closed area, and the drivable area is obtained.

2. The parking environment sensing method of claim 1, wherein, The method further comprises the following steps: when the accuracy is not greater than the preset accuracy, taking the trained initial deep neural network as the initial deep neural network, training the initial deep neural network again according to the training image until the accuracy is greater than the preset accuracy, or taking the trained initial deep neural network as the deep neural network after the training number of the initial deep neural network reaches a preset number.

3. The parking environment sensing method of a parking spaceless line according to any one of claims 1 or 2, characterized in that, The surround view camera comprises a fisheye camera.

4. A vehicle characterized by comprising: The vehicle comprises: an obtaining module configured to obtain a first surround view image corresponding to each surround view camera of a vehicle; a determining module configured to determine a drivable area boundary corresponding to each first surround view image; a projecting module configured to project each drivable area boundary to a bird's eye view, obtain each drivable area boundary in the bird's eye view, and take an area surrounded by each drivable area boundary in the bird's eye view as a drivable area; The determining module is further configured to input each first surround view image into a deep neural network respectively, determine a column boundary point of each column of pixels in each first surround view image through the deep neural network, determine a boundary line of the corresponding drivable area boundary of each first surround view image based on the column boundary point, wherein the drivable area is an area surrounded by each drivable area boundary in the bird's eye view, if the area surrounded by the drivable area is an open area, extend each drivable area boundary in the bird's eye view to make the area surrounded by each drivable area boundary in the extended bird's eye view a closed area, obtain the drivable area, obtain a to-be-trained image and a boundary point label of each column of pixels in each to-be-trained image, input each to-be-trained image into an initial deep neural network for model training, obtain a predicted column boundary point of each column of pixels in each to-be-trained image, determine an accuracy rate of the predicted column boundary point according to the predicted column boundary point and the corresponding boundary point label, and when the accuracy rate is greater than a preset accuracy rate, take the trained initial deep neural network as the deep neural network.

5. A parking environment sensing device without a parking space line, characterized by, The parking environment perception device without parking space lines comprises a memory, a processor, and a parking environment perception program without parking space lines stored on the memory and capable of running on the processor, and the parking environment perception program without parking space lines implements the steps of the parking environment perception method without parking space lines according to any one of claims 1 to 3 when executed by the processor.

6. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a parking environment perception program without parking space lines, and the parking environment perception program without parking space lines implements the steps of the parking environment perception method without parking space lines according to any one of claims 1 to 3 when executed by the processor.

Images