Parking space line detection method and device under aerial view, electronic equipment and storage medium

Abstract

The invention provides a parking space line detection method and device under an aerial view, electronic equipment and a storage medium, and the method comprises the steps: projecting a depth feature and an image feature to a real environment with a vehicle position as an original point of a coordinate system through a homography matrix, and obtaining a visual cone point cloud feature as the position information in a real physical world, according to the method, the shot object can be restored to the position of the real world, then parking space line angular point and parking space entrance line detection is carried out on the view cone point cloud features, the obtained parking space line angular point position close to the parking space entrance line and the detection result of the parking space entrance line are more accurate, and the accuracy of the finally determined parking space line of the parking space is high.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a parking space line detection method, device, electronic equipment and storage medium under a bird's-eye view. Background Art

[0002] Parking line detection is an important part of autonomous parking and can play a role in restraining and protecting the vehicle's driving. Whether in vehicle safety driving systems or in intelligent vehicle parking based on machine vision, parking line detection and recognition is a basic and necessary functional module that can not only reduce the risk of vehicle collision, but also accurately park the vehicle in the designated area.

[0003] At present, mainstream parking operations are based on the solution of vehicle surround perception, that is, to perceive the surrounding scene through LiDAR or surround view camera. However, LiDAR is expensive, so in recent years, pure visual perception tasks based on surround view cameras have become more and more common. The main method is to stitch the images collected by the cameras with four or six perspectives of the vehicle body to obtain a 360° global image, also known as BEV (Bird Eye View), and then introduce tasks such as target detection and tracking to further perceive the surrounding environment.

[0004] When performing parking line detection, the BEV perception currently used is based on the inverse perspective transformation (IPM) method, which is based on the assumption that the ground truth objects are flat, that is, all objects are flattened and assumed to be at the same height, and then surround stitching is performed to restore the appearance of the surrounding environment to the maximum extent, and the perception task (such as parking line detection) is performed on this basis. Although for IPM, it has the function of transforming complex perspective geometry problems into simple geometry problems on the plane, such as lane lines and parking line detection and estimation, its fusion results can both unify the scale and reduce visual occlusion. However, there are also certain disadvantages, such as the loss of edge information: since the image information at the edge of the field of view may be sheared or compressed during the conversion, some useful road surface information may be lost; in addition, IPM relies on the flat road assumption: in many cases, the inverse perspective transformation is based on the assumption that the road is flat, which may cause errors in the case of slopes or uneven roads and the vehicle itself is tilted, such as Figure 1 As shown, when the surround view camera is not kept at the same level, it is very easy to cause incomplete stitching of the stitching image and errors in the distance, which will lead to poor accuracy of subsequent parking space line detection.

[0005] In summary, the traditional IPM-based parking space line detection method has a technical problem that when the cameras of the vehicle body are not aligned horizontally, the captured images have errors in distance or stitching, which leads to poor accuracy of the parking space lines finally detected. Summary of the invention

[0006] In view of this, an object of the present invention is to provide a parking space line detection method, device, electronic device and storage medium under a bird's-eye view, so as to alleviate the technical problem that the parking space line detected by the existing parking space line detection method has poor accuracy.

[0007] In a first aspect, an embodiment of the present invention provides a parking space line detection method under a bird's-eye view, comprising:

[0008] Acquire an original image captured by a surround-view camera of the vehicle, and perform depth estimation and feature extraction on the original image to obtain depth features and image features;

[0009] Projecting the depth feature and the image feature into a real environment with the vehicle position as the origin of the coordinate system through a homography matrix to obtain a view cone point cloud feature, wherein the homography matrix is ​​generated according to the internal and external parameters of the surround view camera;

[0010] Performing parking space line corner point and parking space entrance line detection on the view cone point cloud features to obtain parking space line corner point positions and parking space entrance lines close to the parking space entrance lines, and determining paired parking space line corner point positions and parking space entrance line types close to the parking space entrance lines according to the parking space line corner point positions and the parking space entrance lines;

[0011] The corner point positions of the paired parking lines close to the parking entrance line and the parking entrance line type are determined, and the parking lines of the parking spaces are determined according to the corner point positions of the paired parking lines close to the parking entrance line and the corner point positions of the paired parking lines far from the parking entrance line.

[0012] Furthermore, projecting the depth feature and the image feature into a real environment with the vehicle position as the origin of the coordinate system through a homography matrix includes:

[0013] The depth feature and the image feature are converted by using the homography matrix to obtain a bird's-eye view feature;

[0014] The bird's-eye view features are processed to obtain the view cone point cloud features.

[0015] Further, determining the corner point positions of the paired parking space lines close to the parking space entrance line and the parking space entrance line type according to the corner point positions of the parking space lines close to the parking space entrance line and the parking space entrance line includes:

[0016] Using geometric clues to match the corner point positions of the parking space lines close to the parking space entrance lines to obtain the paired corner point positions of the parking space lines close to the parking space entrance lines;

[0017] The parking space entrance line type is determined according to the positions of the paired parking space line corner points close to the parking space entrance line and the parking space entrance line.

[0018] Furthermore, the parking space entrance line type includes any one of the following: a parking space entrance line of a vertical parking space type, a parking space entrance line of a parallel parking space type, and a parking space entrance line of an oblique parking space type.

[0019] Further, determining the positions of the corner points of the paired parking space lines far from the parking space entrance lines according to the positions of the corner points of the paired parking space lines close to the parking space entrance lines and the types of the parking space entrance lines includes:

[0020] Determine the parking space entrance line angle and the parking space distance according to the parking space entrance line type;

[0021] The positions of the corner points of the paired parking space lines far from the parking space entrance line are calculated according to the parking space entrance line angle, the parking space distance and the corner point positions of the paired parking space lines close to the parking space entrance line.

[0022] Further, calculating the corner point positions of the paired parking space lines far from the parking space entrance line according to the parking space entrance line angle, the parking space distance and the corner point positions of the paired parking space lines close to the parking space entrance line includes:

[0023] Calculate the formula based on the corner position of the paired parking space lines away from the parking space entrance line Calculate the corner point positions of the paired parking lines away from the parking entrance line, where p3 and p4 represent the corner point positions of the paired parking lines away from the parking entrance line, α i represents the parking space entrance line included angle, p1 and p2 represent the positions of the corner points of the paired parking space lines close to the parking space entrance line, and d i Indicates the distance to the parking space.

[0025] Furthermore, the corner points of the parking space line and the parking space entrance line are detected on the view cone point cloud features, including:

[0026] The target detection model is used to detect the parking space line corner points and the parking space entrance line of the view cone point cloud features to obtain the parking space line corner point position close to the parking space entrance line and the parking space entrance line.

[0027] In a second aspect, an embodiment of the present invention further provides a parking space line detection device under a bird's eye view, comprising:

[0028] A depth estimation and feature extraction unit, used to obtain an original image taken by a surround view camera of the vehicle, and perform depth estimation and feature extraction on the original image to obtain depth features and image features;

[0029] A projection unit, configured to project the depth feature and the image feature into a real environment with the vehicle position as the origin of a coordinate system through a homography matrix to obtain a view cone point cloud feature, wherein the homography matrix is ​​generated according to the internal and external parameters of the surround view camera;

[0030] a detection unit, configured to detect parking space line corner points and parking space entrance lines on the view cone point cloud features, obtain parking space line corner point positions and parking space entrance lines close to the parking space entrance lines, and determine paired parking space line corner point positions and parking space entrance line types close to the parking space entrance lines according to the parking space line corner point positions and the parking space entrance lines;

[0031] A determination unit is used to determine the corner point positions of the paired parking lines far from the parking entrance line according to the corner point positions of the paired parking lines close to the parking entrance line and the parking entrance line type, and to determine the parking lines of the parking spaces according to the corner point positions of the paired parking lines close to the parking entrance line and the corner point positions of the paired parking lines far from the parking entrance line.

[0032] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the methods described in the first aspect when executing the computer program.

[0033] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to execute any method described in the first aspect above.

[0034] In an embodiment of the present invention, a parking space line detection method under a bird's-eye view is provided, comprising: acquiring an original image taken by a surround-view camera of a vehicle, and performing depth estimation and feature extraction on the original image to obtain depth features and image features; projecting the depth features and image features into a real environment with a vehicle position as the origin of a coordinate system through a homography matrix to obtain a view cone point cloud feature, wherein the homography matrix is ​​generated according to internal and external parameters of the surround-view camera; performing parking space line corner point and parking space entrance line detection on the view cone point cloud feature to obtain parking space line corner point positions and parking space entrance lines close to the parking space entrance lines, and determining paired parking space line corner point positions and parking space entrance line types close to the parking space entrance lines according to the parking space line corner point positions and the parking space entrance lines; determining paired parking space line corner point positions far from the parking space entrance lines according to the paired parking space line corner point positions and the parking space entrance line types, and determining parking space lines of parking spaces according to the paired parking space line corner point positions close to the parking space entrance lines and the paired parking space line corner point positions far from the parking space entrance lines. From the above description, it can be seen that in the parking space line detection method under the bird's-eye view of the present invention, the depth features and image features are projected into the real environment with the vehicle position as the origin of the coordinate system through the homography matrix, and the obtained view cone point cloud features are the position information in the real physical world, which can restore the photographed object to the position in the real world, and then the parking space line corner points and parking space entrance line detection are performed on the view cone point cloud features. The detection results of the parking space line corner point positions and parking space entrance lines close to the parking space entrance lines are more accurate, and the parking space lines of the parking spaces finally determined are highly accurate, which alleviates the technical problem that the traditional IPM-based parking space line detection method, when the cameras of the vehicle body are not consistent in level, causes the captured images to have distance or splicing errors, which in turn leads to the poor accuracy of the parking space lines finally detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 Schematic diagram of incomplete image stitching caused by IPM provided by traditional solutions;

[0037] Figure 2 A flowchart of a parking space line detection method under a bird's-eye view provided by an embodiment of the present invention;

[0038] Figure 3 A schematic diagram of the parking space entrance line type provided by an embodiment of the present invention;

[0039] Figure 4 A schematic diagram of the relationship between the parking space entrance line type, the parking space entrance line angle, and the parking space distance provided by an embodiment of the present invention;

[0040] Figure 5 A schematic diagram of a parking space line detection device under a bird's eye view provided by an embodiment of the present invention;

[0041] Figure 6 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] In the traditional IPM-based parking space line detection method, when the cameras on the vehicle body are not aligned horizontally, the captured images have errors in distance or stitching, which leads to poor accuracy of the parking space line finally detected.

[0044] Based on this, in the parking space line detection method under the bird's-eye view of the present invention, the depth features and image features are projected into the real environment with the vehicle position as the origin of the coordinate system through the homography matrix, and the obtained view cone point cloud features are the position information in the real physical world, which can restore the photographed object to the position in the real world, and then the parking space line corner point and parking space entrance line detection are performed on the view cone point cloud features. The detection results of the parking space line corner point position and the parking space entrance line close to the parking space entrance line are more accurate, and the parking space line of the parking space finally determined is of high precision.

[0045] To facilitate understanding of this embodiment, a parking space line detection method under a bird's-eye view disclosed in an embodiment of the present invention is first introduced in detail.

[0046] Embodiment 1:

[0047] According to an embodiment of the present invention, an embodiment of a parking space line detection method under a bird's-eye view is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0048] Figure 2 FIG. 4 is a flow chart of a parking space line detection method under a bird's-eye view according to an embodiment of the present invention. Figure 2As shown, the method comprises the following steps:

[0049] Step S202, obtaining an original image captured by a surround view camera of the vehicle, and performing depth estimation and feature extraction on the original image to obtain depth features and image features;

[0050] In an embodiment of the present invention, the above-mentioned surround-view camera can be composed of multiple cameras, which are respectively installed at different positions of the vehicle. For example, the above-mentioned surround-view camera is composed of 4 cameras, which are respectively installed in the front, rear, left and right directions of the vehicle; the above-mentioned surround-view camera can also be composed of 6 cameras, which are respectively installed in the front, rear, upper left, lower left, upper right and lower right directions of the vehicle. The embodiment of the present invention does not specifically limit the above-mentioned surround-view camera.

[0051] Specifically, the number of original images is determined by the number of cameras. After the original image passes through the backbone and neck, a depth distribution score will be generated for each pixel, and then the depth feature of the corresponding pixel (specifically, it can be a depth value) is determined according to the depth distribution score of each pixel. At the same time, the original image is feature extracted to obtain image features (specifically, it can be five-layer features, that is, the features obtained by five-layer downsampling, with sizes of 8*8, 16*16, 32*32, 64*64, and 128*128 respectively).

[0052] Step S204, projecting the depth features and the image features into a real environment with the vehicle position as the origin of the coordinate system through a homography matrix to obtain a view frustum point cloud feature, wherein the homography matrix is ​​generated according to the internal and external parameters of the surround view camera;

[0053] Specifically, the process of projecting the depth features and image features into the real environment with the vehicle position as the origin of the coordinate system through the homography matrix (five images are obtained) is the process of view conversion, and then the view cone point cloud features are obtained.

[0054] Step S206, detecting parking space line corner points and parking space entrance lines on the view cone point cloud features, obtaining parking space line corner point positions and parking space entrance lines close to the parking space entrance lines, and determining paired parking space line corner point positions and parking space entrance line types close to the parking space entrance lines according to the parking space line corner point positions and parking space entrance lines close to the parking space entrance lines;

[0055] Specifically, the above-mentioned parking space line corner point and parking space entrance line detection can be specifically a common 2D detection head, such as: YOLO, Fcos, etc., that is, the parking space line corner point and parking space entrance line detection are performed on the view cone point cloud features through the 2D detection head to obtain the parking space line corner point position and parking space entrance line close to the parking space entrance line.

[0056] Step S208, determining the corner point positions of the paired parking lines far from the parking entrance line according to the corner point positions of the paired parking lines close to the parking entrance line and the parking entrance line type, and determining the parking lines of the parking spaces according to the corner point positions of the paired parking lines close to the parking entrance line and the corner point positions of the paired parking lines far from the parking entrance line.

[0057] In an embodiment of the present invention, a parking space line detection method under a bird's-eye view is provided, comprising: acquiring an original image taken by a surround-view camera of a vehicle, and performing depth estimation and feature extraction on the original image to obtain depth features and image features; projecting the depth features and image features into a real environment with a vehicle position as the origin of a coordinate system through a homography matrix to obtain a view cone point cloud feature, wherein the homography matrix is ​​generated according to internal and external parameters of the surround-view camera; performing parking space line corner point and parking space entrance line detection on the view cone point cloud feature to obtain parking space line corner point positions and parking space entrance lines close to the parking space entrance lines, and determining paired parking space line corner point positions and parking space entrance line types close to the parking space entrance lines according to the parking space line corner point positions and the parking space entrance lines; determining paired parking space line corner point positions far from the parking space entrance lines according to the paired parking space line corner point positions and the parking space entrance line types, and determining parking space lines of parking spaces according to the paired parking space line corner point positions close to the parking space entrance lines and the paired parking space line corner point positions far from the parking space entrance lines. From the above description, it can be seen that in the parking space line detection method under the bird's-eye view of the present invention, the depth features and image features are projected into the real environment with the vehicle position as the origin of the coordinate system through the homography matrix, and the obtained view cone point cloud features are the position information in the real physical world, which can restore the photographed object to the position in the real world, and then the parking space line corner points and parking space entrance line detection are performed on the view cone point cloud features. The detection results of the parking space line corner point positions and parking space entrance lines close to the parking space entrance lines are more accurate, and the parking space lines of the parking spaces finally determined are highly accurate, which alleviates the technical problem that the traditional IPM-based parking space line detection method, when the cameras of the vehicle body are not consistent in level, causes the captured images to have distance or splicing errors, which in turn leads to the poor accuracy of the parking space lines finally detected.

[0058] The above content briefly introduces the parking space line detection method under the bird's-eye view of the present invention, and the specific contents involved are described in detail below.

[0059] In an optional embodiment of the present invention, the depth features and image features are projected into a real environment with the vehicle position as the origin of the coordinate system through a homography matrix, which specifically includes the following steps:

[0060] (1) The depth features and image features are transformed through the homography matrix to obtain the bird's-eye view features;

[0061] (2) Process the bird's-eye view features to obtain the view cone point cloud features.

[0062] Specifically, the process of step S202 to step S204 is actually a method of BEV perception - Lift-Splat-Shoot (LSS). LSS is a depth estimation method that generates depth features and then fuses them with image features. Then, a homography matrix is ​​generated through the internal and external parameters of the camera to restore the original position and depth of the object in the real world. In layman's terms, it is to restore the object photographed by the camera to its real position with the vehicle position as the origin of the coordinate system. The LSS transformation is divided into three parts. The first step is to generate depth features and image features. The second step is to convert the depth features and image features through the homography matrix to obtain the bird's-eye view features. The third step is to process the bird's-eye view features to obtain the cone point cloud features.

[0063] In an optional embodiment of the present invention, determining the corner point positions of the paired parking space lines and the parking space entrance line types close to the parking space entrance line according to the corner point positions of the parking space lines close to the parking space entrance line and the parking space entrance line specifically includes the following steps:

[0064] (1) Use geometric clues to match the corner points of parking space lines close to the parking space entrance lines to obtain the corner points of pairs of parking space lines close to the parking space entrance lines;

[0065] Specifically, Figure 3 As shown, the parking space entrance line type is shown, and the parking space entrance line type includes any of the following: a parking space entrance line of a vertical parking space type, a parking space entrance line of a parallel parking space type, and a parking space entrance line of an oblique parking space type. The parking space is composed of four parking space line corner points, two of which are paired parking space line corner points (red points) close to the parking space entrance line, and the other two parking space line corner points (yellow points) are usually not visible in the surround image due to visual limitations.

[0066] After the parking space line corner points close to the parking space entrance line are detected, the positions of the parking space line corner points close to the parking space entrance line are further matched using geometric clues to obtain the positions of the paired parking space line corner points close to the parking space entrance line. Specifically, the positions of the paired parking space line corner points close to the parking space entrance line are determined based on the relationship between the distance between two adjacent parking space line corner points close to the parking space entrance line and the geometric clue (e.g., a preset distance).

[0067] (2) The parking space entrance line type is determined based on the positions of the paired parking space line corner points close to the parking space entrance line and the parking space entrance line.

[0068] Specifically, if the distance between the corner points of the paired parking lines close to the parking space entrance line is small, and the parking space entrance line is a parking space entrance line with a vertical angle, then the parking space entrance line type is determined to be a parking space entrance line of a vertical parking space type; if the distance between the corner points of the paired parking lines close to the parking space entrance line is large, and the parking space entrance line is a parking space entrance line with a vertical angle, then the parking space entrance line type is determined to be a parking space entrance line of a parallel parking space type; if the distance between the corner points of the paired parking lines close to the parking space entrance line is small, and the parking space entrance line is not a parking space entrance line with a vertical angle, then the parking space entrance line type is determined to be a parking space entrance line of an oblique parking space type.

[0069] In an optional embodiment of the present invention, determining the positions of the corner points of the paired parking space lines far from the parking space entrance line according to the positions of the corner points of the paired parking space lines close to the parking space entrance line and the types of the parking space entrance lines specifically includes the following steps:

[0070] (1) Determine the parking space entrance line angle and parking space distance according to the parking space entrance line type;

[0071] Specifically, Figure 4 As shown, the relationship between the parking space entrance line type, the parking space entrance line angle, and the parking space distance is shown. This relationship is actually the geometric clues and prior knowledge of the parking space.

[0072] (2) The positions of the corner points of the paired parking space lines far from the parking space entrance lines are calculated based on the parking space entrance line angle, the parking space distance, and the corner point positions of the paired parking space lines close to the parking space entrance lines.

[0073] Specifically, the formula is calculated based on the position of the corner points of the paired parking lines away from the parking entrance line Calculate the corner point positions of the paired parking lines away from the parking entrance line, where p3 and p4 represent the corner point positions of the paired parking lines away from the parking entrance line, and α i represents the parking space entrance line angle, p1 and p2 represent the positions of the corner points of the paired parking space lines close to the parking space entrance line, and d i Indicates the parking space distance.

[0075] Specifically, Figure 4 As shown, the above formula can be used to calculate the corner point positions p3 and p4 of the paired parking space lines away from the parking space entrance line.

[0076] In an optional embodiment of the present invention, the corner points of the parking space line and the parking space entrance line are detected based on the view cone point cloud features, which specifically includes the following steps:

[0077] The target detection model is used to detect the parking space line corner points and parking space entrance lines based on the view cone point cloud features, and the parking space line corner point positions and parking space entrance lines close to the parking space entrance lines are obtained.

[0078] The parking space line detection method under the bird's-eye view of the present invention solves the horizontal dependence of the traditional vehicle body IPM holographic image, that is, the inconsistency of the level of the cameras of the vehicle body will cause the captured image to have errors in distance or splicing. Through the process method of the present invention, the captured object can be restored to the position in the real world and the detection result can be obtained. The best use scenario is the parking space line detection during smart parking. Due to the high accuracy requirements, the traditional IPM detection effect is not up to standard. This invention can obtain the real distance between the parking space and the vehicle, with higher accuracy, and the pure visual detection based on the camera is lower in cost than the radar.

[0079] In addition, the IPM recognition method currently used in the market in the field of intelligent parking is extremely dependent on the plane assumption, that is, the body cameras must be on the same plane to obtain a complete surround view stitching. This method is not very effective for detecting parking lines. The present invention uses a traditional image detection algorithm combined with LSS perception to predict the cone point cloud of the parking line, and then restores it to the real world for target detection, thereby solving the above-mentioned horizontal dependence problem.

[0080] Embodiment 2:

[0081] An embodiment of the present invention also provides a parking space line detection device under a bird's-eye view. The parking space line detection device under a bird's-eye view is mainly used to execute the parking space line detection method under a bird's-eye view provided in Embodiment 1 of the present invention. The following is a specific introduction to the parking space line detection device under a bird's-eye view provided in the embodiment of the present invention.

[0082] Figure 5 is a schematic diagram of a parking space line detection device under a bird's eye view according to an embodiment of the present invention, such as Figure 5 As shown, the device includes: a depth estimation and feature extraction unit 10, a projection unit 20, a detection unit 30 and a determination unit 40, wherein:

[0083] A depth estimation and feature extraction unit, used to obtain the original image taken by the vehicle's surround view camera, and perform depth estimation and feature extraction on the original image to obtain depth features and image features;

[0084] A projection unit, used to project the depth features and the image features into a real environment with the vehicle position as the origin of the coordinate system through a homography matrix to obtain a view cone point cloud feature, wherein the homography matrix is ​​generated according to the internal and external parameters of the surround view camera;

[0085] A detection unit is used to detect parking space line corner points and parking space entrance lines based on the view cone point cloud features, obtain the parking space line corner point positions and parking space entrance lines close to the parking space entrance lines, and determine the paired parking space line corner point positions and parking space entrance line types close to the parking space entrance lines based on the parking space line corner point positions and parking space entrance lines close to the parking space entrance lines;

[0086] A determination unit is used to determine the corner point positions of the paired parking lines far from the parking entrance line according to the corner point positions of the paired parking lines close to the parking entrance line and the parking entrance line type, and to determine the parking lines of the parking spaces according to the corner point positions of the paired parking lines close to the parking entrance line and the corner point positions of the paired parking lines far from the parking entrance line.

[0087] In an embodiment of the present invention, a parking space line detection device under a bird's-eye view is provided, comprising: acquiring an original image taken by a surround-view camera of a vehicle, and performing depth estimation and feature extraction on the original image to obtain depth features and image features; projecting the depth features and image features into a real environment with a vehicle position as the origin of a coordinate system through a homography matrix to obtain a view cone point cloud feature, wherein the homography matrix is ​​generated according to internal and external parameters of the surround-view camera; performing parking space line corner point and parking space entrance line detection on the view cone point cloud feature to obtain parking space line corner point positions and parking space entrance lines close to the parking space entrance lines, and determining paired parking space line corner point positions and parking space entrance line types close to the parking space entrance lines according to the parking space line corner point positions and the parking space entrance lines; determining paired parking space line corner point positions far from the parking space entrance lines according to the paired parking space line corner point positions and the parking space entrance line types, and determining the parking space lines of the parking spaces according to the paired parking space line corner point positions close to the parking space entrance lines and the paired parking space line corner point positions far from the parking space entrance lines. From the above description, it can be known that in the parking space line detection device under the bird's-eye view of the present invention, the depth features and image features are projected into the real environment with the vehicle position as the origin of the coordinate system through the homography matrix, and the obtained view cone point cloud features are the position information in the real physical world, which can restore the photographed object to the position in the real world, and then the parking space line corner points and parking space entrance line detection are performed on the view cone point cloud features. The detection results of the parking space line corner point positions and parking space entrance lines obtained close to the parking space entrance lines are more accurate, and the parking space lines of the parking spaces finally determined are of high accuracy, which alleviates the technical problem that the traditional IPM-based parking space line detection method, when the cameras of the vehicle body are inconsistent in level, causes the captured images to have distance or splicing errors, thereby resulting in poor accuracy of the parking space lines finally detected.

[0088] Optionally, the projection unit is further used to: transform the depth feature and the image feature through a homography matrix to obtain a bird's-eye view feature; and process the bird's-eye view feature to obtain a view cone point cloud feature.

[0089] Optionally, the detection unit is also used to: use geometric clues to match the corner point positions of parking space lines close to the parking space entrance line to obtain paired corner point positions of parking space lines close to the parking space entrance line; determine the parking space entrance line type based on the paired corner point positions of parking space lines close to the parking space entrance line and the parking space entrance line.

[0090] Optionally, the parking space entrance line type includes any one of the following: a parking space entrance line of a vertical parking space type, a parking space entrance line of a parallel parking space type, and a parking space entrance line of an oblique parking space type.

[0091] Optionally, the determination unit is also used to: determine the parking space entrance line angle and the parking space distance according to the parking space entrance line type; calculate the corner point positions of the paired parking space lines far away from the parking space entrance line according to the parking space entrance line angle, the parking space distance and the corner point positions of the paired parking space lines close to the parking space entrance line.

[0092] Optionally, the determining unit is further configured to: calculate the formula according to the positions of the corner points of the paired parking space lines away from the parking space entrance line Calculate the corner point positions of the paired parking lines away from the parking entrance line, where p3 and p4 represent the corner point positions of the paired parking lines away from the parking entrance line, and α i represents the parking space entrance line angle, p1 and p2 represent the positions of the corner points of the paired parking space lines close to the parking space entrance line, and d i Indicates the parking space distance.

[0094] Optionally, the detection unit is further used to: use a target detection model to detect parking space line corner points and parking space entrance lines on the view cone point cloud features to obtain the parking space line corner point positions and parking space entrance lines close to the parking space entrance lines.

[0095] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0096] like Figure 6 As shown, an electronic device 600 provided in an embodiment of the present application includes: a processor 601, a memory 602 and a bus, wherein the memory 602 stores machine-readable instructions executable by the processor 601. When the electronic device is running, the processor 601 communicates with the memory 602 through the bus, and the processor 601 executes the machine-readable instructions to perform the steps of the parking space line detection method under the above-mentioned bird's-eye view.

[0097] Specifically, the above-mentioned memory 602 and processor 601 can be general-purpose memories and processors, which are not specifically limited here. When the processor 601 runs the computer program stored in the memory 602, it can execute the above-mentioned parking space line detection method under the bird's-eye view.

[0098] The processor 601 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 601. The above processor 601 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 602, and the processor 601 reads the information in the memory 602 and completes the steps of the above method in combination with its hardware.

[0099] Corresponding to the above-mentioned parking space line detection method under the bird's-eye view, an embodiment of the present application also provides a computer-readable storage medium, which stores machine-executable instructions. When the computer-executable instructions are called and executed by the processor, the computer-executable instructions prompt the processor to execute the steps of the above-mentioned parking space line detection method under the bird's-eye view.

[0100] The parking space line detection device under the bird's-eye view provided in the embodiment of the present application can be specific hardware on the device or software or firmware installed on the device. The implementation principle and technical effects of the device provided in the embodiment of the present application are the same as those of the aforementioned method embodiment. For the sake of brief description, the parts not mentioned in the device embodiment can refer to the corresponding contents in the aforementioned method embodiment. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0101] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0102] For another example, the flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the device, method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0103] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0104] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0105] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or partly contribute to the prior art or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions for an electronic device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the vehicle marking method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0106] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0107] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solution of the present application, rather than to limit it. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the aforementioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solution recorded in the aforementioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solution deviate from the scope of the technical solution of the embodiment of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A parking space line detection method under a bird's-eye view, characterized in that: include: Acquire an original image captured by a surround-view camera of the vehicle, and perform depth estimation and feature extraction on the original image to obtain depth features and image features; Projecting the depth feature and the image feature into a real environment with the vehicle position as the origin of the coordinate system through a homography matrix to obtain a view cone point cloud feature, wherein the homography matrix is ​​generated according to the internal and external parameters of the surround view camera; Performing parking space line corner point and parking space entrance line detection on the view cone point cloud features to obtain parking space line corner point positions and parking space entrance lines close to the parking space entrance lines, and determining paired parking space line corner point positions and parking space entrance line types close to the parking space entrance lines according to the parking space line corner point positions and the parking space entrance lines; The corner point positions of the paired parking lines close to the parking entrance line and the parking entrance line type are determined, and the parking lines of the parking spaces are determined according to the corner point positions of the paired parking lines close to the parking entrance line and the corner point positions of the paired parking lines far from the parking entrance line.

2. The method according to claim 1, characterized in that: Projecting the depth feature and the image feature into a real environment with the vehicle position as the origin of the coordinate system through a homography matrix, including: The depth feature and the image feature are converted by using the homography matrix to obtain a bird's-eye view feature; The bird's-eye view features are processed to obtain the view cone point cloud features.

3. The method according to claim 1, characterized in that Determining the positions of the corner points of the parking space lines close to the parking space entrance lines and the types of the parking space entrance lines according to the corner points of the parking space lines close to the parking space entrance lines and the parking space entrance lines includes: Using geometric clues to match the corner point positions of the parking space lines close to the parking space entrance lines to obtain the paired corner point positions of the parking space lines close to the parking space entrance lines; The parking space entrance line type is determined according to the positions of the paired parking space line corner points close to the parking space entrance line and the parking space entrance line.

4. The method according to claim 3, characterized in that: The parking space entrance line type includes any one of the following: a parking space entrance line of a vertical parking space type, a parking space entrance line of a parallel parking space type, and a parking space entrance line of an oblique parking space type.

5. The method according to claim 1, characterized in that Determining the positions of the corner points of the paired parking space lines far from the parking space entrance line according to the positions of the corner points of the paired parking space lines close to the parking space entrance line and the types of the parking space entrance lines includes: Determine the parking space entrance line angle and the parking space distance according to the parking space entrance line type; The positions of the corner points of the paired parking space lines far from the parking space entrance line are calculated according to the parking space entrance line angle, the parking space distance and the corner point positions of the paired parking space lines close to the parking space entrance line.

6. The method according to claim 5, characterized in that Calculating the positions of the corner points of the paired parking space lines far from the parking space entrance line according to the parking space entrance line included angle, the parking space distance and the corner point positions of the paired parking space lines close to the parking space entrance line includes: Calculate the formula based on the corner position of the paired parking space lines away from the parking space entrance line Calculate the corner point positions of the paired parking lines away from the parking entrance line, where p3 and p4 represent the corner point positions of the paired parking lines away from the parking entrance line, α i represents the parking space entrance line included angle, p1 and p2 represent the positions of the corner points of the paired parking space lines close to the parking space entrance line, and d i Indicates the distance to the parking space.

7. The method according to claim 1, characterized in that Performing parking space line corner point detection and parking space entrance line detection on the view cone point cloud features includes: The target detection model is used to detect the parking space line corner points and the parking space entrance line of the view cone point cloud features to obtain the parking space line corner point position close to the parking space entrance line and the parking space entrance line.

8. A parking space line detection device under a bird's eye view, characterized in that: include: A depth estimation and feature extraction unit, used to obtain an original image taken by a surround view camera of the vehicle, and perform depth estimation and feature extraction on the original image to obtain depth features and image features; A projection unit, configured to project the depth feature and the image feature into a real environment with the vehicle position as the origin of a coordinate system through a homography matrix to obtain a view cone point cloud feature, wherein the homography matrix is ​​generated according to the internal and external parameters of the surround view camera; a detection unit, configured to detect parking space line corner points and parking space entrance lines on the view cone point cloud features, obtain parking space line corner point positions and parking space entrance lines close to the parking space entrance lines, and determine paired parking space line corner point positions and parking space entrance line types close to the parking space entrance lines according to the parking space line corner point positions and the parking space entrance lines; A determination unit is used to determine the corner point positions of the paired parking lines far from the parking entrance line according to the corner point positions of the paired parking lines close to the parking entrance line and the parking entrance line type, and to determine the parking lines of the parking spaces according to the corner point positions of the paired parking lines close to the parking entrance line and the corner point positions of the paired parking lines far from the parking entrance line.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to execute the method according to any one of claims 1 to 7.

Images