A two-stage parking line detection method for stereo parking space

Abstract

The application belongs to the technical field of automatic parking, and particularly relates to a parking line detection method for a three-dimensional parking space. A two-stage parking line detection method for a three-dimensional parking space is provided, which divides automatic parking into two stages of searching for a parking space and parking. In the stage of searching for a parking space, a parking space detection model based on deep learning is used to search and detect all parking spaces in a current frame image, and the parking spaces located in a credible interval are marked as valid parking spaces. After a driver selects a target parking space, the stage of automatic parking is started. In the stage of parking, visual perception no longer detects parking spaces, and only detects parking lines. After the edges of the parking lines are detected, distortion correction is performed on the edges. In the stage of searching for a parking space, the credible interval is set, so that the error of the parking space is reduced. In the stage of parking, control and correction are performed in real time according to the latest parking line position, so that accurate parking line positioning is obtained.

Description

Technical Field

[0001] This invention belongs to the field of automatic parking technology, and specifically relates to a parking line detection method for multi-level parking spaces. Background Technology

[0002] With increasingly scarce parking spaces, multi-level parking spaces, which occupy more space, have emerged in many places. However, because multi-level parking spaces are much narrower than flat parking spaces, parking becomes significantly more difficult. Traditional flat parking generally uses inverse perspective transformation (IPM) technology, which relies on the flatness and grounding of the ground, to create a panoramic top view. Parking spaces are then detected in this panoramic top view and converted to the vehicle's coordinate system for downstream tasks.

[0003] One of the challenges of automated parking systems for multi-level parking spaces is that these spaces typically have a certain height. Directly using the results from a panoramic top-down view is clearly impractical for multi-level parking spaces with height. This is because panoramic stitching algorithms are based on inverse perspective transformation (IPM), which can only represent information about the ground plane. Targets with a certain height will cause distortion in the image, so it's necessary to assume the ground is flat and the target is grounded. For typical flat parking spaces, the ground is generally flat and grounded, so directly identifying and outputting the parking space from the panoramic image is feasible. However, multi-level parking spaces typically have a certain height, which means that directly detecting the parking space in the panoramic view and converting it from pixel coordinates to vehicle coordinates will result in significant deviations in the output parking space coordinates. Summary of the Invention

[0004] The purpose of this invention is to provide a two-stage parking line detection method for multi-level parking spaces in order to improve the accuracy of the final parking position.

[0005] The technical solution of this invention is: a two-stage parking line detection method for three-dimensional parking spaces. This method divides automatic parking into two stages: searching for parking spaces and parking. Stage 1: In the parking space search stage, a deep learning-based parking space detection model searches for and detects all parking spaces in the current frame image, and marks the parking spaces located in the confidence interval as valid parking spaces. After the driver selects the corresponding target parking space, the automatic parking stage begins. Stage 2: In the parking stage, visual perception no longer performs parking space detection, but only detects parking lines. After detecting the edges of the parking lines, distortion correction is performed on them.

[0006] Specifically, in the above scheme, stage one includes the following steps:

[0007] S1.1 Surround view stitching;

[0008] Images are acquired from four onboard surround-view fisheye cameras and stitched together to obtain a surround-view image from the BEV perspective.

[0009] S1.2 Parking space inspection;

[0010] Get all parking spaces in the current frame image;

[0011] S1.3 Coordinate Transformation;

[0012] Transform the coordinates of the parking space in the surround view stitched image to the vehicle's coordinate system;

[0013] S1.4 Confidence Interval Determination;

[0014] Based on the experiment, parking spaces whose four corner coordinates are located within the set range of the vehicle coordinate system are defined as valid parking spaces.

[0015] S1.5 Target Parking Space Selection;

[0016] Displays all available parking spaces, allowing the driver to select their desired parking space.

[0017] Specifically, in the above scheme, stage two includes the following steps:

[0018] S2.1 The control system returns to the target parking space location;

[0019] The system records the location of the parking space and outputs the latest location of the target parking space in real time based on the movement of the vehicle.

[0020] S2.2 Cropping a portion of the target parking space image;

[0021] Images are acquired from four onboard surround-view fisheye cameras and stitched together to obtain a surround-view image from the BEV perspective. The location of the target parking space is determined in the surround-view image. Based on the location of the target parking space, the width is expanded by N pixels in all directions (up, down, left, and right) to obtain a local image of the target parking space. This local image is then cropped.

[0022] S2.3 Parking space edge detection;

[0023] The local image was detected using a deep learning-based relocation line detection model and a parking space edge-assisted detection model based on Hough transform, respectively.

[0024] The detection results of the repositioning line detection model are used as the screening criteria for the detection results of the parking space edge auxiliary detection model. If the parking space edge auxiliary detection model detects a straight line that meets the screening criteria, the average value of the coordinates of the two ends of the selected straight line is taken as the detection repositioning line. If the parking space edge auxiliary detection model does not detect a straight line that meets the screening criteria, the detection results of the repositioning line detection model are used as the detection repositioning line of the parking space edge.

[0025] S2.4 corrects the distortion of the detection repositioning line;

[0026] Correct the distortion caused by the left or right detection repositioning line and determine the four corner points at the actual repositioning line entry line.

[0027] S2.5 coordinate transformation;

[0028] The four corner points of the actual repositioning line are transformed from the pixel coordinate system to the vehicle coordinate system and determined as the left and right edges of the target parking space.

[0029] Beneficial Effects: This invention divides automatic parking into two stages: searching for parking spaces and parking. Compared to saving the parking line positions output by the parking space detection model only during the searching stage, this invention allows for real-time control and correction based on the latest parking line positions throughout the parking stage, resulting in more accurate parking line positioning. Furthermore, by setting a confidence interval during the searching stage and further correcting distortion in the repositioning line during the parking stage, this invention reduces parking space errors. Testing shows that the method described in this invention achieves a parking success rate of over 95% in the tested range of multi-level parking spaces. Attached Figure Description

[0030] Figure 1 This is a flowchart of the present invention.

[0031] Figure 2 This is a flowchart of stage one of the present invention.

[0032] Figure 3 This is a schematic diagram showing the reliable interval of the parking space and the relative position of the vehicle.

[0033] Figure 4 This is a flowchart of stage two of the present invention.

[0034] Figure 5 This is the error map output for parking space edge detection in S2.3.

[0035] Figure 6 This is a schematic diagram of the correction of distortion in the detection relocation line in S2.4. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0037] See appendix Figure 1A two-stage parking line detection method for three-dimensional parking spaces is proposed. This method divides automatic parking into two stages: searching for parking spaces and parking. Stage 1: The parking space search stage uses a deep learning-based parking space detection model to search and detect all parking spaces in the current frame image, and marks the parking spaces located in the confidence interval as valid parking spaces. After the driver selects the corresponding target parking space, the automatic parking stage begins. Stage 2: In the parking stage, visual perception no longer performs parking space detection, but only detects parking lines. After detecting the edges of the parking lines, distortion correction is performed on them.

[0038] See appendix Figure 2 Phase one includes the following steps:

[0039] S1.1 Surround view stitching.

[0040] Images are acquired from four onboard surround-view fisheye cameras and stitched together to create a surround-view image from the BEV's perspective.

[0041] Four fisheye cameras are installed on the front, rear, left, and right sides of the vehicle. These four cameras perceive the surrounding environment and use image stitching to merge the images into a panoramic top-down view. This top-down view displays information around the vehicle in a 360-degree field of view, eliminating blind spots and dead angles.

[0042] The steps for generating a panoramic top view are as follows:

[0043] 1) Perform distortion correction on the image.

[0044] 2) Perform inverse perspective transformation (IPM) on the image, that is, obtain the homography matrix from the camera plane to the ground plane through the calibration of the intrinsic and extrinsic parameters of multiple cameras, and realize the plane-to-plane transformation.

[0045] 3) Perform multi-view image stitching.

[0046] S1.2 Parking space detection.

[0047] Get all parking spaces in the current frame image.

[0048] In the panoramic top-down view, due to image distortion and errors during calibration, the three-dimensional parking space appears as a quadrilateral rather than a rectangle, and the location of the parking space varies across different positions in the panoramic top-down view. Therefore, general object detection methods and rotated rectangle object detection methods cannot be used. In this example, based on the parking space shape in the panoramic top-down view, an object detection method that detects the four corner points of the parking space is used to detect the parking space.

[0049] S1.3 Coordinate Transformation.

[0050] Transform the coordinates of the parking space in the surround view stitched image to the vehicle's coordinate system.

[0051] S1.4 Confidence interval determination.

[0052] For objects with height, significant deformation occurs in the panoramic top view. This leads to substantial deviations in the parking space coordinates output when directly detecting parking spaces in the panoramic view and converting them from pixel coordinates to vehicle coordinates. Therefore, a confidence interval is set here; when a parking space is within this interval, it is considered a parkable location.

[0053] The confidence interval was determined experimentally.

[0054] Extend the two sides of the automated parking space to the ground and place markers on the ground to measure the error between the edge of the automated parking space in the surround view and the actual edge of the automated parking space.

[0055] By saving multiple consecutive frames of images in the above method and observing the parking space edge in the panoramic top view and the actual parking space edge, it was found that: 1) Targets near the plane that is perpendicular to the ground plane and passes through the camera's optical axis have very small vertical distance offset in the panoramic top view, with basically no offset; 2) The deformation of objects in the image is related to their height and distance from the camera. The higher the height, the greater the deformation, and the farther the distance, the greater the deformation.

[0056] Based on measured values, the distortion at the bottom edge of the parking space in the top view (1024x1024) with pixel values ​​between 230-250 is determined to be relatively small compared to the actual parking space edge pixel difference. Since the actual vehicle width cannot be determined due to parking space distortion, a survey of commercially available multi-level parking garages shows widths generally between 2050-2130mm. The average width of the surveyed parking spaces, 2080mm, is taken as the fixed parking space width. The corresponding pixel values ​​and coordinates are transformed to the vehicle coordinate system (with the vehicle center as the origin) using a calibration homography matrix. It is determined that when the ordinates of the four corner points of the parking space are within the interval [442, 3122], the parking space error is small, less than 50mm. Therefore, the interval [442, 3122] is taken as the reliable parking space interval. When the parking space is within this interval, the vehicle is considered parkable and output to the downstream task. The relative position of the reliable parking space interval and the vehicle is as follows: Figure 3 As shown.

[0057] S1.5 Target parking space selection.

[0058] The app displays all available parking spaces, allowing drivers to select their desired parking spot.

[0059] See appendix Figure 4 Phase two includes the following steps:

[0060] S2.1 The system returns to the target parking space location.

[0061] The system records the location of the parking space and outputs the latest location of the target parking space in real time based on the movement of the vehicle.

[0062] S2.2 Cropping a portion of the target parking space image.

[0063] Images are acquired from four onboard surround-view fisheye cameras and stitched together to obtain a surround-view image from the BEV perspective. The location of the target parking space is determined in the surround-view image. Based on the location of the target parking space, the image is expanded by N pixels in all directions (up, down, left, and right) to create a local image of the target parking space. This local image is then cropped.

[0064] S2.3 Parking space edge detection.

[0065] The local image was detected using a deep learning-based relocation line detection model and a parking space edge-assisted detection model based on Hough transform, respectively.

[0066] The detection results of the repositioning line detection model are used as the screening criteria for the detection results of the parking space edge auxiliary detection model. The specific screening criteria are: whether the angle difference between the detection results of the parking space edge auxiliary detection model and the detection results of the repositioning line detection model is within 3 degrees and the distance difference is within 3 pixels.

[0067] If the parking space edge auxiliary detection model detects a straight line that meets the screening criteria, the average of the coordinates of the two ends of the selected straight line is taken as the detection relocation line; if the parking space edge auxiliary detection model does not detect a straight line that meets the screening criteria, the detection result of the relocation line detection model is used as the detection relocation line of the parking space edge.

[0068] S2.4 corrects the distortion of the detection repositioning line.

[0069] Because parking spaces have a certain height, they will be distorted in the panoramic top view. Therefore, edge detection of parking spaces performed in the panoramic top view will inevitably result in a certain error in the output. This error is as follows: Figure 5 As shown. If no correction is made, the vehicle will drift to one side during the parking process, resulting in parking failure.

[0070] Correct the distortion caused by the left or right detection repositioning line, and determine the four corner points at the actual repositioning line entry line.

[0071] See appendix Figure 6 The correction method is as follows:

[0072] S2.4.1 The average value of the angle between the detection repositioning line and the positive x-axis of the vehicle is taken as the repositioning line orientation θ.

[0073] S2.4.2 The difference between the length of the detection repositioning line entering the warehouse and the length of the upstream given parking space entrance is taken as the total error, error = error1 + error2; where error1 is the error between the left detection repositioning line and the left actual repositioning line, and error2 is the error between the right detection repositioning line and the right actual repositioning line.

[0074] S2.4.3 Calculate error1 and error2.

[0075]

[0076] Where: P lx , P′ lx P rx , P′ rx P cx These represent the x-coordinates of the points to the left of the actual repositioning line entry line, the x-coordinates of the points to the left of the detected repositioning line, the x-coordinates of the points to the right of the actual repositioning line entry line, the x-coordinates of the points to the right of the detected repositioning line, and the x-coordinates of the camera, respectively. dist() represents calculating the distance.

[0077] Similarly, error2 is obtained.

[0078] S2.4.4 Determine the corner points.

[0079] P′ lx Moving error1 to the right yields the left front point P of the actual relocation line entry line. lx , will P lx Extend the repositioning line a certain distance toward θ to obtain the left rear endpoint M of the actual repositioning line entrance line.

[0080] Similarly, from P′ rx Obtain the right front end point P of the actual repositioning line entry line. rx , will P rx Extend the repositioning line a certain distance toward θ to obtain the right rear endpoint N of the actual repositioning line entrance line.

[0081] S2.5 coordinate transformation.

[0082] The four corner points of the actual repositioning line are transformed from the pixel coordinate system to the vehicle coordinate system, and determined as the left and right edges of the target parking space, which are then output to the downstream task.

[0083] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A two-stage lane detection method for stereo parking, characterized in that, The method divides automatic parking into two stages of searching for a parking space and parking in; stage one: a parking space searching stage searches for and detects all parking spaces in a current frame image by a parking space detection model based on deep learning, labels parking spaces in a trusted interval as valid parking spaces, and after a driver selects a target parking space, starts a parking stage; stage two: in the parking stage, visual perception no longer detects parking spaces, only detects parking lines, and after detecting parking line edges, corrects distortion of the parking line edges; The stage two includes the following steps: S2.

1. Regulating and returning to a target parking space position; The regulation records the position of the parking space and outputs a latest position of the target parking space in real time according to movement of the vehicle; S2.

2. Cropping a local image of the target parking space; An image is obtained from four ring-view fisheye cameras on the vehicle, a ring-view spliced image in a BEV perspective is obtained through splicing and fusion, the position of the target parking space is determined in the ring-view spliced image, a local image of the target parking space is expanded by N pixel widths upward, downward, leftward and rightward based on the position of the target parking space as a reference, and the local image is cropped; S2.

3. Parking edge detection; A repositioning line detection model based on deep learning and a parking edge auxiliary detection model based on Hough transformation are used to detect the local image respectively; The detection result of the repositioning line detection model is used as a screening condition of the detection result of the parking edge auxiliary detection model, if a straight line meeting the screening condition is detected by the parking edge auxiliary detection model, the average of the coordinates of the two ends of the straight line is taken as a detection repositioning line, and if the straight line meeting the screening condition is not detected by the parking edge auxiliary detection model, the detection result of the repositioning line detection model is used as the detection repositioning line of the parking edge; S2.

4. Correcting distortion of the detection repositioning line; Distortion of the left detection repositioning line or the right detection repositioning line is corrected to determine four corner points at an entrance line of an actual repositioning line; S2.4.

1. Taking the average of the angles between the detection repositioning line and the positive direction of the x-axis of the vehicle as the orientation θ of the repositioning line; S2.4.

2. Taking the difference between the length of the detection repositioning line and the length of the entrance of the given parking space upstream as a total error error, error = error1 + error2; wherein error1 is the error of the left detection repositioning line and the left actual repositioning line, and error2 is the error of the right detection repositioning line and the right actual repositioning line; S2.4.

3. Calculating error1 and error2; wherein: P lx P′ lx P rx P′ rx P cx respectively represent the actual repositioning line entry line left side point horizontal coordinate, the detected repositioning line left side point horizontal coordinate, the actual repositioning line entry line right side point horizontal coordinate, the detected repositioning line right side point horizontal coordinate, the camera horizontal coordinate, and dist() represents the distance. Similarly, error2 is obtained; S2.4.

4. Determining the corner points; P' lx Moving right by errorl gives the left front end point P of the actual repositioning line entry line lx , P lx Extending along the repositioning line a certain distance towards theta gives the left back end point M of the actual repositioning line entry line; Similarly, the right end point P' of the actual repositioning line entry line is obtained by extending P rx rx rx the right end point N of the actual repositioning line entry line is obtained by extending P​​ S2.

5. Coordinate conversion; The four corner points of the actual repositioning line are converted from a pixel coordinate system to a vehicle coordinate system to determine left and right boundary lines of the target parking space.

2. The two-stage lane detection method for stereo parking space according to claim 1, wherein, The stage one includes the following steps: S1.

1. Ring-view splicing; Images are obtained from four ring-view fisheye cameras on the vehicle, and a ring-view spliced image in a BEV perspective is obtained through splicing and fusion; S1.

2. Parking space detection; All parking spaces in a current frame image are obtained; S1.

3. Coordinate conversion; Coordinates of the parking spaces in the ring-view spliced image are converted to a vehicle coordinate system; S1.

4. Trusted interval judgment; According to the experiment, the parking space with the longitudinal coordinate of the four corner points located in the set interval of the ego vehicle coordinate system is set as an effective parking space; S1.5 target parking space selection; Display all effective parking spaces, and select a target parking space by the driver.

3. The two-stage lane detection method for stereo parking space according to claim 2, wherein, In S1.4, the trusted interval is set to the longitudinal direction [442, 3122] in the ego vehicle coordinate system, with the unit of mm.

4. The two-stage lane detection method for stereo parking space according to claim 1, wherein, The screening condition in S2.3 is that whether the angle difference between the detection result of the parking space edge auxiliary detection model and the detection result of the reposition line detection model is within 3 degrees, and the distance difference is within 3 pixels.

Images