A two-stage parking space line detection method for planar parking spaces

Abstract

The application belongs to the technical field of automatic parking, and particularly relates to a parking line detection method for a flat parking space. A two-stage parking line detection method for a flat parking space, the technical scheme of which is that the method divides automatic parking into two stages of searching for a parking space and parking; stage one: the searching for a parking space stage is searched and detected by a parking space detection model based on deep learning to search for an effective parking space, and after a driver selects a corresponding target parking space, the automatic parking stage is started; stage two: in the parking stage, visual perception no longer performs parking space detection, and only image processing is performed on parking line detection. Compared with a parking line position saved only in the searching for a parking space stage and output by a parking space detection model, the application can control and correct in real time according to the latest parking line position in the whole parking stage, so that more accurate parking line positioning can be obtained, and the final parking position is more accurate. Moreover, the application also reduces the requirement for the accuracy of the parking space detection model.

Description

Technical Field

[0001] This invention belongs to the field of automatic parking technology, and specifically relates to a parking line detection method for planar parking spaces. Background Technology

[0002] Finding suitable parking spaces and judging the distance and direction of the vehicle relative to its surroundings requires drivers to invest a significant amount of time and effort in practice and mastery; especially in complex and narrow parking lots or road sections, minor collisions and other traffic accidents are easily caused. Automatic parking technology, on the other hand, uses onboard sensors (such as ultrasonic radar and cameras) to perceive and detect the environment and obstacles around the vehicle, and automatically plans and adjusts the vehicle's direction and speed to avoid unnecessary collisions and damage. The detection and recognition of marked parking spaces in automatic parking technology primarily relies on cameras, directly detecting valid parking spaces on the image—a vision-based parking space detection. By fusing this with ultrasonic radar installed on the vehicle, a superior parking experience can be achieved.

[0003] Visual parking space detection can be broadly categorized into two types: those based on traditional image processing algorithms and those based on deep learning algorithms. Visual parking space detection based on traditional image processing algorithms has low hardware performance requirements but is significantly affected by the environment and has poor generalization ability. Deep learning-based visual parking space detection algorithms offer a significant performance improvement over the former, but still face numerous challenges and difficulties (such as adaptability to environmental changes, lighting conditions, shadows, occlusion, and damage), resulting in large errors in the detected parking space coordinates and orientation, or even failure to detect parking spaces altogether. Summary of the Invention

[0004] The purpose of this invention is to provide a two-stage parking line detection method for planar parking spaces in order to improve the accuracy of the final parking position.

[0005] The technical solution of this invention is: a two-stage parking line detection method for planar parking spaces. This method divides automatic parking into two stages: searching for parking spaces and parking. Stage 1: The parking space search stage uses a deep learning-based parking space detection model to search for and detect valid parking spaces. After the driver selects the corresponding target parking space, the automatic parking stage begins. Stage 2: In the parking stage, visual perception no longer performs parking space detection; instead, parking lines are detected solely based on image processing.

[0006] In the above scheme, stage two includes the following steps:

[0007] S2.1 The system returns to the target parking space location.

[0008] Based on the initial parking space location recorded during the parking space search phase, and according to the vehicle's movement status, the system calculates and outputs the latest location of the target parking space in real time.

[0009] S2.2 Cropping a portion of the target parking space image.

[0010] Images are acquired from four onboard surround-view fisheye cameras and stitched together to obtain a surround-view image from the BEV perspective. The location of the target parking space is determined in the surround-view image. Based on the location of the target parking space, the image is expanded by N pixels in all directions (up, down, left, and right) to create a local image of the target parking space. This local image is then cropped and saved.

[0011] S2.3 Parking line segmentation network is used for prediction and inference.

[0012] The size of the local image is adjusted to the input size of the parking line segmentation network, which then performs prediction and inference to obtain a binary image.

[0013] S2.4 Edge Detection.

[0014] Cany edge detection is performed on the binary image to obtain the edge map.

[0015] S2.5 Linear Detection.

[0016] Line detection is performed on the edge map to obtain line segment detection results.

[0017] S2.6 Segment Clustering and Line Fitting

[0018] Based on the location of the target parking space returned by the traffic control, the positions of the four vertices P1, P2, P3, and P4 of the target parking space on the image are obtained.

[0019] Assume the line connecting P1 and P2 is the entrance line of the target parking space, and P2, P3 and P1, P4 are the left and right side lines of the target parking space. Based on the coordinates of the four points P1, P2, P3, and P4, calculate the initial slope of the entrance line and the left and right side lines of the target parking space. Traverse all the line segments detected in S2.2. When the angle between the detected line segment and the entrance line is less than the threshold T, the line segment is classified as the entrance line category. Otherwise, if the angle between the detected line segment and the left and right side lines is less than the threshold T, determine which category of the two side lines the line segment belongs to based on the distance between the detected line segment and P1, P4 and P2, P3. If the angle between the detected line segment and both the entrance line and the left and right side lines is greater than the threshold T, the line segment is directly discarded.

[0020] After clustering all line segments, a straight line is fitted to all line segments in the same category based on their x and y coordinate values. This straight line is the entrance line or left and right side lines of the target parking space.

[0021] During the parking process, the system continuously returns to the target parking space location in real time, repeating the steps in Phase Two and continuously updating the parking space lines.

[0022] Furthermore, the edge detection step in S2.4 includes:

[0023] S2.4.1 Perform Gaussian smoothing filtering on the image.

[0024] S2.4.2 Calculate the gradient magnitude and direction of the image.

[0025] S2.4.3 Non-maximum suppression to remove spurious edge points.

[0026] S2.4.4 Dual threshold edge filtering sets two thresholds and filters edges by comparing gradient magnitudes.

[0027] S2.4.5 connects the selected edge points to form a complete single-pixel wide edge.

[0028] Beneficial Effects: If the parking space line positions output by the parking space detection model are only saved during the parking space search phase, on the one hand, the vehicle is generally far from the target parking space at this time, resulting in a relatively large error in the parking space line position output by the parking space detection model; on the other hand, vehicle movement introduces a large cumulative error during the parking phase. Therefore, relying solely on the target parking space position saved during the parking space search for automatic parking often results in the final parking position being close to one side of the parking space or not parallel to the direction of the parking space. Compared to saving the parking space line positions output by the parking space detection model only during the parking space search phase, this invention can control and correct in real time according to the latest parking space line positions throughout the entire parking phase, thus achieving more accurate parking space line positioning and making the final parking position more precise. Moreover, this invention also reduces the accuracy requirements of the parking space detection model. Attached Figure Description

[0029] Figure 1 This is a flowchart of the present invention.

[0030] Figure 2 This is a partial image of the target parking space.

[0031] Figure 3 For division Figure 2 A diagram illustrating the annotation results.

[0032] Figure 4 This is a local image of the target parking space extracted from the original surround view stitched image.

[0033] Figure 5 for Figure 4 The binary result obtained after the parking space line segmentation algorithm.

[0034] Figure 6 for Figure 5 The edge map obtained after edge detection by cany.

[0035] Figure 7 for Figure 6 The line segment detection result obtained after straight line detection.

[0036] Figure 8 for Figure 7 The result of line segment clustering and straight line fitting.

[0037] Figure 9 This is a schematic diagram of the vertex of the target parking space. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0039] See appendix Figure 1 A two-stage parking line detection method for planar parking spaces is proposed. This method divides automatic parking into two stages: searching for parking spaces and parking. Stage 1: The parking space search stage uses a deep learning-based parking space detection model to search for and detect valid parking spaces. After the driver selects the corresponding target parking space, the automatic parking stage begins. Stage 2: In the parking stage, visual perception no longer performs parking space detection, but only detects parking lines based on image processing.

[0040] The two-stage parking space line detection method specifically includes the following steps:

[0041] Phase 1: Searching for parking spaces.

[0042] During this stage, a deep learning-based parking space detection model searches for and detects valid parking spaces in real time, and displays all valid parking spaces through the APP. Drivers can then select a target parking space, and the system records the initial position of that parking space.

[0043] Phase Two: Mooring Phase.

[0044] S2.1 The system returns to the target parking space location.

[0045] The system records the location of the parking space and calculates and outputs the latest location of the target parking space in real time based on the initial location of the parking space recorded during the parking space search phase and the movement status of the vehicle.

[0046] S2.2 Cropping a portion of the target parking space image.

[0047] Images are acquired from four onboard surround-view fisheye cameras and stitched together to obtain a BEV-view surround-view image. Based on the target parking space location returned by traffic control, or by directly detecting parking spaces in the surround-view image using a parking space detection model, the target parking space location in the surround-view image can be obtained. Using the target parking space location as a reference, the image is expanded 18 pixels upwards, downwards, leftwards, and rightwards to create a local image of the target parking space. This local image is then cropped and saved, as shown below. Figure 2 As shown.

[0048] The areas corresponding to the parking space lines were labeled using an image annotation tool (such as Labelme). The annotation results are as follows: Figure 3 As shown, a large number of collected and labeled images can be used as training data to train the parking line segmentation algorithm.

[0049] S2.3 Parking line segmentation network is used for prediction and inference.

[0050] The size of the local image is adjusted to the input size of the parking line segmentation network, which then performs prediction and inference to obtain a binary image.

[0051] The parking line segmentation algorithm employs a general image segmentation algorithm to classify each pixel in the image (whether it is a parking line or not). Deep learning algorithms such as Unet or DeepLab can be used. Considering the real-time requirements and the complexity of the parking lines, this method uses a standard Unet network structure to achieve satisfactory results. The Unet image segmentation network is mainly used for image semantic segmentation tasks and consists of an encoder and a decoder. The encoder extracts features from the input image, while the decoder maps these features to their spatial locations in the output image. The Unet image segmentation network has a relatively simple structure and high efficiency. Its decoder uses skip connections to fuse the feature maps from the encoder and decoder, balancing semantic and spatial accuracy. The input image size for the Unet segmentation network is 256*256, and the encoder has 256 channels at its end.

[0052] S2.4 Edge Detection.

[0053] Cany edge detection is performed on the binary image to obtain the edge map.

[0054] The local image of the target parking space extracted from the original surround view stitched image, as shown below. Figure 4 As shown in the figure, the result after the parking space line segmentation algorithm is as follows: Figure 5 The binary result is given, where black pixels represent the background and white pixels indicate that the location may be a parking line. Performing edge detection on 5 pixels yields the following result: Figure 6 The edge map shown.

[0055] The Cany edge detection algorithm mainly includes the following five steps: ① Gaussian smoothing filter is applied to the image; ② Gradient magnitude and direction of the image are calculated; ③ Non-maximum suppression is applied to remove false edge points; ④ Edges are filtered using a dual threshold method, with two thresholds set at 50 and 150 respectively, and edges are filtered by comparing gradient magnitudes; ⑤ The filtered edge points are connected to form a complete single-pixel wide edge.

[0056] S2.5 Linear Detection.

[0057] Line detection is performed on the edge map to obtain line segment detection results.

[0058] In this example, the Hough transform is used to implement line detection. The relevant parameters are as follows: distance precision in pixels is set to 2, angle precision in radians is set to π / 180, minimum number of votes is set to 30, minimum detectable line length is set to 30 pixels, and maximum interval between two lines is set to 20 pixels. Figure 6 Line detection can yield results such as Figure 7 The line segment detection results shown can be indicated by different colors and numbers for different line segments.

[0059] S2.6 Segment Clustering and Line Fitting

[0060] Based on the location of the target parking space returned by the traffic control, the positions of the four vertices P1, P2, P3, and P4 of the target parking space on the image are obtained;

[0061] See appendix Figure 9 Assume that the line connecting P1 and P2 is the entrance line of the target parking space, and P2, P3 and P1, P4 are the left and right side lines of the target parking space. Based on the coordinates of the four points P1, P2, P3, and P4, calculate the initial slope of the entrance line and the left and right side lines of the target parking space. Traverse all the line segments detected in S2.5. When the angle between the detected line segment and the entrance line is less than the threshold T (15°), the line segment is classified as the entrance line category. Otherwise, if the angle between the detected line segment and the left and right side lines is less than the threshold T, determine which category of the two side lines the line segment belongs to based on the distance between the detected line segment and P1, P4 and P2, P3. If the angle between the detected line segment and both the entrance line and the left and right side lines is greater than the threshold T, the line segment is directly discarded.

[0062] like Figure 8 As shown, after clustering all line segments, a straight line is fitted to all line segments in the same category based on their x and y coordinate values. This straight line is the entrance line or left and right side lines of the target parking space.

[0063] During the parking process, the system continuously returns to the target parking space location in real time, repeating the steps in Phase Two and continuously updating the parking space lines.

[0064] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A two-stage parking space line detection method for a planar parking space, characterized in that, The method divides automatic parking into two stages of searching for a parking space and parking in; stage one: the searching for a parking space stage searches and detects an effective parking space based on a deep learning-based parking space detection model, and after a driver selects a corresponding target parking space, the automatic parking in stage starts; In the parking in stage, visual perception no longer performs parking space detection, and only detects parking lines based on image processing; The stage two includes the following steps: S2.

1. regulating and returning to a target parking space position; The regulation calculates and outputs the latest position of the target parking space in real time according to the initial position of the parking space recorded in the searching for a parking space stage and the motion state of the vehicle; S2.

2. cropping a local image of the target parking space; An image is obtained from four fisheye cameras on the vehicle, a ring-view splicing image in a BEV perspective is obtained through splicing and fusion, the position of the target parking space is determined in the ring-view splicing image, a local image of the target parking space is expanded by N pixels in each of the up, down, left and right directions based on the position of the target parking space, and the local image is cropped and saved; S2.

3. a parking line segmentation network performs prediction and reasoning; The size of the local image is adjusted to the input size of the parking line segmentation network, prediction and reasoning are performed by the parking line segmentation network, and a binary image is obtained; S2.

4. edge detection; Canny edge detection is performed on the binary image to obtain an edge image; S2.

5. straight line detection; Straight line detection is performed on the edge image to obtain a line segment detection result; S2.

6. line segment clustering and straight line fitting; The positions of four vertices P1, P2, P3 and P4 of the target parking space on the image are obtained according to the position of the target parking space returned by the regulation; It is assumed that the line connecting P1 and P2 is the entrance line of the target parking space, and P2 and P3 and P1 and P4 are the left and right edge lines of the target parking space; the initial slopes of the entrance line and the left and right edge lines of the target parking space are calculated according to the coordinates of the four points P1, P2, P3 and P4, all line segments detected in S2.2 are traversed, when the included angle between a detected line segment and the entrance line is less than a threshold T, the line segment is classified as the entrance line category; otherwise, if the included angle between the detected line segment and the left and right edge lines is less than the threshold T, the distance between the detected line segment and P1, P4 and P2, P3 is used to determine which category the line segment belongs to; if the included angles between the detected line segment and the entrance line and the left and right edge lines are both greater than the threshold T, the line segment is directly discarded; After clustering all the line segments, a straight line is fitted according to the x and y coordinate values of all the line segments in the same category, and the straight line is the entrance line or the left or right edge line of the target parking space.

2. A two-stage parking space line detection method for a planar parking space according to claim 1, characterized in that, The edge detection step in S2.4 includes the following steps: S2.4.

1. Gaussian smoothing filtering is performed on the image; S2.4.

2. the gradient size and direction of the image are calculated; S2.4.

3. non-maximum suppression is performed to remove false edge points; S2.4.

4. double-threshold edge screening is performed, two thresholds are set, and edges are screened by comparing the gradient size; S2.4.

5. the screened edge points are connected to form a complete single-pixel-wide edge.

Images