A parking space detection method based on parking space key point instance detection

Abstract

This invention discloses a parking space detection method based on key point instance detection, relating to the field of parking space detection. Existing parking space detection methods are easily affected by edge integrity, lighting conditions, and interference from other objects, leading to detection failures and poor accuracy. The proposed solution includes a parking space detection method based on key point instance detection, comprising the following steps: S1, marking four points of the parking space sequentially based on the center point, and labeling each of the four points; S2, calculating the angle of the parking space's edge line; S3, establishing a parking space detection model. This invention features a novel structure and, compared to semantic segmentation-based labeling methods, offers more concise and efficient labeling, is less affected by external factors, has a high parking space detection success rate, and can quickly calculate the entrance position and angle of the parking space from the vertex position, reducing the difficulty of post-processing and simplifying the process.

Description

Technical Field

[0001] This invention relates to the field of parking space detection, and more particularly to a parking space detection method based on the detection of key points in parking spaces. Background Technology

[0002] With the development of vehicle intelligence, people's demand for automatic parking functions is becoming more and more widespread and strong. As an important part of parking functions, how to efficiently and accurately detect parking spaces has become the key to the entire parking function.

[0003] Currently, mainstream visual parking space detection algorithms are mainly divided into traditional algorithms based on image texture and parking space detection methods based on deep learning. Image texture-based parking space detection algorithms mainly extract edges in the image and complete the parking space detection work by geometric constraints such as perpendicularity and parallelism between parking space edges. However, traditional algorithms are generally based on shallow edge or corner information as the basis for parking space detection, which is easily affected by edge integrity, lighting conditions, and interference from other objects, leading to parking space detection failure. Meanwhile, deep learning-based algorithms are mostly based on semantic segmentation and parking space vertex detection, and then complete the assembly of parking spaces through post-processing. Post-processing is often quite complex, has high development difficulty, and increases CPU usage. Therefore, in order to solve these problems, we propose a parking space detection method based on parking space key point instance detection. Summary of the Invention

[0004] The present invention proposes a parking space detection method based on key point instance detection of parking spaces, which solves the problem that existing parking space detection methods are easily affected by edge integrity, lighting conditions, and interference from other objects, leading to parking space detection failure and poor accuracy.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A parking space detection method based on parking space key point instance detection includes the following steps:

[0007] S1. First, mark the four points of the parking space in sequence according to the center point of the parking space, and then label the four points respectively.

[0008] S2, calculates and obtains the angle of the parking space edge line;

[0009] S3, Establish a parking space detection model;

[0010] S4, stitch together the parking space surround view;

[0011] S5 performs heatmap detection on key points in parking spaces;

[0012] S6, detection of the center point of the parking space;

[0013] S7 performs regression analysis on the parking space information to complete the parking space detection.

[0014] Preferably, the parking space markings in S1 are corner1, corner2, corner3, and corner4, and each marking must ensure that corner2 and corner3 are on the entrance side.

[0015] Preferably, the parking space edge angle in S2 is obtained by calculating the angle of the edge line formed by corner1 and corner2 and the angle of another edge line formed by corner3 and corner4, and taking the average to obtain the parking space angle.

[0016] Preferably, the parking space detection model in S3 consists of input, backbone, upsample, and heads.

[0017] Preferably, the bacbone uses Mobilenet_v2 as the feature extraction network to improve the inference speed of the network, and the upsample network consists of three deconvolutional layers and the heads consist of four branches.

[0018] Preferably, the four branches of the heads are: the Park classification branch completes the heatmap detection of the parking space center point, the Keypoints regression branch completes the regression of the parking space vertex, the Keypointsclassification branch completes the heatmap detection of the parking space key points, and the Keypoints offset regression performs parking space key point regression on the Keypoints classification position.

[0019] Preferably, the calculation formulas for the regression loss functions of the parking space vertex and the parking space key point are as follows:

[0020]

[0021] Where L is the total network loss. PC For the parking space detection branch loss, L kr The loss function L regresses from the center of the parking space to the apex of the parking space. kc Parking space vertex detection loss, L ko The parking space keypoint regression loss is calculated based on the keypoints classification location, where λ0, λ1, λ2, and λ3 are the weight coefficients for each loss.

[0022] Preferably, the parking space surround view in S4 is obtained by projecting it onto the ground coordinate system through four fisheye cameras (front, rear, left, and right) after affine transformation.

[0023] Preferably, the key point detection steps in S5 are as follows: first, the center point of the parking space is detected through the park classification branch, and the key points corresponding to the parking space are regressed through the keypoints regression branch. Then, the key points of the parking space are detected through the keypoints classification branch, and the position of the key points is regressed through keypoint offset regression to overcome the quantization error caused by the detection branch. The center point detection of the parking space in S6 is completed through the park classification branch.

[0024] Preferably, in step S7, the key points of the regressed parking space are replaced with key points of higher confidence by using the positions of the regressed parking space key points and the detected parking space key points, thereby obtaining the final parking space instance and its corresponding parking space key point coordinates.

[0025] The beneficial effects of this invention are as follows: Compared with the annotation method based on semantic segmentation, this method is more concise and efficient. At the same time, the parking space detection model outputs parking space vertex information, which is less affected by external factors, has a high parking space detection success rate, and can quickly calculate the entrance position and angle of the parking space through the position of the vertex, reducing the difficulty of parking space post-processing and making parking space post-processing simpler.

[0026] In summary, this invention is easy to use, and compared with semantic segmentation-based annotation methods, its annotation is more concise and efficient, less affected by external factors, and has a high success rate in parking space detection. Furthermore, it can quickly calculate the entrance position and angle of the parking space based on the vertex position, reducing the difficulty of post-processing and making post-processing of parking spaces simpler. It solves the problem that some existing parking space detection methods are easily affected by edge integrity, lighting conditions, and interference from other objects, leading to parking space detection failure and poor accuracy. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of parking space markings in a parking space detection method based on parking space key point instance detection as described in this invention.

[0028] Figure 2 This is a schematic diagram of the detection model in the parking space detection method based on parking space key point instance detection according to the present invention;

[0029] Figure 3 The post-processing flowchart of a parking space detection method based on parking space key point instance detection according to the present invention is shown below.

[0030] Figure 4 The flowchart of a parking space detection method based on parking space key point instance detection according to the present invention. Detailed Implementation

[0031] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0032] Reference Figures 1-4 A parking space detection method based on parking space key point instance detection includes the following steps:

[0033] S1, refer to Figure 1 As shown, first mark the four points of the parking space according to the center point of the parking space, and then label the four points respectively. The parking space labeling in S1 is corner1, corner2, corner3, and corner4. Each time the labeling is done, it must be ensured that corner2 and corner3 are on the entrance side.

[0034] S2, the parking space edge angle is obtained by calculation. The parking space edge angle in S2 is obtained by calculating the angle of the edge line formed by corner1 and corner2 and the angle of another edge line formed by corner3 and corner4, and taking the average to obtain the parking space angle.

[0035] S3, refer to Figure 2 , Figure 3 As shown, a parking space detection model is established. The parking space detection model in S3 consists of input, backbone, upsample, and heads. The backbone uses MobileNet v2 as the feature extraction network to improve the network's derivation speed. The upsample network has three deconvolutional layers. The heads consist of four branches: Park classification (for detecting the parking space center point heatmap), Keypoints regression (for regressing the parking space vertices), Keypoints classification (for detecting the parking space key points heatmap), and Keypoints offset regression (for regressing the parking space key points at the Keypoints classification location). The calculation formulas for the loss functions of the parking space vertex regression and parking space keypoint regression are as follows:

[0036]

[0037] Where L is the total network loss. PC For the parking space detection branch loss, L kr The loss function L regresses from the center of the parking space to the apex of the parking space. kc Parking space vertex detection loss, L ko The parking space key point regression loss is performed based on the keypoints classification location, where λ0, λ1, λ2, and λ3 are the weight coefficients of each loss.

[0038] S4, stitch together the parking space surround view. The parking space surround view in S4 is obtained by projecting it onto the ground coordinate system through four fisheye cameras (front, rear, left, and right) after affine transformation.

[0039] S5, detecting the heatmap of key points in parking spaces. The detection steps of key points in S5 are as follows: first, the center point of the parking space is detected through the parkclassification branch, and the key points corresponding to the parking space are regressed through the keypoints regression branch. Then, the key points of the parking space are detected through the keypoints classification branch, and the position of the key points is regressed through the keypoint offset regression to overcome the quantization error caused by the detection branch.

[0040] S6, Detection of the center point of the parking space, the detection of the center point of the parking space in S6 is completed through the parkclassification branch;

[0041] S7. Regress the parking space information to complete the parking space detection. In S7, the regression of the parking space key points and the location of the detected parking space key points are used to replace the regression key points with key points with higher confidence, thereby obtaining the final parking space instance and its corresponding parking space key point coordinates.

[0042] Example: First, mark four points of the parking space sequentially based on the center point, and label each of the four points as corner1, corner2, corner3, and corner4. Each labeling ensures that corner2 and corner3 are on the entrance side. Then, calculate the angle of the parking space's edge line. First, calculate the angle of the edge line formed by corner1 and corner2, and the angle of another edge line formed by corner3 and corner4. Take the average to obtain the parking space's angle. Then, build a parking space detection model. The Park classification branch in the heads section completes the heatmap detection of the parking space's center point, the Keypoints regression branch completes the regression of the parking space's vertices, Keypointsclassification completes the heatmap detection of key points in the parking space, and Keypoints offset regression completes the keypoints... The classification location is used to regress parking space key points and calculate the regression loss function. Then, the panoramic view captured by four fisheye cameras is stitched together. Next, the heatmap of parking space key points is detected, and the quantization error caused by the detection branch is overcome. Finally, the regressed key points are replaced with key points with higher confidence by using the positions of the regressed parking space key points and the detected parking space key points, so as to obtain the final parking space instance and its corresponding parking space key point coordinates.

[0043] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A parking space detection method based on parking space key point instance detection, characterized in that, Includes the following steps: S1. First, mark the four points of the parking space according to the center point of the parking space in sequence, and label the four points as corner1, corner2, corner3, and corner4 respectively. Each time you label, make sure that corner2 and corner3 are on the entrance side. S2, calculate the angle of the parking space edge line, calculate the angle of the edge line formed by corner1 and corner2 and the angle of the other edge line formed by corner3 and corner4, and take the average to get the angle of the parking space. S3. Establish a parking space detection model, which consists of input, backbone, upsample, and heads. The backbone uses Mobilenet_v2 as the feature extraction network to improve the inference speed of the network. The upsample network has three deconvolutional layers. The heads consist of four branches: Parkclassification branch to complete the parking space center point heatmap detection, Keypoints regression branch to complete the regression of parking space vertices, Keypoints classification branch to complete the parking space key point heatmap detection, and Keypoints offsetregression branch to perform parking space key point regression at the Keypoints classification position. S4, stitch together the parking space surround view; S5 performs heatmap detection on key points in parking spaces; S6, detection of the center point of the parking space; S7 performs regression analysis on parking space information to complete parking space detection; The key point detection steps in S5 are as follows: first, the center point of the parking space is detected through the park classification branch, and the key points corresponding to the parking space are regressed through the keypoints regression branch. Then, the key points of the parking space are detected through the keypointsclassification branch. Finally, the position of the key points is regressed through keypoint offset regression to overcome the quantization error caused by the detection branch. The center point detection of the parking space in S6 is completed through the park classification branch. S7 obtains the final parking space instance and its corresponding parking space key point coordinates by replacing the regressed key points with key points of higher confidence by using the regressed key points and the detected parking space key points.

2. The parking space detection method based on parking space key point instance detection according to claim 1, characterized in that, The parking space surround view in S4 is obtained by projecting it onto the ground coordinate system through four fisheye cameras (front, rear, left, and right) after affine transformation.

Images