Lightweight parking space detection method

Abstract

The application discloses a kind of lightweight parking stall detection methods, belong to computer vision technical field.The algorithm of a kind of lightweight MobileNet+FPN+Aware Headers is designed, and depth separable convolution is used in convolution layer to reduce parameter quantity most;MobileNet is used to compress feature;Different resolution features are fused using FPN, and finally, Aware Headers is used to complete the prediction of parking stall key points.Aware Headers includes 3 prediction heads in total, respectively for predicting the position of key points, the direction of key points and the center point of parking stall entrance.Labels generated by Gaussian kernel with parking stall direction are used to make the direction and position of parking stall have better correlation.In post-processing, according to the center point of parking stall, the corresponding two parking stall key points are matched to determine a complete parking stall.The detection method disclosed by the application has lower calculation and parameter quantity while the detection result is reliable.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to a lightweight parking space detection method. BACKGROUND

[0002] With the development of automatic driving technology, automatic parking assistance system has become a research topic, and with the rise of deep learning, the parking space detection method based on deep convolutional neural network (CNN) has achieved good accuracy. However, due to the complexity of the image, the detection of parking signs is still a very challenging problem.

[0003] The method based on deep learning to detect parking space key points mainly includes regression and heat map generation. The regression method has fast operation speed, but lacks spatial generalization ability, while the heat map based method has strong spatial generalization ability. In the early stage, many works tried to use the regression method to complete it, but the problem of poor spatial generalization ability always exists, and people gradually began to use the heat map generation method to improve the accuracy.

[0004] In recent years, the deep method has continuously increased the network capacity and complexity, so that the parking space detection has the ability to respond to changes. For example, the relatively classic DeepPS adopts a relatively complex backbone network and post-processing matching method, and achieves good results. PSDet adopts a two-stage positioning method to improve the positioning accuracy. SPFCN designs an efficient convolution module, with a parameter of 2.39MB, and completes the lightweight network design, but there is still a lot of compression space. The computational overhead of the network seriously restricts the application of deep learning algorithm in practical engineering application, for example, DeepPS needs powerful GPU to run deep learning algorithm. However, the large-scale production embedded environment only has CPU or weak GPU. Therefore, a lightweight model with further compressed parameter quantity and good detection performance is needed. SUMMARY

[0005] Preferably or optionally,

[0006] To solve the existing technical problems, the present application provides a lightweight parking space detection method, comprising the following steps:

[0007] I. Image enhancement: image enhancement is performed on the original surround view to improve the robustness of the model;

[0008] II. Multi-scale feature fusion: the enhanced image is transmitted into MobileNetV2 for feature extraction, and different scale features are fused by FPN to obtain output feature f';

[0009] III. Predicting key points: The output feature f' is simultaneously transmitted into three perception detection heads, which are used for predicting the position of the parking space key points, the direction of the key points, and the center of the entrance line, respectively. The non-maximum suppression algorithm is used to obtain accurate key points;

[0010] IV. Parking space judgment: Different key points are connected together, and whether the predicted entrance line center point exists on the connecting line and whether the directions of the two key points are consistent are judged to determine whether the two points constitute a parking space.

[0011] Preferably or optionally, the image enhancement method is blur enhancement, random brightness saturation contrast enhancement, and random region cropping to simulate the blur caused by the stretching of the image edge under the surround view angle, the light and ground reflection under various scenes, and various occlusion problems.

[0012] Preferably or optionally, the MobileNetV2 includes an upsampling module composed of a depth separable convolution module and a bilinear interpolation combination, and the fusion adopts a point-by-point addition method. After each upsampling, the fusion with the shallow feature is performed once.

[0013] Preferably or optionally, in the MobileNetV2, the calculation formula of the output feature f' is:

[0014] [f 1 / 32 , f 1 / 16 , f 1 / 8 , f 1 / 4 ]=F MobileNet (1)

[0015] f'=Conv(up(Conv(up(Conv(up(f 1 / 32 )+f 1 / 16 )+f 1 / 8 )+f 1 / 4 ))) (2)

[0016] Where 1 represents an input picture, the image size is 512*512, F MobileNet represents the multi-scale features extracted by MobileNetV2, f 1 / N (N∈{4,8,16,32}) is the feature of different scales obtained after the network, up is the upsampling operation, Conv is the convolution layer, and f' is the output feature, whose size is 1 / 4 of the original input.

[0017] Preferably or optionally, it also includes generating key point labels, extending 2 pixels at the key point position according to the direction of each key point in the original label, and adding a Gaussian kernel to obtain a key point label with direction guidance.

[0018] Preferably or optionally, the perception detection head uses a depthwise separable dilated convolution to process the features of each channel, and then aggregates the information between channels through point-wise convolution. The activation functions of the three detection heads are Sigmoid, Tanh and Sigmoid respectively. Since the cosine value and the sine value of the direction both belong to [-1, 1], the Tanh activation function is used to control the output value within [-1, 1].

[0019] Preferably or optionally, the heat map regression of the perception detection head is:

[0020] f pre =Activate(PwConv(DwDilatedConv)) (3);

[0021] wherein f pre is the heat map regression, Activate is the activation function, PwConv is the point-wise convolution, and DwDilatedConv is the depthwise separable dilated convolution.

[0022] Preferably or optionally, the loss function of the heat map regression is Weighted Wing Loss and SELoss, Weighted Wing Loss is used for the detection head to predict the key point position and the center point of the entrance line, and SELoss is used for the detection head to predict the key point direction; the formula is:

[0023] (4)

[0024] (5);

[0025] wherein x is the distance between the predicted value and the true label, w is a human hyperparameter, which limits the range of the nonlinear part of the function to [-w, w], ε is the curvature for constraining the nonlinear region, and W is the weight generated according to the true label, and C=w-wln(1+x / ε) is used to ensure the continuity of the loss function.

[0026] Beneficial effects: the lightweight parking space detection method provided by the application improves the robustness of the model through image enhancement, and makes the model more compact and lightweight by using a deep separable convolution module and a bilinear interpolation combination to constitute an up-sampling module; the three perception detection heads of parking space key point position prediction, key point direction prediction and entrance line center prediction use deep separable hollow convolution to process the features of each channel, and then use point-by-point convolution to aggregate the information between channels, so as to amplify the micro error in prediction, increase the supervision strength, and obtain more accurate key point positions; the Weighted Wing Loss of the application is used as a loss function to supervise the two detection heads of predicted key points and entrance line center points; the application detects the parking space key points based on the heat map regression mode, which has better spatial generalization ability compared with the coordinate regression mode, and the overall architecture of the model is designed to be lightweight, and the channel number is also compressed as much as possible to reduce the redundancy of features, so that it is very friendly to low computing power platforms. At the same time, the perception segmentation head designed in the patent can perceive the connection between key points far away, so as to ensure the recall rate and precision rate of the prediction result. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 is a neural network structure diagram;

[0028] Figure 2.1 and Figure 2.2 are respectively a Gaussian kernel and a direction guide Gaussian kernel of the application;

[0029] Figure 3 is a parking space detection effect;

[0030] Figure 4 is a recall rate and precision rate curve of the algorithm on the verification set. DETAILED DESCRIPTION

[0031] In the following description, a large number of specific details are given in order to provide a more thorough understanding of the application. However, it is obvious to those skilled in the art that the application can be implemented without one or more of these details. In other examples, some technical features known in the art are not described in order not to obscure the application.

[0032] The application will be further described below in conjunction with examples, and the examples described are intended to explain the application, and cannot be understood as limiting the application. If the specific techniques and reaction conditions are not specified in the examples, they can be carried out according to the techniques or conditions described in the literature in the art or the product instructions. EMBODIMENT

[0033] Parameter setting: For image preprocessing, the image is first adjusted to 512x512, which ensures consistent resolution after 32 times downsampling, and at the same time, reducing the image size also correspondingly reduces the amount of calculation. At the same time, data enhancement adopts random image blur (blur kernel size is 5x5) + random brightness change + random erasure, respectively, to resist the blur caused by the stretching of the image under the look-around view angle, the light and ground reflection under various scenes and the problem of occlusion. The initial learning rate is set to 0.001, the Adam optimizer and the ReduceLROnPlateau learning rate planning are adopted, the batchsize is 32, and the training is 200 epochs. The loss weight ratio of the three detection heads (key points, center points, key point directions) is 1:1:4. In the weighted wing loss, w is set to 2.5, that is, when the distance between the predicted value and the true value is less than 2.5, the nonlinear branch in the loss will be used.

[0034] Model details: as in the attached drawings Figure 1As shown, the input size is 512x512x3, where both the width and height are 512, and 3 represents the red, green, and blue three channels, which is fed into the backbone network MobileNetv2, where s4, s8, s16, and s32 respectively represent down-sampling by 4, 8, 16, and 32, and the number of channels respectively corresponds to 24, 64, 128, and 192. Compared with the conventional convolution operation, MobileNet adopts a depth separable convolution and a point-wise convolution to replace the conventional convolution. The depth separable convolution is a commonly used method in lightweight neural network architecture to compress the parameter amount while maintaining good feature extraction performance, but it does not consider the connection between different channels, and needs to use point-wise convolution to fuse the information between different channels. At the same time, compared with MobileNetv1, the BottleNeck design is added to further reduce the parameter amount, and compared with V3, the SE Block is added to maintain less calculation amount and good accuracy. After MobileNetv2, the output feature s32 is fed into the feature pyramid, and the up-sampling module is designed as conv3x3+relu+conv1x1+bilinear interpolation. After each up-sampling, the shallow features in MobileNet are fused to make up for the information that may be lost in the up-sampling process. In the detection head part, in order to improve the detection performance as much as possible while maintaining a low parameter amount and operation amount, a perception detection head is designed based on the atrous convolution to capture long-distance feature responses, improve the mutual relationship between the parking space key points, and also use a depth separable method to reduce the parameter amount, and then use a point-wise convolution to aggregate the information between channels. Using atrous convolution can increase the receptive field without losing information through pooling, so that the output of each convolution contains a larger range of information, which helps the network to understand the relationship between key points. At the same time, it should be noted that the activation function of the detection head used to predict the direction of the key point adopts Tanh, because the cos and sin values predicted by this branch are distributed between [-1, 1], while Sigmoid will place the values between [0, 1].

[0035] Comparative Examples 1-3

[0036] PSD_L, DeepPS, and SPFCN are used as Comparative Examples 1-3, respectively.

[0037] Experimental result evaluation

[0038] Ps2.0 is the largest and most complete public benchmark dataset, which is widely used for parking space detection. It contains 12165 surround view images, each with a size of 600x600, corresponding to a physical space size of 10m x 10m. It collects typical indoor and outdoor parking spaces under different lighting conditions, and provides the positions of the key points. Therefore, recall rate and precision rate are used as experimental evaluation indicators.

[0039] Recall is for the original label sample, which represents how many positive examples in the sample are predicted correctly. There are two possibilities, one is to predict the original positive class as positive class (TP), and the other is to predict the original positive class as negative class (FN), as follows.

[0040] Recall = TP / (TP + FN) (5)

[0041] Precision represents the overall prediction accuracy, including positive and negative samples. The denominator is the predicted positive class, and the precision is proposed to make the existing prediction results of the model as accurate as possible, where FP is the false positive sample, that is, the negative sample is predicted as a positive sample.

[0042] Precision = TP / (TP + FP) (6)

[0043] The parameter quantity, precision and recall of Example 1 and each comparative example are shown in Table 1 as follows:

[0044] Table 1

[0045] Parameter amount / MB Precision Recall Example 1 0.25 99.49% 96.77% Comparative Example 1 8.38 98.55% 84.64% Comparative Example 2 255 99.54% 98.89% Comparative Example 3 2.39 98.01% 97.31%

[0046] From Table 1, it can be concluded that the recall and precision of each algorithm on Ps2.0 are optimal, but the parameter quantity is also the largest 255 MB, while the implementation example has the smallest parameter quantity while maintaining a high level of recall and precision.

[0047] At the same time, in order to prove the effect of the loss function design, different loss functions are applied to Example 1, and the accuracy and recall of various loss functions are shown in Table 2 as follows:

[0048] Table 2

[0049] Loss function Accuracy Recall Weighted MSE Loss 98.52% 86.81% Weighted MSE Loss (w / o AwareHead) 91.35% 76.02% Focal Loss 99.23% 96.24% Focal Loss (w / o AwareHead) 93.73% 82.45% Weighted Wing Loss (w / o AwareHead) 94.53% 86.82% Weighted Wing Loss 99.49% 96.77%

[0050] From Table 2, it can be seen that the accuracy and recall of Example 1 using Weighted Wing Loss are significantly higher than those of other loss functions. At the same time, experiments also show that using a perception detection head can improve the precision by 6%~7%.

[0051] In addition, it should be noted that each specific technical feature described in the above specific embodiments can be combined in any appropriate manner without contradiction. In order to avoid unnecessary repetition, the present application will not further describe various possible combinations.

Claims

1. A lightweight parking space detection method, characterized by, It comprises the following steps: I. Image enhancement: image enhancement is performed on the original surround view to improve the robustness of the model; II. Multi-scale feature fusion: the enhanced image is input into MobileNetV2 for feature extraction, and different scale features are fused through FPN to obtain output feature f'; III. Predict key points: the output feature f' is input into three perception detection heads at the same time, and the position of the parking space key point, the direction of the key point, and the center of the entrance line are predicted respectively, and the non-maximum suppression algorithm is used to obtain accurate key points; IV. Parking space judgment: different key points are connected together, and whether the predicted entrance line center point exists on the connecting line and whether the directions of the two key points are consistent are judged to determine whether the two points constitute a parking space.

2. The light-weight parking space detection method according to claim 1, characterized in that, The image enhancement method is blur enhancement, random brightness saturation contrast enhancement, and random region cropping to simulate the blur caused by the stretching of the image edge under the surround view angle, the light and ground reflection under various scenes, and various occlusion problems.

3. The light-weight parking space detection method according to claim 1, characterized in that, The MobileNetV2 includes an upsampling module composed of a depth separable convolution module and a bilinear interpolation combination, and the fusion adopts a point-by-point addition method. After each upsampling, the shallow layer feature is fused.

4. The lightweight parking space detection method according to claim 3, characterized by, In the MobileNetV2, the calculation formula of the output feature f' is: [f 1 / 32 , f 1 / 16 , f 1 / 8 , f 1 / 4 ]=F MobileNet (1) f' = Conv(up(Conv(up(Conv(up(f 1 / 32 )+f 1 / 16 )+f 1 / 8 )+f 1 / 4 ))) (2) where 1 represents an input picture, the image size is 512x512, F MobileNet represent multi-scale features extracted by MobileNetV2, f 1 / N (N∈{4, 8, 16, 32}) are features of different scales obtained after the network, up is an up-sampling operation, Conv is a convolution layer, and f' is an output feature, the size of which is 1 / 4 of the original input.

5. The light-weight parking space detection method according to claim 1, characterized in that, It also includes generating key point labels, extending 2 pixels at the key point position according to the direction of each key point in the original label, and adding a Gaussian kernel to obtain a key point label with direction guidance.

6. The light-weight parking space detection method according to claim 1, characterized in that, The perception detection head uses a depth separable hollow convolution to process the features of each channel, and then aggregates the information between channels through point-by-point convolution; the activation functions of the three detection heads of the parking space key point, the entrance center line point, and the key point direction are Sigmoid, Sigmoid, and Tanh, respectively, which control the response range of the detection head output value through the activation function.

7. The lightweight parking space detection method according to claim 1, characterized by, The heat map regression of the perception detection head is: f pre =Activate(PwConv(DwDilatedConv)) (3); where f pre is a heat map regression, Activate is an activation function, PwConv is a point-wise convolution, and DwDilatedConv is a depthwise separable dilated convolution.

8. The lightweight parking space detection method according to claim 7, characterized by, The loss function of the heat map regression is Weighted Wing Loss and SELoss, Weighted Wing Loss is used for the detection head of the predicted key point position and the entrance line center point, and SELoss is used for the detection head of the predicted key point direction; The formula is: （4） （5）； Where x is the distance between the predicted value and the true label, w is a human hyperparameter that limits the range of the non-linear part of the function to [-w, w], ε is the curvature that constrains the non-linear region, W is the weight generated according to the true label, and C=w-wln(1+x / ε) is used to ensure the continuity of the loss function.

Images