Parking space state identification method based on improved YOLOv5s model

Abstract

The invention discloses a parking space state recognition method based on an improved YOLOv5s model, and the method comprises the steps: constructing a vehicle target detection data set, and carrying out the data enhancement; an improved YOLOV5s network model is constructed, and model training is carried out; wherein a part of feature extraction modules CSPDarknet53 in an original network of the YOLOV5s model is replaced with a MyModel feature extraction module suitable for a parking lot background, and backbone and head structures in the YOLOV5s model are improved by using methods such as maximum pooling, expansion convolution and the like; secondly, Focal Loss is introduced to solve the problem of imbalance of difficult samples, so that regression of the model is more accurate; and carrying out parking space calibration on an image to be detected, inputting the image to the improved YOLOv5s network model obtained by training, and comparing the position information of the parking space with the position information of the vehicle target so as to obtain a parking space state identification result. According to the invention, on the basis of self light weight, high parking space state identification speed and precision are effectively ensured, and the method is good in stability and strong in robustness and can be applied to an intelligent parking system.

Description

Technical Field

[0001] The present invention relates to the fields of image processing, computer vision and target detection, and in particular to a parking space status recognition method based on an improved YOLOv5s model. Whether the coordinate position of the center point of the lower edge of the detection frame is within the parking line is used as the basis for judging whether the parking space is occupied. Background Art

[0002] Intelligent Traffic System (ITS) is an effective means to solve urban traffic congestion and other problems in the future. Vehicle target detection is an important part of ITS front-end data collection, providing data guarantee for intelligent traffic control and solving urban parking problems. At present, almost all parking lots in China have achieved video surveillance coverage, and there are very rich urban parking surveillance video resources. Using existing parking lot video surveillance resources for vehicle target detection is the basis for the subsequent realization of parking space status recognition and solving urban parking problems.

[0003] Among various target detection algorithms, the YOLO (You Only Look Once) framework stands out for its excellent balance of speed and accuracy, achieving fast and reliable recognition of objects in images. Since its birth, the YOLO series has gone through many iterations, each time solving limitations and improving performance on the basis of the previous version. The YOLOv5 series is divided into four models: s, m, l, and x according to the model parameters (Params) and floating-point operations (FLOPs). The Params and FLOPs of the s to x models continue to increase. The biggest feature of YOLOv5s is that the model is more lightweight, with fewer Params and FLOPs, and is suitable for real-time target detection on devices with limited resources. The monitoring resource equipment in urban parking scenarios is mostly embedded devices with low computing power, so the present invention uses YOLOv5s with the least parameters as the basic model.

[0004] Although the target detection algorithm is relatively mature, there are still certain problems and difficulties in the target detection task in urban parking scenarios. The main deficiencies of video surveillance images in urban parking scenarios are: the size of vehicle targets in the image is not fixed, the vehicles in the image are distorted, and the vehicles seriously occlude each other. As the convolutional neural network downsamples the image multiple times, the feature size continues to decrease, and the vehicle target features under a specific background are difficult to extract, resulting in false detection and missed detection during the target detection process. Therefore, in the process of vehicle target detection in urban parking scenarios, problems such as poor vehicle target feature extraction, high missed detection rate and false detection rate of target detection are prone to occur, which means that there is still a lot of room for improvement in the accuracy of parking space status recognition. Summary of the invention

[0005] In order to improve the accuracy of vehicle target detection in urban parking scenarios, reduce the false detection and missed detection rates, and thus make parking space status recognition more accurate, the present invention is based on the performance advantages of the Yolov5s model, and on this basis provides a parking space status recognition method based on the improved YOLO V5s model. On the basis of its own lightweight, it effectively guarantees a high speed and accuracy of parking space status recognition, and has good stability and strong robustness, and can be applied to smart parking systems.

[0006] In order to achieve the purpose of the present invention, the present invention provides a parking space state recognition method based on an improved YOLOV5s model, comprising the following steps:

[0007] Step 1: Construct a vehicle target detection dataset in a parking context and divide it into a training set and a test set;

[0008] Step 2: Flip-Mosaic data enhancement technology is used on the dataset to alleviate the problem of sample size imbalance;

[0009] Step 3: Annotate the enhanced data set to form a standardized data set;

[0010] Step 4: Build an improved YOLOV5s network model, in which some feature extraction modules CSPDarknet53 in the original network of the YOLOV5s model are replaced with MyModel feature extraction modules suitable for the parking lot background, and the backbone and head structures in the Yolov5s model are improved by using methods such as maximum pooling and dilated convolution. Secondly, by optimizing the connection method of feature maps at different levels, more hierarchical features are obtained to enhance the model feature extraction capability. Finally, in order to solve the ambiguity problem that the CIoU aspect ratio describes a relative value, EIoU is introduced in the positioning function, and the difference values ​​of width and height are calculated on the basis of CIOU to replace the aspect ratio. At the same time, Focal Loss is introduced to solve the problem of imbalance between difficult and easy samples, making the regression of the model more accurate. The MyModel feature extraction module divides the input information and divides the input into four parts according to the channel direction, each of which is 1 / 4 of the original, and respectively uses the maximum pooling Maxpool, dilated convolution Dilated Conv, deformable convolution DConv and ordinary convolution Conv. This module can obtain feature information from different directions;

[0011] Step 5: Use the labeled vehicle target training set to train the improved YOLOv5s model;

[0012] Step 6: Manually divide parking spaces in the image material to be detected, divide parking spaces with parking space markings into separate parking space recognition areas, divide larger parking areas and set the maximum parking threshold for areas without parking space markings or unclear markings but actually available for parking, and return parking space location information;

[0013] Step 7: Perform vehicle target detection on the image to be detected, return vehicle classification information and target center point location information, match the vehicle location information with the parking space location information, and if the vehicle target center point is within the calibrated parking space, determine that the parking space is occupied, and complete the parking space status judgment;

[0014] Preferably, in step 2, an inverted mosaic algorithm is used to enhance the data by flipping the image left and right and up and down, and retaining the original state of the data. At the same time, some random noise is reasonably introduced to enhance the network model's discrimination ability for small target samples in the image and improve the generalization ability of the model.

[0015] Preferably, in step 4, the calculation formula of the improved loss function Focal-EIoU Loss introduced is as follows:

[0016]

[0017] The EIoU Loss consists of three parts: IoU loss, distance loss, and height-width loss (overlapping area, center point example, and aspect ratio); c With h c are the width and height of the minimum enclosing rectangle of the predicted bounding box and the true bounding box, respectively; ρ is the Euclidean distance between two points.

[0018] Preferably, in step 4, the shallow feature map is fused with the deep feature map after adjusting the width and height through a convolution with a step size of 2, so as to avoid the loss of low-level feature information due to the enhanced feature extraction capability.

[0019] The present invention also provides a parking space status recognition system, comprising:

[0020] Image access module, used to access parking lot image resources to be detected;

[0021] A data enhancement module, used for performing inverse mosaic algorithm data enhancement on the vehicle target image to form an enhanced vehicle target image;

[0022] An image annotation module is used to annotate the enhanced vehicle target image to form a training sample;

[0023] A training module, used for training the improved YOLOv5s network using the training samples to obtain a vehicle target recognition model in a parking lot background; wherein the improved YOLOv5s network is a YOLOv5s network that adds a new Mymodel feature extraction module and optimizes network connections;

[0024] The parking space calibration module is used to calibrate the parking space in the background of the image to be detected and obtain the horizontal and vertical coordinates of the four vertices of the corresponding parking space;

[0025] The parking space status detection module is used to detect the image to be detected using the parking space status detection model to obtain a recognition result.

[0026] Compared with the prior art, the present invention has at least the following beneficial effects:

[0027] The present invention performs parking space status detection based on an improved YOLOv5s model, combining the recognition of parking space status with computer vision technology. Different from the previous method of simultaneously detecting vehicles and parking spaces, the present method only needs to detect vehicle targets and then combines it with parking space calibration, which can provide a more convenient method for real-time detection of parking space status recognition and can more easily and accurately detect the parking space occupancy status in the image.

[0028] The present invention adopts the MyModel feature extraction module for the image features in the parking background to specifically solve the problems of low accuracy, high false detection and missed detection rates of vehicle target detection, thereby making the parking space status recognition more accurate.

[0029] The present invention introduces Focal Loss to solve the problems existing in the CIoU loss function, comprehensively considering the overlapping area, the distance between the center points and the actual distance between the center points, as well as the actual difference between the width and height, which makes the model converge faster and improves the regression accuracy of the prediction box.

[0030] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0032] Figure 1 The present invention provides a flow chart of a parking space status identification method.

[0033] Figure 2 This is a schematic diagram of the camera layout position provided by the present invention.

[0034] Figure 3 Schematic diagram of Mosaic data enhancement in an embodiment of the present invention.

[0035] Figure 4 It is a schematic diagram of the improved YOLOv5 network structure of the present invention.

[0036] Figure 5 It is a schematic diagram of the Mymodel feature extraction module used in the present invention.

[0037] Figure 6 Schematic diagram of the maximum pooling Maxpool adopted in the present invention.

[0038] Figure 7 This is a schematic diagram of the dilated convolution adopted by the present invention.

[0039] Figure 8 It is a schematic diagram of the deformable convolution Dconv adopted in the present invention.

[0040] Fig. 9 It is a schematic diagram of the actual detection effect of the improved model of the present invention.

[0041] Fig.10 It is a schematic diagram of the test results in the embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution will be described clearly and completely in conjunction with the present application. Figure 1 The present invention provides a parking space status recognition method based on an improved YOLO model, comprising the following steps:

[0043] Step 1: Please refer to Figure 2 The schematic diagram of the camera layout position is used to collect vehicle images in the parking background according to the corresponding camera; and when the camera position is needed or rearranged in order to use the camera to judge the parking space status, the camera layout position can be recommended according to the average vehicle height and the standard parking space markings.

[0044] To ensure that the tilt angle offset of the top of the vehicle does not exceed the distance between the vehicles, so that the model can judge that it is in the correct parking space, certain requirements should be met when arranging the camera position so that the recognition position of the top of the vehicle can be correctly judged in the corresponding parking space, that is, it should meet in:

[0045]

[0046] Then h, l should satisfy:

[0047] l<0.6h-1.2

[0048] Among them: h represents the camera height; l represents the distance from the camera to the parking space; α represents the camera tilt angle; H represents the vehicle height, which is 2m; W represents the parking line width, which is 2.4m (garage building design specification); d represents the vehicle top tilt angle offset. Through the preliminary calculation of the offset, the camera is set to meet certain requirements as much as possible, which avoids the error of vehicle occupation position caused by camera angle problems to a certain extent.

[0049] Step 2: In order to simulate the working conditions of infrared cameras at night, part of the data set is processed by grayscale and inversion. Labelimg is used to mark the targets in all images in the data set, and the data set is divided into a training set and a test set.

[0050] Step 3: Use Flip-Mosaic data augmentation technology on the dataset to alleviate the problem of sample size imbalance; see Figure 3 , remove a unit of data from the total data set, randomly take out 4 pictures each time, crop and splice them at random positions, synthesize new pictures, and finally obtain multiple new data of pictures after mosaic data enhancement, and then pass them to the neural network for training to achieve the effect of data enhancement.

[0051] Step 4: Annotate the enhanced data set to form a standardized data set;

[0052] Step 5: Build an improved YOLOV5s network model.

[0053] See also Figure 4 , replace some feature extraction modules CSPDarknet53 in the original network of the YOLOV5s model with MyModel feature extraction modules suitable for the parking lot background, use maximum pooling, dilated convolution and other methods to improve the backbone and head structures in the Yolov5s model, and then optimize the connection method of feature maps at different levels to obtain more hierarchical features and enhance the model feature extraction ability. Finally, in order to solve the ambiguity problem that the CIoU aspect ratio describes the relative value, EIoU is introduced in the positioning function. On the basis of CIOU, the difference values ​​of width and height are calculated respectively to replace the aspect ratio. At the same time, Focal Loss is introduced to solve the problem of imbalance between difficult and easy samples, making the regression of the model more accurate.

[0054] Please refer to Figure 5, MyModel feature extraction module divides the input information into four parts according to the channel direction, each part is 1 / 4 of the original, and passes through the maximum pooling Maxpool, dilated convolution Dilated Conv, deformable convolution DCnv and ordinary convolution Conv respectively. This module can obtain feature information from different directions;

[0055] Maxpool divides the input image into several rectangular regions and outputs the maximum value of each region, such as Figure 6 As shown in the figure, adding the maximum pooling Maxpool can help the model reduce the dimension, enhance the translation invariance, extract features to retain important information, and increase the receptive field of the feature map;

[0056] Dilated convolution Figure 7 As shown in the figure, the size of the convolution kernel is enlarged, thereby increasing the receptive field. Compared with the ordinary convolution calculation process, the resolution of the feature map is reduced. The dilated convolution calculation obtains a wider range of feature information, which reduces the resolution reduction of the feature map and retains more image information. In the vehicle detection task, the use of dilated convolution can perform multi-scale processing on the vehicles in the input image, so as to better capture vehicle targets of different scales. The calculation formula of the dilated convolution convolution coefficient is as follows:

[0057] d=(k-1)×r+1

[0058] Where r is the expansion coefficient; d is the size of the expanded convolution kernel; k is the size of the normal convolution kernel; and after testing, the expansion coefficient is set to 3 in the MyModel model.

[0059] Deformable convolution Dconv Figure 8 As shown in the figure, an offset is applied to each element on the ordinary square convolution kernel, each point in the convolution kernel is offset to the identified object as much as possible, and finally the weighted sum is performed; the specific calculation is as follows:

[0060] Define a regular grid R of ordinary convolution kernels, and sample on the input feature map x; w is the sum of the weights of the sampled values. R defines the size of the receptive field. For example, R is as follows:

[0061] R={(-1,-1), (-1,0),..., (0,1), (1,1)}

[0062] Then the convolution result of input x through R. The value of p0 at any position on y is:

[0063]

[0064] Where: p n is the position vector of R; Δpn is the position offset;

[0065] {Δp n |n=1,2,...,N},N=|R|

[0066] Offset Δp n Usually a decimal, the offset position is obtained by bilinear interpolation:

[0067]

[0068] Where p=p0+p n +Δp n is the position of the decimal; q is the integer position in the feature map; G() is a bilinear interpolation operation.

[0069] In step 4, the calculation formula of the introduced improved loss function Focal-EIoU Loss is as follows:

[0070]

[0071] The EIoU Loss consists of three parts: IoU loss, distance loss, and height-width loss (overlapping area, center point example, and aspect ratio); c With h c are the width and height of the minimum bounding rectangle of the predicted bounding box and the true bounding box, respectively; ρ is the Euclidean distance between two points. The height-width loss directly minimizes the difference in height and width between the predicted target bounding box and the true bounding box, resulting in faster convergence and better positioning results.

[0072] Step 6: Use the labeled vehicle target training set to train the improved YOLOv5s model.

[0073] In the model training parameter setting, the training environment uses Tesla V100 cloud GPU, the optimizer used in all improved training is AdamW, the initial learning rate is 0.01, the weight decay coefficient is 0.0005, the momentum is 0.937, the training iteration round is 300, the batch size is 16, and the input network image is set to 640 pixels * 640 pixels. The improved YOLOV5s model has improved the recognition accuracy, recall rate, and mAP_0.5 by 5%, 7%, and 5% respectively, which is significantly improved compared with the original algorithm; in terms of actual detection effect, such as Fig. 9 As shown in the figure, the improved model has better actual vehicle target detection effect (confidence).

[0074] Step 7: Parking space division: Manually divide the parking spaces in the image material to be detected, divide the parking spaces with parking space markings into separate parking space identification areas, divide the areas without parking space markings or unclear markings but actually available for parking vehicles into larger parking areas and set the maximum parking threshold, and return the parking space location information.

[0075] When calibrating parking spaces, use the cv2.VideoCapture() method to read the background image, use the pylab.show() method of the Matplotlib library to display the image, and define a function for calibrating parking spaces. Draw a parking space by clicking four points, and save the result as a text file.

[0076] Step 8: Load the trained YOLOV5s model, read the trained weight path, detect the vehicle target in the image, video or camera to be detected, return the vehicle classification information and the target center point location information, match the vehicle location information with the parking space location information saved in step 7. If the vehicle target center point is in the calibrated parking space, it is determined that the parking space is occupied and the parking space status judgment is completed. The actual judgment status is as follows: Fig.10 shown.

[0077] In actual parking space judgment, due to the camera's inclination, the coordinates of the center of the target detection frame returned by YOLO are not within the parking space, which can easily lead to misjudgment. Therefore, when the camera is located at the side of the vehicle and has a certain inclination with the parking line plane, when judging whether the parking space is occupied, the coordinate position of the center point of the lower edge of the detection frame is used as the basis for judging whether the parking space is occupied.

[0078] The present invention also provides a parking space status recognition system, comprising:

[0079] Image access module, used to access parking lot image resources to be detected;

[0080] A data enhancement module, used for performing inverse mosaic algorithm data enhancement on the vehicle target image to form an enhanced vehicle target image;

[0081] An image annotation module is used to annotate the enhanced vehicle target image to form a training sample;

[0082] A training module, used for training the improved YOLOv5s network using the training samples to obtain a vehicle target recognition model in a parking lot background; wherein the improved YOLOv5s network is a YOLOv5s network that adds a new Mymodel feature extraction module and optimizes network connections;

[0083] The parking space calibration module is used to calibrate the parking space in the background of the image to be detected and obtain the horizontal and vertical coordinates of the four vertices of the corresponding parking space;

[0084] The parking space status detection module is used to detect the image to be detected using the parking space status detection model to obtain a recognition result.

[0085] Compared with the prior art, the beneficial effects of a parking space status recognition system provided by the present invention are the same as the beneficial effects of a parking space status recognition method described in the above technical solution, which will not be elaborated here.

[0086] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technical solution that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A parking space status recognition method based on an improved YOLOv5s model, characterized by: The steps include: Step 1: Construct a vehicle target detection dataset in a parking context and divide it into a training set and a test set; Step 2: Flip-Mosaic data enhancement technology is used on the data set to alleviate the problem of sample size imbalance; Step 3: Annotate the enhanced data set to form a standardized data set; Step 4: Build an improved YOLOV5s network model; Step 5: Use the labeled vehicle target training set to train the improved YOLOv5s model; Step 6: Manually divide parking spaces in the image material to be detected, divide parking spaces with parking space markings into separate parking space identification areas, divide larger parking areas and set maximum parking thresholds for areas without parking space markings or unclear markings but actually available for parking, and return parking space location information; Step 7: Load the improved YOLOv5s model into the image to be detected for vehicle target detection, return the vehicle classification information and target center point location information, match the vehicle location information with the parking space location information, and if the vehicle target center point is within the calibrated parking space, the parking space is judged to be occupied, and the parking space status judgment is completed.

2. According to claim 1, a parking space status recognition method based on an improved YOLOV5s model is characterized in that: In the step 4, some feature extraction modules CSPDarknet53 in the original network of the YOLOV5s model are replaced with MyModel feature extraction modules suitable for the parking lot background, and the backbone and head structures are improved by using methods such as maximum pooling, dilated convolution, and variable convolution to form an improved YOLOV5s network model.

3. According to claim 2, a parking space status recognition method based on an improved YOLOV5s model is characterized in that: In step 4, the loss function of the improved YOLOv5 network is: The EIoU Loss consists of three parts: IoU loss, distance loss, and height-width loss (overlapping area, center point example, and aspect ratio); c With h c are the width and height of the minimum enclosing rectangle of the predicted bounding box and the true bounding box, respectively; ρ is the Euclidean distance between two points.

4. The parking space status recognition method based on the improved YOLOV5s model according to claim 1 is characterized in that: In step seven, a larger parking area is divided and a maximum parking threshold is set to obtain the parking space coordinates, and the coordinate position of the center point of the lower edge of the vehicle target detection frame is compared with the parking space coordinates to determine whether they are within the parking line, which is used as a basis for judging whether the parking space is occupied.

5. A parking space status recognition system, comprising: Image access module, used to access parking lot image resources to be detected; A data enhancement module, used for performing inverse mosaic algorithm data enhancement on the vehicle target image to form an enhanced vehicle target image; An image annotation module is used to annotate the enhanced vehicle target image to form a training sample; A training module, used for training the improved YOLOv5s network using the training samples to obtain a vehicle target recognition model in a parking lot background; wherein the improved YOLOv5s network is a YOLOv5s network that adds a new Mymodel feature extraction module and optimizes network connections; The parking space calibration module is used to calibrate the parking space in the background of the image to be detected and obtain the horizontal and vertical coordinates of the four vertices of the corresponding parking space; The parking space status detection module is used to detect the image to be detected using the parking space status detection model to obtain a recognition result.

Images