Parking space detection system and method

Abstract

The invention provides a parking space detection system and method, and the system comprises a feature extraction module which is used for receiving an input image and extracting a multi-scale feature map; the training module is connected to the feature extraction module and is used for optimizing network parameters in a model training stage; the total training loss output by the training module in the training stage is formed by weighted summation of detection loss, self-distillation loss and geometric constraint loss; and the optimizer module is connected to the training module and is used for updating network parameters of the system through a back propagation algorithm according to the total training loss. According to the invention, the accuracy of parking space detection and the robustness in a complex scene are obviously improved.

Description

[0001] A parking space detection system and method TECHNICAL FIELD The present application relates to the technical field of assisted parking, in particular to a parking space detection system and method. BACKGROUND

[0002] With the rapid development of automatic driving technology and intelligent parking systems, environmental perception, as its core foundation, has put unprecedentedly high requirements on accuracy, real-time performance and robustness. Among them, high-precision and high-efficiency parking space detection is one of the key technical links to realize autonomous parking function. Its core task is not only to identify the existence of parking spaces, but also to accurately determine their position, angle, size and availability, providing crucial input for vehicle path planning and control.

[0003] Currently, target detection algorithms based on deep learning have occupied an absolute dominant position in this field. Among them, the YOLO (You Only Look Once) series algorithm, due to its excellent balance between speed and accuracy, is widely used in real-time parking detection scenarios. YOLOv8, as one of the latest members of this family, inherits and develops the excellent features of its predecessors, adopting a multi-output head detection architecture. Specifically, its network is equipped with three detection heads at different depths (i.e. different feature scales), usually named P3, P4, P5 layers. This design constitutes an effective feature pyramid network (FPN), greatly improving the model's perception ability for targets of different sizes and significantly improving the model's recall rate.

[0004] However, existing general target detection algorithms usually treat each parking space as an independent and irrelevant detection target, using a "one-size-fits-all" paradigm for learning. This paradigm completely ignores the inherent and strong spatial geometric constraints in actual parking scenarios. In addition, although the use of three detection heads improves target recognition rate and accuracy, the complexity of the model also increases, increasing the model inference time. And the three-head structure will produce several times the number of prediction boxes of the single-head structure, making the non-maximum suppression (NMS) process a computational bottleneck of the entire detection pipeline, significantly increasing the end-to-end processing delay.

[0005] Therefore, there is an urgent need in the art for a special parking space detection scheme that can reduce computational complexity while maintaining high accuracy, and fully utilize scene prior knowledge to improve detection robustness. SUMMARY

[0006] The present application provides a parking space detection system and method, aiming to solve the defects in the prior art and realize significant improvement in the accuracy of parking space detection and robustness in complex scenarios.

[0007] To achieve the above object, the technical scheme adopted by the present application is: The present application provides a parking space detection system, comprising: A feature extraction module for receiving an input image and extracting a multi-scale feature map; A training module connected to the feature extraction module for optimizing network parameters in the model training stage; The total training loss output by the training module in the training stage is composed of the weighted sum of the detection loss, self-distillation loss and geometric constraint loss; An optimizer module connected to the training module for updating the network parameters of the system according to the total training loss through the backpropagation algorithm.

[0008] Specifically, the feature extraction module comprises: A lightweight CSPDarknet-53 backbone network for outputting initial feature maps of the first, second and third scales; An SPPF neck network connected to the backbone network for fusing and enhancing the initial feature maps and outputting enhanced first, second and third scale feature maps to the training module.

[0009] Specifically, the training module comprises a self-distillation unit, a feature alignment unit, a loss calculation and aggregation unit connected in sequence, and a geometric constraint unit.

[0010] Specifically, the self-distillation unit comprises a teacher branch and a student branch; The teacher branch corresponds to the output head of the enhanced first and second scale feature maps; The student branch corresponds to the output head of the enhanced third scale feature map.

[0011] Specifically, the feature alignment unit is used to: Use 1x1 convolution to uniformly adjust the channel number of the teacher branch output feature map to be consistent with the channel number of the student branch output feature map; Use bilinear interpolation up-sampling to uniformly adjust the spatial size of the teacher branch output feature map to be consistent with the size of the student branch output feature map.

[0012] Specifically, the geometric constraint module calculates the geometric constraint loss L angle : L angle =L dis *L adj Where L dis is a Smooth L1 loss for quantifying the difference between the predicted angle and the true angle, and Ladj Judge the loss for the adjacent parking space.

[0013] L dis Determined according to the following formula:

[0014] Wherein, N is the predicted result set, M is the real parking space number, Indicates the predicted parking space inclination angle value, Indicates the real labeled parking space inclination angle value.

[0015] L adj Determined according to the following formula:

[0016] Wherein, The center point of the i-th parking space, The center point of the j-th parking space. τ is a constant according to the actual setting of the system.

[0017] Specifically, the distillation loss Calculated by the following formula:

[0018] Wherein, Adapter(.) is an adapter that can automatically adapt the size of the feature, and is composed of convolution and down-sampling. F p3 , F p4 , F p5 Respectively represent the network features output by P3, P4 and P5 branches, T p3 , T p5 Indicate the category information output by P3 and P5 branches, T indicates the temperature hyperparameter during distillation, δ, λ and ω are hyperparameters.

[0019] Specifically, the loss calculation and aggregation unit calculates the system total loss L total Through the following formula:

[0020] Wherein, L det Is the detection task loss, L sd Is the self-distillation task loss, L angle Is the geometric angle constraint loss, and α, β and γ are hyperparameters.

[0021] Another aspect of the present application provides a parking space detection method, comprising: Step S1: obtaining a training image set containing parking spaces and labeling, the labeling information includes a bounding box, a category, a rotation angle and a parking space center point coordinate; Step S2: constructing the parking space detection system; Step S3: training the system using the training image set, optimizing network parameters by minimizing the total training loss; Step S4: after training, pruning the teacher branch and retaining the student branch to form a lightweight inference model; Step S5: using the inference model to detect parking spaces in an input image, and using a non-maximum suppression algorithm to process the output result of the inference model to remove redundant and overlapping prediction boxes.

[0022] Further, before step S3, a step of pre-processing the training image set is further included, and the pre-processing includes one or more of the following operations: using a Mosaic data enhancement technique to splice four training images into one; randomly horizontally flipping the training image; randomly disturbing the brightness of the training image within a range of ±20%; applying a Gaussian blur process with a kernel size of 3x3 to the training image.

[0023] The present application has the beneficial effects that: the present application constructs an end-to-end parking space detection model by fusing a self-distillation mechanism and a geometric constraint loss function, uses the detailed features of the teacher branch to guide the learning process of the student branch, and introduces a geometric consistency constraint based on the prior of the parking space in the training, thereby significantly improving the accuracy of parking space detection and the robustness in complex scenarios without increasing the inference calculation cost; at the same time, by pruning the teacher branch and retaining the lightweight student network deployment mode, the calculation bottleneck problem caused by multiple detection heads is effectively solved, thereby providing an efficient and reliable environmental perception basis for the automatic driving parking system. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 is a structural schematic diagram of the parking space detection system of the present application; Figure 2 is a flowchart of the parking space detection method of the present application. DETAILED DESCRIPTION

[0025] The embodiments of the present application will be specifically described below with reference to the accompanying drawings, which are only used for reference and illustration, and do not constitute a limitation on the scope of patent protection of the present application.

[0026] In the flow described in the specification, claims, or drawings of the present application, the serial numbers of various steps (such as steps 10, 20, etc.) are included, and the serial numbers are only used to distinguish various steps, and the serial numbers themselves do not represent any execution order. It should be noted that the "first", "second" and the like described herein are only used to distinguish the description objects and the like, and do not represent the order of precedence, nor do the "first", "second" and the like represent different types.

[0027] Embodiment 1 As Figure 1 shown, the present embodiment provides a parking space detection system, comprising: a feature extraction module for receiving an input image and extracting a multi-scale feature map.

[0028] a training module comprising a self-distillation unit, a feature alignment unit, a loss calculation and aggregation unit connected in turn, and further comprising a geometric constraint unit; the self-distillation unit is connected to the feature extraction module and comprises a teacher branch and a student branch; the teacher branch generates a teacher supervision signal based on the enhanced first scale feature map and the second scale feature map, and the student branch makes a prediction based on the third scale feature map and receives distillation guidance of the teacher supervision signal; wherein the scale of the student branch is lower than the scale of the teacher branch; the geometric constraint module is used to calculate a geometric constraint loss according to the spatial prior knowledge of parking arrangement in the model training stage, so as to constrain the angle consistency of the predicted parking space; the feature alignment unit is used to uniformly adjust the spatial size of the teacher branch feature map to be consistent with the size of the student branch feature map; the loss calculation and aggregation unit is used to calculate the total training loss of the system, which is composed of the detection loss, the self-distillation loss and the geometric constraint loss.

[0029] the optimizer module is used to update the system parameters through back propagation according to the total training loss of the system.

[0030] In the present embodiment, the feature extraction module is composed of a backbone network and a neck network in series.

[0031] The backbone network adopts a lightweight CSPDarknet-53 model, and the input image size is 640x640. The backbone network outputs three scales of initial feature maps: a first scale feature map FP3 (80x80 resolution), a second scale feature map FP4 (40x40 resolution), and a third scale feature map FP5 (20x20 resolution); The neck network adopts an SPPF (Spatial Pyramid Pooling - Fast) module to perform feature fusion and enhancement on FP3, FP4, and FP5, and outputs the first, second, and third enhanced feature maps FP3', FP4', and FP5'.

[0032] In this embodiment, the feature alignment unit adjusts the channel number of the teacher branch feature map to be consistent with the student branch (for example, 256 dimensions) through 1x1 convolution, and up-samples or down-samples the size thereof to the same resolution as the student branch feature map through bilinear interpolation.

[0033] In this embodiment, the teacher branch corresponds to the output head (P3) of the first enhanced scale feature map FP3' and the output head (P4) of the second enhanced scale feature map FP4' in the YOLOv8 architecture, and the student branch corresponds to the output head (P5) of the third enhanced feature map FP5' in the YOLOv8 architecture.

[0034] Each output layer of the student branch predicts the following information: Boundary box coordinates x , y , w , h ) (relative to the size of the input image).

[0035] Class confidence c (sigmoid activation, range [0, 1]).

[0036] Angle regression value θ (tanh activation, range [-1, 1], mapped to [-45°, 45°]).

[0037] In the standardized construction of indoor and outdoor parking lots, there is a clear rule for the slope / angle consistency of parking spaces, that is, the direction angle or slope of the boundary line of all parking spaces in the same row is the same or highly consistent.

[0038] Therefore, in this embodiment, the geometric constraint module calculates the geometric constraint loss L angle according to the following formula: L angle =L dis *L adj where L dis is a Smooth L1 loss used to quantify the difference between the predicted angle and the true angle, and L adj is a neighboring parking space judgment loss.

[0039] L dis is determined according to the following formula:

[0040] wherein N is the predicted result set, M is the real number of parking spaces, represents the predicted parking space inclination angle value, represents the real labeled parking space inclination angle value.

[0041] L adj is determined according to the following formula:

[0042] wherein, is the center point of the i-th parking space, is the center point of the j-th parking space. τ is a constant according to the actual setting of the system.

[0043] For example, in the present embodiment, τ is set to 500.

[0044] In the present embodiment, the loss calculation and aggregation unit calculates the system total loss L according to the following formula: total :

[0045] wherein L det is the detection task loss, L sd is the self-distillation task loss, L angle is the geometric angle constraint loss, and α, β, γ are hyperparameters.

[0046] In the present embodiment, α = 0.5, β = 0.3, and γ = 0.3.

[0047] In the present embodiment, the distillation loss L is calculated according to the following formula:

[0048] wherein Adapter(.) is an adapter that can automatically adapt the size of the features, and is composed of convolution and down-sampling. F p3 , F p4 , and F p5 respectively represent the network features output by the P3, P4, and P5 branches, T p3 , T p5 represent the class information output by the P3 and P5 branches, and T represents the temperature hyperparameter during distillation. δ, λ, and ω are hyperparameters.

[0049] In the present embodiment, δ = 0.1, λ = 0.1, ω = 0.5, and T = 4.

[0050] The detection task loss L det is calculated according to the following formula:

[0051] Among them, L Bbox L represents the target detection loss. cls L represents the classification loss. KP This indicates the loss from key point detection.

[0052] In this embodiment, the target detection loss function L Bbox It mainly consists of two parts: one part is the DFL (Distribution Focal Loss) loss L DFL The other part is the CIOU (Complete Intersection over Union) loss L. CIOU In other words, L Bbox Determined according to the following formula:

[0053] Here, ε1 and ε2 are the corresponding hyperparameters.

[0054] The DFL loss is mainly designed to address the issue of insufficient flexibility in the representation of the bounding box. DFL optimizes the two positions to the left and right of the label y. and The probability of L causes the network distribution to focus near the label value. DFL Determined by the following formula:

[0055] Where y represents the true value, yi represents the value of the left endpoint, yi+1 represents the value of the right endpoint, Si represents the probability of the model predicting the left endpoint yi, and Si+1 represents the probability of the model predicting the right endpoint yi+1.

[0056] CIOU loss, on the other hand, adds consideration of the bounding box's length and width to the IOU. CIOU Determined by the following formula:

[0057] Where IOU represents the intersection-union ratio, b represents the center point of the predicted bounding box, and b gt ρ(·) represents the center point of the ground truth box, ρ(·) represents the Euclidean distance between the two center points, and c represents the diagonal distance of the minimum closure region that can simultaneously contain both the predicted box and the ground truth box. η represents the weighting function, and v is a parameter that measures aspect ratio consistency; both are determined by the following formulas:

[0058]

[0059] Among them, w gt h gt The length and width of the ground truth bounding box are given by w and h, respectively, which represent the length and width of the predicted bounding box.

[0060] In this embodiment, the classification loss function L cls for:

[0061] Where C is the number of categories. It is the one-hot encoded vector of the actual label. This is the probability vector predicted by the model, obtained by passing the network output values ​​through the softmax function:

[0062] In this embodiment, the parking space key point loss function L KP It consists of two parts: one part is the L that determines whether the key point is visible. kpv The other part of the loss is L, which represents the cost of comparing the predicted point with the actual point. kpd Loss, i.e.:

[0063] Here, ε3 and ε4 are the corresponding hyperparameters.

[0064] In this embodiment, the key point of the parking space is the corner point of the parking space.

[0065] In this embodiment, L kpv The cross-entropy loss is used for binary classification, i.e.:

[0066] Where V is the number of categories. It is the one-hot encoded vector of the actual label. It is the probability vector predicted by the model, obtained by passing the network output value through the sigmoid function: .

[0067] After training is complete, the system enters the inference phase. At this point, the teacher branches (P3 head and P4 head) and feature alignment units will be fully pruned (removed), and only the lightweight student branch (P5 head) will be retained for the final prediction task.

[0068] Example 2 like Figure 2 As shown, this embodiment provides a parking space detection method, including: Step 1: Obtain and label the training image set containing parking spaces.

[0069] A large number of RGB images in parking lot environment are collected using the fisheye camera of the vehicle-mounted surround view, and the resolution is usually 1920x1080 or 1280x720. Each parking space in the image is finely labeled by a labeler using a labeling tool, and the labeling information includes: Bounding Box: label (x min , y min , x max , y max ) in normalized coordinates.

[0070] Class Label: 0 represents a vertical parking space, 1 represents a horizontal parking space, and 2 represents an inclined parking space.

[0071] Geometric properties: parking space center point coordinates (c x , c y ) (normalized values), and the inclination angle θ of the long side of the parking space (unit: degree, range [-45°, 45°]).

[0072] Step 2, pre-process the training image set.

[0073] The following enhancement strategies are applied to the training set images to improve the generalization ability of the model: Mosaic enhancement: randomly select 4 training images and splice them into a new composite image.

[0074] Random flip: flip the image horizontally / vertically with a probability of 0.5.

[0075] Brightness disturbance: randomly adjust the brightness of the image within ±20% of the original value.

[0076] Gaussian blur: randomly apply a Gaussian blur filter with a kernel size of 3x3.

[0077] All images are finally scaled to 640x640 pixels, and the pixel values are normalized to the range [0, 1], and the mean μ and standard deviation σ of the ImageNet dataset are used for standardization.

[0078] For example, ImageNet statistics: μ =[0.485,0.456,0.406], σ =[0.229,0.224,0.225] are standardized.

[0079] Step 3, build a parking space detection system.

[0080] Build a system model as described in Example 1, and the optimizer can use the Adam optimizer.

[0081] Step 4, training the system using the training image set, optimizing network parameters by minimizing the total training loss.

[0082] In specific implementation, the initial learning rate can be set to 0.001, the batch size is 32, the cosine annealing learning rate scheduling strategy is adopted, and a total of 300 epochs are trained. In each training iteration, the total loss L is calculated by forward propagation total , and all network parameters are updated by the back propagation algorithm.

[0083] Step 5, after training, the teacher branch is pruned, and the student branch is retained to form a lightweight inference model.

[0084] After training, load the model weight with the best performance on the validation set. Then, remove the teacher branch (P3 head, P4 head) and all parameters of the feature alignment module from the model to obtain a lightweight inference model containing only Backbone, Neck and student branch (P5 head), and save it.

[0085] Step 6, using the inference model to detect parking spaces in input images, using the non-maximum suppression algorithm to process the output results of the inference model to remove redundant and overlapping prediction boxes.

[0086] The image to be detected is input into the pruned inference model for forward propagation, and the model directly outputs the prediction results through the P5 head. The prediction results include multiple candidate detection boxes, their confidence, class and angle.

[0087] Finally, the non-maximum suppression (NMS) algorithm is applied, the confidence threshold is set to 0.5, the intersection over union (IoU) threshold is set to 0.45, and the prediction boxes with high redundancy and overlap are filtered out to obtain the final parking space detection result set.

[0088] The above disclosure is only the preferred embodiment of the present application, which cannot limit the scope of protection of the present application. Therefore, any equivalent changes made within the scope of the patent application of the present application are still within the scope of the present application.

Claims

1. A parking space detection system, characterized in that The system comprises: a feature extraction module configured to receive an input image and extract multi-scale feature maps; a training module connected to the feature extraction module and configured to optimize network parameters in a model training phase; a total training loss output by the training module in the training phase is composed of a weighted sum of a detection loss, a self-distillation loss and a geometric constraint loss; an optimizer module connected to the training module and configured to update network parameters of the system according to the total training loss through a back propagation algorithm.

2. The parking space detection system according to claim 1, characterized in that The feature extraction module comprises: a lightweight CSPDarknet-53 backbone network configured to output initial feature maps of first, second and third scales; an SPPF neck network connected to the backbone network and configured to fuse and enhance the initial feature maps and output enhanced feature maps of the first, second and third scales to the training module.

3. The parking space detection system of claim 1, wherein The training module comprises a self-distillation unit, a feature alignment unit, a loss calculation and aggregation unit connected in sequence, and a geometric constraint unit.

4. The parking space detection system according to claim 3, characterized in that The self-distillation unit comprises a teacher branch and a student branch; the teacher branch corresponds to output heads of the enhanced feature maps of the first and second scales; the student branch corresponds to an output head of the enhanced feature map of the third scale.

5. The parking space detection system of claim 3, wherein The feature alignment unit is configured to: use a 1x1 convolution to uniformly adjust the number of channels of the teacher branch output feature map to be consistent with the number of channels of the student branch output feature map; use a bilinear interpolation up-sampling to uniformly adjust the spatial size of the teacher branch output feature map to be consistent with the size of the student branch output feature map.

6. The parking space detection system of claim 1, wherein, The geometric constraint module calculates the geometric constraint loss L by the following formula angle : L angle =L dis *L adj wherein, L dis is a Smooth L1 loss for quantifying the difference between the predicted angle and the true angle, L adj is a neighboring parking space judgment loss; L dis was determined according to the following formula: Wherein, N is the predicted result set, M is the real number of parking spaces, represents the predicted parking space inclination angle value, represents the real labeled parking space inclination angle value; L adj was determined according to the following formula: wherein, is the center point of the i-th parking space, is the center point of the j-th parking space. τ is a constant according to the actual setting of the system.

7. The parking space detection system of claim 1, wherein, The distillation loss By the formula: where Adapter(.) is an adapter that can automatically adapt the size of features, which is composed of convolution and down-sampling. p3 , p4 , p5 respectively represent the network features output by P3, P4, P5 branches, T p3 , p5 represent the class information output by P3, P5 branches, T represents the temperature hyperparameter during distillation, and δ, λ, ω are hyperparameters.

8. The parking space detection system according to claim 7, characterized in that The loss computation and aggregation unit computes the system total loss L by the following equation total : wherein, L det is the detection task loss, L sd is the self-distillation task loss, L angle is the geometric angle constraint loss, and a, b, g are hyperparameters.

9. A parking space detection method using the system according to any one of claims 1 to 8, characterized by, The method comprises the following steps: Step S1: obtaining a training image set containing parking spaces and labeling, the labeling information including a bounding box, a class, a rotation angle and a parking space center point coordinate; Step S2: constructing the parking space detection system; Step S3: training the system using the training image set, and optimizing the network parameters by minimizing the total training loss; Step S4: after training, pruning the teacher branch and retaining the student branch to form a lightweight inference model; Step S5: using the inference model to detect parking spaces in an input image, and using a non-maximum suppression algorithm to process the output result of the inference model to remove redundant and overlapping prediction boxes.

10. The parking space detection method according to claim 9, characterized by, Before step S3, the method further comprises a step of preprocessing the training image set, the preprocessing comprising one or more of the following operations: using a Mosaic data enhancement technique to splice four training images into one; randomly horizontally flipping the training image; randomly disturbing the brightness of the training image within a range of ±20%; applying a Gaussian blur processing with a kernel size of 3x3 to the training image.

Images