A deep learning-based parking space positioning method
By constructing a dual-branch network model with shared weights and updating the loss function, and utilizing the contextual information of the video sequence for parking space localization, the problem of parking space offset caused by vehicle movement is solved, thereby improving the accuracy and robustness of parking space recognition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- FORYOU GENERAL ELECTRONICS
- Filing Date
- 2022-09-24
- Publication Date
- 2026-06-05
AI Technical Summary
Existing deep learning-based parking space detection methods fail to effectively utilize contextual semantic information in video sequences, leading to parking space identification deviations when vehicles are moving, thus affecting accurate positioning.
A dual-branch network model with shared weights is constructed. Parking spaces are located by template similarity matching using contextual information from video sequences. The network model is then updated using a loss function to improve recognition accuracy.
By utilizing the contextual information and loss constraints of video sequences, the problem of parking space offset caused by vehicle movement is effectively solved, improving the accuracy and robustness of parking space recognition.
Smart Images
Figure CN115439829B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of automatic parking technology, and in particular to a parking space location method based on deep learning. Background Technology
[0002] With the continuous development of intelligent vehicles, automated parking technology, as an indispensable component of the intelligent vehicle field, has gradually gained attention. In recent years, deep learning technology has been widely applied in many areas of computer vision. Compared with traditional methods, deep learning can solve problems such as target retrieval and recognition in more complex scenarios. In the field of automated parking, deep learning-based parking space detection methods have gradually demonstrated superior performance. Robustly perceiving the parking space position in each frame of an image during automated parking using deep learning methods is an important research topic.
[0003] Existing deep learning-based parking space detection methods use convolutional neural network models to extract key features frame by frame from the video during the automatic parking process and identify the location of the parking space in each frame.
[0004] The main problems with current deep learning-based parking space detection methods are: First, convolutional neural network models perform independent parking space identification for each frame of the video, without utilizing the contextual semantic information in the video sequence and ignoring the strong correlation between each frame of the video sequence; Second, when a vehicle parks in a target parking space, the relative distance and direction between the vehicle and the target parking space will change, which can easily cause the defined parking space area to shift, affecting the accurate positioning of the target parking space. Summary of the Invention
[0005] This invention provides a deep learning-based parking space location method, aiming to address the shortcomings of existing technologies, solve the problem of parking space misalignment caused by vehicle movement during automatic parking, and improve the accuracy and robustness of parking space recognition.
[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0007] This invention provides a parking space location method based on deep learning, comprising:
[0008] Step 1: Pre-train the initial convolutional neural network to obtain the basic network;
[0009] Step 2: Construct a dual-branch network with shared weights based on the basic network;
[0010] Step 3: Train the dual-branch network to obtain a deep learning network model;
[0011] Step 4: Obtain a top-down view of the vehicle's location;
[0012] Step 5: Using the deep learning network model, identify parking spaces in the top view of the obtained vehicle location.
[0013] Specifically, step 1 includes: using AlexNet as the backbone network of the initial convolutional neural network, and pre-training the backbone network using parking space image data and its corresponding labels.
[0014] Specifically, step 2 includes: using two base networks of different depths to form a set of dual-branch network models, using the deep base network branch as the template branch and the shallow base network branch as the search branch, with the deep base network and the shallow base network sharing weights in the shallow layer.
[0015] Specifically, step 3 includes: using a small-sized feature map as a template, searching for a corresponding template region in a large-sized feature map, and using context information to search for the response value map of the previous frame input image frame by frame until a feature map with high similarity to the template is matched in the search region.
[0016] Specifically, step 3 includes:
[0017] Step 301: Input the i-th frame image into the template branch, perform convolution to extract features, obtain the corresponding first feature map, and use it as the template;
[0018] Step 302: Simultaneously, input the (i+1)th frame image into the search branch, perform convolution to extract features, and obtain the corresponding second feature map;
[0019] Step 303: Convolve the second feature map using the first feature map as the kernel according to a preset rule to obtain a response value map.
[0020] Specifically, the preset rule is as follows:
[0021] f(x i ,x i+1 )=η(x i )*η(x i+1 )+b
[0022] Where, f(x) i ,x i+1 ) represents the image x i and image x i+1 The response value graph, η(x) i ) represents the first feature map, η(x) i+1 ) represents the second feature map, b represents the bias value, and * represents the convolution operation.
[0023] Specifically, step 5 includes: restoring the original image based on the response value map using interpolation to determine the location of the parking space.
[0024] Furthermore, following step 5, the following is also included:
[0025] Step 6: Update the deep learning network model using the loss function.
[0026] Specifically, step 6 includes:
[0027] Step 601: Determine the loss for each sample according to the first formula;
[0028] Step 602: Calculate the total loss of all sample points based on the mean of the losses of all sample points;
[0029] Step 603: Determine the total loss of the deep learning network model training according to the second formula.
[0030] Specifically, the first formula is:
[0031] l(y,s)=log(1+e -ys )
[0032] Where l(y,s) represents the loss for each sample, e is the natural constant, s represents the score of the corresponding sample point in the response graph, and the value is proportional to the probability that the predicted value is a positive sample; y represents the label of the sample point, which indicates the mark of the true value region in the response graph.
[0033] The label y of the sample point is calculated according to the following formula:
[0034]
[0035] Where k is the factor by which the response table is reduced after the input image is convolved, c represents the center of the response table, u represents any point in the response table, ||uc||2 represents the L2 norm, and R represents the distance threshold.
[0036] Specifically, the second formula is:
[0037] L = L id +L(y,s)
[0038] Where L represents the total loss during training of the deep learning network model, L id Let L(y,s) represent the classification loss, and let L(y,s) represent the total loss for all sample points.
[0039] The beneficial effects of this invention are as follows: By constructing a dual-branch network model, the top view of the vehicle position is input frame by frame. Each frame uses the feature map obtained by convolution of the previous frame as a template for template similarity measurement and matching. The contextual information of the video sequence is used to locate the accurate parking space position, thereby improving the accuracy of parking space recognition. Furthermore, by using loss constraints and updating the network model, the problem of parking space deviation caused by vehicle movement during automatic parking is effectively solved, thus improving the accuracy and robustness of parking space recognition. Attached Figure Description
[0040] Figure 1 This is a flowchart illustrating the deep learning-based parking space location method of the present invention.
[0041] Figure 2 This is a schematic diagram of the dual-branch network of the present invention. Detailed Implementation
[0042] The embodiments of the present invention are described in detail below with reference to the accompanying drawings. The drawings are for reference and illustration only and do not constitute a limitation on the scope of protection of the present invention.
[0043] In the process described in the specification, claims, or drawings of this invention, each step is numbered (e.g., step 10, 20, etc.). These numbers are used only to distinguish the steps and do not represent any execution order. It should be noted that the terms "first," "second," etc., used herein are only for distinguishing the objects being described and do not represent a chronological order, nor do they indicate that "first," "second," etc., are different types.
[0044] like Figure 1 As shown, this embodiment provides a parking space recognition method based on deep learning, including:
[0045] Step 1: Pre-train the initial convolutional neural network to obtain the basic network.
[0046] In this embodiment, step 1 includes: using AlexNet as the backbone network of the initial convolutional neural network, and pre-training the backbone network using parking space image data and its corresponding labels.
[0047] To further improve the network's discriminative and adaptive capabilities, the backbone network is pre-trained using classification loss.
[0048] In this embodiment, the classification loss is:
[0049]
[0050] Where p(x) i |y i ) represents the input image xi Belongs to the real label y i The probability of.
[0051] This step yields a pre-trained AlexNet, which we call the base network.
[0052] Step 2: Construct a dual-branch network with shared weights based on the basic network.
[0053] like Figure 2 As shown, in this embodiment, step 2 includes: using two base networks of different depths to form a set of dual-branch network models, using the deep base network branch as the template branch and the shallow base network branch as the search branch, and the deep base network and the shallow base network share weights in the shallow layer.
[0054] Feature maps obtained from deep base networks through convolutions will have smaller feature maps due to the greater number of convolution and pooling layers, while shallow base networks will have larger feature maps.
[0055] Step 3: Train the dual-branch network to obtain a deep learning network model.
[0056] In this embodiment, step 3 includes: using a small-sized feature map as a template, searching for a corresponding template region in a large-sized feature map, and using context information to search frame by frame for the response value map of the previous frame input image until a feature map with high similarity to the template is matched in the search region.
[0057] In this embodiment, the above method is implemented through the following steps:
[0058] Step 301: Input the i-th frame image into the template branch, perform convolution to extract features, and obtain the corresponding first feature map η(x). i ), and use it as a template;
[0059] Step 302: Simultaneously, input the (i+1)th frame image into the search branch, perform convolution to extract features, and obtain the corresponding second feature map η(x). i+1 );
[0060] Step 303: In the second feature map η(x) i+1 The first feature map η(x) is used as the basis for the above. i The convolution kernel is used to perform convolution according to a preset rule to obtain the response value map f(x). i ,x i+1 ).
[0061] In this embodiment, the preset rule is:
[0062] f(x i ,xi+1 )=η(x i )*η(x i+1 )+b
[0063] Where, f(x) i ,x i+1 ) represents the image x i and image x i+1 The response value graph, η(x) i ) represents the first feature map, η(x) i+1 ) represents the second feature map, b represents the bias value, and * represents the convolution operation.
[0064] Step 4: Obtain a top view of the vehicle's location.
[0065] In practice, a video of the vehicle's location can be captured by a surround-view camera around the vehicle, and after distortion correction and stitching fusion, a top-down view video of the vehicle's location can be obtained. Then, the top-down view video can be cropped frame by frame to obtain a top-down view of the vehicle's location.
[0066] Step 5: Using the deep learning network model, identify parking spaces in the top view of the obtained vehicle location.
[0067] In this embodiment, step 5 includes: restoring the original image based on the response value map using interpolation to determine the location of the parking space.
[0068] This step utilizes the contextual information of the video sequence to locate the accurate parking space, overcoming the limitations of existing technologies that do not utilize the contextual semantic information in the video sequence for parking space identification, thereby improving the accuracy of parking space identification.
[0069] In another embodiment of the invention, the method further includes the following after step 5:
[0070] Step 6: Update the deep learning network model using the loss function.
[0071] In practice, points within a certain range of the target area are selected as positive samples, and points outside the range are negative samples. Logistic loss is used to constrain the parking space location area.
[0072] In this embodiment, step 6 includes:
[0073] Step 601: Determine the loss for each sample according to the first formula.
[0074] In this embodiment, the first formula is:
[0075] l(y,s)=log(1+e -ys )
[0076] Where e is the natural constant, s represents the score of the corresponding sample point in the response value graph, and the value is proportional to the probability that the predicted value is a positive sample; y represents the label of the sample point, which indicates the mark of the true value region in the response graph.
[0077] In this embodiment, the label y of the sample point is calculated according to the following formula:
[0078]
[0079] Where k is the factor by which the response table is reduced after the input image is convolved, c represents the center of the response table, u represents any point in the response table, ||uc||2 represents the L2 norm, and R represents the distance threshold.
[0080] When the value of y is 1, it means that the sample point is near the target; when the value of y is -1, it means that the sample point has been located in the wrong position. In this case, the returned loss function value will be very large, and it is necessary to search in the search area of the next frame again.
[0081] Step 602: Calculate the total loss L(y,s) of all sample points based on the mean of the losses of all sample points.
[0082] That is, the total loss L(y,s) is expressed as:
[0083]
[0084] Step 603: Determine the total loss L for training the deep learning network model according to the second formula.
[0085] In this embodiment, the second formula is:
[0086] L = L id +L(y,s)
[0087] Where L represents the total loss during training of the deep learning network model, L id Let L(y,s) represent the classification loss, and let L(y,s) represent the total loss for all sample points.
[0088] By utilizing total loss constraints, the convolutional neural network model is continuously optimized, enabling the dual-branch network model to robustly locate the parking space position as the camera moves during automatic parking.
[0089] The above-disclosed embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Therefore, any equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.
Claims
1. A parking space location method based on deep learning, characterized in that, include: Step 1: Pre-train the initial convolutional neural network to obtain the basic network; Step 2: Construct a dual-branch network with shared weights based on the base network, including: using two base networks of different depths to form a set of dual-branch network models, using the deep base network branch as the template branch and the shallow base network branch as the search branch, wherein the deep base network and the shallow base network share weights in the shallow layer. Step 3: Train the dual-branch network model to obtain a deep learning network model, including: using a small-sized feature map as a template, searching for the corresponding template region in the large-sized feature map, and using context information to search for the response value map of the previous frame input image frame by frame until a feature map with high similarity to the template is matched in the search region. Step 4: Obtain a top-down view of the vehicle's location; Step 5: Using the deep learning network model, identify parking spaces in the top view of the obtained vehicle location.
2. The parking space location method based on deep learning according to claim 1, characterized in that, Step 1 includes: using AlexNet as the backbone network of the initial convolutional neural network, and pre-training the backbone network using parking space image data and its corresponding labels.
3. The parking space location method based on deep learning according to claim 2, characterized in that, Step 3 includes: Step 301: Input the i-th frame image into the template branch, perform convolution to extract features, obtain the corresponding first feature map, and use it as the template; Step 302: Simultaneously, input the (i+1)th frame image into the search branch, perform convolution to extract features, and obtain the corresponding second feature map; Step 303: Convolve the second feature map using the first feature map as the kernel according to a preset rule to obtain a response value map; The preset rule is as follows: f(x i ,x i+1 )=η(x i )*η(x i+1 )+b Where, f(x) i ,x i+1 ) represents the image x i and image x i+1 The response value graph, η(x) i ) represents the first feature map, η(x) i+1 ) represents the second feature map, b represents the bias value, and * represents the convolution operation.
4. The deep learning-based parking space location method according to claim 3, characterized in that, Step 5 includes: restoring the original image based on the response value map using interpolation to determine the location of the parking space.
5. The parking space location method based on deep learning according to claim 1, characterized in that, The process after step 5 also includes: Step 6: Update the deep learning network model using the loss function.
6. The parking space location method based on deep learning according to claim 5, characterized in that, Step 6 includes: Step 601: Determine the loss for each sample according to the first formula; Step 602: Calculate the total loss of all sample points based on the mean of the losses of all sample points; Step 603: Determine the total loss of the deep learning network model training according to the second formula; The first formula is: l(y,s)=log(1+e -ys ) in, l(y,s) The loss for each sample is represented by e, which is the natural constant; s represents the score of the corresponding sample point in the response graph, and its value is proportional to the probability that the predicted value is a positive sample; y represents the label of the sample point, which is the mark of the true value region in the response graph. The label y of the sample point is calculated according to the following formula: Where k is the factor by which the response table is reduced after the input image is convolved, c represents the center of the response table, u represents any point in the response table, ||uc||2 represents the L2 norm, and R represents the distance threshold; The second formula is: L=L id + L(y,s) Where L represents the total loss during training of the deep learning network model, L id Represents classification loss, L(y,s) This represents the total loss for all sample points.