A complete parking space inference method based on parking space features

Abstract

The present application relates to a kind of complete parking space inference methods based on parking space features, comprising steps 1: obtaining the panoramic image around vehicle and inputting panoramic image into feature detector;Step 2: according to the number of proximal parking space mark points in parking space entrance line boundary box, entrance line is paired with proximal parking space mark points;Step 3: according to the distance between parking space entrance line type and paired proximal parking space mark points, determine the type of parking space;Step 4: generate two groups of distal parking space mark points respectively located on the opposite sides of vehicle;Step 5: calculate the intersection-over-union of two parking spaces and vehicle, determine the distal parking space mark point coordinates of actual parking space;Step 6: according to the proximal mark point and distal mark point of parking space, locate parking space, obtain complete parking space.The present application only needs to detect parking space entrance line and mark point, and complete parking space can be inferred through simple calculation.The method is high in accuracy and small in calculation amount.

Description

Technical Field

[0001] This invention relates to the field of automotive parking space recognition technology, and more specifically, to a complete parking space inference method based on parking space features. Background Technology

[0002] Automated parking technology refers to a vehicle intelligently identifying available parking spaces, then calculating the vehicle's trajectory to enter the space. The vehicle control unit continuously adjusts the vehicle's angle and speed to guide it along the predetermined path. The parking process requires no driver intervention, enhancing the driver's experience. With the development of the automotive industry, the safety and accuracy requirements for automated parking technology are constantly increasing, leading to higher precision requirements for parking space recognition. To improve the accuracy of parking space recognition, various technologies have been employed. One existing image-based parking space recognition method uses convolutional neural network technology to extract the corner coordinates, the direction of the parking space dividing lines, and the midpoint coordinates of the parking space entry line as parking space features. These features are then used to infer the parking space's characteristics, achieving the function of detecting parking space features. This scheme trains the neural network model by collecting tens of thousands of panoramic images. Compared to traditional image processing techniques, convolutional neural network technology improves the accuracy of parking space detection. However, this scheme does not make a complete inference about the shape of the parking space; it only identifies the entrance and does not predict the shape of the parking space. Summary of the Invention

[0003] To address the problem that the aforementioned technical solutions do not provide a complete inference of the shape of parking spaces, this invention provides a method for inferring the complete shape of parking spaces based on their features. This method, in addition to identifying the parking space entrance, can accurately infer the complete shape of the parking space.

[0004] The technical solution adopted in this invention is: a complete parking space inference method based on parking space features, comprising the following steps:

[0005] Step 1: Acquire a panoramic image of the area around the vehicle and input the panoramic image into the feature detector. The feature detector selects and acquires parking space features in the panoramic image. The parking space features include the entrance line and the markers on the parking space. The corner point of the parking space closest to the vehicle is identified among the markers. This corner point is the near-end parking space marker.

[0006] Step 2: The feature detector matches the entrance line with the near-end parking space markers based on the number of markers appearing in the boundary box of the parking space entrance line, thus completing the identification of the parking space and determining the type of the parking space entrance line;

[0007] Step 3: Determine the parking space type based on the parking space entrance line type obtained in Step 2 and the distance between the paired near-end parking space markers;

[0008] Step 4: Generate the corner point on the side of the parking space furthest from the vehicle based on the parking space type determined in Step 3. This corner point is the far-end parking space marker. There are two sets of far-end parking space markers, located on opposite sides of the vehicle.

[0009] Step 5: Calculate the intersection-combination ratio of the two parking spaces and vehicles formed by the two sets of far-end parking space markers and near-end parking space markers in Step 4, and determine the coordinates of the far-end parking space marker of the actual parking space.

[0010] Step 6: Locate the parking space using the two near-end markers and two far-end markers to obtain the complete parking space.

[0011] This invention proposes a method for inferring complete parking spaces based on their feature information. Only the detector needs to detect the parking space entrance line and marker points to infer the complete parking space through simple calculations. This method is not only highly accurate and computationally efficient, solving parking space recognition problems in many situations, but it can also be applied to a wider range of scenarios. Furthermore, it does not rely on information from adjacent vehicles or additional communication devices, exhibiting strong robustness.

[0012] Preferably, in step 1, the panoramic camera on the vehicle acquires the scene around the vehicle, and then the image processing unit processes the scene from the panoramic camera to obtain a panoramic image of the vehicle's surroundings. The core of the panoramic camera for the vehicle lies in the addition of multiple cameras to the front and sides of the vehicle, and then the image processing unit stitches and deforms the scenes from the multiple cameras to achieve a 360° panoramic fusion around the vehicle.

[0013] Preferably, in step 2, the matching of the number of near-end parking space markers in the entrance line boundary box includes the following cases:

[0014] (1) If the boundary box of the entrance line contains two near-end parking space markers, then the two are paired near-end parking space markers and can be directly paired.

[0015] (2) If only one near-end parking space marker P1 is contained within the entrance line boundary box, then calculate the position of the near-end parking space marker P2 corresponding to near-end parking space marker P1 within the entrance line boundary box, pair the two near-end parking space markers P1 and P2, and the formula for calculating P2 is:

[0016]

[0017] Among them, b j This represents one of the four vertices of the entry line bounding box; ~b j b j The diagonal vertex; ΔW and ΔH are hyperparameters used to control the width and height of the ingress bounding box; j represents one of the vertices of the ingress bounding box.

[0018] (3) If the entrance line bounding box does not contain any marker points, then by calculating the normalized average intensity value of the four vertex regions of the entrance line bounding box, the point set with the maximum normalized average intensity value NAIV on the diagonal is selected as the pair of near-end parking space marker points P1 and P2. The two near-end parking space marker points P1 and P2 are paired. The formula for calculating the normalized average intensity value NAIV of the center i of the fixed-size pixel region is:

[0019]

[0020] Among them, NAIV i Let N be the NAIV of the fixed-size pixel region center i, max(I) be the maximum pixel intensity value of image I, and N and (x, y) be the regions R. i The number of pixels in the image and their positions on the x and y axes.

[0021] (4) If there are more than two marker points in the boundary box of the entrance line, the two marker points closest to the diagonal vertex are the pair of near-end parking space marker points. Pair the two near-end parking space marker points, and the formula for the relationship between the entrance line and the pair of near-end marker points after matching is as follows:

[0022]

[0023]

[0024]

[0025] Where P(x, y) is the center coordinate of the parking space entrance line bounding box; P1(x, y) and P2(x, y) are the center coordinates of the bounding boxes of the paired marker points that are paired with the entrance line; ΔW and ΔH are hyperparameters used to control the width and height of the entrance line bounding box; W1 is the width of the entrance line; and H1 is the height of the entrance line.

[0026] Preferably, in step 2, after matching the number of near-end parking space markers in the entrance line boundary box, the parking space entrance line type is identified. Based on the angle between the parking space entrance line and the parking space dividing line in the vehicle's forward direction, the parking space entrance line type is divided into right-angle entrance line, acute-angle entrance line, and obtuse-angle entrance line. Specifically, when the parking space entrance line is perpendicular to the parking space dividing line, it is a right-angle entrance line; when the parking space entrance line and the parking space dividing line form an acute angle, it is an acute-angle entrance line; and when the parking space entrance line and the parking space dividing line form an obtuse angle, it is an obtuse-angle dividing line.

[0027] Preferably, in step 3, the process of determining the parking space type includes the following steps:

[0028] (1) Classify parking spaces containing right-angled, acute-angled, or obtuse-angled entrance lines as right-angled parking spaces, acute-angled parking spaces, or obtuse-angled parking spaces.

[0029] (2) Further distinguish the types of right-angle parking spaces, determine whether the right-angle parking space is a perpendicular or parallel parking space, compare the distance L between the marker points with the width b of the parking space, when L > b it is a parallel parking space, when L <= b it is a perpendicular parking space;

[0030] (3) Determine the parking space angle and depth. After steps (1) and (2), the parking spaces are divided into perpendicular parking spaces, parallel parking spaces, acute-angle parking spaces, and obtuse-angle parking spaces. Then, according to the "Specifications for Setting Up Parking Spaces on Urban Roads", the four types of parking spaces applicable to small vehicles are calculated, with corresponding depths d of 6m, 6m, 5.2m, and 5.2m respectively; and corresponding parking space angles of 90°, 90°, 60°, and 120° respectively.

[0031] Preferably, in step 4, the calculation formulas for the invisible far-end parking space markers P3 and P4 in the panoramic image are as follows:

[0032]

[0033]

[0034] Where, α i and d i The parking space angle and depth are represented by P3 and P4, respectively. There are two growth directions for P3 and P4, which can generate two sets of marker points. In step 5, the two intersection-to-intersection ratios formed by the two parking spaces and the vehicle are calculated. If the smaller intersection-to-intersection ratio is less than 0.1, the direction of the parking space corresponding to the smaller intersection-to-intersection ratio is the correct parking direction. If the smaller intersection-to-intersection ratio is greater than 0.1, the process returns to step 2 to re-identify the parking space.

[0035] Preferably, prior to step 1, the feature detector is trained as a neural network to automatically mark parking space markers and entrance lines based on the panoramic vehicle image. The neural network training of the feature detector includes:

[0036] (1) Collect panoramic images of vehicles, mark the bounding boxes of the parking space entrance lines and the near-end parking space markers in the images, and organize the dataset into a format that the feature detector can input;

[0037] (2) Darknet-53, pre-trained on ImageNet, was used as a feature extractor based on the YOLOv3 detector. Then, it was fine-tuned on the dataset. During the fine-tuning process, the batch image size was set to 32, the image size was normalized to 416×416, and the anchor box information was modified.

[0038] Compared with existing technologies, this invention proposes a method for inferring complete parking spaces based on the feature information of parking spaces. Only the detector needs to detect the parking space entrance line and marker points to infer the complete parking space through simple calculations. This method is not only highly accurate and computationally efficient, solving parking space recognition problems in many situations, but it can also be applied to a wider range of scenarios. Furthermore, it does not rely on information from adjacent vehicles or additional communication devices, exhibiting extremely strong robustness. Attached Figure Description

[0039] Figure 1 This is a flowchart of a complete parking space inference method based on parking space features according to the present invention.

[0040] Figure 2 This is a schematic diagram of the boundary box of the parking space entrance line and marker points in a complete parking space inference method based on parking space features according to the present invention.

[0041] Figure 3 This is a schematic diagram of a vertical parking space, which is a complete parking space inference method based on parking space features according to the present invention.

[0042] Figure 4 This is a schematic diagram of parallel parking spaces, representing a complete parking space inference method based on parking space features according to the present invention.

[0043] Figure 5 This is a schematic diagram of two sets of remote parking space markers, representing a complete parking space inference method based on parking space features according to the present invention.

[0044] Figure 6 This is a schematic diagram illustrating the method for determining the parking direction of a parking space based on parking space features, according to the present invention. Detailed Implementation

[0045] The accompanying drawings are for illustrative purposes only and should not be construed as limiting this patent. To better illustrate this embodiment, some components in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings. The positional relationships described in the drawings are for illustrative purposes only and should not be construed as limiting this patent.

[0046] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar parts. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "long," and "short" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present patent. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0047] The technical solution of the present invention will be further described in detail below through specific embodiments and in conjunction with the accompanying drawings:

[0048] Example 1

[0049] like Figures 1-2 The following is an embodiment of a complete parking space inference method based on parking space features, including the following steps:

[0050] Step 1: Acquire a panoramic image of the area around the vehicle and input the panoramic image into the feature detector. The feature detector selects and acquires parking space features in the panoramic image. The parking space features include the entrance line and the markers on the parking space. The corner point of the parking space closest to the vehicle is identified among the markers. This corner point is the near-end parking space marker.

[0051] Step 2: The feature detector matches the entrance line with the near-end parking space markers based on the number of markers appearing in the boundary box of the parking space entrance line, thus completing the identification of the parking space and determining the type of the parking space entrance line;

[0052] Step 3: Determine the parking space type based on the parking space entrance line type obtained in Step 2 and the distance between the paired near-end parking space markers;

[0053] Step 4: Generate the corner point on the side of the parking space furthest from the vehicle based on the parking space type determined in Step 3. This corner point is the far-end parking space marker. There are two sets of far-end parking space markers, located on opposite sides of the vehicle.

[0054] Step 5: Calculate the intersection-combination ratio of the two parking spaces and vehicles formed by the two sets of far-end parking space markers and near-end parking space markers in Step 4, and determine the coordinates of the far-end parking space marker of the actual parking space.

[0055] Step 6: Locate the parking space using the two near-end markers and two far-end markers to obtain the complete parking space.

[0056] The beneficial effects of this embodiment are as follows: Based on the feature information of parking spaces, this invention proposes a method for inferring complete parking spaces. Only the detector needs to detect the parking space entrance line and marker point; the complete parking space can be inferred through simple calculations. This method is not only highly accurate and computationally efficient, solving parking space recognition problems in many situations, but it can also be applied to a wider range of scenarios. Furthermore, it does not rely on information from adjacent vehicles or additional communication devices, exhibiting extremely strong robustness.

[0057] Example 2

[0058] Example 2 of a complete parking space inference method based on parking space features, as follows: Figures 3-6 As shown, steps 1 to 5 are further defined.

[0059] Specifically, in step 2, the matching of the number of near-end parking space markers within the entrance line boundary box includes the following cases:

[0060] (1) If the boundary box of the entrance line contains two near-end parking space markers, then the two are paired near-end parking space markers and can be directly paired.

[0061] (2) If only one near-end parking space marker P1 is contained within the entrance line boundary box, then calculate the position of the near-end parking space marker P2 corresponding to near-end parking space marker P1 within the entrance line boundary box, pair the two near-end parking space markers P1 and P2, and the formula for calculating P2 is:

[0062]

[0063] Among them, b j This represents one of the four vertices of the entry line bounding box; ~b j b j The diagonal vertex; ΔW and ΔH are hyperparameters used to control the width and height of the ingress bounding box, and j represents one of the vertices of the ingress bounding box.

[0064] (3) If the entrance line bounding box does not contain any marker points, then by calculating the normalized average intensity value of the four vertex regions of the entrance line bounding box, the point set with the maximum normalized average intensity value NAIV on the diagonal is selected as the pair of near-end parking space marker points P1 and P2. The two near-end parking space marker points P1 and P2 are paired. The formula for calculating the normalized average intensity value NAIV of the center i of the fixed-size pixel region is:

[0065]

[0066] Among them, NAIV i Let N be the NAIV of the center i of a fixed-size pixel region, max(I) be the maximum pixel intensity value of image I, and N and (x,y) be the values ​​of regions R. iThe number of pixels in the image and their positions on the x and y axes.

[0067] (4) If there are more than two marker points in the boundary box of the entrance line, the two marker points closest to the diagonal vertex are the pair of near-end parking space marker points. Pair the two near-end parking space marker points, and the formula for the relationship between the entrance line and the pair of near-end marker points after matching is as follows:

[0068]

[0069]

[0070]

[0071] Where P(x,y) is the center coordinate of the parking space entrance line bounding box; P1(x,y) and P2(x,y) are the center coordinates of the bounding boxes of the paired marker points that are paired with the entrance line; ΔW and ΔH are hyperparameters used to control the width and height of the entrance line bounding box; W1 is the width of the entrance line; and H1 is the height of the entrance line.

[0072] Specifically, in step 2, after matching the number of parking space markers near the entrance line boundary box, the type of parking space entrance line is identified. Based on the angle between the parking space entrance line and the parking space dividing line in the direction of vehicle movement, the parking space entrance line type is divided into right-angle entrance line, acute-angle entrance line, and obtuse-angle entrance line. Specifically, when the parking space entrance line is perpendicular to the parking space dividing line, it is a right-angle entrance line; when the parking space entrance line and the parking space dividing line form an acute angle, it is an acute-angle entrance line; and when the parking space entrance line and the parking space dividing line form an obtuse angle, it is an obtuse-angle dividing line.

[0073] Specifically, in step 3, the process of determining the parking space type includes the following steps:

[0074] (1) Classify parking spaces containing right-angled, acute-angled, or obtuse-angled entrance lines as right-angled parking spaces, acute-angled parking spaces, or obtuse-angled parking spaces.

[0075] (2) Further distinguish the types of right-angle parking spaces, determine whether the right-angle parking space is a perpendicular or parallel parking space, compare the distance L between the marker points with the width b of the parking space, when L>b it is a parallel parking space, when L<=b it is a perpendicular parking space;

[0076] (3) Determine the parking space angle and depth. After steps (1) and (2), the parking spaces are divided into perpendicular parking spaces, parallel parking spaces, acute-angle parking spaces, and obtuse-angle parking spaces. Then, according to the "Specifications for Setting Up Parking Spaces on Urban Roads", the four types of parking spaces applicable to small vehicles are calculated, with corresponding depths d of 6m, 6m, 5.2m, and 5.2m respectively; and corresponding parking space angles of 90°, 90°, 60°, and 120° respectively.

[0077] Specifically, in step 4, the calculation formulas for the invisible far-end parking space markers P3 and P4 in the panoramic image are as follows:

[0078]

[0079]

[0080] Where, α i and d i The parking space angle and depth are represented by P3 and P4, respectively. There are two growth directions for P3 and P4, which can generate two sets of marker points. In step 5, the two intersection-to-intersection ratios formed by the two parking spaces and the vehicle are calculated. If the smaller intersection-to-intersection ratio is less than 0.1, the direction of the parking space corresponding to the smaller intersection-to-intersection ratio is the correct parking direction. If the smaller intersection-to-intersection ratio is greater than 0.1, the process returns to step 2 to re-identify the parking space.

[0081] The beneficial effects of this embodiment are: the detailed description of steps 1 to 5 simplifies the computational workload of this method.

[0082] Example 3

[0083] Example 3 of a complete parking space inference method based on parking space features, further restricting the training of the feature detector.

[0084] Before step 1, the feature detector is trained as a neural network to automatically mark parking space markers and entrance lines based on the panoramic vehicle image. The neural network training for the feature detector includes:

[0085] (1) Collect panoramic images of vehicles, mark the bounding boxes of the parking space entrance lines and the near-end parking space markers in the images, and organize the dataset into a format that the feature detector can input;

[0086] (2) Darknet-53, pre-trained on ImageNet, was used as a feature extractor based on the YOLOv3 detector. Then, it was fine-tuned on the dataset. During the fine-tuning process, the batch image size was set to 32, the image size was normalized to 416×416, and the anchor box information was modified.

[0087] The beneficial effects of this embodiment are as follows: The training method for the feature detector is specifically described, which enables the feature detector to accurately identify parking space features.

[0088] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A complete parking space inference method based on parking space features, characterized in that, Includes the following steps: Step 1: Acquire a panoramic image of the area around the vehicle and input the panoramic image into a feature detector. The feature detector selects and acquires parking space features in the panoramic image. The parking space features include the entrance line and the markers on the parking space. The detector also identifies the corner point of the parking space that is close to the vehicle among the markers. This corner point is the near-end parking space marker. Step 2: The feature detector matches the entrance line with the near-end parking space markers based on the number of markers appearing in the boundary box of the parking space entrance line, thus completing the identification of the parking space and determining the type of the parking space entrance line; Step 3: Determine the parking space type based on the parking space entrance line type obtained in Step 2 and the distance between the paired near-end parking space markers; Step 4: Generate the corner point on the side of the parking space furthest from the vehicle based on the parking space type determined in Step 3. This corner point is the far-end parking space marker. There are two sets of far-end parking space markers, located on opposite sides of the vehicle. Step 5: Calculate the intersection-combination ratio of the two parking spaces and vehicles formed by the two sets of far-end parking space markers and near-end parking space markers in Step 4, and determine the coordinates of the far-end parking space marker of the actual parking space. Step 6: Locate the parking space using the two near-end markers and two far-end markers to obtain the complete parking space; In step 2, matching the number of near-end parking space markers within the entrance line boundary includes the following cases: (1) If the boundary box of the entrance line contains two near-end parking space markers and ,but and These are paired parking space markers, directly matched; (2) If there is only one near-end parking space marker If the parking space is included within the entrance line boundary box, then the distance between the parking space marker and the nearest parking space marker within the entrance line boundary box is calculated. Corresponding near-end parking space marker The positions of the two near-end parking space markers and pair; (3) If the entrance line boundary box does not contain any marker points, then the normalized average intensity values ​​of the four vertex regions of the entrance line boundary box are calculated, and the point set with the maximum normalized average intensity value on the diagonal is selected as the pair of near-end parking space marker points. and The two near-end parking space markers and pair; (4) If there are more than two marker points in the boundary box of the entrance line, the two marker points closest to the diagonal vertex are the pair of near-end parking space marker points. and The two near-end parking space markers and pair; After matching the number of parking space markers near the entrance line boundary, the type of parking space entrance line is identified. Based on the angle between the parking space entrance line and the parking space dividing line in the direction of vehicle movement, the parking space entrance line type is divided into right-angle entrance line, acute-angle entrance line, and obtuse-angle entrance line. Specifically, when the parking space entrance line is perpendicular to the parking space dividing line, it is a right-angle entrance line; when the parking space entrance line and the parking space dividing line form an acute angle, it is an acute-angle entrance line; and when the parking space entrance line and the parking space dividing line form an obtuse angle, it is an obtuse-angle dividing line. In step 3, the process of determining the parking space type includes the following steps: (1) Parking spaces containing right-angled, acute-angled, or obtuse-angled entrance lines are classified as right-angled parking spaces, acute-angled parking spaces, or obtuse-angled parking spaces; (2) Further distinguish the types of right-angle parking spaces, determine whether the right-angle parking space is a perpendicular or parallel parking space, compare the distance L between the marker points with the width b of the parking space, when L>b it is a parallel parking space, when L<=b it is a perpendicular parking space; (3) Determine the angle and depth of the parking space. After steps (1) and (2), the parking space is divided into perpendicular parking space, parallel parking space, acute angle parking space and obtuse angle parking space, and their corresponding depths d are 6m, 6m, 5.2m and 5.2m respectively; and their corresponding parking space angles are 90°, 90°, 60° and 120° respectively. In step 4, the distant parking space markers that are not visible in the panoramic image. and The calculation formula is as follows: in, and These refer to the angle and depth of the parking space, respectively. and There are two growth directions, and calculations can generate two sets of marker points.

2. The complete parking space inference method based on parking space features according to claim 1, characterized in that, In step 1, the panoramic camera on the car acquires the scene around the car, and then the image processing unit processes the scene from the panoramic camera to obtain a panoramic image of the car's surroundings.

3. The complete parking space inference method based on parking space features according to claim 1, characterized in that, In step 5, if the smaller of the two intersection ratios formed by the two parking spaces and the vehicle is less than 0.1, then the direction of the parking space corresponding to the smaller intersection ratio is the correct parking direction; if the smaller of the two intersection ratios formed by the two parking spaces and the vehicle is greater than 0.1, then return to step 2 to re-identify the parking space.

4. The complete parking space inference method based on parking space features according to claim 1, characterized in that, Before step 1, the feature detector is trained as a neural network so that it can automatically mark parking space markers and entrance lines based on the panoramic image of the vehicle.

5. The complete parking space inference method based on parking space features according to claim 4, characterized in that, The neural network training of the feature detector includes: (1) Collect panoramic images of vehicles, mark the bounding boxes of the parking space entrance lines and the near-end parking space markers in the images, and organize the dataset into a format that the feature detector can input; (2) Darknet-53, pre-trained on ImageNet, was used as a feature extractor based on the YOLOv3 detector. Then, it was fine-tuned on the dataset. During the fine-tuning process, the image size was batched and the anchor box information was modified.

6. The complete parking space inference method based on parking space features according to claim 5, characterized in that, The image batch size is set to 32, and the image size is normalized to 416×416.

Images