A parking space fusion calculation method and system

Abstract

This invention provides a fusion calculation method for parking spaces. It includes the following steps: acquiring image information of a target parking space, first obstacle information, and second obstacle information, wherein the first obstacle is parked in a first parking space, the second obstacle is parked in a second parking space, and the target parking space is located between the first and second parking spaces; calculating parking space information of the target parking space based on the image information, the first obstacle information, and the second obstacle information, wherein the parking space information includes entrance point information and non-entrance point information of the target parking space; if the parking space information meets the parking needs of the target vehicle, controlling the target vehicle to automatically park in the target parking space; if the parking space information does not meet the parking needs of the target vehicle, controlling the target vehicle to continue moving. This invention solves the problems of slow speed in querying valid parking spaces and bottlenecks in non-entrance point detection in existing technologies.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving, and in particular to a fusion calculation method and system for parking spaces. Background Technology

[0002] Automation and intelligence are current industry trends, especially in the field of intelligent driving vehicles, which has been developing for many years. Assisted parking and automatic parking are also good options, greatly alleviating the difficulties faced by novice drivers. Accurate detection of parking spaces and the precision of parking availability play a crucial role in successful parking. Providing stable parking space coordinate information faster and more accurately is essential for assisted and automatic parking. Parking space integration is a vital component of this process.

[0003] The principle of visual and ultrasonic fusion in existing methods is as follows: For parking spaces detected visually, a feasible binary region value is used to search for whether a parking space is available within an a*a search sampling step. During tracking, a temporal association is performed using the parking space IoU method. If the IoU is greater than a preset threshold, the lifespan of that parking space is incremented by 1; if the lifespan of a parking space exceeds the preset threshold, then the parking space truly exists. The intersection of the bird's-eye view of the drivable area generated by the visual system and the ultrasonic radar system is fused in the world coordinate system. Furthermore, obstacle motion estimation is performed. The obstacle trajectory prediction method is based on a uniform kinematics model and uses Kalman filtering to estimate the obstacle motion.

[0004] This method has three drawbacks. First, querying valid parking spaces on a binary map is slow. Second, visual detection makes it difficult to accurately obtain two points at the non-entrance end of the parking space when it is obstructed. The IoU calculation of parking spaces obtained by visual and ultrasonic radar detections has errors, which reduces accuracy and makes threshold setting difficult. Third, the use of motion models for parking space tracking is inappropriate because parking lot terrain is complex and may have sharp turns, straight lines, etc., making it difficult to predict the trajectory of the parking space. Summary of the Invention

[0005] One objective of this invention is to provide a fusion calculation method for parking spaces to solve problems such as slow query speed for valid parking spaces and bottlenecks in non-entry point detection in the prior art.

[0006] Specifically, the present invention provides a method for fusion calculation of parking spaces, comprising the following steps:

[0007] Acquire image information of the target parking space, information of a first obstacle, and information of a second obstacle. The first obstacle is parked in the first parking space, and the second obstacle is parked in the second parking space. The target parking space is located between the first parking space and the second parking space.

[0008] The parking space information of the target parking space is calculated based on the image information, the first obstacle information, and the second obstacle information. The parking space information includes the entrance point information and non-entrance point information of the target parking space.

[0009] If the parking space information meets the parking needs of the target vehicle, then control the target vehicle to automatically park in the target parking space;

[0010] If the parking space information does not meet the parking requirements of the target vehicle, then the target vehicle is controlled to continue moving.

[0011] Furthermore, the step of calculating the parking space information of the target parking space based on the image information, the first obstacle information, and the second obstacle information, wherein the parking space information includes the entrance point information and non-entrance point information of the target parking space, further includes:

[0012] Image information of the target parking space is obtained by a surround-view fisheye camera, and information on the first obstacle and the second obstacle is obtained by an ultrasonic radar.

[0013] By fusing the first obstacle information, the second obstacle information, and the image information, the two entrance points and two non-entrance points of the target parking space in the image are calculated.

[0014] The Euclidean distance to the target parking space is calculated based on the two entrance points. If the Euclidean distance meets the parking requirements of the target vehicle, the target vehicle is controlled to automatically park in the target parking space.

[0015] Furthermore, the step of fusing the first obstacle information, the second obstacle information, and the image information to calculate the two entrance points and two non-entrance points of the target parking space in the image also includes:

[0016] If the two non-entry point locations cannot be completely obtained, then starting from the two entry points, the target parking space is completed by following the direction of the non-entry points and passing through the non-entry points, using the standard parking space length.

[0017] If both non-entry point locations can be obtained completely, no completion is required.

[0018] Furthermore, the step of supplementing the target parking space by taking the two entrance points as starting points, moving along the direction of the non-entry points and passing through the non-entry points, and using the standard parking space length, if the two non-entry point locations cannot be completely obtained, also includes:

[0019] If the line connecting the entrance point and the non-entry point has a slope, then the target parking space is completed.

[0020] Further, the step of calculating the parking space information of the target parking space based on the image information, the first obstacle information, and the second obstacle information, wherein the parking space information includes the entrance point information and non-entrance point information of the target parking space, further includes:

[0021] The first sub-information of the target parking space is calculated based on the image information;

[0022] The second sub-information of the target parking space is calculated based on the first obstacle information and the second obstacle information;

[0023] Match the first sub-information with the second sub-information;

[0024] If the first sub-information and the second sub-information meet the preset conditions, then the image information, the first obstacle information, and the second obstacle information are the parking space information of the same target parking space.

[0025] Furthermore, the step of matching the first sub-information and the second sub-information also includes the following steps:

[0026] If the first sub-information is greater than the second sub-information, then the second entry point information in the second sub-information and the first entry point information in the first sub-information are used as the fused entry point information.

[0027] The parking space information is recalculated and updated by taking the second non-entry point information in the second sub-information and the non-entry point information in the first sub-information as the fused non-entry point information.

[0028] Furthermore, it also includes the following steps:

[0029] If the first sub-information is less than the second sub-information, then the first entrance point information and the first non-entrance point information in the first sub-information are used as the parking space information.

[0030] Furthermore, it also includes the following steps:

[0031] If the first sub-information and the second sub-information partially overlap, then the minimum value between the first sub-information and the second sub-information shall be taken as the parking space information.

[0032] Furthermore, the step of taking the minimum value between the first sub-information and the second sub-information as the parking space information if the first sub-information and the second sub-information partially overlap further includes the following steps:

[0033] The second entry point information of the second sub-information with the minimum value on one side is merged with the first entry point information of the first sub-information, and the second non-entry point information of the second sub-information with the minimum value on one side is merged with the first non-entry point information of the first sub-information.

[0034] The third entry point information and the third non-entry point information on the other side are obtained through the surround-view fisheye camera;

[0035] Recalculate and update the parking space information.

[0036] The present invention also discloses a parking space fusion system, including a control device, the control device including a memory and a processor, the memory storing a control program, the control program being executed by the processor to implement the parking space fusion calculation method described above.

[0037] This invention acquires image information of a target parking space, information about a first obstacle, and information about a second obstacle. The first obstacle is positioned in a first parking space, and the second obstacle is positioned in a second parking space. The target parking space is located between the first and second parking spaces. Based on the image information, the first obstacle information, and the second obstacle information, parking space information is calculated. This parking space information includes entrance point information and non-entrance point information for the target parking space. If the parking space information meets the parking needs of the target vehicle, the vehicle is automatically controlled to park in the target parking space. If the parking space information does not meet the parking needs of the target vehicle, the vehicle continues to move. This solves the problems of slow query speed for valid parking spaces and bottlenecks in non-entrance point detection in existing technologies.

[0038] Furthermore, this invention improves robustness to occlusion and perception bottlenecks by calculating the Euclidean distance matching between two points at each parking space entrance, and also increases computational speed. By using a nonlinear model or not using a motion model for prediction at low speeds, both speed and accuracy can be improved.

[0039] The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments of the invention in conjunction with the accompanying drawings. Attached Figure Description

[0040] The following sections will describe some specific embodiments of the invention in detail by way of example and not limitation, with reference to the accompanying drawings. The same reference numerals in the drawings denote the same or similar parts or portions. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0041] Figure 1 This is a schematic diagram of a parking space according to an embodiment of the present invention;

[0042] Figure 2 This is a first process diagram according to an embodiment of the present invention;

[0043] Figure 3 This is a schematic diagram of the second process according to an embodiment of the present invention;

[0044] Figure 4 This is a schematic diagram of the third process according to an embodiment of the present invention. Detailed Implementation

[0045] Figure 1 This is a schematic diagram of a parking space according to an embodiment of the present invention. Figure 2 This is a first flowchart illustrating an embodiment of the present invention. In one embodiment, as shown... Figure 1 and Figure 2 As shown. The fusion calculation method for this parking space includes the following steps:

[0046] S1. Obtain image information of the target parking space, information of the first obstacle, and information of the second obstacle. The first obstacle is parked in the first parking space, the second obstacle is parked in the second parking space, and the target parking space is located between the first parking space and the second parking space.

[0047] S2. Calculate the parking space information of the target parking space based on the image information, the first obstacle information, and the second obstacle information. The parking space information includes the entrance point information and non-entry point information of the target parking space.

[0048] S3. If the parking space information meets the parking needs of the target vehicle, then control the target vehicle to automatically park in the target parking space.

[0049] S4. If the parking space information does not meet the parking needs of the target vehicle, then control the target vehicle to continue moving.

[0050] In this embodiment, image information of the target parking space, first obstacle information, and second obstacle information are acquired. Then, parking space information of the target parking space is calculated based on the image information, first obstacle information, and second obstacle information. The parking space information includes entrance point information and non-entrance point information of the target parking space. If the parking space information meets the parking needs of the target vehicle, the target vehicle is controlled to automatically park in the target parking space. This solves the problems of slow speed in querying valid parking spaces and bottlenecks in non-entrance point detection in the prior art.

[0051] Figure 3 This is a second process diagram according to an embodiment of the present invention. Figure 4 This is a schematic diagram of the third process according to an embodiment of the present invention. Figure 3 and Figure 4As shown, step S2, which calculates the parking space information of the target parking space based on image information, first obstacle information, and second obstacle information, including the entrance point information and non-entrance point information of the target parking space, further includes:

[0052] S20. Obtain image information of the target parking space through a surround-view fisheye camera, and obtain information on the first obstacle and the second obstacle through ultrasonic radar;

[0053] S21. The information of the first obstacle, the information of the second obstacle, and the image information are fused together to calculate the two entrance points and two non-entrance points of the target parking space in the image.

[0054] S22. If the two non-entry point locations cannot be completely obtained, then starting from the two entry points, follow the direction of the non-entry points and pass through the non-entry points to complete the target parking space using the standard parking space length.

[0055] S23. If two non-entry point locations can be completely obtained, no completion is required;

[0056] S24. Calculate the Euclidean distance to the target parking space based on the locations of the two entrance points. If the Euclidean distance meets the parking requirements of the target vehicle, control the target vehicle to automatically park in the target parking space.

[0057] Specifically, the main sensor is a surround-view fisheye camera, and the auxiliary sensor is ultrasonic radar. The surround-view fisheye camera generates an image after stitching and correction. Visual perception then identifies the four corner points of the parking space and whether it is occupied. However, the stitched image from the surround-view fisheye camera may not show a complete parking space; the length detected by visual perception might be much shorter than the standard parking space. In this case, by connecting the entrance point and its incomplete non-entry points, the slope of the line can be calculated. Starting from the entrance point, the line is drawn along the direction of the non-entry points, passing through them, and the parking space is completed using the standard parking space length. The ultrasonic radar acquires the location information of obstacles in the parking lot, using the coordinates of the obstacle boundary points.

[0058] In this embodiment, the coordinates of the parking space entrance point are obtained using ultrasonic radar. The parking space where this point is located is searched in the image information, and the length of the parking space and the slope between the entrance point and the non-entrance point are obtained. These are used as the parking space entrance slope of the ultrasonic radar. Taking the two entrance points detected by the ultrasonic radar as the starting point, the extension of the obtained parking space length along the slope can complete the parking space information obtained by the ultrasonic radar. If both the entrance point and the non-entrance point of the parking space can be obtained by the ultrasonic radar during the process of entering the parking space, no completion is required.

[0059] In one embodiment, such as Figure 3 and Figure 4As shown. Step S2: Calculate the parking space information of the target parking space based on the image information, the first obstacle information, and the second obstacle information. The parking space information includes the entrance point information and non-entrance point information of the target parking space, and also includes:

[0060] S2a. Calculate the first sub-information of the target parking space based on the image information;

[0061] S2b: Calculate the second sub-information of the target parking space based on the first obstacle information and the second obstacle information;

[0062] S2c, Match the first sub-information and the second sub-information;

[0063] S2d. If the first sub-information is greater than the second sub-information, then the second entry point information in the second sub-information and the first entry point information in the first sub-information are used as the fused entry point information, and the second non-entry point information in the second sub-information and the non-entry point information in the first sub-information are used as the fused non-entry point information. The parking space information is then recalculated and updated.

[0064] S2e. If the first sub-information is less than the second sub-information, then the first entrance point information and the first non-entrance point information in the first sub-information are used as parking space information.

[0065] S2f: If the first sub-information and the second sub-information partially overlap, the minimum value between the first sub-information and the second sub-information is used as the parking space information. The second entry point information of the second sub-information with the minimum value on one side is merged with the first entry point information of the first sub-information. The second non-entry point information of the second sub-information with the minimum value on one side is merged with the first non-entry point information of the first sub-information. The third entry point information and the third non-entry point information on the other side are obtained through the surround-view fisheye camera. The parking space information is then recalculated and updated.

[0066] S2g, when the first sub-information and the second sub-information meet the preset conditions, the image information and the first obstacle information and the second obstacle information are parking space information of the same target parking space.

[0067] Specifically, surround-view fisheye cameras are easier and more scalable for identifying parking spaces, and can handle various scenarios. For example, they can detect parking spaces in open areas as long as there are parking lines. Ultrasonic radar, on the other hand, is better at detecting obstacles. If a car is parked closer to the entrance or a vacant space between two occupied parking spaces, the risk of collision can be reduced. Therefore, surround-view fisheye cameras and ultrasonic radar parking spaces can be fused as follows: when the ultrasonic radar completes the parking space information and the fisheye camera's perception results are combined, the distance to the parking space at the entrance point can be calculated to match them as belonging to the same parking space.

[0068] When the width of the parking space perceived by the surround-view fisheye camera is wider, it is more likely that a vehicle in another parking space is encroaching on the target parking space. Therefore, the parking space information of ultrasonic radar is considered. The intersection of the line connecting the entrance point of the ultrasonic radar parking space and the entrance point of the parking space perceived by the surround-view fisheye camera is used as the fused entrance point. The intersection of the line connecting the non-entrance point of the ultrasonic radar parking space and the non-entrance point of the parking space perceived by the surround-view fisheye camera is used as the fused non-entrance point. The length and width of the line are then recalculated to update the fused parking space information.

[0069] When the parking spaces in the fisheye camera's perception results are narrower, it means that the parking space lines in the fisheye camera are narrower than the available parking space in the ultrasonic radar. In this case, the parking spaces in the fisheye camera's perception results are selected and used as the entry and non-entry points for fusion. Their length and width information are updated to the fused parking space length and width information.

[0070] When the parking space lines in the surround-view fisheye camera's perception result overlap with those in the ultrasonic radar's perception result, the innermost overlapping parking space line is considered as the parking space. One of the entrance points is obtained by extending the line connecting the entrance point of the ultrasonic radar parking space's inner parking space line and the entrance point of the parking space perceived by the fisheye camera. Similarly, one of the non-entrance points is obtained by extending the line connecting the non-entrance point of the ultrasonic radar parking space and the non-entrance point of the parking space perceived by the surround-view fisheye camera. Using one entrance point and one non-entrance point of the parking space perceived by the surround-view fisheye camera, the other entrance point and non-entrance point are updated, and their length and width are recalculated to update the merged parking space information.

[0071] In this embodiment, by calculating the Euclidean distance matching between the two points at each parking space entrance, its robustness to occlusion and perception bottlenecks is improved, and the calculation speed is also increased. By using a nonlinear model or not using a motion model for prediction at low speeds, both speed and accuracy can be improved.

[0072] The present invention also discloses a parking space fusion system, including a control device, the control device including a memory and a processor, the memory storing a control program, the control program being executed by the processor to implement the parking space fusion calculation method described above.

[0073] Therefore, those skilled in the art should recognize that although numerous exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications conforming to the principles of the present invention can be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the invention. Thus, the scope of the present invention should be understood and construed as covering all such other variations or modifications.

Claims

1. A method for merging and calculating parking spaces, characterized in that, Includes the following steps: Acquire image information of the target parking space, information of a first obstacle, and information of a second obstacle. The first obstacle is parked in the first parking space, and the second obstacle is parked in the second parking space. The target parking space is located between the first parking space and the second parking space. The parking space information of the target parking space is calculated based on the image information, the first obstacle information, and the second obstacle information. The parking space information includes the entrance point information and non-entrance point information of the target parking space. If the parking space information meets the parking needs of the target vehicle, then control the target vehicle to automatically park in the target parking space; If the parking space information does not meet the parking needs of the target vehicle, then control the target vehicle to continue moving; The step of calculating the parking space information of the target parking space based on the image information, the first obstacle information, and the second obstacle information, wherein the parking space information includes the entrance point information and non-entry point information of the target parking space, includes: The first sub-information of the target parking space is calculated based on the image information; The second sub-information of the target parking space is calculated based on the first obstacle information and the second obstacle information; Match the first sub-information with the second sub-information; If the first sub-information and the second sub-information meet the preset conditions, then the image information, the first obstacle information, and the second obstacle information are the parking space information of the same target parking space; The step of matching the first sub-information and the second sub-information includes: If the first sub-information is greater than the second sub-information, then the second entry point information in the second sub-information and the first entry point information in the first sub-information are used as the fused entry point information, and the second non-entry point information in the second sub-information and the first non-entry point information in the first sub-information are used as the fused non-entry point information, and the parking space information is recalculated and updated. If the first sub-information is less than the second sub-information, then the first entrance point information and the first non-entrance point information in the first sub-information are taken as the parking space information; If the first sub-information and the second sub-information partially overlap, then the minimum value between the first sub-information and the second sub-information shall be taken as the parking space information; The step of using the minimum value between the first sub-information and the second sub-information as the parking space information if the first sub-information and the second sub-information partially overlap includes: The second entry point information of the second sub-information with the minimum value on one side is merged with the first entry point information of the first sub-information, and the second non-entry point information of the second sub-information with the minimum value on one side is merged with the first non-entry point information of the first sub-information. Information on the third entry point and the third non-entry point on the other side is obtained through a panoramic fisheye camera; Recalculate and update the parking space information.

2. The fusion computing method according to claim 1, characterized in that, The step of calculating the parking space information of the target parking space based on the image information, the first obstacle information, and the second obstacle information, wherein the parking space information includes the entrance point information and non-entry point information of the target parking space, further includes: Image information of the target parking space is obtained by a surround-view fisheye camera, and information on the first obstacle and the second obstacle is obtained by an ultrasonic radar. By fusing the first obstacle information, the second obstacle information, and the image information, the two entrance points and two non-entrance points of the target parking space in the image are calculated. The Euclidean distance to the target parking space is calculated based on the two entrance points. If the Euclidean distance meets the parking requirements of the target vehicle, the target vehicle is controlled to automatically park in the target parking space.

3. The fusion computing method according to claim 2, characterized in that, The step of fusing the first obstacle information, the second obstacle information, and the image information to calculate the two entrance points and two non-entrance points of the target parking space in the image further includes: If the two non-entry point locations cannot be completely obtained, then starting from the two entry points, the target parking space is completed by following the direction of the non-entry points and passing through the non-entry points, using the standard parking space length. If both non-entry point locations can be obtained completely, no completion is required.

4. The fusion computing method according to claim 3, characterized in that, The step of supplementing the target parking space by taking the two entrance points as the starting point, moving along the direction of the non-entry points and passing through the non-entry points, and using the standard parking space length, if the two entrance points cannot be completely obtained, further includes: If the line connecting the entrance point and the non-entry point has a slope, then the target parking space is completed.

5. A parking space integration system, characterized in that, The system includes a control device, which comprises a memory and a processor. The memory stores a control program, which, when executed by the processor, is used to implement the fusion calculation method for parking spaces according to any one of claims 1-4.

Images