A method of determining a parking space corner point and a related device

Abstract

The application discloses a method for determining a parking space corner point and a related device, and the method comprises the following steps: when automatic parking, obtaining a vehicle surrounding parking space overhead view centered on the vehicle by looking around the vehicle; obtaining an empty parking space center point heat map, an occupied parking space center point heat map, a parking space front corner point heat map and a parking space rear corner point heat map according to the vehicle surrounding parking space overhead view and a preset detection algorithm; determining an empty parking space center point, an occupied parking space center point, a parking space front corner point and a parking space rear corner point according to the empty parking space center point heat map, the occupied parking space center point heat map, the parking space front corner point heat map, the parking space rear corner point heat map and a confidence of a pixel point; and determining a parking space corner point according to the empty parking space center point, the occupied parking space center point, the parking space front corner point, the parking space rear corner point and a preset parking space matching algorithm. The accuracy of determining the parking space corner point is improved, the accuracy of determining the parking space is improved, and the accuracy of the automatic parking technology is improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method and related apparatus for determining the corner point of a parking space. Background Technology

[0002] With the improvement of living standards, most users choose to travel by private car. The increase in vehicles has made parking a problem for users. To alleviate the parking burden, automatic parking technology has emerged. However, automatic parking technology needs to determine parking spaces. Parking spaces are generally described by the coordinates of the four corner points of the parking space. The magnitude of the positioning error of these corner points directly affects the accuracy of parking space determination, and thus the accuracy of automatic parking technology.

[0003] Currently, one method for determining parking space corner points is to use a Harris feature corner detector. However, this method is limited by the Harris feature settings and is subject to interference in practice, resulting in inaccurate determination of parking space corner points. Another method is to detect and fit parking space lines, and then determine the parking space corner points by finding the intersection of the parking space lines. However, this method is limited by the fitting accuracy of the parking space lines. If the parking space lines are not clear, the determination of parking space corner points will be inaccurate.

[0004] The inventors discovered through research that existing methods cannot accurately locate the corner points of parking spaces, meaning that the determination of parking space corner points is not accurate enough. This leads to inaccurate parking space determination, reduces the accuracy of automatic parking technology, and thus affects the user experience. Summary of the Invention

[0005] In view of this, embodiments of this application provide a method and related apparatus for determining the corner point of a parking space, thereby improving the accuracy of determining the corner point of a parking space, improving the accuracy of determining the parking space, enhancing the accuracy of automatic parking technology, and thus improving the user experience.

[0006] In a first aspect, embodiments of this application provide a method for determining the corner point of a parking space, the method comprising:

[0007] Obtain a top-down view of the parking spaces surrounding the vehicle, centered on the vehicle, during automatic parking;

[0008] Based on the top-down view of the parking spaces around the vehicle and a preset detection algorithm, heatmaps of the center points of empty parking spaces, occupied parking spaces, front corners, and rear corners of parking spaces are obtained. The preset detection algorithm is obtained by pre-training a deep learning model based on the top-down view samples of the parking spaces around the vehicle and the positions of the center points of empty parking spaces, occupied parking spaces, front corners, and rear corners of parking spaces in the top-down view samples of the parking spaces around the vehicle.

[0009] Based on the heat map of the center point of the empty parking space, the heat map of the center point of the occupied parking space, the heat map of the front corner point of the parking space, the heat map of the rear corner point of the parking space, and the confidence level of the pixels, the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, and the rear corner point of the parking space are determined.

[0010] The corner points of the parking spaces are determined based on the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, the rear corner point of the parking space, and a preset parking space matching algorithm.

[0011] Optionally, determining the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, the rear corner point of the parking space, and the confidence level of the pixels based on the heat map of the center point of the empty parking space, the heat map of the center point of the occupied parking space, the heat map of the front corner point of the parking space, the heat map of the rear corner point of the parking space, and the confidence level of the pixels includes:

[0012] The heat map of the center point of the empty parking space, the heat map of the center point of the occupied parking space, the heat map of the front corner of the parking space, and the heat map of the rear corner of the parking space are divided into multiple regions to obtain multiple heat map regions.

[0013] The pixels are filtered based on the confidence levels of the multiple heatmap regions and the pixels to determine the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, and the rear corner point of the parking space.

[0014] Optionally, the step of filtering the pixels based on the confidence levels of the multiple heatmap regions and the pixels to determine the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, and the rear corner point of the parking space includes:

[0015] Based on a preset confidence level, the pixel with the highest confidence level in each heatmap region is selected to obtain multiple first candidate pixels;

[0016] Based on the confidence levels of the first candidate pixel and its surrounding pixels, the first candidate pixel is filtered to obtain multiple second candidate pixels;

[0017] Gaussian surface fitting is performed based on the multiple second candidate pixels to determine the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, and the rear corner point of the parking space.

[0018] Optionally, the step of filtering the pixel with the highest confidence level in each heatmap region according to a preset confidence level to obtain multiple first candidate pixels includes:

[0019] Find the pixel with the highest confidence level in each of the heatmap regions to obtain multiple initial candidate pixels;

[0020] If the confidence level of the initial candidate pixel is less than the preset confidence level, the initial candidate pixel is deleted, and multiple first candidate pixels are obtained.

[0021] Optionally, the step of filtering the first candidate pixel based on the confidence levels of the first candidate pixel and surrounding pixels to obtain multiple second candidate pixels includes:

[0022] If the distance between any two first candidate pixels is less than a preset distance, delete the first candidate pixel with the lower confidence level.

[0023] If the confidence level of pixels in the preset neighboring region of the first candidate pixel is less than the preset confidence level, the first candidate pixel is deleted, and multiple second candidate pixels are obtained.

[0024] Optionally, determining the corner point of the parking space based on the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, the rear corner point of the parking space, and a preset parking space matching algorithm includes:

[0025] Take the center point of each empty parking space and the center point of each occupied parking space as the current parking space center point, and draw a circle with the current parking space center point as the center and a preset radius value as the radius to obtain the current circle;

[0026] By combining two front corner points and two rear corner points of the parking spaces within the current circle, multiple quadrilaterals are obtained.

[0027] If the target quadrilateral among the multiple quadrilaterals includes only the center point of the current parking space, and the area of ​​the target quadrilateral is the smallest, then the two front corner points and two rear corner points of the parking space corresponding to the target quadrilateral are determined as the parking space corner points.

[0028] Optionally, obtaining a top-down view of the parking spaces surrounding the vehicle, centered on the vehicle, during automatic parking includes:

[0029] During automatic parking, acquire four fisheye images of the vehicle from the front, rear, left, and right.

[0030] By stitching together the front, rear, left, and right fisheye images of the vehicle, a top-down view of the parking spaces surrounding the vehicle is obtained.

[0031] Secondly, embodiments of this application provide a device for determining the corner point of a parking space, the device comprising:

[0032] The first obtaining unit is used to obtain a top view of the parking spaces around the vehicle centered on the vehicle by looking around the parking spaces around the vehicle during automatic parking.

[0033] The second obtaining unit is used to obtain a heat map of the center point of an empty parking space, a heat map of the center point of an occupied parking space, a heat map of the front corner of a parking space, and a heat map of the rear corner of a parking space based on the top view of the parking spaces around the vehicle and a preset detection algorithm; the preset detection algorithm is obtained by pre-training a deep learning model based on the top view samples of the parking spaces around the vehicle and the positions of the center points of empty parking spaces, occupied parking spaces, front corners of parking spaces, and rear corners of parking spaces in the top view samples of the parking spaces around the vehicle.

[0034] The first determining unit is used to determine the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, the rear corner point of the parking space, and the confidence level of the pixels based on the heat map of the center point of the empty parking space, the heat map of the center point of the occupied parking space, the heat map of the front corner point of the parking space, the heat map of the rear corner point of the parking space, and the confidence level of the pixels.

[0035] The second determining unit is used to determine the corner point of the parking space based on the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, the rear corner point of the parking space and a preset parking space matching algorithm.

[0036] Optionally, the first determining unit includes:

[0037] The first obtaining subunit is used to divide the heat map of the center point of the empty parking space, the heat map of the center point of the occupied parking space, the heat map of the front corner point of the parking space, and the heat map of the rear corner point of the parking space into multiple regions to obtain multiple heat map regions.

[0038] The first determining subunit is used to filter the pixels based on the confidence level of the multiple heat map regions and the pixels, and to determine the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, and the rear corner point of the parking space.

[0039] Optionally, the first determining subunit includes:

[0040] The first obtaining module is used to filter the pixel with the highest confidence in each heat map region according to a preset confidence level, and obtain multiple first candidate pixel points;

[0041] The second obtaining module is used to filter the first candidate pixel based on the confidence level of the first candidate pixel and the surrounding pixels to obtain a plurality of second candidate pixels;

[0042] The determination module is used to perform Gaussian surface fitting based on the plurality of second candidate pixels to determine the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, and the rear corner point of the parking space.

[0043] Optionally, the first obtaining module includes:

[0044] The first submodule is used to find the pixel with the highest confidence in each heatmap region and obtain multiple initial candidate pixels.

[0045] The second obtaining submodule is used to delete the initial candidate pixel if the confidence level of the initial candidate pixel is less than the preset confidence level, and obtain a plurality of the first candidate pixel.

[0046] Optionally, the second obtaining module includes:

[0047] The deletion submodule is used to delete the first candidate pixel with lower confidence if the distance between any two first candidate pixels is less than a preset distance.

[0048] The third obtaining submodule is used to delete the first candidate pixel and obtain multiple second candidate pixels if the confidence level of the pixels in the preset adjacent area of ​​the first candidate pixel is less than the preset confidence level.

[0049] Optionally, the second determining unit includes:

[0050] The second obtaining subunit is used to take the center point of each empty parking space and the center point of each occupied parking space as the current parking space center point, and draw a circle with the current parking space center point as the center and a preset radius value as the radius to obtain the current circle.

[0051] The third sub-unit is used to combine two front corner points and two rear corner points of the parking space within the current circle to obtain multiple quadrilaterals.

[0052] The second determining subunit is used to determine the two front corner points and two rear corner points of the parking space corresponding to the target quadrilateral as the parking space corner points if the target quadrilateral among the multiple quadrilaterals only includes the center point of the current parking space and the area of ​​the target quadrilateral is the smallest.

[0053] Optionally, the first obtaining unit includes:

[0054] The acquisition subunit is used to acquire four fisheye images of the vehicle (front, rear, left, and right) during automatic parking.

[0055] The fourth sub-unit is used to stitch together the front, rear, left and right fisheye images of the vehicle to obtain a top view of the parking spaces around the vehicle.

[0056] Thirdly, embodiments of this application provide a terminal device, the terminal device including a processor and a memory:

[0057] The memory is used to store program code and transmit the program code to the processor;

[0058] The processor is configured to execute the method for determining the corner point of a parking space as described in any of the first aspects above, according to the instructions in the program code.

[0059] Fourthly, embodiments of this application provide a computer-readable storage medium for storing program code for executing the method for determining the corner point of a parking space as described in any of the first aspects above.

[0060] Compared with the prior art, this application has at least the following advantages:

[0061] Using the technical solution of this application embodiment, during automatic parking, a top-down view of the parking spaces surrounding the vehicle is obtained by looking around the parking spaces. Based on the top-down view of the parking spaces surrounding the vehicle and a preset detection algorithm, heatmaps of the center points of empty parking spaces, occupied parking spaces, front corners, and rear corners of parking spaces are obtained. The preset detection algorithm is based on the sample of the top-down view of the parking spaces surrounding the vehicle and the positions of the center points of empty and occupied parking spaces. The locations of the parking space points, including the front and rear corners, are obtained through a pre-trained deep learning model. The center points of the empty and occupied parking spaces, as well as the front and rear corners, are determined based on the heatmaps of the empty, occupied, and rear parking spaces, along with the confidence scores of the pixels. Finally, the corner points of the parking spaces are determined using the center points of the empty and occupied spaces, the front and rear corners, and a pre-defined parking space matching algorithm.

[0062] For a top-down view of parking spaces around the vehicle, a four-channel heatmap is obtained using a preset detection algorithm, representing the confidence levels of the center points of empty parking spaces, occupied parking spaces, the front corner of a parking space, and the rear corner of a parking space. Based on this four-channel heatmap, the center points of empty parking spaces, occupied parking spaces, the front corner of a parking space, and the rear corner of a parking space are accurately located. Furthermore, a preset parking space matching algorithm is used to accurately locate the corner points of parking spaces. This method significantly improves the accuracy of determining parking space corner points, thereby enhancing the accuracy of automatic parking technology and ultimately improving the user experience. Attached Figure Description

[0063] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This is a schematic diagram of the system framework involved in one application scenario in the embodiments of this application;

[0065] Figure 2 A flowchart illustrating a method for determining the corner point of a parking space, provided in an embodiment of this application;

[0066] Figure 3 A schematic diagram of a top view of parking spaces around a vehicle, a heat map of the center point of an empty parking space, a heat map of the center point of an occupied parking space, a heat map of the front corner of a parking space, and a heat map of the rear corner of a parking space, provided for embodiments of this application;

[0067] Figure 4 This is a schematic diagram of a device for determining the corner point of a parking space, provided in an embodiment of this application. Detailed Implementation

[0068] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0069] Currently, parking space corner points are detected using Harris feature corner detectors. However, this method is limited by the Harris feature settings and is susceptible to interference in practice, leading to inaccurate corner point determination. Alternatively, parking space lines can be detected and fitted to determine the corner points by finding their intersections. However, this method is limited by the fitting accuracy of the parking lines; if the lines are not clearly defined, the corner point determination will also be inaccurate. In short, none of these methods can accurately locate parking space corner points, resulting in inaccurate corner point determination. This reduces the accuracy of automatic parking technology and negatively impacts the user experience.

[0070] To address this issue, in this embodiment of the application, during automatic parking, a top-down view of the parking spaces surrounding the vehicle is obtained by scanning the surrounding parking spaces. Based on the top-down view and a preset detection algorithm, heatmaps of the center points of empty parking spaces, occupied parking spaces, the front corner of the parking space, and the rear corner of the parking space are obtained. The preset detection algorithm is based on the sample of the top-down view of the parking spaces surrounding the vehicle, including the center points of empty parking spaces and occupied parking spaces. The center point, front corner point, and rear corner point of the parking space are obtained through a pre-trained deep learning model. The center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, the rear corner point of the parking space, and the confidence level of the pixels are used to determine the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, and the rear corner point of the parking space. The corner points of the parking space are determined based on the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, the rear corner point of the parking space, and a preset parking space matching algorithm.

[0071] For a top-down view of parking spaces around the vehicle, a four-channel heatmap is obtained using a preset detection algorithm, representing the confidence levels of the center points of empty parking spaces, occupied parking spaces, the front corner of a parking space, and the rear corner of a parking space. Based on this four-channel heatmap, the center points of empty parking spaces, occupied parking spaces, the front corner of a parking space, and the rear corner of a parking space are accurately located. Furthermore, a preset parking space matching algorithm is used to accurately locate the corner points of parking spaces. This method significantly improves the accuracy of determining parking space corner points, thereby enhancing the accuracy of automatic parking technology and ultimately improving the user experience.

[0072] For example, one scenario in the embodiments of this application can be applied to, such as Figure 1 The scenario shown includes a vehicle surround view system 101 and a vehicle parking system 102. During automatic parking, the vehicle surround view system 101 surveys the surrounding parking spaces of the vehicle to obtain a surround view image, which is then sent to the vehicle parking system 102. Based on this image, the vehicle parking system 102 executes various implementation methods provided in this application to determine the corner point of the parking space, so that the vehicle parking system 102 can then automatically park the vehicle according to the corner point of the parking space.

[0073] First, in the above application scenarios, although the action description of the implementation method provided in this application is executed by the vehicle controller 102, the implementation method of this application is not limited in terms of the execution subject, as long as the actions disclosed in the implementation method provided in this application are executed.

[0074] Secondly, the above scenario is only one example provided by the embodiments of this application, and the embodiments of this application are not limited to this scenario.

[0075] The following, in conjunction with the accompanying drawings, describes in detail the specific implementation of the method and related devices for determining parking space corners in the embodiments of this application.

[0076] Exemplary methods

[0077] See Figure 2 This illustration shows a flowchart of a method for determining the corner point of a parking space according to an embodiment of this application. In this embodiment, the method may include, for example, the following steps:

[0078] Step 201: During automatic parking, obtain a top-down view of the parking spaces around the vehicle, centered on the vehicle.

[0079] In this embodiment of the application, determining the corner point of a parking space in automatic parking technology requires obtaining a top-down view of the parking spaces surrounding the vehicle, centered on the vehicle. To facilitate obtaining this top-down view, a vehicle surround-view system can be used to detect the parking spaces around the vehicle, obtaining four fisheye views (front, rear, left, and right) of the vehicle. Based on this, the four fisheye images can be stitched together to obtain a top-down view of the parking spaces surrounding the vehicle, centered on the vehicle. Therefore, in an optional implementation of this embodiment, step 201 may include, for example, steps A-B of the following steps:

[0080] Step A: Acquire four fisheye images of the vehicle (front, rear, left, and right) during automatic parking;

[0081] Step B: Stitch together the front, rear, left, and right fisheye images of the vehicle to obtain a top-down view of the parking spaces around the vehicle.

[0082] In the specific implementation of step B, the intrinsic and extrinsic parameters of the four fisheye cameras installed on the front, rear, left, and right sides of the vehicle can be obtained. By calculating the intrinsic and extrinsic parameters of the four fisheye cameras, the correspondence between objects in the real world and the pixels in the four fisheye images of the vehicle can be obtained. Based on this, the four fisheye images of the vehicle are stitched together, and the pixels in the four fisheye images of the vehicle are displayed as a top view of the parking spaces around the vehicle.

[0083] Step 202: Based on the top-view of the parking spaces around the vehicle and the preset detection algorithm, obtain the heat map of the center point of the empty parking space, the heat map of the center point of the occupied parking space, the heat map of the front corner of the parking space, and the heat map of the rear corner of the parking space; the preset detection algorithm is obtained by pre-training a deep learning model based on the top-view samples of the parking spaces around the vehicle and the positions of the center points of the empty parking spaces, the center points of the occupied parking spaces, the front corners of the parking spaces, and the rear corners of the parking spaces in the top-view samples of the parking spaces around the vehicle.

[0084] In this embodiment of the application, in order to solve the problem that existing methods cannot accurately locate the corner points of parking spaces, i.e., the determination of the corner points of parking spaces is not accurate enough, the following measures are taken: First, the historical top-view of parking spaces around the vehicle with known positions of the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, and the rear corner point of the parking space is used as training samples. That is, the deep learning model is iteratively trained using the top-view samples of parking spaces around the vehicle and the positions of the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, and the rear corner point of the parking space. The trained deep learning model is then used as the preset detection algorithm.

[0085] Then, the top-down view of the parking spaces around the vehicle obtained in step 201 is input into the preset detection algorithm, and a four-channel heat map is output, namely, the heat map of the center point of the empty parking space, the heat map of the center point of the occupied parking space, the heat map of the front corner of the parking space, and the heat map of the rear corner of the parking space.

[0086] In the heatmap of the center point of an empty parking space, the pixel value represents the confidence level that the pixel is the center point of an empty parking space; the higher the confidence level, the greater the probability that the pixel is the center point of an empty parking space. In the heatmap of the center point of an occupied parking space, the pixel value represents the confidence level that the pixel is the center point of an occupied parking space; the higher the confidence level, the greater the probability that the pixel is the center point of an occupied parking space. In the heatmap of the front corner of a parking space, the pixel value represents the confidence level that the pixel is the front corner of a parking space; the higher the confidence level, the greater the probability that the pixel is the front corner of a parking space. In the heatmap of the rear corner of a parking space, the pixel value represents the confidence level that the pixel is the rear corner of a parking space; the higher the confidence level, the greater the probability that the pixel is the rear corner of a parking space.

[0087] As an example, such as Figure 3 The diagram shows a top view of parking spaces around a vehicle, a heat map of the center point of an empty parking space, a heat map of the center point of an occupied parking space, a heat map of the front corner of a parking space, and a heat map of the rear corner of a parking space. The first image on the left is a top view of parking spaces around a vehicle, and the four images on the right are heat maps of the front corner of a parking space, the rear corner of a parking space, the center point of an empty parking space, and the center point of an occupied parking space, respectively.

[0088] Step 203: Determine the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, the rear corner point of the parking space, and the confidence level of the pixels based on the heat map of the empty parking space center point, the heat map of the occupied parking space center point, the heat map of the front corner point of the parking space, and the rear corner point of the parking space.

[0089] In this embodiment of the application, after obtaining the heat map of the center point of the empty parking space, the heat map of the center point of the occupied parking space, the heat map of the front corner point of the parking space, and the heat map of the rear corner point of the parking space in step 202, it is necessary to locate the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, and the rear corner point of the parking space based on the confidence level of the pixels in the above heat maps.

[0090] In the specific implementation of step 202, due to the characteristic that the confidence level of pixels in the heatmap decreases from the center point outwards, directly sorting the confidence levels on the heatmap to find multiple parking space corner points and parking space center points requires a huge amount of computation. Furthermore, the confidence level sorting causes the top n pixels with the highest confidence levels to cluster together, making it impossible to simultaneously locate multiple parking space corner points and parking space center points. To save significant computational power while simultaneously locating multiple parking space corner points and parking space center points, considering the non-adjacent nature of parking space corner points and parking space center points, a divide-and-conquer algorithm is applied to the four channels of heatmaps: the heatmap of the empty parking space center point, the heatmap of the occupied parking space center point, the heatmap of the front corner point of the parking space, and the heatmap of the rear corner point of the parking space. That is, each heatmap is first divided into several heatmap regions, and then pixels are filtered based on their confidence levels to locate multiple parking space corner points and parking space center points. Therefore, in an optional embodiment of this application, step 203 may include steps C-D of the following steps:

[0091] Step C: Divide the heat map of the center point of the empty parking space, the heat map of the center point of the occupied parking space, the heat map of the front corner of the parking space, and the heat map of the rear corner of the parking space into multiple regions to obtain multiple heat map regions.

[0092] For the front and rear corner heatmaps, the image size can be, for example, 416*416 pixels. The interval between the front corners of the same parking space is the distance from the parking space entrance line. The image size corresponding to the perpendicular parking space entrance line is greater than 60 pixels. Considering that the front corners of adjacent parking spaces are close together, that is, the two parking spaces are not collinear but are very close to each other, the heatmap is divided into 2704 heatmap areas with an 8*8 pixel size. During the division process, if the size is less than 8*8 pixels, it is calculated as one heatmap area. Theoretically, an 8*8 area will not contain two parking space corners.

[0093] For the heatmaps of empty parking space center points and occupied parking space center points, the image size can be, for example, 416*416 pixels. The interval between the center points of adjacent parking spaces is the distance from the parking space entrance line, corresponding to an image size greater than 60 pixels. The interval between the center points of non-adjacent parking spaces is also greater than the distance from the parking space entrance line. Therefore, the image is divided into 49 heatmap areas of 60*60 pixels each. Any area smaller than 60*60 pixels is counted as one heatmap area. Theoretically, a 60*60 area will not contain two parking space center points.

[0094] Step D: Filter the pixels based on the confidence levels of the multiple heatmap regions and the pixels to determine the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, and the rear corner point of the parking space.

[0095] In the specific implementation of step D, since the higher the confidence level of a pixel, the greater the probability that the pixel is a corner point or center point of a parking space, and a lower confidence level representing the pixel being a corner point or center point of a parking space is set as a preset confidence level, the pixels with the highest confidence level in each heatmap region need to be selected first using the preset confidence level. Pixels with the highest confidence level in each heatmap region that are greater than the preset confidence level are retained, resulting in multiple first candidate pixels. Then, based on the characteristic that the corner point, center point, and surrounding pixels of a parking space have high confidence levels, while other pixels have low confidence levels, multiple first candidate pixels are further selected to obtain multiple second candidate pixels. This is to avoid dividing the corner point, center point, and surrounding pixels of a parking space into multiple heatmap regions and to avoid the influence of image noise on the location of the corner point and center point of a parking space. Finally, considering that a heatmap area may have multiple second candidate pixels with the same confidence level, in order to further accurately locate the corner point or center point of the parking space, based on the fact that the heatmap conforms to the standard Gaussian surface distribution characteristics, it is also necessary to perform Gaussian surface fitting on the multiple second candidate pixels to determine the corner point or center point of the parking space. Therefore, in an optional embodiment of this application, step D may include, for example, steps D1-D3 of the following steps:

[0096] Step D1: Filter the pixel with the highest confidence in each heatmap region according to the preset confidence level to obtain multiple first candidate pixel points.

[0097] In the specific implementation of step D1, firstly, the pixel with the highest confidence level needs to be found in each heatmap region as an initial candidate pixel, thus obtaining multiple initial candidate pixels. Then, for each initial candidate pixel, if the initial candidate pixel is less than a preset confidence level, it means that the initial candidate pixel cannot be a parking space corner point or parking space center point, so the initial candidate pixel is deleted, thus obtaining multiple first candidate pixels. Therefore, in an optional embodiment of this application, step D1 may include, for example, steps D11-D12 of the following steps:

[0098] Step D11: Find the pixel with the highest confidence in each of the heatmap regions to obtain multiple initial candidate pixels;

[0099] Step D12: If the confidence level of the initial candidate pixel is less than the preset confidence level, delete the initial candidate pixel and obtain multiple first candidate pixels.

[0100] Step D2: Filter the first candidate pixel based on the confidence level of the first candidate pixel and the surrounding pixels to obtain multiple second candidate pixels.

[0101] In the specific implementation of step D1, firstly, based on the high confidence of the parking space corner point, parking space center point, and surrounding pixels, in order to avoid the parking space corner point or parking space center point and surrounding pixels being divided into multiple heat map regions, which would affect the location of the parking space corner point or parking space center point, it is necessary to merge multiple first candidate pixels. When the distance between any two first candidate pixels is less than a preset distance, it means that the parking space corner point or parking space center point and surrounding pixels are divided into multiple heat map regions. The first candidate pixel with higher confidence among the two first candidate pixels needs to be retained, and the first candidate pixel with lower confidence does not need to be retained and can be deleted.

[0102] Then, to avoid the influence of image noise on the location of parking space corners and center points, even when image noise makes the confidence of a single pixel very high, and the confidence of the parking space corners, center points, and surrounding pixels is also relatively high, the first candidate pixel is only retained when the confidence of all pixels in the preset neighboring area of ​​the first candidate pixel is greater than the preset confidence. When the confidence of all pixels in the preset neighboring area of ​​the first candidate pixel is less than the preset confidence, the first candidate pixel does not need to be retained and can be deleted.

[0103] Therefore, in an optional embodiment of this application, step D2 may include, for example, steps D21-D22 of the following steps:

[0104] Step D21: If the distance between any two first candidate pixels is less than a preset distance, delete the first candidate pixel with the lower confidence level.

[0105] Step D22: If the confidence level of the pixels in the preset neighboring region of the first candidate pixel is less than the preset confidence level, delete the first candidate pixel and obtain multiple second candidate pixels.

[0106] Step D3: Perform Gaussian surface fitting based on the multiple second candidate pixels to determine the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, and the rear corner point of the parking space.

[0107] Specifically, let (x0, y0) be the coordinates of the center point of the Gaussian surface equation, (σ x ,σ y If ) represents the standard deviation, then the equation of the Gaussian surface can be expressed as:

[0108]

[0109] Where K represents the peak value.

[0110] To find (x0, y0), take a logarithmic transformation on the Gaussian surface equation:

[0111]

[0112] A total of 49 data points were used for Gaussian surface fitting. The equation after logarithmic transformation was viewed as a matrix form A = BC, where:

[0113] A i =f i *ln f i ;

[0114]

[0115]

[0116] It can be seen that A is a 49*1 vector, B is a 49*5 matrix, and C is a 5*1 vector.

[0117] Assuming the error vector E = A - BC, the objective of the Gaussian surface equation is to minimize E, i.e., to minimize the MSE:

[0118]

[0119] To reduce computational complexity, B is decomposed into QR, i.e., B = QR, where Q is a 49*49 orthogonal matrix. T Q = I (where I is the identity matrix), and R is a 49*5 upper triangular matrix. The derivation is as follows:

[0120]

[0121] Among them, Q T Let A be a 49*1 vector, let Where S is a 5x1 vector, T is a 4x4 vector, and R1 is a 5x5 upper triangular matrix, then:

[0122]

[0123] When S = R1C, MSE reaches its minimum value. Therefore, we can conclude that:

[0124]

[0125]

[0126] Step 204: Determine the corner point of the parking space based on the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, the rear corner point of the parking space, and the preset parking space matching algorithm.

[0127] In this embodiment of the application, after determining the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, and the rear corner point of the parking space in step 203, it is also necessary to match the parking space using a preset parking space matching algorithm in order to accurately determine the corner point of the parking space.

[0128] In the specific implementation of step 204, firstly, a circle needs to be drawn with the center point of the parking space as the center and a preset radius value to obtain a current circle; then, the current circle may contain multiple parking space corner points and parking space center points, and four of the parking space corner points need to be combined to obtain multiple quadrilaterals; finally, when a quadrilateral formed by the four parking space corner points contains only the parking space center point and not other parking space corner points or parking space center points, and the area of ​​the quadrilateral is the smallest among the multiple quadrilaterals, it is considered that the four parking space corner points form a qualified parking space, and the four parking space corner points are parking space corner points. Therefore, in an optional embodiment of this application, step 204 may include, for example, steps E-G of the following steps:

[0129] Step E: Take the center point of each empty parking space and the center point of each occupied parking space as the current parking space center point, and draw a circle with the current parking space center point as the center and a preset radius value as the radius to obtain the current circle.

[0130] Step F: Combine two front corner points and two rear corner points of the parking spaces within the current circle to obtain multiple quadrilaterals.

[0131] Step G: If the target quadrilateral among the multiple quadrilaterals includes only the center point of the current parking space, and the area of ​​the target quadrilateral is the smallest, determine the two front corner points and two rear corner points of the parking space corresponding to the target quadrilateral as the parking space corner points.

[0132] Through the various implementation methods provided in this embodiment, during automatic parking, a top-down view of the parking spaces surrounding the vehicle is obtained by looking around the vehicle; based on the top-down view of the parking spaces surrounding the vehicle and a preset detection algorithm, heatmaps of the center points of empty parking spaces, occupied parking spaces, front corners, and rear corners of parking spaces are obtained; the preset detection algorithm is based on the sample of the top-down view of the parking spaces surrounding the vehicle and the positions of the center points of empty and occupied parking spaces. The center point, front corner point, and rear corner point of the parking space are obtained by a pre-trained deep learning model. The center point of the empty parking space, the center point of the occupied parking space, the front corner point, and the rear corner point of the parking space are determined based on the heat map of the center point of the empty parking space, the heat map of the center point of the occupied parking space, the heat map of the front corner point, and the rear corner point of the parking space, as well as the confidence of the pixels. The corner point of the parking space is determined based on the center point of the empty parking space, the center point of the occupied parking space, the front corner point, and the rear corner point of the parking space, and a preset parking space matching algorithm.

[0133] For a top-down view of parking spaces around the vehicle, a four-channel heatmap is obtained using a preset detection algorithm, representing the confidence levels of the center points of empty parking spaces, occupied parking spaces, the front corner of a parking space, and the rear corner of a parking space. Based on this four-channel heatmap, the center points of empty parking spaces, occupied parking spaces, the front corner of a parking space, and the rear corner of a parking space are accurately located. Furthermore, a preset parking space matching algorithm is used to accurately locate the corner points of parking spaces. This method significantly improves the accuracy of determining parking space corner points, thereby enhancing the accuracy of automatic parking technology and ultimately improving the user experience.

[0134] Exemplary device

[0135] See Figure 4 The diagram illustrates a structural schematic of a device for determining the corner point of a parking space according to an embodiment of this application. In this embodiment, the device may specifically include, for example:

[0136] The first obtaining unit 401 is used to obtain a top view of the parking spaces around the vehicle centered on the vehicle by looking around the parking spaces around the vehicle during automatic parking.

[0137] The second obtaining unit 402 is used to obtain a heat map of the center point of an empty parking space, a heat map of the center point of an occupied parking space, a heat map of the front corner of a parking space, and a heat map of the rear corner of a parking space based on the top view of the parking spaces around the vehicle and a preset detection algorithm; the preset detection algorithm is obtained by pre-training a deep learning model based on the top view samples of the parking spaces around the vehicle and the positions of the center points of empty parking spaces, occupied parking spaces, front corners of parking spaces, and rear corners of parking spaces in the top view samples of the parking spaces around the vehicle.

[0138] The first determining unit 403 is used to determine the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, the rear corner point of the parking space, and the confidence level of the pixels based on the heat map of the center point of the empty parking space, the heat map of the center point of the occupied parking space, the heat map of the front corner point of the parking space, the heat map of the rear corner point of the parking space, and the confidence level of the pixels.

[0139] The second determining unit 404 is used to determine the corner point of the parking space based on the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, the rear corner point of the parking space and a preset parking space matching algorithm.

[0140] In one optional embodiment of this application, the first determining unit 403 includes:

[0141] The first obtaining subunit is used to divide the heat map of the center point of the empty parking space, the heat map of the center point of the occupied parking space, the heat map of the front corner point of the parking space, and the heat map of the rear corner point of the parking space into multiple regions to obtain multiple heat map regions.

[0142] The first determining subunit is used to filter the pixels based on the confidence level of the multiple heat map regions and the pixels, and to determine the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, and the rear corner point of the parking space.

[0143] In an optional embodiment of this application, the first determining subunit includes:

[0144] The first obtaining module is used to filter the pixel with the highest confidence in each heat map region according to a preset confidence level, and obtain multiple first candidate pixel points;

[0145] The second obtaining module is used to filter the first candidate pixel based on the confidence level of the first candidate pixel and the surrounding pixels to obtain a plurality of second candidate pixels;

[0146] The determination module is used to perform Gaussian surface fitting based on the plurality of second candidate pixels to determine the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, and the rear corner point of the parking space.

[0147] In one optional embodiment of this application, the first obtaining module includes:

[0148] The first submodule is used to find the pixel with the highest confidence in each heatmap region and obtain multiple initial candidate pixels.

[0149] The second obtaining submodule is used to delete the initial candidate pixel if the confidence level of the initial candidate pixel is less than the preset confidence level, and obtain a plurality of the first candidate pixel.

[0150] In one optional embodiment of this application, the second obtaining module includes:

[0151] The deletion submodule is used to delete the first candidate pixel with lower confidence if the distance between any two first candidate pixels is less than a preset distance.

[0152] The third obtaining submodule is used to delete the first candidate pixel and obtain multiple second candidate pixels if the confidence level of the pixels in the preset adjacent area of ​​the first candidate pixel is less than the preset confidence level.

[0153] In one optional embodiment of this application, the second determining unit 404 includes:

[0154] The second obtaining subunit is used to take the center point of each empty parking space and the center point of each occupied parking space as the current parking space center point, and draw a circle with the current parking space center point as the center and a preset radius value as the radius to obtain the current circle.

[0155] The third sub-unit is used to combine two front corner points and two rear corner points of the parking space within the current circle to obtain multiple quadrilaterals.

[0156] The second determining subunit is used to determine the two front corner points and two rear corner points of the parking space corresponding to the target quadrilateral as the parking space corner points if the target quadrilateral among the multiple quadrilaterals only includes the center point of the current parking space and the area of ​​the target quadrilateral is the smallest.

[0157] In an optional embodiment of this application, the first obtaining unit 401 includes:

[0158] The acquisition subunit is used to acquire four fisheye images of the vehicle (front, rear, left, and right) during automatic parking.

[0159] The fourth sub-unit is used to stitch together the front, rear, left and right fisheye images of the vehicle to obtain a top view of the parking spaces around the vehicle.

[0160] Through the various implementation methods provided in this embodiment, during automatic parking, a top-down view of the parking spaces surrounding the vehicle is obtained by looking around the vehicle; based on the top-down view of the parking spaces surrounding the vehicle and a preset detection algorithm, heatmaps of the center points of empty parking spaces, occupied parking spaces, front corners, and rear corners of parking spaces are obtained; the preset detection algorithm is based on the sample of the top-down view of the parking spaces surrounding the vehicle and the positions of the center points of empty and occupied parking spaces. The center point, front corner point, and rear corner point of the parking space are obtained by a pre-trained deep learning model. The center point of the empty parking space, the center point of the occupied parking space, the front corner point, and the rear corner point of the parking space are determined based on the heat map of the center point of the empty parking space, the heat map of the center point of the occupied parking space, the heat map of the front corner point, and the rear corner point of the parking space, as well as the confidence of the pixels. The corner point of the parking space is determined based on the center point of the empty parking space, the center point of the occupied parking space, the front corner point, and the rear corner point of the parking space, and a preset parking space matching algorithm.

[0161] For a top-down view of parking spaces around the vehicle, a four-channel heatmap is obtained using a preset detection algorithm, representing the confidence levels of the center points of empty parking spaces, occupied parking spaces, the front corner of a parking space, and the rear corner of a parking space. Based on this four-channel heatmap, the center points of empty parking spaces, occupied parking spaces, the front corner of a parking space, and the rear corner of a parking space are accurately located. Furthermore, a preset parking space matching algorithm is used to accurately locate the corner points of parking spaces. This method significantly improves the accuracy of determining parking space corner points, thereby enhancing the accuracy of automatic parking technology and ultimately improving the user experience.

[0162] Furthermore, this application embodiment also provides a terminal device, the terminal device including a processor and a memory:

[0163] The memory is used to store program code and transmit the program code to the processor;

[0164] The processor is used to execute the method for determining the corner point of a parking space as described in the above method embodiment according to the instructions in the program code.

[0165] This application also provides a computer-readable storage medium for storing program code for executing the method for determining the corner point of a parking space as described in the above method embodiments.

[0166] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0167] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0168] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0169] The above description is merely a preferred embodiment of this application and is not intended to limit the application in any way. Although this application has disclosed preferred embodiments above, it is not intended to limit the application. Any person skilled in the art can make many possible variations and modifications to the technical solutions of this application using the methods and techniques disclosed above, or modify them into equivalent embodiments with equivalent changes, without departing from the scope of the technical solutions of this application. Therefore, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of this application without departing from the content of the technical solutions of this application shall still fall within the protection scope of the technical solutions of this application.

Claims

1. A method for determining the corner point of a parking space, characterized in that, include: Obtain a top-down view of the parking spaces surrounding the vehicle, centered on the vehicle, during automatic parking; Based on the top-down view of the parking spaces around the vehicle and a preset detection algorithm, heatmaps of the center points of empty parking spaces, occupied parking spaces, front corners, and rear corners of parking spaces are obtained. The preset detection algorithm is obtained by pre-training a deep learning model based on the top-down view samples of the parking spaces around the vehicle and the positions of the center points of empty parking spaces, occupied parking spaces, front corners, and rear corners of parking spaces in the top-down view samples of the parking spaces around the vehicle. The heat map of the center point of the empty parking space, the heat map of the center point of the occupied parking space, the heat map of the front corner of the parking space, and the heat map of the rear corner of the parking space are divided into multiple regions to obtain multiple heat map regions. Based on a preset confidence level, the pixel with the highest confidence level in each heatmap region is selected to obtain multiple first candidate pixels; Based on the confidence levels of the first candidate pixel and its surrounding pixels, the first candidate pixel is filtered to obtain multiple second candidate pixels; Gaussian surface fitting is performed based on the multiple second candidate pixels to determine the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, and the rear corner point of the parking space; The corner points of the parking spaces are determined based on the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, the rear corner point of the parking space, and a preset parking space matching algorithm.

2. The method according to claim 1, characterized in that, The step of filtering the pixel with the highest confidence level in each heatmap region according to a preset confidence level to obtain multiple first candidate pixel points includes: Find the pixel with the highest confidence level in each of the heatmap regions to obtain multiple initial candidate pixels; If the confidence level of the initial candidate pixel is less than the preset confidence level, the initial candidate pixel is deleted, and multiple first candidate pixels are obtained.

3. The method according to claim 1, characterized in that, The step of filtering the first candidate pixel based on the confidence level of the first candidate pixel and surrounding pixels to obtain multiple second candidate pixels includes: If the distance between any two first candidate pixels is less than a preset distance, delete the first candidate pixel with the lower confidence level. If the confidence level of pixels in the preset neighboring region of the first candidate pixel is less than the preset confidence level, the first candidate pixel is deleted, and multiple second candidate pixels are obtained.

4. The method according to claim 1, characterized in that, The step of determining the corner point of a parking space based on the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, the rear corner point of the parking space, and a preset parking space matching algorithm includes: Take the center point of each empty parking space and the center point of each occupied parking space as the current parking space center point, and draw a circle with the current parking space center point as the center and a preset radius value as the radius to obtain the current circle; By combining two front corner points and two rear corner points of the parking spaces within the current circle, multiple quadrilaterals are obtained. If the target quadrilateral among the multiple quadrilaterals includes only the center point of the current parking space, and the area of ​​the target quadrilateral is the smallest, then the two front corner points and two rear corner points of the parking space corresponding to the target quadrilateral are determined as the parking space corner points.

5. The method according to claim 1, characterized in that, The method of obtaining a top-down view of the parking spaces surrounding the vehicle, centered on the vehicle, during automatic parking includes: During automatic parking, acquire four fisheye images of the vehicle from the front, rear, left, and right. By stitching together the front, rear, left, and right fisheye images of the vehicle, a top-down view of the parking spaces surrounding the vehicle is obtained.

6. A device for determining the corner point of a parking space, characterized in that, include: The first obtaining unit is used to obtain a top view of the parking spaces around the vehicle centered on the vehicle by looking around the parking spaces around the vehicle during automatic parking. The second obtaining unit is used to obtain a heat map of the center point of an empty parking space, a heat map of the center point of an occupied parking space, a heat map of the front corner of a parking space, and a heat map of the rear corner of a parking space based on the top view of the parking spaces around the vehicle and a preset detection algorithm; the preset detection algorithm is obtained by pre-training a deep learning model based on the top view samples of the parking spaces around the vehicle and the positions of the center points of empty parking spaces, occupied parking spaces, front corners of parking spaces, and rear corners of parking spaces in the top view samples of the parking spaces around the vehicle. The first determining unit divides the heat map of the center point of the empty parking space, the heat map of the center point of the occupied parking space, the heat map of the front corner of the parking space, and the heat map of the rear corner of the parking space into multiple regions to obtain multiple heat map regions. Based on a preset confidence level, the pixel with the highest confidence level in each heatmap region is selected to obtain multiple first candidate pixels; Based on the confidence levels of the first candidate pixel and its surrounding pixels, the first candidate pixel is filtered to obtain multiple second candidate pixels; Gaussian surface fitting is performed based on the multiple second candidate pixels to determine the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, and the rear corner point of the parking space; The second determining unit is used to determine the corner point of the parking space based on the center point of the empty parking space, the center point of the occupied parking space, the front corner point of the parking space, the rear corner point of the parking space and a preset parking space matching algorithm.

7. A terminal device, characterized in that, The terminal device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the method for determining the corner point of a parking space as described in any one of claims 1-5 according to the instructions in the program code.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program code for performing the method for determining the corner point of a parking space as described in any one of claims 1-5.

Images