Parking space matching method, device, electronic device and computer-readable storage medium

Abstract

The present invention discloses a parking space matching method, device, electronic device and computer-readable storage medium. The method includes: obtaining the current coordinates and heading angle information of a vehicle in the world coordinate system at the same moment and a single-frame panoramic image of the vehicle; performing parking space entrance corner point detection on the single-frame panoramic image to obtain the comprehensive confidence of each parking space entrance corner point and the picture coordinates of each parking space entrance corner point on the single-frame panoramic image; wherein, the comprehensive confidence is obtained according to the confidence information in three aspects of corner point confidence, position confidence and distance confidence; according to the current coordinates and heading angle information of the vehicle and the comprehensive confidence of each parking space entrance corner point, mapping the picture coordinates of each parking space entrance corner point into the world coordinate system to obtain a world corner point map; and matching a parking space according to the coordinates of the parking space entrance corner points in the world corner point map. The solution of the present invention can improve the accuracy of parking space matching.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic parking, and particularly relates to a parking space matching method, device, electronic device and computer-readable storage medium. Background Art

[0002] Currently, the automatic parking function has been installed in many vehicles. When the driver drives the vehicle into the garage, during the driving process, the in-vehicle system stitches the images captured by four or more cameras around the vehicle into a panoramic view, and then sends it to the parking space matching algorithm to match the parking space. When the parking space matching algorithm of the vehicle detects a suitable parking space, it will be displayed to the user. After the user selects a certain parking space, the parking program will automatically park the vehicle into the designated parking space, and the vehicle does not need to be controlled by the user at all during the process of parking into the parking space.

[0003] An important algorithm used in the parking function is the parking space matching algorithm. There are many implementations of this algorithm. Some detect the parking space lines through the fisheye view or panoramic view using the segmentation algorithm and then combine them into a parking space. Some detect the corner points of the parking space entrance through the panoramic view and then combine the entrance corner points into a parking space. There are many techniques for detecting the entrance corner points. For example, first detect the entrance area, then extract the image of the entrance area to accurately locate the entrance corner points, and finally obtain the corner points of the parking space entrance, or directly regress the entrance corner points at one time. The detected corner points of the parking space entrance are combined in pairs to form a parking space.

[0004] The accuracy of the parking space matching algorithm depends on the stability of the image, that is, all regions of the image need to be clearly visible. However, since most of the cameras used on vehicles are fisheye cameras, the image quality in the area directly facing the camera is high, but at the edge of the camera, especially in the overlapping area of the views of two cameras, the quality of the panoramic view is relatively poor. The panoramic view generated by the panoramic stitching algorithm is generally blurred in the overlapping area of the views of two cameras, and there are even ghosting phenomena. This is very unfriendly to the parking space matching algorithm, which will cause the parking space matching algorithm to calculate incorrect coordinates or angles of the corner points of the parking space entrance, or even directly fail to detect the corner points of the parking space entrance, thus greatly affecting the accuracy of the parking space matching algorithm. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a parking space matching method, device, electronic device and computer-readable storage medium, which can reduce the decrease in the accuracy of parking space matching caused by unstable image quality in each region of a single-frame panoramic view.

[0006] According to the first aspect of the present invention, an embodiment of a parking space matching method is provided, including:

[0007] Obtain the current coordinates and heading angle information of the vehicle in the world coordinate system at the same moment and a single-frame panoramic view of the vehicle;

[0008] Perform corner detection on the single-frame panoramic image to obtain the comprehensive confidence of each parking space entrance corner point and the image coordinates of each parking space entrance corner point on the single-frame panoramic image; wherein, the comprehensive confidence is obtained based on the confidence information in three aspects: corner confidence, position confidence, and distance confidence of the parking space entrance corner point from the vehicle.

[0009] According to the current coordinates and heading angle information of the vehicle and the comprehensive confidence of each parking space entrance corner point, map the image coordinates of each parking space entrance corner point to the world coordinate system to obtain a world corner point map.

[0010] Match parking spaces based on the coordinates of the parking space entrance corner points in the world corner point map.

[0011] As an improvement to the above solution, the matching of parking spaces based on the coordinates of the parking space entrance corner points in the world corner point map includes:

[0012] Perform corner detection on each subsequent single-frame panoramic image to obtain the comprehensive confidence of each parking space entrance corner point of each single-frame panoramic image and the image coordinates of each parking space entrance corner point on the single-frame panoramic image.

[0013] Map the image coordinates of each parking space entrance corner point of each subsequent single-frame panoramic image obtained to the world coordinate system to obtain the coordinates of the parking space entrance corner points in the world coordinate system.

[0014] Update the world corner point map according to the comprehensive confidence of each parking space entrance corner point of multiple subsequent single-frame panoramic images and the coordinates of each parking space entrance corner point in the world coordinate system.

[0015] Match parking spaces based on the coordinates of the parking space entrance corner points in the updated world corner point map.

[0016] As an improvement to the above solution, the single-frame panoramic view of the vehicle is formed by stitching multiple surrounding environment images collected in real time by the vehicle's camera. Performing corner detection on the single-frame panoramic image to obtain the comprehensive confidence of each parking space entrance corner point includes:

[0017] Input the single-frame panoramic image into a parking space matching model to obtain the corner confidence of each parking space entrance corner point.

[0018] Obtain the position confidence of the parking space entrance corner point according to the stitching quality of the single-frame panoramic view and the coordinates of the parking space entrance corner point on the single-frame panoramic image.

[0019] Obtain the distance confidence of the parking space entrance corner point according to the distance between the parking space entrance corner point and the nearest camera on the vehicle.

[0020] Obtain the comprehensive confidence of the parking space entrance corner point based on the corner point confidence, the position confidence, and the distance confidence.

[0021] As an improvement to the above solution, the obtaining the comprehensive confidence of the parking space entrance corner point according to the corner point confidence, the position confidence, and the distance confidence includes:

[0022] Input the corner point confidence, the position confidence, and the distance confidence of each detected parking space entrance corner point into the following weighted formula to obtain the comprehensive confidence of each parking space entrance corner point:

[0023] TotalConf = αC corner + βC img + γC dist

[0024] where C corner is the corner point confidence, C img is the position confidence, C dist is the distance confidence, and α, β, and γ are weight coefficients.

[0025] As an improvement to the above solution, the updating the world corner point map according to the comprehensive confidence of each parking space entrance corner point of multiple subsequent single-frame omnidirectional view images and the coordinates of each parking space entrance corner point in the world coordinate system includes:

[0026] When a detected parking space entrance corner point has been detected in a previous frame, compare the newly calculated comprehensive confidence of the parking space entrance corner point with its current comprehensive confidence. If the newly calculated comprehensive confidence is greater than the current comprehensive confidence, update the coordinates of the parking space entrance corner point in the world coordinate system to the new coordinates;

[0027] When a certain parking space entrance corner point is detected for the first time, directly update the coordinates of the parking space entrance corner point detected for the first time in the world coordinate system to the world corner point map.

[0028] According to a second aspect of the present invention, there is provided an embodiment of a parking space matching device, including:

[0029] A vehicle information acquisition module, configured to acquire the current coordinates and heading angle information of the vehicle in the world coordinate system at the same moment and the single-frame omnidirectional view image of the vehicle;

[0030] An entrance corner point detection module for detecting the parking space entrance corner points of the single-frame panoramic view image, obtaining the comprehensive confidence of each parking space entrance corner point and the picture coordinates of each parking space entrance corner point on the single-frame panoramic view image; wherein, the comprehensive confidence is obtained according to the confidence information in three aspects: corner point confidence, position confidence, and distance confidence of the parking space entrance corner point from the vehicle.

[0031] A coordinate mapping module for mapping the picture coordinates of each parking space entrance corner point into the world coordinate system according to the current coordinates and heading angle information of the vehicle and the comprehensive confidence of each parking space entrance corner point to obtain a world corner point map.

[0032] A parking space matching module for matching parking spaces according to the coordinates of the parking space entrance corner points in the world corner point map.

[0033] As an improvement of the above solution, the parking space matching module includes a corner point map updating unit, and the corner point map updating unit is used for:

[0034] Detecting the parking space entrance corner points of each subsequent single-frame panoramic view image, obtaining the comprehensive confidence of each parking space entrance corner point of each single-frame panoramic view image and the picture coordinates of each parking space entrance corner point on the single-frame panoramic view image;

[0035] Mapping the picture coordinates of each parking space entrance corner point of each subsequent single-frame panoramic view image obtained onto the world coordinate system to obtain the coordinates of the parking space entrance corner points in the world coordinate system;

[0036] Updating the world corner point map according to the comprehensive confidence of each parking space entrance corner point of multiple subsequent single-frame panoramic view images and the coordinates of each parking space entrance corner point in the world coordinate system;

[0037] The parking space matching module is specifically used for matching parking spaces according to the coordinates of the parking space entrance corner points in the updated world corner point map.

[0038] As an improvement of the above solution, the vehicle information acquisition module includes a panoramic view stitching unit, and the panoramic view stitching unit is used for stitching multiple surrounding environment images collected by the vehicle camera in real time into a single-frame panoramic view image.

[0039] The entrance corner point detection module includes:

[0040] A corner point confidence unit for inputting the single-frame panoramic view image into a parking space matching model to obtain the corner point confidence of each parking space entrance corner point.

[0041] A position confidence unit, configured to obtain the position confidence of the parking space entrance corner point according to the stitching quality of the single-frame panoramic view and the coordinates of the parking space entrance corner point on the single-frame panoramic image;

[0042] A distance confidence unit, configured to obtain the distance confidence of the parking space entrance corner point according to the distance between the parking space entrance corner point and the nearest camera on the vehicle;

[0043] A comprehensive confidence calculation unit, configured to obtain the comprehensive confidence of the parking space entrance corner point according to the corner point confidence, the position confidence, and the distance confidence.

[0044] As an improvement to the above solution, the comprehensive confidence calculation unit is specifically configured to,

[0045] Input the corner point confidence, the position confidence, and the distance confidence of each detected parking space entrance corner point into the following weighted formula to obtain the comprehensive confidence of each parking space entrance corner point:

[0046] TotalConf = αC corner + βC img + γC dist

[0047] Where C corner is the corner point confidence, C img is the position confidence, C dist is the distance confidence, and α, β, and γ are weight coefficients.

[0048] As an improvement to the above solution, the corner point map update unit is specifically configured to,

[0049] In the case that a detected parking space entrance corner point has been detected in a previous frame, compare the newly calculated comprehensive confidence of the parking space entrance corner point with its current comprehensive confidence. If the newly calculated comprehensive confidence is greater than the current comprehensive confidence, update the coordinates of the parking space entrance corner point in the world coordinate system to the new coordinates;

[0050] In the case of first detecting a parking space entrance corner point, directly update the coordinates of the first-detected parking space entrance corner point in the world coordinate system to the world corner point map.

[0051] According to a third aspect of the present invention, an embodiment of an electronic device is provided, including: a memory and a processor,

[0052] A computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the steps of the foregoing parking space matching method.

[0053] According to the fourth aspect of the present invention, there is also provided an embodiment of a computer-readable storage medium, on which one or more computer programs are stored, and when the one or more computer programs are executed by a processor, the steps of the foregoing parking space matching method are implemented.

[0054] Implementing the embodiments of the present invention has the following beneficial effects:

[0055] In the embodiments of the present invention, by obtaining the current coordinates and heading angle information of the vehicle in the world coordinate system and the single-frame panoramic image of the vehicle at the same moment, and then detecting the corner points of the parking space entrance for the single-frame panoramic image, the comprehensive confidence of each corner point of the parking space entrance and the picture coordinates of each corner point of the parking space entrance on the single-frame panoramic image are obtained. Then, according to the current coordinates and heading angle information of the vehicle and the comprehensive confidence of each corner point of the parking space entrance, the picture coordinates of each corner point of the parking space entrance are mapped into the world coordinate system to obtain a world corner point map. After that, the parking space is matched according to the coordinates of the corner points of the parking space entrance in the world corner point map. Since the embodiments of the present invention perform coordinate matching of the corner points of the parking space entrance in the world corner point map, considering that the corner points of the parking space entrance are stable in the world coordinate system, and the world corner point map is obtained according to the comprehensive confidence, and the comprehensive confidence synthesizes the confidence information in three aspects: corner point confidence, position confidence, and distance confidence. Therefore, the comprehensiveness and authenticity of the coordinate information of the corner points of the parking space entrance in the world corner point map are ensured, and the defect of low parking space matching accuracy caused by unstable image quality in each area of the single-frame panoramic image can be overcome to a certain extent, and the accuracy of parking space matching is improved.

[0056] In a preferred embodiment, the world corner point map is updated by the comprehensive confidence of each corner point of the parking space entrance in subsequent multiple single-frame panoramic images and the coordinates of each corner point of the parking space entrance in the world coordinate system, and then the parking space is matched according to the coordinates of the corner points of the parking space entrance in the updated world corner point map. Since the corner points of the parking space entrance are stable in the world coordinate system, even if the same corner point of the parking space entrance is detected at different positions in different frames of panoramic images, it is still at the same position when mapped into the world coordinate system. Therefore, the preferred embodiment can fuse the information of the corner points of the parking space entrance in multiple frames and only retain the coordinate data of the corner points of the parking space entrance when the parking space is clear, which can greatly improve the accuracy of parking space matching. Description of the Drawings

[0057] By reading the following detailed description of the preferred embodiments, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention. And throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0058] Figure 1It is a schematic flowchart of an embodiment of a parking space matching method provided by the present invention;

[0059] Figure 2 It is a schematic diagram showing the high and low position confidence levels on a surround view image provided by the present invention;

[0060] Figure 3 It is a block diagram of an embodiment of a parking space matching device provided by the present invention;

[0061] Figure 4 It is a block diagram of another embodiment of a parking space matching device provided by the present invention;

[0062] Figure 5 It is a schematic structural diagram of an embodiment of an electronic device provided by the present invention. Detailed implementation manners

[0063] Hereinafter, exemplary embodiments of the present invention will be described in more detail with reference to the accompanying drawings. These embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein.

[0064] Figure 1 It is a schematic flowchart of an embodiment of a parking space matching method provided by the present invention. Refer to Figure 1 , and this method embodiment includes the following steps S110 to step S160:

[0065] Step S110, obtain the current coordinates and heading angle information of the vehicle in the world coordinate system at the same moment and a single-frame surround view image of the vehicle.

[0066] To implement the intelligent parking function, generally at least four cameras are installed around the vehicle, distributed on the left side, right side, rear side, and front side of the vehicle body. The multi-channel surrounding environment images include the left-side image of the vehicle body, the right-side image of the vehicle body, the rear-side image of the vehicle body, and the front-side image of the vehicle body, but are not limited thereto. The in-vehicle system splices the images of four or more cameras around the vehicle into a surround view, and then sends it to the parking space matching algorithm to match the parking space. The surround view image is also called a top view image. The in-vehicle system will fuse the camera images of these cameras into a single top view image, just as if looking down from directly above the vehicle.

[0067] Generally, the spliced surround view image on the vehicle will be displayed on the central control screen to facilitate the user to view the environment around the vehicle. The surround view image is very suitable for detecting parking spaces, and the coordinates in the surround view image can be easily converted into the world coordinate system, which is convenient for guiding the parking program to control the vehicle to park in the parking space.

[0068] Cameras on vehicles are basically fisheye cameras, because fisheye cameras have a relatively large field of view, and generally the horizontal and vertical FOV (Field of View) are more than 120°. The fisheye image output by the fisheye camera has a relatively high image quality in the center area of ​​the camera, but the edge image is severely distorted and of relatively low quality. After being stitched into a surround view image, the image quality of the intersection area of ​​the field of view of the two cameras is poor, which is very unfavorable to the parking space search model and will reduce the accuracy of the parking space search in the intersection area of ​​the field of view.

[0069] While acquiring the multi-channel surrounding environment images collected by the vehicle camera in real time, the vehicle's current coordinates and heading angle information in the world coordinate system reported in real time by the vehicle computing unit are also acquired synchronously. The world coordinate system refers to the plane coordinate system constructed with the ground as the plane and the vehicle's current location as the origin when the vehicle starts the parking function. This ensures that the stitched single-frame surround view image corresponds to the vehicle's current location.

[0070] Step S120, performing parking space entrance corner point detection on the single-frame surround view image to obtain a comprehensive confidence of each parking space entrance corner point and the image coordinates of each parking space entrance corner point on the single-frame surround view image; wherein the comprehensive confidence is obtained based on three aspects of confidence information: corner point confidence, position confidence, and distance confidence of the parking space entrance corner point from the vehicle.

[0071] The parking space entrance corner points refer to the points on both sides of the parking space entrance, and together with the other two points, a rectangular parking space can be formed. The embodiment of the present invention uses a pre-trained parking space matching model to directly detect the parking space entrance corner points in the surround view. The input of the parking space matching model is a single-frame surround view image, and the output of the parking space matching model is the image coordinates of each detected parking space entrance corner point on the single-frame surround view image and the comprehensive confidence of each parking space entrance corner point.

[0072] Confidence appears in various deep learning algorithms. In object detection algorithms, it is mainly used to characterize the reliability of object detection results. In key point algorithms, it is mainly used to characterize the reliability of key point detection results. The value range is generally 0-1. The larger the value, the more reliable the result.

[0073] In the embodiment of the method, during the parking space entrance corner point detection process for a single frame surround view image, the parking space matching model outputs not only the corner point confidence but also the position confidence and the distance confidence between the parking space entrance corner point and the vehicle for each detected parking space entrance corner point.

[0074] Considering that the single-frame surround view of the vehicle is a mosaic of multiple surrounding environment images collected by the vehicle camera in real time, the parking entrance corner point detection is performed on the single-frame surround view image to obtain the comprehensive confidence of each parking entrance corner point, which can be specifically:

[0075] Input a single-frame panoramic image into the parking space matching model to obtain the corner confidence of each corner point of the parking space entrance; according to the stitching quality of the single-frame panoramic view and the coordinates of the corner point of the parking space entrance on this single-frame panoramic image, obtain the position confidence of the corner point of the parking space entrance; according to the distance between the corner point of the parking space entrance and the nearest camera on the vehicle, obtain the distance confidence of the corner point of the parking space entrance; then obtain the comprehensive confidence of the corner point of the parking space entrance according to the corner confidence, position confidence and distance confidence.

[0076] The closer the object is to the camera, the more accurate the detected information such as coordinates and angles is. Therefore, it is necessary to add the distance factor between the corner point of the parking space entrance and the nearest camera on the vehicle as the distance confidence.

[0077] The calculation formula for the distance confidence can be, for example, C dist = 2(1 - sigmoid(dist)), where dist is the distance between the corner point of the parking space entrance on the panoramic image and the nearest camera to it. Among them, the Sigmoid function is a common S-shaped function in biology, also known as the S-shaped growth curve. In information science, due to its properties such as monotonic increase and monotonic increase of the inverse function, the Sigmoid function is often used as the threshold function of the neural network to map the variable to between 0 and 1.

[0078] At the positions at the same distance from the camera, the blur degree is different in different regions of the panoramic view. Therefore, it is also necessary to add the coordinate factor of the corner point of the parking space entrance on the single-frame panoramic image as the position confidence.

[0079] Since there are differences in the image quality of the stitched panoramic image in different regions. Generally speaking, the image quality in the area directly facing the camera is high, but the image quality in the overlapping area of the two camera fields of view is relatively poor. Therefore, the position confidence can be roughly calculated according to the coordinates of the corner point of the parking space entrance on the single-frame panoramic image.

[0080] Figure 2 is a schematic diagram showing the high and low position confidence on the panoramic image provided by the present invention. See Figure 2 , the first area 21 is the area directly facing the camera, and the pictures in these areas have higher brightness, representing higher position confidence. The second area 22 is the overlapping area of the two camera fields of view, and the pictures in these areas have lower brightness, representing lower position confidence.

[0081] In a specific embodiment, obtaining the comprehensive confidence of the corner point of the parking space entrance according to the corner confidence, position confidence and distance confidence includes:

[0082] The corner confidence, position confidence, and distance confidence of each detected corner point of the parking space entrance are input into the following weighted formula to obtain the comprehensive confidence of each corner point of the parking space entrance:

[0083] TotalConf = αC corner + βC img + γC dist

[0084] where C corner is the corner confidence, C img is the position confidence, C dist is the distance confidence, and α, β, and γ are weight coefficients.

[0085] The weighted formula for calculating the comprehensive confidence has been proven effective through numerous experiments. The factors to be considered for the values of the weight coefficients include the installation position of the camera, the stitching quality of the single-frame panoramic image, etc.

[0086] It can be seen that the comprehensive confidence comprehensively considers information such as the installation position of the camera and the stitching quality of the single-frame panoramic image. Compared with the solution that only uses the single-model confidence (the model only outputs the corner confidence), the parking space positioning is more accurate.

[0087] Step S130: According to the current coordinates and heading angle information of the vehicle and the comprehensive confidence of each corner point of the parking space entrance, map the picture coordinates of each corner point of the parking space entrance to the world coordinate system to obtain the world corner map.

[0088] This step S130 constructs the world corner map based on the output result of the parking space matching model and the current coordinates and heading angle information of the vehicle reported by the in-vehicle computing unit in the world coordinate system. Among them, the world corner map (global cornermap) refers to the plane graph formed after projecting the corner points of the parking space entrance detected in the single-frame panoramic image into the world coordinate system. Each corner point of the parking space entrance in the world corner map records its corresponding comprehensive confidence.

[0089] Specifically, this step S130 can first screen out the corner points of the parking space entrance whose comprehensive confidence is greater than a certain confidence threshold. This confidence threshold can be, for example, 0.5, but it is not limited to this. Then, according to the currently obtained vehicle coordinates and heading angle information, map the picture coordinates of these screened corner points of the parking space entrance on the single-frame panoramic image to the coordinates in the world coordinate system, that is, map from the graphic coordinate system to the world coordinate system to construct the world corner map, and the world corner map records the comprehensive confidence corresponding to each corner point of the parking space entrance. Of course, it is also possible to map all the detected corner points of the parking space entrance from the graphic coordinate system to the world coordinate system without using the confidence threshold to screen the corner points of the parking space entrance.

[0090] By constructing a world corner point map, multiple parking space entrance corner points detected in a single-frame panoramic image can be drawn on the same plane of the world corner point map, and then parking spaces can be matched pairwise according to the coordinates of the parking space entrance corner points on the world corner point map, which helps to quickly identify parking spaces.

[0091] Step S140: Match parking spaces according to the coordinates of the parking space entrance corner points in the world corner point map.

[0092] Specifically, in this step S140, the coordinates of all the parking space entrance corner points in the world corner point map are combined pairwise to match the points on both sides of the parking space entrance, and adding two other points can form a rectangular parking space.

[0093] It should be noted that the factors considered in the matching process are not limited to the coordinate values of the parking space entrance corner points in the world corner point map, but may also include the types of the entrance corner points, such as "T"-shaped parking space corners, "L"-shaped parking space corners, etc., and may also include factors such as the directions of the entrance corner points. Information such as the coordinates, types, and directions of the parking space entrance corner points can all be output by training a parking space matching model.

[0094] In the embodiment of this method, the coordinate matching of the parking space entrance corner points is carried out in the world corner point map. Considering that the parking space entrance corner points are stable in the world coordinate system, and this world corner point map is obtained based on the comprehensive confidence, and the comprehensive confidence synthesizes the confidence information in three aspects: corner point confidence, position confidence, and distance confidence. Therefore, the comprehensiveness and authenticity of the coordinate information of the parking space entrance corner points in the world corner point map are ensured, which can overcome to a certain extent the defect of low parking space matching accuracy caused by unstable image quality in each region of a single-frame panoramic image, and improve the accuracy of parking space matching.

[0095] Considering the fact that as the vehicle moves continuously during the process of searching for a parking space, the parking spaces in the camera view cross region will always appear in the high-quality region facing the camera as the vehicle moves. In a preferred embodiment, in the process of matching parking spaces according to the coordinates of the parking space entrance corner points in the world corner point map in the above step S140, it includes:

[0096] Detect the parking space entrance corner points for each subsequent single-frame panoramic image to obtain the comprehensive confidence of each parking space entrance corner point in each single-frame panoramic image and the image coordinates of each parking space entrance corner point on this single-frame panoramic image;

[0097] Map the image coordinates of each parking space entrance corner point in each subsequent single-frame panoramic image obtained to the world coordinate system to obtain the coordinates of the parking space entrance corner points in the world coordinate system;

[0098] Update the world corner point map according to the comprehensive confidence of each parking space entrance corner point in multiple subsequent single-frame surround-view images and the coordinates of each parking space entrance corner point in the world coordinate system;

[0099] Match the parking spaces according to the coordinates of the parking space entrance corner points in the updated world corner point map.

[0100] Since the parking space entrance corner points are stable in the world coordinate system, even if the parking space matching model scans the same parking space entrance corner point at different positions in surround-view images of different frames, it is still at the same position when mapped to the world coordinate system.

[0101] When fusing multiple frames, if the same parking space entrance corner point detected in the new frame of surround-view image has a higher confidence, update the information recorded for this parking space entrance corner point in the world corner point map; otherwise, discard it. Therefore, the step of "updating the world corner point map according to the comprehensive confidence of each parking space entrance corner point in multiple subsequent single-frame surround-view images and the coordinates of each parking space entrance corner point in the world coordinate system" can specifically be:

[0102] When it is detected that a certain parking space entrance corner point has been detected in a previous frame, compare the newly calculated comprehensive confidence of this parking space entrance corner point with its current comprehensive confidence. If the newly calculated comprehensive confidence is greater than the current comprehensive confidence, update the coordinates of this parking space entrance corner point in the world coordinate system to the new coordinates;

[0103] When a certain parking space entrance corner point is detected for the first time, directly update the coordinates of the parking space entrance corner point detected for the first time in the world coordinate system to the world corner point map.

[0104] That is to say, if a certain parking space entrance corner point detected in a subsequent single-frame surround-view image has been detected in a previous frame, compare the new comprehensive confidence of this parking space entrance corner point with its current comprehensive confidence. If the new comprehensive confidence is greater than the current comprehensive confidence, use the new coordinates of this parking space entrance corner point to update the current coordinates in the world coordinate system; otherwise, discard the new coordinates of this parking space entrance corner point. If a certain parking space entrance corner point detected in a subsequent single-frame surround-view image has not been detected in a previous frame, record the coordinates of this parking space entrance corner point in the world coordinate system.

[0105] In this way, by fusing multiple frames of surround-view images, record the coordinates of the parking space entrance corner points detected in each frame of surround-view image in the world corner point map, and only retain the coordinates of the parking space entrance corner points with high confidence for the same parking space, thereby continuously updating the world corner point map to make the world corner point map only retain the coordinates of the parking space entrance corner points when the image is clearest.

[0106] Since the vehicle is constantly moving, there is always a moment when the same parking space can clearly appear in the high-quality area of the camera. Therefore, in the world corner point map obtained by fusing multiple frames of panoramic images, the coordinates of the parking space entrance corner points recorded on it basically correspond to the high-quality areas of the panoramic images. Even if the parking space moves to the low-quality area as the vehicle moves forward, it can still be accurately identified with the help of the world corner point map, thus greatly improving the accuracy of parking space search.

[0107] Belonging to the same technical concept as the aforementioned parking space matching method, the present invention also provides an embodiment of a parking space matching device. Figure 3 It is a block diagram of an embodiment of a parking space matching device provided by the present invention. Refer to Figure 3 , the parking space matching device 300 of this embodiment includes:

[0108] A vehicle information acquisition module 310, configured to acquire the current coordinates and heading angle information of the vehicle in the world coordinate system at the same moment, as well as a single-frame panoramic image of the vehicle.

[0109] An entrance corner point detection module 320, configured to perform parking space entrance corner point detection on a single-frame panoramic image to obtain the comprehensive confidence of each parking space entrance corner point and the picture coordinates of each parking space entrance corner point on the single-frame panoramic image; wherein, the comprehensive confidence is obtained based on the confidence information in three aspects: corner point confidence, position confidence, and distance confidence of the parking space entrance corner point from the vehicle.

[0110] A coordinate mapping module 330, configured to map the picture coordinates of each parking space entrance corner point to the world coordinate system according to the current coordinates and heading angle information of the vehicle and the comprehensive confidence of each parking space entrance corner point to obtain a world corner point map.

[0111] A parking space matching module 340, configured to match a parking space according to the coordinates of the parking space entrance corner points in the world corner point map.

[0112] Wherein, the world coordinate system refers to a plane coordinate system constructed with the ground as the plane and the current location of the vehicle as the origin when the vehicle starts the parking function. By synchronously acquiring multiple surrounding environment images collected by the vehicle camera in real time, as well as the current coordinates and heading angle information of the vehicle in the world coordinate system reported by the in-vehicle computing unit in real time, it can be ensured that the single-frame panoramic image obtained by splicing corresponds to the current position of the vehicle.

[0113] Based on the above Figure 3 provided embodiment of the parking space matching device, the present invention also provides another embodiment of the parking space matching device. Figure 4 It is a block diagram of another embodiment of a parking space matching device provided by the present invention. Refer to Figure 4, the parking space matching device 400 of this embodiment includes: a vehicle information acquisition module 410, an entrance corner point detection module 420, a coordinate mapping module 430, and a parking space matching module 440. For the function descriptions of each module, please refer to Figure 3 the parking space matching device 300.

[0114] In a preferred embodiment, please refer to Figure 4 , the parking space matching module 440 includes a corner point map updating unit 441, and the corner point map updating unit 441 is used for:

[0115] Detect the entrance corner points of the parking space for each subsequent single-frame panoramic image, and obtain the comprehensive confidence of each entrance corner point of the parking space in each single-frame panoramic image and the picture coordinates of each entrance corner point of the parking space on the single-frame panoramic image;

[0116] Map the picture coordinates of each entrance corner point of the parking space in each subsequent single-frame panoramic image obtained to the world coordinate system to obtain the coordinates of the entrance corner point of the parking space in the world coordinate system;

[0117] Update the world corner point map according to the comprehensive confidence of each entrance corner point of the parking space in multiple subsequent single-frame panoramic images and the coordinates of each said entrance corner point of the parking space in the world coordinate system;

[0118] Correspondingly, the parking space matching module 440 is specifically used to match the parking space according to the coordinates of the entrance corner points of the parking space in the updated world corner point map.

[0119] In a preferred embodiment, still refer to Figure 4 , the vehicle information acquisition module 410 includes a panoramic view stitching unit 411, and the panoramic view stitching unit 411 is used to stitch multiple surrounding environment images collected by the vehicle camera in real time into a single-frame panoramic image.

[0120] The entrance corner point detection module 420 includes:

[0121] A corner point confidence unit 421, which is used to input the single-frame panoramic image into the parking space matching model to obtain the corner point confidence of each entrance corner point of the parking space;

[0122] A position confidence unit 422, which is used to obtain the position confidence of the entrance corner point of the parking space according to the stitching quality of the single-frame panoramic view and the coordinates of the entrance corner point of the parking space on the single-frame panoramic image;

[0123] A distance confidence unit 423, which is used to obtain the distance confidence of the entrance corner point of the parking space according to the distance between the entrance corner point of the parking space and the nearest camera on the vehicle;

[0124] A comprehensive confidence calculation unit 424, which is used to obtain the comprehensive confidence of the entrance corner point of the parking space according to the corner point confidence, the position confidence, and the distance confidence.

[0125] That is to say, the corner confidence corresponds to the model confidence output by the parking space matching model. The position confidence is calculated based on the stitching quality of the single-frame panoramic image and the coordinates of the parking space entrance corner on the single-frame panoramic image. The distance confidence is calculated based on the distance between the parking space entrance corner and the nearest on-vehicle camera on the vehicle.

[0126] In a preferred embodiment, the comprehensive confidence calculation unit 424 is specifically configured to

[0127] Input the corner confidence, position confidence, and distance confidence of each detected parking space entrance corner into the following weighted formula to obtain the comprehensive confidence of each parking space entrance corner:

[0128] TotalConf = αC corner + βC img + γC dist

[0129] Where C corner is the corner confidence, C img is the position confidence, C dist is the distance confidence, and α, β, and γ are weight coefficients. The factors to be considered for the values of each weight coefficient include the installation position of the camera, the stitching quality of the single-frame panoramic image, etc.

[0130] In a preferred embodiment, the corner map update unit 441 is specifically configured to:

[0131] When a detected parking space entrance corner has been detected in a previous frame, compare the newly calculated comprehensive confidence of the parking space entrance corner with its current comprehensive confidence. If the newly calculated comprehensive confidence is greater than the current comprehensive confidence, update the coordinates of the parking space entrance corner in the world coordinate system to the new coordinates;

[0132] When a parking space entrance corner is detected for the first time, directly update the coordinates of the parking space entrance corner detected for the first time in the world coordinate system to the world corner map.

[0133] For the implementation processes of each unit module in the two embodiments of the parking space matching device of the present invention, reference may specifically be made to the corresponding steps in the method embodiment, which will not be elaborated herein.

[0134] Belonging to the same technical concept as the foregoing parking space matching method, the present invention also provides an embodiment of an electronic device. Refer to Figure 5, an embodiment of the electronic device 500 provided by the present invention includes: a memory 510 and a processor 520. Among them, the memory 510 may be a memory, such as a high-speed random access memory (Random-Access Memory, RAM), or may be a non-volatile memory, such as at least one disk memory, etc. A computer program is stored in the memory 510, and this computer program is loaded and executed by the processor 520 to implement the steps of the foregoing embodiment of the parking space matching method.

[0135] At the hardware level, the electronic device 500 may also selectively include hardware such as a display panel 530, an interface module 540, and a communication module 550 required by the service. The memory 510, the processor 520, and the display panel 530, the interface module 540, the communication module 550, etc. may be interconnected through an internal bus, and this internal bus may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, Figure 5 only a two-way arrow is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0136] The present invention also proposes an embodiment of a computer-readable storage medium. This computer-readable storage medium stores one or more computer programs, and when the one or more computer programs are executed by a processor, the steps of the foregoing embodiment of the parking space matching method are implemented, and specifically used to execute:

[0137] Obtain the current coordinates and heading angle information of the vehicle in the world coordinate system at the same moment and the single-frame panoramic image of the vehicle;

[0138] Perform parking space entrance corner point detection on the single-frame panoramic image to obtain the comprehensive confidence of each parking space entrance corner point and the picture coordinates of each parking space entrance corner point on the single-frame panoramic image; among them, the comprehensive confidence is obtained according to the corner point confidence, the position confidence, and the distance confidence of the parking space entrance corner point from the vehicle in three aspects of confidence information;

[0139] According to the current coordinates and heading angle information of the vehicle and the comprehensive confidence of each parking space entrance corner point, map the picture coordinates of each parking space entrance corner point to the world coordinate system to obtain a world corner point map;

[0140] Match parking spaces according to the coordinates of the parking space entrance corner points in the world corner point map.

[0141] Those skilled in the art should understand that the embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer programs.

[0142] These computer programs can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured article including an instruction device, and the instruction device implements the operations in the process Figure 1 one process or multiple processes and / or blocks Figure 1 the functions specified in one block or multiple blocks.

[0143] These computer programs can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 the functions specified in one block or multiple blocks.

[0144] In a typical configuration, a computer device includes one or more processors (CPUs), an input / output interface, a network interface, and memory. The memory may include non-permanent memory in the computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable storage medium.

[0145] Computer-readable storage media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable storage media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0146] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0147] The above are only embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A parking space matching method, characterized in that, Including: Obtaining the current coordinates and heading angle information of the vehicle in the world coordinate system at the same moment, and the single-frame panoramic image of the vehicle; Performing parking space entrance corner point detection on the single-frame panoramic image to obtain the comprehensive confidence of each parking space entrance corner point and the image coordinates of each parking space entrance corner point on the single-frame panoramic image; wherein, the comprehensive confidence is obtained based on the confidence information in three aspects: corner point confidence, position confidence, and distance confidence of the parking space entrance corner point from the vehicle; According to the current coordinates and heading angle information of the vehicle and the comprehensive confidence of each parking space entrance corner point, mapping the image coordinates of each parking space entrance corner point to the world coordinate system to obtain a world corner point map; Matching parking spaces according to the coordinates of the parking space entrance corner points in the world corner point map.

2. The method according to claim 1, wherein The matching of parking spaces according to the coordinates of the parking space entrance corner points in the world corner point map includes: Performing parking space entrance corner point detection on each subsequent single-frame panoramic image to obtain the comprehensive confidence of each parking space entrance corner point of each single-frame panoramic image and the image coordinates of each parking space entrance corner point on the single-frame panoramic image; Mapping the image coordinates of each parking space entrance corner point of each subsequent single-frame panoramic image obtained to the world coordinate system to obtain the coordinates of the parking space entrance corner point in the world coordinate system; Updating the world corner point map according to the comprehensive confidence of each parking space entrance corner point of multiple subsequent single-frame panoramic images and the coordinates of each parking space entrance corner point in the world coordinate system; Matching parking spaces according to the coordinates of the parking space entrance corner points in the updated world corner point map.

3. The method according to claim 1 or 2, characterized in that, The single-frame panoramic view of the vehicle is formed by stitching multiple surrounding environment images collected in real time by vehicle cameras. Performing parking space entrance corner point detection on the single-frame panoramic image to obtain the comprehensive confidence of each parking space entrance corner point includes: Inputting the single-frame panoramic image into a parking space matching model to obtain the corner point confidence of each parking space entrance corner point; Obtaining the position confidence of the parking space entrance corner point according to the stitching quality of the single-frame panoramic view and the coordinates of the parking space entrance corner point on the single-frame panoramic image; Obtaining the distance confidence of the parking space entrance corner point according to the distance between the parking space entrance corner point and the nearest camera on the vehicle; Obtaining the comprehensive confidence of the parking space entrance corner point according to the corner point confidence, the position confidence, and the distance confidence.

4. The method according to claim 3, characterized in that, The obtaining of the comprehensive confidence of the parking space entrance corner point according to the corner point confidence, the position confidence, and the distance confidence includes: Inputting the corner point confidence, position confidence, and distance confidence of each detected parking space entrance corner point into the following weighted formula to obtain the comprehensive confidence of each parking space entrance corner point: TotalConf = αC corner + βC img + γC dist Among them, C corner is the corner confidence, C img is the position confidence, C dist is the distance confidence, and α, β, and γ are weight coefficients.

5. The method according to claim 2, wherein The updating of the world corner point map according to the comprehensive confidence of each parking space entrance corner point of multiple subsequent single-frame panoramic images and the coordinates of each parking space entrance corner point in the world coordinate system includes: When a detected corner point of a parking space entrance has been detected in a previous frame, compare the newly calculated comprehensive confidence of the corner point of the parking space entrance with its current comprehensive confidence. If the newly calculated comprehensive confidence is greater than the current comprehensive confidence, update the coordinates of the corner point of the parking space entrance in the world coordinate system to the new coordinates; When a corner point of a parking space entrance is detected for the first time, directly update the coordinates of the corner point of the parking space entrance detected for the first time in the world coordinate system to the world corner point map.

6. A parking space matching device, characterized in that, Including: A vehicle information acquisition module, configured to acquire the current coordinates and heading angle information of a vehicle in the world coordinate system at the same moment, as well as a single-frame panoramic image of the vehicle; An entrance corner point detection module, configured to detect corner points of a parking space entrance in the single-frame panoramic image, and obtain the comprehensive confidence of each corner point of the parking space entrance and the picture coordinates of each corner point of the parking space entrance on the single-frame panoramic image; wherein, the comprehensive confidence is obtained according to the confidence information in three aspects: corner point confidence, position confidence, and distance confidence of the corner point of the parking space entrance from the vehicle; A coordinate mapping module, configured to map the picture coordinates of each corner point of the parking space entrance to the world coordinate system according to the current coordinates and heading angle information of the vehicle and the comprehensive confidence of each corner point of the parking space entrance, to obtain a world corner point map; A parking space matching module, configured to match a parking space according to the coordinates of the corner points of the parking space entrance in the world corner point map.

7. The device according to claim 6, characterized in that, The parking space matching module includes a corner point map updating unit, and the corner point map updating unit is configured to: Detect corner points of a parking space entrance in each subsequent single-frame panoramic image, and obtain the comprehensive confidence of each corner point of the parking space entrance in each single-frame panoramic image and the picture coordinates of each corner point of the parking space entrance on the single-frame panoramic image; Map the picture coordinates of each corner point of the parking space entrance in each subsequent single-frame panoramic image obtained to the world coordinate system, to obtain the coordinates of the corner point of the parking space entrance in the world coordinate system; Update the world corner point map according to the comprehensive confidence of each corner point of the parking space entrance in multiple subsequent single-frame panoramic images and the coordinates of each corner point of the parking space entrance in the world coordinate system; The parking space matching module is specifically configured to match a parking space according to the coordinates of the corner points of the parking space entrance in the updated world corner point map.

8. The device according to claim 6, characterized in that, The vehicle information acquisition module includes a panoramic view stitching unit, and the panoramic view stitching unit is configured to stitch multiple surrounding environment images collected by a vehicle camera in real time into a single-frame panoramic image; The entrance corner point detection module includes: A corner point confidence unit, configured to input the single-frame panoramic image into a parking space matching model to obtain the corner point confidence of each corner point of the parking space entrance; A position confidence unit, configured to obtain the position confidence of the corner point of the parking space entrance according to the stitching quality of the single-frame panoramic view and the coordinates of the corner point of the parking space entrance on the single-frame panoramic image; A distance confidence unit, configured to obtain the distance confidence of the corner point of the parking space entrance according to the distance between the corner point of the parking space entrance and the nearest camera on the vehicle; A comprehensive confidence calculation unit for obtaining the comprehensive confidence of the parking space entrance corner point according to the corner point confidence, the position confidence, and the distance confidence.

9. An electronic device, comprising a memory and a processor, wherein a computer program is stored in the memory and is loaded and executed by the processor to implement the steps of the method according to any one of claims 1 to 5.

10. A computer-readable storage medium having one or more computer programs stored thereon, characterized in that, When the one or more computer programs are executed by the processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images