Mechanical parking space recognition method and device and automatic parking system

Abstract

This application discloses a method, device, and automatic parking system for mechanical parking space recognition, belonging to the field of intelligent driving technology. The method includes: during the vehicle's parking space search phase, processing a first bird's-eye view image corresponding to a first image captured by an onboard fisheye lens using a parking search model to obtain first parking space information for at least one mechanical parking space, the first parking space information including initial position information; determining a target parking space to be parked in from the at least one mechanical parking space based on the first parking space information; after the vehicle enters the parking phase targeting the target parking space, processing a second bird's-eye view image corresponding to a second image captured by the onboard fisheye lens using a parking model to obtain second parking space information for the target parking space, the second parking space information including entrance corner coordinate information and a mechanical parking space line segmentation mask; correcting the initial position information of the target parking space based on the second parking space information to obtain the target position information of the target parking space. This application can improve the accuracy of mechanical parking space corner points.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, specifically to mechanical parking space recognition methods, devices, and automatic parking systems. Background Technology

[0002] In real-world parking scenarios, parking spaces come in various types, with mechanical parking spaces and conventional parking spaces being the most common. Conventional parking spaces offer relatively more space and are easier to park in. Typical automated parking systems can easily identify and park in conventional spaces using conventional sensor data acquisition and processing algorithms. However, mechanical parking spaces present entirely different characteristics. They are usually compact and narrow, with structures incorporating lifting and sliding mechanisms. This results in dynamically changing boundaries and available space, significantly increasing the complexity of the parking environment. Furthermore, mechanical parking spaces have extremely strict requirements regarding the vehicle's parking position and angle, demanding that automated parking systems acquire high-precision corner coordinates. However, during the vehicle search phase for mechanical parking spaces, the obtained corner coordinates are often inaccurate due to the considerable distance, and this coordinate information is not corrected after being transmitted to the planning module, potentially leading to errors. Summary of the Invention

[0003] This application provides a mechanical parking space recognition method, device, and automatic parking system, aiming to solve the problem of insufficient accuracy of corner coordinates of mechanical parking spaces.

[0004] Firstly, a mechanical parking space recognition method is provided, including: During the vehicle's parking search phase, the parking search model is used to process the first bird's-eye image corresponding to the first image captured by the vehicle's fisheye lens to obtain the first parking space information of at least one mechanical parking space. The first parking space information includes the initial position information. Based on the first parking space information, determine the target parking space to be parked in from at least one of the aforementioned mechanical parking spaces; After the vehicle enters the parking phase of the target parking space, the second bird's-eye image corresponding to the second image captured by the on-board fisheye lens is processed using the parking model to obtain the second parking space information of the target parking space. The second parking space information includes the entrance corner point coordinate information and the mechanical parking space line segmentation mask. The initial position information of the target parking space is corrected based on the information of the second parking space to obtain the target position information of the target parking space.

[0005] In some embodiments, before processing the first bird's-eye image corresponding to the first image acquired by the vehicle-mounted fisheye lens using a library search model, the method further includes: The first image is converted into a first bird's-eye view image using an inverse perspective transformation projection algorithm; Before processing the second bird's-eye image corresponding to the second image captured by the vehicle-mounted fisheye lens using the parking model, the process also includes: The second image is converted into a second bird's-eye view image using an inverse perspective transformation projection algorithm.

[0006] In some embodiments, the first parking space information further includes at least one of parking space type, mechanical parking space line segmentation mask, and slope line segmentation mask.

[0007] In some embodiments, correcting the initial location information of the target parking space based on the second parking space information includes: Based on the mechanical parking line segmentation mask in the second parking space information, determine the center point coordinates of the mechanical parking line of the target parking space; Based on the above entrance corner coordinates and center point coordinates, the initial position information of the target parking space is corrected.

[0008] In some embodiments, the initial location information includes the coordinate information of four corner points, and the aforementioned entrance corner point coordinate information includes the coordinate information of the two entrance corner points of the target parking space; Based on the above entrance corner coordinates and center point coordinates, the initial position information of the target parking space is corrected, including: Based on the coordinate information of the two entrance corner points and the coordinates of the center point, determine the coordinate information of the other corner points of the target parking space besides the two entrance corner points. Use the coordinate information of the two entrance corner points and the other corner points mentioned above to replace the coordinate information of the corresponding corner points in the initial position information of the target parking space.

[0009] In some embodiments, the second parking space information further includes slope information, which includes slope angle coordinate information and / or slope line segmentation mask; Based on the coordinates of the two entrance corner points and the center point, determine the coordinates of the other corner points of the target parking space besides the two entrance corner points, including: Based on the coordinates of the two entrance corner points, the coordinates of the center point, and the slope information, the coordinates of the other corner points are determined.

[0010] In some embodiments, the library search model includes a parking space detection module and a parking space segmentation module. The parking space detection module is used for parking space corner point regression and parking space type classification, and the parking space segmentation module is used for mechanical parking space line segmentation and slope line segmentation. The parking model includes a corner detection module and a mechanical segmentation module. The corner detection module is used for entrance corner regression, slope corner regression, and corner type classification, while the mechanical segmentation module is used for mechanical parking space line segmentation and slope line segmentation.

[0011] In some embodiments, parking space corner regression, entrance corner regression, and slope corner regression are all achieved by regressing the offset of the pixel anchor point within the parking space to the corresponding corner point.

[0012] Secondly, a mechanical parking space recognition device is provided, comprising: The parking space search processing module is configured to process the first bird's-eye view image corresponding to the first image captured by the onboard fisheye lens during the parking space search phase of the vehicle using the parking space search model, so as to obtain the first parking space information of at least one mechanical parking space, the first parking space information including initial position information. The determination module is configured to determine the target parking space to be parked in from at least one mechanical parking space based on the first parking space information. The parking processing module is configured to process the second bird's-eye image corresponding to the second image captured by the onboard fisheye lens using the parking model after the vehicle enters the parking stage for the target parking space, and obtain the second parking space information of the target parking space. The second parking space information includes the entrance corner point coordinate information and the mechanical parking space line segmentation mask. The correction module is configured to correct the initial position information of the target parking space based on the second parking space information, so as to obtain the target position information of the target parking space.

[0013] Thirdly, an automatic parking system is provided, including a mechanical parking space recognition device as described in the second aspect.

[0014] Beneficial effects: The solution provided in this application, during the vehicle's parking search phase, processes the first bird's-eye view image corresponding to the first image captured by the vehicle-mounted fisheye lens using a parking search model to obtain first parking space information for at least one mechanical parking space. This first parking space information includes initial position information. Then, based on the first parking space information, a target parking space to be parked in is determined from among these at least one mechanical parking space. Next, after the vehicle enters the parking phase targeting the target parking space, the second bird's-eye view image corresponding to the second image captured by the vehicle-mounted fisheye lens using a parking model to obtain second parking space information for the target parking space. This second parking space information includes entrance corner coordinate information and a mechanical parking space line segmentation mask. Then, based on the second parking space information, the initial position information of the target parking space is corrected to obtain the target position information of the target parking space. Thus, two large-view models with different resolutions can be used to distinguish between the vehicle's parking search phase and the parking phase. After finding the position of the mechanical parking space, the detection corner points of the mechanical parking space are dynamically adjusted, improving the accuracy of the mechanical parking space corner points. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of the library search model in the embodiments of this application; Figure 2 This is a schematic diagram of the docking model in the embodiments of this application; Figure 3 This is an exemplary schematic diagram of a mechanical parking space from a Bev's perspective, provided in an embodiment of this application; Figure 4 This is a flowchart illustrating the mechanical parking space recognition method provided in the embodiments of this application; Figure 5 This is a schematic diagram of the mechanical parking space recognition device provided in the embodiments of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] In the description of this application, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0019] "A and / or B" includes the following three combinations: A only, B only, and a combination of A and B.

[0020] The use of "applies to" or "configured to" in this application implies open and inclusive language, which does not exclude the applicability to or configuration to devices performing additional tasks or steps. Additionally, the use of "based on" implies openness and inclusivity, because processes, steps, calculations, or other actions "based on" one or more conditions or values ​​may in practice be based on additional conditions or values ​​beyond those stated.

[0021] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0022] With the major trend of intelligent transformation in the automotive industry, automatic parking technology, as a key technology to improve driving convenience and safety, is receiving increasing attention. Automatic parking technology, also known as Automated Parking Assistant (APA) or Automated Parking System (APS), relies on onboard sensors and intelligent control systems to allow vehicles to autonomously complete parking operations without significant driver intervention, bringing drivers a completely new parking experience.

[0023] With the rapid development of technology and the continuous advancement of urbanization, the number of cars in cities has increased dramatically. This change has made parking problems increasingly severe, becoming a major factor restricting the efficient operation of urban traffic. Traditional manual parking methods have gradually revealed many shortcomings in dealing with the massive parking demand. Manual parking is not only inefficient and prone to causing traffic congestion during peak hours, but also, due to the scarcity of urban parking resources and the limited space in ordinary parking spaces, drivers may cause scratches, collisions, and other safety accidents if they are not careful while parking, resulting in losses to themselves and others.

[0024] The emergence of automatic parking technology offers an innovative solution to the parking problem. This technology accurately senses the surrounding environment and intelligently controls vehicle movements, effectively reducing the probability of accidents caused by driver error and significantly improving driving comfort and safety. At the same time, automatic parking technology improves parking accuracy and reliability, allowing vehicles to be parked more precisely and optimizing parking space utilization.

[0025] In real-world parking scenarios, parking spaces come in a variety of types, with mechanical parking spaces and conventional parking spaces being the most common. Conventional parking spaces offer relatively more space and are easier to park in. Typical automated parking systems can easily identify and park in conventional spaces using conventional sensor data acquisition and processing algorithms. However, mechanical parking spaces present entirely different characteristics. They are usually compact and narrow, with structures incorporating lifting and sliding mechanisms. This results in dynamically changing boundaries and available space, significantly increasing the complexity of the parking environment. Furthermore, mechanical parking spaces have extremely strict requirements on the vehicle's parking position and angle. If the parking deviation exceeds a certain range, the vehicle may be unable to enter or exit the space, or even the mechanical components may be damaged. Therefore, accurately and efficiently identifying and detecting mechanical parking spaces in automated parking systems is a key challenge in ensuring the reliable operation of automated parking functions in mechanical parking space scenarios.

[0026] Currently, the process of determining a parking space in an automatic parking system includes: using visual detection methods such as deep learning to detect and classify parking spaces; determining whether parking is possible and generating a planned route based on information from multiple sensors; and executing control to allow the vehicle to park autonomously.

[0027] Commonly used visual algorithms for parking space detection include Algorithm 1, Algorithm 2, and Algorithm 3, as shown below: Algorithm 1: Based on the YOLOv2 network, a rectangular box is detected for the corner point on the entry line of each parking space; the area of ​​the entry line is cropped and fed into the CNN segmentation network to output the parking space angle type; based on the prior knowledge of the parking space, the corner points of the other two parking spaces are inferred.

[0028] Algorithm 2: Based on a CNN network, output the type and coordinates of the parking space corner points; based on the shape and direction of the corner points output by the network, obtain the parking space type through matching; based on the prior knowledge of the parking space, infer the other two parking space corner points.

[0029] Algorithm 3: Based on the first stage output of the MobileNetV2 network, the parking space type and rotation angle are obtained; the final rotated Anchor box is obtained based on the image and type angle, and the parking space coordinates are obtained.

[0030] The relevant mechanical parking space recognition technologies include Technology 1, Technology 2, and Technology 3, as shown below: Technology 1: Based on the target detection model, detect the edge points and edge lines of the mechanical parking space; based on the segmentation model, detect the passable area space of the mechanical parking space and the protruding edges at the boundary of the mechanical parking space; based on the depth prediction model, predict the pixel depth value of the monocular camera; based on the ultrasonic model, detect obstacles around the waiting vehicle; the fusion module outputs the mechanical parking space and its location coordinates based on the model fusion perception results.

[0031] Technique 2: Utilizing SLAM technology for parking space location throughout the entire process of finding and parking; during parking space detection and measurement, a fisheye camera is used in the finding stage, and a binocular camera is used in the parking stage.

[0032] Technique 3: Detecting the inner edge of a parking space in an image; the parking space image is obtained by capturing the mechanical parking space with a monocular camera. The direction of the inner edge of the parking space is from the entrance to the end of the parking space. The inner edge of the parking space includes: a first inner edge and a second inner edge; a first plane is constructed based on the first inner edge and the optical center position of the monocular camera; a second plane is constructed based on the second inner edge and the optical center position; the parking direction of the mechanical parking space is determined according to the straight line intersecting the first and second planes. The intersecting straight line can accurately reflect the actual direction and angle of the mechanical parking space in three-dimensional space, avoiding the limitation of the ground plane assumption, thus accurately detecting the parking direction of the mechanical parking space.

[0033] The above-mentioned technologies have the following disadvantages; Detection algorithms are multi-stage and multi-sensor, and require complex post-processing, making it impossible to obtain accurate corner coordinate information end-to-end through modeling. Mechanical parking spaces are narrower and more complex than regular parking spaces, lacking obvious characteristic information (such as the markings on regular parking spaces), and do not differentiate between spaces with slopes or other features. During the parking phase, the visual perception information from the parking space search phase becomes invalid and inaccurate. The above method fails to accurately distinguish between the parking space search and parking phases, and fails to effectively update perception information and correct the corner position of the parking space in a timely manner.

[0034] The related technologies are multi-stage and complex in acquiring all parking space corner points. Given the diverse and complex scenarios of mechanical parking spaces, including those with slopes, a single method cannot simultaneously handle both types of spaces. Furthermore, during the vehicle search phase for mechanical parking spaces, the obtained corner points are inaccurate due to the considerable distance, and this corner point coordinate information is not corrected after being transmitted to the planning module, potentially introducing errors. While the parking phase does not provide complete overall parking space information, dividing lines, entrance corner points, and slope points can be used as corrections to reduce parking space errors; however, the related technologies do not consider the issue of corner point information correction.

[0035] In view of this, embodiments of this application provide a mechanical parking space recognition method, device, and automatic parking system. By using only fisheye lenses to acquire images for mechanical parking space classification and recognition detection, and employing two different resolution field-of-view models to distinguish between the vehicle search stage and the parking stage, the detection corner points of the mechanical parking spaces are dynamically adjusted after the mechanical parking space is located to improve the accuracy of the mechanical parking space corner points, thereby solving at least one of the above-mentioned technical problems.

[0036] In some embodiments, the mechanical parking space recognition method in this application can be performed by an automatic parking system.

[0037] In some embodiments, this application employs a dual-model architecture to distinguish between the vehicle search and parking stages. The model structure may include detection and segmentation tasks, and can also achieve end-to-end identification of mechanical parking spaces with and without slopes, and dynamically adjust the corner coordinates of parking spaces to reduce errors.

[0038] As an example, in this application, a library-finding model for long-distance library-finding and a berthing model for short-distance berthing can be pre-built. Additionally, datasets for training the library-finding model and the berthing model can be pre-built.

[0039] For example, images captured by a vehicle-mounted fisheye lens (referred to as fisheye images) can be converted to a Bird's-Eye View (Bev) to transform them into a top-down view. The Bev images are then manually labeled. The labels required by the model searching for the parking space can include: parking space location information (e.g., coordinates of the four corner points), parking space type (e.g., other parking space types, mechanical parking spaces), mechanical parking space line segmentation masks, and slope line segmentation masks. The labels required by the parking model can include: mechanical parking space line segmentation masks, slope line segmentation masks, entrance corner point coordinate information (e.g., coordinates of the two entrance corner points), and slope corner point coordinate information (e.g., coordinates of the two slope corner points). Then, the labels in the dataset are uniformly converted to the format required by the modeling objective.

[0040] In some embodiments, both the parking space search model and the parking space entry model are multi-task models. As an example, the overall model structure of the parking space search model and the parking space entry model is the same, only the input image and output content are different. The output of the parking space search stage is the parking space in the distance obtained by searching the parking space, and the output of the parking space entry stage is the nearby entrance point, slope point (if there is a slope) and segmentation. The above elements are then used to refine the parking space.

[0041] Figure 1 This is a schematic diagram of the library search model in an embodiment of this application. For example... Figure 1As shown, the library-finding model can include a parking space detection module and a parking space segmentation module. The parking space detection module is used for parking space corner point regression and parking space type classification, while the parking space segmentation module is used for mechanical parking line segmentation and slope line segmentation. Additionally, the library-finding model includes a backbone network for image feature extraction. The input to the library-finding model is a BEV image (which can be called a bird's-eye view image) used for long-distance library-finding. This BEV image is obtained by converting a captured fisheye image. The fisheye image is an image captured by an onboard fisheye lens.

[0042] Figure 2 This is a schematic diagram of the docking model in an embodiment of this application. For example... Figure 2 As shown, the parking model can include a corner detection module and a mechanical segmentation module. The corner detection module is used for entrance corner regression, slope corner regression, and corner type classification, while the mechanical segmentation module is used for mechanical parking line segmentation and slope line segmentation. Additionally, the parking model includes a backbone network for image feature extraction. The input to the library search model is a bird's-eye view image for close-range parking, obtained by transforming a captured fisheye image.

[0043] As one implementation method, the above-mentioned parking space corner point regression, entrance corner point regression, and slope corner point regression are all achieved by regressing the offset of the pixel anchor point within the parking space to the corresponding corner point.

[0044] It should be noted that the backbone network in this application can be composed of a series of stacked convolutional layers (such as a backbone + neck structure), which extracts image features by continuously stacking convolutional, upsampling, and downsampling modules. The parking space detection and segmentation modules in the library search model, as well as the corner detection and mechanical segmentation modules in the parking arrival model, can all be composed of a series of convolutional and upsampling modules. In one example, the parking space detection module, parking space segmentation module, corner detection module, and mechanical segmentation module can be referred to as the parking space detection head, parking space segmentation head, corner detection head, and mechanical segmentation head, respectively.

[0045] In the database search model, the parking space detection module performs the parking space detection task. This task can include a type classification subtask and a point regression subtask.

[0046] In the type classification subtask, a classification approach can be used to distinguish the parking space type for each parking space. In this application, the parking space types can be divided into other parking space types and mechanical parking spaces. Furthermore, the classification loss for parking space types can be calculated using the formula shown below:

[0047] in, Indicates the classification loss for parking space type. This can represent the number of samples of pixel anchor points within a parking space. and It can represent a coefficient. It can represent the first The prediction result of the parking space type (such as the prediction probability value) of the pixel anchor point within each parking space. It can represent the first Category labels for pixel anchor points within each parking space.

[0048] In the point regression subtask, since the projected Bev image still has some edge distortion, the conventional bounding rectangle method is not applicable. This application can adopt the method of regression learning of the four corner points of the parking space to regress the offset of the pixel anchor point in the parking space to the four corner points (dx1, dy1, dx2, dy2, dx3, dy3, dx4, dy4). Figure 3 This is an exemplary schematic diagram of a mechanical parking space from a Bev's perspective, provided in an embodiment of this application. For example... Figure 3 As shown, (x1, y1), (x2, y2), (x3, y3), and (x4, y4) are the coordinates of the four corner points of the mechanical parking space. dx1 represents the x-direction offset of the pixel anchor point within the parking space to the corner point (x1, y1), dy1 represents the y-direction offset of the pixel anchor point within the parking space to the corner point (x1, y1), dx2 represents the x-direction offset of the pixel anchor point within the parking space to the corner point (x2, y2), and dy2 represents the x-direction offset of the pixel anchor point within the parking space to the corner point (x2, y2). The y-direction offset of the pixel anchor point within the parking space to the corner point (x2, y2), dx3 can represent the x-direction offset of the pixel anchor point within the parking space to the corner point (x3, y3), dy3 can represent the y-direction offset of the pixel anchor point within the parking space to the corner point (x3, y3), dx4 can represent the x-direction offset of the pixel anchor point within the parking space to the corner point (x4, y4), and dy4 can represent the y-direction offset of the pixel anchor point within the parking space to the corner point (x4, y4).

[0049] In the point regression subtask, the regression loss for parking space corner points can be calculated using the following formula:

[0050] in, This can represent the regression loss at the parking space corner. This can represent the number of samples of pixel anchor points within a parking space. Indicates the predicted first The pixel anchor point within each parking space to the first corner points Directional offset This can represent the predicted first... The pixel anchor point within each parking space to the first corner points Directional offset It can represent the first The pixel anchor point within each parking space to the first corner points Direction truth offset, It can represent the first The pixel anchor point within each parking space to the first corner points Directional truth offset.

[0051] Additionally, in the library search model, the parking space segmentation module performs the parking space segmentation task. This task can include a mechanical parking line segmentation subtask and a mechanical parking slope segmentation subtask, both with identical loss calculations. In both subtasks, lines can be labeled using a conventional polygon method. It should be noted that segmenting and labeling slope lines can be used to control vehicle parking posture and speed. The center points of the segmented parking line segments can be used to adjust the corner coordinates of the parking space detection.

[0052] In both the mechanical parking space line segmentation subtask and the mechanical parking space slope segmentation subtask, the segmentation loss can be calculated using the following formula:

[0053] in, This represents the number of samples for the feature points. and Represents the coefficient. Indicates the first The category labels of feature points Indicates the first Predicted probability values ​​of feature points.

[0054] In the docking model, the corner detection module is used to perform corner detection tasks, which can include type classification subtasks and point regression subtasks.

[0055] In the type classification subtask of the corner detection task, a classification approach can be used to distinguish the type of each corner point. The type can include slope points and entry line points. The formula used to calculate the classification loss for corner point types in this type classification subtask is similar to the formula for calculating the classification loss for parking space types introduced earlier, and can be referred to the relevant explanations above, so it will not be repeated here.

[0056] In the point regression subtask of the corner detection task, the points can still be labeled in the form of bounding boxes (4 points). Therefore, the learning method is the same as that for parking space corners, which uses regression learning of the four corner points to regress the offset of the pixel anchor point to the four corner points.

[0057] In addition, in the parking model, the mechanical segmentation module is used to perform parking space segmentation tasks, which can include mechanical parking space slope segmentation subtasks and mechanical parking space line segmentation subtasks. The loss calculation for these two subtasks is the same. In this application, lines can be labeled using a conventional polygon method. Specifically, the slope lines are segmented and labeled to control the vehicle's parking posture and speed. The center point of the segmented parking space lines is used to adjust the corner coordinates of the parking space detection. It should be noted that the formulas used to calculate the segmentation loss for these two subtasks are similar to those used in the mechanical parking space line segmentation subtask and mechanical parking space slope segmentation subtask described above; please refer to the relevant explanations above, which will not be repeated here.

[0058] Figure 4 This is a flowchart illustrating the mechanical parking space recognition method provided in this application embodiment, as shown below. Figure 4 As shown, the mechanical parking space identification method includes the following steps: S401: During the vehicle's parking search phase, the parking search model is used to process the first bird's-eye image corresponding to the first image captured by the vehicle's fisheye lens to obtain the first parking space information of at least one mechanical parking space. The first parking space information includes initial position information. S403: Based on the first parking space information, determine the target parking space to be parked in from at least one of the above-mentioned mechanical parking spaces; S405: After the vehicle enters the parking phase of the target parking space, the second bird's-eye view image corresponding to the second image captured by the on-board fisheye lens is processed using the parking model to obtain the second parking space information of the target parking space. The second parking space information includes the entrance corner point coordinate information and the mechanical parking space line segmentation mask. S407: Correct the initial position information of the target parking space based on the second parking space information to obtain the target position information of the target parking space.

[0059] Figure 4 The corresponding implementation provides a solution that can use two field-of-view models with different resolutions to distinguish between the vehicle search stage and the parking stage. After finding the location of the mechanical parking space, the detection corner point of the mechanical parking space is dynamically adjusted to improve the accuracy of the mechanical parking space corner point.

[0060] Steps S401 to S407 will be explained below.

[0061] In step S401, during the vehicle's parking space search phase, the parking space search model can be used to process the first bird's-eye view image corresponding to the first image captured by the vehicle-mounted fisheye lens to obtain first parking space information for at least one mechanical parking space. The first parking space information includes initial position information. Further, the first parking space information may also include at least one of parking space type, mechanical parking space line segmentation mask, and slope line segmentation mask.

[0062] The first bird's-eye view image is obtained by transforming a first image. For example, the first image can be transformed into the first bird's-eye view image using an inverse perspective mapping (IPM) projection algorithm.

[0063] In step S403, the target parking space to be parked in can be determined from the at least one mechanical parking space based on the first parking space information; for example, the target parking space to be parked in can be determined from the at least one mechanical parking space based on the initial position information in the first parking space information. It should be understood that when the first parking space information also includes other information besides the initial position information, the target parking space to be parked in can be determined from the at least one mechanical parking space based on the initial position information and the other information. Alternatively, the target parking space to be parked in can be determined from the at least one mechanical parking space based on the vehicle's current position and the first parking space information.

[0064] In step S405, after the vehicle enters the parking phase for the target parking space, the parking model can be used to process the second bird's-eye view image corresponding to the second image captured by the onboard fisheye lens to obtain the second parking space information of the target parking space. The second parking space information includes entrance corner point coordinate information (such as the coordinate information of two entrance corner points) and a mechanical parking space line segmentation mask. Further, the second parking space information may also include slope information, which may include slope corner point coordinate information (such as the coordinate information of two slope corner points) and / or a slope line segmentation mask.

[0065] The second bird's-eye view image is obtained by transforming a second image. For example, the second image can be transformed into a second bird's-eye view image using an inverse perspective transformation projection algorithm.

[0066] In step S407, the initial position information of the target parking space is corrected according to the second parking space information to obtain the target position information of the target parking space.

[0067] Specifically, the center point coordinates of the mechanical parking line of the target parking space can be determined based on the mechanical parking line segmentation mask in the second parking space information. Then, the initial position information of the target parking space can be corrected based on the aforementioned entrance corner coordinate information and the aforementioned center point coordinates.

[0068] Furthermore, the initial position information includes the coordinate information of four corner points. The aforementioned entrance corner point coordinate information includes the coordinate information of the two entrance corner points of the target parking space. Based on the aforementioned entrance corner point coordinate information and the aforementioned center point coordinates, correcting the initial position information of the target parking space may include: determining the coordinate information of other corner points of the target parking space besides the aforementioned two entrance corner points based on the aforementioned two entrance corner point coordinates and the aforementioned center point coordinates; and replacing the coordinate information of the corresponding corner points in the initial position information of the target parking space with the coordinate information of the aforementioned two entrance corner points and the aforementioned other corner points.

[0069] Furthermore, the second parking space information also includes slope information, which includes slope corner point coordinate information and / or slope line segmentation mask; based on the coordinate information of the two entrance corner points and the coordinates of the center point, determining the coordinate information of other corner points of the target parking space other than the two entrance corner points may include: determining the coordinate information of the other corner points based on the coordinate information of the two entrance corner points, the coordinates of the center point and the slope information.

[0070] In some embodiments, the slope information includes slope corner coordinates and a slope line segmentation mask, and the first parking space information of the target parking space also includes the slope line segmentation mask. The slope line segmentation mask in the first parking space information can be modified based on the slope line segmentation mask in the slope information to obtain a modified slope line segmentation mask. Therefore, by combining the modified slope line segmentation mask and the slope corner coordinates, accurate positioning of the slope characteristics of the target parking space can be achieved.

[0071] The solution provided in this application separates the parking space search stage and the parking entry stage into two models. The parking space search stage obtains a rough estimate of the parking space location and type; after switching to the parking entry stage model, the corner point positions are fine-tuned based on the detection points and segmentation. In addition, the two models in this application have basically the same structure, only the input and output elements are different, and they are end-to-end models with multiple tasks, including both detection and segmentation.

[0072] This application adopts a multi-task joint learning structure, making full use of perception information to merge mechanical parking space detection and mechanical parking space segmentation into an end-to-end structure, distinguishing between the parking search and parking stages, and splitting them into two models. The parking search model is used to detect and segment to determine the approximate parking space location, and more accurate parking space corner point information is obtained during the parking process.

[0073] Figure 5 This is a schematic diagram of the mechanical parking space recognition device provided in an embodiment of this application. Figure 5 As shown, the mechanical parking space recognition device includes: The parking space search processing module 501 is configured to process the first bird's-eye view image corresponding to the first image captured by the onboard fisheye lens using the parking space search model during the parking space search phase of the vehicle, and obtain the first parking space information of at least one mechanical parking space, the first parking space information including initial position information. The determination module 502 is configured to determine the target parking space to be parked in from at least one mechanical parking space based on the first parking space information. The parking processing module 503 is configured to process the second bird's-eye image corresponding to the second image captured by the onboard fisheye lens using the parking model after the vehicle enters the parking stage for the target parking space, and obtain the second parking space information of the target parking space. The second parking space information includes the entrance corner point coordinate information and the mechanical parking space line segmentation mask. The correction module 504 is configured to correct the initial position information of the target parking space based on the second parking space information to obtain the target position information of the target parking space.

[0074] In some embodiments, the library search processing module 501 is further configured to convert the first image into a first bird's-eye image by using an inverse perspective transformation projection algorithm before processing the first bird's-eye image corresponding to the first image acquired by the vehicle-mounted fisheye lens using the library search model; the parking processing module 503 is further configured to convert the second image into a second bird's-eye image by using an inverse perspective transformation projection algorithm before processing the second bird's-eye image corresponding to the second image acquired by the vehicle-mounted fisheye lens using the parking model.

[0075] In some embodiments, the first parking space information further includes at least one of parking space type, mechanical parking space line segmentation mask, and slope line segmentation mask.

[0076] In some embodiments, the correction module 504 is specifically configured as follows: Based on the mechanical parking line segmentation mask in the second parking space information, determine the center point coordinates of the mechanical parking line of the target parking space; Based on the above entrance corner coordinates and center point coordinates, the initial position information of the target parking space is corrected.

[0077] In some embodiments, the initial position information includes the coordinate information of four corner points, wherein the aforementioned entrance corner point coordinate information includes the coordinate information of the two entrance corner points of the target parking space; the correction module 504 is specifically configured to: Based on the coordinate information of the two entrance corner points and the coordinates of the center point, determine the coordinate information of the other corner points of the target parking space besides the two entrance corner points. Use the coordinate information of the two entrance corner points and the other corner points mentioned above to replace the coordinate information of the corresponding corner points in the initial position information of the target parking space.

[0078] In some embodiments, the second parking space information further includes slope information, which includes slope corner coordinate information and / or slope line segmentation mask; the correction module 504 is specifically configured to: determine the coordinate information of the other corner points based on the coordinate information of the two entrance corner points, the coordinates of the center point and the slope information.

[0079] In some embodiments, the library search model includes a parking space detection module and a parking space segmentation module. The parking space detection module is used for parking space corner regression and parking space type classification, and the parking space segmentation module is used for mechanical parking space line segmentation and slope line segmentation. The parking entry model includes a corner detection module and a mechanical segmentation module. The corner detection module is used for entrance corner regression, slope corner regression and corner type classification, and the mechanical segmentation module is used for mechanical parking space line segmentation and slope line segmentation.

[0080] In some embodiments, parking space corner regression, entrance corner regression, and slope corner regression are all achieved by regressing the offset of the pixel anchor point within the parking space to the corresponding corner point.

[0081] For details regarding the implementation of the mechanical parking space recognition device and its modules, as well as its beneficial effects, please refer to the relevant descriptions in the method embodiments above; they will not be repeated here.

[0082] This application also provides an automatic parking system, which includes the above-mentioned mechanical parking space recognition device.

[0083] This application also provides an electronic device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, it implements the method of any of the above embodiments.

[0084] This embodiment also provides a computer-readable storage medium storing a computer program thereon, which is loaded by a processor to execute the steps in any of the methods described in the above embodiments. In this application embodiment, the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0085] This application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the method of any of the above embodiments.

[0086] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0087] The mechanical parking space recognition method, device, and automatic parking system provided in the embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A mechanical parking space recognition method, characterized in that, include: During the vehicle's parking search phase, the parking search model is used to process the first bird's-eye image corresponding to the first image captured by the vehicle's fisheye lens to obtain the first parking space information of at least one mechanical parking space, which includes initial position information. Based on the first parking space information, determine the target parking space to be parked in from the at least one mechanical parking space; After the vehicle enters the parking phase for the target parking space, the second bird's-eye image corresponding to the second image captured by the onboard fisheye lens is processed using the parking model to obtain the second parking space information of the target parking space. The second parking space information includes entrance corner point coordinate information and mechanical parking space line segmentation mask. The initial position information of the target parking space is corrected based on the second parking space information to obtain the target position information of the target parking space.

2. The mechanical parking stall identification method of claim 1, wherein, Before processing the first bird's-eye image corresponding to the first image acquired by the vehicle-mounted fisheye lens using the library search model, the process further includes: The first image is converted into the first bird's-eye view image using an inverse perspective transformation projection algorithm; Before processing the second bird's-eye image corresponding to the second image captured by the vehicle-mounted fisheye lens using the parking model, the method further includes: The second image is converted into the second bird's-eye view image using the inverse perspective transformation projection algorithm.

3. The mechanical parking stall identification method of claim 1, wherein, The first parking space information also includes at least one of the following: parking space type, mechanical parking space line segmentation mask, and slope line segmentation mask.

4. The mechanical parking stall identification method of claim 1, wherein, The step of correcting the initial position information of the target parking space based on the second parking space information includes: Based on the mechanical parking space line segmentation mask, determine the coordinates of the center point of the mechanical parking space line of the target parking space; The initial position information of the target parking space is corrected based on the coordinates of the entrance corner point and the coordinates of the center point.

5. The mechanical parking stall identification method of claim 4, wherein, The initial position information includes the coordinate information of four corner points, and the entrance corner point coordinate information includes the coordinate information of the two entrance corner points of the target parking space; The step of correcting the initial position information of the target parking space based on the entrance corner coordinates and the center point coordinates includes: Based on the coordinate information of the two entrance corner points and the coordinates of the center point, determine the coordinate information of the other corner points of the target parking space besides the two entrance corner points; The coordinate information of the two entrance corner points and the other corner points is used to replace the coordinate information of the corresponding corner points in the initial position information of the target parking space.

6. The mechanical parking stall identification method of claim 5, wherein, The second parking space information also includes slope information, which includes slope angle coordinates and / or slope line segmentation mask; The step of determining the coordinate information of the other corner points of the target parking space besides the two entrance corner points based on the coordinate information of the two entrance corner points and the coordinates of the center point includes: Based on the coordinate information of the two entrance corner points, the coordinates of the center point, and the slope information, the coordinate information of the other corner points is determined.

7. The mechanical parking space identification method according to any one of claims 1-6, characterized in that, The database search model includes a parking space detection module and a parking space segmentation module. The parking space detection module is used for parking space corner point regression and parking space type classification. The parking space segmentation module is used for mechanical parking space line segmentation and slope line segmentation. The parking model includes a corner detection module and a mechanical segmentation module. The corner detection module is used for entrance corner regression, slope corner regression, and corner type classification. The mechanical segmentation module is used for mechanical parking line segmentation and slope line segmentation.

8. The mechanical parking stall identification method of claim 7, wherein, The regression of parking space corner points, the regression of entrance corner points, and the regression of slope corner points are all achieved by regressing the offset of pixel anchor points within the parking space to the corresponding corner points.

9. A mechanical parking space recognition device, characterized in that include: The parking space search processing module is configured to process the first bird's-eye view image corresponding to the first image captured by the onboard fisheye lens during the parking space search phase of the vehicle using the parking space search model, so as to obtain the first parking space information of at least one mechanical parking space, the first parking space information including initial position information. The determining module is configured to determine the target parking space to be parked in from the at least one mechanical parking space based on the first parking space information; The parking processing module is configured to, after the vehicle enters the parking phase for the target parking space, process the second bird's-eye image corresponding to the second image captured by the on-board fisheye lens using the parking model to obtain the second parking space information of the target parking space, the second parking space information including entrance corner point coordinate information and mechanical parking space line segmentation mask; The correction module is configured to correct the initial position information of the target parking space based on the second parking space information to obtain the target position information of the target parking space.

10. An automatic parking system, characterized by Includes the mechanical parking space identification device as described in claim 9.

Images