Automated parking mapping system for a vehicle and method thereof

By installing monocular cameras on both sides of the vehicle to construct two-dimensional or three-dimensional point cloud maps and removing dynamic feature points, the problems of large storage capacity and inaccurate positioning of three-dimensional point cloud maps are solved, achieving efficient automatic parking map construction and accurate parking area positioning.

CN116758814BActive Publication Date: 2025-12-16OTOBRITE ELECTRONICS INC
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210210106.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-04
Publication Date
2025-12-16
Estimated Expiration
2042-03-04

AI Technical Summary

Technical Problem

In existing technologies for automatic parking functions, the storage capacity of 3D point cloud maps is large and affects the operation of other functions of the vehicle system, resulting in inaccurate positioning.

Method used

Two monocular cameras are installed on both sides of the vehicle to capture continuous image frames around the vehicle. The processing unit constructs a two-dimensional or three-dimensional point cloud map, and semantic operations are combined to remove dynamic feature points, reducing storage requirements and improving accuracy.

Benefits of technology

It saves map storage capacity, improves the completeness and accuracy of 3D point cloud maps, and ensures precise positioning of the automatic parking function.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116758814B_ABST
    Figure CN116758814B_ABST
Patent Text Reader

Abstract

The application discloses an automatic parking mapping system loaded on a vehicle, comprising a first camera installed on a first side of the vehicle to shoot a first continuous image frame; a second camera installed on a second side opposite to the first side to shoot a second continuous image frame; an image information receiving module; a vehicle information assembly interface receiving driving data information; a processing unit defining a first detection area and a second detection area; when the first detection area and the second detection area include a recognition target, the processing unit constructs a two-dimensional point cloud map corresponding to the recognition target by using the first continuous image frame and the second continuous image frame; when the recognition target is not included in the first detection area or the second detection area, the processing unit generates a feature point with depth information by using the first continuous image frame and the second continuous image frame, and matches the driving data information of the vehicle to construct a three-dimensional point cloud map; and a storage unit storing the two-dimensional point cloud map and the three-dimensional point cloud map.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, in particular to an automatic parking mapping method. BACKGROUND

[0002] The pros and cons of the automatic parking function mainly come from the establishment and positioning of the parking field map. The positioning of the vehicle comes from the accurate map establishment. The general common indoor parking field is limited by the poor reception of the global satellite positioning system (GPS), so it is more dependent on the establishment and recognition of the field map by the vehicle itself to achieve automatic parking. The simultaneous localization and mapping (SLAM) technology often uses a three-dimensional point cloud map. The use of a three-dimensional point cloud map greatly improves the shortcomings of a two-dimensional point cloud map, especially affecting the positioning and pose of the vehicle. A three-dimensional point cloud map can be drawn through different sensors, such as Lidar, ultrasonic waves, radar, and cameras. However, the storage capacity occupied by the three-dimensional map of the parking field established by the camera image shooting method is very large, which affects the operation or storage of other functions of the vehicle safety driving system. SUMMARY

[0003] To solve the above technical problems, an automatic parking mapping system loaded on a vehicle and a method thereof are provided.

[0004] The object of the present application can be achieved by the following technical solutions:

[0005] An automatic parking mapping system loaded on a vehicle, comprising:

[0006] a first camera mounted on a first side of a vehicle for shooting a first continuous image frame of the environment around the vehicle,

[0007] a second camera mounted on a second side opposite to the first side of the vehicle for shooting a second continuous image frame of the environment around the vehicle,

[0008] an image information receiving module for receiving the first continuous image and the second continuous image,

[0009] a vehicle information assembly interface for receiving driving data information from the vehicle,

[0010] a processing unit for defining a first detection area and a second detection area corresponding to the first continuous image frame and the second continuous image frame respectively, and judging whether the first detection area and the second detection area contain a recognition target,

[0011] wherein, when the first detection area and the second detection area include the identified target, the processing unit constructs a two-dimensional point cloud map corresponding to the identified target using the first continuous image frame and the second continuous image frame; when the identified target is not included in the first detection area or the second detection area, the processing unit generates feature points with depth information using the first continuous image frame and the second continuous image frame, and matches the driving data information of the vehicle to construct a three-dimensional point cloud map,

[0012] a storage unit configured to store the two-dimensional point cloud map and the three-dimensional point cloud map.

[0013] The identified target can be set under multiple conditions, and can be set in an intersection or union manner according to multiple conditions.

[0014] The three-dimensional point cloud map is constructed from the first continuous image frame and the second continuous image frame not including the identified target, and the first continuous image frame and the second continuous image frame including the identified target taken within a predetermined time before and / or after determining that the identified target is not included in the first detection area or the second detection area.

[0015] The processing unit has semantic operation, which can remove dynamic feature points in the three-dimensional point cloud map.

[0016] The first detection area and the second detection area are bird's-eye view areas projected on the ground.

[0017] and,

[0018] A method for automatic parking mapping of a vehicle, comprising:

[0019] capturing a first continuous image frame of an environment around a first side of the vehicle,

[0020] capturing a second continuous image frame of an environment opposite to the first side of the vehicle,

[0021] receiving the first continuous image frame and the second continuous image frame,

[0022] receiving driving data information from the vehicle,

[0023] defining a first detection area and a second detection area corresponding to the first continuous image frame and the second continuous image frame, respectively,

[0024] determining whether the first detection area and the second detection area include an identified target,

[0025] when the identification target is included in the first detection area and the second detection area, a two-dimensional point cloud map corresponding to the identification target is constructed by using the first continuous image frame and the second continuous image frame; when the identification target is not included in the first detection area or the second detection area, a feature point with depth information is generated by using the first continuous image frame and the second continuous image frame, and vehicle driving data information is matched to construct a three-dimensional point cloud map,

[0026] the two-dimensional point cloud map and the three-dimensional point cloud map are stored.

[0027] The identification target can be set under multiple conditions, and can be set in an intersection or union manner according to multiple conditions.

[0028] The three-dimensional point cloud map is constructed by the first continuous image frame and the second continuous image frame not containing the identification target, and the first continuous image frame and the second continuous image frame containing the identification target obtained within a predetermined time before and / or after the identification target is determined not to be included in the first detection area or the second detection area.

[0029] The three-dimensional point cloud map further includes removing dynamic feature points in the three-dimensional point cloud map.

[0030] The first detection area and the second detection area are bird's-eye areas projected on the ground.

[0031] The present application has the following advantages:

[0032] 1. When an automatic parking establishment parking field map is established, the system of the present application receives continuous image frames of the environment around the vehicle taken by at least two monocular cameras, and by identifying an identification target in a specific detection area, a two-dimensional or three-dimensional point cloud map of the parking field can be selectively constructed to save the storage capacity of the map. At the same time, in order to achieve a more accurate three-dimensional point cloud map of the parking field, the present application can remove dynamic feature points in the three-dimensional point cloud map through semantic operation.

[0033] 2. Since the monocular camera of the present application is loaded on the front, rear and both sides of the vehicle, the replacement method of the front and rear image frames can reduce the distortion of the image edge caused by the lens, and improve the completeness and accuracy of the three-dimensional point cloud map of the parking field. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 is a structural schematic view of an automatic parking mapping system loaded on a vehicle of the present application;

[0035] Figure 2 is a flow chart of an automatic parking mapping and positioning method of the present application;

[0036] Figure 3 is a schematic view of camera arrangement of the first embodiment of the present application on a vehicle;

[0037] Figure 4 is a schematic view of automatic parking mapping generation of the first embodiment of the present application;

[0038] Figure 5 is a schematic view of camera arrangement of the second embodiment of the present application on a vehicle. DETAILED DESCRIPTION

[0039] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0040] In the description of the present application, it should be understood that the orientation or positional relationship indicated by the terms "upper", "lower", and the like is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or component referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application.

[0041] In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise explicitly specified and limited.

[0042] In the present application, unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connecting", "fixing" and the like should be understood broadly, for example, can be fixed connection, or detachable connection, or integral; can be mechanical connection, or electrical connection; can be directly connected, or indirectly connected through an intermediate medium; can be the internal communication of two components or the interaction relationship between two components. For those of ordinary skill in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0043] The present application will be further described in detail below with reference to the drawings and specific embodiments:

[0044] Please refer to Figure 1 The application discloses an automatic parking mapping system 100 loaded on a vehicle, comprising a first camera 101, a second camera 102, an image information receiving module 103, a vehicle information assembly interface 104, a processing unit 105 and a storage module 106.

[0045] The first camera 101 and the second camera 102 are usually single-purpose cameras, which are usually composed of a shell, a lens, an image sensing circuit board, an image processing circuit board, a connector, a power line, an image output line or a control line. The lens is exposed to the outside through a lens slot hole of the shell, generally has a fisheye lens or a wide-angle lens, and the image sensing circuit board is provided with a photosensitive component corresponding to the lens, which converts the light signal from the lens into an electronic image signal. The first camera 101 and the second camera 102 can be installed on the front, rear and / or side of the vehicle, each having a different field of view of the surrounding environment of the vehicle. The image processing circuit board is provided with a processing module and a memory, the memory temporarily stores the electronic image signal, and the processing module processes the electronic image signal to generate a processed image. The connector electrically connects the image sensing circuit board and the image processing circuit board. The power line passes through the shell to electrically connect the image processing circuit board to provide power, and the image output line passes through the shell to electrically connect the image processing circuit board. The control line passes through the shell to electrically connect the image processing circuit board to input a control signal. The processing module selectively outputs the processed image from the image output line according to the control signal. The camera is usually installed on the outside of the vehicle to avoid being blocked by the field of view of the vehicle itself, and is mainly used to shoot continuous image frames of the surrounding environment of the vehicle. In order to obtain better image quality, the format of the continuous image frames can be LVDS format information.

[0046] In the application, the image information receiving module 103 connects the first camera 101 and the second camera 102, and is used for receiving first continuous image frames of the surrounding environment of the first side of the vehicle shot by the first camera 101 and receiving second continuous image frames of the surrounding environment opposite to the first side of the vehicle shot by the second camera 102. The image information receiving module 103 also has an image processing unit (ISP, Image Signal Processor), which can process functions such as lens correction, pixel correction, color interpolation, Bayer noise removal, white balance correction, color correction, gamma correction and color space conversion. The image information receiving module 103 usually has an LVDS (Low Voltage Differential Signaling) or MIPI CSI transmission interface (not marked).

[0047] The vehicle information assembly interface 104 is connected to the vehicle CAN bus to receive driving data from the vehicle. The driving data from the vehicle includes, for example, vehicle speed, engine speed, steering angle, acceleration, gear position, etc., and can even include sensing information from an inertial measurement unit (IMU) and a wheel speed sensor, GPS, etc. that can measure the distance of vehicle movement.

[0048] The processing unit 105 is the main computing unit of the present application, which is built-in with optimized artificial intelligence and has the ability to perform semantic operations on images. The processor of the processing unit 105 is usually a DSP (digital signal processor). DSP is suitable for performing various multiplication and addition operations (SOP: Sum of Products), such as FIR (Finite Impulse Response) filtering operation, IIR (Infinite Impulse Response) filtering operation, DFT (Discrete Fourier Transform), DCT (Discrete Cosine Transform), dot product operation, convolution operation, and matrix polynomial evaluation operation, etc. The processing unit 105 is connected to the image information receiving module 103 to process the converted continuous image frames from the image information receiving module 103. The processing unit 105 is also connected to the vehicle information assembly interface 104 to receive driving data from the vehicle. Therefore, according to the continuous image frames of the vehicle surroundings and the driving data, at least two continuous image frames of monocular cameras can be obtained when the automatic parking mapping function is turned on, and the point cloud map can be constructed.

[0049] The storage module 106 is mainly used to store continuous image frames from monocular cameras and / or various driving data, and to store the point cloud map after the automatic parking mapping is completed. The storage module 106 can be an internal integrated circuit memory or an external storage device such as an SSD or an SD card. The storage module 106 is connected to the processing unit 105 to store data according to the information generated by the processing unit 105.

[0050] Please refer to Figures 1 to 4 An embodiment of the operation of the automatic parking mapping system 100 according to the present application loaded on a vehicle is provided. The automatic parking mapping system 100 loaded on the vehicle according to the present application performs the following steps:

[0051] Step S01, capturing a first continuous image frame of a first side surrounding environment of the vehicle and a second continuous image frame of a second side surrounding environment of the vehicle opposite to the first side. When the user starts the automatic parking function on the vehicle, the first mapping mode must be performed first, so after starting the automatic parking function, the user must drive to the parking space (usually with a painted grid line) or a specific parking space in the parking area, the system 100 of the present application is loaded on the vehicle, and two monocular cameras are connected to the outside of the vehicle to capture the surrounding environment image of the vehicle, effectively establishing the scene map of the automatic parking. The two monocular cameras usually have a wide image viewing angle, so that the continuous images captured by the monocular cameras installed on the right side and the left side of the vehicle cover as much as possible the field of view of the surrounding environment of the vehicle, that is, constitute a nearly panoramic view. In this embodiment, the monocular cameras are installed on the right side and the left side of the vehicle, which are a first camera 101 installed on a first side of a vehicle for capturing a first continuous image frame of a surrounding environment of the vehicle, and a second camera 102 installed on a second side opposite to the first side of the vehicle for capturing a second continuous image frame of a surrounding environment of the vehicle. That is, the continuous image frames captured by the two sides are different and opposite surrounding environment image frames of the vehicle.

[0052] Step S02, receiving the first continuous image frame and the second continuous image frame and receiving driving data information from the vehicle. The system 100 of the present application has an image information receiving module 103 for receiving the first continuous image frame captured by the first camera 101 and the second continuous image frame captured by the second camera 102. These continuous image frames involve the internal and external parameters of the monocular camera, so they must be corrected. In the present application, the image information receiving module 103 has image processing functions and can receive the first continuous image frame and the second continuous image frame and perform image correction and other processing. At the same time, a vehicle information assembly interface 104 is connected to the vehicle CAN bus to receive driving data information from the vehicle. These driving data information of the vehicle mainly includes sensing information for measuring the moving distance and speed of the vehicle, which serves as actual distance information for matching the continuous image information of the surrounding environment of the vehicle.

[0053] Step S03, defining a first detection area and a second detection area corresponding to the first continuous image frame and the second continuous image frame, respectively. Please refer to Figure 3After receiving the continuous image frames processed by the image information receiving module 103, the processing unit 105 in the system is used to define the first detection area 201 and the second detection area 202 corresponding to the first continuous image frame and the second continuous image frame respectively. The monocular camera of the present application is used to capture the field features in the parking field that can be effectively memorized, especially the two-dimensional features that can be accessed and do not occupy large storage capacity, such as parking spaces in the parking field (usually with painted grid lines) or ground markings, indicator symbols or other devices on the ground of the parking field. The memorized and effective two-dimensional features have the actual distance value estimated from the continuous image frames between the vehicle and the features, which is sufficient to serve as the data for building the two-dimensional map of the parking field. However, due to the limitations of the image captured by the wide-angle monocular camera (such as image distortion), the distance of the object in the environment estimated from the image is limited by the specifications of the monocular camera itself, so there is a certain effective detection range on the image. In the present embodiment, the length (parallel to the side of the vehicle) of the first detection area 201 and the second detection area 202 can be 5-6 meters, and the width (perpendicular to the side of the vehicle) can be 3-5 meters. However, the present application is not limited thereto, and the effective detection range of the first detection area 201 and the second detection area 202 can be increased according to the improvement of the image resolution of the monocular camera. In the present embodiment, the calculation of the first detection area 201 and the second detection area 202 by the processing unit 105 can be the three-dimensional space range of the first continuous image frame and the second continuous image frame, or the effective range of the length and the width after the bird's eye view conversion of the first continuous image frame and the second continuous image frame, i.e. the first detection area 201 and the second detection area 202 are the bird's eye view areas projected on the ground.

[0054] Step S04, judging whether the first detection area and the second detection area contain the recognition target. The processing unit 105, due to the image recognition of artificial intelligence, can perform specific recognition target recognition on the detection area of the previous step, and establish an automatic parking map according to the recognition result. The recognition target can be a single setting or a multiple condition setting, and can be set by intersection or union of multiple conditions, that is, two or more different recognition targets can be set, intersection represents that different recognition targets exist at the same time, and union represents that any recognition target exists. The system can be determined by the developer. Generally, the recognition target can include pedestrians, various vehicles, ground signs, lane lines, sidewalks, lane types, curbs, parking spaces, ground locks, limit stop rods, etc. In this embodiment, since the recognition target of the parking area is established to establish an effective parking map, especially in the establishment of a two-dimensional map, simple and accurate recognition targets are needed, so it is mostly fixed objects in the parking area, such as painted parking grid lines, unpainted parking grid lines, parking space numbers, ground locks, limit stop rods, ground signs, and parking lot pillars, curved lines, etc.

[0055] After defining the automatic parking area as an effective recognition target for map establishment, please refer to Figure 4 The establishment method of the automatic parking map of the present application is:

[0056] (1) The result of the recognition of the processing unit 105 is that when the first detection area 201 and the second detection area 202 contain the recognition target, the processing unit 105 constructs a two-dimensional point cloud map corresponding to the recognition target from the first continuous image frame and the second continuous image frame. That is, when the first detection area 201 and the second detection area 202 contain the recognition target, the processing unit 105 establishes a two-dimensional point cloud map according to the relative position of the recognition target relative to the vehicle, such as Figure 4 The vehicle is at position A in the two-dimensional point cloud map. Such a two-dimensional point cloud map is established from the bird's eye view. Further, the processing unit 105 can calculate the proportion or recognition rate of the recognition target existing in the first detection area 201 and the second detection area 202 through the two-dimensional point cloud map, and once the proportion or recognition rate reaches a certain threshold, it is considered to meet the condition for establishing a two-dimensional point cloud map. Similarly, such a threshold can be determined by the system developer.

[0057] (2) When the processing unit 105 identifies that the recognition target is not included in the first detection area 201 or the second detection area 202, the processing unit 105 generates feature points with depth information from the first continuous image frame and the second continuous image frame, and matches the driving data information of the vehicle to construct a three-dimensional point cloud map. That is, in the first detection area 201 or the second detection area 202, if one of them fails to detect the recognition target, or fails to reach the threshold value for establishing a two-dimensional point cloud map, the processing unit 105 will operate the first continuous image frame and the second continuous image frame to generate feature points with depth information, while matching the driving data information of the vehicle to construct a three-dimensional point cloud map, such as Figure 4 the position of the vehicle at B. In detail, the processing unit 105 extracts feature points from the first continuous image frame and the second continuous image frame by using a model-based method instead of a computer vision method. The method mainly estimates the descriptors of feature points in each frame of the image, especially the points, edges, corners, and textures of columns, walls, road signs, symbols, etc. (but not limited to the above) commonly seen in parking lots. With the movement of the vehicle, the descriptors of the feature points in the front and back frames are paired one by one, and the projection changes of the paired feature points in the stereo coordinates are calculated. Finally, the feature points with depth information in each frame of the continuous image information are obtained, and then the feature points are matched with the driving data information of the vehicle. Since the image is composed of a large number of continuous frames, the front and back frames of the image represent a time difference. Even if the processing unit 105 generates feature points with depth information from each frame of the continuous image, there will be a certain difference with the actual distance traveled by the vehicle. Therefore, in this embodiment, the processing unit 105 receives the driving data information of the vehicle from the vehicle information assembly interface 104, especially the inertial measurement unit (IMU) and the wheel speed sensor, which can measure the sensing information of the vehicle movement distance. When the vehicle movement distance corresponding to the front and back frames of the image is known, the actual depth information of the feature points can be obtained by matching the feature points with the driving information of the vehicle based on the principle of triangulation.

[0058] However, to establish a better three-dimensional point cloud map, in addition to establishing a three-dimensional point cloud map from the corresponding first continuous image frame and second continuous image frame when the processing unit 105 fails to recognize the recognition target in the first detection area 201 or the second detection area 202, feature points with depth information are also generated in the front or back of the corresponding first continuous image frame and second continuous image frame within a predetermined time (and several frames before and after), to construct a three-dimensional point cloud map, that is Figure 4The vehicle travels to the front or rear of the B position for a predetermined time (or a number of frames), and the feature points with depth information in the continuous image frames can be determined. Therefore, according to the present application, the three-dimensional point cloud map can be constructed from the first continuous image frame not containing the recognition target, the second continuous image frame, and the first continuous image frame and the second continuous image frame containing the recognition target before and / or after a predetermined time before and / or after determining that the recognition target is not included in the first detection area or the second detection area. The three-dimensional point cloud map and the two-dimensional point cloud map are included in the same number of continuous frames before and after. Similarly, the predetermined time (or the number of frames) can be determined by the system developer.

[0059] In addition, for the labeling of all feature points in the image, the processing unit 105 has a semantic operation, which can remove the dynamic feature points in the three-dimensional point cloud map. The purpose is that for the first time mapping of the user, the parking area may include many dynamic objects, such as parked vehicles, pedestrians, and movable fire-fighting equipment. If these dynamic objects are considered as feature points and become part of the three-dimensional point cloud map when mapping, it will cause the failure of the automatic parking function positioning when the dynamic objects change or move during the subsequent automatic parking of the user. Therefore, through the semantic operation of the processing unit 105, the dynamic feature points in the area framed by the semantic operation in each frame of the continuous image frames are deleted, that is, the information with dynamic feature points is not stored in the three-dimensional point cloud map.

[0060] Step S05, store the two-dimensional point cloud map and the three-dimensional point cloud map. This step is to store the two-dimensional point cloud map and the three-dimensional point cloud map established in the storage module 106. The stored information can include continuous image information captured by each monocular camera, and the relative coordinates of the two-dimensional point cloud map and the three-dimensional point cloud map corresponding to the monocular camera. Even the storage module 106 can store the maps established by the user in different parking areas, or the maps established at different times in the same parking area, and further can store the mapping completeness index after the semantic operation of the dynamic feature points in each parking area, and the GPS information, inertial measurement unit information, etc. from the vehicle. The information stored in the storage module 106 can be read, rewritten, or transmitted to the cloud server through the network for the purpose of optimizing the map and information exchange.

[0061] Please refer to Figure 5A second embodiment of the automatic parking mapping system 100 according to the present application is described. The second embodiment is similar to the first embodiment, but in the second embodiment, the monocular cameras are mounted on the vehicle to capture the continuous image information of the environment around the vehicle, i.e. the image information receiving module 103 receives the continuous image frames in front of the vehicle, the continuous image frames behind the vehicle, and the continuous image frames on the left and right sides of the vehicle to form a panoramic view. Since the monocular cameras have a wide field of view, the reliability of the depth information of the relative images is reduced, and the lenses mounted on the monocular cameras can easily cause distortion at the edges of the images. The distorted image information can cause poor matching, especially when matching the feature points of the image frames, and even abnormal matching of the feature points on the same object. Therefore, the processing unit 105 generates the two-dimensional point cloud map and the three-dimensional point cloud map by identifying the target in the first detection area 201 (corresponding to the image of the environment around the vehicle captured by the monocular camera on the right side of the vehicle) and the second detection area 202 (corresponding to the image of the environment around the vehicle captured by the monocular camera on the left side of the vehicle). In addition to receiving the continuous image frames on the left and right sides of the vehicle, the two-dimensional point cloud map and the three-dimensional point cloud map are also established by simultaneously receiving the continuous image frames from the front and rear of the vehicle. The images on the left and right sides of the vehicle overlap with the images in front of the vehicle and the images behind the vehicle. Since the image information captured by the monocular cameras is synchronous and real-time, in the overlapping area, in addition to determining that the three-dimensional point cloud map of the panoramic view can be completely covered by the field of view of the monocular cameras, the processing unit 105 marks the overlapping area without calculating the feature points of the image information. As the vehicle moves and changes its pose, the overlapping and possibly high-distortion images can be compensated by the image frames in front and behind to greatly increase the accuracy of the three-dimensional point cloud map.

[0062] The use of the present application in the mapping steps of the parking area can greatly save the memory capacity of the parking area map storage, and achieve effective and accurate map information to realize the application of automatic parking in different parking areas. In order to achieve a more accurate parking area map, the steps of S01 to S05 are repeated until the accuracy of the parking area is optimized.

[0063] The above merely describes preferred embodiments of the present application, and is not intended to limit the present application in any form. Although the present application has been disclosed with reference to the preferred embodiments above, it is not intended to limit the present application, and any person skilled in the art can make some changes or modifications to the above disclosed methods and technical contents to make equivalent embodiments with equivalent changes, as long as the changes or modifications do not depart from the technical solutions of the present application. Any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present application still falls within the scope of the technical solutions of the present application.

Claims

1. An automatic parking mapping system for a vehicle, comprising: a first camera mounted on a first side of a vehicle for capturing a first continuous image frame of an environment surrounding the vehicle; a second camera mounted on a second side of the vehicle opposite the first side for capturing a second continuous image frame of the environment surrounding the vehicle; an image information receiving module for receiving the first continuous image frame and the second continuous image frame; a vehicle information assembly interface for receiving driving data information from the vehicle; a processing unit for defining a first detection area and a second detection area corresponding to the first continuous image frame and the second continuous image frame, respectively; determining whether an identification target is included in the first detection area and the second detection area, wherein the identification target is set by multiple conditions and is set by an intersection or a union of the multiple conditions; characterized in that, when the identification target is included in the first detection area and the second detection area, the processing unit constructs a two-dimensional point cloud map corresponding to the identification target from the first continuous image frame and the second continuous image frame; when the identification target is not included in the first detection area or the second detection area, the processing unit generates feature points with depth information from the first continuous image frame and the second continuous image frame, and matches the driving data information of the vehicle to construct a three-dimensional point cloud map, wherein the three-dimensional point cloud map is constructed from the first continuous image frame and the second continuous image frame not containing the identification target, and the first continuous image frame and the second continuous image frame containing the identification target taken within a predetermined time before and / or after determining that the identification target is not included in the first detection area or the second detection area; and a storage unit for storing the two-dimensional point cloud map and the three-dimensional point cloud map. The processing unit has semantic operations to remove dynamic feature points in the three-dimensional point cloud map.

2. The automatic parking mapping system of claim 1, wherein, The first detection area and the second detection area are bird's-eye areas projected on the ground.

3. The automatic parking mapping system of claim 1, wherein, comprising:

4. A method of automatic parking mapping loaded on a vehicle, characterized by, capturing a first continuous image frame of an environment surrounding a first side of a vehicle; capturing a second continuous image frame of an environment surrounding a second side of the vehicle opposite the first side of the vehicle; receiving the first continuous image frame and the second continuous image frame; receiving driving data information from the vehicle; defining a first detection area and a second detection area corresponding to the first continuous image frame and the second continuous image frame, respectively; determining whether an identification target is included in the first detection area and the second detection area, wherein the identification target is set by multiple conditions and is set by an intersection or a union of the multiple conditions; ​ When the first detection area and the second detection area include the identified target, a two-dimensional point cloud map corresponding to the identified target is constructed from the first continuous image frame and the second continuous image frame; when the identified target is not included in the first detection area or the second detection area, a feature point with depth information is generated from the first continuous image frame and the second continuous image frame, and the driving data information of the vehicle is matched to construct a three-dimensional point cloud map, wherein the three-dimensional point cloud map is constructed from the first continuous image frame and the second continuous image frame not containing the identified target, and the first continuous image frame and the second continuous image frame containing the identified target taken within a predetermined time before and / or after determining that the identified target is not included in the first detection area or the second detection area; The two-dimensional point cloud map and the three-dimensional point cloud map are stored.

5. The method of automatically mapping for parking according to claim 4, wherein, Dynamic feature points in the three-dimensional point cloud map are removed.

6. The method of automatically parking mapping of claim 4, wherein, The first detection area and the second detection area are bird's-eye view areas projected on the ground.

Citation Information

Patent Citations

  • Parking space detection method for autonomous vehicle

    CN110766979A

  • Map construction method and device, SLAM system and storage medium

    CN112446845A