Asynchronous correlation for parking space measurement

Abstract

A vision-based perception system for a vehicle includes N cameras arranged on the vehicle and configured to generate a plurality of images, where N is an integer greater than 1. An image analysis module is configured to identify a plurality of features of a parking space in the plurality of images and generate a plurality of keyframe measurements therefrom. A keyframe selection module is configured to identify selected keyframe measurements in the plurality of keyframe measurements. A database is configured to receive the selected keyframe measurements in the plurality of keyframe measurements and selectively output unassociated keyframe measurements and existing features of the parking space within a predetermined range of the vehicle. A data association module is configured to receive the unassociated keyframe measurements and the existing features of the parking space from the database.

Description

[0001] introduction

[0002] The information provided in this section is intended to provide a general overview of the background of this disclosure. Within the scope of this section, the work of the presently designated inventors, and descriptions that may not conform to the prior art at the time of submission, are neither explicitly nor implicitly acknowledged as prior art to this disclosure. Technical Field

[0003] This disclosure relates to vision-based systems for vehicles, and more specifically to parking space location and mapping systems for autonomous driving systems. Background Technology

[0004] The vehicle may include an autonomous driving system that can operate the vehicle without driver input. The autonomous driving system receives images from multiple cameras pointing in different directions. For example, the vehicle may include one or more forward-facing cameras, one or more side-facing cameras, and one or more rear-facing cameras. The autonomous driving system needs to be able to sense the location of open parking spaces in a parking lot. The autonomous driving system may not know the location of parking spaces in the parking lot and / or whether each parking space is currently occupied before arriving at the parking lot. The autonomous driving system uses images generated by the cameras as it drives through the parking lot to determine the location of available parking spaces, and then parks the vehicle in one of the available parking spaces. Summary of the Invention

[0005] A vision-based perception system for a vehicle includes N cameras arranged on the vehicle and configured to generate multiple images, where N is an integer greater than 1. An image analysis module is configured to identify multiple features of a parking space in the multiple images and generate multiple keyframe measurements based on them. A keyframe selection module is configured to identify a selected keyframe measurement from the multiple keyframe measurements. A database is configured to receive the selected keyframe measurement from the multiple keyframe measurements and selectively output unassociated keyframe measurements and existing features of the parking space within a predetermined range of the vehicle. A data association module is configured to receive unassociated keyframe measurements and existing features of the parking space.

[0006] Among the other features, at least two of the unassociated keyframe measurements and the existing features of the parking space are asynchronous measurements. The data association module is configured to selectively associate selected keyframe measurements with existing features of the parking space, and associate other selected keyframe measurements with new features of the parking space.

[0007] Among other features, the data association module is configured to generate measurements between unassociated keyframe measurements and existing features of parking spaces, identify associations between measurements, and identify associations between landmarks.

[0008] Among other features, the data association module is configured to generate a measurement between unassociated keyframe measurements and existing features of the parking space by converting features of the parking space measured by the unassociated keyframe into local frames and calculating the distance to existing features of the parking space.

[0009] Among other features, when the distance is less than a predetermined distance, the data association module is configured to define an exact association between a corresponding unassociated keyframe measurement and a corresponding existing feature of the parking space. The data association module is configured to identify associations between measurements by calculating the distance between unassociated keyframe measurements and clustering the unassociated keyframe measurements into one or more clusters based on said distance.

[0010] Among other features, the data association module is configured to identify associations between measurements by selecting one of one or more clusters, calculating an initial estimate of the parking space corresponding to one of the one or more clusters, adding the parking space to the database, and repeating the calculation and addition for the remaining clusters of the one or more clusters.

[0011] Among other features, the data association module is configured to identify associations between landmarks by retrieving existing neighbor associations, retrieving relevant parking spaces from the database, selecting candidate parking spaces, and determining whether the orientation of adjacent parking spaces matches that of candidate parking spaces.

[0012] Among other features, the data association module is configured to identify associations between landmarks by: if a candidate parking space does not yet have an adjacent parking space in a predetermined neighbor location, selecting the best candidate as a neighbor from two or more candidates, updating the association links in the database, and repeating the selection and update for other candidate parking spaces.

[0013] Among other features, the data association module is configured to select the best candidate from two or more candidates based on the distance between two or more candidates and the candidate parking space.

[0014] A method for identifying and associating parking spaces includes generating multiple images using N cameras deployed on a vehicle, where N is an integer greater than 1; identifying multiple features of parking spaces in the multiple images and generating multiple keyframe measurements based thereon; identifying selected keyframe measurements from the multiple keyframe measurements; storing the selected keyframe measurements in a database; selecting unassociated keyframe measurements and existing features of parking spaces within a predetermined range of the vehicle from the database; and selectively associating the selected keyframe measurements with the existing features of the parking spaces, and associating other selected keyframe measurements with new features of the parking spaces. At least two of the unassociated keyframe measurements and existing features of the parking spaces are asynchronous measurements.

[0015] Among other features, selectively associating selected keyframe measurements from the keyframe measurements with existing features of the parking space, and associating other selected keyframe measurements from the keyframe measurements with new features of the parking space, includes generating measurements between unassociated keyframe measurements and existing features of the parking space; identifying associations between measurements; and identifying associations between landmarks.

[0016] Among other features, generating a measurement between an unassociated keyframe measurement and existing features of the parking space includes converting the parking space features of the unassociated keyframe measurement into a local frame and calculating the distance to the existing features of the parking space. When the distance is less than a predetermined distance, the method includes defining an exact association between a corresponding feature in the unassociated keyframe measurement and a corresponding feature in the existing features of the parking space.

[0017] Among other features, identifying associations between measurements includes calculating the distance between unassociated keyframe measurements; and clustering unassociated keyframe measurements into one or more clusters based on the distance.

[0018] Among other features, performing the correlation between measurements includes selecting one of one or more clusters; calculating an initial estimate of the parking spaces corresponding to one of the one or more clusters; adding the parking spaces to the database; and repeating the calculation and addition for the remaining clusters in the one or more clusters.

[0019] Among other features, identifying associations between landmarks includes retrieving existing neighbor associations; retrieving relevant parking spaces from the database; selecting candidate parking spaces; and determining whether the orientation of adjacent parking spaces matches that of candidate parking spaces.

[0020] Among other features, identifying associations between landmarks includes selecting the best candidate as a neighbor from two or more candidates if a candidate parking space does not already have an adjacent parking space in a predetermined neighbor location; updating the association links in the database; and repeating the selection and updating for other candidate parking spaces.

[0021] A vision-based perception system for a vehicle includes N cameras arranged on the vehicle and configured to generate multiple images, where N is an integer greater than 1. An image analysis module is configured to identify multiple features of a parking space in the multiple images and generate multiple keyframe measurements based thereon. A keyframe selection module is configured to identify a selected keyframe measurement from the multiple keyframe measurements. A database is configured to receive the selected keyframe measurement from the multiple keyframe measurements and selectively output unassociated keyframe measurements and existing features of the parking space within a predetermined range of the vehicle. A data association module is configured to receive the unassociated keyframe measurements and existing features of the parking space from the database. At least two of the unassociated keyframe measurements and existing features of the parking space are asynchronous measurements. The data association module is configured to selectively associate the selected keyframe measurement with existing features of the parking space and associate other selected keyframe measurements with new features of the parking space. The data association module is configured to generate measurements between the unassociated keyframe measurements and existing features of the parking space, identify associations between measurements, and identify associations between landmarks.

[0022] Other applicable areas of this disclosure will become apparent from the detailed description, claims, and drawings. The detailed description and specific examples are for illustrative purposes only and are not intended to limit the scope of this disclosure. Attached Figure Description

[0023] This disclosure will be more fully understood through detailed description and accompanying drawings, in which:

[0024] Figure 1 This is a functional block diagram of an example vehicle including a vision-based parking space location module according to the present disclosure, the parking space location module being configured to map and locate parking spaces in a parking lot;

[0025] Figure 2A It is a floor plan of an example of a vehicle passing through a parking lot with parking spaces and parking in one of the parking spaces, according to this disclosure;

[0026] Figure 2B The vehicle in the parking space is shown according to this disclosure;

[0027] Figure 3 This is a functional block diagram of an example parking space positioning module based on this disclosure;

[0028] Figure 4 and 5A This is a flowchart illustrating an example of a method for identifying parking spaces in a parking lot according to the present disclosure;

[0029] Figure 5B A comparison example of the center of a parking space according to this disclosure with the center of an existing parking space is shown; and

[0030] Figures 5C to 10 This is a flowchart illustrating an example of a method for identifying parking spaces in a parking lot according to this disclosure.

[0031] In the accompanying drawings, reference numerals may be used repeatedly to identify similar and / or identical elements. Detailed Implementation

[0032] Autonomous driving systems rely on vision-based systems, including multiple cameras. Associating measurements made by these cameras requires synchronizing the cameras for optimal performance. Furthermore, multi-frame association necessitates saving incomplete calculations and aggregating data until a stable solution is reached. Other issues include odometry drift, which can cause straight lines to curve slowly, thus distorting the sensed representation of the environment.

[0033] This disclosure relates to systems and methods for generating correlations between multi-camera, multi-frame measurements without requiring explicit synchronization using a database and global asynchronous queries. Retrieved measurements are correlated using an aggregation algorithm and associated with pre-existing landmarks using a one-to-many approach with robust outlier rejection.

[0034] Parking spaces are semantically insensitive. In other words, all objects at each location look similar or identical. Detector noise and odometer drift can cause straight parking lanes to appear curved. The systems and methods described in this paper use prior knowledge of the structure of parking spaces and parking lots to improve accuracy. The systems and methods described in this paper allow synchronization between different cameras without adding latency or forcing hardware synchronization.

[0035] Now for reference Figure 1 Vehicle 10 includes various sensors, such as a Global Positioning System (GPS) 14, an Inertial Measurement Unit (IMU) 18, and wheel speed sensors 20. Vehicle 10 includes one or more cameras 34-1, 34-2, ..., 34-N that generate images relative to the front, sides, and rear of vehicle 10, where N is an integer greater than 1. In some examples, cameras 34-1, 34-2, ..., 34-N include one or more forward-facing cameras, one or more side-facing cameras, and one or more rear-facing cameras.

[0036] The controller 42 includes an image analysis module 44 configured to receive images from cameras 34-1, 34-2, ..., 34-N to identify features of landmarks or parking spaces in the images (e.g., corners or centers of parking spaces) and generate keyframe measurements from the corresponding cameras to the landmarks in the images. In some examples, the image analysis module 44 may include a neural network configured to recognize various objects, such as corners of parking spaces.

[0037] The keyframe selection module 46 selects keyframe measurements from the images (each measurement has a timestamp). Cameras 34-1, 34-2, ..., 34-N generate a large number of samples, and not all samples provide new or useful information. For example, when the vehicle is stationary or moving slowly, measurements in subsequent keyframes may or may not change significantly over consecutive timestamps. Not all images contain parking spaces. The keyframe selection module 46 is configured to select some or all keyframes based on the values ​​of the data.

[0038] Simultaneous Localization and Mapping (SLAM) module 47 is configured to perform simultaneous localization and mapping. In some examples, SLAM module 47 includes the Georgia Tech Smoothing and Mapping (GTSAM) or g2o module, but other SLAM modules may also be used.

[0039] SLAM module 47 includes data association module 48, which is configured to receive keyframes and associate the keyframes with existing states (e.g., identified parking spaces) or new states (e.g., previously unidentified new parking spaces).

[0040] In some examples, optimization module 52 is configured to manage and optimize SLAM states (e.g., vehicle location and the location of parking space features). In some examples, SLAM states correspond to vehicle and / or camera locations and parking space features such as the corner or center of the parking space. In some examples, optimization module 52 includes a cost function extended using parking space / parking lot-specific factors. Examples of parking space / parking lot factors include that most parking spaces are quadrilaterals, opposite sides of parking spaces are generally equal in length, opposite sides are generally parallel, most are generally rectangular, adjacent parking spaces generally have similar dimensions, adjacent parking spaces are generally parallel, and so on. Further details regarding the extension of optimization module 52 using parking space factors can be found in commonly assigned U.S. Patent Application Serial No. xx / xxx, xxx (Attorney's File No. P110856), entitled "Optimization System for Mapping Parking Spaces Using Synchronous Localization and Mapping," the entire contents of which are incorporated herein by reference.

[0041] In some examples, SLAM module 47 also includes a scaling estimation module 50, which is configured to estimate the scale of measurements on a per-camera and / or per-measurement basis. Further details relating to the scaling estimation module 50 can be found in U.S. Patent Application Serial No. xx / xxx, xxx (Attorney General's File No. P110844), entitled "Incremental Scaling Estimation for Vision-Based Autonomous Driving Systems," the entire contents of which are incorporated herein by reference.

[0042] Vehicle 10 includes an autonomous driving module 56 configured to control actuators 60 to adjust the steering, braking, and / or acceleration of vehicle 10. For example, autonomous driving module 56 can park the vehicle in one of the identified and mapped parking spaces.

[0043] Now for reference Figure 2A Vehicle 10 enters parking lot 70, which includes multiple parking spaces 72. Vehicle 10 generates images using cameras 34-1, 34-2, ..., 34-N. Some images are selected as keyframes. Some of the selected keyframes may include parking spaces identified by features such as center or corner locations. Keyframes include the location of a corner of one or more parking spaces at a given timestamp. In some examples, keyframes are generated by a single camera (e.g., a parking space located to one side of vehicle 10). In some examples, keyframes at a given timestamp may be generated by cameras on both sides of the vehicle, a front camera, and / or a rear camera.

[0044] Some parking spaces 72 are occupied by vehicle 80, and other parking spaces 72 are open. Vehicle 10 can proceed directly to one of the parking spaces 72, such as parking space 72-1 or 72-2, or vehicle 10 can drive around the parking lot 70 before selecting a parking space (such as parking space 72-3).

[0045] Now for reference Figure 2B The example shown includes parking spaces with corners 1, 2, 3, and 4. In some examples, the positions of corners 2 and 3 can be detected (with high confidence and low covariance) because the vehicles are closer to corners 2 and 3, and the positions of corners 1 and 4 can be estimated (with lower confidence and higher covariance).

[0046] Now for reference Figure 3 Cameras 34-1, 34-2, ..., 34-N output images to image analysis module 44. Image analysis module 44 outputs keyframes to keyframe selection module 46, which selects keyframes for further analysis. The selected keyframes are stored in database 90, such as a Structured Query Language (SQL) database, along with odometer / IMU data 92.

[0047] The data association module 48 is configured to determine whether a measurement (e.g., the corner location of a parking space) corresponds to an existing parking space or a new parking space, as will be described further below.

[0048] Now for reference Figure 4At point 110, the data association module 48 performs a measurement between the location of an unassociated measurement (e.g., a parking space not yet associated in the keyframe) and an existing parking space. At point 114, the data association module 48 identifies the association between the measurements. At point 118, the data association module 48 identifies the association between landmarks.

[0049] Now for reference Figures 5A to 5C This shows the measurement of existing parking spaces ( Figure 3 Additional details regarding step 110). Figure 5A In step 210, the data association module 48 focuses on related but unassociated measurements. Related refers to unassociated measurements within a moving window around the vehicle. In other words, the data association module 48 selects images that may be associated with parking spaces but are not yet associated with known parking spaces. At 214, the data association module 48 converts the center of possible parking spaces into a local frame and determines the number of candidates whose centers fall within a predetermined center region 320-1. Candidates falling into regions 320-2 and 320-3 are unacceptable because they imply overlapping parking spaces, which could lead to a collision. Candidates located in the outer region 310-4 can be associated with other parking spaces, such as parking space 320.

[0050] At position 218, data association module 48 determines the number of candidates within region 320-1. If data association module 48 determines at position 218 that the number of candidates is equal to 1, then data association module 48 treats the candidate as an exact association at position 222 and adds the link to the database at position 226. If data association module 48 determines at position 218 that the number of candidates is greater than 1, then data association module 48 treats the candidate as an indefinite association at position 222 and adds the database link at position 226.

[0051] exist Figure 5C The example of step 210 is shown below. At 410, the latest keyframe from the involved state is considered. At 414, all keyframes within the search radius surrounding the latest keyframe are considered. At 418, keyframes marked as deleted are ignored. At 422, unassociated measurements from the remaining keyframes are considered.

[0052] Now for reference Figure 6 The diagram illustrates the steps performed by the data association module 48 when identifying associations between measurements. At 450, the method focuses on related but unassociated measurements. At 454, the method creates a new state. At 458, the method transforms / rotates the measurements onto the map. At 462, the method calculates the distance between the measurements. At 466, the method uses the distance to cluster measurements that are within a predetermined distance from each other.

[0053] At 470, the method selects the first cluster. At 474, the method calculates an initial estimate of the parking spaces. At 478, the method adds the landmark or parking space to the database. At 482, the method links the measurement results to the landmark / parking space in the database. At 486, the method represents the first keyframe in the cluster as a reference. At 490, the method determines whether another cluster exists. If true, the method returns to 474. If false, the method terminates.

[0054] Now for reference Figure 7 The steps for calculating the distance between measurements are as follows: At 510, the distance between features of the parking spaces (such as center or corner) is calculated. At 514, the distance between the corners of the swapped parking spaces (offset by 180°) is calculated. At 524, the maximum corner distance is determined for both regular and swapped orientations. At 530, the minimum distance between the regular and swapped orientations is selected.

[0055] Now for reference Figures 8 to 10 This shows how the association between landmarks is identified (118). Figure 4 The steps executed by the data association module 48 at that time. Figure 8 and Figure 9 In this method, existing associations are retrieved at position 610. Figure 9 The diagram illustrates an example of extracting existing associations. At 710, the method retrieves database links for adjacent parking spaces along with parking space identifiers (ids). At 714, the method identifies missing connections (e.g., right-hand side (RHS) or left-hand side (LHS)).

[0056] Back Figure 8 This method retrieves all candidate parking spaces from the database at position 614 for association. Figure 10 The example shown illustrates retrieving all candidate parking spaces from the database at position 614 for association. Figure 10 In the example, at point 760, parking spaces already associated with the current parking space were excluded. At point 764, parking spaces located outside the area surrounding the current parking space (e.g., windows) were excluded.

[0057] Return to reference Figure 8The method selects one of the candidate parking spaces at 618. At 622, the method determines whether the orientations of adjacent parking spaces match. If 622 is false, the method returns to 618. If 622 is true, the method transforms the candidate to the current parking space coordinates at 626. At 630, the method determines whether the candidate already has neighbors (e.g., in front (F) (for parallel parking), behind (R), left (LHS), and / or right (RHS)). If 630 is true, the method returns to 618. If 630 is false, the method stores the valid candidate as a neighbor association (along with the positions such as F, R, LHS, RHS) at 634. At 638, the method selects the best candidate for adjacent parking spaces based on a predetermined minimum distance.

[0058] At 642, the method determines whether the parking space has a matching adjacent parking space. If 642 is true, the method updates the database connection at 644. If 642 is false, the method does not update the database connection at 646. If it is determined at 650 that another candidate parking space exists, the method returns to 618.

[0059] The foregoing description is illustrative in nature and is in no way intended to limit this disclosure, its application, or use. The broad teachings of this disclosure can be implemented in various forms. Therefore, while this disclosure includes specific examples, its true scope should not be so limited, as other modifications will become apparent upon study of the accompanying drawings, specification, and appended claims. It should be understood that one or more steps in the method may be performed in a different order (or simultaneously) without altering the principles of this disclosure. Furthermore, although each embodiment is described above as having certain features, any one or more of these features described with respect to any embodiment of this disclosure can be implemented and / or combined with features of any other embodiment, even if such combination is not explicitly described. In other words, the described embodiments are not mutually exclusive, and arrangements between one or more embodiments remain within the scope of this disclosure.

[0060] Spatial and functional relationships between components (e.g., between modules, circuit elements, semiconductor layers, etc.) are described using various terms, including “connection,” “joint,” “coupled,” “adjacent,” “next to,” “top,” “above,” “below,” and “set.” Unless explicitly described as “direct,” when describing the relationship between first and second components in the foregoing disclosure, the relationship can be a direct relationship where no other intervening component exists between the first and second components, but it can also be an indirect relationship where one or more intervening components (spatially or functionally) exist between the first and second components. As used herein, the phrase “at least one of A, B, and C” should be interpreted as using the non-exclusive logic OR to represent logic (A OR B OR C), and should not be understood as “at least one of A, at least one of B, and at least one of C.”

[0061] In a diagram, the direction of an arrow (as indicated by the arrow) typically represents the flow of information (such as data or instructions) of interest. For example, when elements A and B exchange various types of information, but the information transmitted from element A to element B is relevant to the diagram, the arrow can point from element A to element B. This unidirectional arrow does not imply that no other information is transmitted from element B to element A. Furthermore, for information sent from element A to element B, element B can send a request for or confirmation of receipt of the information to element A.

[0062] In this application, the terms "module" or "controller" are used in accordance with the following definitions, and may be replaced by the term "circuit". The term "module" may refer to, constitute, or include: application-specific integrated circuit (ASIC); digital, analog, or mixed analog / digital discrete circuit; digital, analog, or mixed analog / digital integrated circuit; combinational logic circuit; field-programmable gate array (FPGA); processor circuitry (shared, dedicated, or grouped) that executes code; memory circuitry (shared, dedicated, or grouped) that stores code executed by the processor circuitry; other suitable hardware components that provide the aforementioned functionality; or a combination of some or all of the above.

[0063] This module may include one or more interface circuits. In some examples, the interface circuits may include wired or wireless interfaces connected to a local area network (LAN), the Internet, a wide area network (WAN), or a combination thereof. The functionality of any given module in this disclosure can be distributed among multiple modules connected via the interface circuits. For example, multiple modules can allow for load balancing. In another example, a server (also known as a remote or cloud) module may perform some functions on behalf of a client module.

[0064] As used above, the term "code" can include software, firmware, and / or microcode, and can refer to programs, routines, functions, classes, data structures, and / or objects. The term "shared processor circuitry" includes a single processor circuitry that executes some or all of the code from multiple modules. The term "group processor circuitry" includes processor circuitry that executes some or all of the code from one or more modules in conjunction with additional processor circuitry. References to multiprocessor circuitry include multiple processor circuitry on a discrete die, multiple processor circuitry on a single die, multiple cores of a single processor circuit, multiple threads of a single processor circuit, or combinations thereof. The term "shared memory circuitry" includes a single memory circuitry that stores some or all of the code from multiple modules. The term "group memory circuitry" includes memory circuitry that stores some or all of the code from one or more modules in conjunction with additional memory.

[0065] The term memory circuit is a subset of the term computer-readable medium. As used herein, the term computer-readable medium does not include transient electrical or electromagnetic signals propagating through a medium (e.g., on a carrier wave); therefore, the term computer-readable medium can be considered tangible and non-transient. Non-limiting examples of non-transient tangible computer-readable media are non-volatile memory circuits (such as flash memory circuits, erasable programmable read-only memory circuits, or mask read-only memory circuits), volatile memory circuits (such as static random access memory circuits or dynamic random access memory circuits), magnetic storage media (such as analog or digital magnetic tape or hard disk drives), and optical storage media (such as CDs, DVDs, or Blu-ray discs).

[0066] The apparatus and methods described in this application can be partially or fully implemented by a special-purpose computer created by configuring a general-purpose computer to execute one or more specific functions embodied in a computer program. The aforementioned function blocks, flowchart components, and other elements serve as software specifications that can be converted into computer programs through the routine work of skilled technicians or programmers.

[0067] A computer program includes processor-executable instructions stored on at least one non-transitory tangible computer-readable medium. A computer program may also include or depend on stored data. A computer program may include a basic input / output system (BIOS) for interacting with dedicated computer hardware, device drivers for interacting with specific devices of the dedicated computer, one or more operating systems, user applications, background services, background applications, etc.

[0068] Computer programs may include: (i) descriptive text to be parsed, such as HTML (Hypertext Markup Language), XML (Extensible Markup Language), or JSON (JavaScript Object Notation); (ii) assembly code; (iii) object code generated by the compiler from the source code; (iv) source code executed by the interpreter; (v) source code compiled and executed by a just-in-time compiler, etc. As an example only, source code can be from C, C++, C#, Objective-C, Swift, Haskell, Go, SQL, R, Lisp, etc. Fortran, Perl, Pascal, Curl, OCaml, HTML5 (Hypertext Markup Language 5th Edition), Ada, ASP (Active Server Pages), PHP (PHP: Hypertext Preprocessor), Scala, Eiffel, Smalltalk, Erlang, Ruby, Visual Lua, MATLAB, SIMULINK and The syntax of languages ​​such as [list of languages].

Claims

1. A vision-based perception system for a vehicle, comprising: N cameras are deployed on the vehicle and configured to generate multiple images, where N is an integer greater than 1; An image analysis module is configured to identify multiple features of parking spaces in the multiple images and generate multiple keyframe measurements based thereon. The keyframe selection module is configured to identify a selected keyframe among the plurality of keyframe measurements; A database configured to receive selected keyframe measurements from the plurality of keyframe measurements and selectively output unrelated keyframe measurements within a predetermined range of the vehicle and existing features of the parking space; and The data association module is configured to receive unassociated keyframe measurements and existing features of parking spaces from the database.

2. The vision-based perception system of claim 1, wherein at least two of the unassociated keyframe measurements and the existing features of the parking space are asynchronous measurements.

3. The vision-based perception system of claim 2, wherein the data association module is configured to selectively associate selected keyframe measurements in the keyframe measurements with existing features of the parking space, and associate other selected keyframe measurements in the keyframe measurements with new features of the parking space.

4. The vision-based perception system according to claim 1, wherein the data association module is configured as follows: Generate measurements between unassociated keyframe measurements and existing features of the parking space; Identify correlations between measurements; and Identify the connections between landmarks.

5. The vision-based perception system of claim 4, wherein the data association module is configured to generate a measurement between the unassociated keyframe measurement and the existing features of the parking space by converting the features of the parking space measured by the unassociated keyframe into a local frame and calculating the distance to the existing features of the parking space.

6. The vision-based perception system of claim 5, wherein when the distance is less than a predetermined distance, the data association module is configured to define an exact association between a corresponding unassociated keyframe measurement and a corresponding existing feature of the parking space.

7. The vision-based perception system of claim 4, wherein the data association module is configured to identify associations between measurements in the following manner: Calculate the distance between unrelated keyframe measurements; and Unassociated keyframe measurements are aggregated into one or more clusters based on the distance.

8. The vision-based perception system of claim 7, wherein the data association module is configured to identify associations between measurements in the following manner: Select one of the one or more clusters; Calculate an initial estimate of the parking spaces corresponding to one of the one or more clusters; Add parking spaces to the database; as well as Repeat the calculation and addition for the remaining clusters in one or more of the said clusters.

9. The vision-based perception system of claim 4, wherein the data association module is configured to identify associations between landmarks in the following manner: Retrieve existing neighbor associations; Retrieve relevant parking spaces from the database; Select a candidate parking space; and Determine whether the orientation of adjacent parking spaces matches that of candidate parking spaces.

10. The vision-based perception system of claim 9, wherein the data association module is configured to identify associations between landmarks in the following manner: If a candidate parking space does not already have an adjacent parking space in the predetermined neighbor locations, the best candidate is selected as the neighbor from two or more candidates. Update the related links in the database; as well as Repeat the selection and update process for other candidate parking spaces.

Images