Incremental scale estimation for vision-based autonomous driving systems

Abstract

The present disclosure relates to incremental scale estimation for vision-based autonomous driving systems. A vision-based perception system for a vehicle includes N cameras arranged on the vehicle and configured to generate a plurality of images. An image analysis module is configured to identify a plurality of parking space features in the plurality of images and to generate a plurality of keyframe measurements based thereon. A keyframe selection module is configured to identify selected keyframe measurements of the plurality of keyframe measurements. A simultaneous localization and mapping (SLAM) module is configured to generate SLAM state values and store them in a database, and to selectively output unassociated keyframe measurements and existing features of parking spaces within a predetermined range of the vehicle. A per-camera scaler is configured to receive the selected keyframe measurements of the keyframe measurements from the keyframe selection module, apply a per-camera scale to the selected keyframe measurements of the keyframe measurements, and output the scaled keyframe measurements to the database.

Description

Technical Field

[0001] The information provided in this section is for the purpose of generally setting out the context of this disclosure. The presently listed inventors’ work and aspects described in this section, which may not otherwise conform to the prior art at the time of filing, are neither an explicit nor an implicit admission that they are prior art to this disclosure.

[0002] This disclosure relates to vehicles including autonomous driving systems, and more particularly to scaling estimators for vision-based autonomous driving systems. Background Technology

[0003] The vehicle may include an autonomous driving system that operates the vehicle without driver input. The autonomous driving system receives images from multiple cameras pointing in different directions. For example, the vehicle may include one or more forward-facing cameras, one or more side-facing cameras, and one or more rear-facing cameras. The autonomous driving system needs to be able to identify objects in the images and the positions of said objects relative to the moving vehicle within the images. Summary of the Invention

[0004] A vision-based perception system for a vehicle includes N cameras deployed on the vehicle and configured to generate multiple images, where N is an integer greater than one. An image analysis module is configured to identify multiple parking space features in the multiple images and generate multiple keyframe measurements based on these features. A keyframe selection module is configured to identify selected keyframe measurements from the multiple keyframe measurements. A Simultaneous Localization and Measurement (SLAM) module is configured to generate SLAM state values ​​and store them in a database, and selectively output unassociated keyframe measurements and existing features of parking spaces within a predetermined range of the vehicle. A per-camera zoomer is configured to receive selected keyframe measurements from the keyframe selection module, apply per-camera zoom to the selected keyframe measurements, and output the zoomed keyframe measurements to the database.

[0005] Among other features, the data association module is configured to receive unassociated measurements and existing states within a predetermined distance from the vehicle from a database. The data association module is also configured to selectively associate selected keyframe measurements from the keyframe measurements with the existing state and associate other selected keyframe measurements with a new state.

[0006] Among other features, the scaling estimation module includes a per-measurement scaling calculator configured to calculate the per-measurement scaling of keyframe measurements for the current state and the new state. The keyframe measurements for at least one of the current and new states include n rays, each ray having a start position at a corresponding camera among the N cameras, an end position at a corresponding parking space feature among the N cameras, and a first distance between the start and end positions, where n is an integer greater than two.

[0007] Among the other features, the per-measurement scaling calculator is configured to calculate the estimated position L of one of the parking space features by averaging the end positions of the n rays. The per-measurement scaling calculator is configured to use a least mean square function to determine the n intermediate positions of the estimated position L of one of the parking space features on the n rays that are closest to the parking space features.

[0008] Among other features, the per-measurement scaling calculator is configured to calculate the scaling of each measurement of the n rays based on a ratio of a first distance from the start position of each of the n rays to the corresponding intermediate position among the n intermediate positions, divided by a second distance from the start position of each of the n rays to the end position of each of the n rays. The scaling estimation module is configured to calculate a confidence level for each of the n rays. The scaling estimation module includes a per-camera scaling calculator configured to update the scaling of each camera in response to the scaling of each measurement of the n rays.

[0009] Among other features, the scaling database is configured to store the scaling of each measurement of n rays and the scaling of each camera updated by the scaling of each measurement of n rays.

[0010] A vision-based perception system for a vehicle includes an image analysis module configured to identify multiple parking space features in multiple images and generate multiple keyframe measurements based on these features. A keyframe selection module is configured to identify selected keyframe measurements from the multiple keyframe measurements. A Simultaneous Localization and Measurement (SLAM) module is configured to generate SLAM state values ​​and store them in a database, and selectively output unassociated keyframe measurements and existing parking space features within a predetermined range of the vehicle. A per-camera zoomer is configured to receive selected keyframe measurements from the keyframe selection module, apply per-camera zoom to the selected keyframe measurements, and output the zoomed keyframe measurements to the database. A data association module is configured to receive unassociated measurements and existing states within a predetermined distance of the vehicle from the database. The data association module is configured to selectively associate selected keyframe measurements with the existing state and associate other keyframe measurements with a new state. A zoom estimation module includes a per-measurement zoom calculator configured to calculate the per-measurement zoom of the keyframe measurements for the existing state and the keyframe measurements for the new state.

[0011] Among other features, keyframe measurements of at least one of the existing state and the new state include n rays, each ray having a start position at a corresponding camera among N cameras of the vehicle, an end position at a corresponding parking space feature in the parking space features, and a first distance between the start position and the end position, where N is an integer greater than one and n is an integer greater than two. A per-measurement scaling calculator is configured to calculate an estimated position L of one parking space feature among the parking space features by averaging the end positions of the n rays. A per-measurement scaling calculator is configured to use a least mean square function to determine n intermediate positions on the n rays that are closest to the estimated position L of one parking space feature among the parking space features. A per-measurement scaling calculator is configured to calculate the scaling of each measurement of the n rays based on the ratio of the first distance from the start position of each of the n rays to the corresponding intermediate position among the n intermediate positions to the second distance from the start position of each of the n rays to the end position of each of the n rays.

[0012] Among other features, the scaling estimation module is configured to calculate the confidence level for each of the n rays. The scaling estimation module includes a per-camera scaling calculator configured to update the scaling of each camera in response to the scaling of each measurement of the n rays.

[0013] Among other features, the scaling database is configured to store the scaling of each measurement of n rays and the scaling of each camera updated by the scaling of each measurement of n rays.

[0014] A scaling estimation module for a vision-based perception system for a vehicle includes a per-measurement scaling calculator configured to calculate per-measurement scaling for keyframe measurements. The keyframe measurements include n rays, each ray having a start position at one of N cameras deployed on the vehicle, an end position at a parking space feature, and a first distance between the start and end positions, where N is an integer greater than one and n is an integer greater than two. The per-measurement scaling calculator is configured to: calculate an estimated position L of the parking space feature by averaging the end positions of the n rays; determine n intermediate positions on the n rays that are closest to the estimated position L of the parking space feature using a least mean square function; and calculate the per-measurement scaling of the n rays based on the ratio of the first distance from the start position of each of the n rays to the corresponding intermediate position among the n intermediate positions to the second distance from the start position of each of the n rays to the end position of each of the n rays.

[0015] Other applicable areas of this disclosure will become apparent from the detailed description, claims, and drawings. The detailed description and specific examples are intended for illustrative purposes only and are not intended to limit the scope of this disclosure. Attached Figure Description

[0016] This disclosure will be more fully understood through detailed description and the accompanying drawings, in which:

[0017] Figure 1 This is a functional block diagram of an example of a vehicle, including a vision-based perception system, in accordance with this disclosure;

[0018] Figure 2A It is a floor plan of an example of a vehicle passing through a parking lot with parking spaces and parking in one of the parking spaces, in accordance with this disclosure;

[0019] Figure 2B The illustration shows an example of a vehicle in a parking space according to this disclosure;

[0020] Figure 3 This is a flowchart illustrating an example of a vision-based object detection method performed without scaling estimation.

[0021] Figure 4 This is a flowchart illustrating an example of a vision-based object detection method employing scaling estimation according to this disclosure;

[0022] Figure 5 The illustration shows an example of rays representing the same landmark according to this disclosure;

[0023] Figure 6This is a flowchart of an example scaling estimation system according to this disclosure.

[0024] In the accompanying drawings, reference numerals may be used repeatedly to identify similar and / or identical elements. Detailed Implementation

[0025] Although the scaling estimation system for vision-based perception systems is described below in the context of autonomous vehicles, scaling estimation can be used in other types of vision-based systems.

[0026] The vehicle may include an autonomous driving system that operates the vehicle without driver input. The autonomous driving system receives images from multiple cameras pointing in two or more different directions. For example, the vehicle may include one or more forward-facing cameras, one or more side-facing cameras, and one or more rear-facing cameras. The vision-based autonomous driving system identifies objects in the images and their relative positions within the images.

[0027] Current vision-based systems assume a fixed scaling for each camera or situation. Using fixed scaling introduces inaccuracies and poor adaptation in dynamic environments. Scaling is not directly observable for monocular SLAM. Therefore, scaling is estimated, leading to potential inconsistencies and ambiguities. Using incorrect scaling distorts trajectories, maps, and / or localization. The assumption of fixed scaling limits performance in dynamic environments. Scaling errors can also degrade the reliability and loop closure of Simultaneous Localization and Mapping (SLAM) (e.g., identifying previously visited locations and updating beliefs accordingly).

[0028] The vision-based system disclosed herein dynamically estimates the scaling of keyframe measurements. More specifically, the scaling estimator performs online incremental per-measurement scaling estimation for vision-based simultaneous localization and mapping (VSLAM) systems. The scaling estimator resolves scaling ambiguities without fixed assumptions about the basis of per-camera or keyframe measurements.

[0029] Using factor graph-based MAP estimation, the scaling estimator dynamically optimizes the scaling of each estimated measurement, ensuring adaptability and robustness. The scaling estimator is decoupled from SLAM state estimation to enhance stability, while a per-camera initializer improves convergence and data correlation. In some examples, the SLAM state corresponds to vehicle and / or camera positions and parking space features such as corners or centers. The scaling estimator supports multiple camera and sensor setups. In some examples, the scaling estimator uses one or more scaling uncertainty estimation methods (to generate confidence levels) and achieves real-time performance.

[0030] Now refer to Figure 1The vehicle 10 includes various sensors, such as a Global Positioning System (GPS) 14, an Inertial Measurement Unit (IMU) 18, and wheel speed sensors 20. The vehicle 10 includes one or more cameras 34-1, 34-2, ..., and 34-N that generate images relative to the front, side, and / or rear of the vehicle 10, where N is an integer greater than one. In some examples, cameras 34-1, 34-2, ..., and 34-N include one or more forward-facing cameras, one or more side-facing cameras, and one or more rear-facing cameras.

[0031] The controller 42 includes an image analysis module 44 configured to receive images from cameras 34-1, 34-2, ..., and 34-N, identify landmarks in the images (e.g., corners of parking spaces), and generate measurements of the landmarks in the images from the corresponding cameras. In some examples, the image analysis module 44 may include a neural network configured to identify various objects such as corners of parking spaces.

[0032] The keyframe selection module 46 selects keyframes (each with a timestamp) from the images. Cameras 34-1, 34-2, ..., and 34-N generate a large number of samples, and not all samples provide new or useful information. For example, measurements may or may not change significantly in subsequent keyframes during sequential timestamps when the vehicle is stationary or moving slowly. Not all images include parking spaces. The keyframe selection module 46 is configured to select some or all of the keyframes based on the values ​​of the data.

[0033] Simultaneous Localization and Mapping (SLAM) module 47 is configured to perform simultaneous localization and mapping. In some examples, SLAM module 47 includes the Georgia Tech Smoothing and Mapping (GTSAM) or g2o module, although other SLAM modules can be used.

[0034] In some examples, optimization module 52 is configured to manage and optimize SLAM states (e.g., vehicle locations and parking space features). In some examples, SLAM states correspond to vehicle and / or camera locations and parking space features such as the corner or center of a parking space. In some examples, optimization module 52 includes a cost function extended with parking space / parking lot-specific factors. Examples of parking space / parking lot factors include that most parking spaces are quadrilaterals, opposite sides of parking spaces are typically equal in length, opposite sides are generally parallel, most are generally rectangular, adjacent parking spaces generally have similar dimensions, adjacent parking spaces are generally cooperatively parallel, and so on. Additional details relating to the extension of optimization module 52 with parking space factors can be found in commonly assigned U.S. Patent Application Serial No. 19 / 195048 (Attorney Docket No. P110856, entitled “Optimization System for Mapping Parking Slots Using Simultaneous Localization and Mapping”), which is incorporated herein by reference in its entirety.

[0035] Controller 42 further includes a scaling estimation module 56 configured to estimate the scaling of a measurement on a per-camera and / or per-measurement basis, as will be further described below.

[0036] The controller 42 includes an autonomous driving module 58 configured to control actuators 60 to adjust the steering, braking, and / or acceleration of the vehicle 10. For example, the autonomous driving module 58 can park the vehicle in a parking space identified and mapped by the SLAM module 47.

[0037] Now refer to Figure 2A Vehicle 10 enters a parking lot 70 comprising multiple parking spaces 72. Vehicle 10 uses cameras 34-1, 34-2, ..., and 34-N to generate images. Some of these images are selected as keyframes. Some of the selected keyframes may include parking spaces identified by center and / or corner locations. Keyframes include locations at some or all corners of one or more parking spaces at a given timestamp. In some examples, keyframes are generated by a single camera (e.g., a parking space located on one side of vehicle 10). In some examples, keyframes at a given timestamp can be generated by cameras on both sides of the vehicle, a front camera, and / or a rear camera.

[0038] Some parking spaces in parking space 72 are occupied by vehicle 80, and other parking spaces in parking space 72 are empty. Vehicle 10 can proceed directly to a parking space in parking space 72, such as parking space 72-1 or parking space 72-2, or vehicle 10 can drive around parking space 70 before selecting a parking space, such as parking space 72-3.

[0039] Now refer to Figure 2B The diagram shows examples of parking spaces including corners 1, 2, 3, and 4. In some examples, the locations of corners 2 and 3 can be detected (with high confidence and low covariance) because the vehicles are closer to them, while corners 1 and 4 can be estimated (with lower confidence and higher covariance).

[0040] Now refer to Figure 3 Previous methods for identifying parking spaces without scaling estimates included receiving new measurements at 110. At 114, the method performed data correlation. At 118, the method performed optimization.

[0041] Figure 4 The present invention illustrates a method for identifying parking spaces, including scaling estimation, according to this disclosure. The method includes receiving a new measurement at 120. In some examples, the new measurement or keyframe includes landmarks such as the center or corner of the parking space. At 124, per-camera scaling is applied. At 128, the method includes performing data association. At 132, per-measurement scaling is applied. At 136, the method includes using a SLAM module to perform optimization.

[0042] Now refer to Figure 5 The input measurement corresponds to the measurement from p i Extending to P i The ray (where i is in the range from 0 to n and n>2). The ray may have incorrect scaling. In some examples, the p-th of the ray... i This corresponds to the position of the camera that captured the image. In some examples, the P of the ray... i This corresponds to the location of the landmark L, which may have incorrect scaling. The scaling estimation system transforms the ray into a vector (a ray with corrected scaling). In some examples, the ray is generated by the same camera at different times (asynchronously) or by different cameras at the same or different times.

[0043] After association, three or more rays (e.g., rays p0 to P0, p1 to P1, and p2 to P2) form a set representing the same landmark L. In some examples, an initial guess about the location of landmark L is made. For example, the initial guess may include averaging the locations P0, P1, and P2 of the rays (with incorrect scaling or with per-camera scaling applied), although other methods can be used. The scaling suitable for each measurement is then estimated using triangulation. For example, the least mean square function can be used to identify the nearest points P'0, P'1, and P'2 on the rays (p0 to P0, p1 to P1, and p2 to P2) to landmark L. The scaling is then calculated by calculating the ratios P'0 / P0, P'1 / P1, and P'2 / P2. The scaling estimate is decoupled from the SLAM state estimate to better control system robustness.

[0044] For example, the minimum mean square function corresponds to Where L is the location of a landmark that is unknown priori, and P′ i `i` represents the closest point to `L` in the `i`-th row, and `n` is the number of measurements (e.g., greater than or equal to three to perform triangulation). The scaling measurements may also include a covariance matrix that can be used to weight the estimated scaling.

[0045] In some examples, one or more additional calculations may be performed to generate confidence levels using the Gaussian noise assumption. Examples of additional calculations include Var(S), which is the estimated scaling variance (e.g., with the lower bound (left side) and upper bound (right side) of the Cauchy-Schwartz inequality):

[0046]

[0047] in, Var(L) is the variance of the measurement, and Var(L) is the variance of the landmark. It is the standard deviation of the measurement. It is the average of the measurements, σ L It is the standard deviation of the landmark location, and μ L It is the average of the landmark locations.

[0048] Now refer to Figure 6 The new keyframe measurement is output at 210 to the per-camera scaler 214 of the scaling estimation module 50. The per-camera scaler 214 uses the per-camera estimated scaling value to scale the new measurement, and the per-camera scaling measurement is stored in the database 218 associated with the SLAM module 47.

[0049] Unassociated keyframe measurements (not associated with a parking space) and existing states (e.g., known parking spaces) are output to data association module 45. Data association module 45 associates the measurements with the existing state at 226 and with the new state at 232. Examples of data association can be found in commonly assigned U.S. Patent Application No. 19 / 194253 (Attorney Docket No. P110843, entitled "A-Synchronous Association for Parking Slot Measurements").

[0050] The output of the data association module 45 corresponding to the new keyframe measurement of the existing state or the new keyframe measurement of the new state is output to the per-measurement scaling calculator 228. The per-measurement scaling calculator 228 calculates the estimated scaling for each measurement (as described above). Figure 4 and Figure 5 (as described in the document), and stores the estimated scaling values ​​for each measurement along with the corresponding measurement and / or state in a scaling database 250. In some examples, a per-camera scaling calculator 240 updates the estimated per-camera scaling based on the most recently estimated per-measurement scaling.

[0051] In one example, the unscaled original measurement is approximately 20% larger than the actual ground view. After scaling, the scaling error is approximately 1%. In one example involving a parking space location, the original measurement ranges from 0.31 meters (m) to 0.46 meters. The scaling error per camera ranges from 0.14 meters to 0.23 meters. The scaling error per measurement ranges from 0.1 meters to 0.21 meters. As can be appreciated, scaling per measurement shows an improvement in accuracy ranging from 20% to 50% compared to the unscaled solution.

[0052] The foregoing description is merely illustrative and is by no means intended to limit this disclosure, its application, or its use. The broad teachings of this disclosure can be implemented in a wide variety of forms. Therefore, while this disclosure includes specific examples, its true scope should not be limited thereto, as other modifications will become apparent upon examination of the drawings, specification, and the following claims. It should be understood that one or more steps within the method may be performed in different orders (or concurrently) without altering the principles of this disclosure. Furthermore, while each of the embodiments is described above as having certain features, one or more of those features described for any embodiment of this disclosure can be implemented and / or combined with those features in any embodiment of other embodiments, even if such a combination is not explicitly described. In other words, the described embodiments are not mutually exclusive, and substitutions of one or more embodiments remain within the scope of this disclosure.

[0053] The spatial and functional relationships between elements (e.g., between modules, circuit elements, semiconductor layers, etc.) are described using a variety of terms including “connected,” “joined,” “coupled,” “adjacent,” “closely adjacent,” “above,” “under,” and “set.” Unless explicitly described as “direct,” when the relationship between the first and second elements is described in the above disclosure, that relationship can be a direct relationship where no other intermediate elements are present between the first and second elements, but it can also be an indirect relationship where one or more intermediate elements (spatially or functionally) are present between the first and second elements. As used herein, the phrase “at least one of A, B, and C” should be understood to mean using non-exclusive logic OR (or) to represent the logic (A OR B OR C), and should not be understood to mean “at least one of A, at least one of B, and at least one of C.”

[0054] In the accompanying drawings, the direction of the arrows generally illustrates the flow of information (such as data or instructions) of interest. For example, when components A and B exchange a variety of information, but the information transmitted from component A to component B is relevant to the illustration, the arrow may point from component A to component B. This unidirectional arrow does not imply that no other information is transmitted from component B to component A. Furthermore, for information sent from component A to component B, component B may send a request for said information or an acknowledgment of receipt of said information to component A.

[0055] In this application, which includes the following definitions, the term "module" or "controller" may be replaced by the term "circuit". The term "module" may refer to, be part of, or include the following: application-specific integrated circuit (ASIC); digital, analog, or mixed-signal analog / digital discrete circuit; digital, analog, or mixed-signal analog / digital integrated circuit; combinational logic circuit; field-programmable gate array (FPGA); processor circuitry (shared, dedicated, or group) that executes code; memory circuitry (shared, dedicated, or group) that stores code executed by said processor circuitry; other suitable hardware components that provide the described functionality; or combinations of some or all of the above, such as in a system-on-a-chip.

[0056] The module may include one or more interface circuits. In some examples, the interface circuits may include a wired or wireless interface connected to a local area network (LAN), the Internet, a wide area network (WAN), or a combination thereof. The functionality of any given module of this disclosure may be distributed among multiple modules connected via the interface circuits. For example, multiple modules may allow for load balancing. In another example, a server (also known as a remote or cloud) module may perform a certain function on behalf of a client module.

[0057] As used above, the term "code" can include software, firmware, and / or microcode, and can refer to programs, routines, functions, classes, data structures, and / or objects. The term "shared processor circuitry" includes a single processor circuitry that executes some or all of the code from multiple modules. The term "group processor circuitry" includes processor circuitry that executes some or all of the code from one or more modules in combination with additional processor circuitry. References to multiple processor circuitry include multiple processor circuitry on a discrete die, multiple processor circuitry on a single die, multiple cores of a single processor circuitry, multiple threads of a single processor circuitry, or a combination thereof. The term "shared memory circuitry" includes a single memory circuitry that stores some or all of the code from multiple modules. The term "group memory circuitry" includes memory circuitry that stores some or all of the code from one or more modules in combination with additional memory circuitry.

[0058] The term "memory circuit" is a subset of the term "computer-readable medium." As used herein, the term "computer-readable medium" does not include transient electrical or electromagnetic signals propagating through a medium (such as on a carrier wave); the term "computer-readable medium" can therefore be considered tangible and non-transient. Non-limiting examples of non-transient tangible computer-readable media are non-volatile memory circuits (such as flash memory circuits, erasable programmable read-only memory circuits, or mask read-only memory circuits), volatile memory circuits (such as static random access memory circuits or dynamic random access memory circuits), magnetic storage media (such as analog or digital magnetic tape or hard disk drives), and optical storage media (such as CDs, DVDs, or Blu-ray discs).

[0059] The apparatus and methods described in this application can be implemented, partially or entirely, by a dedicated computer created by configuring a general-purpose computer to perform one or more specific functions implemented in a computer program. The function blocks, flowchart components, and other elements described above serve as software specifications that can be converted into computer programs through the routine work of skilled technicians or programmers.

[0060] A computer program includes processor-executable instructions stored on at least one non-transitory tangible computer-readable medium. The computer program may also include or rely on stored data. The computer program may contain a basic input / output system (BIOS) for interacting with the hardware of a special-purpose computer, device drivers for interacting with specific devices of the special-purpose computer, one or more operating systems, user applications, background services, background applications, etc.

[0061] Computer programs may include: (i) descriptive text to be parsed, such as HTML (Hypertext Markup Language), XML (Extensible Markup Language), or JSON (JavaScript Object Markup), (ii) assembly code, (iii) object code generated by a compiler from source code, (iv) source code for execution by an interpreter, (v) source code for compilation and execution by a just-in-time (JIT) compiler, and so on. As an example only, source code may be from languages ​​including C, C++, C#, Objective-C, Swift, Haskell, Go, SQL, R, Lisp, etc. Fortran, Perl, Pascal, Curl, OCaml, HTML5 (Hypertext Markup Language, 5th Revision), Ada, ASP (Active Server Pages), PHP (PHP: Hypertext Preprocessor), Scala, Eiffel, Smalltalk, Erlang, Ruby, Visual Lua, MATLAB, SIMULINK and The syntax of the language was written.

Claims

1. A vision-based perception system for vehicles, comprising: N cameras are deployed on the vehicle and configured to generate multiple images, where N is an integer greater than one; The image analysis module is configured to identify multiple parking space features in the multiple images and generate multiple keyframe measurements based on these features; A keyframe selection module is configured to identify the selected keyframe measurement among the plurality of keyframe measurements; Simultaneous Localization and Measurement (SLAM) module, configured to generate SLAM state values ​​and store them in a database, and selectively output unassociated keyframe measurements and existing features of parking spaces within a predetermined range of the vehicle; as well as Each camera scaler is configured to receive selected keyframe measurements from the keyframe measurements from the keyframe selection module, apply per-camera scaling to the selected keyframe measurements from the keyframe measurements, and output the scaled keyframe measurements to the database.

2. The vision-based perception system of claim 1, further comprising a data association module configured to receive unassociated measurements and current status within a predetermined distance from the vehicle from the database.

3. The vision-based perception system as described in claim 2, wherein, The data association module is configured to selectively associate selected keyframe measurements with an existing state and associate other selected keyframe measurements with a new state.

4. The vision-based perception system of claim 3, further comprising a scaling estimation module, the scaling estimation module including a per-measurement scaling calculator, the per-measurement scaling calculator being configured to calculate the per-measurement scaling of the keyframe measurements in the current state and the keyframe measurements in the new state.

5. The vision-based perception system as described in claim 4, wherein, The keyframe measurement of at least one of the existing state and the new state includes n rays, each ray having a start position at a corresponding camera among the N cameras, an end position at a corresponding parking space feature among the parking space features, and a first distance between the start position and the end position, where n is an integer greater than two.

6. The vision-based perception system as described in claim 5, wherein, The per-measurement scaling calculator is configured to calculate the estimated position L of one of the parking space features by averaging the end positions of the n rays.

7. The vision-based perception system as described in claim 6, wherein, The per-measurement scaling calculator is configured to use a minimum mean square function to determine the n intermediate positions of the estimated position L on the n rays that is closest to one of the parking space features.

8. The vision-based perception system as described in claim 7, wherein, The per-measurement scaling calculator is configured to calculate the scaling of the n rays per measurement based on a ratio of a first distance from the start position of each of the n rays to the corresponding intermediate position among the n intermediate positions to a second distance from the start position of each of the n rays to the end position of each of the n rays.

9. The vision-based perception system as described in claim 8, wherein, The scaling estimation module is configured to calculate the confidence level for each of the n rays.

10. The vision-based perception system as described in claim 9, wherein, The scaling estimation module includes a per-camera scaling calculator configured to update the scaling of each camera in response to the scaling of each measurement of the n rays.

Images