Optimized system for mapping parking spaces using simultaneous localization and mapping
Patent Information
- Application Number
- CN202511388402.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2025-04-30
- Filing Date
- 2025-09-26
- Publication Date
- 2026-10-30
Smart Images
Figure CN123016387A_ABST
Abstract
Description
[0001] introduction The information provided in this section is for the purpose of presenting the overall context of this disclosure. To the extent described in this section, the work of the currently named inventors and aspects that may not otherwise conform to the description as prior art at the time of submission are neither explicitly nor implicitly considered as prior art contrary to this disclosure.
[0002] This disclosure relates to vision-based perception systems, and more specifically to vision-based perception systems that combine simultaneous localization and mapping.
[0003] The vehicle may include an autonomous driving system that operates without driver input. The autonomous driving system receives images from multiple cameras pointing in different directions. For example, the vehicle may include one or more forward-facing cameras, one or more side cameras, and one or more rear-facing cameras. The autonomous driving system needs to be able to sense the location of parking spaces in a parking lot. The autonomous driving system may not know the location of parking spaces and / or whether each parking space is currently occupied before arriving at the parking lot. The autonomous driving system uses images generated by the cameras to determine the location of available parking spaces as it drives through the parking lot, and then parks the vehicle in one of the available parking spaces. Summary of the Invention
[0004] A vision-based perception system for a vehicle includes N cameras arranged on the vehicle and configured to generate multiple images, where N is an integer greater than 1. An image analysis module is configured to identify multiple features of parking spaces in the multiple images and generate multiple keyframe measurements based thereon. A keyframe selection module is configured to identify selected keyframe measurements from the multiple keyframe measurements. A database is configured to receive the selected keyframe measurements from the multiple keyframe measurements and selectively output irrelevant keyframe measurements and existing features of parking spaces within a predetermined range of the vehicle. A data association module is configured to receive irrelevant keyframe measurements and existing features of parking spaces from the database; and selectively associate the selected keyframe measurements with existing features of parking spaces, and associate other selected keyframe measurements with new features of parking spaces. An optimization module is configured to optimize the location of parking spaces in a local map using a Simultaneous Localization and Mapping (SLAM) cost function extended by at least one parking lot / parking space factor.
[0005] Among other features, at least two of the unrelated keyframe measurements and the existing features of the parking space are asynchronous measurements. The data association module is configured to generate measurements between the unrelated keyframe measurements and the existing features of the parking space; identify associations between measurements; and identify associations between landmarks.
[0006] Among other features, the at least one parking lot / parking space factor includes a quadrilateral factor that evaluates a keyframe measurement of a selected parking space relative to a quadrilateral parking space. The at least one parking lot / parking space factor includes a rectangle factor that evaluates a keyframe measurement of a selected parking space relative to a rectangle. The at least one parking lot / parking space factor includes a line length factor that evaluates the equality of the lengths of opposite sides of a selected parking space. The at least one parking lot / parking space factor includes a parallel line factor that evaluates the parallelism of opposite sides of a selected parking space. The at least one parking lot / parking space factor includes a neighbor size factor that evaluates the size of a selected parking space relative to the size of a neighboring parking space. The at least one parking lot / parking space factor includes a neighbor co-parallel factor that evaluates the parallelism of a selected parking space relative to the sides of a neighboring parking space.
[0007] Among other features, the at least one parking lot / parking space factor includes a quadrilateral factor that evaluates a keyframe measurement of a selected parking space relative to a quadrilateral parking space. The at least one parking lot / parking space factor also includes two or more parking lot / parking space factors selected from the group consisting of: a line length factor evaluating the equality of the lengths of opposite sides of the selected parking space; a parallel line factor evaluating the parallelism of opposite sides of the selected parking space; a rectangle factor evaluating the keyframe measurement of the selected parking space relative to a rectangle; a neighbor size factor evaluating the size of the selected parking space relative to neighboring parking spaces; and a neighbor co-parallelism factor evaluating the parallelism of the sides of the selected parking space relative to neighboring parking spaces.
[0008] A vision-based perception system for a vehicle includes N cameras arranged on the vehicle and configured to generate multiple images, where N is an integer greater than 1. An image analysis module is configured to identify multiple features of parking spaces in the multiple images and generate multiple keyframe measurements based thereon. A keyframe selection module is configured to identify selected keyframe measurements from the multiple keyframe measurements. A database is configured to receive the selected keyframe measurements from the multiple keyframe measurements and selectively output irrelevant keyframe measurements and existing features of parking spaces within a predetermined range of the vehicle. A data association module is configured to receive irrelevant keyframe measurements and existing features of parking spaces from the database; and selectively associate the selected measurements in the keyframe measurements with existing features of parking spaces, and associate other selected measurements in the keyframe measurements with new features of parking spaces. An optimization module is configured to optimize the location of parking spaces using a Simultaneous Localization and Mapping (SLAM) cost function that includes at least one parking lot / parking space factor. The at least one parking lot / parking space factor includes a quadrilateral factor that evaluates the keyframe measurements of selected parking spaces relative to a quadrilateral. At least two of the unrelated keyframe measurements and the existing features of the parking space are measured asynchronously.
[0009] Among other features, the data association module is configured to generate measurements between unrelated keyframe measurements and existing features of the parking space; identify associations between measurements; and identify associations between landmarks.
[0010] Among other features, the at least one parking lot / parking space factor includes a rectangle factor that evaluates a keyframe measurement of the selected parking space relative to a rectangle. The at least one parking lot / parking space factor includes a line length factor that evaluates the equality of lengths of opposite sides of the selected parking space. The at least one parking lot / parking space factor includes a parallel line factor that evaluates the parallelism of opposite sides of the selected parking space. The at least one parking lot / parking space factor includes a neighbor size factor that evaluates the size of the selected parking space relative to the size of neighboring parking spaces. The at least one parking lot / parking space factor includes a neighbor co-parallelism factor that evaluates the parallelism of the selected parking space relative to the sides of adjacent parking spaces. The at least one parking lot / parking space factor includes one or more parking lot / parking space factors selected from the group consisting of: a line length factor that evaluates the equality of the lengths of the opposite sides of the selected parking space, a parallel line factor that evaluates the parallelism of the opposite sides of the selected parking space, a rectangle factor that evaluates the selected parking space relative to a keyframe measurement of a rectangle, a neighbor size factor that evaluates the size of the selected parking space relative to the size of a neighboring parking space, and a neighbor coparallelism factor that evaluates the parallelism of the selected parking space relative to the sides of a neighboring parking space.
[0011] A vision-based perception system for a vehicle includes N cameras arranged on the vehicle and configured to generate multiple images, where N is an integer greater than 1. An image analysis module is configured to identify multiple features of parking spaces in the multiple images and generate multiple keyframe measurements based thereon. A keyframe selection module is configured to identify selected keyframe measurements from the multiple keyframe measurements. A database is configured to receive the selected keyframe measurements from the multiple keyframe measurements and selectively output irrelevant keyframe measurements and existing features of parking spaces within a predetermined range of the vehicle. A data association module is configured to receive irrelevant keyframe measurements and existing features of parking spaces from the database; and selectively associate the selected measurements in the keyframe measurements with existing features of parking spaces, and associate other selected measurements in the keyframe measurements with new features of parking spaces. An optimization module is configured to optimize at least one of the vehicle position and parking space position using a Simultaneous Localization and Mapping (SLAM) cost function that includes at least one parking lot / parking space factor. At least one parking lot / parking space factor includes three or more parking lot / parking space factors selected from the group consisting of: a quadrilateral factor that evaluates the keyframe measurement of the selected parking space relative to a quadrilateral, a line length factor that evaluates the equality of the lengths of the opposite sides of the selected parking space, a parallel line factor that evaluates the parallelism of the opposite sides of the selected parking space, a rectangle factor that evaluates the keyframe measurement of the selected parking space relative to a rectangle, a neighbor size factor that evaluates the size of the selected parking space relative to the size of a neighboring parking space, and a neighbor co-parallel factor that evaluates the parallelism of the selected parking space relative to the sides of a neighboring parking space.
[0012] Among the other features, at least two of the unrelated keyframe measurements and the existing features of the parking space are asynchronous measurements.
[0013] This invention also includes the following technical solutions: Option 1. A vision-based perception system for vehicles, comprising: N cameras are deployed on the vehicle and configured to generate multiple images, where N is an integer greater than 1; The image analysis module is configured to identify multiple features of parking spaces in multiple images and generate multiple keyframe measurements based on these features. The keyframe selection module is configured to identify the selected measurement among the plurality of keyframe measurements; The database is configured to receive selected measurements from the plurality of keyframe measurements and selectively output irrelevant keyframe measurements and existing features of parking spaces within a predetermined range of the vehicle; and The data association module is configured as follows: Receive irrelevant keyframe measurements and existing features of the parking space from the database; and Selectively associate chosen measurements in keyframe measurements with existing features of the parking space, and associate other selected measurements in keyframe measurements with new features of the parking space; and The optimization module is configured to optimize the location of parking spaces in the local map using a Simultaneous Localization and Mapping (SLAM) cost function extended by at least one parking lot / parking space factor.
[0014] Option 2. According to the vision-based perception system described in Option 1, at least two of the unrelated keyframe measurements and the existing features of the parking space are measured asynchronously.
[0015] Option 3. The vision-based perception system according to Option 1, wherein the data association module is configured as follows: Generate measurements between irrelevant keyframe measurements and existing features of the parking space; Identify correlations between measurements; and Identify the relationships between boundary markers.
[0016] Option 4. The vision-based perception system according to Option 1, wherein the at least one parking lot / parking space factor includes a quadrilateral factor, the quadrilateral factor evaluating keyframe measurements of a selected parking space relative to a quadrilateral parking space.
[0017] Option 5. The vision-based perception system according to Option 1, wherein the at least one parking lot / parking space factor includes a rectangle factor that evaluates keyframe measurements of a selected parking space relative to a rectangle of parking spaces.
[0018] Option 6. The vision-based perception system according to Option 1, wherein the at least one parking lot / parking space factor includes a line length factor that evaluates the equality of the lengths of opposite sides of the selected parking space.
[0019] Option 7. The vision-based perception system according to Option 1, wherein the at least one parking lot / parking space factor includes a parallel line factor that evaluates the parallelism of opposite sides of a selected parking space.
[0020] Option 8. The vision-based perception system according to Option 1, wherein the at least one parking lot / parking space factor includes a neighbor size factor that evaluates the size of the selected parking space among the parking spaces relative to the size of the adjacent parking spaces.
[0021] Option 9. The vision-based perception system according to Option 1, wherein the at least one parking lot / parking space factor includes a neighbor co-parallelism factor that evaluates the parallelism of the selected parking space among the parking spaces relative to the edges of neighboring parking spaces.
[0022] Option 10. The vision-based perception system according to Option 1, wherein: The at least one parking lot / parking space factor includes a quadrilateral factor that evaluates keyframe measurements of a selected parking space relative to a quadrilateral; and The at least one parking lot / parking space factor also includes two or more parking lot / parking space factors selected from the group consisting of: The line length factor is used to evaluate the equality of the lengths of opposite sides of selected parking spaces. The parallelism factor is used to evaluate the parallelism between the opposite sides of the selected parking spaces. The rectangle factor is evaluated relative to keyframe measurements of the selected parking space within a rectangle. The neighbor size factor evaluates the size of the selected parking space relative to the size of neighboring parking spaces. The neighbor parallelism factor assesses the parallelism of a selected parking space relative to the edges of neighboring parking spaces.
[0023] Option 11. A vision-based perception system for a vehicle, comprising: N cameras are deployed on the vehicle and configured to generate multiple images, where N is an integer greater than 1; The image analysis module is configured to identify multiple features of parking spaces in multiple images and generate multiple keyframe measurements based on these features. The keyframe selection module is configured to identify the selected measurement among the plurality of keyframe measurements; The database is configured to receive selected keyframe measurements from the plurality of keyframe measurements and selectively output irrelevant keyframe measurements and existing features of parking spaces within a predetermined range of the vehicle; and The data association module is configured as follows: Receive irrelevant keyframe measurements and existing features of the parking space from the database; and Selectively associate chosen measurements in keyframe measurements with existing features of the parking space, and associate other selected keyframe measurements with new features of the parking space; and The optimization module is configured to optimize the location of parking spaces using a Simultaneous Localization and Mapping (SLAM) cost function that includes at least one parking lot / parking space factor. Wherein, the at least one parking lot / parking space factor includes a quadrilateral factor, the quadrilateral factor evaluating keyframe measurements of a selected parking space relative to a quadrilateral parking space, and At least two of the unrelated keyframe measurements and the existing features of the parking space are measured asynchronously.
[0024] Solution 12. The vision-based perception system according to Solution 11, wherein the data association module is configured as follows: Generate measurements between irrelevant keyframe measurements and existing features of the parking space; Identify correlations between measurements; and Identify the relationships between boundary markers.
[0025] Option 13. The vision-based perception system according to Option 11, wherein the at least one parking lot / parking space factor includes a rectangle factor that evaluates keyframe measurements of a selected parking space relative to a rectangle of parking spaces.
[0026] Option 14. The vision-based perception system according to Option 11, wherein the at least one parking lot / parking space factor includes a line length factor that evaluates the equality of the lengths of opposite sides of a selected parking space.
[0027] Option 15. The vision-based perception system according to Option 11, wherein the at least one parking lot / parking space factor includes a parallel line factor that evaluates the parallelism of opposite sides of a selected parking space in a parking space.
[0028] Option 16. The vision-based perception system according to Option 11, wherein the at least one parking lot / parking space factor includes a neighbor size factor, the neighbor size factor evaluating the size of the selected parking space among parking spaces relative to the size of neighboring parking spaces.
[0029] Option 17. The vision-based perception system according to Option 11, wherein the at least one parking lot / parking space factor includes a neighbor co-parallelism factor, the neighbor co-parallelism factor evaluating the parallelism of a selected parking space relative to the edges of neighboring parking spaces.
[0030] Option 18. The vision-based perception system according to Option 1, wherein the at least one parking lot / parking space factor includes one or more parking lot / parking space factors selected from the group consisting of: The line length factor evaluates the equality of the lengths of opposite sides of a selected parking space. The parallelism factor is used to evaluate the parallelism between the opposite sides of a selected parking space. The rectangle factor is evaluated relative to keyframe measurements of the selected parking space within a rectangle. The neighbor size factor evaluates the size of the selected parking space relative to the size of neighboring parking spaces. The neighbor parallelism factor assesses the parallelism of the selected parking space relative to the edge of the neighboring parking space.
[0031] Option 19. A vision-based perception system for a vehicle, comprising: N cameras are deployed on the vehicle and configured to generate multiple images, where N is an integer greater than 1; The image analysis module is configured to identify multiple features of parking spaces in multiple images and generate multiple keyframe measurements based on these features. The keyframe selection module is configured to identify the selected measurement among the plurality of keyframe measurements; The database is configured to receive selected keyframe measurements from the plurality of keyframe measurements and selectively output irrelevant keyframe measurements and existing features of parking spaces within a predetermined range of the vehicle; and The data association module is configured as follows: Receive irrelevant keyframe measurements and existing features of the parking space from the database; and Selectively associate chosen measurements in keyframe measurements with existing features of the parking space, and associate other selected measurements in keyframe measurements with new features of the parking space; and The optimization module is configured to optimize at least one of vehicle location and parking space location using a Simultaneous Localization and Mapping (SLAM) cost function that includes at least one parking lot / parking space factor. Wherein, the at least one parking lot / parking space factor includes three or more parking lot / parking space factors selected from the group consisting of: The quadrilateral factor is evaluated relative to the keyframe measurement of the selected parking space within the quadrilateral parking space. The line length factor evaluates the equality of the lengths of opposite sides of a selected parking space. The parallelism factor is used to evaluate the parallelism between the opposite sides of a selected parking space. Evaluate the rectangle factor of the keyframe measurement relative to the selected parking space within the rectangle. The neighbor size factor evaluates the size of the selected parking space relative to the size of neighboring parking spaces. The neighbor parallelism factor assesses the parallelism of a selected parking space relative to the edges of neighboring parking spaces.
[0032] Option 20. The vision-based perception system according to Option 19, wherein at least two of the unrelated keyframe measurements and the existing features of the parking space are measured asynchronously.
[0033] Further applications of this disclosure will become apparent from the detailed description, claims, and accompanying drawings. The detailed description and specific examples are intended for illustrative purposes only and are not intended to limit the scope of this disclosure. Attached Figure Description
[0034] This disclosure will become more fully understood based on the detailed description and accompanying drawings, wherein: Figure 1 This is a functional block diagram of an example vehicle including a vision-based system according to the present disclosure, the vision-based system being configured to map and locate parking spaces in a parking lot; Figure 2A This is a floor plan of an example of a vehicle parked in one of the parking spaces in a parking lot, according to this disclosure; Figure 2B The illustration shows a vehicle in a parking space according to this disclosure; Figure 3 This is a functional block diagram of an example optimization module according to the present disclosure, which uses a Simultaneous Localization and Mapping (SLAM) cost function that includes parking space / parking lot related extensions to optimize a local map; Figure 4 This is an example of simultaneous location and mapping of parking spaces in a parking lot based on graphics, according to this disclosure; and Figure 5 The functional block diagram of the SLAM optimization module according to this disclosure has a Simultaneous Localization and Mapping (SLAM) cost function extended by one or more parking lot / parking space related factors.
[0035] In the accompanying drawings, reference numerals may be used repeatedly to identify similar and / or identical elements. Detailed Implementation
[0036] Parking spaces are large, static, flat objects with robust and consistent structures. During Simultaneous Localization and Mapping (SLAM), expansion factors associated with parking lots / parking spaces are utilized to reduce cumulative localization errors by providing fixed, robust landmarks. Implementing prior constraints (e.g., quadrilateral shape and parallelism) in the SLAM cost function improves data association, minimizes false matches, and / or enhances mapping. Figure 1 Consistency. Utilizing inter-slot geometry allows elements outside the field of view to contribute, which further offsets odometer measurement noise and detection uncertainty.
[0037] Now for reference Figure 1 Vehicle 10 includes various sensors, such as a Global Positioning System (GPS) 14, an Inertial Measurement Unit (IMU) 18, and wheel speed sensors 20. Vehicle 10 includes one or more cameras 34-1, 34-2, ..., 34-N that generate images relative to the front, sides, and rear of vehicle 10, where N is an integer greater than 1. In some examples, cameras 34-1, 34-2, ..., 34-N include one or more forward-facing cameras, one or more side-facing cameras, and one or more rear-facing cameras.
[0038] The controller 42 includes an image analysis module 44 configured to receive images from cameras 34-1, 34-2, ..., 34-N, identify landmarks or parking space features in the images (e.g., corners or centers of parking spaces), and generate keyframe measurements from the corresponding camera to the landmarks in the images. In some examples, the keyframe measurements include the corners or centers of the parking spaces. In some examples, the image analysis module 44 may include a neural network configured to recognize various objects, such as the center or corners of parking spaces.
[0039] The keyframe selection module 46 selects keyframe measurements (each with a timestamp) from the images. Cameras 34-1, 34-2, ..., 34-N generate numerous samples, and not all samples provide new or useful information. For example, when the vehicle is stationary or moving slowly, the measurements may or may not change significantly in subsequent keyframes over a consecutive timestamp period. Not all images will include the parking space. The keyframe selection module 46 is configured to select some or all of the keyframes based on the values of the data.
[0040] Simultaneous Localization and Mapping (SLAM) module 47 is configured to perform localization and mapping. In some examples, SLAM module 47 includes graph-based SLAM. In some examples, SLAM module 47 includes the Georgia Tech Smoothing and Mapping (GTSAM) module or the g2o module, but other types of optimization modules can be used.
[0041] SLAM module 47 includes a data association module 48 configured to receive keyframes and associate the keyframes with existing states (e.g., already identified parking spaces) or new states (e.g., new parking spaces not previously identified). Additional details relating to data association module 48 can be found in U.S. Patent Application Serial No. 19 / 194,253 (Attorney General's File No. P110843), entitled "A-Synchronous Association for Parking Slot Measurements," which is incorporated herein by reference in its entirety.
[0042] In some examples, SLAM module 47 includes optimization module 52 for managing and optimizing SLAM state. In some examples, SLAM state corresponds to vehicle and / or camera position and parking space characteristics (e.g., corner or center of the parking space).
[0043] In some examples, SLAM module 47 also includes a scale estimation module 50, which is configured to estimate the scale of the measurement based on each camera and / or each measurement. Additional details about the scale estimation module 50 can be found in U.S. Patent Application Serial No. 19 / 194,338 (Attorney General's File No. P110844), entitled "Incremental Scale Estimation for Vision-Based Autonomous Driving Systems," which is incorporated herein by reference in its entirety.
[0044] The controller 42 includes an autonomous driving module 56 configured to control actuators 60, such as steering, braking, and / or acceleration of the vehicle 10. For example, the autonomous driving module 56 can park the vehicle in one of the parking spaces identified and mapped by the parking space location module 54.
[0045] Now for reference Figure 2A Vehicle 10 enters parking lot 70, which includes multiple parking spaces 72. Vehicle 10 uses cameras 34-1, 34-2, ..., 34-N to generate images. Some of the images are selected as keyframes. Some of the selected keyframes may include parking spaces identified by parking space features, such as center or corner locations. Keyframes include the corner locations of one or more parking spaces at a given timestamp. In some examples, keyframes are generated by a single camera (e.g., a parking space located on one side of vehicle 10). In some examples, keyframes at a given timestamp may be generated by cameras on both sides of the vehicle, a front camera, and / or a rear camera.
[0046] Some of the parking spaces 72 are occupied by vehicle 80, while the others in parking spaces 72 are open. Vehicle 10 can proceed directly to one of the parking spaces 72 (e.g., parking space 72-1 or parking space 72-2), or vehicle 10 can drive around the parking lot 70 before selecting one of the parking spaces (e.g., parking space 72-3).
[0047] Now for reference Figure 2B The example shown includes parking spaces with corners 1, 2, 3, and 4. In some examples, since the vehicles are closer to corners 2 and 3, the positions of the near corners 2 and 3 can be detected (with high confidence and low covariance), while the positions of the far corners 1 and 4 can be estimated (with lower confidence and higher covariance).
[0048] Now for reference Figure 3 Cameras 34-1, 34-2, ..., and 34-N output images to image analysis module 44. Image analysis module 44 outputs keyframes to keyframe selection module 46, which selects some of the keyframes for further analysis. The selected keyframes, along with odometer / IMU data 92 corresponding to the timestamps of the keyframe measurements, are stored in database 90, such as a Structured Query Language (SQL) database.
[0049] Data association module 74 is configured to determine whether unassociated keyframe measurements (e.g., parking space features such as the center of a corner location of a parking space) correspond to a previously detected existing parking space or a new parking space. In some examples, data association module 74 makes measurements between unassociated keyframe measurements (e.g., parking spaces in unassociated keyframes) and the locations of existing parking spaces. In some examples, data association module 74 identifies associations between measurements. In some examples, data association module 74 identifies associations between landmarks. Additional details relating to data association module 48 can be found in U.S. Patent Application Serial No. 19 / 194,253 (Attorney General's File No. P110843), entitled "A-Synchronous Association for Parking Slot Measurements".
[0050] The optimization module 52 uses a Simultaneous Localization and Mapping (SLAM) cost function to optimize the SLAM state and the local map. This cost function is extended by one or more parking space / parking lot-related factors, which will be further described below. The map publisher 96 retrieves data from the database 90 and generates and publishes a local map in a sliding window around the vehicle. In some examples, the autonomous driving module 56 uses the local map to select a parking space and park the vehicle there.
[0051] Now for reference Figure 4 The diagram illustrates a graph-based map generated for vehicles and parking spaces in a parking lot during SLAM. In the graph, a vehicle moves from vehicle state x0 to x1, from x1 to x2, and from x2 to x3. In each vehicle state, the vehicle's camera perception corresponds to the parking space markers I0 and I1.
[0052] Now for reference Figure 5 The optimization module 52 is extended by adjusting the Simultaneous Localization and Mapping (SLAM) cost function 110 to include other factors, such as parking lot / parking space factor 112. The SLAM cost function can also include other typical SLAM factors 114. Examples of parking lot / parking space factors 112 include quadrilateral factor, line length factor, parallel line factor, rectangle factor, neighbor size factor, and neighbor co-parallel factor. The SLAM cost function associated with the corresponding parking space factor increases when a specific attribute of a parking space or parking lot changes from the expected value. The optimization module 52 attempts to minimize the SLAM cost function by iterating and changing the estimated location of the parking space and re-evaluating the SLAM cost function.
[0053] The quadrilateral factor is a component of the SLAM cost function that evaluates whether a parking space is quadrilateral (since most parking spaces are quadrilateral). The quadrilateral factor compares keyframe measurements to the 3D positions of the four corners of the parking space. In some examples, the comparison is made in centroid (CG) coordinates. In other words, the SLAM cost function receives keyframe measurements of the parking space corners and the quadrilateral representing the parking space in global coordinates, and uses these parking space measurements to align these states.
[0054] The line length factor is a component of the SLAM cost function that evaluates whether opposite sides of parking spaces (e.g., 3-4 and 1-2) are of equal length (because most opposite lines of parking spaces have equal length). In other words, the line length factor forces opposite sides of parking spaces 1-2 and 3-4 to be of equal length. The parallel line factor is a component of the SLAM cost function that evaluates whether the opposite sides of a parking space are parallel (e.g., 3-4 is parallel to 1-2) (because most parking spaces are usually parallel).
[0055] Jacobian style: Covariance: The rectangle factor determines whether a parking space is rectangular. A rectangle is a parallelogram whose sides form right angles.
[0056] The neighbor size factor is a component of the SLAM cost function that evaluates whether neighboring parking spaces have similar sizes (because most parking spaces in a given parking lot or within a given sliding window have a uniform shape). If inter-parking space regions exist, these regions are also constrained to be parallelograms. The neighbor co-parallelism factor is a component of the SLAM cost function that evaluates whether neighboring parking spaces (slots) are co-parallel (because most parking lots have aligned parking spaces).
[0057] Extending the SLAM optimization module by using a cost function that optimizes parking lot / parking space factors significantly improves the accuracy of local maps.
[0058] The foregoing description is merely illustrative in nature and is in no way intended to limit this disclosure, its application, or use. The broad teachings of this disclosure can be implemented in many forms. Therefore, while this disclosure includes specific examples, its true scope should not be so limited, as other modifications will become apparent upon examination of the drawings, specification, and the following claims. It should be understood that one or more steps within the method may be performed in different orders (or simultaneously) without altering the principles of this disclosure. Furthermore, while each of the embodiments is described above as having certain features, any one or more of those features described with respect to any embodiment of this disclosure may be implemented in any other embodiment and / or combined with features of any other embodiment, even if that combination is not explicitly described. In other words, the described embodiments are not mutually exclusive, and substitution of one or more embodiments for each other remains within the scope of this disclosure.
[0059] Various terms (including “connection,” “joint,” “coupled,” “adjacent,” “next to,” “on top of,” “above,” “below,” and “set up”) are used to describe spatial and functional relationships between elements (e.g., between modules, circuit elements, semiconductor layers, etc.). Unless explicitly described as “direct,” when describing a relationship between first and second elements in the above disclosure, that relationship can be a direct relationship where no other intervening element exists between the first and second elements, but it can also be an indirect relationship (spatially or functionally) between the first and second elements with one or more intervening elements. As used herein, the phrase “at least one of A, B, and C” should be interpreted as meaning a logic using the non-exclusive logic “OR” (A or B or C) and should not be interpreted as meaning “at least one of A, at least one of B, and at least one of C.”
[0060] In various diagrams, the direction of arrows, indicated by arrowheads, typically shows the flow of information of interest to the diagram (such as data or instructions). For example, when elements A and B exchange various types of information, but the information transmitted from element A to element B is relevant to the diagram, the arrow can point from element A to element B. This unidirectional arrow does not mean that no other information is transmitted from element B to element A. Furthermore, for information sent from element A to element B, element B can send a request for or confirmation of receipt of that information to element A.
[0061] In this application, which includes the definitions below, the term "module" or "controller" may be replaced by the term "circuit". The term "module" may refer to, be part of, or include the following: application-specific integrated circuit (ASIC); digital, analog, or mixed-signal analog / digital discrete circuit; digital, analog, or mixed-signal analog / digital integrated circuit; combinational logic circuit; field-programmable gate array (FPGA); processor circuitry (shared, dedicated, or grouped) that executes code; memory circuitry (shared, dedicated, or grouped) that stores code executed by the processor circuitry; other suitable hardware components that provide the described functionality; or some or all of the foregoing, such as in a system-on-a-chip.
[0062] A module may include one or more interface circuits. In some examples, the interface circuits may include wired or wireless interfaces that connect to a local area network (LAN), the Internet, a wide area network (WAN), or a combination thereof. The functionality of any given module in this disclosure may be distributed among multiple modules connected via the interface circuits. For example, multiple modules may allow for load balancing. In another example, a server (also referred to as a remote or cloud) module may perform some functions on behalf of a client module.
[0063] The term "code," as used above, can include software, firmware, and / or microcode, and can refer to programs, routines, functions, classes, data structures, and / or objects. The term "shared processor circuitry" includes a single processor circuitry that executes some or all of the code from multiple modules. The term "group processor circuitry" includes processor circuitry that, in conjunction with other processor circuitry, executes some or all of the code from one or more modules. References to multiple processor circuitry include multiple processor circuitry on a discrete die, multiple processor circuitry on a single die, multiple cores of a single processor circuitry, multiple threads of a single processor circuitry, or a combination of the above. The term "shared memory circuitry" includes a single memory circuitry that stores some or all of the code from multiple modules. The term "group memory circuitry" includes memory circuitry that, in conjunction with other memory, stores some or all of the code from one or more modules.
[0064] The term "memory circuit" is a subset of the term "computer-readable medium." As used herein, the term "computer-readable medium" does not include transient electrical or electromagnetic signals propagating through a medium (e.g., on a carrier wave); therefore, the term "computer-readable medium" can be considered tangible and non-transitory. Non-limiting examples of non-transitory tangible computer-readable media are non-volatile memory circuits (e.g., flash memory circuits, erasable programmable read-only memory circuits, or mask read-only memory circuits), volatile memory circuits (e.g., static random access memory circuits or dynamic random access memory circuits), magnetic storage media (e.g., analog or digital magnetic tape or hard disk drives), and optical storage media (e.g., CDs, DVDs, or Blu-ray discs).
[0065] The apparatus and methods described in this application can be implemented, in part or in whole, by a dedicated computer created by configuring a general-purpose computer to perform one or more specific functions contained in a computer program. The function blocks, flowchart components, and other elements described above serve as software specifications that can be translated into computer programs from the routine work of technicians or programmers.
[0066] A computer program includes processor-executable instructions stored on at least one non-transitory tangible computer-readable medium. A computer program may also include or depend on stored data. A computer program may include a basic input / output system (BIOS) for interacting with the hardware of a special-purpose computer, device drivers for interacting with specific devices of the special-purpose computer, one or more operating systems, user applications, background services, background applications, etc.
[0067] Computer programs may include: (i) descriptive text to be parsed, such as HTML (Hypertext Markup Language), XML (Extensible Markup Language), or JSON (JavaScript Object Notation); (ii) assembly code; (iii) object code generated from source code by a compiler; (iv) source code executed by an interpreter; (v) source code compiled and executed by a real-time compiler, etc. As an example only, source code can be written using syntax from languages including C, C++, C#, Objective-C, Swift, Haskell, Go, SQL, R, Lisp, Java®, Fortran, Perl, Pascal, Curl, OCaml, JavaScript®, HTML5 (Hypertext Markup Language, Fifth Revision), Ada, ASP (Active Server Pages), PHP (PHP: Hypertext Preprocessor), Scala, Eiffel, Smalltalk, Erlang, Ruby, Flash®, Visual Basic®, Lua, MATLAB, SIMULINK, and Python®.
Claims
1. A vision-based perception system for a vehicle, comprising: N cameras are deployed on the vehicle and configured to generate multiple images, where N is an integer greater than 1; The image analysis module is configured to identify multiple features of parking spaces in multiple images and generate multiple keyframe measurements based on these features. The keyframe selection module is configured to identify the selected measurement among the plurality of keyframe measurements; The database is configured to receive selected measurements from the plurality of keyframe measurements and selectively output irrelevant keyframe measurements and existing features of parking spaces within a predetermined range of the vehicle; and The data association module is configured as follows: Receive irrelevant keyframe measurements and existing features of the parking space from the database; and Selectively associate the measurements chosen in the keyframe measurements with existing features of the parking space, and associate other selected measurements in the keyframe measurements with new features of the parking space; and The optimization module is configured to optimize the location of parking spaces in the local map using a Simultaneous Localization and Mapping (SLAM) cost function extended by at least one parking lot / parking space factor.
2. The vision-based perception system of claim 1, wherein at least two of the unrelated keyframe measurements and the existing features of the parking space are measured asynchronously.
3. The vision-based perception system according to claim 1, wherein, The data association module is configured as follows: Generate measurements between irrelevant keyframe measurements and existing features of the parking space; Identify correlations between measurements; and Identify the relationships between boundary markers.
4. The vision-based perception system according to claim 1, wherein, The at least one parking lot / parking space factor includes a quadrilateral factor that evaluates keyframe measurements of a selected parking space relative to a quadrilateral parking space.
5. The vision-based perception system according to claim 1, wherein, The at least one parking lot / parking space factor includes a rectangle factor that evaluates keyframe measurements of a selected parking space relative to a rectangle.
6. The vision-based perception system of claim 1, wherein the at least one parking lot / parking space factor includes a line length factor that evaluates the equality of the lengths of opposite sides of a selected parking space.
7. The vision-based perception system of claim 1, wherein the at least one parking lot / parking space factor includes a parallel line factor that evaluates the parallelism of opposite sides of a selected parking space.
8. The vision-based perception system according to claim 1, wherein, The at least one parking lot / parking space factor includes a neighbor size factor that evaluates the size of the selected parking space relative to the size of adjacent parking spaces.
9. The vision-based perception system according to claim 1, wherein, The at least one parking lot / parking space factor includes a neighbor co-parallelism factor that evaluates the parallelism of the selected parking space among the parking spaces relative to the edges of neighboring parking spaces.
10. The vision-based perception system according to claim 1, wherein: The at least one parking lot / parking space factor includes a quadrilateral factor that evaluates keyframe measurements of a selected parking space relative to a quadrilateral parking space. and The at least one parking lot / parking space factor also includes two or more parking lot / parking space factors selected from the group consisting of: The line length factor is used to evaluate the equality of the lengths of opposite sides of selected parking spaces. The parallelism factor is used to evaluate the parallelism between the opposite sides of the selected parking spaces. The rectangle factor is evaluated relative to keyframe measurements of the selected parking space within a rectangle. The neighbor size factor evaluates the size of the selected parking space relative to the size of neighboring parking spaces. The neighbor parallelism factor assesses the parallelism of a selected parking space relative to the edges of neighboring parking spaces.