Vehicle positioning using map data and visual data

Abstract

This document describes vehicle positioning using map and visual data. For example, this document describes a positioning system that obtains the map centerline points and visual centerline points of road lanes. The positioning system also obtains the vehicle's position. The positioning system can then compare the map centerline points and the visual centerline points to generate lateral and longitudinal corrections relative to the vehicle's position. The lateral and longitudinal corrections are used to generate the corrected position. In this way, the described positioning system can provide accurate vehicle positioning, which resolves potential drift caused by interrupted or inaccurate positioning data, and allows driver assistance systems and autonomous driving systems to operate at higher speeds and on roads with sharper curves.

Description

Background Technology

[0001] Advanced driver assistance systems (ADAS) and autonomous driving systems use map data to perform various ADAS functions (e.g., route planning, lane centering, automatic lane changing, autonomous driving). These functions require accurate vehicle position (e.g., plus or minus ten centimeters) to meet performance and safety requirements. Under ideal conditions, satellite-based positioning (e.g., via Global Positioning System (GPS) or Global Navigation Satellite System (GNSS)) can achieve the necessary accuracy. However, under non-ideal conditions (e.g., when line-of-sight to a satellite is impossible), satellite-based positioning cannot provide the necessary accuracy. Therefore, vehicle position drift (e.g., errors of up to two meters) can occur, negatively impacting ADAS systems. Summary of the Invention

[0002] This document describes techniques, apparatus, and systems for vehicle positioning using map data and visual data. For example, this document discloses a positioning system that can obtain the map centerline point and visual centerline point of the lane in which the vehicle is traveling. The positioning system can also use positioning data to obtain the vehicle's position. The positioning system can then compare the map centerline point and the visual centerline point to generate lateral and longitudinal corrections relative to the vehicle's position. The lateral and longitudinal corrections are used to generate the corrected position. The vehicle can then be operated on the road based on the corrected position. In this way, the described positioning system can provide accurate vehicle positioning that addresses potential drift caused by lapses or inaccurate positioning data. Corrected vehicle positioning using map data and visual data allows assisted driving systems and autonomous driving systems to operate at higher speeds and on roads with sharper curves.

[0003] This document also describes other operations of the systems, techniques, and apparatuses summarized above, as well as other methods described herein, and apparatuses for performing these methods.

[0004] This invention provides a simplified concept for vehicle positioning using map and visual data, which is further described in the detailed description and accompanying drawings. This invention is not intended to identify essential features of the claimed subject matter, nor is it intended to define the scope of the claimed subject matter. Attached Figure Description

[0005] This document describes in detail one or more aspects of vehicle positioning using map and visual data with reference to the following figures. The same figures are used throughout the figures to refer to similar features and components:

[0006] Figure 1 An example road environment is shown in which the locator can use map data and visual data to perform vehicle positioning according to the techniques described in this disclosure;

[0007] Figure 2 A vehicle software component for performing vehicle positioning using map data and visual data, according to the technology described in this disclosure, is shown;

[0008] Figure 3 An example concept diagram is shown indicating how visual center line points and map center line points are managed as inputs for vehicle positioning;

[0009] Figure 4 An example concept diagram of a software method for locating vehicles using map data and visual data according to the technology described in this disclosure is shown;

[0010] Figure 5 An example concept diagram shows a software method that uses map data and visual data to determine the recapture distance as part of vehicle positioning;

[0011] Figure 6 An example conceptual diagram illustrates a software method that uses map data and visual data to capture trail points as part of vehicle localization;

[0012] Figure 7 An example conceptual diagram illustrates a software method that uses map data and visual data to update trajectories as part of vehicle localization; and

[0013] Figure 8 An example flowchart is shown as an example process for performing vehicle positioning using map data and visual data. Detailed Implementation

[0014] Overview

[0015] Vehicle positioning is a crucial aspect of driver assistance systems (ADAS) and autonomous driving systems. ADAS and autonomous driving systems can provide route planning, lane centering, automatic lane changing, and autonomous driving functions. However, these functions require accurate positioning of the primary vehicle to meet performance and safety requirements. Positioning using satellite-based systems (e.g., GPS, GNSS) can achieve the necessary accuracy under ideal conditions. However, such systems cannot provide the required accuracy in all environments (e.g., when the line of sight to the satellite is obstructed by overpasses or skyscrapers). In such cases, estimating the vehicle's positional drift (e.g., errors of up to two meters) can render the accuracy of many ADAS functions unacceptable.

[0016] Some vehicle-based systems fuse map and visual data to correct vehicle positioning. These systems typically use sophisticated algorithms (e.g., Simultaneous Localization and Mapping (SLAM), Kalman filters, 3D image processing) to correlate map and visual data. Other systems use hybrid cube formulas to link map data to visual data, which requires significant computational overhead that is not typically available in vehicles.

[0017] In contrast, this document describes a computationally efficient vehicle positioning system using both map and visual data. For example, this document discloses a positioning system that obtains map centerline points and visual centerline points of road lanes. The positioning system also uses the positioning data to determine the vehicle's position. The positioning system can then compare the map centerline points and the visual centerline points to generate lateral and longitudinal corrections relative to the vehicle's position. These lateral and longitudinal corrections are applied to the vehicle's position to generate a corrected position. The described system can provide accurate vehicle positioning that addresses potential drift caused by data interruptions or insufficient data in the positioning system. This improved vehicle positioning allows assisted driving systems and autonomous driving systems to operate at higher speeds and on roads with sharper curves.

[0018] This section describes only one example of how the described technology and system use map and visual data to perform vehicle localization. Other examples and implementations are described in this document.

[0019] Operating environment

[0020] Figure 1 An example road environment 100 is shown, in which the locator 104 can use map data and visual data to perform vehicle positioning according to the technology described in this disclosure. Figure 1 A locator 104 is shown as part of a system (not shown) implemented within vehicle 102. Although presented as a passenger car, vehicle 102 could represent other motorized vehicles (e.g., motorcycles, buses, tractor-trailers, semi-trailer trucks, or construction equipment). Typically, the manufacturer can mount or install the locator 104 into any mobile platform that travels on a road.

[0021] Vehicle 102 is traveling along the road. Despite... Figure 1 The road is presented as a road with lanes and lane markings (e.g., highway), but the road can be any type of designated route for vehicles, including, for example, virtual waterways used by ships and ferries, virtual airways used by unmanned aerial vehicles (UAVs) and other aircraft, train tracks, tunnels, or virtual underwater channels.

[0022] The locator 104 acquires visual centerline points 106 and map centerline points 108 associated with the road. The array of visual centerline points 106 represents the lateral center of the corresponding lane as determined by one or more visual sensors (e.g., cameras). The array of map centerline points 108 represents the lateral center of the corresponding lane as stored in a map database (e.g., a high-definition (HD) map database). The map centerline points 108 may include corresponding geographic locations (e.g., latitude and longitude coordinates) provided according to the desired direction of travel of vehicles on the road segment. Each visual centerline point 106 and map centerline point 108 may include latitude and longitude coordinates in a map coordinate system.

[0023] The road comprises one or more lanes, which are represented by visual centerline point 106, map centerline point 108, lane segments 110, and lane segment groups (LGS) 112. Lane segments 110 represent corresponding portions of road lanes. For example, lane segments 110-1, 110-3, and 110-5 represent corresponding portions of the current lane in which vehicle 102 is traveling. One or more lane segments 110 with the same direction of travel are included in LSG 112. LSG 112 typically represents corresponding portions of lane groups with unchanged lane markings and no splitting. For example, LSG 112-1 includes lane segments 110-1 and 110-2. LSG 112-2 includes lane segments 110-3 and 110-4. LSG 112-3 includes lane segments 110-5 and 110-6. Each of the LSGs 112 may include multiple lines (e.g., vectors of points, lane markings). In some implementations, each of the LSGs 112 may include a predetermined origin. The origin may be located at the lateral center of the respective LSG 112, and at the starting point of each LSG 112. Without departing from the scope of this disclosure, the position of the origin relative to the respective LSG 112 may vary.

[0024] In the depicted environment 100, one or more sensors (not shown) are mounted to or integrated within the vehicle 102. The sensors may include a visual sensor (e.g., a camera) and a position sensor that provide visual and position data to the locator 104, respectively. The position sensor may include a GPS and / or GNSS system or an inertial measurement unit (IMU). The locator 104 may also acquire map data stored locally or remotely in a map database. The locator 104 uses the map data and visual data to provide accurate vehicle positioning (e.g., for lane centering or autonomous driving) to the vehicle 102's driver assistance and autonomous driving systems. For example, the locator may acquire a visual centerline point 106 and a map centerline point 108 of the road on which the vehicle is traveling. The locator 104 compares the visual centerline point 106 and the map centerline point 108 to generate lateral and longitudinal corrections relative to the vehicle 102's position (which is obtained based on the positioning data). The locator 104 applies the lateral and longitudinal corrections to the vehicle's position to generate a corrected position for the vehicle. In this way, the locator 104 uses map data and visual data to provide more accurate vehicle positioning, thereby allowing driver assistance systems and autonomous driving systems to avoid drift in the positioning system and operate smoothly on curved roads at higher speeds.

[0025] Example Architecture

[0026] Figure 2 A vehicle software component for performing vehicle positioning using map data and visual data, according to the technology described in this disclosure, is illustrated. Vehicle 102 includes one or more processors 202, a computer-readable storage medium (CRM) 204, one or more communication components 220, and one or more vehicle-based systems 224. Vehicle 102 may also include one or more sensors (e.g., cameras, radar systems, Global Positioning System (GPS), Global Navigation Satellite System (GNSS), lidar systems, inertial measurement units (IMUs)) to provide input data to the locator 104 and the vehicle-based system 224.

[0027] As a non-limiting example, processor 202 may include a system-on-a-chip (SoC), application processor (AP), electronic control unit (ECU), central processing unit (CPU), or graphics processing unit (GPU). Processor 202 may be a single-core or multi-core processor implemented using a homogeneous or heterogeneous core architecture. Processor 202 may include a hardware-based processor implemented as a hardware-based logic, circuitry, processing core, etc. In some aspects, the functionality of processor 202 and other components of locator 104 is provided via an integrated processing, communication, or control system (e.g., an SoC), which can implement various operations of the vehicle 102 embodying the system.

[0028] The CRM 204 described herein does not include propagation signals. The CRM 204 may include any suitable memory or storage device that can be used to store device data (not shown), map data 208, location data 210, and visual data 212 of the data manager 206, such as random access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM), non-volatile RAM (NVRAM), read-only memory (ROM), or flash memory.

[0029] Processor 202 executes computer-executable instructions stored within CRM 204 to perform the techniques described herein. For example, processor 202 may execute data manager 206 to process and access location data 210, or enable locator 104 to perform vehicle positioning using map data 208 and visual data 212.

[0030] Data manager 206 includes map data 208, location data 210, and visual data 212. Data manager 206 can store map data 208, process updated map data received from remote sources, and retrieve portions of map data 208 for locator 104. In the depicted system, data manager 206 is shown located on or within vehicle 102. In other implementations, data manager 206, or another implementation of data manager 206, may be located remotely from vehicle 102 (e.g., in the cloud or on a remote computer system) and provide map data 208, or a subset of map data 208, to vehicle 102 and locator 104.

[0031] Map data 208 may include lane geometry data, including map centerline point 108, lane segments 110, and LSG 112. For example, map data 208 may include lane geometry data around the current position or attitude of vehicle 102 or along a predetermined proximity to the navigation route. Lane geometry data for each lane of environment 100 may include an array of points representing the lateral (e.g., x) and longitudinal (e.g., y) positions of map centerline point 108 in a global or map coordinate system, and offsets of the distance traveled by vehicle 102 within a particular lane segment 110 and / or LSG 112. Lane geometry data may also include other details associated with the road (e.g., curvature, traffic control devices, stop lines, positioning data, and 3D data). Map data 208 may be stored locally or received from a remote source or database providing lane geometry data for road segments.

[0032] Location data 210 provides attitude or position with lateral (e.g., x) and longitudinal (e.g., y) positions in a global or map coordinate system, as well as the heading of vehicle 102. Location data 210 can be generated from sensor data from satellite systems (e.g., GPS, GNSS) and / or motion sensors (e.g., IMU).

[0033] Visual data 212 may include a visual centerline point 106 and may be generated from sensor data of a visual sensor (e.g., a camera). Data manager 206 may determine the lateral offset of vehicle 102 relative to the lane center based on the left and right lane lines detected from visual data 212. The lateral offset may provide the lateral (e.g., x) and longitudinal (e.g., y) distances of vehicle 102 from the visual centerline point 106, which may be set at a longitudinal distance in front of vehicle 102 based on its current speed. The visual centerline point 106 may be obtained in a map coordinate system. When neither lane line is visible (e.g., at a lane merge), data manager 206 may also apply different logic and algorithms to determine the centerline.

[0034] Similarly, processor 202 can execute locator 104 to accurately locate vehicle 102. Locator 104 may include a position module 214, a lane crawler 216, and a fusion module 218. When satellite system data is intermittent, position module 214 can apply a position model to position data 210 to maintain the estimated attitude of vehicle 102. Position module 214 can be designed to avoid step function changes in the estimated attitude or heading of vehicle 102 that could negatively impact vehicle-based system 224. Position module 214 can provide attitude and heading to lane crawler 216.

[0035] Lane crawler 216 determines the position of vehicle 102 on or within map data 208 based on the estimated attitude or heading provided by position module 214. Lane crawler 216 can output the lateral offset of vehicle 102 relative to the centerline of the current driving lane. The lateral offset can be determined as the longitudinal distance in front of the vehicle based on the vehicle's speed to match the lateral offset of data manager 206 for processing visual data 212.

[0036] The fusion module 218 can compare the trajectory or series of visual centerline points 106 with the series or trajectory of map centerline points to determine a fused trajectory. Visual centerline points 106 and map centerline points 108 are initially in a map coordinate system. Data manager 206 or fusion module 218 can transform the trajectory points from map coordinates to a vehicle coordinate system. The fused trajectory is obtained by using a transformation to best fit (e.g., minimum error fit) the match between the two trajectories, thereby producing a corrected attitude for locating vehicle 102. The corrected attitude includes lateral correction, longitudinal correction, and / or heading or rotation correction. Lateral correction can represent values ​​in the vehicle coordinate system used to adjust the attitude of vehicle 102 in the lateral direction (e.g., along the x-axis of the vehicle coordinate system). Longitudinal correction can represent values ​​in the vehicle coordinate system used to adjust the attitude of vehicle 102 in the longitudinal direction (e.g., along the y-axis of the vehicle coordinate system). Rotation correction can represent values ​​in radians (or degrees) used to correct the heading of vehicle 102. The obtained lateral, longitudinal, and rotational corrections are transformed into the map coordinate system to be applied to the current vehicle attitude estimation and to determine the corrected attitude.

[0037] The fusion module 218 can apply several rules to maintain a stable fused trajectory. Visual centerline point 106 and map centerline point 108 are typically sampled at periodic travel distances (e.g., every six meters). Visual centerline point 106 and map centerline point 108 include synchronized samples of the lateral offset of the centerline relative to the estimated attitude of the vehicle 102. For cases where visual centerline point 106 or map centerline point 108 is invalid (e.g., during a lane change where the centerline switches to a new lane), trajectory points are skipped or not sampled. The fusion module 218 can also set a minimum effective trajectory length for determining the corrected attitude. Furthermore, the fusion module 218 can allow configurable gaps in the trajectory samples to maintain corrections during transient events (e.g., lane changes, lane merging).

[0038] Communication component 220 may include a vehicle-based system interface 222. The vehicle-based system interface 222 can transmit data between various components of vehicle 102 or between components of vehicle 102 and external components via the communication network of vehicle 102. For example, when data manager 206 and locator 104 are integrated within vehicle 102, the vehicle-based system interface 222 can facilitate data transmission between them. When a portion of data manager 206 is located remotely from vehicle 102, the vehicle-based system interface 222 can facilitate data transmission between vehicle 102 and a remote entity having data manager 206. Communication component 220 may also include a sensor interface (not shown) to relay measurement data from sensors as input to data manager 206, vehicle-based system 224, or other components of vehicle 102.

[0039] The vehicle-based system interface 222 can transmit the corrected attitude to the vehicle-based system 224 or another component of the vehicle 102. Generally, the corrected attitude and corrected heading provided by the vehicle-based system interface 222 are in a format usable by the vehicle-based system 224.

[0040] The vehicle-based system 224 can operate the vehicle 102 on a road using corrected attitude and corrected heading from the locator 104. The vehicle-based system 224 may include driver assistance systems and autonomous driving systems (e.g., Traffic Jam Assist (TJA) systems, Lane Centering Assist (LCA) systems, and Level 3 / Level 4 autonomous driving (L3 / L4) systems on highways). Generally, the vehicle-based system 224 requires accurate positioning data (e.g., accuracy within ±10 cm) to meet performance requirements. As described in more detail below, the locator 104 extends the examples of providing sufficiently accurate positioning data to the vehicle-based system 224. The locator 104 corrects the position data 210 using visual detection of lane lines by effectively using the trajectory of fused data points. In this way, the locator 104 can handle lane changes, sharp curves in the road, and poor satellite reception.

[0041] The vehicle-based system 224 can move the vehicle 102 to a specific location on the road while operating the vehicle 102 based on a calibrated attitude provided by the locator 104. The autonomous driving system can also move the vehicle 102 to a specific location on the road to avoid collisions with objects detected by other systems on the vehicle 102 (e.g., radar systems, lidar systems) and return the vehicle 102 to its original navigation route.

[0042] Trajectory generation

[0043] Figure 3 A sample concept diagram 300 illustrates how visual centerline point 106 and map centerline point 108 are managed as inputs for vehicle positioning. The locator 104 acquires and manages visual centerline point 106 and map centerline point 108 to generate a vehicle trajectory.

[0044] As vehicle 102 travels along the road, locator 104 acquires a visual left lane line 302 and a visual right lane line 304. The visual left lane line 302 and visual right lane line 304 indicate the current lateral offset of the left and right lane lines relative to vehicle attitude 308, respectively. The lateral offset is provided as a vehicle coordinate system value. Locator 104 can set the lateral offset relative to vehicle attitude 308 based on vehicle speed and steering angle.

[0045] Vehicle attitude 308 represents the estimated position of vehicle 102 in the map coordinate system based on location data 210. As described above, location data 210 can be obtained from a GPS sensor, GNSS sensor, IMU sensor, or wheel encoder system. Vehicle attitude 308 also includes heading (e.g., in radians) to allow for point transformation between the map coordinate system and the vehicle coordinate system.

[0046] The locator 104 uses a visual centerline point 106-1 to determine the lateral offset of the visual centerline relative to the vehicle's posture, where visual centerline point 106-1 represents the visual centerline point 106 of the current vehicle posture. This lateral offset of the visual centerline is transformed to a map coordinate system to determine the longitudinal (e.g., x) and lateral (e.g., y) coordinates to be sampled for addition to the vehicle's trajectory. Because the locator 104 does not focus on processing lane lines in front of the vehicle 102, the described vehicle localization is robust to errors in visual processing of lane line trajectories.

[0047] The locator 104 maintains the visual centerline point 106 as a visual trajectory point. The visual centerline point 106 is generated by lateral offset from the visual centerline. The visual trajectory points are maintained in a map coordinate system and are spaced approximately evenly based on the distance traveled by the vehicle 102. For example, the visual centerline point 106 can be sampled based on a specified distance traveled by the vehicle 102 instead of at specific clock intervals.

[0048] The locator 104 also maintains the map centerline points 108 as map trajectory points. The map centerline points 108 are sparsely spaced in the map coordinate system, and their density is generally inversely proportional to the curvature of the road. The locator 104 determines interpolated map centerline points 306 based on the map centerline points 108. In contrast to the map centerline points 108, the interpolated map centerline points 306 are generally evenly spaced in the map coordinate system and paired with their corresponding visual centerline points 106.

[0049] The locator 104 uses an interpolated map centerline point 306-1 to determine the lateral offset of the map centerline relative to the vehicle's attitude. The interpolated map centerline point 306-1 represents the interpolated map centerline point 306 of the current vehicle attitude 308. The longitudinal position of the interpolated map centerline point 306-1 is determined to correspond to the longitudinal position of the visual centerline point 106-1. The lateral position of the interpolated map centerline point 306-1 is determined by interpolation between the two nearest map centerline points 108. The lateral offset of the map centerline is either in or translated into the map coordinate system and is paired with the associated visual centerline point 106-1.

[0050] The locator 104 maintains pairs of visual centerline points 106 and interpolated map centerline points 306 as vehicle trajectories. In this way, the locator 104 uses a single trajectory of sampled centerline points from map and visual data to locate the vehicle. As the vehicle 102 travels along the road, vehicle trajectory points accumulate. The locator 104 can analyze the length of the vehicle trajectory and the gaps within it to determine whether the vehicle trajectory is effective for determining the corrected vehicle position. The locator 104 can determine whether the length of the vehicle trajectory is less than a length threshold. The length threshold can be a configurable distance. For example, the length threshold can be dynamically adjusted based on vehicle speed and road curvature to achieve shorter trajectories on road segments with curves and longer trajectories on straight road segments. The locator 104 can consider vehicle trajectories shorter than the length threshold as invalid for position correction and can wait for the application of lateral or longitudinal corrections until the vehicle trajectory is effective. If the vehicle trajectory is longer than the length threshold, the vehicle trajectory is sufficient for corrected position determination.

[0051] The locator 104 can also maintain a list of filtered vehicle trajectory points, indicating when trajectory points are skipped based on filtering criteria. Trajectory points can be filtered when transients exist in the visual centerline point 106 (e.g., during lane changes). If the gaps in a vehicle trajectory exceed a gap threshold, the locator 104 can mark the vehicle trajectory as invalid and can wait for lateral or longitudinal corrections to be applied until the vehicle trajectory is valid. Vehicle trajectories may exceed the gap threshold due to persistent issues that cause trajectory points to be filtered out (e.g., temporary loss or blockage of lane markings).

[0052] Attitude correction

[0053] Figure 4 An example concept diagram 400 is shown, illustrating a software method for locating vehicles using map data and visual data according to the techniques described in this disclosure. Specifically, concept diagram 400 illustrates a software method for managing trajectory points and applying attitude or position corrections. Figure 1 and Figure 2 The locator 104 or another component can perform the software method shown in concept diagram 400.

[0054] The locator 104 can execute a software method at a loop rate to determine attitude correction and locate the vehicle 102. For example, the locator 104 can process map data 208, location data 210, and visual data 212 with a loop time of no more than 20 milliseconds for the concept map 400.

[0055] At 402, locator 104 determines the lateral and longitudinal errors associated with map centerline point 108 and visual centerline point 106. Locator 104 can determine the lateral error relative to vehicle attitude 308. The lateral error can be expressed in meters. When executing the software method, locator 104 determines attitude corrections that minimize the lateral and longitudinal errors between map data 208 and visual data 212. Locator 104 maintains the trajectory of visual centerline point 106 and map centerline point 108, as well as the interpolated map centerline 306, to infer the shape of the road and minimize both the lateral and longitudinal errors. Locator 104 can use an Nth-order polynomial equation to represent the centerline of the road segment starting from vehicle attitude 308, where the lateral error represents a first coefficient or offset relative to vehicle attitude 308.

[0056] At 404, locator 104 determines the changes or derivatives of the lateral and longitudinal errors associated with the map centerline 108. Specifically, locator 104 determines the change in map visual error between the current and previous values. This change is used to detect transient conditions indicating a significant deviation between the map centerline and the visual centerline. Locator 104 primarily uses the change to detect lane changes indicating a change in the sign of the lateral offset of the visual centerline (e.g., from negative to positive).

[0057] At 406, locator 104 determines the distance traveled by vehicle 102. Locator 104 determines the distance traveled since the last sampling of the map and visual lateral offset to generate trajectory points. The traveled distance can be based on the Euclidean distance between the currently estimated vehicle attitude 308 (e.g., from location data 210) and the corrected vehicle attitude of the most recent trajectory point.

[0058] At point 408, locator 104 determines the recapture distance based on a configurable distance between trajectory points (e.g., 6 meters when traveling at 75 mph). The configurable distance can be extended to allow skipping trajectory points under transient conditions such as lane changes. (Reference) Figure 5 Operation 408 is described in more detail.

[0059] At point 410, when the distance traveled by vehicle 102 exceeds the re-acquisition distance, locator 104 captures the trajectory point. (Reference) Figure 6 and Figure 7 The operation of creating trajectory points is described in more detail 410.

[0060] At 412, the locator 104 can reset the trajectory under specific conditions where sensor data is invalid or a significant transient occurs. In this case, the trajectory can be considered invalid and reset.

[0061] At 414, the locator 104 applies a correction to the vehicle attitude 308. When the trajectory is valid for both length and clearance, the locator 104 applies the attitude correction to the vehicle attitude 308 to generate a corrected vehicle attitude. In this way, the locator 104 can use map data 208 and visual data 212 to generate map trajectories and visual trajectories, and locate the vehicle 102.

[0062] Figure 5 A sample concept diagram 500 illustrates a software method for determining recapture distance as part of vehicle localization using map data and visual data. Specifically, concept diagram 500 shows the implementation... Figure 4 The software method for operation 408 (e.g., determining the recapture distance). Figure 1 and Figure 2 The locator 104 or another component can execute the software method shown in concept diagram 500.

[0063] At 502, positioner 104 determines the lateral error or the change in lateral error (e.g., from...). Figure 4 The conditions (402 and 404) are invalid for capturing trajectory points during transient conditions (e.g., during lane changes) if the lateral error or its variation exceeds a configurable threshold.

[0064] At 504, if an invalid condition exists, the locator 104 determines a new recapture distance by adding a configurable transient gap distance (e.g., in meters) to the current distance traveled by the vehicle 102.

[0065] At point 506, locator 104 terminates the software method for determining the reacquisition distance and returns to the reference. Figure 4 The software method described.

[0066] At point 508, if no invalid condition exists, the locator 104 determines whether visual data 212 is valid or sufficient to determine the visual centerline point 106 based on both the left and right lane lines. If visual data 212 is valid, the locator 104 proceeds to operation 506 and terminates the software method for determining the recapture distance.

[0067] At 510, if visual data 212 is not valid, locator 104 determines a new recapture distance by adding a configurable visual gap distance (e.g., in meters) to the current distance traveled by vehicle 102.

[0068] Figure 6 A sample concept diagram 600 illustrates a software method for capturing trajectory points using map data and visual data as part of vehicle localization. Specifically, concept diagram 600 shows the implementation... Figure 4 Software methods for operations 410 (e.g., capturing trajectory points). Figure 1 and Figure 2 The locator 104 or another component can perform the software method shown in concept diagram 600.

[0069] At 602, the locator 104 determines whether the vehicle 104 has traveled more than the sampling distance (e.g., six meters for highway speed) since the latest trajectory point. If so, the data on the vehicle's attitude is used as a candidate for recording as a trajectory point.

[0070] At point 604, if vehicle 102 has not yet traveled beyond the sampling distance, locator 104 terminates the software method for capturing trajectory points and returns to the reference. Figure 4 The software method described. At point 606, if the vehicle 102 has traveled beyond the sampling distance, the locator 104 resets the distance of the last trajectory point to allow for the next sampling.

[0071] At 608, the locator 104 determines whether the vehicle 102 has traveled beyond the recapture distance. The locator 104 may default to setting the recapture distance to the last traveled distance. If transient or error conditions exist, the locator 104 may adjust the recapture distance. If the vehicle 102 has not yet traveled beyond the recapture distance, the locator 104 proceeds to operation 604 and terminates the software method for capturing trajectory points.

[0072] If vehicle 102 has traveled beyond the recapture distance, locator 104 samples the data and creates trajectory points. At 610, locator 104 transforms the visual lateral offset into map coordinates. The visual lateral offset is typically in the vehicle coordinate system (e.g., relative to vehicle 102), and locator 104 translates and rotates it into the map coordinate system.

[0073] At point 612, locator 104 transforms the map lateral offset into map coordinates. The map lateral offset represents the offset of vehicle 102 from the centerline interpolated between map centerline points 108. Locator 104 translates and rotates the map lateral offset into the map coordinate system.

[0074] At position 614, locator 104 creates a fused trajectory point that stores the visual lateral offset and map lateral offset in map coordinates. The distance traveled is also stored to support distance determination and trajectory length management. Each fused trajectory point is added to a queue to maintain the fused trajectory in memory.

[0075] At position 616, locator 104 updates the fused trajectory by managing the trajectory length and determining the match between map data and visual data. (Reference) Figure 7 The operation of updating the trajectory is described in more detail in section 616.

[0076] Figure 7 A sample concept diagram 700 illustrates a software method for updating trajectories using map data and visual data as part of vehicle localization. Specifically, concept diagram 700 shows the implementation... Figure 6 The software method for operation 616 (e.g., updating the trajectory). Figure 1 and Figure 2 The locator 104 or another component can perform the software method shown in concept diagram 700.

[0077] At 702, the locator 104 adds trajectory points from the fused data to the end of the vector or array of trajectory points.

[0078] At 704, locator 104 truncates the trajectory. When the distance of the earliest trajectory point (e.g., the first trajectory point in the vector or array of trajectory points) exceeds the configurable trajectory distance, locator 104 removes the earliest trajectory point from the vector or array of trajectory points.

[0079] At point 706, locator 104 determines whether the trajectory is valid. Specifically, locator 104 determines whether the trajectory length is longer than a configurable value and whether the gap between the two most recent trajectory points is less than another configurable value. If both conditions are met, the trajectory is considered valid.

[0080] At point 708, if locator 104 determines that the trajectory is not valid, locator 104 terminates the software method for updating the trajectory and returns to the reference. Figure 6 The software method described.

[0081] At 710, if the locator 104 determines that the trajectory is valid, then the locator 104 determines the minimum error point match between the latest visual centerline point 106 and the map centerline point 108. The locator 104 compares the visual centerline point 106 with the map centerline point 108 to determine the transformation used to minimize the error between the map data and the visual data.

[0082] Example Method

[0083] Figure 8 An example flowchart is shown as an example process for performing vehicle positioning using map data and visual data. Flowchart 800 is shown as multiple sets of operations (or actions) to be performed, but is not limited to the order or combination of operations shown herein. Furthermore, one or more operations may be repeated, combined, or rearranged to provide other methods. References can be made in the various sections discussed below. Figures 1 to 7 The locator 104 and the entities detailed therein are referred to by way of example only. This technique is not limited to being performed by one or more entities.

[0084] At point 802, the map centerline point and visual centerline point of the lane on the road in which the main vehicle is traveling are obtained. For example, locator 104 can obtain visual centerline point 106 and map centerline point 108 of vehicle 102. Visual centerline point 106 can be obtained from visual data 212 of the vision-based system of vehicle 102. Map centerline point 108 can be obtained from a map database including map data 208. Both visual centerline point and map centerline point 108 can be obtained in a map coordinate system.

[0085] The locator 104 can obtain the visual center line point 106 by using visual data 212 to determine the left and right lane lines of the road. The visual center line point 106 can then be determined as the lateral center line point between the left and right lane lines.

[0086] The locator 104 can obtain the map centerline point 108 by using map data 208 in the map database to determine the two nearest database centerline points from the lane to the vehicle's location. The locator 104 can interpolate the two nearest database centerline points to obtain the map centerline point 108. The longitudinal position of the map centerline point 108 along the road is approximately equal to the longitudinal position of the visual centerline point 106.

[0087] At position 804, the location of the main vehicle can be obtained. For example, locator 104 can use location data 210 to obtain the location of vehicle 102. Location data 210 can be obtained from at least one of Global Positioning System (GPS), Global Navigation Satellite System (GNSS), or Inertial Measurement Unit (IMU). The vehicle's location can be obtained in a map coordinate system. Locator 104 can also use location data 210 to obtain the vehicle's heading.

[0088] The locator 104 can transform the map centerline point 108 and the visual centerline point 106 from the map coordinate system to the vehicle coordinate system. The vehicle coordinate system is relative to the position and heading of the vehicle.

[0089] At point 806, a comparison between the map centerline point and the visual centerline point can be performed to generate lateral and longitudinal corrections relative to the position of the primary vehicle. For example, locator 104 can compare map centerline point 108 and visual centerline point 106 to generate lateral and longitudinal corrections relative to the position of vehicle 102. Locator 104 can match map centerline point 108 and visual centerline point 106 by applying a least-squares fitting algorithm.

[0090] Lateral and longitudinal corrections can be obtained in the vehicle coordinate system. The locator can transform the lateral and longitudinal corrections to the map coordinate system. The locator 104 can also generate a heading correction for the vehicle 102 based on the map between the map centerline point 108 and the visual centerline point 106.

[0091] At 808, lateral and longitudinal corrections can be applied to the position of the main vehicle to provide a corrected position of the main vehicle. For example, locator 104 can apply lateral and longitudinal corrections (e.g., add or subtract) to the position of vehicle 102 to generate a corrected position. Locator 104 can also apply heading corrections to the heading of the vehicle to provide a corrected heading.

[0092] The locator 104 can add the visual centerline point 106 to the visual trajectory. The visual trajectory does not include the visual centerline point 106 located longitudinally in front of the vehicle 102 along the road. The locator 104 can also add the map centerline point 108 to the map trajectory. The locator 104 can maintain the map trajectory and the visual trajectory in the data manager 206 or another component.

[0093] At 810, the primary vehicle can be operated in the road based on the calibrated location. For example, locator 104 can provide the calibrated location to vehicle-based system 224, which uses the calibrated location to locate and operate vehicle 102 in the road.

[0094] Example

[0095] Examples are provided in the following sections.

[0096] Example 1. A method comprising: obtaining a map centerline point and a visual centerline point of a lane of a road in which a primary vehicle is traveling, the map centerline point being obtained from a map database and the visual centerline point being obtained from visual data from a vision-based system; obtaining the position of the primary vehicle; comparing the map centerline point and the visual centerline point to generate lateral and longitudinal corrections relative to the position of the primary vehicle; applying the lateral and longitudinal corrections to the position of the primary vehicle to provide a corrected position of the primary vehicle; and operating the primary vehicle in the road based on the corrected position.

[0097] Example 2. The method of Example 1, wherein the map center line point, the visual center line point, and the position of the main vehicle are obtained in the map coordinate system.

[0098] Example 3. The method of Example 2, which further includes: transforming the map centerline point and the visual centerline point from the map coordinate system to the vehicle coordinate system, the vehicle coordinate system being relative to the position and heading of the main vehicle.

[0099] Example 4. A method of any of the preceding examples, wherein obtaining the visual centerline point of a lane includes: using visual data to determine the left lane line and the right lane line of the road lane; and determining the visual centerline point of the lane as the lateral centerline point between the left lane line and the right lane line.

[0100] Example 5. A method of any of the preceding examples, wherein obtaining the map centerline point of the lane comprises: using a map database to determine the two nearest database centerline points of the lane to the main vehicle; and interpolating the two nearest database centerline points to obtain the map centerline point, the map centerline point being approximately equal to the longitudinal position of the visual centerline point along the road.

[0101] Example 6. A method of any of the preceding examples, the method further comprising: adding a map centerline point to a map trajectory; adding a visual centerline point to a visual trajectory; and maintaining the map trajectory and the visual trajectory.

[0102] Example 7. The method of Example 6, wherein: the map trajectory and the visual trajectory are maintained as vehicle trajectories; and the method further includes: determining the length of the vehicle trajectory from the position of the primary vehicle; determining whether one or more gaps in the vehicle trajectory are greater than a gap threshold; and in response to a trajectory length greater than a trajectory length threshold and one or more gaps less than a gap threshold, marking the vehicle trajectory as valid and applying lateral and longitudinal corrections to the position of the primary vehicle to provide a corrected position, the trajectory length threshold being adjusted based on at least one of the speed of the primary vehicle or the curvature of the road; or in response to a trajectory length less than a trajectory length threshold or one or more gaps greater than a gap threshold, marking the vehicle trajectory as invalid and waiting for the application of lateral and longitudinal corrections until the vehicle trajectory is valid.

[0103] Example 8. The method of Example 6, wherein the visual trajectory does not include the visual centerline point located longitudinally in front of the main vehicle along the road.

[0104] Example 9. The method of any of the preceding examples, wherein the position of the primary vehicle is obtained from at least one of a Global Positioning System (GPS), a Global Navigation Satellite System (GNSS), or an Inertial Measurement Unit (IMU).

[0105] Example 10. A method of any of the foregoing examples, the method further comprising: obtaining the heading of the primary vehicle; generating a heading correction of the primary vehicle based on a comparison of a map centerline point and a visual centerline point; and applying the heading correction to the heading of the primary vehicle to provide a corrected heading of the primary vehicle.

[0106] Example 11. The method of any of the preceding examples, wherein comparing the map centerline point and the visual centerline point includes applying a least-squares fitting algorithm to the map centerline point and the visual centerline point.

[0107] Example 12. The method of any of the preceding examples, wherein the lateral correction and longitudinal correction are obtained in a vehicle coordinate system, and the method further includes: transforming the lateral correction and longitudinal correction to a map coordinate system.

[0108] Example 13. A computer-readable storage medium including computer-executable instructions that, when executed, cause a processor in a primary vehicle to perform any of the methods described above.

[0109] Example 14. A system comprising a processor configured to perform a method of any one of Examples 1 through 12.

[0110] Conclusion

[0111] While various embodiments of the present disclosure have been described in the foregoing description and illustrated in the accompanying drawings, it should be understood that the present disclosure is not limited thereto, but can be practiced in various ways within the scope of the following claims. It will be apparent from the foregoing description that various modifications can be made without departing from the spirit and scope of the present disclosure as defined by the following claims.

Claims

1. A method for a means of transport, the method comprising: The map centerline point and visual centerline point of the lane of the road in which the main vehicle is traveling are obtained, wherein the map centerline point is obtained from a map database and the visual centerline point is obtained from visual data of a vision-based system. Add the map centerline point to the map trajectory; Add the visual center line point to the visual trajectory; Maintain the map trajectory and the visual trajectory; Obtain the location of the main vehicle; The map centerline point and the visual centerline point are compared to generate lateral and longitudinal corrections relative to the position of the main vehicle. The length of the vehicle's trajectory is determined from the position of the main vehicle; Determine whether one or more gaps in the trajectory of the vehicle are greater than a gap threshold; In response to the trajectory length being greater than a trajectory length threshold and the one or more gaps being less than the gap threshold, the vehicle trajectory is identified as valid and the lateral correction and the longitudinal correction are applied to the position of the main vehicle to provide a corrected position of the main vehicle, the trajectory length threshold being adjusted based on at least one of the speed of the main vehicle and the curvature of the road; or In response to the trajectory length being less than the trajectory length threshold or the one or more gaps being greater than the gap threshold, the vehicle trajectory is marked as invalid and awaits the application of the lateral correction and the longitudinal correction until the vehicle trajectory becomes valid; as well as Based on the corrected position, the main vehicle is operated on the road.

2. The method as described in claim 1, characterized in that, The map center line point, the visual center line point, and the position of the main vehicle are obtained in the map coordinate system.

3. The method of claim 2, further comprising: The map centerline point and the visual centerline point are transformed from the map coordinate system to the vehicle coordinate system, which is relative to the position and heading of the main vehicle.

4. The method as described in claim 1, characterized in that, Obtaining the visual centerline point of the lane includes: The visual data is used to determine the left and right lane lines of the lanes of the road; and The visual centerline point of the lane is determined as the lateral centerline point between the left lane line and the right lane line.

5. The method as described in claim 1, characterized in that, Obtaining the map centerline point of the lane includes: The map database is used to determine the two nearest database centerline points from the lane to the location of the main vehicle; and The map centerline point is obtained by interpolating the two nearest database centerline points, and the map centerline point is approximately equal to the longitudinal position of the visual centerline point along the road.

6. The method as described in claim 1, characterized in that, The visual trajectory does not include the visual centerline point located longitudinally in front of the main vehicle along the road.

7. The method as described in claim 1, characterized in that, The location of the main vehicle is obtained from at least one of the Global Positioning System (GPS), Global Navigation Satellite System (GNSS), or Inertial Measurement Unit (IMU).

8. The method of claim 1, further comprising: Obtain the heading of the main vehicle; Based on the comparison between the map centerline point and the visual centerline point, the heading correction of the main vehicle is generated; as well as The heading correction is applied to the heading of the main vehicle to provide a corrected heading for the main vehicle.

9. The method as described in claim 1, characterized in that, Comparing the map centerline point and the visual centerline point includes applying a least-squares fitting algorithm to both the map centerline point and the visual centerline point.

10. The method as described in claim 1, characterized in that, The lateral correction and the longitudinal correction are obtained in the vehicle coordinate system, and the method further includes: Transform the horizontal and vertical corrections to the map coordinate system.

11. A computer-readable storage medium comprising computer-executable instructions, which, when executed, cause a processor in a primary vehicle to: The map centerline point and visual centerline point of the lane of the road in which the main vehicle is traveling are obtained, the map centerline point is obtained from a map database, and the visual centerline point is obtained from visual data of a vision-based system. Add the map centerline point to the map trajectory; Add the visual center line point to the visual trajectory; Maintain the map trajectory and the visual trajectory; Obtain the location of the main vehicle; The map centerline point and the visual centerline point are compared to generate lateral and longitudinal corrections relative to the position of the main vehicle. The length of the vehicle's trajectory is determined from the position of the main vehicle; Determine whether one or more gaps in the trajectory of the vehicle are greater than a gap threshold; In response to the trajectory length being greater than a trajectory length threshold and the one or more gaps being less than the gap threshold, the vehicle trajectory is identified as valid and the lateral correction and the longitudinal correction are applied to the position of the main vehicle to provide a corrected position of the main vehicle, the trajectory length threshold being adjusted based on at least one of the speed of the main vehicle and the curvature of the road; or In response to the trajectory length being less than the trajectory length threshold or the one or more gaps being greater than the gap threshold, the vehicle trajectory is marked as invalid and awaits the application of the lateral correction and the longitudinal correction until the vehicle trajectory becomes valid; as well as Based on the corrected position, the main vehicle is operated on the road.

12. The computer-readable storage medium as claimed in claim 11, characterized in that, The map center line point, the visual center line point, and the position of the main vehicle are obtained in the map coordinate system.

13. The computer-readable storage medium as claimed in claim 12, characterized in that, The computer-readable storage medium includes computer-executable instructions, which, when executed, further cause the processor in the main vehicle to: The map centerline point and the visual centerline point are transformed from the map coordinate system to the vehicle coordinate system, which is relative to the position and heading of the main vehicle.

14. The computer-readable storage medium as claimed in claim 11, characterized in that, The computer-readable storage medium includes computer-executable instructions that, when executed to obtain the map centerline point of the lane, cause the processor in the primary vehicle to: The map database is used to determine the two nearest database centerline points to the location of the main vehicle; and The map centerline point is obtained by interpolating the two nearest database centerline points, and the map centerline point is approximately equal to the longitudinal position of the visual centerline point along the road.

15. The computer-readable storage medium as claimed in claim 11, characterized in that, The visual trajectory does not include the visual centerline point located longitudinally in front of the main vehicle along the road.

16. The computer-readable storage medium as claimed in claim 11, characterized in that, The computer-readable storage medium includes computer-executable instructions, which, when executed, further cause the processor in the main vehicle to: Obtain the heading of the main vehicle; Based on the comparison between the map centerline point and the visual centerline point, the heading correction of the main vehicle is generated; as well as The heading correction is applied to the heading of the main vehicle to provide a corrected heading for the main vehicle.

17. A system for a vehicle, the system comprising a processor configured to: The map centerline point and visual centerline point of the lane of the road in which the main vehicle is traveling are obtained, wherein the map centerline point is obtained from a map database and the visual centerline point is obtained from visual data of a vision-based system. Add the map centerline point to the map trajectory; Add the visual center line point to the visual trajectory; Maintain the map trajectory and the visual trajectory; Obtain the location of the main vehicle; The map centerline point and the visual centerline point are compared to generate lateral and longitudinal corrections relative to the position of the main vehicle. The length of the vehicle's trajectory is determined from the position of the main vehicle; Determine whether one or more gaps in the trajectory of the vehicle are greater than a gap threshold; In response to the trajectory length being greater than a trajectory length threshold and the one or more gaps being less than the gap threshold, the vehicle trajectory is identified as valid and the lateral correction and the longitudinal correction are applied to the position of the main vehicle to provide a corrected position of the main vehicle, the trajectory length threshold being adjusted based on at least one of the speed of the main vehicle and the curvature of the road; or In response to the trajectory length being less than the trajectory length threshold or the one or more gaps being greater than the gap threshold, the vehicle trajectory is marked as invalid and awaits the application of the lateral correction and the longitudinal correction until the vehicle trajectory becomes valid; as well as Based on the corrected position, the main vehicle is operated on the road.

Images