GNSS error quantization and global map alignment

Abstract

A method for aligning a plurality of local maps with a global coordinate system includes quantifying a Global Navigation Satellite System (GNSS) error at each of a plurality of locations in an environment. The method further includes determining a plurality of anchor points in the environment based at least in part on the GNSS error at each of the plurality of locations. The method further includes aligning the plurality of local maps with the global coordinate system based at least in part on the plurality of anchor points.

Description

Technical Field

[0001] This invention relates to systems and methods for vehicle navigation, route selection, and environmental mapping. Background Technology

[0002] To enhance occupant awareness and convenience, vehicles can be equipped with Advanced Driver Assistance Systems (ADAS) and / or Automatic Deployment Systems (ADS). ADAS systems utilize various sensors, such as cameras, radar, and lidar, to detect and identify objects around the vehicle, including other vehicles, pedestrians, road configurations, traffic signs, and road markings. ADAS systems can take actions based on environmental conditions, such as applying the brakes or alerting vehicle occupants. ADS systems can use various sensors to detect objects in the vehicle's environment and control the vehicle, navigating it to a predetermined destination. ADAS and ADS systems can also utilize the vehicle's position obtained through Global Navigation Satellite Systems (GNSS) combined with globally aligned maps for navigation route selection, path planning, lane recognition, obstacle avoidance, and more. However, current GNSS systems and map alignment methods may not account for GNSS errors introduced by variations in the regional environment, such as geographical features or tall buildings.

[0003] Therefore, while GNSS systems and map alignment methods have achieved their intended purpose, a new and improved system and method are needed for aligning multiple local maps with a global coordinate system. Summary of the Invention

[0004] According to several aspects, the present invention provides a method for aligning multiple local maps with a global coordinate system. The method may include quantifying Global Navigation Satellite System (GNSS) errors at each of a plurality of locations in an environment. The method may also include determining a plurality of anchor points in the environment based at least in part on the GNSS errors at each of the plurality of locations. The method may further include aligning the multiple local maps with the global coordinate system based at least in part on the plurality of anchor points.

[0005] In another aspect of the invention, quantizing the GNSS error at each of the multiple locations may further include collecting multiple local maps from multiple vehicles. Quantizing the GNSS error at each of the multiple locations may further include identifying multiple locations within the multiple local maps. Quantizing the GNSS error at each of the multiple locations may further include quantizing the GNSS error at each of the multiple locations based at least in part on the multiple local maps.

[0006] In another aspect of the invention, collecting multiple local maps may further include collecting multiple local maps from multiple vehicles using Simultaneous Localization and Mapping (SLAM). Each of the multiple local maps includes multiple observation points. Each of the multiple observation points includes observation data, local map coordinates, and GNSS coordinates.

[0007] In another aspect of the invention, identifying multiple locations in multiple local maps may further include identifying multiple pairs of neighboring observation points based on multiple observation points in each of the multiple local maps. Each pair of neighboring observation points includes a first observation point and a second observation point, the second observation point being located within a first predetermined radius of the first observation point in the environment, and being at least partially based on observation data from each of the multiple observation points in each of the multiple local maps. Identifying multiple locations in multiple local maps may further include identifying multiple locations. Each of the multiple locations includes a first observation point from one of the multiple pairs of neighboring observation points.

[0008] In another aspect of the invention, quantifying the GNSS error at each of the plurality of locations based at least in part on a plurality of local maps may further include: performing a relocation algorithm to determine a transformation vector between a first observation point and a second observation point. The transformation vector describes the positional differences in the environment between the first and second observation points. Quantifying the GNSS error at each of the plurality of locations based at least in part on a plurality of local maps may further include determining the GNSS error between the first and second observation points. The GNSS error is:

[0009] e GNSS =‖GNSS1-GNSS2‖-‖T‖

[0010] Among them, e GNSS Let GNSS be the GNSS error between the first and second observation points, GNSS1 be the GNSS coordinates of the first observation point, GNSS2 be the GNSS coordinates of the second observation point, and T be the transformation vector. Quantifying the GNSS error at each of the multiple locations, at least in part based on multiple local maps, may also include updating the GNSS coordinates of the first observation point, at least in part based on the transformation vector.

[0011] In another aspect of the invention, updating the GNSS coordinates of the first observation point may further include transforming the GNSS coordinates of the second observation point using a transformation vector to determine the transformed GNSS coordinates of the second observation point. Updating the GNSS coordinates of the first observation point may further include averaging the GNSS coordinates of the first observation point with the transformed GNSS coordinates of the second observation point to determine the updated GNSS coordinates of the first observation point.

[0012] In another aspect of the invention, determining a plurality of anchor points may further include determining a plurality of anchor positions. The plurality of anchor positions includes a subset of multiple positions. Determining a plurality of anchor points may further include determining a plurality of anchor points. Each of the plurality of anchor points corresponds to one of the plurality of anchor positions.

[0013] In another aspect of the invention, determining a plurality of anchor locations may further include determining a plurality of anchor locations, wherein each of the plurality of anchor locations has a GNSS error less than or equal to a predetermined error threshold.

[0014] In another aspect of the invention, determining a plurality of anchor positions may further include determining a plurality of anchor positions, wherein each of the plurality of anchor positions has a GNSS error less than or equal to a predetermined error threshold, and wherein each of the plurality of anchor positions has at least a first predetermined number of a plurality of observation points located within a second predetermined radius of each of the plurality of anchor positions.

[0015] In another aspect of the invention, aligning multiple local maps with a global coordinate system may further include performing a first-stage alignment using a first alignment algorithm and a first subset of multiple anchor points.

[0016] In another aspect of the invention, aligning multiple local maps with a global coordinate system may further include performing a second-stage alignment using a second alignment algorithm and a second subset of the multiple anchor points. The second alignment algorithm differs from the first alignment algorithm. The second subset of the multiple anchor points is smaller than the first subset of the multiple anchor points.

[0017] According to several aspects, the present invention provides a system for aligning multiple local maps with a global coordinate system. The system may include multiple vehicle sensors, including at least a vehicle global navigation satellite system (GNSS), vehicle perception sensors, and a vehicle communication system. The system may also include a vehicle controller in electrical communication with the multiple vehicle sensors. The vehicle controller is programmed to collect multiple local maps of the environment using Simultaneous Localization and Mapping (SLAM). Each of the multiple local maps includes multiple observation points. Each of the multiple observation points includes observation data, including observations made using the vehicle perception sensors, local map coordinates, and GNSS coordinates determined using the vehicle GNSS. The vehicle controller is also programmed to transmit the multiple local maps to a server system using the vehicle communication system.

[0018] In another aspect of the invention, the system further includes a server system. The server system may include a server communication system and a server controller in electrical communication with the server communication system. The server controller is programmed to receive multiple local maps using the server communication system. The server controller is also programmed to identify multiple locations within the multiple local maps. The server controller is further programmed to quantify the GNSS error at each of the multiple locations, at least in part based on the multiple local maps. The server controller is also programmed to determine multiple anchor points, at least in part based on the GNSS error at each of the multiple locations. The server controller is further programmed to align the multiple local maps with a global coordinate system, at least in part based on the multiple anchor points.

[0019] In another aspect of the invention, to identify multiple locations in multiple local maps, the server controller is further programmed to identify multiple pairs of neighboring observation points based on each of multiple observation points in the multiple local maps. Each of the multiple pairs of neighboring observation points includes a first observation point and a second observation point, the second observation point being located within a first predetermined radius of the first observation point in the environment, the first and second observation points being at least partially based on observation data from each of the multiple observation points in each of the multiple local maps. To identify multiple locations in the multiple local maps, the server controller is also programmed to identify multiple locations. Each of the multiple locations includes a first observation point from one of the multiple pairs of neighboring observation points.

[0020] In another aspect of the invention, to quantify the GNSS error at each of the multiple locations, the server controller is also programmed to execute a relocation algorithm to determine a transformation vector between a first observation point and a second observation point. The transformation vector describes the positional differences in the environment between the first and second observation points. To quantify the GNSS error at each of the multiple locations, the server controller is also programmed to determine the GNSS error between the first and second observation points. The GNSS error is:

[0021] e GNSS =‖GNSS1-GNSS2‖-‖T‖

[0022] Among them, e GNSS GNSS is the GNSS error between the first and second observation points, GNSS1 is the GNSS coordinates of the first observation point, GNSS2 is the GNSS coordinates of the second observation point, and T is the transformation vector.

[0023] In another aspect of the invention, to determine the plurality of anchor points, the server controller is also programmed to determine a plurality of anchor locations. The plurality of anchor locations comprises a subset of multiple locations. Each of the plurality of anchor locations has a GNSS error less than or equal to a predetermined error threshold. Each of the plurality of anchor locations has at least a first predetermined number of observation points located within a second predetermined radius of each of the plurality of anchor locations. To determine the plurality of anchor points, the server controller is also programmed to determine the plurality of anchor points, wherein each of the plurality of anchor points corresponds to one of the plurality of anchor locations, and wherein the GNSS coordinates of each of the plurality of anchor points are the median of the GNSS coordinates of a first observation point and a second observation point included in each of the plurality of anchor locations.

[0024] In another aspect of the invention, in order to align multiple local maps with a global coordinate system, the server controller is also programmed to perform a first-stage alignment using a first alignment algorithm and a first subset of multiple anchor points.

[0025] In another aspect of the invention, to align multiple local maps with a global coordinate system, the server controller is also programmed to perform a second-stage alignment using a second alignment algorithm and a second subset of the multiple anchor points. The second alignment algorithm differs from the first alignment algorithm. The second subset of the multiple anchor points is smaller than the first subset of the multiple anchor points.

[0026] According to several aspects, the present invention provides a method for aligning multiple local maps with a global coordinate system. The method may include: collecting multiple local maps of an environment from multiple vehicles using Simultaneous Localization and Mapping (SLAM). Each of the multiple local maps includes multiple observation points. Each of the multiple observation points includes observation data, local map coordinates, and GNSS coordinates. The method may further include: identifying multiple pairs of neighboring observation points based on the multiple observation points of each of the multiple local maps. Each of the multiple pairs of neighboring observation points includes a first observation point and a second observation point, the second observation point being located within a first predetermined radius of the first observation point in the environment, the first and second observation points being at least partially based on observation data of each of the multiple observation points of each of the multiple local maps. The method may further include identifying multiple locations. Each of the multiple locations includes a first observation point of one of the multiple pairs of neighboring observation points. The method may further include performing a relocation algorithm to determine a transformation vector between the first and second observation points. The transformation vector describes the positional differences in the environment between the first and second observation points. The method may further include determining the GNSS error at each of the multiple locations. The GNSS error at one of the multiple locations is:

[0027] e GNSS =‖GNSS1-GNSS2‖-‖T‖

[0028] Among them, e GNSS GNSS is the GNSS error between the first and second observation points, GNSS1 is the GNSS coordinates of the first observation point, GNSS2 is the GNSS coordinates of the second observation point, and T is the transformation vector.

[0029] In another aspect of the invention, the method may further include determining a plurality of anchor locations. The plurality of anchor locations comprises a subset of multiple locations. Each of the plurality of anchor locations has a GNSS error less than or equal to a predetermined error threshold. The method may further include determining a plurality of anchor points. Each of the plurality of anchor points corresponds to one of the plurality of anchor locations. The GNSS coordinates of each of the plurality of anchor points are the median of the GNSS coordinates of a first observation point and a second observation point included in each of the plurality of anchor locations.

[0030] In another aspect of the invention, the method may further include performing a first-stage alignment using a first alignment algorithm and a first subset of the plurality of anchor points. The method may also include performing a second-stage alignment using a second alignment algorithm and a second subset of the plurality of anchor points. The second alignment algorithm differs from the first alignment algorithm. The second subset of the plurality of anchor points is smaller than the first subset of the plurality of anchor points.

[0031] Further areas of application will become apparent from the description provided herein. It should be understood that these descriptions and specific examples are for illustrative purposes only and are not intended to limit the scope of the invention. Attached Figure Description

[0032] The accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of the invention in any way.

[0033] Figure 1 This is a schematic diagram of a system for aligning multiple local maps with a global coordinate system according to an exemplary embodiment;

[0034] Figure 2 This is a flowchart of a method for aligning multiple local maps with a global coordinate system according to an exemplary embodiment; and

[0035] Figure 3 This is a schematic diagram of multiple local maps in an environment according to an exemplary embodiment. Detailed Implementation

[0036] The following description is merely exemplary in nature and is not intended to limit the invention, application, or use.

[0037] In various aspects of this invention, the accuracy of location information determined using Global Navigation Satellite Systems (GNSS) is increasingly important for automobiles and vehicle use cases, including, for example, navigation route selection, Advanced Driver Assistance Systems (ADAS), and Automatic Deployment Systems (ADS). However, GNSS data can contain significant noise or steady-state errors caused by regional environmental factors such as tall building obstructions (also known as “urban canyons”). Therefore, in various aspects of this invention, it is advantageous to quantify GNSS errors at specific locations for constructing and aligning globally aligned maps for vehicle applications. Accordingly, this invention provides a novel and improved system and method for aligning multiple local maps to a global coordinate system, including the accurate quantification of GNSS errors at specific locations even in the absence of known reference points or ground fact information.

[0038] refer to Figure 1 A system for aligning multiple local maps with a global coordinate system is illustrated, and the system is generally indicated by reference numeral 10. System 10 generally includes a vehicle system 12a and a server system 12b. Vehicle system 12a is shown together with an exemplary vehicle 14. Although a passenger vehicle is shown, it should be understood that vehicle 14 can be any type of vehicle without departing from the scope of the invention. Vehicle system 12a typically includes a vehicle controller 16 and multiple vehicle sensors 18.

[0039] The vehicle controller 16 includes at least one processor 20 and a non-transitory computer-readable storage device or medium 22. The processor 20 may be a custom or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor among several processors associated with the vehicle controller 16, a semiconductor-based microprocessor (in the form of a microchip or chipset), a macroprocessor, a combination thereof, or generally a means for executing instructions.

[0040] Computer-readable storage device or medium 22 may include volatile and non-volatile storage such as read-only memory (ROM), random access memory (RAM), and keep-alive memory (KAM). KAM is a persistent or non-volatile memory that can be used to store various operational variables when processor 20 is powered off. Computer-readable storage device or medium 22 may be implemented through a variety of storage devices such as PROM (programmable read-only memory), EPROM (electrical PROM), EEPROM (electrically erasable PROM), flash memory, or another electrical, magnetic, optical, or combined storage device capable of storing data, some of which represents executable instructions that can be used by vehicle controller 16 to control various systems of vehicle 12.

[0041] The vehicle controller 16 may also consist of multiple controllers that are electrically communicating with each other. The vehicle controller 16 may interconnect with other systems and / or controllers of the vehicle 12, thereby allowing the vehicle controller 16 to access data such as the speed, acceleration, braking and steering angle of the vehicle 12.

[0042] The vehicle controller 16 communicates electrically with a plurality of vehicle sensors 18. In one exemplary embodiment, electrical communication is established using, for example, a CAN network, a FLEXRAY network, a local area network (e.g., WiFi, Ethernet, etc.), a Serial Peripheral Interface (SPI) network, etc. It should be understood that various other wired and wireless technologies and communication protocols for communicating with the vehicle controller 16 are within the scope of this invention. It should also be understood that, within the scope of this invention, electrical communication also includes the transfer of power and / or energy between electrical devices (e.g., using wired and / or wireless power transmission technologies).

[0043] Multiple vehicle sensors 18 are used to acquire information related to the vehicle 14. In one exemplary embodiment, the multiple vehicle sensors 18 include a vehicle global navigation satellite system (GNSS) 24, a vehicle perception sensor 26, a vehicle communication system 28, and a vehicle inertial measurement unit (IMU) 30.

[0044] Vehicle GNSS 24 is used to determine the geographic location of vehicle 14. In one exemplary embodiment, vehicle GNSS 24 is a Global Positioning System (GPS). In a non-limiting example, the GPS includes a GPS receiver antenna (not shown) and a GPS controller (not shown) in electrical communication with the GPS receiver antenna. The GPS receiver antenna receives signals from multiple satellites, and the GPS controller calculates the geographic location of vehicle 14 based on the signals received by the GPS receiver antenna. In one exemplary embodiment, vehicle GNSS 24 also includes a map. The map includes information about infrastructure, such as municipal boundaries, roads, railways, sidewalks, buildings, etc. Therefore, map information is used to contextualize the geographic location of vehicle 14. In one non-limiting example, the map is obtained from a remote source using a wireless connection. In another non-limiting example, the map is stored in a database of vehicle GNSS 24. It should be understood that various other types of satellite-based radio navigation systems, such as the Global Positioning System (GPS), Galileo system, GLONASS, and BeiDou Navigation Satellite System (BDS), are within the scope of this invention. Vehicle GNSS 24 is in electrical communication with vehicle controller 16 as described above.

[0045] Vehicle perception sensor 26 is used to sense objects in the environment 32 surrounding vehicle 14 and / or measure distances. In one exemplary embodiment, vehicle perception sensor 26 includes a surround-view camera system comprising multiple cameras (also referred to as satellite cameras) arranged to provide views of the environment 32 adjacent to all sides of vehicle 14. In a non-limiting example, the camera system includes a front-facing camera (e.g., mounted in the front grille of vehicle 14), a rear-facing camera (e.g., mounted on the tailgate of vehicle 14), and two side cameras (e.g., mounted below each of the two side mirrors of vehicle 14). In another non-limiting example, the camera system further includes an additional rear-view camera mounted near the center high-mounted brake light of vehicle 14. It should be understood that camera systems with other cameras and / or other mounting locations are within the scope of this invention.

[0046] In another exemplary embodiment, the vehicle perception sensor 26 includes a stereo camera with distance measurement capabilities. In one example, the vehicle perception sensor 26 is fixed inside the vehicle 14, for example, in the roof lining of the vehicle 14, and has a view through the windshield of the vehicle 14. In another example, the vehicle perception sensor 26 is fixed outside the vehicle 14, for example, on the roof of the vehicle 14, and has a view of the environment 32 surrounding the vehicle 14.

[0047] It should be understood that various additional types of sensing devices, such as LiDAR sensors, ultrasonic ranging sensors, radar sensors, and / or time-of-flight sensors, are within the scope of this invention. The vehicle perception sensor 26 is in electrical communication with the vehicle controller 16, as described above.

[0048] Vehicle controller 16 uses vehicle communication system 28 to communicate with other systems outside vehicle 14. For example, vehicle communication system 28 has the capability to communicate with vehicles (“V2V” communication), infrastructure (“V2I” communication), remote systems at remote call centers (e.g., General Motors’ ON-STAR), and / or personal devices. Generally, the term vehicle-to-everything (“V2X” communication) refers to communication between vehicle 14 and any remote system (e.g., vehicles, infrastructure, and / or remote systems).

[0049] In some embodiments, the vehicle communication system 28 is a wireless communication system configured to communicate via a wireless local area network (WLAN) using the IEEE 802.11 standard or by using cellular data communication (e.g., using GSMA standards such as SGP.02, SGP.22, SGP.32, etc.). Therefore, the vehicle communication system 28 may also include an embedded universal integrated circuit card (eUICC) configured to store at least one cellular connectivity configuration profile, such as an embedded subscriber identity module (eSIM) profile.

[0050] The vehicle communication system 28 is also configured to communicate via a personal area network (e.g., Bluetooth), near field communication (NFC), and / or any additional type of radio frequency communication. However, additional or alternative communication methods, such as Dedicated Short Range Communication (DSRC) channels and / or mobile telecommunications protocols based on 3GPP standards, are also considered to be within the scope of this invention. A DSRC channel refers to a unidirectional or bidirectional short- to medium-range wireless communication channel designed specifically for automotive use, along with a set of corresponding protocols and standards. 3GPP refers to a partnership among multiple standards organizations that develop mobile telecommunications protocols and standards. 3GPP standards are structured as “versions.” Therefore, communication methods based on 3GPP versions 14, 15, 16, and / or future 3GPP versions are considered to be within the scope of this invention.

[0051] Therefore, the vehicle communication system 28 may include one or more antennas and / or communication transceivers for receiving and / or transmitting signals, such as Cooperative Sensing Messages (CSM). The vehicle communication system 28 is configured to wirelessly transmit information between vehicle 14 and another vehicle. Furthermore, the vehicle communication system 28 is configured to wirelessly transmit information between vehicle 14 and infrastructure or other vehicles. It should be understood that, without departing from the scope of the invention, the vehicle communication system 28 may be integrated with the vehicle controller 16 (e.g., on the same circuit board as the vehicle controller 16 or otherwise as part of the vehicle controller 16). The vehicle communication system 28 communicates electrically with the vehicle controller 16, as described above.

[0052] The vehicle IMU 30 is used to determine the direction of action, velocity, and gravity on the vehicle 14. In one exemplary embodiment, the vehicle IMU 30 includes several sensors, including an accelerometer, a gyroscope, and / or a magnetometer. In a non-limiting example, the vehicle IMU 30 includes a three-axis accelerometer and a three-axis gyroscope integrated as a single unit. The accelerometer measures linear acceleration along each axis, while the gyroscope measures angular velocity about each axis. The vehicle IMU 30 processes data from the sensors to calculate the current orientation, velocity, heading, yaw rate (i.e., rate of change of heading), and acceleration of the vehicle 14 in three-dimensional space. The vehicle IMU 30 is in electrical communication with the vehicle controller 16, as described above.

[0053] It should be understood that the plurality of vehicle sensors 18 may include other sensors without departing from the scope of the invention. In one exemplary embodiment, the plurality of vehicle sensors 18 may also include sensors for determining performance data about the vehicle 14. In a non-limiting example, the plurality of vehicle sensors 18 may also include at least one of a motor speed sensor, a motor torque sensor, an electric drive motor voltage and / or current sensor, an accelerator pedal position sensor, a brake position sensor, a coolant temperature sensor, a cooling fan speed sensor, and a transmission fluid temperature sensor.

[0054] In another exemplary embodiment, the plurality of vehicle sensors 18 further include sensors for determining information about the environment within the vehicle 14. In a non-limiting example, the plurality of vehicle sensors 18 also include at least one of a seat occupancy sensor, a cabin air temperature sensor, a cabin motion detection sensor, a cabin camera, a cabin microphone, etc.

[0055] In another exemplary embodiment, the plurality of vehicle sensors 18 further include sensors for determining information about the environment 32 surrounding the vehicle 14. In a non-limiting example, the plurality of vehicle sensors 18 also include at least one of an ambient air temperature sensor, an atmospheric pressure sensor, and / or a photographic camera and / or a video camera, positioned to observe the environment 32 in front of the vehicle 14. The plurality of vehicle sensors 18 are in electrical communication with the vehicle controller 16, as described above.

[0056] Continue to refer to Figure 1 The figure shows server system 12b, which is generally indicated by reference numeral 12b. Server system 12b generally includes server controller 40 that is in electrical communication with server database 42 and server communication system 44. In a non-limiting example, server system 12b is located in a server cluster, data center, etc., and is connected to the Internet.

[0057] Server controller 40 is used to implement method 100 for aligning multiple local maps with a global coordinate system, as described below. Server controller 40 includes at least one server processor 46 and a server non-transitory computer-readable storage device or server medium 48. The description of the type and configuration given above for vehicle controller 16 also applies to server controller 40. In some examples, server controller 40 differs from vehicle controller 16 in that server controller 40 can have higher processing speeds, include more memory, include more inputs / outputs, etc. In a non-limiting example, the server processor 46 and server medium 48 of server controller 40 are structurally and / or functionally similar to the processor 20 and medium 22 of vehicle controller 16, as described above.

[0058] Server database 42 is used to store data received from vehicle 14, including, for example, maps, information about lane boundaries, road geometry, speed limits, traffic signs, and / or other relevant features. Although Figure 1 A single vehicle 14 is shown, but it should be understood that server system 12b can communicate with multiple vehicles (not shown) to crowdsource data, as discussed in more detail below. In one exemplary embodiment, server database 42 includes one or more mass storage devices, such as hard disk drives, tape drives, magneto-optical drives, optical disks, solid-state drives, and / or additional means operable to store data in a persistent and machine-readable manner. In some examples, the one or more mass storage devices may be configured to provide redundancy in the event of hardware failure and / or data corruption, for example, using a redundant array of independent disks (RAID). In a non-limiting example, server controller 40 may execute software such as a database management system (DBMS), thereby allowing the organization and access of data stored on one or more mass storage devices. Server database 42 is in electrical communication with server controller 40.

[0059] Server communication system 44 is used to communicate with external systems (e.g., vehicle controller 16) via vehicle communication system 28. In a non-limiting example, server communication system 44 is structurally and / or functionally similar to vehicle communication system 28 as described above. In some examples, server communication system 44 differs from vehicle communication system 28 in that server communication system 44 is capable of higher power signal transmission, more sensitive signal reception, higher bandwidth transmission, additional transmission / reception protocols, etc. Server communication system 44 communicates electrically with server controller 40.

[0060] refer to Figure 2 A flowchart of a method 100 for aligning multiple local maps with a global coordinate system is shown. Method 100 begins at box 102 and proceeds to box 104. At box 104, server system 12b collects multiple local maps 50 from multiple vehicles (not shown) (including, for example, vehicle 14) in environment 32. Figure 3 ).

[0061] refer to Figure 3A schematic diagram of multiple local maps 50 is shown. The multiple local maps 50 include a first local map 52a and a second local map 52b. In one exemplary embodiment, the first local map 52a and the second local map 52b are collected by multiple vehicle sensors 18 of vehicle 14 during two separate driving passes through the same area of ​​environment 32. In another exemplary embodiment, the first local map 52a is collected by multiple vehicle sensors 18 of vehicle 14, and the second local map 52b is collected by multiple vehicle sensors of another vehicle (not shown) among multiple vehicles (not shown) driving through the same area of ​​environment 32. In the following disclosure, for illustrative purposes, the collection of the multiple local maps 50 will be discussed from the perspective of vehicle 14 using vehicle system 12a. It should be understood that in some embodiments, one or more of the multiple local maps 50 are collected by one or more of multiple vehicles (not shown) in environment 32, and the multiple vehicles (not shown) are equipped with systems having similar or equivalent structure and / or function to vehicle system 12a.

[0062] In one exemplary embodiment, each of the plurality of local maps 50 includes a plurality of observation points 54. Each of the plurality of observation points 54 represents an observation of environment 32 from a specific location within environment 32. Each of the plurality of observation points 54 includes observation data collected by a plurality of vehicle sensors, local map coordinates oriented within one of the plurality of local maps 50, and GNSS coordinates measured at the location of observation point 54 using vehicle GNSS 24.

[0063] Simultaneous Localization and Mapping (SLAM) algorithms are used to determine the local map coordinates of each of a plurality of observation points 54. In the following disclosure, the SLAM algorithm will be discussed from the perspective of a vehicle system 12a used to generate one of the plurality of local maps 50. However, it should be understood that the following disclosure applies to any of the plurality of vehicles (not shown). The vehicle controller 16 uses the SLAM algorithm to simultaneously determine the position of vehicle 14 in environment 32 while also constructing a local map of environment 32. In one exemplary embodiment, the SLAM algorithm employs observation data collected by a plurality of vehicle sensors 18 (e.g., vehicle perception sensors 26 and / or vehicle IMU 30) to achieve this function. In a non-limiting example, the SLAM algorithm includes four software components: a localization module, a map building module, a location recognition module, and a relocalization module.

[0064] The localization module uses observation data to estimate the pose (position and orientation) of vehicle 14 relative to objects in environment 32 (e.g., lane lines, road edges, landmarks, structures, trees, etc.). The localization module employs techniques such as Bayesian filtering or Kalman filtering to fuse the observation data and accurately predict the current state of vehicle 14. Simultaneously, the map building module constructs a local map of environment 32 based on observation data and odometry data collected during motion (e.g., visual data from perception sensor 26 and inertial / odometer data from vehicle IMU 30). The map building module incrementally constructs and updates the local map by capturing multiple observation points 54 and calculating the local map coordinates of each of the multiple observation points 54 relative to the local map coordinate system.

[0065] The location recognition module is responsible for identifying nearby locations within environment 32 based on observation data. The location recognition module executes a location recognition algorithm to compare two sets of observation data (e.g., current observation data observed by vehicle 14 and observation data stored in a local map) to identify familiar locations or landmarks. In a non-limiting example, the location recognition algorithm identifies the similarity between two sets of observation data and quantifies the degree of similarity between them. If the similarity is greater than a predetermined threshold, the two sets of observation data are determined to be close in location (e.g., within a first predetermined radius of each other, such as within ten meters). After identifying nearby locations, a relocation module is used.

[0066] The relocation module is used to determine a transformation vector between two nearby locations identified by the location identification module (e.g., the current location of vehicle 14 and a previously visited location identified by the location identification module). In a non-limiting example, as vehicle 14 approaches a previously visited location identified by the location identification module, the relocation module uses a relocation algorithm to compute a transformation vector describing the difference in location within environment 32, where the difference is the difference between the current location of vehicle 14 and a nearby previously visited location. Generally, the relocation algorithm is configured to receive any two sets of nearby observation data (e.g., from two of a plurality of observation points 54 identified as nearby by the location identification module) and compute a transformation vector from one set of observation data to the other.

[0067] Refer again Figure 2 And continue to refer to Figure 3At block 104, multiple vehicles (not shown) collect multiple local maps 50 using the SLAM algorithm described above and send the multiple local maps 50 to the server communication system 44. In a non-limiting example, vehicle system 12a of vehicle 14 executes the SLAM algorithm to collect at least one of the multiple local maps 50 using observation data and odometry data from multiple vehicle sensors 18, and transmits at least one of the multiple local maps 50 to server system 12b using vehicle communication system 28. In an exemplary embodiment, the multiple local maps 50 are stored in server database 42 of server system 12b. After block 104, method 100 proceeds to block 106.

[0068] Refer again at box 106. Figure 3 And continue to refer to Figure 2 The server controller 40 identifies multiple pairs of neighboring observation points 56. Within the scope of this invention, the multiple pairs of neighboring observation points 56 comprise multiple pairs of observation points 54 located within a predetermined radius of each other on multiple local maps 50. Each of the multiple pairs of neighboring observation points 56 includes a first observation point 58a from a first local map (e.g., first local map 52a) and a second observation point 58b from a second local map (e.g., second local map 52b). The local map coordinates of each of the multiple observation points 54 orient each of the multiple observation points 54 within one of the multiple local maps 50, but do not provide location information in a global coordinate system. The GNSS coordinates of each of the multiple observation points 54 provide location information in a global coordinate system, but may be affected by GNSS errors. Therefore, the server controller 40 executes the SLAM location identification algorithm as described above to identify the multiple pairs of neighboring observation points 56 based on observation data from each of the multiple observation points 54 in each of the multiple local maps 50.

[0069] After identifying each of the multiple neighboring observation pairs 56, the server controller 40 identifies multiple locations 60. Each of the multiple locations 60 is a first observation point 58a in one of the multiple neighboring observation pairs 56. (See again...) Figure 2 After box 106, method 100 proceeds to box 108.

[0070] At block 108, server controller 40 determines the transformation vector for each of the plurality of locations 60. Within the scope of the invention, the transformation vector of one of the plurality of locations 60 describes the positional difference in environment 32 between a first observation point 58a and a second observation point 58b among the plurality of locations 60. The local map coordinates of each of the plurality of observation points 54 orient each of the plurality of observation points 54 within one of the plurality of local maps 50, but do not provide positional information in a global coordinate system. The GNSS coordinates of each of the plurality of observation points 54 provide positional information in a global coordinate system, but may be affected by GNSS errors. Therefore, server controller 40 executes the SLAM relocalization algorithm as described above to determine the transformation vector for each of the plurality of locations 60 based on the observation data of each of the plurality of observation points 54. Following block 108, method 100 proceeds to block 110.

[0071] At box 110, server controller 40 determines the GNSS error at each of the plurality of locations 60. Within the scope of this invention, GNSS error includes noise and steady-state errors in GNSS measurements, including, for example, the estimated horizontal position error (EHPE). GNSS error can be caused by a variety of factors. For example, tall buildings or environmental features can block signals from GNSS satellites, leading to increased GNSS error. Generally, GNSS error will vary at different locations throughout environment 32 due to differences in geography, urbanization, and other environmental factors. To determine the GNSS error at each of the plurality of locations 60, server controller 40 uses the formula:

[0072] e GNSS =‖GNSS1-GNSS2‖-‖T‖ (1)

[0073] Among them, e GNSS GNSS is the GNSS error between a first observation point 58a and a second observation point 58b, one of a plurality of locations 60. GNSS1 is the GNSS coordinate of the first observation point 58a, one of a plurality of locations 60, GNSS2 is the GNSS coordinate of the second observation point 58b, one of a plurality of locations 60, and T is the transformation vector of one of the plurality of locations 60 determined at block 108. In an exemplary embodiment, when method 100 is subsequently performed using additional observation points, the GNSS error can be updated using the previously determined GNSS error and the average, exponential average, moving average, median, etc., of the new GNSS error determined based on the additional observation points.

[0074] In one exemplary embodiment, server controller 40 further updates the GNSS coordinates of the first observation point 58a based at least in part on a transformation vector. Server controller 40 first determines the transformed GNSS coordinates of the second observation point 58b. In a non-limiting example, the transformed GNSS coordinates of the second observation point 58b are determined by transforming the GNSS coordinates of the second observation point 58b with a transformation vector. Server controller 40 then updates the GNSS coordinates of the first observation point 58a. In a non-limiting example, the GNSS coordinates of the first observation point 58a are averaged with the transformed GNSS coordinates of the second observation point 58b to determine the updated GNSS coordinates of the first observation point 58a. In another non-limiting example, the transformed GNSS coordinates of the second observation point 58b are saved during multiple executions of method 100, and the median of the multiple transformed GNSS coordinates of the second observation point 58b is then calculated to determine the updated GNSS coordinates of the first observation point 58a. After block 110, method 100 proceeds to block 112.

[0075] Refer again Figure 3 And continue to refer to Figure 2 At box 112, server controller 40 determines a plurality of anchor positions 62. Within the scope of this invention, the plurality of anchor positions 62 includes a subset of the plurality of positions 60 determined at box 106. The plurality of anchor positions 62 will be used to align a plurality of local maps 50 with a global coordinate system, as will be discussed in more detail below.

[0076] In one exemplary embodiment, the plurality of anchor locations 62 comprises a subset of a plurality of locations 60 having a GNSS error less than or equal to a predetermined error threshold as determined at block 110 (e.g., the magnitude of the GNSS error is less than or equal to two meters). In another exemplary embodiment, the plurality of anchor locations 62 comprises a subset of a plurality of locations 60 having a GNSS error less than or equal to a predetermined error threshold as determined at block 110 and having at least a first predetermined number (e.g., 20) of a plurality of observation points 54 located within a second predetermined radius (e.g., five meters) of each of the subsets of locations 60. In another exemplary embodiment, each of the plurality of anchor locations 62 is selected at least a predetermined distance (e.g., 100 meters) from any other anchor location 62, such as to achieve spatial diversity of the plurality of anchor locations 62 in environment 32. In another exemplary embodiment, a location 60 with a high relocation error covariance, as determined during relocation at block 108, is not selected as one of the plurality of anchor locations 62. After block 112, method 100 proceeds to block 116.

[0077] Continue to refer to Figures 2 to 3At box 116, server controller 40 determines a plurality of anchor points 64. Each of the plurality of anchor points 64 corresponds to one of a plurality of anchor positions 62 determined at box 112. Within the scope of the invention, each of the plurality of anchor points 64 is a location in environment 32 defined by GNSS coordinates. In an exemplary embodiment, the GNSS coordinates of each of the plurality of anchor points 64 are selected as the median or average of the GNSS coordinates of a first observation point 58a and a second observation point 58b in each of the plurality of anchor positions 62. After box 116, method 100 proceeds to box 118.

[0078] Refer again Figure 2 At box 118, a first-stage alignment of each of the plurality of local maps 50 with a global coordinate system is performed. Within the scope of this invention, a global coordinate system is a substantially globally recognized system used, for example, to identify locations on Earth using two-dimensional or three-dimensional coordinate points according to ISO 6709. In one exemplary embodiment, the first-stage alignment is performed by server controller 40. In another exemplary embodiment, the first-stage alignment is performed by vehicle controller 16. While the following disclosure is explained from the perspective of server controller 40, it also applies to embodiments where the first-stage alignment is performed by vehicle controller 16.

[0079] Generally, the goal of the first-stage alignment is to apply transformations (e.g., translation, rotation, scaling, etc.) to multiple local maps 50 in order to fit the multiple local maps 50 to a global coordinate system based on multiple anchor points 64. In one exemplary embodiment, the first-stage alignment is performed using a first subset of the multiple anchor points 64. The first subset of the multiple anchor points 64 is relatively large. In a non-limiting example, the first subset of the multiple anchor points 64 includes all of the multiple anchor points 64. In another non-limiting example, the first subset of the multiple anchor points 64 includes anchor points 64 having a plurality of observation points 54 with a second predetermined number (e.g., 3) located within a third predetermined radius (e.g., 5 meters) of each of the first subset of the multiple anchor points 64.

[0080] To perform the first-stage alignment, a first subset of the plurality of anchor points 64 and a plurality of local maps 50 are provided as input to a first alignment algorithm. Generally, the first alignment algorithm is a point cloud registration algorithm configured to determine the transformations required to align each of the plurality of local maps 50 with the first subset of the plurality of anchor points 64. In one exemplary embodiment, the first alignment algorithm is a first machine learning alignment algorithm.

[0081] In a non-limiting example, the first machine learning algorithm comprises multiple layers, including an input layer and an output layer, as well as one or more hidden layers. The input layer receives a first subset of multiple anchor points 64 and multiple local maps 50 as input. These inputs are then passed to the hidden layers. Each hidden layer applies a transformation (e.g., a non-linear transformation) to the data and passes the result to the next hidden layer, until the last hidden layer. The output layer produces the transformation required to align each of the multiple local maps 50 with the first subset of the multiple anchor points 64.

[0082] To train the first machine learning alignment algorithm, the input dataset and its corresponding optimal output were used. The algorithm was trained by adjusting the internal weights between nodes in each hidden layer to minimize the prediction error. During training, optimization techniques (such as gradient descent) were used to adjust the internal weights to reduce the prediction error. The training process was repeated on the entire dataset until the prediction error was minimized, and then the resulting trained model was used to process new input data.

[0083] After sufficient training, the first machine learning alignment algorithm is able to accurately and precisely determine the transformations required to align each of the multiple local maps 50 with the first subset of the multiple anchor points 64, based on a first subset of multiple anchor points 64 and multiple local maps 50. By adjusting the weights between nodes in each hidden layer during training, the algorithm “learns” to identify patterns in the data that indicate the transformations required to align each of the multiple local maps 50 with the first subset of the multiple anchor points 64.

[0084] In another exemplary embodiment, the first alignment algorithm is the Umeyama algorithm, also known as the Kabsch algorithm or the Kabsch-Umeyama algorithm. S. Umeyama discusses the Umeyama algorithm in detail in "Least-squares estimation of transformation parameters between two point patterns" (IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 13, No. 4, pp. 376-380, April 1991), the entire contents of which are incorporated herein by reference. It should be understood that the first alignment algorithm may include alternative or other algorithms without departing from the scope of the invention. After block 118, method 100 proceeds to block 120.

[0085] At box 120, a second-stage alignment of each of the plurality of local maps 50 with the global coordinate system is performed. In one exemplary embodiment, the second-stage alignment is performed by server controller 40. In another exemplary embodiment, the second-stage alignment is performed by vehicle controller 16. Although the following disclosure is explained from the perspective of server controller 40, it also applies to embodiments in which the second-stage alignment is performed by vehicle controller 16.

[0086] Generally, the goal of the second-stage alignment is to refine the transformations (e.g., translation, rotation, scaling, etc.) determined by the first-stage alignment at box 118 to increase the accuracy of adapting multiple local maps 50 to the global coordinate system based on multiple anchor points 64. In one exemplary embodiment, the second-stage alignment is performed using a second subset of the multiple anchor points 64. The second subset of the multiple anchor points 64 is relatively small. In a non-limiting example, the second subset of the multiple anchor points 64 includes anchor points 64 having a greater than a third predetermined number (e.g., 20) of observation points 54 located within a fourth predetermined radius (e.g., 5 meters) of each of the second subset of the multiple anchor points 64. In one exemplary embodiment, the third predetermined number is greater than the second predetermined number discussed above with reference to box 118. Therefore, the second subset of the multiple anchor points 64 can be understood as having a higher “confidence” than the first subset of the multiple anchor points 64 because the second subset of the multiple anchor points 64 is supported by a larger number of observations. Furthermore, the second subset of the multiple anchors 64 is smaller than the first subset of the multiple anchors 64 (i.e., contains fewer anchors 64).

[0087] To perform the second-stage alignment, a second subset of the plurality of anchor points 64 and a plurality of local maps 50 are provided as input to a second alignment algorithm. Generally, the second alignment algorithm is a point cloud registration algorithm configured to determine the transformations required to align each of the plurality of local maps 50 with the second subset of the plurality of anchor points 64. In one exemplary embodiment, the second alignment algorithm is a second machine learning alignment algorithm.

[0088] In a non-limiting example, the second machine learning alignment algorithm comprises multiple layers, including an input layer and an output layer, and one or more hidden layers. The input layer receives a second subset of multiple anchor points 64 and multiple local maps 50 as input. These inputs are then passed to the hidden layers. Each hidden layer applies a transformation (e.g., a non-linear transformation) to the data and passes the result to the next hidden layer, until the last hidden layer. The output layer produces the transformation required to align each of the multiple local maps 50 with the second subset of the multiple anchor points 64.

[0089] To train the second machine learning alignment algorithm, the input to the dataset and its corresponding optimal output are used. The algorithm is trained by adjusting the internal weights between nodes in each hidden layer to minimize the prediction error. During training, optimization techniques (such as gradient descent) are used to adjust the internal weights to reduce the prediction error. The training process is repeated on the entire dataset until the prediction error is minimized, and then the resulting trained model is used to process new input data.

[0090] After sufficient training, the second machine learning alignment algorithm is able to accurately and precisely determine the transformations required to align each of the multiple local maps 50 with the second subset of the multiple anchor points 64, based on a second subset of multiple anchor points 64 and multiple local maps 50. By adjusting the weights between nodes in each hidden layer during training, the algorithm “learns” to identify patterns in the data that indicate the transformations required to align each of the multiple local maps 50 with the second subset of the multiple anchor points 64.

[0091] In another exemplary embodiment, the second alignment algorithm is a pose graph optimization (PGO) algorithm. The PGO algorithm is discussed in detail, for example, by F. Lu and E. Milios in "Globally Consistent RangeScan Alignment for Environment Mapping" (Autonomous Robots, Vol. 4, pp. 333-349, October 1997), the entire contents of which are incorporated herein by reference. It should be understood that the second machine learning alignment algorithm may include alternative or other algorithms without departing from the scope of the invention.

[0092] After performing the second-stage alignment, multiple local maps 50 are aligned with the global coordinate system. One or more of the aligned local maps can be transmitted to vehicle 14 using server communication system 44. Vehicle 14 can use one or more of the aligned local maps for navigation route selection, path planning, lane recognition, obstacle avoidance, etc. Furthermore, server controller 40 can use one or more of the aligned local maps to create a global map stored in server database 42 and / or interfering with vehicle 14 and / or multiple vehicles (not shown). After block 120, method 100 proceeds to block 122, entering a standby state.

[0093] In one exemplary embodiment, method 100 repeatedly exits the standby state 122 and restarts at block 102. In a non-limiting example, method 100 restarts periodically, for example, every three hundred milliseconds. The repeated execution of method 100 allows for continuous updates to the GNSS error and re-execution of the location identification and relocation algorithm to increase accuracy.

[0094] The system 10 and method 100 of the present invention offer several advantages. Accurate determination of the GNSS error at a given location within environment 32 allows for the selection of optimal anchor point locations, even when GNSS service capabilities are poor in a portion of environment 32. More specifically, even in the absence of known reference points or ground condition information, method 100 allows for the accurate quantification of the GNSS error at a specific location within environment 32 by utilizing SLAM-based relocation techniques.

[0095] The description of this invention is merely exemplary in nature, and variations that do not depart from the spirit of the invention are intended to fall within its scope. These variations should not be considered as departing from the spirit and scope of the invention.

Claims

1. A method for aligning multiple local maps with a global coordinate system, the method comprising: Quantify the Global Navigation Satellite System (GNSS) error at each of multiple locations in the environment; Multiple anchor points in the environment are determined based at least in part on the GNSS error at each of the multiple locations; as well as The multiple local maps are aligned with the global coordinate system, at least in part based on the multiple anchor points.

2. The method according to claim 1, wherein, Quantifying the GNSS error at each of the plurality of locations further includes: Collect the multiple local maps from multiple vehicles; Identify the multiple locations in the multiple local maps; and The GNSS error at each of the plurality of locations is quantified, at least in part, based on the plurality of local maps.

3. The method according to claim 2, wherein, Collecting the aforementioned multiple local maps also includes: Simultaneous Localization and Mapping (SLAM) is used to collect multiple local maps from the multiple vehicles, wherein each of the multiple local maps includes multiple observation points, and wherein each of the multiple observation points includes observation data, local map coordinates, and GNSS coordinates.

4. The method according to claim 3, wherein, Identifying the multiple locations in the multiple local maps further includes: Multiple neighboring observation point pairs are identified based on the multiple observation points in each of the multiple local maps, wherein each of the multiple neighboring observation point pairs includes a first observation point and a second observation point, the second observation point being located within a first predetermined radius of the first observation point in the environment, and the first observation point and the second observation point being at least partially based on the observation data of each of the multiple observation points in each of the multiple local maps; and Identify the plurality of locations, wherein each of the plurality of locations includes the first observation point of one of the plurality of neighboring observation point pairs.

5. The method according to claim 4, wherein, Quantifying the GNSS error at each of the plurality of locations, based at least in part on the plurality of local maps, further includes: A relocation algorithm is executed to determine a transformation vector between the first observation point and the second observation point, wherein the transformation vector describes the positional differences in the environment between the first observation point and the second observation point; Determine the GNSS error between the first observation point and the second observation point, wherein the GNSS error is: e GNSS =‖GNSS1-GNSS2‖-‖T‖ Among them, e GNSS The GNSS error between the first observation point and the second observation point, where GNSS1 is the GNSS coordinates of the first observation point, GNSS2 is the GNSS coordinates of the second observation point, and T is the transformation vector; and The GNSS coordinates of the first observation point are updated at least in part based on the transformation vector.

6. The method according to claim 5, wherein, Updating the GNSS coordinates of the first observation point also includes: The GNSS coordinates of the second observation point are transformed using the transformation vector to determine the transformed GNSS coordinates of the second observation point; and The updated GNSS coordinates of the first observation point are determined by averaging the GNSS coordinates of the first observation point with the transformed GNSS coordinates of the second observation point.

7. The method according to claim 4, wherein, Determining the plurality of anchor points also includes: Determine multiple anchor locations, wherein the multiple anchor locations include a subset of the multiple locations; and A plurality of anchor points are determined, wherein each of the plurality of anchor points corresponds to one of the plurality of anchor positions.

8. The method according to claim 7, wherein, Determining the plurality of anchor positions also includes: The plurality of anchor positions are determined, wherein each of the plurality of anchor positions has a GNSS error less than or equal to a predetermined error threshold.

9. The method according to claim 8, wherein, Determining the plurality of anchor positions also includes: The plurality of anchor positions are determined, wherein each of the plurality of anchor positions has a GNSS error less than or equal to a predetermined error threshold, and wherein each of the plurality of anchor positions has at least a first predetermined number of observation points located within a second predetermined radius of each of the plurality of anchor positions.

10. The method according to claim 1, wherein, Aligning the plurality of local maps with the global coordinate system also includes: The first-stage alignment is performed using a first alignment algorithm and a first subset of the plurality of anchor points; and A second-stage alignment is performed using a second alignment algorithm and a second subset of the plurality of anchor points, wherein the second alignment algorithm is different from the first alignment algorithm, and wherein the second subset of the plurality of anchor points is smaller than the first subset of the plurality of anchor points.

Images