Navigation map generation and filtering

Abstract

Techniques are described for generating, updating and using a sensor-based navigation map. An input map is generated based on sensor data acquired by a first sensor of a first vehicle. The input map is filtered based on one or more criteria to generate a filtered map corresponding to a three-dimensional representation of a route traveled by the first vehicle. The filtering may be based on detecting features using sensor data acquired by a second sensor of the first vehicle. The one or more criteria may include object classification, distance criteria, and / or other criteria related to attributes of features in the sensor data acquired by the first sensor and / or the sensor data acquired by the second sensor. The filtered map may be stored for transmission to a second vehicle and for use in detecting the location of the second vehicle while the second vehicle is traveling along the same route.

Description

[Technical field]

[0001] 1. Field of invention The present invention relates generally to the field of vehicle navigation, and more particularly to the use of sensor data to generate digital maps that can be referenced during navigation to determine vehicle position. Summary of the Invention

[0002] 2. Description of Related Technology Navigational maps can be generated in a variety of ways. Some maps are generated by satellite imagery for use in combination with satellite-aided navigation, e.g., aid from a global navigation satellite system such as the Global Positioning System (GPS). Maps generated using satellite imagery tend to be relatively accurate, and when combined with a satellite receiver on board the vehicle, allow the vehicle's position or geographic location to be estimated with high accuracy (e.g., within a few meters or even tens of centimeters). However, satellites cannot image all locations, as some locations are not well visible from a satellite view. Furthermore, the availability of satellite-aided navigation can depend on the vehicle's configuration (e.g., the presence or absence of an on-board GPS receiver) as well as environmental factors. For example, when driving through tunnels or mountainous areas, the GPS signal may be weak or nonexistent.

[0003] As an alternative to satellite imagery, maps can be generated using data from sensors such as cameras, light detection and ranging (LIDAR) sensors, and / or radar sensors. Such sensors can provide more detailed information than satellite imagery because the sensors are typically much closer to the captured environment. The increased level of detail can allow for a location to be detected with greater accuracy, e.g., within one or two centimeters, compared to satellite-based location. However, forming a map from sensor data requires processing a significantly larger amount of information commensurate with the increased level of detail. Sensor-based maps also tend to include extraneous information that is not necessary to determine location. The extraneous information increases the map size and requires additional storage space. The increased map size can also lead to transmission latency when the map is communicated to the vehicle for navigation use.

[0004] overview The embodiments described herein address these and other problems by providing a system and method for generating concise maps of geographic areas for use in connection with vehicle navigation. Aspects of the present disclosure relate to generating maps by capturing sensor data and then using automated processing to filter the sensor data to remove irrelevant information (e.g., data points that meet one or more criteria). The removed information generally corresponds to features that are not necessary for detecting a location and can therefore be classified as being of low or no interest. Examples of features that can be removed without adversely affecting the ability to detect a location include vehicles (e.g., vehicles parked on the side of a road or oncoming vehicles), animated objects (e.g., pedestrians), and various categories of inanimate objects (e.g., trees). Features that are retained and useful for detecting a location can include, for example, roads, traffic signs, landmarks, or other points of interest (POIs).

[0005] Automated filtering may include the fusion of data from different types of sensors to identify features for removal. For example, in some implementations, data from LIDAR or radar sensors is filtered based on features detected using images captured by a camera. Filtering can be performed algorithmically, for example, via a software algorithm configured to apply conditional logic to determine whether one or more criteria related to the presence of a particular feature are met. In some implementations, machine learning (e.g., deep learning algorithms or neural networks) is used to identify features for removal. For example, deep learning may include applying a trained machine learning model to detect the physical boundaries of objects that belong to a particular object classification or category.

[0006] In one aspect, a method of generating a map includes generating, by a computer system, an input map corresponding to a three-dimensional representation of a route traveled by a first vehicle based on sensor data acquired by a first sensor of the first vehicle. The method further includes filtering, by the computer system, the input map based on one or more criteria to generate a filtered map. The filtering includes automatically removing features that meet the one or more criteria. The features correspond to objects that are irrelevant to locating a second vehicle with reference to the filtered map. The method further includes transmitting, by the computer system, the filtered map to the second vehicle. The filtered map is processed by a navigation system of the second vehicle to locate the second vehicle while the second vehicle is traveling the same route.

[0007] In an aspect, the system includes one or more processors and a memory that stores instructions executable by the one or more processors to cause the one or more processors to perform the above-described method for generating a map. The memory can include a non-transitory computer-readable storage medium that stores the instructions. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram of a system incorporating one or more embodiments. [Diagram 2] FIG. 2 shows an example of a 2D display that can be used as input for filtering in accordance with one or more embodiments. [Figure 3A] FIG. 3A illustrates an example of a filterable 3D display in accordance with one or more embodiments. [Figure 3B] FIG. 3B shows the result of filtering the 3D display of FIG. 3A to generate filtered data. [Figure 4] FIG. 4 illustrates an example sequence of operations that may be performed to generate a filtered map, according to one embodiment. [Figure 5A] FIG. 5A shows examples of bounding boxes applied to a 3D representation of an environment. [Figure 5B] FIG. 5B shows an example of filtering the 3D display in FIG. 5A. [Figure 6] FIG. 6 is a flow diagram of a process for generating and using a filtered map according to one embodiment. [Figure 7] FIG. 7 is a flowchart of a process for generating a filtered map according to one embodiment. [Figure 8] FIG. 8 is a flowchart of a process for filtering an input map according to one embodiment. [Figure 9]FIG. 9 is a flowchart of a process for transmitting a filtered map to a vehicle according to an embodiment. [Figure 10A] 10A-10C show examples of filtered maps received over a period of time according to some embodiments. [Figure 10B] 10A-10C show examples of filtered maps received over a period of time according to some embodiments. [Figure 10C] 10A-10C show examples of filtered maps received over a period of time according to some embodiments. [Figure 11] FIG. 11 illustrates an example of a filtered map according to some embodiments. [Figure 12] FIG. 12 is a flowchart of a process for updating a filtered map according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Like reference numbers in the various figures indicate like elements according to certain exemplary embodiments. Additionally, multiple instances of an element may be indicated by a first number for that element followed by a letter or a hyphen and a second number. For example, multiple instances of element 110 may be indicated as 110-1, 110-2, 110-3, etc., or as 110A, 110B, 110C, etc. When referring to such an element using only the first number, any instance of that element will be understood (e.g., element 110 in the previous example would refer to elements 110-1, 110-2, and 110-3, or elements 110A, 110B, and 110C).

[0010] Detailed Description Certain exemplary embodiments will now be described with reference to the accompanying drawings, which form a part hereof. Certain specific embodiments in which one or more aspects of the disclosure may be practiced are described below, but other embodiments may be utilized, and various changes may be made, without departing from the scope of the disclosure or the spirit of the appended claims.

[0011] Aspects of the present disclosure relate to generating a map by acquiring sensor data and then filtering the sensor data to remove extraneous information, e.g., unwanted data points from a point cloud. Removing extraneous information may include subjecting an input map derived from the sensor data to automated processing to generate a filtered map that includes less extraneous information. The input map and / or the filtered map may be a two-dimensional (2D) or three-dimensional (3D) representation of the physical environment around the sensor. For example, in some implementations, the input map is a 3D map generated from a LIDAR point cloud, and the output map corresponds to the 3D map after filtering using 2D images acquired by a camera. Thus, the map may be generated using a 3D representation, a 2D representation, or a combination of 2D and 3D representations of the physical environment, which are acquired by multiple sensors. The automated filtering allows for the creation of a filtered map without relying on manual removal of extraneous information. In this manner, the filtering techniques described herein can be applied to create filtered maps in a more timely manner through processing large amounts of sensor data with little or no manual oversight.

[0012] A map may be a collection of digital information that represents a geographic area. In addition to showing the physical layout of the geographic area, a map typically includes labels that indicate places and features in the geographic area, such as names of roads, cities, bodies of water, buildings, etc. As previously mentioned, maps may be 2D or 3D. Thus, locations within the geographic area may be specified as geodetic locations that are displayed using 2D or 3D coordinates (e.g., latitude, longitude, and altitude). Alternatively, locations may be specified as civic locations (e.g., addresses).

[0013] Depending on the size of the geographic area, the map may be divided into one or more segments (e.g., square or rectangular tiles), with each segment formed using sensor data acquired by one or more sensors of the vehicle as the vehicle travels through the area corresponding to the segment. In some implementations, the vehicle may be part of a fleet of vehicles traveling around the geographic area to acquire LIDAR point clouds, camera images, and / or other types of views as part of mapping the geographic area. Thus, the map may be a product of sensor data provided by at least one vehicle, and in some examples is generated through an aggregation of sensor data from multiple vehicles.

[0014] After the map is filtered to generate a filtered map, the filtered map can be communicated to a vehicle for use as a reference while traveling through the geographic area covered by the filtered map. Typically, the vehicle using the filtered map is a different vehicle than the vehicle that provided the sensor data for the input map from which the filtered map was generated. When referencing the filtered map, the vehicle may compare information in the filtered map with information obtained using one or more of the vehicle's own sensors to identify the location of the vehicle. The filtered map can be transferred to the vehicle's memory for storage. In some examples, the filtered map is preloaded into the vehicle's memory before traveling. In other examples, the filtered map is transmitted wirelessly in real time while the vehicle is traveling through the geographic area covered by the map.

[0015] The filtered map could potentially be transmitted at one time, for example as a continuous, uninterrupted stream of wireless data. However, in practice, due to bandwidth constraints, in a given communication transaction, only a portion of the filtered map that is relevant to a local area around the vehicle (e.g., one or two segments representing an approaching area) may be transmitted. Because the filtered map contains less information than the input (unfiltered) map, the filtered map can be transmitted with less latency, which helps enable the vehicle to access map information relevant to the vehicle's current environment. Efficient communication of the filtered map is particularly beneficial in situations where satellite aiding is unavailable (e.g., due to lack of a GPS signal). For example, if the GPS signal is suddenly lost, the sensor-based map can be streamed to the vehicle as part of switching from GPS positioning to positioning using a sensor-based map. The switch in positioning method can be made quickly so that navigation instructions and / or a map representing the vehicle's location can continue to be presented to the driver with minimal interruption. If the vehicle is an autonomous vehicle, the switch may allow a control unit of the autonomous vehicle to have substantially uninterrupted access to information regarding the autonomous vehicle's current location.

[0016] 1 is a block diagram of a system 100 incorporating one or more embodiments of the present disclosure. The system 100 includes a set of vehicles 102 (e.g., vehicle 102A, vehicle 102B, and vehicle 102N) and a computer system 150 that is communicatively coupled to the vehicles 102 via one or more communication networks 130.

[0017] The one or more networks 130 may include any of a variety of public and / or private networks, such as the Internet, one or more cellular networks (e.g., 4G, 5G, or Long Term Evolution (LTE), a Wi-Fi (IEEE 802.11X) network, a WiMax (Wireless Inter-operability for Microwave Access) network, and / or the like. The vehicle 102 may be wirelessly coupled to the network 130, for example, via a cellular connection, a Wi-Fi connection, or a Bluetooth connection. In some examples, the vehicle 102 may have a wired (e.g., Ethernet) connection to the network, for example, to transfer sensor data acquired by the vehicle to the computer system 150.

[0018] 1, the vehicle 102 may include sensors 110, an electronic control unit (ECU) 112, a global navigation satellite system (GNSS) receiver 114, a navigation system 116, a communication system 118, a memory 120, and input / output (I / O) devices 122. The vehicle 102 may be an automobile or other type of motorized vehicle, such as a car, motorcycle, bus, truck, etc. However, the techniques described herein are applicable to any vehicle equipped with sensors and are not limited to land-based vehicles. Thus, the disclosed embodiments may also be used in watercraft, aircraft, and / or spacecraft.

[0019] The ECU 112 may include one or more processing units. Each processing unit may be a general-purpose processor, a special-purpose processor (e.g., a digital signal processor (DSP) or a graphics accelerator processor), an application specific integrated circuit (ASIC), and / or other hardware circuitry or combination of circuits. The ECU 112 may execute instructions stored in the memory 120.

[0020] The memory 120 may include storage and / or working memory implemented using volatile storage (e.g., RAM), non-volatile storage (e.g., ROM), or a combination of volatile and non-volatile storage. The memory 120 may store program code corresponding to software applications executable by the ECU 112 to generate and filter maps, for example. Additionally or alternatively, the memory 120 may store software applications executable by the ECU 112 to obtain and process filtered maps in connection with detecting the location of the vehicle 102. Furthermore, the memory 120 may include storage space for sensor data obtained by one or more sensors 110 in connection with generating the filtered maps and / or detecting the location using the filtered maps. Thus, the memory 120 may include storage for executable program instructions to configure the ECU 112 and / or other vehicle components (e.g., sensors 110) to provide various map-related functions described herein.

[0021] The sensors 110 may include one or more instances of cameras, LIDAR sensors, radar sensors, ultrasonic sensors, and / or other types of sensors capable of acquiring data from which a 2D or 3D representation of the environment around the sensors can be formed. For example, a front camera may be mounted behind the windshield of the vehicle 102A to obtain a 2D image of the environment in front of the vehicle 102A, while a LIDAR or radar sensor may generate a 3D point cloud representing the environment in front of the vehicle 102A. Each sensor has a corresponding field of view. Multiple instances of sensors may be employed to capture the environment in different directions (e.g., sensors mounted on the sides and rear of the vehicle). The vehicle 102 may be configured to perform sensor fusion as part of generating the input map and / or filtered map. For example, a composite image may be formed by stitching together images from cameras pointed in different directions. In a stereo camera arrangement, an image from a first camera and an image from a second camera may be combined to generate a 3D representation based on knowledge of the positions of the first and second cameras, for example by comparing both images to extract depth information based on camera parallax. Thus, through combining data from different sensors, an input map can be generated.

[0022] Sensor fusion may include data from different types of sensors. Sensor data provided by one type of sensor may be supplemented and in some examples may overlap with sensor data provided by another type of sensor. For example, a LIDAR sensor may generate a point cloud including points representing the outline of a road, and a camera may acquire an image of the same road, including lane markers that cannot be detected by the LIDAR (e.g., because the lane markers are painted and not 3D). In this manner, different sensors and / or different types of sensors may contribute to a more comprehensive view of the environment around the vehicle 102. An example of using data from a first sensor type to filter data from a second sensor type is described below.

[0023] The GNSS receiver 114 is configured to communicate with one or more satellites of the GNSS. For example, the GNSS receiver 114 may include a GPS receiver capable of tuning to a carrier frequency of signals from satellites in the GPS constellation. The GNSS receiver 114 may also have transmission capabilities, for example, to request assistance information from a land-based server in connection with determining the location of the GNSS receiver 114. When used in connection with GNSS-based positioning, the GNSS receiver 114 may receive reference signals from multiple satellites, which may be processed (e.g., by the navigation system 116) to geometrically determine the location of the vehicle 102, for example, by trilateration. In some examples, GNSS-based positioning may involve communication with terrestrial entities, such as a base station or another GNSS receiver. For example, the location of the vehicle 102 may be determined using information transmitted from a nearby base station at a known location, including measurements of the reference signals and / or GPS signals received by the base station.

[0024] The navigation system 116 is configured to access a map stored in the memory 120 in connection with detecting the location of the vehicle 102. The process of detecting the location of the vehicle in relation to the map may be referred to as "localization" and, when performed using a sensor-based map, may include comparing data acquired by one or more sensors 110 to data included in the sensor-based map. For example, by comparing a point cloud acquired by a LIDAR or radar sensor to a point cloud included in the sensor-based map, the navigation system 116 may detect that the vehicle 102 is currently located on a particular portion of a road. The comparison may include matching features included in the real-time sensor data with corresponding features included in the sensor-based map. For example, the navigation system 116 may identify features (e.g., road contours or building outlines) from points in the real-time point cloud and determine whether there is a matching feature in the sensor-based map.

[0025] The navigation system 116 may be implemented in software and / or hardware. For example, the navigation system 116 may be a software application residing in the memory 120 and executed by a processor of the ECU 112. The navigation system 116 may detect the location of the vehicle 102 in connection with providing navigation guidance to a driver of the vehicle 102. For example, the navigation system 116 may use the location of the vehicle 102 to plan a route to a driver-specified destination, to display the vehicle's location on a display within the vehicle, or to retrieve traffic notifications or other reports related to the vehicle's location from a remote server. Similarly, if the vehicle 102 is an autonomous vehicle, the location of the vehicle may be provided as an input for determining an autonomous behavior (e.g., a driving maneuver) designed to get the vehicle to a driver-specified destination, or whether the planned autonomous behavior should be modified to account for traffic, weather, hazards, or other conditions present around the vehicle's location.

[0026] The communication system 118 may include a wireless communication interface through which data is communicated between the vehicle 102 and the computer system 150. Thus, the communication system 118 may include radio frequency components such as a Wi-Fi receiver, a cellular receiver, and / or the like. A radio frequency component operating as a receiver may also include a transmit capability. The radio frequency components may share hardware. For example, the communication system 118 may include an antenna array including one or more antennas used by the GNSS receiver 114 to receive signals from GPS satellites and one or more antennas configured to receive signals from cellular base stations. In some implementations, separate antennas may be provided for use with different wireless communication protocols. For example, the GNSS receiver 114 may include its own antenna array separate from the antenna array of the radio frequency receiver of the communication system 118.

[0027] The navigation system 116 may be configured to request the sensor-based map using the communication system 118. A request for the sensor-based map may be sent to the computer system 150 to cause the computer system 150 to transmit the sensor-based map or a relevant portion of the sensor-based map. For example, the request may include an indication of the last known or estimated location of the vehicle, and the computer system 150 may provide map data covering a distance around the indicated location. The sensor-based map may be provided to the vehicle 102 substantially in real-time while the vehicle 102 is traveling within a geographic area covered by the sensor-based map. The sensor-based map may also be communicated to the vehicle in advance and stored in the memory 120 before traveling within the covered geographic area.

[0028] The sensor-based map received through the communication system 118 may include a filtered map generated using sensor data acquired by one or more vehicles 102. The filtered map may be generated by removing irrelevant information from the unfiltered map. In some implementations, the process for generating the filtered map is performed by the computer system 150 based on the sensor data and / or the unfiltered map received from one or more vehicles 102. Thus, the communication system 118 may also be used to provide the sensor data and / or the unfiltered map to the computer system 150. For example, the vehicles 102A, 102B, and 102N may be part of a fleet of vehicles that travel throughout a geographic area to obtain 2D and / or 3D views using their respective sensors 110 and upload the views to the computer system 150 for forming and filtering the sensor-based map. Each vehicle 102 in the fleet that provides sensor data for the sensor-based map may be assigned to travel along a different portion of the geographic area. In some examples, the portions traveled by two vehicles may overlap. Redundancy in the sensor data provided by one or more vehicles 102 can be removed when the sensor data is combined by the computer system 150 .

[0029] The sensor-based map can be updated based on newer sensor data. For example, when navigating using a filtered map provided by the computer system 150, the vehicle 102 may transmit sensor data to the computer system 150 to enable the computer system 150 to compare data from the filtered map with sensor data acquired using one or more sensors 110 of the vehicle 102 as well as to determine whether to update the filtered map based on changes reflected in the sensor data. Examples of changes that may cause an update to the filtered map include the presence of a detour, a closed road, a new building, and a lane closure. Thus, the filtered map may be the product of sensor data collected over multiple drives across a geographic area by one or more vehicles.

[0030] The I / O devices 122 may include one or more input devices configured to receive input from a driver or other occupants of the vehicle 102, such as, for example, a keyboard or keypad, a touch screen, buttons, knobs, a microphone, and / or the like. The input devices may be located in the vehicle cabin, such as, for example, in a center console, a steering wheel, an overhead console, etc. The I / O devices 122 may also include one or more output devices configured to present audio, visual, and / or other types of output (e.g., tactile output) to the driver or other occupants. Examples of output devices include a display monitor, a loudspeaker, light emitting diodes (LEDs), and / or the like.

[0031] The computer system 150 is configured to generate and / or transmit the filtered map to the vehicle 102. Thus, the computer system 150 may include a map generation system 152 and a communication system 154. The computer system 150 may be implemented using one or more computing devices, which may include, for example, a general purpose computer, a storage server, or a network of computers. The computer system 150 may include one or more I / O devices 156 similar to the I / O devices 122 of the vehicle 102, such as a keyboard, a display monitor, a loudspeaker, and / or the like. As previously mentioned, the filtered map may be generated in an automated manner with little or no manual supervision. In particular, the process of identifying and removing irrelevant information from the unfiltered or input map may be performed without manual review of the unfiltered map. However, manual input can be provided, for example using I / O device 156, to further refine the filtered map (e.g., by manually designating one or more features for deletion) or to override a decision by computer system 150 to delete a feature.

[0032] The map generation system 152 may be configured to generate an unfiltered map using sensor data from one or more vehicles 102. The sensor data may be provided via a communication system 154, for example, sent via one or more communication networks 130 while the vehicle 102 is traveling within the geographic area or after the vehicle 102 has completed traveling within the geographic area. The map generation system 152 may be implemented in hardware and / or software and may perform sensor fusion to identify features for removal. The sensor fusion performed by the map generation system 152 may include combining disparate sensor data (e.g., camera images and LIDAR point clouds) to detect and classify features to be removed. In some implementations, the unfiltered map is generated using 3D data from a LIDAR or radar sensor and then filtered based on camera images or data from another type of sensor. The sensor fusion performed as part of the filtering of the unfiltered map may include not only removing features related to irrelevant information, but also augmenting the sensor data of the unfiltered map to include more detailed information of the features that are retained. For example, camera images may provide color information not present in the LIDAR point cloud, or sensor fusion may be performed prior to filtering to generate an unfiltered map as a product of sensor data from multiple types of sensors.

[0033] In some implementations, the map generation system 152 is configured to apply machine learning to identify features related to removal. For example, the map generation system 152 may execute a machine learning algorithm or use a trained neural network to classify objects represented in the sensor data according to a set of predefined categories (people, trees, roads, lanes, vehicles, etc.) and identify the boundaries of each object. The machine learning algorithm or neural network may be pre-trained using supervised learning on the sensor data (e.g., camera images and / or LIDAR point clouds) where the locations and boundaries of objects are known. Alternatively, unsupervised learning techniques may be used. The input to the machine learning algorithm or neural network may be, for example, a 3D representation depicting the same location and a corresponding 2D representation. The output of the machine learning or neural network may be a labeled or segmented 3D representation depicting different classes of objects and the boundaries of such objects.

[0034] Whether performed algorithmically or by machine learning, the filtering process may include a series of steps that first segment the 3D representation into different regions and assign a class label to each region. After segmenting the 3D representation, the boundaries of each object may be determined, for example, based on a clustering of points in the point cloud. The segmentation and object boundaries can be determined taking into account information from the 2D representation, which can also be segmented and classified. In this respect, the 3D representation may include depth information that is not available from the 2D representation, while color information in the 2D representation may allow objects that are close to each other (e.g., a vehicle parked behind another vehicle) to be more easily distinguished. Thus, the 2D and 3D representations (or, in general, data from heterogeneous sensors) may collectively provide a more comprehensive view of the environment. The object boundaries may be class specific. For example, 3D bounding box templates of different shapes may be available for fitting to the segmented regions of the 2D and / or 3D representations. In this manner, the map generation system 152 can determine a bounding box for an object in the 3D representation and / or a corresponding bounding box for the same object in the 2D representation.

[0035] After detecting boundaries for the detected objects, the map generation system 152 may identify one or more features for removal from the 3D representation to generate an output 3D representation for inclusion as part of the filtered map. To identify an object for removal, the map generation system 152 may determine whether one or more criteria are met. The criteria may be explicitly defined, for example, using if-then-else statements, case statements, or other programming logic, or hardware logic (e.g., combinatorial logic circuits). Alternatively, the criteria may be defined through training of a machine learning model to set the parameters (e.g., weights of a neural network) of the machine learning model. One criterion may be that the feature belongs to one or more predefined categories or object classifications. For example, features detected to be vehicles, pedestrians, trees, or other types of objects that are not suitable for use as reference features (e.g., because the objects are ephemeral or may change appearance) may be selected as candidates for removal. Features that may serve as reference features and therefore may be retained include signs or boards along the road (e.g., signs indicating the location of nearby exits), raised curbs that define the edges of the road, buildings and other points of interest, fences or retaining walls, and / or the like. The following is a non-exhaustive list of object classifications that may be filtered out: lanes, moving objects, pedestrians, other vehicles (e.g., cars, bikes, trucks, etc.), plants, and animals.

[0036] Another criterion that the map generation system 152 may apply is a distance threshold. Features that are within a certain distance from the vehicle 102 that acquired the sensor data for the 3D representation may be retained. For example, a tree 20 meters away may be removed. In contrast, another vehicle 5 meters away may be retained. Thus, not all features that may be unnecessary for detecting the location of the vehicle are removed. Since each filtered map may include multiple 3D representations, each corresponding to a "snapshot" of the environment around the vehicle 102 at a different location within the geographic area covered by the filtered map, limiting the filtering to features beyond the threshold distance may reduce the time it takes to generate the filtered map.

[0037] Other criteria can be applied to filter out features that are not necessary for detecting a location. For example, the map generation system 152 may apply one or more criteria related to the attributes of the detected objects, such as absolute speed, speed relative to the sensing vehicle, physical state (e.g., moving or stationary), age, length, width, height, and shape. Thus, the filtered out objects may include objects moving above a threshold speed, objects below a threshold age (detectable based on comparing multiple sets of sensor data collected about the same location over time), objects below a threshold length / width / height, or objects that meet certain shape requirements (e.g., objects with a number of edges above a threshold or objects with edges with curvature above a threshold). Another example is the confidence of the object classification. In some implementations, the map generation system 152 is configured to determine a confidence value that represents the likelihood that a class label assigned to a feature is a correct label for that feature. To avoid removal due to false positive identification, objects with a confidence value below a confidence threshold may be retained. Thus, map generation system 152 can apply any number or combination of criteria to filter out a feature if it meets such criteria. For example, map generation system 152 can be configured with multiple sets of criteria (e.g., different combinations of the criteria described above) such that a feature is filtered out if it meets any set in the multiple sets of criteria.

[0038] The computer system 150 may further include a memory 158. Similar to the memory 120, the memory 158 may include storage memory and / or working memory. The memory 158 may store software applications configured to provide map-related functions. For example, the map generation system 152 may include one or more processors configured to execute program instructions in the memory 158 to generate and filter a map. Additionally, the memory 158 may store input and / or output data for the map generation system 152. Such data may include filter criteria 160, a filtered map 162, or an unfiltered map (not depicted). The filtered map 162 may be stored for subsequent transmission to the vehicle 102, for example, upon request from a particular vehicle or transmitted to multiple vehicles simultaneously.

[0039] FIG. 2 illustrates an example of a 2D display that can be used as input for filtering according to one or more embodiments. FIG. 2 is provided to illustrate an example of features (described in connection with FIGS. 3A and 3B) that can be filtered out without adversely affecting the ability to detect a location. The 2D display of FIG. 2 is a camera image 200 that may be captured, for example, using a front camera of the vehicle 102. The image 200 may be a color image or a monochrome (e.g., black and white or grayscale) image. As shown in FIG. 2, the image 200 depicts a scene along a multi-lane road with two directions of traffic and vehicles in both directions. The vehicles in the image 200 include vehicles traveling ahead of the vehicle capturing the image 200 (e.g., leading vehicle 210) and vehicles traveling in the opposite direction (e.g., oncoming vehicle 220). Image 200 further shows the edge of the road 250 (defined in this example by a raised curb), lane markers 260, and structures along the sides of the road (eg, building 230 and road sign 240).

[0040] FIG. 3A illustrates an example of a 3D view that may be filtered according to one or more embodiments. The 3D view in FIG. 3A is a point cloud 300 that may be acquired, for example, using a LIDAR sensor on the vehicle 102 (e.g., the same vehicle that acquired the image 200). Comparing the image 200 to the point cloud 300, it can be seen that the two views show the same scene, but the viewpoints are not identical. For example, as shown in FIG. 3A, the point cloud 300 includes a first point cloud related to the forward vehicle 210, a second point cloud related to the oncoming vehicle 220, a third point cloud related to a building 230, a fourth point cloud related to a road sign 240, and a fifth point cloud related to a road edge 250. The points in the point cloud 300 may be formed based on measurements of laser light or radar signals reflected from objects in the environment. The difference in viewpoint may be the result of the sensors that acquired the image 200 and the point cloud 300 having different fields of view, being pointed in different directions, being mounted in different locations, and / or having other differences in sensor configuration. The point cloud 300 also includes ground reflections 302 that correspond to points generated as a result of reflections from the ground.

[0041] FIG. 3B shows the result of filtering the 3D view of FIG. 3A to form a filtered point cloud 310. Comparing FIG. 3A and FIG. 3B, it can be seen that the filtered point cloud 310 contains a significantly smaller amount of points. For example, points related to the forward vehicle 210 and the oncoming vehicle 220 are not present in the filtered point cloud 310. However, points related to the buildings 230, the road signs 240 and the road edges 250 remain. As mentioned above, the filtering process may include applying one or more criteria to identify features in the sensor data acquired using the first sensor that meet the one or more criteria, and may be performed with the help of sensor data from the second sensor. For example, the filtered point cloud 310 may be generated as a result of filtering the point cloud 300 based on information contained in the image 200. The filtering of the point cloud 300 may be performed after segmenting the point cloud 300 into different regions corresponding to groups or clusters of related points (e.g., the first to fifth point clouds described above) and assigning a class label to each group of points. Filtered point cloud 310 can be combined with other filtered point clouds to form a filtered map, for example, by constructing a 3D view of an extended section of the same road depicted in FIG. 3B using points from filtered point cloud 310 and points from one or more additional filtered point clouds depicting different portions of the road.

[0042] FIG. 4 shows an example of a sequence of steps 400 that can be performed to generate a filtered map, according to one embodiment. The steps depicted in FIG. 4 can be executed using one or more processors of a computer system, such as the processor implementing the map generation system 152 of FIG. 1. The sequence of steps 400 starts with an object detection stage 410. The object detection stage 410 may include sensor fusion using sensor data 402 from multiple sensors. Sensor data 402 from different sensor types, such as data from a LIDAR point cloud and data from a camera image, can be processed together to identify features in the sensor data as objects belonging to one or more predefined object classes. For example, clusters of points that are located a threshold distance apart from each other in the point cloud can be identified as belonging to the same object. Similarly, pixels of the same or similar color / brightness in the camera image can be identified as belonging to the same object. Objects 412 detected as a result of processing in the object detection stage 410 are input to a bounding box generation stage 420.

[0043] In the bounding box generation stage 420, a bounding box may be formed around each object 412 to generate a set of bounding boxes 422. The bounding boxes 422 may be formed in two dimensions (2D boxes) and / or three dimensions (3D boxes) depending on the type of sensor data 402 provided as input to the object detection stage 410. In some implementations, the bounding box generation is applied only to the 2D sensor data, which is then mapped to corresponding regions of the 3D sensor data, e.g., boundaries detected using the 2D sensor data, to corresponding points in the point cloud, as part of subsequent processing in the filtering stage 430. The bounding boxes may be rectangular or some other geometric shape depending on the level of accuracy desired. For complex shapes, a predefined shape template (e.g., sedan shape, van shape, truck shape, etc.) may be applied using one or more geometric transformations, such as scaling and rotation, to form a bounding box that best fits the detected object.

[0044] The bounding boxes 422 are input to a filtering stage 430 together with unfiltered map data 424. The unfiltered map data 424 corresponds to the map to be filtered and may include a 3D view of the same environment represented by the sensor data 402. In some implementations, the unfiltered map data 424 includes a subset of the sensor data 402, such as the same LIDAR point cloud. However, there may or may not be overlap between the data input to the object detection stage 410 and the data input to the filtering stage 430. For example, the object detection stage 410 may be performed without using the point cloud to be filtered. The filtering stage 430 includes determining which of the objects 412 defined by the corresponding bounding boxes 422 are removed from the unfiltered map data 424. As described above, the filtering may include identifying objects that meet one or more criteria, which provide a basis for distinguishing objects that are not required for location detection. Removal of such objects results in filtered map data 432 that can be incorporated into a filtered map. For example, the filtered map data 432 may correspond to the filtered point cloud 310 of FIG. 3B.

[0045] The filtering stage 430 may include multiple filtering steps. For example, a first filtering step may remove ground reflections (e.g., ground reflection 302 in FIG. 3A) and moving objects (e.g., forward vehicle 210 and oncoming vehicle 220). Motion can be detected, for example, by point motion classification, which identifies groups of points that move a different amount (e.g., distance) or in a different direction than the capturing vehicle. After removing ground reflections and moving objects, the remaining objects (associated with some or all of bounding box 422) can be identified as static, non-moving objects that belong to one or more object classes (vehicles, pedestrians, animals, etc.) and can be removed by object-based filtering using one or more criteria. For example, the first filtering may filter out moving vehicles, while the object-based filtering may filter out stationary or parked vehicles. As mentioned above, one example of a filter criterion is a threshold distance, which may define a region of interest within which features that otherwise meet one or more criteria are still retained. For example, filtering may be limited to features that are at least 15 meters away in any direction. An example process for multi-step filtering is shown in Figure 8 below.

[0046] The filtered map data 432 includes a 3D representation (e.g., a 3D grid of points) that corresponds to an instantaneous snapshot of the environment surrounding the vehicle. To generate a complete filtered map, for example, multiple sets of filtered map data 432 can be combined, such that each set of filtered map data 432 forms a separate 3D slice within an overall 3D volume representing a geographic area.

[0047] FIG. 5A shows an example of a bounding box applied to a 3D representation of an environment. Although the underlying data is three-dimensional, FIG. 5A corresponds to a top-down view of the environment and is therefore shown using a two-dimensional coordinate system in which the horizontal axis (x-axis) represents horizontal distance in meters and the vertical axis (y-axis) represents vertical distance in meters. Distances in FIG. 5A are relative to the vehicle 520 from which the 3D representation was acquired. The vehicle 520 is therefore located at the origin (0,0) of the two-dimensional coordinate system. In the example of FIG. 5A, the 3D representation is a point cloud with points scattered around bounding boxes 510A, 510B, 510C, and 510D. Bounding boxes 510A-D represent other vehicles around vehicle 520. The point cloud further includes points associated with other physical features in the environment, such as road edges 550. As shown in FIG. 5A, each object represented by bounding box 510 can be assigned a unique identifier (ID).

[0048] FIG. 5B shows an example of filtering the 3D display in FIG. 5A. FIG. 5B includes the points remaining after filtering has been performed. As shown in FIG. 5B, there are no points associated with the bounding boxes 510A, 510B, and 510C. This is a result of identifying the objects corresponding to the bounding boxes 510A, 510B, and 510C for removal. However, there are relevant points in the road edge 550 in addition to the irrelevant points 530A, 530B, and 540. The irrelevant points 530A and 530B may correspond to noise or irrelevant results, but are not labeled as belonging to the class of objects to be removed, and can, however, be removed optionally depending on further filtering, and do not significantly affect the detection of the location of the vehicle. Also, the irrelevant point 540 does not significantly affect the location detection, and represents a point associated with the vehicle corresponding to the bounding box 510D, and remains as a result of applying the threshold distance of 15 meters as a criterion for removal, even though it belongs to the class of objects to be removed (e.g. other vehicles). As mentioned above, features that are within a threshold distance can be retained to reduce processing time. In this example, point 540 is within 15 meters of vehicle 520, while the points around bounding boxes 510A, 510B, and 510C are all located beyond 15 meters.

[0049] FIG. 6 is a flow diagram of a process 600 for generating and using a filtered map, according to an embodiment. The process 600 can be performed by a first vehicle (A), a second vehicle (B), and a computer system (e.g., computer system 150). At 602, vehicle A acquires sensor data while traveling along a route. The sensor data is acquired using a first sensor mounted on vehicle A and may include a 3D representation, such as a point cloud acquired using a LIDAR or radar sensor. The sensor data may further include data acquired using a second sensor mounted on vehicle A, such as a camera, a radar sensor, or another LIDAR sensor. The second sensor acts as a reference sensor that acquires data that can be used to filter the 3D representation. The data acquired by the second sensor may be acquired simultaneously with the data acquired by the first sensor and may depict the same scene (e.g., the same portion of a road along the route). The route extends through the geographic area being mapped and includes one or more drivable portions of the geographic area, such as one or more roads, highways, bridges, etc. The route may be pre-planned or, in some cases, may be a route selected on the fly by the driver of vehicle A. As shown in Figures 2 and 3A, the sensor data may include an instantaneous representation of the environment surrounding the vehicle acquired at any given time. The sensor data acquired in 602 may include multiple sets of 2D and / or 3D representations acquired at different times depicting different portions of the route.

[0050] At 604, vehicle A transmits the acquired sensor data to the computer system. The sensor data can be transmitted substantially in real time using wireless communication, such as while traveling along the route or immediately after completing the route. Alternatively, the sensor data can be stored in a memory of vehicle A and uploaded to the computer system using a wired connection, if vehicle A is available to be communicatively connected to the computer system via a wire.

[0051] At 606, the computer system generates an unfiltered map using the sensor data. The unfiltered map may include a 3D representation of the entire route traveled by vehicle A, and can be generated, for example, by combining a set of point clouds or other 3D representations acquired along the route.

[0052] At 608, the computer system filters the map based on one or more criteria to remove irrelevant information. Filtering removes features that meet one or more criteria and can be performed, for example, according to the process depicted in FIG. 4. Thus, the function at 608 may include segmenting a first view acquired using a first sensor (e.g., camera images) to detect that one or more regions in the first view correspond to a class of objects that can potentially be removed, and then determining a bounding box for each of the one or more regions. The function at 608 may further include applying one or more criteria to determine which bounding boxes are associated with features to be removed from a second view acquired using a second sensor (e.g., a LIDAR point cloud) to generate filtered map data for inclusion in the filtered map. The filtering process can be repeated for a set of corresponding views (e.g., point clouds and camera images depicting the same portion of a route) to generate filtered map data for the entire route traveled by vehicle A.

[0053] At 610, the computer system stores the filtered map in a memory accessible to the computer system, such as a local memory or an external server. The computer system makes the filtered map available to vehicle B. For example, the filtered map can be retrieved from storage and sent to vehicle B and other vehicles in communication with the computer system or sent in response to a request (at 612) from vehicle B. In some examples, the filtered map can include additional sensor data provided by other vehicles traveling the same route or additional routes nearby. For example, vehicle A may be part of a fleet of vehicles tasked with mapping various portions of a geographic area. The additional sensor data can be filtered in the same manner as the sensor data acquired by vehicle A.

[0054] At 612, vehicle B requests map data from the computer system. The request at 612 may indicate the location for which map data is requested, e.g., a geographic area including a route traveled by vehicle A. The location indicated in the request can be specified at various levels of specificity. For example, the request can include the last known or estimated coordinates of vehicle B and / or can identify by name a geographic area or a sub-region within the geographic area. The request at 612 can be transmitted over one or more communication networks via a wireless connection (e.g., a cellular, Wi-Fi, or WiMax connection). The request at 612 can be timed such that the filtered map or a relevant portion thereof is received by vehicle B shortly before vehicle B arrives at the location depicted in the filtered map, e.g., a few seconds or minutes before the vehicle arrives at the location.

[0055] At 614, the computer system responds to the request at 612 by transmitting the filtered map to vehicle B, e.g., over the same wireless connection over which request 612 was transmitted. Depending on the amount of data to be transmitted, the computer system may transmit all or part of the filtered map. Figures 10A-10C, described below, show examples in which filtered map data is downloaded by a vehicle over time such that the vehicle receives incremental updates of map data related to the local area surrounding the vehicle.

[0056] At 616, vehicle B acquires sensor data while traveling along the same route traveled by vehicle A. Vehicle B does not necessarily travel the same route traveled by vehicle A at 602. However, the sensor data acquired at 616 overlaps with the sensor data acquired at 602 in that it includes one or more indications of the same locations along the route. In particular, the sensor data acquired at 616 may include an indication of vehicle B's current surroundings, which are also displayed in the map data that vehicle B received at 614. Thus, by comparing the sensor data acquired at 616 with the data in the filtered map, it may be possible to determine where vehicle B is located.

[0057] At 618, vehicle B compares the filtered map data with the sensor data acquired at 616 to detect vehicle B's location. That comparison at 618 may include identifying one or more matching features in the filtered map and determining that vehicle B is currently located at a particular location along a route previously traveled by vehicle A based on the presence of the matching feature or features. For example, referring to FIG. 3B, vehicle B may acquire a point cloud depicting road signs 240 and road edges 250, possibly from a slightly different perspective. The difference in perspective may be due to sensor configuration (e.g., vehicle B's LIDAR sensor is mounted at a different height than vehicle A's LIDAR sensor), vehicle position (e.g., vehicle B is traveling in a different lane than vehicle A was in), and / or other factors. By comparing the relative positions of road marks 240, road edges 250, and / or other features in the point cloud acquired by vehicle B, vehicle B may determine that such features match features contained in the filtered point cloud 310 and therefore that vehicle B's current location is the location depicted in the filtered point cloud 310.

[0058] 7 is a flow chart of a process 700 for generating a filtered map, according to an embodiment. Process 700 may be executed by one or more processors of a computer system configured to receive sensor data from one or more vehicles, such as computer system 150. At 702, the computer system generates an input map corresponding to a 3D representation of a route traveled by a first vehicle (e.g., vehicle A of FIG. 6) based on sensor data acquired by a first sensor of the first vehicle. The functionality at 702 may be implemented according to the process described in connection with 606 of FIG. 6 to generate an unfiltered or "raw" map as the input map.

[0059] At 704, the computer system filters the input map based on one or more criteria to generate a filtered map. The functionality at 704 may be implemented according to the process described in connection with 608 of FIG. 6, for example, using the filtering process illustrated in FIG. 4. Thus, the filtering at 704 may include automatically removing features that satisfy the one or more criteria. As described above, the criteria may be selected to correspond to attributes associated with features that are not necessary for location detection. Features that satisfy the one or more criteria may correspond to objects that are not relevant to detecting the location of a second vehicle (e.g., vehicle B of FIG. 6) with reference to the filtered map.

[0060] At 706, the computer system transmits the filtered map to the second vehicle. The computer system may store the filtered map prior to transmission, for example, as part of filtered map 162 in FIG. 1. After the second vehicle receives the filtered map, the filtered map may be processed by a navigation system of the second vehicle to detect the location of the second vehicle while the second vehicle is traveling along the same route. As discussed above in connection with 618 of FIG. 6, processing by the second vehicle may include comparing sensor data acquired by a sensor of the second vehicle with data from the filtered map to identify one or more matching features.

[0061] The filtered map can be updated after being transmitted at 706. The filtered map update can be based on sensor data acquired during subsequent trips on the same route. The subsequent trips can be performed by vehicle A or other vehicles in communication with the computer system. For example, the filtered map update can be performed based on sensor data acquired by a second vehicle. The computer system can compare the sensor data acquired during the subsequent trips to determine whether there are any significant changes to the filtered map, such as roads closed by barriers, lane diversions as indicated by traffic cones, detours, etc. In some implementations, the filtered map update can include manual input to select or confirm automatic selection of changes. The filtering of the sensor data used to update the filtered map can be performed in the same automated manner as when the filtered map was originally generated.

[0062] FIG. 8 is a flow chart of a process 800 for filtering an input map according to an embodiment. The process 800 can be executed by one or more processors of a computer system to implement the functionality in block 704 of FIG. 7. At 802, the computer system identifies features of the input map generated using sensor data from the first vehicle. The features identified at 802 correspond to ground reflections and / or moving objects. As described above in connection with FIG. 4, moving objects can be detected using point motion classification to identify groups of points that move in different directions or by different amounts from the acquisition vehicle (here, the first vehicle). Ground reflections can be identified by defining a ground surface based on detected boundaries of roads or drivable surfaces in the filtered map. Points or features that are located on the ground surface and are not classified as objects can be identified as ground reflections. Other techniques for identifying ground reflections may be used.

[0063] At 804, the computer system performs a first filtering step to remove features identified at 802 as corresponding to ground reflections and / or moving objects. The first filtering step produces a partially filtered map that contains less information (e.g., fewer points) than the input map but can be further filtered. The first filtering step is performed to remove features that can be identified as unwanted without relying on object classification. Because object classification can be computationally intensive compared to identifying ground reflections or moving objects, the first filtering step can speed up processing time by reducing the amount of data to which object-based filtering is applied.

[0064] At 806, the computer system identifies features in the input map that correspond to objects belonging to one or more classes. For example, the features identified at 806 include points associated with a bounding box labeled as a vehicle. Alternatively or additionally, the computer system identifies features that are greater than a threshold distance from the first vehicle. Identifying features that correspond to objects belonging to one or more classes can be performed as part of object-based filtering, using one or more object classifications as filter criteria. Identifying features that are greater than a threshold distance can be performed as part of distance-based filtering, using a threshold distance as filter criteria. For example, a threshold distance of 15 meters can be used to define a rectangular region of interest such that features that are greater than 15 meters away in either a horizontal or vertical direction are identified for removal.

[0065] At 808, the computer system performs a second filtering step to remove the features identified at 806. The second filtering step produces a filtered map, which can then be transmitted to the second vehicle.

[0066] FIG. 9 is a flow chart of a process 900 for transmitting a filtered map to a vehicle, according to an embodiment. Process 900 may be performed by one or more processors of a computer system in communication with the vehicle, such as computer system 150. Process 900 may be performed by the same computer system as the computer system that generates the filtered map or a different computer system. At 902, the computer system receives a request for map data from the vehicle. The request may be transmitted wirelessly through one or more communication networks, such as a 4G or 5G cellular network. The request may be transmitted in conjunction with the vehicle switching from GNSS-based navigation to sensor-based navigation. For example, the request may be transmitted when the vehicle is unable to receive and / or decode signals from GPS satellites. The request for map data does not necessarily have to occur in conjunction with the switch to sensor-based navigation. For example, the request at 902 may occur during GPS-based navigation or even during the course of a journey without navigation assistance, such that the filtered map is available if the vehicle needs to use the filtered map in the future.

[0067] At 904, the computer system identifies a region of the filtered map based on location information included in the request. The location information may correspond to an approximate location of the vehicle. For example, the request at 902 may include the nearest known or estimated coordinates of the vehicle (e.g., nearest location detected using a GNSS-based positioning method). Alternatively or additionally, the request may specify a region by name (e.g., street name or city name). The region may encompass the region ahead of the vehicle, e.g., the region the vehicle is expected to reach based on the vehicle's current trajectory. The region may further encompass areas the vehicle has already passed through. In some implementations, the filtered map is a 3D map (e.g., a 3D grid of points) that is divided into different regions along two dimensions (e.g., latitude and longitude) such that each region corresponds to a respective tile (rectangular block) in the 3D grid. Thus, the region identified at 904 may include one or more tiles that cover a portion of the geographic area represented in the filtered map.

[0068] At 906, the computer system transmits filtered map data for the area identified at 904 to the vehicle. The filtered map data may include data for one or more tiles in the 3D grid. For example, the filtered map may be in point cloud data (PCD) format, whereby the transmission at 906 includes transmitting one or more PCD files (e.g., a separate PCD file for each tile) that include data for one or more tiles. The process 900 may return to block 902 with a subsequent request for map data such that another area of ​​the filtered map may be identified and transmitted to the vehicle. In this manner, the vehicle may receive the filtered map data periodically (e.g., every few seconds or minutes). The filtering reduces the overall size of the map, which allows the filtered map to be transmitted to the vehicle with less latency. Reducing the map size may also reduce the time it takes to find the location of the vehicle, for example, because there are fewer points or features to match.

[0069] 10A-10C show an example of a filtered map 1000 received over a period of time, according to some embodiments. The filtered map 1000 is a 3D grid divided into square or rectangular tiles. Each tile represents a different segment of the filtered map and may contain one or more points corresponding to features within the covered geographic area. For example, the outline of a road 1010 is indicated by points located along the edge of the road. The filtered map 1000 as depicted in FIGS. 10A-10C is intended to illustrate how map data can be made available to a vehicle based on changes in the vehicle's location over time. The filtered map 1000 can be displayed to the driver of the vehicle, although display is optional.

[0070] In FIG. 10A, the filtered map 1000 is shown with concentric rings 1002. The rings 1002 are centered about the vehicle's current location. The outermost ring indicates the maximum detection range of a sensor (e.g., a LIDAR sensor) mounted on the vehicle. As shown in FIG. 10A, the filtered map 1000 covers a geographic area that encompasses not only a portion of a road 1010 at the vehicle's current location, but also an approaching portion of the road beyond the maximum detection range of the sensor and located in a region 1020 toward which the vehicle is heading. Thus, at the time instant depicted in FIG. 10A, the vehicle may have available map data that covers the geographic area up to and including the region 1020. However, map data for areas beyond the region 1020 (e.g., tiles corresponding to more distant sections of the road 1010) may not be available. The unavailability of some portions of the filtered map 1000 may be a result of bandwidth constraints and / or the data size of the filtered map 1000. If bandwidth is limited, or the size of the filtered map makes it impractical to transmit the filtered map all at once, the filtered map may instead be transmitted to the vehicles in portions.

[0071] In Figure 10B, a new tile (additional map segment 1030) has been added to the filtered map 1000 received by the vehicle. As shown in Figure 10B, the vehicle is now closer to region 1020 than in Figure 10A. The additional map segment 1030 may have been selected for transmission based on the current location of the vehicle, for example, such that the vehicle has map data for tiles up to a certain distance away from the vehicle's current location. Figure 10B also shows that the new tiles can be limited to the area around the current travel route, for example tiles covering road 1010.

[0072] Figure 10C shows another view of the filtered map 1000 at a time when the vehicle is located in the area represented by the additional map segment 1030. As shown in Figure 10C, the filtered map 1000 includes a dot representing a road 1010 that splits into two forks, with the vehicle located in the right fork, which forks are also shown in Figure 10B.

[0073] FIG. 11 shows an example of a filtered map 1100. The filtered map 1100 is shown from a closer perspective compared to the filtered map 1000 in FIGS. 10A-10C and may correspond to, for example, a zoomed-in view of one of the tiles in the filtered map 1000. FIG. 11 is provided to illustrate an example of features that may be used to match corresponding features acquired by a vehicle's sensor to detect the location of the vehicle. The filtered map 1100 shows a multi-lane road including a traffic lane 1110 in a first direction and a traffic lane 1120 in an opposite direction. The individual lanes are not visible. Instead, the road is represented as a collection of points indicating the edges of the road along both directions 1110 and 1120. Trees 1112 adjacent to the side of the road along the first direction 1110 and trees 1114 adjacent to the side of the road along the opposite direction 1120 are visible. Trees 1112 and 1114 may correspond to points remaining after applying a threshold distance, as described above. Trees 1112 and 1114 are not necessary for location detection, and in fact may be difficult to match due to changing appearance over time (e.g., seasonal foliage). However, it is still possible to identify the location of a vehicle, for example by matching it against road edges. Other features that may be matched to detect vehicle location may include, for example, overhead signs 1140 and barriers / walls 1130.

[0074] 12 is a flowchart of a process 1200 for updating a filtered map, according to one embodiment. Process 1200 may be performed by one or more processors of a computer system that has access to the filtered map, such as computer system 150. Process 1200 is described as being performed by the same computer system that generates the filtered map; however, the update may occur on a separate computer system. At 1202, the computer system generates an input map using sensor data acquired by one or more sensors of the first vehicle as the first vehicle travels along a route.

[0075] At 1204, the computer system filters the input map based on one or more criteria to generate a first filtered map. The functions at 1202 and 1204 can be implemented using the processes described in connection with blocks 702 and 704 of FIG. 7. However, unlike the example of FIG. 7, the first filtered map at 1204 is used to update an existing map, rather than as an initial example of a map. The existing map is a second filtered map generated in a similar manner to the first filtered map, using sensor data acquired by the first vehicle or another vehicle while traveling along the same route. In this manner, the first filtered map and the second filtered map at least partially overlap in terms of the geographic area covered. However, the first filtered map can cover a larger or smaller area than the area covered by the second filtered map. For example, if the second filtered map is the filtered map 1000 in FIG. 10A, the first filtered map can cover only some tiles (e.g., area 1020) or a portion of a single tile.

[0076] At 1206, the computer system identifies differences between the first filtered map and the second filtered map. Identifying differences may include comparing points or features in the first filtered map with corresponding points or features in the second filtered map. This may be done in a manner similar to the matching done as part of detecting a vehicle location. The comparison allows the computer system to identify areas that appear in both the first filtered map and the second filtered map. Furthermore, the comparison allows the computer system to determine the extent to which this common / overlapping area has changed from the time of the second filtered map to the time of the first filtered map.

[0077] At 1208, the computer system updates the second filtered map based on the differences identified at 1206. Not all differences are significant enough to justify updating the second filtered map. Instead, the computer system may be configured to update the second filtered map to reflect only changes in vehicle access along the route. Examples of changes in vehicle access include road closures (e.g., lane detours marked with barriers or traffic cones restricting access), detours (closure of highway entrances or exits at certain times of the day), and changes in road geometry (e.g., widening or narrowing of roads or addition of new road sections). In some examples, the computer system may automatically update the second filtered map, for example, by adding or removing points based on point cloud data included in the first filtered map. However, some types of updates may need to be manually verified before being made. For example, a human reviewer may verify that two lanes have merged due to construction, that a highway entrance is closed at certain times as indicated by a traffic sign, etc. The computer system may flag differences between the first and second filtered maps for manual review, for example, by displaying corresponding portions of the first and second filtered maps side-by-side.

[0078] Those skilled in the art will appreciate that substantial variations are possible depending on specific requirements. For example, customized hardware might also be used and / or particular elements might be implemented in hardware, software (including portable software such as applets), or both. Further, connections to other computing devices, such as network input / output devices, might be employed.

[0079] With reference to the accompanying figures, components that may include memory may include non-transitory machine-readable media. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any storage medium that participates in providing data that causes a machine to operate in a specific manner. In the embodiments provided above, various machine-readable media may participate in providing instructions / code to a processing unit and / or other device or devices for execution. Additionally or alternatively, machine-readable media may be used to store and / or convey such instructions / code. In many implementations, computer-readable media are physical and / or tangible storage media. Such media may take many forms, including but not limited to non-volatile media, volatile media, and transmission media. Common forms of computer-readable media include, for example, magnetic and / or optical media, punch cards, paper tape, any other physical media with patterns of holes, RAM, PROM, EPROM, FLASH-EPROM, any other memory chip or cartridge, carrier wave, or any other medium from which a computer can read instructions and / or code.

[0080] The methods, systems, and devices discussed herein are examples. Various embodiments may omit, substitute, or add various procedures or components as appropriate. For example, features described with respect to one embodiment may be combined in various other embodiments. Various aspects and elements of the embodiments may be combined in a similar manner. Various components of the diagrams provided herein may be embodied in hardware and / or software. Also, because technology evolves, many elements are examples, and the examples are not intended to limit the scope of the disclosure to those specific examples.

[0081] Although several embodiments have been described, various modifications, alternative constructions, and equivalents may be used without departing from the spirit of the disclosure. For example, the above elements may only be components of a larger system, and other rules may take precedence over or otherwise modify the application of the embodiments. Also, multiple steps may be taken before, during, or after considering the above elements. Thus, the above description is not intended to limit the scope of the disclosure to the precise embodiments described.

Claims

1. 1. A method for generating a map, comprising: generating, by the computer system, an input map corresponding to a three-dimensional representation of a route traveled by the first vehicle based on sensor data acquired by a first sensor of the first vehicle; filtering, by the computer system, the input map based on one or more criteria to generate a filtered map, the filtering including automatically removing features that satisfy the one or more criteria, the features corresponding to objects that are irrelevant to detecting a location of a second vehicle with reference to the filtered map; transmitting, by the computer system, the filtered map to the second vehicle, wherein the filtered map is processed by a navigation system of the second vehicle to determine a location of the second vehicle as the second vehicle travels along the same route; The method includes:

2. The method of claim 1 , wherein the one or more criteria include at least one class of objects to be removed, the at least one class including at least one of a vehicle, a pedestrian, an animal, or a plant.

3. The method of claim 1 , wherein the one or more criteria include at least one attribute of the objects to be removed, the at least one attribute including at least one of age, length, width, height, speed or shape.

4. The method of claim 1 , wherein the one or more criteria include a distance from the first vehicle, the distance being a threshold distance beyond which features are to be removed.

5. The method of claim 1 , wherein the filtered map shows edges of multi-lane roads along the route, and the filtered map excludes lanes within the multi-lane roads.

6. The method of claim 1 , wherein the first sensor is a LIDAR sensor, the input map includes a point cloud acquired by the LIDAR sensor, and the filtering includes removing points associated with the object.

7. The filtering step includes: identifying the features that satisfy the one or more criteria based on additional sensor data acquired by a second sensor of the first vehicle, the additional sensor data being acquired contemporaneously with the point cloud acquired by the LIDAR sensor, the second sensor being a camera, a radar sensor, or another LIDAR sensor; The method of claim 6 further comprising:

8. Identifying the features that satisfy the one or more criteria includes: detecting a boundary of the object using the additional sensor data; and mapping the boundary to corresponding points in the point cloud; The method of claim 7, comprising:

9. The filtering may include: performing a first filtering step in which features corresponding to ground reflections and moving objects are removed from the input map; and performing a second filtering step after the first filtering step, the second filtering step removing features corresponding to objects belonging to one or more classes; The method of claim 1 , comprising:

10. The method of claim 1 , wherein the filtered map is transmitted to the second vehicle via a wireless connection while the second vehicle is traveling along the same route, and features contained in the filtered map are matched with corresponding features in sensor data acquired by a sensor of the second vehicle to detect the location of the second vehicle.

11. one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the one or more processors to: generating an input map corresponding to a three-dimensional representation of a route traveled by the first vehicle based on sensor data acquired by a first sensor of the first vehicle; filtering the input map based on one or more criteria to generate a filtered map, the one or more processors being configured to automatically remove features that satisfy the one or more criteria, the features corresponding to objects that are irrelevant to detecting a location of a second vehicle with reference to the filtered map; transmitting the filtered map to the second vehicle, the filtered map being usable by a navigation system of the second vehicle to detect a location of the second vehicle while the second vehicle is traveling along the same route. system.

12. The system of claim 11 , wherein the one or more criteria include at least one class of objects to be removed, the at least one class including at least one of a vehicle, a pedestrian, an animal, or a plant.

13. The system of claim 11 , wherein the one or more criteria include at least one attribute of the objects to be removed, the at least one attribute including at least one of age, length, width, height, speed, or shape.

14. The system of claim 11 , wherein the one or more criteria include a distance from the first vehicle, the distance being a threshold distance beyond which features will be removed.

15. The system of claim 11 , wherein the filtered map shows edges of multi-lane roads along the route, and the filtered map excludes lanes within multi-lane roads.

16. the first sensor is a LIDAR sensor; the input map includes a point cloud acquired by the LIDAR sensor; the one or more processors are configured to identify the features that satisfy the one or more criteria based on additional sensor data acquired by a second sensor of the first vehicle; the additional sensor data is acquired contemporaneously with the point cloud acquired by the LIDAR sensor; the second sensor is a camera, a radar sensor, or another LIDAR sensor; the one or more processors are further configured to remove points associated with the object. The system of claim 11.

17. To identify the features that satisfy the one or more criteria, the one or more processors: Detecting a boundary of the object using the additional sensor data; mapping the boundary to a corresponding point in the point cloud; The system of claim 16 configured to:

18. To filter the input map, the one or more processors: performing a first filtering step in which features corresponding to ground reflections and moving objects are removed from the input map; performing a second filtering step after the first filtering step, the second filtering step removing features corresponding to objects belonging to one or more classes; The system of claim 11 configured to:

19. 12. The system of claim 11, wherein the filtered map is transmitted to the second vehicle via a wireless connection while the second vehicle is traveling along the same route, and the navigation system of the second vehicle is configured to detect the location of the second vehicle by matching features contained in the filtered map with corresponding features in sensor data acquired by a sensor of the second vehicle.

20. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors of a computer system, generating an input map corresponding to a three-dimensional representation of a route traveled by the first vehicle based on sensor data acquired by a first sensor of the first vehicle; filtering the input map based on one or more criteria to generate a filtered map, the filtering including automatically removing features that satisfy one or more criteria, the features corresponding to objects that are irrelevant to detecting a location of a second vehicle with reference to the filtered map; and transmitting the filtered map to the second vehicle, the filtered map being usable by a navigation system of the second vehicle to detect a location of the second vehicle while the second vehicle is traveling along the same route; a non-transitory computer-readable storage medium for causing the one or more processors to execute the

Images