Vehicle environment map methods and systems

Abstract

An example method for generating at least one environment map representing a spatial area, the environment map useable at least partially for vehicle navigation, comprises: using a plurality of vehicles, at least partially scanning the spatial area to obtain sensed data representing the spatial area; transmitting the sensed data from the plurality of vehicles to an environment map computing system; receiving, at the environment map computing system, the transmitted sensed data; using the received sensed data, generating, at the environment map computing system, the at least one environment map; and transmitting the generated at least one environment map from the environment map computing system to the plurality of vehicles.

Description

VEHICLE ENVIRONMENT MAP METHODS AND SYSTEMSCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] The application claims the benefit of United States Provisional Patent Application No. 63 / 665,290 filed on June 28, 2024, hereby incorporated by reference herein.FIELD

[0002] The present disclosure relates generally to mapping systems and methods and, in particular, to mapping systems and methods for autonomous vehicles or Advance Driver Assistance Systems (ADAS) equipped vehicles.BACKGROUND

[0003] Vehicles equipped with autonomous driving functionality, ADAS, or both typically include one or more sensors. Such a vehicle may use the one or more sensors to obtain data representing an area surrounding the vehicle. Using the obtained data, the vehicle may generate an environment map which is specific to that vehicle to represent the area surrounding the vehicle. Generating the environment map typically requires significant computational resources from the vehicle as well as significant amounts of power. As such, improved methods and systems for generating environment maps are desirable.SUMMARY

[0004] According to at least one embodiment, a method for generating at least one environment map representing a spatial area, the environment map useable at least partially for vehicle navigation, comprises: using a plurality of vehicles, at least partially scanning the spatial area to obtain sensed data representing the spatial area; transmitting the sensed data from the plurality of vehicles to an environment map computing system; receiving, at the environment map computing system, the transmitted sensed data; using the received sensed data, generating, at the environment map computing system, the at least one environment map; and transmitting the generated at least one environment map from the environment map computing system to the plurality of vehicles.

[0005] In some embodiments, the method further comprises aligning the received sensed data prior to generating the at least one environment map.

[0006] In some embodiments, the received sensed data comprises data of a plurality of types.

[0007] In some embodiments, the method further comprises combining a first type of the received sensed data with a second type of the received sensed data.

[0008] In some embodiments, the first type of the received sensed data is aligned and the second type of the received sensed data is aligned prior to the first type of the received sensed data being combined with the second type of the received sensed data.

[0009] In some embodiments, the first type of the received sensed data corresponds to data obtained from a first sensor type and the second type of the received sensed data corresponds to data obtained from a second sensor type.

[0010] In some embodiments, aligning the received sensed data comprises temporally aligning the received sensed data, spatially aligning the received sensed data or both.

[0011] In some embodiments, generating the at least one environment map comprises segmenting the spatial area and for each segment generating a corresponding environment map.

[0012] In some embodiments, the method further comprises transmitting the corresponding environment maps from the environment map computing system to vehicles of the plurality of vehicles based on locations of the vehicles within the spatial area.

[0013] In some embodiments, segmenting the spatial area comprises segmenting the spatial area such that adjacent segments overlap by at least a threshold amount.

[0014] In some embodiments, each segment comprises a circular boundary and the spatial area is segmented according to a circle packing process.

[0015] In some embodiments, the method further comprises controlling, by the environment map computing system, the scanning of the spatial area by the plurality of vehicles based on a density of the received sensed data.

[0016] In some embodiments, the method further comprises updating, by the environment map computing system, the generated at least one environment map in response to receiving, at the environment map computing system, new data from the plurality of vehicles.

[0017] In some embodiments, transmitting the generated at least one environment map to the plurality of vehicles comprises transmitting a first portion of the generated at least one environment map to a first vehicle of the plurality of vehicles and a second portion of the generated at least one environment map to a second vehicle of the plurality of vehicles.

[0018] In some embodiments, the first portion and the second portion are different.

[0019] In some embodiments, at least partially scanning the spatial area to obtain sensed data representing the spatial area comprises scanning the spatial area with at least a first vehicle comprising a first set of sensors and a second vehicle comprising a second set of sensors.

[0020] In some embodiments, the first set of sensors and the second set of sensors are different.

[0021] In some embodiments, the plurality of vehicles are communicatively coupled to the environment map computing system by a communication network.

[0022] In some embodiments, the communication network is a cellular network.

[0023] In some embodiments, the cellular network is a 5G cellular network.

[0024] According to at least one embodiment, a method for execution by an environment map computing system comprises causing the environment map computing system to at least: receive, from a plurality of vehicles, sensed data representing a spatial area; and generate, based at least partially on the received sensed data, at least one environment map representing the spatial area, the environment map useable at least partially for vehicle navigation.

[0025] In some embodiments, the method further comprises causing the environment map computing system to, at least, transmit the generated environment map to the plurality of vehicles.

[0026] In some embodiments, the plurality of vehicles are communicatively coupled to the environment map computing system by a communication network.

[0027] In some embodiments, the communication network is a cellular network.

[0028] In some embodiments, the cellular network is a 5G cellular network.

[0029] Other aspects and features will become apparent to those ordinarily skilled in the art upon review of the following description of illustrative embodiments in conjunction with the accompanying figures.BRIEF DESCRIPTION OF THE DRAWINGS

[0030] While the invention is claimed in the concluding portions hereof, example embodiments are provided in the accompanying detailed description which may be best understood in conjunction with the accompanying figures where like parts in each of the several figures are labeled with like numerals, and where:

[0031] FIG. 1 is a schematic illustration of an environment map system according to one embodiment;

[0032] FIG. 2 is a schematic illustration of a vehicle according to one embodiment;

[0033] FIG. 3 is a schematic illustration of an environment map according to one embodiment;

[0034] FIG. 4 is a block diagram showing a method for generating an environment map according to one embodiment;

[0035] FIG. 5 is a block diagram showing a method for transmitting data according to one embodiment;

[0036] FIG. 6 is a block diagram showing a method for combining data according to one embodiment;

[0037] FIG. 7 is a schematic illustration of a spatial area segmented into a plurality of zones according to one embodiment; and

[0038] FIG. 8 is a block diagram showing a method for generating an environment map according to one embodiment.DETAILED DESCRIPTION

[0039] The following discussion provides many example embodiments of the inventive subject matter. Although each embodiment represents a single combination of inventive elements, the inventive subject matter is considered to include all possible combinations of the disclosed elements.

[0040] Referring to FIG. 1 , a system for generating at least one environment map representing a spatial area useable by vehicles to at least partially navigate the spatial area is shown generally at 10. The system 10 includes an environment map computing system 11. Vehicles 12 may be communicatively coupled to the environment map computing system 11 with a communication network 13. As described herein, a vehicle 12 may be configured to (or may be operable to) collect (or sense) data which at least partially represents an area surrounding the vehicle 12. The data collected (or sensed) by the vehicles 12 may be transmitted to the environment map computing system 11. The environment map computing system 11 may be configured to (or may be operable to) receive the data from the vehicles 12 and to generate the environment map based at least partially on the received data. The environment map computing system 11 may provide the generated environment map (or portions thereof) to the vehicles 12.

[0041] In the illustrated embodiment, the environment map computing system 11 includes a central processing unit (CPU) 15. However, alternative embodiments may include one or more alternatives to the CPU 15, such as one or more microprocessors, one or more analog circuits, one or more configurable logic blocks, one or more application-specific integrated circuits (ASICs), or one or more field programmable gate arrays (FPGAs), for example. The computingsystem 11 also includes a network interface (or network interface controller) 16 and a data-storage device 17 in communication with the CPU 15.

[0042] The network interface 16 is configured to (or is operable to) communicatively couple the CPU 15 with the communication network 13. The network interface 16 may include various signal interfaces, analog-to-digital converters (ADCs), receivers, transmitters and / or other circuitry to receive, produce and transmit signals over the communication network 13.

[0043] The data-storage device 17 may include one or more of the same or different computer- readable and / or computer writable data-storage media, which in various embodiments may include one or more of a read-only memory (ROM), a random access memory (RAM), a hard disc drive (HDD), a solid-state drive (SDD) and other computer-readable and / or computer-writable data-storage media.

[0044] The data-storage device 17 includes a program-codes store 18 storing program codes that, when executed by the CPU 15, cause the environment map computing system 11 to implement functions such as those described herein, for example.

[0045] The data-storage device 17 may also include an environment map store 19 storing data representing the environment map (or a plurality of environment maps).

[0046] The environment map computing system 11 may be a distributed computing system or a non-distributed computing system.

[0047] The communication network 13 may include, or be, a computer network. In some embodiments, the communication network 13 is a cellular network. In some embodiments, the communication network 13 is a fifth generation (5G) 3GPP cellular network. In some embodiments, the communication network 13 is a secure network.

[0048] Vehicles 12 may include, or be, any vehicle. For example, vehicles 12 may include vehicles such as passenger cars, passenger trucks, transport trucks, transport vans, busses, motorcycles, emergency vehicles (e.g. police cars, ambulances, fire trucks, etc.), construction vehicles, municipal works vehicles (e.g. municipal maintenance vehicles, snow plows, garbage trucks, etc.), etc. A vehicle 12 may be configured to (or may be operable to) be autonomouslydriven. Additionally, or alternatively, a vehicle 12 may include one or more Advanced Driver Assistance Systems (ADAS) such as, for example, autonomous emergency braking (front and / or rear), autonomous lane keep assistance, lane change assistance, autonomous blind spot monitoring, autonomous pedestrian detection and / or avoidance, cross-traffic detection and / or avoidance, automatic cruise control, collision mitigation assistance, etc. The autonomous driving functionality, at least one ADAS, or both of a vehicle 12 may at least partially use (or be based on) the environment map generated by the environment map computing system 11.

[0049] With reference to FIG. 2, an example embodiment of a vehicle 12 is shown. In the illustrated embodiment, the vehicle 12 includes one or more sensors 21 . The one or more sensors 21 may be configured to (or may be operable to) sense one or more characteristics of the vehicle (e.g. heading of the vehicle, orientation of the vehicle, speed of the vehicle, location of the vehicle, etc.), one or more characteristics of an area surrounding the vehicle (e.g. road markings or signs, other vehicles, pedestrians, potential obstacles, etc.) or both.

[0050] The one or more sensors 21 may include: a location sensor configured to determine a current location of the vehicle (the location sensor may, for example, include, or be, a Global Positioning System (GPS) module operable to provide a current location of the vehicle); an inertial measurement unit (IMU); a speed sensor configured to determine a current speed of the vehicle; an odometer; a compass; one or more image capture devices (or cameras) configured to (or operable to) capture still or video images of the area surrounding the vehicle; one or more Light Detection and Ranging (LiDAR) systems; one or more radar sensors; one or more ultrasonic sensors; combinations of two or more thereof; etc.

[0051] The one or more sensors 21 may be positioned at various locations on or within the vehicle 12. A positioning of a particular sensor 21 may at least partially control what portion of an area that surrounds the vehicle 12 is sensed by the particular sensor 21. In some embodiments, a particular sensor 21 is positioned on the vehicle 12 such that the particular sensor 21 is front facing, rear facing, side facing or a combination of two or more thereof. In some embodiments, the one or more sensors 21 collectively sense about a 360° area surrounding the vehicle 12.

[0052] Different ones of the sensors 21 may have different sensing ranges. In some embodiments, at least one of the sensors 21 is a short range sensor (e.g. a sensor having a sensing range of less than about 2m). In some embodiments, at least one of the sensors 21 is amedium range sensor (e.g. a sensor having a sensing range in a range from about 2m to about 50m). In some embodiments, at least one of the sensors 21 is a long range sensor (e.g. a sensor having a sensing range in a range greater than about 50m).

[0053] The one or more sensors 21 may be in communication with a CPU 22. Alternative embodiments of the vehicle 12 may include one or more alternatives to the CPU 22, such as one or more microprocessors, one or more analog circuits, one or more configurable logic blocks, one or more application-specific integrated circuits (ASICs), or one or more field programmable gate arrays (FPGAs), for example.

[0054] The illustrated example vehicle 12 also includes a data-storage device 23 in communication with the CPU 22. The data-storage device 23 may include one or more of the same or different computer-readable and / or computer writable data-storage media, which in various embodiments may include one or more of a read-only memory (ROM), a random access memory (RAM), a hard disc drive (HDD), a solid-state drive (SDD) and other computer-readable and / or computer-writable data-storage media. The data-storage device 23 includes a sensor data store 24 storing data sensed (or collected) by the one or more sensors 21. The data-storage device 23 also may include an environment map data store 25 storing an environment map received from the environment map computing system 11.

[0055] The illustrated example vehicle 12 also includes a network interface (or network interface controller) 26 in communication with the CPU 22. The network interface 26 is configured to (or is operable to) communicatively couple the CPU 22 with the communication network 13. The network interface 26 may include various signal interfaces, analog-to-digital converters (ADCs), receivers, transmitters and / or other circuitry to receive, produce and transmit signals over the communication network 13.

[0056] Different ones of the vehicles 12 may be the same or different. In some embodiments, different ones of the vehicles 12 include different ones of the sensors 21 , different configurations of the sensors 21 or both.

[0057] An environment map includes data representing a spatial area. The environment map may, for example, be used by a vehicle 12 to navigate the spatial area (e.g. if the vehicle 12 is a vehicle which may be autonomously driven) or may at least partially be used by an ADAS of thevehicle 12. The environment map may identify objects which are present in the spatial area represented by the environment map. The environment map may additionally identify spatial locations of the objects within the spatial area represented by the environment map. Additionally, the environment map may identify one or more spatial relationships between the objects which are present in the spatial area represented by the environment map.

[0058] The environment map may include, or be, an image of the spatial area represented by the environment map. The environment map may be two-dimensional (2D) or three-dimensional (3D). The environment map may comprise a plurality of pixels (if the environment map is 2D) or voxels (if the environment map is 3D). The value of a pixel or a voxel may identify what object (if any) is represented by the pixel or voxel. The location of the pixel or the voxel within the environment map may identify a location of the object within the spatial area (or a spatial relationship). The location of the object may at least partially be relative to a current location of the viewer (e.g. a vehicle 12) of the environment map.

[0059] The spatial area represented by the environment map may vary with time. For example, an object which is present within the spatial area at a first time may no longer be present in the spatial area at a second time that is different than the first time. As another example, an object moving within the spatial area at a first heading at a first time may be moving within the spatial area at a second heading at a second time that is different than the first time. As such, the environment map may represent a snapshot of the spatial area at a particular time. The environment map may include data identifying a time to which the environment map corresponds to. For example, if the data identifies the time as 5:03pm, then the environment map represents the spatial area as it was at 5:03pm. In some embodiments, the data identifying a time to which the environment map corresponds to also includes a date.

[0060] Additionally, or alternatively, the environment map may include data identifying a location of the spatial area represented by the environment map. For example, the environment map may comprise GPS coordinates identifying the spatial area represented by the environment map.

[0061] The environment map may identify various objects. For example, the objects may include roads or roadways, vehicles, pedestrians, road signs, road markings, animals, buildings, traffic lights, plants or combinations of two or more thereof. An object may be moving or non-moving. Different objects may move at different headings and / or speeds. Additionally, or alternatively, anobject may have a varying state. For example, if the object is a traffic light, the traffic light may cycle through a first state corresponding to a green light, a second state corresponding to a yellow light and a third state corresponding to a red light. The state of the object may be different depending on a time that the environment map corresponds to. Other example objects which may have varying states include dynamically varying speed limits, lane control lights, rail-road crossings, crosswalk signals, pedestrians, vehicles, etc. Additionally, or alternatively, an object may encode information which directly or indirectly affects navigation of the spatial area by a vehicle. For example, a speed limit sign encodes a maximum speed at which vehicles should travel at. The maximum speed may directly affect navigation of the spatial area by the vehicle (e.g. the vehicle may be controlled to not exceed the maximum speed). As another example, detection of brake lights being activated on one or more other vehicles may indirectly cause the vehicle to be controlled to slow down. Additionally, or alternatively, relationships between objects may directly or indirectly affect navigation of the spatial area by the vehicle. For example, a pedestrian being identified as proximal to a crosswalk may cause a vehicle to be controlled to slow down as the vehicle approaches the crosswalk.

[0062] The environment map may represent various spatial areas. For example, the spatial area may include, or be, a portion of a municipal block, an entire municipal block, a plurality of municipal blocks, an intersection, a plurality of intersections, a municipality, an entire road network of a province or a state, an entire road network of a country, etc. Different environment maps representing different spatial areas may be combined together.

[0063] As data is collected or sensed by the vehicles 12, the environment map may be varied or updated in view of the newly collected or updated data.

[0064] Referring to FIG. 3, an example of an environment map 30 representing a road 31 is shown. The illustrated example environment map 30 is a 2D representation of the road 31. The environment map 30 includes a subset of pixels 32 identifying a center road line, a subset of pixels 33 identifying a roadway, a subset of pixels 34 identifying a vehicle present in the roadway and a subset of pixels 35 identifying a pedestrian.

[0065] As described elsewhere herein, the vehicles 12 collect (or sense) data representing their surroundings. The vehicles 12 transmit the collected data to the environment map computing system 11. The environment map computing system 11 may collate (or gather or combine) thedata received from the vehicles 12 and may be caused to generate at least one environment map from the collated data. The generated at least one environment map may then be provided to the vehicles 12. Such process may be repeated continuously, periodically, non-periodically, etc.

[0066] Generating the environment map using the environment map computing system 11 rather than a vehicle 12 generating its own environment map from its collected (or sensed) data reduces the computational power required by the vehicle 12, decreases power consumption of the vehicle 12 or both. Additionally, computational efficiency is improved by centrally processing the collected data from the vehicles 12 at the environment map computing system 11 to generate an environment map that may be used by a plurality of the vehicles 12.

[0067] In some cases, the one or more sensors 21 may have sensor occlusions or blind spots (e.g. areas surrounding a vehicle 12 from which its sensors 21 cannot cover). In some cases, the sensor occlusions or blind spots are caused due to physical limitations of the sensors. In some cases, the sensor occlusions or blind spots are caused as a result of weather or environmental factors (e.g. fog may occlude one or more sensors, snow build-up on a vehicle 12 may occlude one or more sensors, etc.). By collating (or collecting) sensor data from multiple vehicles 12, the occurrence of sensor occlusions or blind spots may be reduced or eliminated. Reducing or eliminating occlusions or blind spots may improve accuracy or a confidence level of a generated environment map. For example, an occluded region of a first vehicle 12 may be sensed by a second vehicle 12 that is adjacent (or otherwise proximal) to the first vehicle 12. As another example, if dense fog is present reducing the range of one or more cameras of a vehicle 12, data collected by other vehicles 12 may replace the missing data caused by the fog-induced reduced range of the one or more cameras.

[0068] The environment map may be continuously evolving. As data is collected by the vehicles 12, the environment map computing system 11 may update the environment map. Since data is collected by multiple vehicles 12, a vehicle 12 may receive in advance an environment map corresponding to a spatial area the vehicle will be navigating through in the future which was generated based on data collected by other vehicles 12 presently in that area.

[0069] Referring to FIG. 4, an example method 40 for generating an environment map is shown.

[0070] At block 41 , data representing areas surrounding the vehicles 12 is collected by the vehicles 12. The vehicles 12 may collect sets of sensed data 42 (e.g. a set of sensed data 42-1 corresponding to a vehicle 12-1 , a set of sensed data 42-2 corresponding to a vehicle 12-2, ... , a set of sensed data 42-N corresponding to a vehicle 12-N). The sensed data 42 from a vehicle 12 may be collected with the one or more sensors 21 of the vehicle 12. The sensed data 42 of a vehicle 12 may include a time at which the data was acquired, a location of the vehicle 12 when the data was acquired, an orientation of the vehicle 12 when the data was acquired or combinations thereof. In some embodiments, the sensed data 42 is at least temporarily stored by vehicles 12 (e.g. in sensor data stores 24).

[0071] The sensed data 42 may be transmitted from the vehicles 12 to the environment map computing system 11 at block 43. The sensed data 42 may, for example, be transmitted from the vehicles 12 to the environment map computing system 11 over the communication network 13.

[0072] At block 44, the environment map computing system 11 may receive the transmitted sensed data 42 from the vehicles 12. In some embodiments, the environment map computing system 11 at least temporarily stores the received sensed data 42. For example, the environment map computing system 11 may store the received sensed data 42 in the data-storage device 17.

[0073] At block 45, the environment map computing system 11 may combine the received sensed data 42 from the vehicles 12 together into a combined dataset 46. As described elsewhere herein, combining the received sensed data 42 into a combined dataset 46 may include temporally and / or spatially aligning the data from the different vehicles 12, fusing the data of a particular type from the different vehicles 12 (e.g. data from a particular sensor type but from different vehicles 12) together and fusing different types of data (e.g. data from different sensor types) together.

[0074] Using the combined dataset 46, the environment map computing system 11 at block 47 may generate at least one environment map 48 representing a spatial area. The environment map computing system 11 may store the generated environment map 48 in, for example, the environment map store 19. The generated environment map 48 may include data identifying a time at which the environment map 48 was generated. The generated environment map 48 may be updated. For example, as new sensed data 42 is received, the environment map computing system 11 may update the generated environment map 48 in view of the new sensed data 42which was received. In some embodiments, the generated environment map 48 is continuously updated.

[0075] The generated environment map 48 may be transmitted from the environment map computing system 11 to the vehicles 12 at block 49. The generated environment map 48 may, for example, be transmitted from the environment map computing system 11 to the vehicles 12 over the communication network 13. When a vehicle 12 receives the environment map 48, the vehicle 12 may store the received environment map 48 in the environment map data store 25 of the vehicle 12.

[0076] The blocks of method 40 need not happen sequentially. Blocks of method 40 may occur in parallel. For example, while the environment map computing system 11 is combining data (e.g. block 45) or generating the environment map 48 (e.g. block 47), at least a portion of the vehicles 12 may collect new (or updated) sensed data sets 42 (e.g. block 41), transmit the new (or updated) sensed data sets 42 to the environment map computing system 11 (e.g. block 43) or both. The blocks of method 40 may also repeat until generation of the environment map 48 is no longer desired.

[0077] Blocks or steps of the method 40 will now be described in further detail.

[0078] The one or more sensors 21 of a vehicle 12 may be configured to collect (or sense) data representing the vehicle 12’s surroundings by, for example, the CPU 22 of the vehicle 12. The one or more sensors 21 may be configured by the CPU 22 to collect (or sense) data at a specified frequency. The specified frequency may be different for different ones of the one or more sensors 21. In some embodiments, at least one sensor 21 is configured to continuously collect (or sense) data. In some embodiments, at least one sensor 21 is configured to collect (or sense) data after a threshold amount of time has passed (e.g. every 10 seconds, every 30 seconds, every minute, every five minutes, etc.). In some embodiments, at least one sensor 21 is configured to collect (or sense) data after the vehicle 12 has travelled a threshold distance (e.g. every one metre, every two metres, every five metres, every 10 metres, every 50 metres, every 100 metres, every 500 metres, every one kilometre, every 10 kilometres, etc.).

[0079] As described elsewhere herein, the data collected (or sensed) by the one or more sensors 21 may include a time at which the data was acquired, a location of the vehicle 12 at the time thedata was acquired, an orientation of the vehicle 12 at the time the data was acquired or combinations thereof. In some embodiments, the CPU 22 of the vehicle 12 is caused to include (or associate) a time (e.g. based on a clock of the vehicle 12), a location (e.g. obtained from a location module such as a GPS module), an orientation (e.g. based on a reading of an IMU of the vehicle 12, obtained from a location module such as a GPS module, etc.) or a combination thereof with data of the one or more sensors 21 .

[0080] In some embodiments, which ones of the one or more sensors 21 of a vehicle 12 collect (or sense) data and / or at what frequency the data is collected (or sensed) may at least partially be set by the environment map computing system 11. For example, the environment map computing system 11 may need more data of a first type than data of a second type to generate the environment map 48. The environment map computing system 11 may cause the one or more sensors 21 configured to collect data of the first type to collect data at a higher frequency than a frequency at which sensors are configured to collect data of the second type. As another example, based on how much data the environment map computing system 11 has of an area (e.g. how dense received sensed data 42 is), the environment map computing system 11 may control the one or more sensors 21 of a vehicle 12 to collect more data, the same amount of data, less data or no data. The environment map computing system 11 may control the one or more sensors 21 of a vehicle 12 by, for example, transmitting to the vehicle 12 what data is required by the environment map computing system 11 to generate the environment map 48. Additionally, or alternatively, the environment map computing system 11 may transmit to the vehicle 12 at what frequency the required data should be acquired at.

[0081] Referring to FIG. 5, an example method 50 for transmitting collected (or sensed) data from a vehicle 12 to the environment map computing system 11 is shown.

[0082] At block 51 , the one or more sensors 21 of the vehicle 12 collect (or sense) data. As described elsewhere herein, the one or more sensors 21 of the vehicle 12 may be configured to collect (or sense) data continuously, at a desired frequency, etc.

[0083] At block 52, the method 50 optionally determines whether the collected (or sensed) data includes new data. “New data” means collected (or sensed) data whose value varies from previously transmitted data by at least a threshold amount. The threshold amount may be a threshold change in a value represented by the data, a threshold change in time, a thresholdchange in location of the vehicle 12, a threshold change in orientation of the vehicle 12 or combinations thereof, for example. If no new data has been collected (or sensed), then method 50 may return to block 51 . If, however, new data has been collected (or sensed), then method 50 may proceed to block 53.

[0084] Alternatively, block 51 may proceed directly to block 53.

[0085] At block 53, the collected (or sensed) data (or newly collected (or newly sensed) data) is transmitted to the environment map computing system 11. As described elsewhere herein, the collected (or sensed) data may be transmitted to the environment map computing system 11 over the communication network 13.

[0086] Data from different ones of the one or more sensors 21 may be transmitted to the environment map computing system 11 at the same time or at different times. In some embodiments, a vehicle 12 is caused to transmit data collected from (or sensed by) each of its one or more sensors 21 simultaneously. In some embodiments, a vehicle 12 is caused to transmit data collected from (or sensed by) different ones of its one or more sensors 21 at different times. In some embodiments, a vehicle 12 is caused to transmit new data collected from (or sensed by) any sensor of the one or more sensors 21 in response to the new data being collected (or sensed).

[0087] The sensed data 42 received by the environment map computing system 11 may be associated with a plurality of times and / or a plurality of locations corresponding to the different times at which the data was collected (or sensed) and the different vehicles 12 which collected (or sensed) the data. The environment map computing system 11 may be configured to (or operable to) align the received sensed data 42 based on a time at which the data was acquired (i.e. “temporally align” the data). Additionally, or alternatively, the environment map computing system 11 may be configured to align the received sensed data 42 based on a location at which the data was acquired (i.e. “spatially align” the data).

[0088] In some embodiments, the vehicles 12 provide time information to the environment map computing system 11 in a Coordinated Universal Time (UTC) format. In some embodiments, the vehicles 12 provide time information to the environment map computing system 11 in a time format of the environment map computing system 11. In some embodiments, the vehicles 12 provide time information to the environment map computing system 11 in a time format of thecommunication network 13. In some embodiments, each of the vehicles 12 may be configured to provide time information to the environment map computing system 11 in the same format.

[0089] Different ones of the one or more sensors 21 may have different data transmission or publication rates. The environment map computing system 11 may be configured to apply a thresholding method to the received sensed data 42. For example, a thresholding method may be applied by the environment map computing system 11 between two temporal samples of data to at least partially temporally align the received sensed data 42 (e.g. data having a time between a first time and a second time may be allocated to the first time or second time by the environment map computing system 11 applying the thresholding method). The environment map computing system 11 may be configured to correct a result of the thresholding if it is detected by the environment map computing system 11 that the thresholding was incorrectly applied.

[0090] As described elsewhere herein, the received sensed data 42 may include data identifying a location (or locations) of the area which are represented by the received sensed data 42. The data identifying the location (or locations) may, for example, be absolute (e.g. such as GPS coordinates) or relative (e.g. a location may be identified relative to one or more other objects such as a stop sign, a cross-walk, a building, a pedestrian, etc.).

[0091] To determine a location (or locations) represented by data from the one or more sensors 21 of a vehicle 12, a calibration transform may, for example, be applied to a measured location (e.g. a GPS location) of the vehicle 12. The calibration transform may translate the measured location of the vehicle 12 to one or more actual locations represented by the data from the one or more sensors 21 of the vehicle 12. The calibration transform may be at least partially based on a range of a sensor, a field of view of the sensor, sensor placement relative to the vehicle 12, etc. The calibration transform may include, for example, one or more calibration values which may be applied to the measured location of the vehicle 12 to determine one or more locations of the sensed data. In some embodiments, the calibration transform includes a matrix of calibration values which may be applied to the measured location of the vehicle 12. In some embodiments, each sensor type of a vehicle 12 has a corresponding calibration transform. In some embodiments, each sensor of a vehicle 12 has a corresponding calibration transform. The calibration transform may be applied to sensed data by the vehicle 12, the environment map computing system 11 or both.

[0092] The received sensed data 42 may be at least partially spatially aligned by the environment map computing system 11 based on the calibrated location data. For example, data representing the same location may be aligned together by the environment map computing system 11.

[0093] Additionally, or alternatively, the received sensed data 42 may be spatially aligned by the environment map computing system 11 based on one or more objects represented by the received sensed data 42. The one or more objects may be stationary (e.g. buildings, road signs, benches, etc.) or non-stationary (e.g. pedestrians, another vehicle, etc.). By aligning received sensed data 42 which represents the same object (or objects), the environment map computing system 11 may at least partially spatially align the received data 42 together.

[0094] In some embodiments, the sensed data 42 is grouped by type. The type of data may be based on or correspond to a sensor (or sensors) which was used to acquire the data. For example, data of the sensed data 42 which was acquired by radar sensors may be grouped into a first type or group corresponding to radar data, data of the sensed data 42 which was acquired by cameras may be grouped into a second type or group corresponding to camera data, data of the sensed data 42 which was acquired by LiDAR systems may be grouped into a third type or group corresponding to LiDAR data, etc. In some embodiments, the different types of data are based on different data formats that the data was stored in. For example, data which is in RGB-D format may be of a first type while data which is in JPEG format may be of a second type.

[0095] For each group of a type of data of the sensed data 42, the environment map computing system 11 may be configured to (or operable to) align the type of data with respect to time and / or location as described elsewhere herein. Once the data of a group corresponding to a type of data is aligned with respect to time and / or location, the data of that group may be combined or fused together into a single dataset. For example, the data of the group may be combined or fused together into a single image map. The single datasets of the different groups of data (or different types of data) may then be combined by the environment map computing system 11 into a single overall dataset (e.g. the combined dataset 46) which may be used by the environment map computing system 11 to generate the environment map 48.

[0096] In some embodiments, the sensed data 42 is in RGB-D format.

[0097] Different ones of the sensors 21 may be configured to store data using the same or different data formats.

[0098] In some embodiments, the environment map computing system 11 is configured to (or is operable to) convert a data format of at least a portion of the received sensed data 42 such that the received sensed data 42 includes, or is in, a uniform data format. In some embodiments, the uniform data format includes, or is, RGB-D formatted data.

[0099] Referring to FIG. 6, an example method 60 for combining received sensed data 42 is shown.

[0100] At block 61 , data of a particular type is aligned with respect to time (i.e. “temporally aligned”) by the environment map computing system 11.

[0101] At block 62, data of the particular type is aligned with respect to location (i.e. “spatially aligned”) by the environment map computing system 11.

[0102] Blocks 61 and 62 may be performed simultaneously in parallel or sequentially. In some embodiments, block 61 is performed prior to block 62. In some embodiments, block 62 is performed prior to block 61 .

[0103] At block 63, aligned data of the particular type is combined into a single data set (e.g. a single RGB-D dataset) by the environment map computing system 11.

[0104] At block 64, method 60 or the environment map computing system 11 may determine whether the received sensed data 42 contains another type of data. If so, method 60 may perform blocks 61 , 62 and 63 for the next type of data. Otherwise, method 60 may proceed to block 65.

[0105] At block 65, the combined single datasets corresponding to each type of data that was present in the received sensed data 42 may be combined together into a single overall combined dataset (e.g. the combined dataset 46) by the environment map computing system 11 . Combining the single datasets corresponding to each type of data may include aligning the single datasets with respect to time and / or location by the environment map computing system 11.

[0106] As described elsewhere herein, the environment map computing system 11 may at least partially temporally align different ones of the single datasets by applying a thresholding method.

[0107] Additionally, or alternatively, the environment map computing system 11 may at least partially spatially align different ones of the single datasets based on the location information of the datasets, one or more objects represented by the datasets or both.

[0108] In some embodiments, the environment map computing system 11 first spatially aligns the datasets based on location information (e.g. GPS coordinates, calibrated coordinates as described elsewhere herein, etc.). To fine tune or adjust the spatial alignment, the environment map computing system 11 may then further spatially align the datasets based on objects or other features represented by the data (e.g. the environment map computing system 11 spatially aligns all of the data representing the same stop sign). A location of at least one of the objects or other features used to spatially align the datasets may be known by the environment map computing system 11.

[0109] In some embodiments, the environment map computing system 11 translates location information (e.g. GPS coordinates) from a vehicle 12 into a location format of the environment map computing system 11.

[0110] In some embodiments, the environment map computing system 11 first spatially and / or temporally aligns together the datasets from a vehicle 12 prior to spatially and / or temporally aligning together the datasets from different ones of the vehicles 12.

[0111] In some embodiments, aligning the received sensed data 42 or combining the received sensed data 42 includes inferring data or generating additional data from the received sensed data 42. For example, aligning the received sensed data 42 and / or combining the received sensed data 42 may include interpolating and / or extrapolating data values based on the received sensed data 42.

[0112] The environment map computing system 11 may use the combined dataset 46 to generate the environment map 48. As the aligned data 46 is updated or refreshed, the environment map 48 may be updated or refreshed to take into account the updated data. Additionally, or alternatively, the environment map computing system 11 may request new or updated data fromthe vehicles 12. For example, the environment map computing system 11 may cause the vehicles 12 to collect new or updated data.

[0113] In some embodiments, a spatial area to be represented by the environment map 48 is segmented into a plurality of zones. A corresponding environment map (e.g. corresponding environment maps 48-1 , 48-2, ... , 48-N, where N is the number of zones) may be generated for each of the zones. By generating an environment map for each zone, computational efficiency may be improved (i.e. the environment map computing system 11 needs to only generate a map for each zone rather than a collective environment map). From the perspective of a single vehicle 12, for example, the environment map computing system 11 need not generate the environment map 48 representing the entire spatial area but may rather generate the environment maps corresponding to the different zones as the vehicle 12 travels through the different zones thereby reducing the computational power that is required of the environment map computing system 11 at any given time. Different zones may be of the same size or of different sizes.

[0114] Each of the zones may overlap. The overlap amount may be the same or different for different overlapping zones. In some embodiments, the zones overlap by a threshold amount. The threshold may be sufficiently large such that a vehicle 12 transitioning from a first zone to a second zone does not cause the vehicle 12 to lose any environment map 48 data (i.e. the vehicle 12 may be continuously provided an environment map despite the transition between zones). In some embodiments, the segmenting of the spatial area into a plurality of zones (or segments) includes, or is, a circle packing method or process.

[0115] Referring to FIG. 7, an example spatial area 70 segmented into a plurality of zones 71 is shown. For example, zone 71-1 may correspond to an environment map 48-1 , zone 71-2 may correspond to an environment map 48-2, zone 71-3 may correspond to an environment map 48- 3 and so on. As a vehicle 12 approaches a zone boundary or a region of overlap, the vehicle 12 may request or obtain from the environment map computing system 11 the corresponding environment map for the next zone. In such a manner, the vehicle 12 may be continuously provided with an environment map.

[0116] Referring to FIG. 8, an example method 80 for generating an environment map 48 by the environment map computing system 11 is shown.

[0117] At block 81 , a spatial area that is to be represented by the environment map 48 is segmented into a plurality of zones (or segments) by the environment map computing system 11. As described elsewhere herein, segmenting the spatial area may include a circle packing method or process.

[0118] At block 82, environment maps for the segmented zones are generated by the environment map computing system 11 . Alternatively, corresponding environment maps may only be generated for the segmented zones which currently (or may imminently) have at least one vehicle 12 within the segmented zones (e.g. the corresponding environment maps are generated “on demand”).

[0119] At block 83, the generated environment maps are provided to (or transmitted to) the vehicles 12 by the environment map computing system 11. As described elsewhere herein, the generated environment maps may be transmitted from the environment map computing system 11 to the vehicles 12 over the communication network 13. In some embodiments, the generated environment maps are provided based on an orientation and speed of a vehicle 12. For example, the environment map corresponding to the zone the vehicle 12 will be visiting next may be provided in advance. As another example, the faster a vehicle 12 is travelling, the larger the provided environment map may be. Conversely, the slower a vehicle 12 is travelling, the smaller the provided environment map may be. By varying a size of the provided environment map, the environment map computing system 11 may increase computational efficiency.

[0120] In some embodiments, updated environment map data is provided periodically. Based on the period, vehicle orientation and vehicle speed, the environment map computing system 11 may determine how big of an environment map 48 to provide to any given vehicle 12. The corresponding environment maps may be provided to different ones of the vehicles 12 at different times (e.g. the environment maps may be provided based at least partially on a current status (e.g. speed, location, orientation, etc.) of the different ones of the vehicles 12).

[0121] In some embodiments, the environment map computing system 11 at least partially controls acquisition of data by the vehicles 12 based on a density of the sensed data 42 the environment map computing system 11 receives as described elsewhere herein. For example, if a high number of vehicles 12 is present in a given area, higher density data may be received by the environment map computing system 11 and the environment map computing system 11 mayreduce the amount of data each vehicle 12 collects. Reducing an amount of data that a vehicle 12 collects may reduce power use by the vehicle 12 and / or improve computational efficiency of the vehicle 12. Additionally, or alternatively, the environment map computing system 11 may cause some of the vehicles 12 to stop collecting any data. If the received sensed data 42 from a subset of vehicles 12 is dense enough that the area is sufficiently and accurately represented by the received sensed data 42, then other vehicles 12 in the area need not provide any additional data. Conversely, if a low number of vehicles is present in a given area, the environment map computing system 11 may increase the amount of data each vehicle 12 collects. Density of the received sensed data 42 may vary, for example, with time (e.g. different times of day may have more or less vehicles 12), with weather (e.g. different weather patterns may results in more or less vehicles 12) or other events.

[0122] In some cases, if the received sensed data 42 is not dense enough at a particular time (e.g. there is only one vehicle 12 in the vicinity), the environment map computing system 11 may be configured to (or may be operable to) update an existing environment map 48 for the area with the new received sensed data 42 (even if the newly received sensed data 42 alone is insufficient to completely represent the area). The environment map computing system 11 may, for example, retrieve an existing environment map 48 from the environment map store 19.

[0123] In some embodiments, the environment map computing system 11 at least partially obtains data from extrinsic sources such as maps, etc. For example, data representing stationary objects which do not change such as roads, buildings, etc. may be obtained from maps. Obtaining data from the extrinsic sources may reduce data collection burdens on each of the vehicles 12 (e.g. the vehicles 12 may not need to collect data representing the stationary buildings if an adequate representation of the stationary buildings may be obtained from at least one extrinsic source).

[0124] In some embodiments, the system 10 includes one or more infrastructure sensors 90 (e.g. as shown in FIG. 1). The one or more infrastructure sensors 90 include sensors which are not part of vehicles 12 and which may be configured to sense one or more characteristics of the spatial area represented by the environment map. For example, the one or more infrastructure sensors 90 may include surveillance cameras installed on buildings within the spatial area, traffic cameras, weather stations, etc. The environment map computing system 11 may receive sensed data from the one or more infrastructure sensors 90 and may at least partially use the receivedsensed data from the one or more infrastructure sensors 90 to generate the environment map 48. In some embodiments, the environment map computing system 11 combines data from the one or more infrastructure sensors with the sensed data 42 received from the vehicles 12.

[0125] As described above, the methods and method steps described herein may be stored in the program-codes store 18 as program codes that, when executed by the CPU 15, cause the environment map computing system 11 to implement the methods (or functions) described herein.

[0126] It will be appreciated by those skilled in the art that changes could be made to the various aspects of the subject application described above without departing from the inventive concept thereof. It is to be understood, therefore, that this subject application is not limited to the particular aspects disclosed, but it is intended to cover modifications as defined by the appended claims. Also, it should be appreciated that not all features are required in all embodiments.

[0127] When introducing elements of the present invention or the embodiments thereof, the articles “a,” “an,” “the,” and “said” are intended to mean that there are one or more of the elements. The terms “comprising,” “including,” and “having” are intended to be inclusive and mean that there may be additional elements other than the listed element.

Claims

CLAIMS1 . A method for generating at least one environment map representing a spatial area, the environment map useable at least partially for vehicle navigation, the method comprising: using a plurality of vehicles, at least partially scanning the spatial area to obtain sensed data representing the spatial area; transmitting the sensed data from the plurality of vehicles to an environment map computing system; receiving, at the environment map computing system, the transmitted sensed data; using the received sensed data, generating, at the environment map computing system, the at least one environment map; and transmitting the generated at least one environment map from the environment map computing system to the plurality of vehicles.

2. The method of claim 1 , further comprising aligning the received sensed data prior to generating the at least one environment map.

3. The method of claim 2, wherein the received sensed data comprises data of a plurality of types.

4. The method of claim 3, further comprising combining a first type of the received sensed data with a second type of the received sensed data.

5. The method of claim 4, wherein the first type of the received sensed data is aligned and the second type of the received sensed data is aligned prior to the first type of the received sensed data being combined with the second type of the received sensed data.

6. The method of claim 4 or claim 5, wherein the first type of the received sensed data corresponds to data obtained from a first sensor type and the second type of the received sensed data corresponds to data obtained from a second sensor type.

7. The method of any one of claims 2 to 6, wherein aligning the received sensed data comprises temporally aligning the received sensed data, spatially aligning the received sensed data or both.

8. The method of any one of claims 1 to 7, wherein generating the at least one environment map comprises segmenting the spatial area and for each segment generating a corresponding environment map.

9. The method of claim 8, further comprising transmitting the corresponding environment maps from the environment map computing system to vehicles of the plurality of vehicles based on locations of the vehicles within the spatial area.

10. The method of claim 8 or claim 9, wherein segmenting the spatial area comprises segmenting the spatial area such that adjacent segments overlap by at least a threshold amount.

11. The method of any one of claims 8 to 10, wherein each segment comprises a circular boundary and the spatial area is segmented according to a circle packing process.

12. The method of any one of claims 1 to 11 , further comprising controlling, by the environment map computing system, the scanning of the spatial area by the plurality of vehicles based on a density of the received sensed data.

13. The method of any one of claims 1 to 12, further comprising updating, by the environment map computing system, the generated at least one environment map in response to receiving, at the environment map computing system, new data from the plurality of vehicles.

14. The method of any one of claims 1 to 13, wherein transmitting the generated at least one environment map to the plurality of vehicles comprises transmitting a first portion of the generated at least one environment map to a first vehicle of the plurality of vehicles and a second portion of the generated at least one environment map to a second vehicle of the plurality of vehicles.

15. The method of claim 14, wherein the first portion and the second portion are different.

16. The method of any one of claims 1 to 15, wherein at least partially scanning the spatial area to obtain sensed data representing the spatial area comprises scanning the spatial area with at least a first vehicle comprising a first set of sensors and a second vehicle comprising a second set of sensors.

17. The method of claim 16, wherein the first set of sensors and the second set of sensors are different.

18. The method of any one of claims 1 to 17, wherein the plurality of vehicles are communicatively coupled to the environment map computing system by a communication network.

19. The method of claim 18, wherein the communication network is a cellular network.

20. The method of claim 19, wherein the cellular network is a 5G cellular network.

21. A method for execution by an environment map computing system, the method comprising causing the environment map computing system to at least: receive, from a plurality of vehicles, sensed data representing a spatial area; and generate, based at least partially on the received sensed data, at least one environment map representing the spatial area, the environment map useable at least partially for vehicle navigation.

22. The method of claim 21, further comprising causing the environment map computing system to, at least, transmit the generated environment map to the plurality of vehicles.

23. The method of claim 21 or claim 22, wherein the plurality of vehicles are communicatively coupled to the environment map computing system by a communication network.

24. The method of claim 23, wherein the communication network is a cellular network.

25. The method of claim 24, wherein the cellular network is a 5G cellular network.

Images