Method and system for generating and using positioning reference data

Abstract

This application relates to methods and systems for generating and using positioning reference data. The present invention discloses methods and systems for improving positioning accuracy relative to a digital map, and the methods and systems are preferably used for high-level and fully autonomous driving applications, and they can use positioning reference data associated with the digital map. The present invention is further extended to methods and systems for generating positioning reference data associated with a digital map.

Description

[0001] Relevant information on divisional applications

[0002] This is a divisional application. The parent case of this divisional application is a patent application for invention with the application date of August 3, 2016, application number 201680044930.4, and invention title "Method and System for Generating and Using Positioning Reference Data". Technical Field

[0003] In some aspects and embodiments, the present invention relates to methods and systems for improving positioning accuracy relative to a digital map, and such methods and systems are required for highly and fully autonomous driving applications. Such methods and systems can use positioning reference data associated with the digital map. In other aspects, the present invention relates to the generation of positioning reference data associated with the digital map, including the format of the reference data and the use of the reference data. For example, embodiments of the present invention relate to using reference data by comparison with data sensed from a vehicle to accurately position the vehicle on the digital map. Other embodiments relate to using the reference data for other purposes, not necessarily in technologies that also use data sensed from the vehicle. For example, additional embodiments relate to using the generated reference data to reconstruct views from a camera associated with the vehicle. Background Art

[0004] In recent years, it has become common for vehicles to be equipped with navigation devices, which can be in the form of a portable navigation device (PND) removably positioned within the vehicle or in the form of a system integrated into the vehicle. These navigation devices include means for determining the current position of the device; typically a global navigation satellite system (GNSS) receiver, such as GPS or GLONASS. However, it should be understood that other means can be used, such as using a mobile telecommunications network, surface beacons, or the like.

[0005] The navigation device may also access a digital map representing a navigable network on which the vehicle is traveling. A digital map (or sometimes called a mathematical map) in its simplest form is actually a database containing data representing nodes (most commonly representing road intersections) and the lines between those nodes representing the roads between those intersections. In a more detailed digital map, the lines can be divided into road segments bounded by a starting node and an ending node. These nodes can be "real" where they represent road intersections where at least 3 lines or road segments intersect, or they can be "artificial" where they are provided as anchor points for road segments not bounded by real nodes at one or both ends to provide, in particular, means for providing shape information for a particular road segment or for identifying positions along the road where certain characteristics of the road (such as speed limits) change. In virtually all modern digital maps, the nodes and road segments are further defined by various attributes, which are also represented by data in the database. For example, each node will typically have geographical coordinates to define its real-world location, e.g., latitude and longitude. Nodes will typically also have live data associated with them, which indicates whether it is possible to move from one road to another at the intersection; and the road segments will also have associated attributes such as the maximum allowed speed, lane size, number of lanes, whether there is a divider in the middle, etc. For the purposes of this application, this form of digital map is referred to as a "standard map".

[0006] The navigation device is arranged to be able to perform a number of tasks using the current position of the device and the standard map, such as guidance regarding a determined route, and providing traffic and travel information relative to the current position or a predicted future position based on the determined route.

[0007] However, it has been recognized that the data contained within the standard map is not sufficient for various next-generation applications, such as highly automated driving where the vehicle is able to automatically control (e.g.) acceleration, braking, and steering without input from the driver, and even fully automated "driverless" vehicles. For such applications, a more precise digital map is required. This more detailed digital map typically includes a three-dimensional vector model where each lane of the road is represented separately along with data on its connectivity to other lanes. For the purposes of this application, this form of digital map will be referred to as a "planning map" or a "high-definition (HD) map".

[0008] Figure 1 A representation of a portion of the planning map is shown, where each line represents the centerline of a lane. Figure 2 Another exemplary portion of the planning map is shown, but this time overlaid on an image of the road network. The data within these maps is typically accurate to within one meter or less, and can be collected using a variety of techniques.

[0009] An exemplary technique for collecting data to build such a planning map is to use a mobile mapping system; an example of which is depicted in Figure 3 . The mobile mapping system 2 includes a survey vehicle 4, a digital camera 40 mounted on top 8 of the vehicle 4, and a laser scanner 6. The survey vehicle 4 further includes a processor 10, a memory 12, and a transceiver 14. Additionally, the survey vehicle 4 includes an absolute positioning device 20 (e.g., a GNSS receiver) and a relative positioning device 22 that includes an inertial measurement unit (IMU) and a distance measurement instrument (DMI). The absolute positioning device 20 provides the geographical coordinates of the vehicle, and the relative positioning device 22 is used to improve the accuracy of the coordinates measured by the absolute positioning device 20 (and to replace the absolute positioning device in those instances where signals from navigation satellites cannot be received). The laser scanner 6, camera 40, memory 12, transceiver 14, absolute positioning device 20, and relative positioning device 22 are all configured to communicate with the processor 10 (as indicated by line 24). The laser scanner 6 is configured to scan the environment in 3D with a laser beam and create a point cloud representing the environment; each point indicates the position of the surface of the object from which the laser beam is reflected. The laser scanner 6 is also configured as a time-of-flight laser rangefinder to measure the distance to each incident position of the laser beam on the object surface.

[0010] In use, as Figure 4 shown, the survey vehicle 4 travels along a road 30 that includes a surface 32 on which road markings 34 are painted. The processor 10 determines the position and orientation of the vehicle 4 at any given moment based on position and orientation data measured using the absolute positioning device 20 and the relative positioning device 22, and stores the data in the memory 12 with an appropriate timestamp. Additionally, the camera 40 repeatedly captures images of the road surface 32 to provide a plurality of road surface images; the processor 10 adds a timestamp to each image and stores the images in the memory 12. The laser scanner 6 also repeatedly scans the surface 32 to provide at least a plurality of measured distance values; the processor adds a timestamp to each distance value and stores it in the memory 12. Figure 5 and 6 show examples of data obtained from the laser scanner 6. Figure 5 shows a 3D view, and Figure 6 shows a side view projection; the color in each picture represents the distance to the road. All of the data obtained from these mobile mapping vehicles can be analyzed and used to create a planning map of a portion of the navigable (or road) network traveled by the vehicle.

[0011] The applicant has recognized that, in order to use such planning maps for highly and fully automated driving applications, it is necessary to know the position of the vehicle relative to the planning map with high precision. Conventional techniques for determining the current position of a device using navigation satellites or ground beacons provide the absolute position of the device with an accuracy of approximately 5 to 10 meters; this absolute position is then matched with the corresponding position on the digital map. While this level of accuracy is sufficient for most conventional applications, it is not accurate enough for next-generation applications, where the position relative to the digital map is required to be at sub-meter accuracy, even when traveling at high speed on a road network. Therefore, improved positioning methods are needed.

[0012] The applicant has also recognized that there is a need for an improved method of generating positioning reference data associated with a digital map, for example, for providing a "planning map" that can be used to determine the position of a vehicle relative to the map and in other contexts. SUMMARY OF THE INVENTION

[0013] According to a first aspect of the present invention, there is provided a method of generating positioning reference data associated with a digital map, the positioning reference data providing a compressed representation of the environment around at least one navigable element of a navigable network represented by the digital map, the method comprising performing the following operations for at least one navigable element represented by the digital map:

[0014] generating positioning reference data comprising at least one depth map indicating the environment around the navigable element projected onto a reference plane, the reference plane being defined by a reference line associated with the navigable element, each pixel in the at least one depth map being associated with a position in the reference plane associated with the navigable element, and the pixel comprising a depth channel representing the distance along a predetermined direction from the associated position of the pixel in the reference plane to the surface of an object in the environment; and

[0015] associating the generated positioning reference data with the digital map data.

[0016] It should be understood that the digital map (in this aspect or embodiment of the present invention and any other aspect or embodiment) includes data representing navigable elements of a navigable network, for example, roads of a road network.

[0017] According to a first aspect of the present invention, positioning reference data is generated associated with one or more navigable elements of a navigable network represented by a digital map. This data can be generated for at least part and preferably all of the navigable elements represented by the map. The generated data provides a compressed representation of the environment around the navigable elements. This is achieved using at least one depth map that indicates the environment around the elements projected onto a reference plane defined by a reference line, which in turn is defined relative to the navigable element. Each pixel of the depth map is associated with a position in the reference plane and includes a depth channel representing the distance along a predetermined direction from the position of the pixel in the reference plane to the surface of an object in the environment.

[0018] Various features of the at least one depth map of the positioning reference data will now be described. It should be understood that such features can alternatively or additionally be applied to the at least one depth map of real-time scan data used in certain other aspects or embodiments of the present invention, provided they are not mutually exclusive.

[0019] The reference line associated with the navigable element and used to define the reference plane can be set in any way with respect to the navigable element. The reference line is defined by one or more points associated with the navigable element. The reference line can have a predetermined orientation relative to the navigable element. In a preferred embodiment, the reference line is parallel to the navigable element. This may be suitable for providing positioning reference data (and / or real-time scan data) related to the lateral environment on one or more sides of the navigable element. The reference line can be linear or non-linear, i.e., depending on whether the navigable element is straight. The reference line can include straight and non-linear, e.g., curved portions, such as remaining parallel to the navigable element. It should be understood that in some other embodiments, the reference line may not be parallel to the navigable element. For example, as described below, the reference line can be defined by a radius centered on a point associated with the navigable element (e.g., a point on the navigable element). The reference line can be circular. This can then provide a 360-degree representation of the environment around the junction point.

[0020] The reference line is preferably a longitudinal reference line and can be, for example, the edge or boundary of the navigable element or its lane, or the centerline of the navigable element. Then, the positioning reference data (and / or real-time scan data) will provide a representation of the environment on one or more sides of the element. The reference line can be located on the element.

[0021] In an embodiment, since the reference line of a navigable element (e.g., the edge or center line of the navigable element) and the associated depth information may undergo a mapping to a linear reference line, the reference line can be linear even when the navigable element is curved. This mapping or transformation is described in more detail in WO 2009 / 045096 A1; WO2009 / 045096A1 is incorporated herein by reference in its entirety.

[0022] The reference plane defined by the reference line is preferably oriented perpendicular to the surface of the navigable element. The reference plane used herein refers to a two-dimensional surface, which can be curved or non-curved.

[0023] In the case where the reference line is parallel to the longitudinal reference line of the navigable element, the depth channel of each pixel preferably represents the lateral distance to the surface of an object in the environment.

[0024] Each depth map can be in the form of a raster image. It should be understood that each depth map represents the distance from the surface of an object in the environment to the reference plane at a plurality of longitudinal positions and elevations (i.e., corresponding to the position of each pixel associated with the reference plane) along a predetermined direction. The depth map includes a plurality of pixels. Each pixel of the depth map is associated with a specific longitudinal position and elevation in the depth map (e.g., raster image).

[0025] In some preferred embodiments, the reference plane is defined by a line parallel to the longitudinal reference line of the navigable element and is oriented perpendicular to the surface of the navigable element. Then, each pixel includes a depth channel representing the lateral distance to the surface of an object in the environment.

[0026] In a preferred embodiment, at least one depth map can have a fixed longitudinal resolution and a variable vertical and / or depth resolution.

[0027] According to a second aspect of the present invention, there is provided a method of generating positioning reference data associated with a digital map, the positioning reference data providing a compressed representation of the environment around at least one navigable element of a navigable network represented by the digital map, the method comprising performing the following operations for at least one navigable element represented by the digital map:

[0028] Generate positioning reference data, the positioning reference data including at least one depth map indicative of the environment around the navigable element projected onto a reference plane, the reference plane being bounded by a longitudinal reference line parallel to the navigable element and oriented perpendicular to the surface of the navigable element, each pixel in the at least one depth map being associated with a position in the reference plane associated with the navigable element, and the pixel including a depth channel representing a lateral distance along a predetermined direction from the associated position of the pixel in the reference plane to the surface of an object in the environment, preferably wherein the at least one depth map has a fixed longitudinal resolution and a variable vertical and / or depth resolution; and

[0029] Associate the generated positioning reference data with the digital map data.

[0030] The invention according to this further aspect may include any or all of the features described in other aspects of the invention, provided they are not mutually contradictory.

[0031] Regardless of the orientation of the reference line, reference plane, and the line along which the environment is projected onto the reference plane, in its various aspects and embodiments, according to the invention, it is advantageous for at least one depth map to have a fixed longitudinal resolution and a variable vertical and / or depth resolution. At least one depth map of the positioning reference data (and / or real-time scan data) preferably has a fixed longitudinal resolution and a variable vertical and / or depth resolution. The variable vertical and / or depth resolution is preferably non-linear. Parts of the depth map (e.g., raster image) closer to the ground and closer to the navigable element (and thus, closer to the vehicle) can be shown at a higher resolution than parts of the depth map (e.g., raster image) higher above the ground and farther from the navigable element (and thus, farther from the vehicle). This maximizes the information density at heights and depths that are more important for detection by vehicle sensors.

[0032] Regardless of the orientation of the reference line and plane and the resolution of the depth map in various directions, the projection of the environment onto the reference plane is along a predetermined direction, which can be selected as needed. In some embodiments, the projection is an orthogonal projection. In these embodiments, the depth channel of each pixel represents the distance from the associated position of the pixel in the reference plane along a direction perpendicular to the reference plane to the surface of an object in the environment. Thus, in some embodiments where the distance represented by the depth channel is a lateral distance, the lateral distance is along a direction perpendicular to the reference plane (although non-orthogonal projections are not limited to cases where the depth channel is related to a lateral distance). The use of orthogonal projection may be advantageous in some contexts because this will result in any height information being independent of the distance from the reference line (and thus independent of the distance from the reference plane).

[0033] In other embodiments, it has been found that using non-orthogonal projections can be advantageous. Thus, in some embodiments of the invention in any of its aspects, unless mutually exclusive, the depth channel of each pixel represents the distance from the associated position of the pixel in the reference plane to the surface of an object in the environment along a direction that is not perpendicular to the reference plane (whether the pre-determined distance is a lateral distance or not). The use of non-orthogonal projections has the advantage that information about surfaces oriented perpendicular to the navigable element (i.e., where the reference line is parallel to the element) can be preserved. This can be achieved without providing an additional data channel associated with the pixel. Thus, information about objects near the navigable element can be captured more efficiently and in more detail without increasing the storage capacity. The pre-determined direction can be along any desired direction relative to the reference plane, for example at 45 degrees.

[0034] It has also been found that using non-orthogonal projections is useful in preserving a greater amount of information about the surfaces of objects that can be detected by one or more cameras of a vehicle in dark conditions, and is thus particularly useful in combination with some aspects and embodiments of the invention in which a reference image or point cloud is compared with an image or point cloud obtained from real-time data sensed by the cameras of the vehicle.

[0035] According to another aspect of the invention, there is provided a method of generating positioning reference data associated with a digital map, the positioning reference data providing a compressed representation of the environment around at least one navigable element of a navigable network represented by the digital map, the method comprising, for at least one navigable element represented by the digital map:

[0036] generating positioning reference data comprising at least one depth map indicative of the environment around the navigable element projected onto a reference plane bounded by a reference line parallel to the navigable element, each pixel in the at least one depth map being associated with a position in the reference plane associated with the navigable element, and the pixel comprising a depth channel representing the distance along a pre-determined direction from the associated position of the pixel in the reference plane to the surface of an object in the environment, wherein the pre-determined direction is not perpendicular to the reference plane; and

[0037] associating the generated positioning reference data with digital map data indicative of the navigable element.

[0038] The invention according to this other aspect can include any or all of the features described in other aspects of the invention, provided they are not mutually contradictory.

[0039] According to any aspect or embodiment of the present invention, the localization reference data (and / or real-time scan data) is based on scan data obtained by scanning the environment around the navigable element using one or more sensors. The one or more scanners may include one or more of the following: a lidar scanner, a radar scanner, and a camera, for example, a single camera or a pair of stereo cameras.

[0040] Preferably, the distance to the surface of the object represented by the depth channel of each pixel of the localization reference data (and / or real-time scan data) is determined based on a set of multiple sensing data points, each indicating the distance from the position of the pixel to the surface of the object along a predefined direction. The data points can be obtained when scanning the environment around the navigable element. The set of sensing data points can be obtained from one or more types of sensors. However, in some preferred embodiments, the sensing data points include or comprise a set of data points sensed by a lidar scanner. In other words, the sensing data points include or comprise lidar measurements.

[0041] It has been found that using the average of multiple sensing data points when determining the distance value of the depth channel of a given pixel can lead to incorrect results. This is because there is a possibility that at least some of the sensing data points indicating the position of the surface of the object along the applicable predefined direction from the reference plane and considered to be mapped to a particular pixel may be associated with the surface of a different object. It should be understood that due to the compressed data format, an extended area of the environment can be mapped to the area of pixels in the reference plane. A relatively large amount of sensing data, i.e., several sensing data points, can thus be applicable to that pixel. Within that area, there may be objects located at different depths relative to the reference plane, including objects that may overlap with another object only a short distance in any dimension, such as trees, lamp posts, walls, and moving objects. The depth value to the surface of the object represented by the sensor data points applicable to a particular pixel can thus exhibit a significant variation.

[0042] According to any aspect or embodiment of the present invention, the distance to the surface of an object represented by the depth channel of each pixel of the localization reference data (and / or real-time scan data) is determined based on a set of multiple sensed data points, each sensed data point indicating the sensed distance along a predetermined direction from the position of the pixel to the surface of the object. Preferably, the distance represented by the depth channel of the pixel is not based on the average of the set of multiple sensed data points. In a preferred embodiment, the distance represented by the depth channel of the pixel is the sensed distance closest to the surface of the object from among the set of sensed data points, or the closest mode value obtained using the distribution of the sensed depth values. It should be understood that one or more closest values detected are likely to most accurately reflect the depth from the surface of the object to the pixel. For example, consider a situation where a tree is located between a building and a road. The different sensed depth values applicable to a particular pixel may be based on the detection of the building or the tree. If all of these sensed values are taken into account to provide an average depth value, the average value will indicate that the depth from the pixel to the surface of the object is somewhere between the depth to the tree and the depth to the building. This will result in a misleading depth value for the pixel, which can cause problems when correlating real-time vehicle sensing data with reference data and may potentially be dangerous because it is very important to know with certainty how close an object is to the road. In contrast, the closest depth value or closest mode value is likely to be related to the tree rather than the building, thus reflecting the true position of the closest object.

[0043] According to another aspect of the present invention, there is provided a method of generating localization reference data associated with a digital map, the localization reference data providing a compressed representation of the environment around at least one navigable element of a navigable network represented by the digital map. The method includes performing the following operations for at least one navigable element represented by the digital map:

[0044] Generating localization reference data including at least one depth map indicating the environment around the navigable element projected onto a reference plane, the reference plane being defined by a reference line associated with the navigable element. Each pixel in the at least one depth map is associated with a position in the reference plane associated with the navigable element, and the pixel includes a depth channel representing the distance along a predetermined direction from the associated position of the pixel in the reference plane to the surface of an object in the environment, wherein the distance to the surface of the object represented by the depth channel of each pixel is determined based on a set of multiple sensed data points, each sensed data point indicating the sensed distance along the predetermined direction from the position of the pixel to the surface of the object, and wherein the distance to the surface of the object represented by the depth channel of the pixel is based on the closest distance, or closest mode distance, of the set of sensed data points; and

[0045] Associate the generated positioning reference data with digital map data.

[0046] According to this aspect of the invention, any or all of the features described in other aspects of the invention may be included, provided they are not mutually contradictory.

[0047] According to any aspect or embodiment of the invention, each pixel (in the positioning reference data and / or the real-time scan data) includes a depth channel representing the distance to the surface of an object in the environment. In a preferred embodiment, each pixel includes one or more additional channels. This can provide a depth map with one or more additional information layers. Each channel preferably indicates a value of a property obtained based on one or more sensed data points and preferably based on a set of multiple sensed data points. The sensed data can be obtained from one or more of the sensors described earlier. In a preferred embodiment, the or each pixel includes at least one channel indicating a value of sensed reflectivity of a given type. Each pixel can include one or more of the following: a channel indicating a value of sensed laser reflectivity; and a channel indicating a value of sensed radar reflectivity. The sensed reflectivity value of a pixel indicated by a channel is related to the sensed reflectivity in the applicable portion of the environment represented by the pixel. The sensed reflectivity value of a pixel preferably indicates the sensed reflectivity around a distance from a reference plane corresponding to the depth of the pixel from the reference plane indicated by the depth channel of the pixel, i.e., the sensed reflectivity around the depth value of the pixel. This can then be considered to indicate the relevant reflectivity properties of the object present at that depth. Preferably, the sensed reflectivity is an average reflectivity. The sensed reflectivity data can be based on reflectivities associated with the same data points used to determine the depth values of a larger set of data points. For example, reflectivities associated with the sensed depth values applicable to a pixel (and in addition to those nearest values preferably used to determine the depth values of the depth channel) can be considered.

[0048] In this way, a multi-channel depth map, such as a raster image, is provided. This format can enable more efficient compression of a larger amount of data related to the environment around navigable elements, facilitating storage and processing, and providing the ability to implement improved correlation with real-time data sensed by a vehicle under different conditions, and the vehicle does not necessarily need to have the same type of sensors as those used to generate the reference positioning data. As will be described in more detail below, this data can also help reconstruct the data sensed by the vehicle, or images of the environment around navigable elements obtained using the vehicle's camera under specific conditions (such as at night). For example, radar or laser reflectivity can enable the identification of those objects that will be visible under specific conditions (such as at night).

[0049] According to another aspect of the present invention, there is provided a method of generating location reference data associated with a digital map, the location reference data providing a compressed representation of the environment around at least one navigable element of a navigable network represented by the digital map, the method comprising performing the following operations for at least one navigable element represented by the digital map:

[0050] Generating location reference data including at least one depth map indicating the environment around the navigable element projected onto a reference plane, the reference plane being defined by a reference line associated with the navigable element, each pixel in the at least one depth map being associated with a position in the reference plane associated with the navigable element, and the pixel including a depth channel representing the distance along a predetermined direction from the associated position of the pixel in the reference plane to the surface of an object in the environment, wherein each pixel further includes one or more of the following: a channel indicating a value of sensed laser reflectivity; and a channel indicating a value of sensed radar reflectivity; and

[0051] Associating the generated location reference data with digital map data.

[0052] The present invention according to this other aspect may include any or all of the features described in other aspects of the present invention, provided that they are not mutually contradictory.

[0053] According to any aspect or embodiment of the present invention, other channels associated with the pixels may alternatively or additionally be used. For example, additional channels may be channels indicating one or more of the following: the thickness of an object near the distance indicated by the depth channel of the pixel from the position of the pixel in the reference plane along a predetermined direction; the density of reflection data points near the distance indicated by the depth channel of the pixel from the position of the pixel in the reference plane along a predetermined direction; the color near the distance indicated by the depth channel of the pixel from the position of the pixel in the reference plane along a predetermined direction; and the texture near the distance indicated by the depth channel of the pixel from the position of the pixel in the reference plane along a predetermined direction. Each channel may include a value indicating the relevant property. The value is based on the applicable sensor data obtained, which may optionally be obtained from one or more different types of sensors, for example, a camera for color or texture data. Each value may be based on a plurality of sensed data points and may be an average of the plurality of sensed data points.

[0054] It should be understood that while the depth channel indicates the distance of an object from a reference plane at a position along a pre-determined direction from a pixel, other channels may indicate other properties of the object, such as the reflectivity of the object, or its color, texture, etc. This can be useful in reconstructing scan data that may be expected to have been sensed by a vehicle and / or camera images taken by a vehicle. Data indicating the thickness of an object can be used to recover information related to the surface of the object perpendicular to a navigable element, where an orthogonal projection of the environment onto the reference plane is used. This can provide an alternative to the embodiments described above for determining information related to such surfaces of an object that use a non-orthogonal projection.

[0055] In many embodiments, localization reference data is used to provide a compressed representation of the environment on one or more sides of a navigable element, i.e., to provide a side depth map. Then, reference lines can be parallel to the navigable element, where the depth channel of a pixel indicates the lateral distance of an object surface from the reference plane. However, using depth maps can also be helpful in other contexts. The applicant has recognized that it will be useful to provide a circular depth map in the area of a junction (e.g., an intersection). This can provide an improved ability to locate a vehicle relative to a junction (e.g., an intersection), or, if desired, to reconstruct data indicating the environment around the junction (e.g., an intersection). A 360-degree representation of the environment around the junction is preferably provided, although it should be understood that the depth map need not extend around a complete circle and can thus extend around less than 360 degrees. In some embodiments, the reference plane is defined by reference lines bounded by a radius centered on a reference point associated with the navigable element. In these embodiments, the reference lines are curved and preferably circular. The reference point is preferably located on a navigable section at the junction. For example, the reference point can be located at the center of the junction (e.g., an intersection). The radius defining the reference line can be selected as needed, e.g., depending on the size of the junction.

[0056] According to another aspect of the present invention, there is provided a method of generating localization reference data associated with a digital map representing elements of a navigable network, the localization reference data providing a compressed representation of the environment around at least one junction of the navigable network represented by the digital map, the method comprising performing the following operations for at least one junction represented by the digital map:

[0057] Generate positioning reference data, the positioning reference data including at least one depth map indicative of the environment around the junction point projected onto a reference plane, the reference plane being defined by a reference line bounded by a radius centered on a reference point associated with the junction point, each pixel in the at least one depth map being associated with a position in the reference plane associated with the junction point, and the pixel including a depth channel representing the distance along a predetermined direction from the associated position of the pixel in the reference plane to the surface of an object in the environment; and

[0058] Associate the generated positioning reference data with digital map data indicative of the junction point.

[0059] As described with respect to earlier embodiments, the junction point may be an intersection. The reference point may be located at the center of the junction point. The reference point may be associated with a node of a digital map representing the junction point or a navigable element at the junction point. These additional aspects or embodiments of the present invention may be used in conjunction with side depth maps representing the environment on the side of a navigable element away from the junction point.

[0060] The present invention according to this other aspect may include any or all of the features described with respect to other aspects of the present invention, provided they are not mutually contradictory.

[0061] According to any aspect or embodiment of the present invention related to the generation of positioning reference data, the method may include associating the generated positioning reference data regarding a navigable element or a junction point with digital map data indicative of the element or the junction point. The method may include storing the generated positioning data associated with the digital map data, for example, associated with the navigable element or junction point to which it relates.

[0062] In some embodiments, the positioning reference data may include reference scans representing, for example, the lateral environment on the left side and the right side of a navigable element. The positioning reference data for each side of the navigable element may be stored in a combined data set. Thus, data from multiple parts of a navigable network may be stored in an efficient data format. The data stored in the combined data set may be compressed, thereby allowing more data from the navigable network to be stored within the same storage capacity. If the reference scan data is transmitted to a vehicle via a wireless network connection, then data compression will also allow the use of reduced network bandwidth. However, it should be understood that the positioning reference data does not necessarily need to relate to the lateral environment on either side of a navigable element. For example, as discussed in some of the above embodiments, the reference data may relate to the environment around a junction point.

[0063] The present invention also extends to a data product storing positioning reference data generated according to any aspect or embodiment of the present invention.

[0064] The data product in any of these further aspects or embodiments of the invention may have any suitable form. In some embodiments, the data product may be stored on a computer-readable medium. The computer-readable medium may be, for example, a floppy disk, a CD ROM, a ROM, a RAM, a flash memory, or a hard disk. The invention extends to computer-readable media comprising a data product according to any aspect or embodiment of the invention.

[0065] The positioning reference data generated according to any aspect or embodiment of the invention related to the generation of this data can be used in various ways. In a further aspect related to the use of data, the step of obtaining reference data may extend to generating data, or generally include retrieving data. The reference data is preferably generated by a server. The step of using the data is preferably performed by a device (such as a navigation device or similar device) that may be associated with a vehicle.

[0066] In some preferred embodiments, the data is used to determine the position of the vehicle relative to a digital map. The digital map thus includes data representing navigable elements along which the vehicle travels. The method may include: obtaining positioning reference data associated with the digital map for a supposed current position of the vehicle along a navigable element of a navigable network; determining real-time scan data by scanning the environment around the vehicle using at least one sensor, wherein the real-time scan data includes at least one depth map indicating the environment around the vehicle, each pixel in the at least one depth map being associated with a position in a reference plane associated with the navigable element, and the pixel containing a depth channel representing the distance determined using the at least one sensor from the associated position of the pixel in the reference plane to the surface of an object in the environment along a predetermined direction; calculating a correlation between the positioning reference data and the real-time scan data to determine an alignment offset between the depth maps; and using the determined alignment offset to adjust the supposed current position to determine the position of the vehicle relative to the digital map. It should be understood that the obtained positioning reference data is related to the navigable elements along which the vehicle travels. The depth map of the positioning reference data indicating the environment around the navigable element thus indicates the environment around the vehicle.

[0067] According to another aspect of the invention, there is provided a method of determining the position of a vehicle relative to a digital map, the digital map including data representing navigable elements of a navigable network along which the vehicle travels, the method comprising:

[0068] Obtaining positioning reference data associated with the digital map for a perceived current position of the vehicle along a navigable element of the navigable network, wherein the positioning reference data includes at least one depth map indicative of the environment around the vehicle projected onto a reference plane bounded by a reference line associated with the navigable element, each pixel in the at least one depth map being associated with a position in the reference plane of the navigable element along which the vehicle travels, and the pixel including a depth channel representing the distance from the associated position of the pixel in the reference plane to the surface of an object in the environment along a pre-determined direction;

[0069] Determining real-time scan data by scanning the environment around the vehicle using at least one sensor, wherein the real-time scan data includes at least one depth map indicative of the environment around the vehicle, each pixel in the at least one depth map being associated with a position in the reference plane of the navigable element along which the vehicle travels, and the pixel including a depth channel representing the distance from the associated position of the pixel in the reference plane to the surface of an object in the environment along the pre-determined direction determined using the at least one sensor;

[0070] Calculating a correlation between the positioning reference data and the real-time scan data to determine an alignment offset between the depth maps; and

[0071] Adjusting the perceived current position using the determined alignment offset to determine the position of the vehicle relative to the digital map.

[0072] According to this another aspect of the invention, it may include any or all of the features described in other aspects of the invention, provided that they are not mutually contradictory.

[0073] In further aspects and embodiments of the invention related to using positioning reference data and real-time scan data in determining the position of a vehicle, the current position of the vehicle may be a longitudinal position. The real-time scan data may be related to the lateral environment around the vehicle. Then, the depth maps of the positioning reference data and / or the real-time sensor data will be bounded by a reference line parallel to the navigable element and include depth channels representing the lateral distance to the surface of an object in the environment. Then, the determined offset may be a longitudinal offset.

[0074] According to another aspect of the invention, there is provided a method of determining the position of a vehicle relative to a digital map, the digital map including data representing junctions through which the vehicle travels, the method comprising:

[0075] Obtain positioning reference data associated with the digital map for a perceived current position of the vehicle in the navigable network, wherein the positioning reference data includes at least one depth map indicative of the environment around the vehicle projected onto a reference plane, the reference plane being defined by a reference line bounded by a radius centered at a reference point associated with the junction point, each pixel in the at least one depth map being associated with a position in the reference plane associated with the junction point through which the vehicle travels, and the pixel including a depth channel representing the distance along a predetermined direction from the associated position of the pixel in the reference plane to the surface of an object in the environment;

[0076] Determine real-time scan data by scanning the environment around the vehicle using at least one sensor, wherein the real-time scan data includes at least one depth map indicative of the environment around the vehicle, each pixel in the at least one depth map being associated with a position in the reference plane associated with the junction point, and the pixel including a depth channel representing the distance determined using the at least one sensor along the predetermined direction from the associated position of the pixel in the reference plane to the surface of an object in the environment;

[0077] Calculate a correlation between the positioning reference data and the real-time scan data to determine an alignment offset between the depth maps; and

[0078] Adjust the perceived current position using the determined alignment offset to determine the position of the vehicle relative to the digital map.

[0079] The invention according to this other aspect may include any or all of the features described in other aspects of the invention, provided they are not mutually contradictory.

[0080] According to another aspect of the invention, there is provided a method of determining the position of a vehicle relative to a digital map, the digital map including data representing navigable elements of a navigable network along which the vehicle travels, the method comprising:

[0081] Obtain positioning reference data associated with the digital map for a supposed current position of the vehicle along a navigable element of the navigable network, wherein the positioning reference data includes at least one depth map indicative of the environment around the vehicle, each pixel in the at least one depth map being associated with a position in a reference plane associated with the navigable element, the reference plane being defined by a longitudinal reference line parallel to the navigable element and oriented perpendicular to the surface of the navigable element, and each pixel including a depth channel representing a lateral distance to the surface of an object in the environment, optionally, wherein the at least one depth map has a fixed longitudinal resolution and a variable vertical and / or depth resolution;

[0082] Determine real-time scan data by scanning the environment around the vehicle using at least one sensor;

[0083] Determine real-time scan data using the sensor data, wherein the real-time scan data includes at least one depth map indicative of the environment around the vehicle, each pixel in the at least one depth map being associated with a position in a reference plane associated with the navigable element, and each pixel including a depth channel representing a lateral distance to the surface of an object in the environment determined according to the sensor data, optionally, wherein the at least one depth map has a fixed longitudinal resolution and a variable vertical and / or depth resolution;

[0084] Calculate a correlation between the positioning reference data and the real-time scan data to determine an alignment offset between the depth maps; and

[0085] Adjust the supposed current position using the determined alignment offset to determine the position of the vehicle relative to the digital map.

[0086] The invention according to this further aspect may include any or all of the features described in other aspects of the invention, provided they are not mutually contradictory.

[0087] In a further aspect of the invention related to the use of positioning reference data, data may be generated according to any earlier aspect of the invention. The real-time scan data used in determining the position of a vehicle or otherwise should have a form corresponding to the positioning reference data. Thus, the determined depth map will include pixels having positions in a reference plane defined relative to a reference line associated with a navigable element in the same manner as the positioning reference data, such that the real-time scan data and the positioning reference data are related to each other. The depth channel data of the depth map may be determined in a manner corresponding to that of the reference data, for example without using the average of the sensed data, and may thus include the closest distance from a plurality of sensed data points to the surface. The real-time scan data may contain any additional channels. In the case where the depth map of the positioning reference data has a fixed longitudinal resolution and variable vertical and / or depth resolution, the depth map of the real-time scan data may also have this resolution.

[0088] Thus, according to these aspects or embodiments of the invention, there is provided a method for continuously determining the position of a vehicle relative to a digital map; the digital map includes data representing navigable elements (e.g., roads) of a navigable network (e.g., a road network) along which the vehicle travels. The method includes: receiving real-time scan data obtained by scanning the environment around the vehicle; retrieving positioning reference data associated with the digital map for a supposed current position of the vehicle relative to the digital map (e.g., where the positioning reference data includes a reference scan of the environment around the supposed current position), optionally, where the reference scan has been obtained from at least one device that has previously traveled along a route throughout the digital map; comparing the real-time scan data with the positioning reference data to determine an offset between the real-time scan data and the positioning reference data; and adjusting the supposed current position based on the offset. The position of the vehicle relative to the digital map can thus always be known with high precision. Examples in the prior art have attempted to determine the position of a vehicle by comparing the data collected with known reference data of predetermined landmarks along a route. However, the landmarks may be sparsely distributed on many routes, resulting in significant estimation errors in the vehicle position when the vehicle travels between the landmarks. This is a problem in cases such as highly automated driving systems, where such errors can lead to catastrophic consequences, such as a vehicle collision accident causing serious injury or loss of life. The present invention solves this problem in at least some aspects by having reference scan data throughout the digital map and by scanning the environment around the vehicle in real time. In this way, the present invention may allow the comparison of the real-time scan data with the reference data, such that the position of the vehicle relative to the digital map is always known with high precision.

[0089] According to another aspect of the present invention, there is provided a method for determining a longitudinal position of a vehicle relative to a digital map, the digital map including data representing navigable elements of a navigable network along which the vehicle travels, the method comprising:

[0090] Obtaining positioning reference data associated with the digital map for a supposed current position of the vehicle along the navigable elements of the navigable network, wherein the positioning reference data includes outlines of objects in the environment around the vehicle projected onto a reference plane, the reference plane being defined by a longitudinal reference line parallel to the navigable element and perpendicular to the surface of the navigable element;

[0091] Obtaining sensor data by scanning the environment around the vehicle using at least one sensor;

[0092] Determining real-time scan data using the sensor data, wherein the real-time scan data includes outlines of objects in the environment around the vehicle projected onto the reference plane determined from the sensor data;

[0093] Calculating a correlation between the positioning reference data and the real-time scan data to determine a longitudinal alignment offset; and

[0094] Adjusting the supposed current position using the determined alignment offset to determine the longitudinal position of the vehicle relative to the digital map.

[0095] The invention according to this other aspect may incorporate any or all of the features described in other aspects of the invention, provided they are not mutually contradictory.

[0096] The localization reference data can be stored in association with a digital map, for example, in association with relevant navigable elements, such that the outlines of objects in the environment around the vehicle projected onto a reference plane have been determined. However, in other embodiments, the localization reference data can be stored in a different format, and the stored data is processed to determine the outlines. For example, in an embodiment, as in the aspects described earlier in the present invention, the localization reference data includes one or more depth maps, such as raster images, each depth map representing the lateral distance to surfaces in the environment at multiple longitudinal positions and elevations. The depth map can be according to any earlier aspect and embodiment. In other words, the localization reference data includes at least one depth map, such as a raster image, which indicates the environment around the vehicle, wherein each pixel of the at least one depth map is associated with a position in the reference plane, and each pixel includes a channel representing the lateral distance (e.g., perpendicular to the reference plane) to the surface of an object in the environment. In such embodiments, the relevant depth map, such as a raster image, is processed using an edge detection algorithm to generate the outlines of objects in the environment. The edge detection algorithm can include the Canny operator, the Prewitt operator, and similar operators. However, in a preferred embodiment, the edge detection is performed using the Sobel operator. The edge detection operator can be applied to both the height (or elevation) and longitudinal domains, or to only one of the domains. For example, in a preferred embodiment, the edge detection operator is applied only to the longitudinal domain.

[0097] Similarly, the outlines of objects in the environment around the vehicle projected onto a reference plane can be directly determined based on sensor data obtained by at least one sensor. Alternatively, in other embodiments, the sensor data can be used to determine one or more depth maps, such as raster images, each depth map representing the lateral distance to surfaces in the environment at multiple longitudinal positions and elevations. In other words, the real-time scan data includes at least one depth map, such as a raster image, which indicates the environment around the vehicle, wherein each pixel of the at least one depth map is associated with a position in the reference plane, and each pixel includes a channel representing the lateral distance (e.g., perpendicular to the reference plane) to the surface of an object in the environment determined using at least one sensor. Then, the relevant depth map, such as a raster image, can be processed using an edge detection algorithm, preferably using the same edge detection algorithm applied to the localization reference data, to determine the outlines of the real-time scan data. The edge detection operator can be applied to both the height (or elevation) and longitudinal domains, or to only one of the domains. For example, in a preferred embodiment, the edge detection operator is applied only to the longitudinal domain.

[0098] In an embodiment, a blurring operator is applied to the contour of at least one of the localization reference data and the real-time scan data before correlating the two sets of data. The blurring operator can be applied to both the height (or altitude) and longitudinal domains, or to only one of the domains. For example, in a preferred embodiment, the blurring operator is applied only to the height domain. When obtaining the real-time scan data and / or the localization reference data, the blurring operator can take into account any tilt of the vehicle, such that for example the contour is slightly shifted up or down in the height domain.

[0099] According to any aspect or embodiment of the present invention, the supposed current (e.g.) longitudinal position of the vehicle can be obtained at least initially from an absolute positioning system, such as a satellite navigation device (e.g., GPS, GLONASS), the European Galileo positioning system, the COMPASS positioning system, or the IRNSS (Indian Regional Navigation Satellite System). However, it should be understood that other position determination means can be used, such as using mobile telecommunications, surface beacons, or the like.

[0100] The digital map can include a three-dimensional vector model representing navigable elements (e.g., roads of a road network) of a navigable network, where each lane of a navigable element (e.g., a road) is represented separately. Thus, the lateral position of the vehicle on the road can be known by determining the lane in which the vehicle is traveling, e.g., by image processing of a camera mounted on the vehicle. In such embodiments, the longitudinal reference line can be, for example, the edge or boundary of a lane of a navigable element or the centerline of a lane of a navigable element.

[0101] Real-time scan data can be obtained on the left side of the vehicle and on the right side of the vehicle. This helps to reduce the influence of transient features on the position estimate. Such transient features can be, for example, parked vehicles, overtaking vehicles, or vehicles traveling in the opposite direction on the same route. Thus, the real-time scan data can record the features present on both sides of the vehicle. In some embodiments, the real-time scan data can be obtained from the left side of the vehicle or from the right side of the vehicle.

[0102] In embodiments where the localization reference data and the real-time scan data are each regarding the left side and the right side of the vehicle, the comparison of the real-time scan data from the left side of the vehicle with the localization reference data from the left side of the navigable element and the comparison of the real-time scan data from the right side of the vehicle with the localization reference data from the right side of the navigable element can be a single comparison. Thus, when the scan data includes data from the left side of the navigable element and data from the right side of the navigable element, the scan data can be compared as a single data set, thereby significantly reducing the processing requirements compared to a situation where the comparison for the left side of the navigable element and the comparison for the right side of the navigable element are performed separately.

[0103] Whether it relates to the left or right side of the vehicle, comparing the real-time scan data with the positioning reference data may include calculating the cross-correlation between the real-time scan data and the positioning reference data, preferably the normalized cross-correlation. The method may include determining the position at which the data sets are most aligned. Preferably, the alignment offset between the determined depth maps is at least a longitudinal alignment offset, and the position at which the data sets are most aligned is a longitudinal position. The step of determining the longitudinal position at which the data sets are most aligned may include longitudinally shifting a depth map (e.g., a raster image provided by the depth map based on the real-time scan data) relative to a depth map (e.g., a raster image provided by the depth map based on the positioning reference data) until the depth maps are aligned. This may be performed in the image domain.

[0104] The determined longitudinal alignment offset is used to adjust the perceived current position to adjust the longitudinal position of the vehicle relative to the digital map.

[0105] As an alternative or preferably in addition to determining the longitudinal alignment offset between the depth maps, it is desirable to determine the lateral alignment offset between the depth maps. Then, the determined lateral alignment offset can be used to adjust the perceived current lateral position of the vehicle and thus determine the position of the vehicle relative to the digital map. Preferably, the longitudinal alignment offset is determined, which can be implemented in any of the ways described above, and additionally the lateral alignment offset is determined. Then, the determined lateral and longitudinal alignment deviations are used together to adjust both the longitudinal and lateral positions of the vehicle relative to the digital map.

[0106] The method may include determining the longitudinal alignment offset between the depth maps, e.g., by calculating the correlation between the positioning reference data and the real-time scan data, and may further include: determining the lateral offset between the depth maps; and using the determined lateral and longitudinal alignment offsets to adjust the perceived current position to determine the position of the vehicle relative to the digital map.

[0107] The longitudinal alignment offset is preferably determined before the lateral alignment offset. According to certain embodiments described below, the lateral alignment offset may be determined based on first determining the longitudinal offset between the depth maps and longitudinally aligning the depth maps relative to each other based on the offset.

[0108] The lateral offset is preferably determined based on the most common lateral offset between corresponding pixels of the depth maps, i.e., the modal lateral offset.

[0109] According to another aspect of the present invention, there is provided a method of determining the position of a vehicle relative to a digital map, the digital map including data representing navigable elements of a navigable network along which the vehicle travels, the method comprising:

[0110] Obtain positioning reference data associated with the digital map for a supposed current position of the vehicle along a navigable element of the navigable network, wherein the positioning reference data includes at least one depth map indicating the environment around the vehicle projected onto a reference plane, the reference plane being defined by a reference line associated with the navigable element, each pixel of the at least one depth map being associated with a position in the reference plane of the navigable element along which the vehicle travels, and the pixel including a depth channel representing the distance along a predetermined direction from the associated position of the pixel in the reference plane to the surface of an object in the environment;

[0111] Determine real-time scan data by scanning the environment around the vehicle using at least one sensor, wherein the real-time scan data includes at least one depth map indicating the environment around the vehicle, each pixel of the at least one depth map being associated with a position in the reference plane of the navigable element, and the pixel including a depth channel representing the distance along a predetermined direction from the associated position of the pixel in the reference plane to the surface of an object in the environment determined using the at least one sensor;

[0112] Determine a longitudinal alignment offset between the depth maps of the positioning reference data and the real-time scan data by calculating a correlation between the positioning reference data and the real-time scan data;

[0113] Determine a lateral alignment offset between the depth maps, wherein the lateral offset is based on the most common lateral offset between corresponding pixels of the depth maps; and

[0114] Adjust the supposed current position using the determined longitudinal and lateral alignment offsets to determine the position of the vehicle relative to the digital map.

[0115] The invention according to this other aspect may include any or all of the features described in other aspects of the invention, provided they are not mutually contradictory.

[0116] According to these aspects and embodiments of the present invention in which a lateral alignment offset is determined, the most common lateral alignment offset can be determined by considering the depth channel data of corresponding pixels of the depth maps. The most common lateral alignment offset is a determined lateral alignment offset determined between corresponding pairs of located pixels of the depth maps, and preferably is based on the lateral alignment offset of each pair of corresponding pixels. To determine the lateral alignment offset between corresponding pixels of the depth maps, corresponding pairs of pixels in the depth maps must be identified. The method may include identifying corresponding pairs of pixels in the depth maps. Preferably, a longitudinal alignment offset is determined prior to the lateral alignment offset. The depth maps are desirably shifted relative to each other until they are longitudinally aligned to enable identification of corresponding pixels in each depth map.

[0117] Accordingly, the method may further include longitudinally aligning the depth maps relative to each other based on the determined longitudinal alignment offset. The step of longitudinally aligning the depth maps with each other may include longitudinally shifting one or both of the depth maps. The longitudinal shift of the depth maps relative to each other may be implemented in the image domain. Accordingly, the step of aligning the depth maps may include longitudinally shifting the raster images corresponding to each depth map relative to each other. The method may further include cropping the size of the image provided by the located reference data depth map to correspond to the size of the image provided by the real-time scan data depth map. This may facilitate comparison between the depth maps.

[0118] Once corresponding pixels in two depth maps have been identified, the lateral offset between each pair of corresponding pixels can be determined. This can be achieved by comparing the distances along a pre-determined direction from the position of the pixel in the reference plane to the surface of the object in the environment as indicated by the depth channel data associated with each pixel. As described earlier, the depth maps preferably have a variable depth resolution. The lateral alignment offset between each pair of corresponding pixels can be based on the difference in the distances indicated by the depth channel data of the pixels. The method may include using a histogram to identify the most common lateral alignment offset between corresponding pixels of the depth maps. The histogram may indicate the frequency of occurrence of different lateral alignment offsets between corresponding pixel pairs. The histogram may indicate the probability density function of the lateral alignment offset, where the mode reflects the most likely shift.

[0119] In some embodiments, each pixel has a color indicative of the value of the depth channel of the pixel. Accordingly, comparison of the depth values of corresponding pixels may include comparing the colors of corresponding pixels of the depth maps. The difference in color between corresponding pixels may indicate the lateral alignment offset between the pixels, for example when the depth maps have a fixed depth resolution.

[0120] In these embodiments in which a lateral alignment offset has been determined, the current longitudinal and lateral position of the vehicle relative to the digital map can be adjusted.

[0121] According to any aspect or embodiment of the present invention in which the current position of a vehicle (whether longitudinal and / or lateral position) is adjusted, the adjusted current position may be an estimate of the current position obtained in any suitable manner, such as from an absolute position determination system or other position determination system as described above. For example, GPS or dead reckoning may be used. As should be understood, the absolute position is preferably matched to a digital map to determine an initial position relative to the digital map; then, a longitudinal and / or lateral correction is applied to the initial position to improve the position relative to the digital map.

[0122] The Applicant has recognized that while the techniques described above may be useful in adjusting the position of a vehicle relative to a digital map, they will not correct the forward direction of the vehicle. In a preferred embodiment, the method further comprises adjusting the perceived forward direction of the vehicle using positioning reference data and a depth map of real-time scan data. This additional step is preferably carried out in addition to determining the longitudinal and lateral alignment offsets of the depth map according to any of the embodiments described above. In these embodiments, the perceived forward direction of the vehicle may be determined in any suitable manner, such as using GPS data etc., as described with respect to determining the perceived position of the vehicle.

[0123] It has been found that when the perceived forward direction of the vehicle is incorrect, the lateral alignment offset between corresponding pixels of the depth map will vary in the longitudinal direction along the depth map (i.e., along the depth map image). It has been found that the forward direction offset may be determined based on a function indicating the variation of the lateral alignment offset between corresponding pixels of the depth map relative to the longitudinal position along the depth map. The step of determining the forward direction offset may incorporate any of the features described earlier with respect to determining the lateral alignment offset of corresponding pixels. Thus, the method preferably first comprises shifting the depth maps relative to each other to longitudinally align the depth maps.

[0124] Thus, the method may further comprise: determining a longitudinal alignment offset between the depth maps; determining a function indicating the variation of the lateral alignment offset between corresponding pixels of the depth map relative to the longitudinal position of the pixels along the depth map; and using the determined function to adjust the perceived current forward direction of the vehicle to determine the forward direction of the vehicle relative to the digital map.

[0125] The determined lateral alignment offset between corresponding pixels is, as described above, preferably based on the difference in values indicated by the depth channel data of the pixels, for example by reference to the color of the pixels.

[0126] In these aspects or embodiments, the determined function indicates the forward direction offset of the vehicle.

[0127] The step of determining a function indicative of a change in a lateral alignment offset relative to a longitudinal position may include determining an average (i.e., mean) lateral alignment offset of corresponding pixels of a depth map in each of a plurality of vertical sections that traverse the depth map along a longitudinal direction of the depth map. The function may then be obtained based on a change in the average lateral alignment offset determined for each vertical section along the longitudinal direction of the depth map. It should be understood that at least some, and optionally each, of the corresponding pixel pairs in the depth map are considered in determining the function.

[0128] According to another aspect of the present invention, there is provided a method of determining a position of a vehicle relative to a digital map, the digital map including data representing navigable elements of a navigable network along which the vehicle travels, the method comprising:

[0129] obtaining positioning reference data associated with the digital map for a supposed current position of the vehicle along the navigable elements of the navigable network, wherein the positioning reference data includes at least one depth map indicative of an environment around the vehicle projected onto a reference plane, the reference plane being defined by a reference line associated with the navigable element, each pixel of the at least one depth map being associated with a position in the reference plane associated with the navigable element along which the vehicle travels, and the pixel including a depth channel representing a distance along a predetermined direction from the associated position of the pixel in the reference plane to a surface of an object in the environment;

[0130] determining real-time scan data by scanning the environment around the vehicle using at least one sensor, wherein the real-time scan data includes at least one depth map indicative of the environment around the vehicle, each pixel of the at least one depth map being associated with a position in the reference plane associated with the navigable element, and the pixel including a depth channel representing a distance along a predetermined direction from the associated position of the pixel in the reference plane to a surface of an object in the environment determined using the at least one sensor;

[0131] determining a function indicative of a change in a lateral alignment offset between corresponding pixels of the positioning reference data and the real-time sensor data depth map relative to a change in a longitudinal position of the pixel along the depth map; and

[0132] using the determined function to adjust the supposed current forward direction of the vehicle to determine the forward direction of the vehicle relative to the digital map.

[0133] The present invention according to this other aspect may include any or all of the features described in other aspects of the present invention, provided they are not mutually inconsistent.

[0134] In these aspects and embodiments of the present invention, additional steps may be taken to improve the determined forward direction offset, for example by filtering out noise pixels or by weighting the average pixel depth difference within the section by reference to the number of significant pixels considered within a longitudinal section of the depth map or image.

[0135] As mentioned above, the depth map of the localization reference data and thus also the depth map of the real-time data can be transformed so as to always be associated with a linear reference line. Due to this linearization of the depth map, it has been found that when the navigable element is curved, it is not possible to directly apply the determined longitudinal, lateral and / or forward direction corrections. The applicant has identified computationally efficient methods of adjusting or correcting the current position of the vehicle relative to the digital map which involve applying each of the corrections in a series of incremental independent linear update steps.

[0136] Thus, in a preferred embodiment, the determined longitudinal offset is applied to the current position of the vehicle relative to the digital map and at least one depth map of the real-time scan data is recalculated based on the adjusted position. The determined lateral offset using the recalculated real-time scan data is then applied to the adjusted position of the vehicle relative to the digital map and at least one depth map of the real-time scan data is recalculated based on another adjusted position. The determined skew, i.e., forward direction offset, using the recalculated real-time scan data is then applied to another adjusted position of the vehicle relative to the digital map and at least one depth map of the real-time scan data is recalculated based on the again adjusted position. These steps are preferably repeated any number of times as required until there is zero or substantially zero longitudinal offset, lateral offset and skew.

[0137] It should be understood that the generated localization reference data obtained in accordance with any aspect or embodiment of the present invention can be used in other ways in conjunction with the real-time scan data to determine the more precise position of the vehicle or, indeed, for other purposes. In particular, the applicant has recognized that it may not always be possible or at least not always convenient to use the real-time scan data to determine the corresponding depth map for comparison with the depth map of the localization reference scan data. In other words, it may be inappropriate to perform the comparison of the data sets in the image domain. In particular, this may be the case where the type of sensors available on the vehicle is different from the type of sensors used to obtain the localization reference data.

[0138] According to some additional aspects and embodiments of the present invention, the method includes using the localization reference data to determine a reference point cloud indicative of the environment around the navigable element, the reference point cloud comprising a set of first data points in a three-dimensional coordinate system, wherein each first data point represents the surface of an object in the environment.

[0139] In another aspect of the present invention, there is provided a method for generating positioning reference data associated with a digital map, the positioning reference data providing a compressed representation of the environment around at least one navigable element of a navigable network represented by the digital map. The method includes performing the following operations for at least one navigable element represented by the digital map:

[0140] Generating positioning reference data including at least one depth map indicating the environment around the navigable element projected onto a reference plane, the reference plane being defined by a reference line associated with the navigable element. Each pixel in the at least one depth map is associated with a position in the reference plane associated with the navigable element, and the pixel includes a depth channel representing the distance along a predetermined direction from the associated position of the pixel in the reference plane to the surface of an object in the environment;

[0141] Associating the generated positioning reference data with the digital map data; and

[0142] Using the positioning reference data to determine a reference point cloud indicating the environment around the navigable element, the reference point cloud including a set of first data points in a three-dimensional coordinate system, where each first data point represents the surface of an object in the environment.

[0143] The present invention according to this other aspect may include any or all of the features described in other aspects of the present invention, provided that they are not mutually contradictory.

[0144] In another aspect of the present invention, there is provided a method for generating positioning reference data associated with a digital map representing an element of a navigable network, the positioning reference data providing a compressed representation of the environment around at least one junction point of the navigable network represented by the digital map. The method includes performing the following operations for at least one junction point represented by the digital map:

[0145] Generating positioning reference data including at least one depth map indicating the environment around the junction point projected onto a reference plane, the reference plane being defined by a reference line defined by a radius centered on a reference point associated with the junction point. Each pixel in the at least one depth map is associated with a position in the reference plane associated with the junction point, and the pixel includes a depth channel representing the distance along a predetermined direction from the associated position of the pixel in the reference plane to the surface of an object in the environment;

[0146] Associating the generated positioning reference data with the digital map data indicating the junction point; and

[0147] Using the positioning reference data, a reference point cloud indicating the environment around the junction point is determined, the reference point cloud including a set of first data points in a three-dimensional coordinate system, wherein each first data point represents the surface of an object in the environment.

[0148] The invention according to this further aspect may include any or all of the features described in other aspects of the invention, provided that they are not mutually contradictory.

[0149] A reference point cloud including a set of first data points in a three-dimensional coordinate system (wherein each first data point represents the surface of an object in the environment) may be referred to herein as a "3D point cloud". The 3D point cloud obtained according to these further aspects of the invention may be used in determining the positioning of a vehicle.

[0150] In some embodiments, the method may include using the generated positioning reference data in any aspect or embodiment of the invention in determining the position of a vehicle relative to a digital map, the digital map including data representing navigable elements of a navigable network along which the vehicle travels, the method including:

[0151] Obtaining positioning reference data associated with the digital map for a supposed current position of the vehicle along a navigable element or junction point of the navigable network, using the positioning reference data to determine a reference point cloud indicating the environment around the vehicle, the reference point cloud including a set of first data points in a three-dimensional coordinate system, wherein each first data point represents the surface of an object in the environment;

[0152] Determining real-time scan data by scanning the environment around the vehicle using at least one sensor, the real-time scan data including a point cloud indicating the environment around the vehicle, the point cloud including a set of second data points in a three-dimensional coordinate system, wherein each data point represents the surface of an object in the environment determined using the at least one sensor;

[0153] Calculating a correlation between the point cloud of the real-time scan data and the point cloud of the obtained positioning reference data to determine an alignment offset between the point clouds; and

[0154] Using the determined alignment offset to adjust the supposed current position to determine the position of the vehicle relative to the digital map.

[0155] The invention according to this further aspect may include any or all of the features described in other aspects of the invention, provided that they are not mutually contradictory.

[0156] According to another aspect of the present invention, there is provided a method for determining the position of a vehicle relative to a digital map, the digital map including data representing navigable elements of a navigable network along which the vehicle travels, the method comprising:

[0157] Obtaining positioning reference data associated with the digital map for a supposed current position of the vehicle along a navigable element of the navigable network, wherein the positioning reference data includes at least one depth map indicating the environment around the vehicle projected onto a reference plane, the reference plane being defined by a reference line associated with the navigable element, each pixel in the at least one depth map being associated with a position in the reference plane associated with the navigable element along which the vehicle travels, and the pixel including a depth channel representing the distance from the associated position of the pixel in the reference plane to the surface of an object in the environment along a pre-determined direction;

[0158] Using the positioning reference data to determine a reference point cloud indicating the environment around the vehicle, the reference point cloud including a set of first data points in a three-dimensional coordinate system, wherein each first data point represents the surface of an object in the environment;

[0159] Determining real-time scan data by scanning the environment around the vehicle using at least one sensor, the real-time scan data including a point cloud indicating the environment around the vehicle, the point cloud including a set of second data points in a three-dimensional coordinate system, wherein each data point represents the surface of an object in the environment determined using the at least one sensor;

[0160] Calculating a correlation between the point cloud of the real-time scan data and the point cloud of the obtained positioning reference data to determine an alignment offset between the point clouds; and

[0161] Using the determined alignment offset to adjust the supposed current position to determine the position of the vehicle relative to the digital map.

[0162] The invention according to this other aspect may include any or all of the features described in other aspects of the invention, provided that they are not mutually contradictory.

[0163] According to yet another aspect of the present invention, there is provided a method for determining the position of a vehicle relative to a digital map, the digital map including data representing junctions of a navigable network through which the vehicle travels, the method comprising:

[0164] Obtain positioning reference data associated with the digital map for a supposed current position of the vehicle at a junction of the navigable network, wherein the positioning reference data includes at least one depth map indicating the environment around the vehicle projected onto a reference plane, the reference plane being defined by a reference line bounded by a radius centered on a reference point associated with the junction, each pixel in the at least one depth map being associated with a position in the reference plane associated with the junction through which the vehicle travels, and the pixel including a depth channel representing the distance along a predetermined direction from the associated position of the pixel in the reference plane to the surface of an object in the environment;

[0165] Use the positioning reference data to determine a reference point cloud indicating the environment around the vehicle, the reference point cloud including a set of first data points in a three-dimensional coordinate system, wherein each first data point represents the surface of an object in the environment;

[0166] Determine real-time scan data by scanning the environment around the vehicle using at least one sensor, the real-time scan data including a point cloud indicating the environment around the vehicle, the point cloud including a set of second data points in a three-dimensional coordinate system, wherein each data point represents the surface of an object in the environment determined using the at least one sensor;

[0167] Calculate a correlation between the point cloud of the real-time scan data and the point cloud of the obtained positioning reference data to determine an alignment offset between the point clouds; and

[0168] Use the determined alignment offset to adjust the supposed current position to determine the position of the vehicle relative to the digital map.

[0169] A reference point cloud including a set of second data points in a three-dimensional coordinate system (wherein each second data point represents the surface of an object in the environment) in these additional aspects may be referred to herein as a "3D point cloud".

[0170] In these further aspects or embodiments of the invention, localization reference data is used to obtain a 3D reference point cloud. This indication data relates to the environment around the navigable element or junction point, and thus indicates the environment around the vehicle as the vehicle travels along the navigable element or through the junction point. The point cloud of the real-time sensor data relates to the environment around the vehicle, and thus can also be said to relate to the environment around the navigable element or junction point where the vehicle is located. In some preferred embodiments, the 3D point cloud obtained based on the localization reference data is compared with the 3D point cloud indicating the environment around the vehicle (i.e., when traveling along the relevant element or through the junction point) obtained based on real-time scan data. Then, the position of the vehicle can be adjusted based on this comparison, rather than the comparison of depth maps (e.g., raster images).

[0171] A point cloud of real-time scan data is obtained using one or more sensors associated with the vehicle. A single sensor or multiple such sensors can be used, and in the latter case, any combination of sensor types can be used. The sensors can include any one or some of the following: a set of one or more laser scanners; a set of one or more radar scanners; and a set of one or more cameras, such as a single camera or a pair of stereo cameras. A single laser scanner, radar scanner, and / or camera can be used. In the case where the vehicle is associated with one or more cameras, the images obtained from the one or more cameras can be used to construct a three-dimensional scene indicating the environment around the vehicle, and the three-dimensional scene can be used to obtain a 3D point cloud. For example, in the case where the vehicle uses a single camera, the point cloud can be determined from it by the following steps: obtaining a sequence of two-dimensional images from the camera as the vehicle travels along the navigable element or through the junction point; using the sequence of two-dimensional images to construct a three-dimensional scene; and using the three-dimensional scene to obtain a three-dimensional point cloud. In the case where the vehicle is associated with a stereo camera, the images obtained from the camera can be used to obtain a three-dimensional scene, which is then used to obtain a three-dimensional point cloud.

[0172] By transforming the depth map of the localization reference data into a 3D point cloud, it can be compared with the 3D point cloud obtained by real-time scanning using vehicle sensors, regardless of what sensors the vehicle sensors may be. For example, the localization reference data can be based on a reference scan using multiple sensor types, including laser scanners, cameras, and radar scanners. The vehicle may or may not have a corresponding set of sensors. For example, typically the vehicle may only include one or more cameras.

[0173] Localization reference data can be used to determine a reference point cloud corresponding to a point cloud expected to be generated by at least one sensor of a vehicle, which indicates the environment around the vehicle. In the case where the reference point cloud is obtained using a sensor of the same type as the sensor type of the vehicle, this can be direct, and all the localization reference data can be used in constructing the 3D point cloud. Similarly, under certain conditions, data sensed by one type of sensor can be similar to data sensed by another sensor. For example, an object sensed by a lidar sensor when providing reference localization data is also expected to be sensed by a camera of the vehicle during the day. However, the method can include only those points in the 3D point cloud that are expected to be detected by one or more sensors of the type associated with the vehicle and / or are expected to be detected under the current conditions. The localization reference data can include data enabling the generation of an appropriate reference point cloud.

[0174] In some embodiments, as described above, each pixel of the localization reference data further includes at least one channel indicating a sensed reflectivity value. Each pixel can include one or more of the following: a channel indicating a sensed lidar reflectivity value and a channel indicating a sensed radar reflectivity value. Preferably, channels indicating both radar and lidar reflectivities are provided. Then, the step of generating the 3D point cloud based on the localization reference data preferably uses the sensed reflectivity data. The generation of the 3D point cloud can also be based on the type of one or more sensors of the vehicle. The method can include using the reflectivity data and data indicating the type of one or more sensors of the vehicle to select 3D points included in the reference 3D point cloud. The data of the reflectivity channel is used to select data from the depth channel for generating the 3D point cloud. The reflectivity channel gives an indication of whether a particular object will be sensed by the relevant sensor type (under appropriate circumstances, under the current conditions).

[0175] For example, in the case where the reference data is based on data obtained from a lidar scanner and a radar scanner and the vehicle has only a radar scanner, the radar reflectivity value can be used to select those points included in the 3D points expected to be sensed by the radar scanner of the vehicle. In some embodiments, each pixel includes a channel indicating radar reflectivity, and the method includes the step of using the radar reflectivity data to generate a 3D reference point cloud containing only those points that will be sensed by the radar sensor. In the case where the method further includes comparing the 3D reference point cloud with the 3D point cloud obtained based on real-time scan data, the 3D point cloud of the real-time scan data is thus based on data obtained from the radar scanner. The vehicle can include only a radar scanner.

[0176] When a vehicle may include a radar and / or a lidar scanner, in many cases, the vehicle may include only one camera or multiple cameras. The lidar reflectivity data can provide a way to obtain a 3D reference point cloud related to the 3D point cloud that is expected to be sensed by a vehicle having only one camera or multiple cameras as sensors under dark conditions. The lidar reflectivity data provides an indication of those objects that can be expected to be detected by the camera at night. In some embodiments, each pixel includes a channel indicating the lidar reflectivity, and the method includes the step of using the lidar reflectivity data to generate a 3D reference point cloud containing only those points that will be sensed by the vehicle's camera during dark conditions. In cases where the method further includes comparing the 3D reference point cloud with the 3D point cloud obtained based on real-time scan data, the 3D point cloud of the real-time scan data can thus be based on data obtained from the camera under dark conditions.

[0177] It is believed that obtaining reference localization data in the form of a three-dimensional point cloud, and using this data to reconstruct a reference map, such as an image that can be expected to be obtained from one or more cameras of a vehicle under applicable conditions, and then comparing it with the image obtained by the camera, is itself advantageous.

[0178] In some embodiments, the method may include using the generated localization reference data in any aspect or embodiment of the present invention in reconstructing a view that can be expected to be obtained from one or more cameras associated with a vehicle traveling along a navigable element of a navigable network or passing through a junction represented by a digital map, the method including: obtaining localization reference data associated with a digital map for a supposed current position of the vehicle along or at a navigable element or junction of a navigable network; using the localization reference data to determine a reference point cloud indicating the environment around the vehicle, the reference point cloud including a set of first data points in a three-dimensional coordinate system, where each first data point represents the surface of an object in the environment; and using the reference point cloud to reconstruct a reference view that can be expected to be obtained by one or more cameras associated with the vehicle when passing through the navigable element or junction under applicable conditions. The method may further include using one or more cameras to determine a real-time view of the environment around the vehicle, and comparing the reference view with the real-time view obtained by the one or more cameras.

[0179] According to another aspect of the present invention, there is provided a method of reconstructing a view that can be expected to be obtained from one or more cameras associated with a vehicle traveling along a navigable element of a navigable network represented by a digital map under applicable conditions, the method including:

[0180] Obtain positioning reference data associated with a digital map for a supposed current position of the vehicle along a navigable element of a navigable network, wherein the positioning reference data includes at least one depth map indicative of an environment around the vehicle projected onto a reference plane, the reference plane being defined by a reference line associated with the navigable element, each pixel in the at least one depth map being associated with a position in the reference plane of the navigable element along which the vehicle travels, and the pixel including a depth channel representing a distance along a predetermined direction from the associated position of the pixel in the reference plane to a surface of an object in the environment;

[0181] Use the positioning reference data to determine a reference point cloud indicative of the environment around the vehicle, the reference point cloud including a set of first data points in a three-dimensional coordinate system, wherein each first data point represents a surface of an object in the environment;

[0182] Use the reference point cloud to reconstruct a reference view that is expected to be obtained by one or more cameras associated with the vehicle when crossing the navigable element under applicable conditions;

[0183] Use the one or more cameras to determine a real-time view of the environment around the vehicle; and

[0184] Compare the reference view with the real-time view obtained by the one or more cameras.

[0185] The invention according to this other aspect may include any or all of the features described in other aspects of the invention, provided that they are not mutually contradictory.

[0186] According to another aspect of the invention, there is provided a method of reconstructing a view that is expected to be obtained by one or more cameras associated with a vehicle traveling through a junction of a navigable network represented by a digital map under applicable conditions, the method comprising:

[0187] Obtain positioning reference data associated with a digital map for a supposed current position of the vehicle along a navigable element of a navigable network, wherein the positioning reference data includes at least one depth map indicative of an environment around the vehicle projected onto a reference plane, the reference plane being defined by a reference line bounded by a radius centered on a reference point associated with the junction, each pixel in the at least one depth map being associated with a position in the reference plane of the junction through which the vehicle travels, and the pixel including a depth channel representing a distance along a predetermined direction from the associated position of the pixel in the reference plane to a surface of an object in the environment;

[0188] Use the positioning reference data to determine a reference point cloud indicative of the environment around the vehicle, the reference point cloud comprising a set of first data points in a three-dimensional coordinate system, wherein each first data point represents the surface of an object in the environment;

[0189] Use the reference point cloud to reconstruct a reference view that is expected to be obtained by one or more cameras associated with the vehicle when traversing the navigable element under applicable conditions;

[0190] Use the one or more cameras to determine a real-time view of the environment around the vehicle; and

[0191] Compare the reference view with the real-time view obtained by the one or more cameras.

[0192] The invention according to this further aspect may include any or all of the features described in other aspects of the invention, provided that they are not mutually contradictory.

[0193] These aspects of the invention are particularly advantageous in allowing the construction of a reference view that can be compared with the real-time view obtained by the vehicle's camera, but that is based on positioning reference data that can be obtained from different types of sensors. It has been recognized that in practice, many vehicles will be equipped only with one or more cameras, rather than more specific or complex sensors, such as those that can be used to obtain reference data.

[0194] In these further aspects and embodiments of the invention, the comparison result of the reference view and the real-time view can be used as needed. For example, the comparison result can be used to determine the position of the vehicle as in the aspects and embodiments described earlier. The method may include calculating a correlation between the real-time view and the reference view to determine an alignment offset between the views; and using the determined alignment offset to adjust the perceived current position of the vehicle to determine the position of the vehicle relative to a digital map.

[0195] The applicable conditions are those conditions that are appropriate at the current time and may be lighting conditions. In some embodiments, the applicable conditions are dark conditions.

[0196] According to any of the embodiments described above, a reference view is reconstructed using a 3D reference point cloud obtainable from localization reference data. The step of reconstructing a reference view expected to be obtained by one or more cameras preferably includes using data from a reflectivity data channel associated with a pixel of a depth map of the localization reference data. Preferably, therefore, each pixel of the localization reference data further includes at least one channel indicating a value of sensed laser reflectivity, and the step of generating a 3D point cloud based on the localization reference data is performed using the sensed laser reflectivity data. The laser reflectivity data can be used to select data from the depth channel for generating a reference 3D point cloud to result in a reconstructed reference view corresponding to a view expected to be obtained by one or more cameras of a vehicle, e.g., the view includes those objects that are desired to be visible under applicable conditions (e.g., darkness). The one or more cameras of the vehicle can be a single camera, or a pair of stereo cameras, as described above.

[0197] The comparison of real-time scan data with localization reference data, which can be performed according to various aspects and embodiments of the present invention, whether by comparison of depth maps or by comparison of point clouds or by comparison of reconstructed images with real-time images, can be performed on a data window. The data window is a data window in the direction of travel, e.g., longitudinal data. Thus, windowed data allows the comparison to consider a subset of the available data. The comparison can be performed periodically for overlapping windows. At least some overlap of the windows of data used for comparison is desirable. For example, this can ensure that the difference between adjacent calculated values such as longitudinal offset values is smoothed for the data. The window can have a length sufficient to prevent the accuracy of the offset calculation from changing with transient features, preferably a length of at least 100 m. Such transient features can be, for example, parked vehicles, overtaking vehicles, or vehicles traveling in the opposite direction along the same route. In some embodiments, the length is at least 50 m. In some embodiments, the length is 200 m. In this way, the sensed environmental data is determined for a section of the road (e.g., a longitudinal section) (‘window’, e.g., 200 m), and the resulting data is then compared with the localization reference data for that section. By performing the comparison on a section of this size (i.e., a section substantially larger than the length of the vehicle), non-stationary or temporary objects (e.g., other vehicles on the road, vehicles parked beside the road, etc.) generally do not affect the comparison result.

[0198] At least a portion of the localization reference data used according to any aspect or embodiment of the present invention can be stored remotely. Preferably, in the case of a vehicle, at least a portion of the localization reference data is stored locally on the vehicle. Thus, even if the localization reference data is available throughout the route, it does not need to be continuously transmitted to the vehicle and the comparison can be performed on the vehicle.

[0199] The positioning reference data can be stored in a compressed format. The positioning reference data can have a size corresponding to 30 KB / km or less.

[0200] The positioning reference data can be stored for at least part (and preferably all) of the navigable elements of the navigable network represented in the digital map. Thus, the position of the vehicle can be continuously determined anywhere along the route traveled by the vehicle.

[0201] In an embodiment, the reference positioning data may have been obtained from a reference scan using at least one device positioned on a mobile mapping vehicle that has previously traveled along the navigable element subsequently traveled by the vehicle. Thus, a different vehicle than the current vehicle whose position is continuously determined may have been used to obtain the reference scan. In some embodiments, the mobile mapping vehicle has a similar design to the vehicle whose position is continuously determined.

[0202] At least one rangefinder sensor can be used to obtain real-time scan data and / or reference scan data. The rangefinder sensor can be configured to operate along a single axis. The rangefinder sensor can be arranged to perform scans on a vertical axis. When performing scans on the vertical axis, distance information of planes at multiple heights is collected, and thus the resulting scan is significantly more detailed. Alternatively or additionally, the rangefinder sensor can be arranged to perform scans on a horizontal axis.

[0203] The rangefinder sensor can be a laser scanner. The laser scanner can include a laser beam that scans the lateral environment using a mirror. Additionally or alternatively, the rangefinder sensor can be a radar scanner and / or a pair of stereo cameras.

[0204] The invention extends to a device, such as a navigation device, a vehicle, etc., that has components, such as one or more processors, which are arranged (e.g., programmed) to perform any of the methods described herein.

[0205] The step of generating the positioning reference data described herein is preferably performed by a server or another similar computing device.

[0206] The components for implementing any step of the method can include a set of one or more processors configured (e.g., programmed) to do so. A given step can be implemented using the same or different set of processors as any other step. Any given step can be implemented using a combination of processor sets. The system can further include a data storage component, such as a computer memory, for storing, for example, digital maps, positioning reference data, and / or real-time scan data.

[0207] In a preferred embodiment, the method of the present invention is implemented by a server or a similar computing device. In other words, the proposed method of the present invention is preferably a computer-implemented method. Thus, in an embodiment, the system of the present invention includes a server or a similar computing device including components for implementing the various steps described, and the method steps described herein are implemented by the server.

[0208] The present invention is further extended to a computer program product including computer-readable instructions executable to perform or cause a device to perform any method described herein. The computer program product is preferably stored in a non-transitory physical storage medium.

[0209] As will be appreciated by those skilled in the art, aspects and embodiments of the present invention may and preferably do, as appropriate, incorporate any one or more or all of the preferred and optional features of the present invention described herein with respect to any other aspect of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0210] Embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0211] Figure 1 is a representation of a portion of a planning map;

[0212] Figure 2 shows a portion of a planning map overlaid on an image of a road network;

[0213] Figure 3 and 4 shows an exemplary mobile mapping system that can be used to collect data for constructing a map;

[0214] Figure 5 shows a 3D view of data obtained from a laser scanner, while Figure 6 shows a side view projection of data obtained from a laser scanner;

[0215] Figure 7 shows a vehicle sensing its surrounding environment while traveling along a road according to an embodiment;

[0216] Figure 8 shows a comparison of positioning reference data compared to sensed environmental data (e.g., collected by a vehicle as described by Figure 7 );

[0217] Figure 9 shows an exemplary format in which positioning reference data can be stored;

[0218] Figure 10A shows an exemplary point cloud obtained by a ranging sensor mounted on a vehicle traveling along a road, while Figure 10BShow that this point cloud data has been converted into two depth maps;

[0219] Figure 11 Show the offsets determined according to the normalized cross - correlation calculation in the embodiments;

[0220] Figure 12 Show another instance of the correlation performed between the "reference" data set and the "local measurement" data set;

[0221] Figure 13 Show the system located inside the vehicle according to the embodiment;

[0222] Figure 14A Show an exemplary raster image as part of a piece of positioning reference data;

[0223] Figure 14B The Figure 14A bird's - eye perspective view of the data is shown as two separate planes on the left and right sides of the road;

[0224] Figure 15A Show the fixed longitudinal resolution and variable (e.g., non - linear) vertical and / or depth resolution of the positioning reference data and the real - time scan data;

[0225] Figure 15B Show the function that maps the height on the reference line to the pixel Y - coordinate value;

[0226] Figure 15C Show the function that maps the distance from the reference line to the pixel depth value;

[0227] Figure 15D Show the fixed longitudinal pixel resolution, variable vertical pixel resolution, and variable depth value resolution in the three - dimensional graph;

[0228] Figure 16A Show the orthogonal projection onto the reference plane bounded by the reference line associated with the road element;

[0229] Figure 16B Show the side depth map obtained using the orthogonal projection;

[0230] Figure 16C Show the non - orthogonal projection onto the reference plane bounded by the reference line associated with the road element;

[0231] Figure 16D Show the side depth map obtained using the non - orthogonal projection;

[0232] Figure 17 Show the multi - channel data format of the depth map;

[0233] Figure 18Show circular and linear reference lines that can be used to construct a depth map at an intersection;

[0234] Figure 19A Show ways in which an object can be projected onto a circular depth map at different angular positions;

[0235] Figure 19B Show the orthogonal projection of an object for which a depth map is to be provided onto a reference plane;

[0236] Figure 20A Show a reference depth map and a corresponding real-time depth map;

[0237] Figure 20B Show a longitudinal correction derived from the longitudinal correlation of the reference and real-time depth maps;

[0238] Figure 20C Show a lateral correction derived from the histogram difference between the pixel depth values of corresponding pixels in the reference and real-time depth maps;

[0239] Figure 20D Show how the longitudinal position of a vehicle on a road can be corrected and then the lateral position can be corrected;

[0240] Figure 21A Show a set of vertical slices passing through corresponding portions of the reference depth map;

[0241] Figure 21B Show the average pixel depth difference of the vertical slices plotted against the longitudinal distance along the vertical slices of the depth map;

[0242] Figure 22 Show an image of a curved road of a road and a corresponding linear reference image;

[0243] Figure 23A and 23B Show a method for establishing the position of a vehicle, e.g., in a non-linear environment;

[0244] Figure 24 Show an exemplary system in which a data vehicle sensor is correlated with reference data to locate the vehicle relative to a digital map;

[0245] Figure 25A 、 25B and 25C show a first example use case in which a reference depth map is used to construct a 3D point cloud, and then the 3D point cloud is compared with a 3D point cloud obtained from a vehicle lidar sensor;

[0246] Figure 26A 、 26B, 26C and 26D illustrate a second example use case, where a reference depth map is used to construct a 3D point cloud or view, and then the 3D point cloud or view is compared with a 3D scene or view obtained from multiple vehicle cameras or a single vehicle camera;

[0247] Figure 27A , 27B and 27C illustrate a third example use case, where the reflectivity data of a depth map is used to construct a 3D point cloud or view, and then the 3D point cloud or view is compared with a 3D scene or view obtained from a vehicle camera;

[0248] Figure 28A and 28B illustrate a fourth example use case, where the radar data of a depth map is used to construct a 3D point cloud, and then the 3D point cloud is compared with a 3D scene obtained using vehicle radar;

[0249] Figure 29 illustrates different coordinate systems used in embodiments of the present invention;

[0250] Figure 30 depicts steps performed when correlating vehicle sensor data with reference data to determine the position of a vehicle;

[0251] Figure 31 describes steps performed to determine Figure 30 the laser point cloud in the method;

[0252] Figure 32A describes a first exemplary method for performing Figure 30 the correlation steps in the method; and

[0253] Figure 32B describes a second exemplary method for performing Figure 30 the correlation steps in the method. Specific embodiments

[0254] It has been recognized that there is a need for an improved method for determining the position of a device (such as a vehicle) relative to a digital map (representing a navigable network, such as a road network). In particular, there is a need to be able to accurately determine (e.g., with sub-meter accuracy) the longitudinal position of the device relative to the digital map. The term "longitudinal" in this application refers to the direction along the portion of the navigable network on which the device (such as a vehicle) moves; in other words, along the length of the road on which the vehicle travels. The term "lateral" in this application has its usual meaning perpendicular to the longitudinal direction and thus refers to the direction along the width of the road.

[0255] As will be appreciated, when the digital map includes a planned map as described above (e.g., a three-dimensional vector model where each lane of a road is represented separately (as opposed to the center line of the road in a standard map), the lateral position of a device (e.g., a vehicle) simply involves determining the lane in which the device is currently traveling. Various techniques are known for performing this determination. For example, the determination can be made using only information obtained from a global navigation satellite system (GNSS) receiver. Additionally or alternatively, information from cameras, lasers, or other imaging sensors associated with the device can be used; for example, a great deal of research has been done in recent years where (e.g.) image data from one or more cameras mounted in a vehicle is analyzed using various image processing techniques to detect and track the lane in which the vehicle is traveling. An exemplary technique is described in the paper "Multi-lane detection in urban driving environments using conditional random fields" by Junhwa Hur, Sean-Nam Kang, and Seung-Woo Seo, which was published in the proceedings of the Intelligent Vehicles Symposium, pages 1297 to 1302, Institute of Electrical and Electronics Engineers (IEEE), (2013). Here, the device can have data feeds from cameras, radar, and / or lidar sensors and use appropriate algorithms to process the received data in real time to determine the current lane of the device or vehicle in which it is traveling. Alternatively, another device or equipment (e.g., a Mobileye system available from Mobileye N.V. of Nevada) can provide a determination of the current lane of the vehicle based on these data feeds and then feed the determination of the current lane to the device, e.g., via a wired connection or a Bluetooth connection.

[0256] In an embodiment, the longitudinal position of a vehicle can be determined by comparing a real-time scan of the environment around the vehicle (and preferably on one or both sides of the vehicle) with a reference scan of the environment associated with the digital map. Based on this comparison, a longitudinal offset (if any) can be determined, and the determined offset can be used to match the position of the vehicle with the digital map. Thus, the position of the vehicle relative to the digital map can always be known with high accuracy.

[0257] A real-time scan of the environment around a vehicle can be obtained using at least one rangefinder sensor located on the vehicle. The at least one rangefinder sensor can take any suitable form, but in a preferred embodiment includes a laser scanner, i.e., a LIDAR device. The laser scanner can be configured to scan a laser beam throughout the environment and create a point cloud representation of the environment; each point indicates the position of the surface of an object that reflects the laser. As should be understood, the laser scanner is configured to record the time it takes for the laser beam to return to the scanner after reflecting from the surface of an object, and the recorded time can then be used to determine the distance to each point. In a preferred embodiment, the rangefinder sensor is configured to operate along a single axis to obtain data within a certain acquisition angle (e.g., between 50 and 90°, e.g., 70°); for example, when the sensor includes a laser scanner, a mirror within the device is used to scan the laser beam.

[0258] Figure 7 An embodiment is shown in which vehicle 100 is traveling along a road. The vehicle is equipped with rangefinder sensors 101, 102 located on each side of the vehicle. Although the sensors are shown on each side of the vehicle, in other embodiments, only a single sensor may be used on one side of the vehicle. Preferably, the sensors are appropriately aligned such that data from each sensor can be combined, as discussed in more detail below.

[0259] WO 2011 / 146523 A2 provides an example of a scanner that can be used on a vehicle to capture reference data in the form of a 3D point cloud, or it can also be used on an autonomous vehicle to obtain real-time data related to the surrounding environment.

[0260] As discussed above, the rangefinder sensor can be arranged to operate along a single axis. In one embodiment, the sensor can be arranged to perform scans in the horizontal direction (i.e., in a plane parallel to the road surface). This is shown (for example) in Figure 7 By continuously scanning the environment as the vehicle travels along the road, sensed environmental data as shown in Figure 8 can be collected. Data 200 is data collected from the left sensor 102 and shows object 104. Data 202 is data collected from the right sensor 101 and shows objects 106 and 108. In other embodiments, the sensor can be arranged to perform scans in the vertical direction (i.e., in a plane perpendicular to the road surface). By continuously scanning the environment as the vehicle travels along the road, it is possible to collect environmental data in the manner of Figure 6 It will be understood that by performing scans in the vertical direction, distance information for planes at multiple heights is collected, and thus the resulting scans are significantly more detailed. Of course, it will be understood that scans can be performed along any axis as needed.

[0261] A reference scan of the environment is obtained from one or more vehicles that have previously traveled along a road, and is then appropriately aligned with and associated with a digital map. The reference scan is stored in a database associated with the digital map, and is referred to herein as localization reference data. When matched with the digital map, the combination of the localization reference data can be referred to as a localization map. As will be appreciated, the localization map is created remotely from the vehicle; typically provided by a digital map making company (such as TomTom International B.V. or HERE, a Nokia company).

[0262] The reference scan can be obtained from a dedicated vehicle, such as a mobile mapping vehicle (e.g., as shown in Figure 3 ). However, in a preferred embodiment, the reference scan can be determined from sensed environmental data collected by the vehicle as it travels along a navigable network. This sensed environmental data can be stored and periodically sent to a digital map making company to create, maintain, and update the localization map.

[0263] Although the localization reference data is preferably stored locally at the vehicle, it should be appreciated that the data can be stored remotely. In an embodiment, and particularly when storing the localization reference data locally, the data is stored in a compressed format.

[0264] In an embodiment, localization reference data is collected for each side of a road in a road network. In such embodiments, the reference data for each side of the road can be stored separately, or alternatively it can be stored together in a combined dataset.

[0265] In an embodiment, the localization reference data can be stored as image data. The image data can be a color (e.g., RGB) image or a grayscale image.

[0266] Figure 9 A exemplary format showing how the localization reference data can be stored is presented. In this embodiment, the reference data for the left side of the road is provided on the left side of the image, and the reference data for the right side of the road is provided on the right side of the image; the datasets are aligned such that the left reference dataset for a particular longitudinal position is shown opposite the right reference dataset for the same longitudinal position.

[0267] In the Figure 9 image, and for illustrative purposes only, the longitudinal pixel size is 0.5 m, and there are 40 pixels on each side of the centerline. It has also been determined that the image can be stored as a grayscale image rather than a color (RGB) image. By storing the image in this format, the localization reference data has a size corresponding to 30 KB / km.

[0268] In Figure 10A and 10BAnother example can be seen in Figure 10A shows an example point cloud obtained by a ranging sensor mounted on a vehicle traveling along a road. In Figure 10B this, the point cloud data has been converted into two depth maps; one for the left side of the vehicle and the other for the right side of the vehicle, which have been placed close to each other to form a composite image.

[0269] As discussed above, the sensed environmental data determined by the vehicle is compared with the positioning reference data to determine if there is an offset. Any determined offset can then be used to adjust the position of the vehicle so that it accurately matches the correct position on the digital map. This determined offset is referred to herein as the correlation index.

[0270] In an embodiment, the sensed environmental data is determined for a longitudinal section (e.g., 200 m), and the resulting data (e.g., image data) is then compared with the positioning reference data for the section. By performing the comparison over a section of this size (i.e., substantially larger than the length of the vehicle), non-stationary or temporary objects (e.g., other vehicles on the road, vehicles parked beside the road, etc.) will generally not affect the comparison result.

[0271] Preferably, the comparison is performed by calculating the cross-correlation between the sensed environmental data and the positioning reference data in order to determine the longitudinal position where the degree of alignment of the data sets is highest. The difference between the longitudinal positions of the two data sets with the maximum alignment allows determination of the longitudinal offset. This can be seen (e.g.) by the offset indicated between Figure 8 the sensed environmental data and the positioning reference data of

[0272] In an embodiment, when the data sets are provided as images, the cross-correlation includes a normalized cross-correlation operation such that differences in brightness, lighting conditions, etc. between the positioning reference data and the sensed environmental data can be alleviated. Preferably, the comparison is performed periodically on overlapping windows (e.g., 200 m long) such that any offset is continuously determined as the vehicle travels along the road. Figure 11 shows, in an exemplary embodiment, the offset determined based on the normalized cross-correlation calculation between the depicted positioning reference data and the depicted sensed environmental data.

[0273] Figure 12 illustrates another example of the correlation performed between a "reference" data set and a "local measurement" data set (which is obtained by the vehicle as it travels along the road). The result of the correlation between the two images can be seen in a plot of "shift" versus "longitudinal correlation index", where the position of the maximum peak is used to determine the best-fit shift illustrated, which can then be used to adjust the longitudinal position of the vehicle relative to the digital map.

[0274] As can be seen from Figure 9 , 10B , 11 and 12, the positioning reference data and the sensed environmental data are preferably in the form of a depth map, where each element (e.g., a pixel when the depth map is stored as an image) includes: a first value indicating a longitudinal position (along the road); a second value indicating a height (i.e., a height above the ground); and a third value indicating a lateral position (across the road). Each element (e.g., pixel) of the depth map thus effectively corresponds to a portion of the surface of the environment around the vehicle. As will be appreciated, the size of the surface represented by each element (e.g., pixel) will vary with the amount of compression such that the element (e.g., pixel) will represent a larger surface area with a higher compression level of the depth map (or image).

[0275] In an embodiment, where the positioning reference data is stored in a data storage component (e.g., a memory) of the device, the comparison step can be performed on one or more processors within the vehicle. In other embodiments, where the positioning reference data is stored away from the vehicle, the sensed environmental data can be sent to a server via a wireless connection, e.g., via a mobile telecommunications network. The server, which has access to the positioning reference data, will then return (e.g., also using the mobile telecommunications network) any determined offset to the vehicle.

[0276] Figure 13 depicts an exemplary system located within a vehicle according to an embodiment of the present invention. In this system, a processing device, referred to as a correlation index provider unit, receives data feeds from a range sensor positioned to detect the environment to the left of the vehicle and a range sensor positioned to detect the environment to the right of the vehicle. The processing device also accesses a digital map (which is preferably in the form of a planning map) and a database of positioning reference data that is suitably matched to the digital map. The processing device is arranged to perform the method described above and thus optionally compares the data feeds from the range sensors with the positioning reference data after converting the data feeds into a suitable form (e.g., image data that combines data from the two sensors) to determine a longitudinal offset and thus the accurate position of the vehicle relative to the digital map. The system also includes a horizon provider unit, and the horizon provider unit uses the determined position of the vehicle and the data within the digital map to provide information (referred to as "horizon data") about an upcoming portion of the navigable network that the vehicle is about to cross. This horizon data can then be used to control one or more systems within the vehicle to perform various assisted or autonomous driving operations, e.g., adaptive cruise control, automatic lane change, emergency braking assistance, etc.

[0277] In summary, the present invention relates, at least in the preferred embodiments, to a longitudinal-correlation-based positioning method. The 3D space around the vehicle is represented in the form of two depth maps, which cover the left and right sides of the road and can be combined into a single image. A reference image stored in a digital map is cross-correlated with the depth maps from the vehicle's laser or other range sensors to accurately position the vehicle longitudinally along the representation of the road in the digital map. In an embodiment, the depth information can then be used to position the vehicle transversely across the road.

[0278] In a preferred embodiment, the 3D space around the vehicle is projected onto two grids parallel to the road trajectory, and the projected values are averaged within each cell of the grid. The pixels of the longitudinal-correlator depth map have a size of approximately 50 cm along the travel direction and a height of approximately 20 cm. The depth encoded by the pixel values is quantized to approximately 10 cm. Although the depth-map image resolution along the travel direction is 50 cm, the positioning resolution is much higher. The cross-correlation image represents the grid in which the laser points are distributed and averaged. Appropriate upsampling enables finding the shift vector of the sub-pixel coefficients. Similarly, the depth quantization of approximately 10 cm does not imply a positioning accuracy of 10 cm across the road, because the quantization error is averaged over all the relevant pixels. Thus, in practice, the positioning accuracy is mainly limited by the laser accuracy and calibration, and the quantization error of the longitudinal-correlator index has only a minimal contribution.

[0279] Therefore, it should be understood that the positioning information (e.g., depth map (or image)) is always available (even when there are no clear objects in the surroundings), compact (it is possible to store the road network of the entire world), and enables an accuracy comparable to or even better than other methods (which is attributed to its availability everywhere and thus the higher possibility of error averaging).

[0280] Figure 14A A exemplary raster image is shown as part of a piece of positioning reference data. The raster image is formed by orthogonally projecting the collected 3D laser-point data onto a hyperplane defined by a reference line and oriented perpendicular to the road surface. Due to the orthogonality of the projection, any height information is independent of the distance from the reference line. The reference line itself generally extends parallel to the lane / road boundary. The actual representation of the hyperplane is in a raster format with a fixed horizontal resolution and a non-linear vertical resolution. This method aims to maximize the information density regarding those heights that are important for vehicle sensor detection. Experiments have shown that a raster-plane height of 5 to 10 meters is sufficient to capture enough relevant information necessary for later vehicle positioning. Each individual pixel in the raster reflects a set of laser measurements. Just like the vertical resolution, the resolution in the depth information is also represented in a non-linear manner but is typically stored as an 8-bit value (i.e., as a value from 0 to 255).Figure 14A Show data on both sides of the road. Figure 14B The Figure 14A bird's-eye perspective view of the data is shown as two separate planes on the left and right sides of the road.

[0281] As discussed above, a vehicle equipped with a front- or side-mounted horizontally-mounted laser scanner sensor can generate in real time a 2D plane similar to the 2D plane of the positioning reference data. The positioning of the vehicle relative to the digital map is achieved by the correlation of the image space of the prior mapping data with the data sensed and processed in real time. The longitudinal vehicle positioning is obtained by applying an average non-negative normalized cross-correlation (NCC) operation calculated in an overlapping moving window to an image having 1-pixel blur in the height domain and a Sobel operator in the longitudinal domain.

[0282] Figure 15A Show a fixed longitudinal resolution and a variable (e.g., non-linear) vertical and / or depth resolution of the positioning reference data and the real-time scan data. Thus, although the longitudinal distances represented by the values a, b, and c are the same, the height ranges represented by the values D, E, and F are different. Specifically, the height range represented by D is less than the height range represented by E, and the height range represented by E is less than the height range represented by F. Similarly, the depth range represented by the value 0 (i.e., the surface closest to the vehicle) is less than the depth range represented by the value 100, and the depth range represented by the value 100 is less than the depth range represented by the value 255, i.e., the surface farthest from the vehicle. For example, the value 0 may represent a depth of 1 cm, while the value 255 may represent a depth of 10 cm.

[0283] Figure 15B Illustrate how the vertical resolution can vary. In this example, the vertical resolution varies based on a non-linear function that maps the height above the reference line to the pixel Y coordinate value. As Figure 15B shown, pixels closer to the reference line (equal to 40 at Y in this example) represent lower heights. Also as Figure 15B shown, the vertical resolution is closer to the reference line, i.e., the change in height relative to the pixel position is smaller for pixels closer to the reference line and larger for pixels farther from the reference line.

[0284] Figure 15C Illustrate how the depth resolution can vary. In this example, the depth resolution varies based on a non-linear function that maps the distance from the reference line to the pixel depth (color) value. As Figure 15C shown, lower pixel depth values represent shorter distances from the reference line. Also as Figure 15CAs shown, the depth resolution is greater at lower pixel depth values, i.e., the distance change relative to the pixel depth value is smaller for lower pixel depth values and greater for higher pixel depth values.

[0285] Figure 15D Illustrate how a subset of pixels can be mapped to distances along a reference line. As Figure 15D shown, each pixel along the reference line is the same width, such that the longitudinal pixel resolution is fixed. Figure 15D Also illustrate how a subset of pixels can be mapped to heights above a reference line. As Figure 15D shown, the pixels gradually widen at greater distances from the reference line, such that the vertical pixel resolution is lower at greater heights above the reference line. Figure 15D Also illustrate how a subset of pixel depth values can be mapped to distances from a reference line. As Figure 15D shown, the distance covered by the pixel depth values gradually widens at greater distances from the reference line, such that the depth resolution is lower at greater depth distances from the reference line.

[0286] Some additional embodiments and features of the present invention will now be described.

[0287] As described with respect to Figure 14A , a depth map (e.g., a raster image) of the localization reference data can be provided by orthogonally projecting onto a reference plane defined by a reference line associated with a road element. Figure 16A Illustrate the results of using this projection. The reference plane is perpendicular to the shown road reference line. Here, although the height information is independent of the distance from the reference line, which can provide some advantages, one limitation of the orthogonal projection is that information related to surfaces perpendicular to the road element may be lost. This is illustrated by the Figure 16B side depth map obtained using the orthogonal projection.

[0288] If a non-orthogonal projection is used, e.g., at 45 degrees, then this information related to surfaces perpendicular to the road element can be preserved. This is shown by Figure 16C and 16D shown. Figure 16C Illustrate a 45-degree projection onto a reference plane that is again defined as perpendicular to the road reference line. As Figure 16D shown, the side depth map obtained using this projection contains more information about those surfaces of objects perpendicular to the road element. By using a non-orthogonal projection, information about such perpendicular surfaces can be captured by the depth map data without the need to include additional data channels or otherwise increase the storage capacity. It should be understood that in the case of using this non-orthogonal projection for the depth map data of the localization reference data, the corresponding projection should then be used for the real-time sensing data to be compared with it.

[0289] Each pixel of the depth map data of the localization reference data is based on a set of sensing measurements, e.g., laser measurements. These measurements correspond to sensor measurements indicating the distance of an object from a reference plane along a pre-determined direction at the location of the pixel. Due to the way the data is compressed, a set of sensor measurements will be mapped to a particular pixel. Instead of determining a depth value corresponding to the average of different distances according to the set of sensor measurements to be associated with the pixel, it has been found that using the closest distance among the distances corresponding to various sensor measurements for the pixel depth value can achieve greater accuracy. Importantly, the depth value of the pixel accurately reflects the distance from the reference plane to the nearest surface of the object. This is of most concern when accurately determining the position of a vehicle in a way that will minimize the risk of collision. If the average of a set of sensor measurements is used to provide the depth value of the pixel, then there is a possibility that the depth value will indicate a greater distance to the object surface than is actually the case at the pixel location. This is because an object may be temporarily located between the reference plane and another, more distant object, e.g., a tree may be in front of a building. In this case, some of the sensor measurements used to provide the pixel depth value will be associated with the building and other sensor measurements will be associated with the tree, as a result of the sensor measurements being mapped to an area of pixels extending to one or more sides of the tree. The applicant has recognized that using the closest of the various sensor measurements as the depth value associated with the pixel is the safest and most reliable in order to ensure that the distance to the surface of the nearest object, which in this case is the tree, is reliably captured. Alternatively, a distribution of the sensor measurements for the pixel can be derived and a closest mode can be employed to provide the pixel depth. This will provide a more reliable indication of the pixel depth in a manner similar to the closest distance.

[0290] As described above, the pixels of the depth map data of the localization reference data include a depth channel that contains data indicating the depth from the location of the pixel in the reference plane to the surface of the object. One or more additional pixel channels may be included in the localization reference data. This will result in a multi-channel or multi-layer depth map and thus a raster image. In some preferred embodiments, the second channel contains data indicating the laser reflectivity of the object at the location of the pixel, and the third channel contains data indicating the radar reflectivity of the object at the pixel location.

[0291] Each pixel has a position corresponding to a specific distance along a road reference line (x - direction) and a height above the road reference line (y - direction). The depth value associated with a pixel in the first channel c1 indicates the distance of the pixel in the reference plane to the surface of the nearest object along a pre - determined direction (which may depend on whether the reference plane of the used projection is orthogonal or non - orthogonal to the reference plane), preferably corresponding to the nearest distance of a set of sensing measurements used to obtain the pixel depth value. Each pixel may have a laser reflectivity value in the second channel c2, which indicates the average local reflectivity of the laser spot in the vicinity of the distance c1 from the reference plane. In the third channel c3, the pixel may have a radar reflectivity value, which indicates the average local reflectivity of the radar spot at a distance approximately c1 from the reference plane. This is shown, for example, in Figure 17 as follows. The multi - channel format allows for a large amount of data to be included in the depth map. Other possible channels that can be used are object thickness (which can be used to recover information about the surface perpendicular to the road trajectory using orthogonal projection), reflectance point density, and color and / or texture (e.g., obtained from a camera used to provide reference scan data).

[0292] Although the invention has been described with respect to embodiments where the depth map in which the reference data is located relates to the environment on the lateral side of the road, it has been recognized that depth maps with different configurations can be useful for assisting in positioning a vehicle at an intersection. These additional embodiments can be used in conjunction with side depth maps of areas away from the intersection.

[0293] In some additional embodiments, the reference line is defined as circular. In other words, the reference line is non - linear. The circle is defined by a given radius centered at the center of the digital map intersection. The radius of the circle can be selected depending on the side of the intersection. The reference plane can be defined as a 2 - D surface perpendicular to this reference line. Then, a (circular) depth map can be defined, where each pixel contains a channel indicating the distance (i.e., the depth value) from the position of the pixel in the reference plane to the surface of the object along a pre - determined direction in the same manner as when using a linear reference line. The projection onto the reference plane can similarly be orthogonal or non - orthogonal, and each pixel can have multiple channels. The depth value of a given pixel is preferably based on the nearest sensing distance to the object.

[0294] Figure 18 Indicating circular and linear reference lines, which can be used to construct depth maps at intersections and away from intersections, respectively. Figure 19A Illustrating the way an object can be projected onto a circular depth map at different angular positions. Figure 19B Indicating the use of orthogonal projection to project each of the objects onto the reference plane to provide a depth map.

[0295] It has been described how a depth map of the localization reference data (whether circular or otherwise) can be compared with real-time sensor data obtained from a vehicle to determine a longitudinal alignment offset between the reference and the real-time sensed data. In some additional embodiments, a lateral alignment offset is also obtained. This involves a series of steps that can be performed in the image domain.

[0296] Referring to an example using side depth maps, in a first step of the process, a longitudinal alignment offset between a reference-based side depth map and a side depth map based on real-time sensor data is determined in the manner previously described. The depth maps are shifted relative to each other until they are longitudinally aligned. Next, the reference depth map, which is a raster image, is cropped to correspond in size to the depth map based on real-time sensor data. Then, the depth values of the pixels in corresponding positions of the reference-based side depth map and the side depth map based on real-time sensors, i.e., the values of the depth channels of the pixels, are compared. The difference in the depth values of each pair of corresponding pixels indicates the lateral offset of the pixels. This can be evaluated by considering the color difference of the pixels, where the depth value of each pixel is represented by a color. The most common lateral offset (mode difference) determined between corresponding pixel pairs is determined and is considered to correspond to the lateral alignment offset between the two depth maps. The most common lateral offset can be obtained using a histogram of the depth differences between the pixels. Once the lateral offset is determined, it can be used to correct the perceived lateral position of the vehicle on the road.

[0297] Figure 20A Illustrate a reference depth map (i.e., an image) that can be compared to determine the lateral offset alignment of depth maps with a corresponding depth map or image based on real-time sensor data from a vehicle. As Figure 20B Illustrated, first the images are shifted relative to each other to longitudinally align them. Next, after cropping the reference image, a lateral alignment offset between the depth maps is determined using a histogram of the differences in the pixel depth values of corresponding pixels in the two depth maps - Figure 20C . Figure 20D Illustrate how this can achieve the longitudinal position and then how to correct the lateral position of the vehicle on the road.

[0298] Once the lateral alignment offset between the reference-based depth map and the real-time data-based depth map has been obtained, the forward direction of the vehicle can also be corrected. It has been found that in the case where there is an offset between the actual forward direction of the vehicle and the perceived forward direction, this will result in a non-constant lateral alignment offset being determined between corresponding pixels in the reference-based depth map and the real-time sensed data-based depth map that varies according to the longitudinal distance along the depth map.

[0299] Figure 21AA set of vertical slices showing corresponding portions through a reference depth map image (top) and a real-time sensor-based depth map image (bottom). The average difference in pixel depth values of corresponding pixels in each slice (i.e., the lateral alignment offset) is plotted against the longitudinal distance (x-axis) along the map / image (y-axis). In Figure 21B this figure is shown. Then, a function describing the relationship between the average pixel depth distance and the longitudinal distance along the depth map can be derived through a suitable regression analysis. The gradient of this function indicates the forward direction offset of the vehicle.

[0300] The depth maps used in embodiments of the present invention can be transformed so as to always be relative to a straight reference line, i.e., so as to become a linear reference image, for example, as described in WO 2009 / 045096 A1. This has the advantages as Figure 22 shown in. On the Figure 22 left side of is an image of a curved road. To mark the center line of the curved road, a number of markers 1102 must be placed. On the Figure 22 right hand side of, a corresponding linear reference image corresponding to the curved road in the left side of the figure is shown. To obtain the linear reference image, the center line of the curved road is mapped to the straight reference line of the linear reference image. In view of this transformation, the reference line can now be simply defined by two end points 1104 and 1106.

[0301] When on a perfectly straight road, the shift calculated from the comparison of the reference depth map and the real-time depth map can be directly applied. Due to the non-linear nature of the linearization process used to generate the linear reference image, it is not possible to directly apply the calculated shift on a curved road. Figure 23A and 23B show computationally efficient methods for establishing the position of a vehicle in a non-linear environment through a series of incremental independent linear update steps. As Figure 23AAs shown, the method involves applying vertical correction in a series of increasing independent linear update steps, then applying horizontal correction, and then applying forward direction correction. Specifically, in step (1), the vertical offset is determined using vehicle sensor data and a reference depth map based on the current assumed position of the vehicle relative to a digital map (e.g., obtained using GPS). Then, the vertical offset is applied to adjust the assumed position of the vehicle relative to the digital map, and the reference depth map is recalculated based on the adjusted position. Then, in step (2), the horizontal offset is determined using vehicle sensor data and the recalculated reference depth map. Then, the horizontal offset is applied to further adjust the assumed position of the vehicle relative to the digital map, and the reference depth map is recalculated again based on the adjusted position. Finally, at step (3), the forward direction offset or skew is determined using vehicle sensor data and the recalculated reference depth map. Then, the forward direction offset is applied to further adjust the assumed position of the vehicle relative to the digital map, and the reference depth map is recalculated again based on the adjusted position. These steps are repeated the number of times required for there to be substantially zero vertical offset, horizontal offset, and forward direction offset between the real-time depth map and the reference depth map. Figure 23B Show continuously and repeatedly applying vertical, horizontal, and forward direction offsets to the point cloud generated from vehicle sensor data until that point cloud is substantially aligned with the point cloud generated from the reference depth map.

[0302] Also depict a series of exemplary use cases for the positioning reference data.

[0303] For example, in some embodiments, instead of using the depth map of the positioning reference data for the purpose of comparing with the depth map based on real-time sensor data, the depth map of the positioning reference data is used to generate a reference point cloud, including a set of data points in a three-dimensional coordinate system, each point representing the surface of an object in the environment. This reference point cloud can be compared with the corresponding three-dimensional point cloud based on the real-time sensor data obtained by the vehicle sensor. The comparison can be used to determine the alignment offset between the depth maps, and thus adjust the determined position of the vehicle.

[0304] The reference depth map can be used to obtain a reference 3D point cloud, which can be compared with the corresponding point cloud based on the real-time sensor data of the vehicle (regardless of what type of sensors that vehicle has). Although the reference data can be based on sensor data obtained from various types of sensors, including lidar scanners, radar scanners, and cameras, the vehicle may not have a corresponding set of sensors. The 3D reference point cloud can be constructed from the reference depth map, which can be compared with the 3D point cloud obtained based on the specific type of real-time sensor data available for the vehicle.

[0305] For example, in a case where a depth map of reference localization data includes a channel indicating radar reflectivity, this can be taken into account when generating a reference point cloud, which can be compared with a 3D point cloud obtained using real-time sensor data of a vehicle having only a radar sensor. The radar reflectivity data associated with pixels helps identify those data points that should be included in the 3D reference point cloud, i.e., the data points represent the surfaces of objects that the vehicle radar sensor is expected to detect.

[0306] In another example, a vehicle may have only one or more cameras for providing real-time sensor data. In this case, data from the laser reflectivity channel of the reference depth map can be used to construct a 3D reference point cloud that includes data points related only to surfaces that are likely to be detected by the vehicle's cameras in the current state. For example, at night, only relatively reflective objects should be included.

[0307] A 3D point cloud based on the vehicle's real-time sensing data can be obtained as needed. In a case where the vehicle includes only a single camera as a sensor, "structure from motion" techniques can be used, where a series of images from the camera are used to reconstruct a 3D scene, and the 3D point cloud can be obtained from the 3D scene. In a case where the vehicle includes a stereo camera, a 3D scene can be directly generated and used to provide a 3D point cloud. This can be achieved using a disparity-based 3D model.

[0308] In still other embodiments, instead of comparing the reference point cloud with the real-time sensor data point cloud, the reference point cloud is used to reconstruct an image that is expected to be seen by one or more cameras of the vehicle. Then, the images can be compared and used to determine the alignment offset between the images, which can in turn be used to correct the assumed position of the vehicle.

[0309] In these embodiments, additional channels of the reference depth map can be used as described above to reconstruct an image based on including only those points in the 3D reference point cloud that are expected to be detected by the vehicle's cameras. For example, in the dark, the laser reflectivity channel can be used to select those points included in the 3D point cloud that correspond to the surfaces of objects that can be detected by the camera in the dark. It has been found that using a non-orthogonal projection onto a reference plane is particularly useful in this context when determining the reference depth map, thus preserving more information about the surfaces of objects that can still be detected in the dark.

[0310] Figure 24Depict an exemplary system according to an embodiment of the present invention, wherein data collected by one or more vehicle sensors (lasers, cameras, and radars) is used to generate an "actual occupancy area" of the environment as seen by the vehicle. The "actual occupancy area" is compared (i.e., correlated) with a corresponding "reference occupancy area" determined from reference data associated with a digital map, where the reference data includes at least one distance channel and may include a laser reflectivity channel and / or a radar reflectivity channel, as discussed above. Through this correlation, the position of the vehicle can be accurately determined relative to the digital map.

[0311] In a first example use case, as Figure 25A depicted in, the actual occupancy area is determined from a laser-based distance sensor (e.g., a LIDAR sensor) in the vehicle and correlated with a reference occupancy area determined from data in the distance channel of the reference data to enable continuous positioning of the vehicle. Figure 25B Show a first method, where a laser point cloud determined by a laser-based distance sensor is converted into a depth map in the same format as the reference data, and the two depth map images are compared. Figure 25C A second alternative method is shown in, where a laser point cloud is reconstructed from the reference data, and this reconstructed point cloud is compared with the laser point cloud as seen by the vehicle.

[0312] In a second example use case, as Figure 26A depicted in, the actual occupancy area is determined from a camera in the vehicle and correlated with a reference occupancy area determined from data in the distance channel of the reference data to enable continuous positioning of the vehicle, although only during the day. In other words, in this example use case, a reference depth map is used to construct a 3D point cloud or view, which is then compared with a 3D scene or view obtained from multiple vehicle cameras or a single vehicle camera. In Figure 26B a first method is shown in, where a stereo vehicle camera is used to establish a disparity-based 3D model, which is then used to construct a 3D point cloud for correlation with the 3D point cloud constructed from the reference depth map. In Figure 26C a second method is shown in, where a sequence of vehicle camera images is used to construct a 3D scene, and the 3D scene is then used to construct a 3D point cloud for correlation with the 3D point cloud constructed from the reference depth map. Finally, in Figure 26D a third method is shown in, where vehicle camera images are compared with a view created from a 3D point cloud constructed from the reference depth map.

[0313] In a third example use case, as Figure 27AAs shown, there is a modification to the use case of the second instance, where the laser reflectivity data of the reference data located in the channels of the depth map can be used to construct a 3D point cloud or view, which can be compared with the 3D point cloud or view based on the images captured by one or more cameras. In Figure 27B The first method is shown, where a sequence of vehicle camera images is used to construct a 3D scene. Then, a 3D point cloud is constructed using the 3D scene to be correlated with the 3D point cloud constructed from the reference depth map (using both the distance and laser reflectivity channels). In Figure 27C The second method is shown, where the vehicle camera images are compared with the view created based on the 3D point cloud constructed from the reference depth map (again using both the distance and laser reflectivity channels).

[0314] In the use case of the fourth instance, as Figure 28A depicted, the actual occupied area is determined from the radar-based distance sensor in the vehicle and correlated with the reference occupied area determined from the data in the distance and radar reflectivity channels of the reference data, in order to achieve sparse localization of the vehicle. In Figure 28B The first method is shown, where the reference data is used to reconstruct a 3D scene, and the data in the radar reflectivity channel is used to leave only the radar reflection points. Then, this 3D scene is correlated with the radar point cloud as seen by the vehicle.

[0315] Of course, it should be understood that various use cases, i.e., fusions, can be used together to allow for more precise localization of the vehicle relative to the digital map.

[0316] A method for correlating vehicle sensor data with reference data to determine the position of the vehicle, for example, as discussed above, will now be described with reference to Figures 29 to 32B as follows. Figure 29 Depicts various coordinate systems used in the method: the local coordinate system (local CS); the vehicle frame coordinate system (CF CS); and the linear reference coordinate system (LRCS) along the vehicle trajectory. Another coordinate system, although not depicted, is the World Geodetic System (WGS), where, as is known in the art, the position is given as a pair of latitude and longitude coordinates. In Figure 30 A general method is shown, where the details of the steps are carried out to determine Figure 31 the laser point cloud shown in Figure 32A Shows the execution of Figure 30 the first exemplary method of the relevant steps, where the position of the vehicle is corrected by image correlation between, for example, the depth map raster image of the reference data and the corresponding depth map raster image created from the vehicle sensor data. Figure 32B Shows the execution of Figure 30A second exemplary method of the relevant steps, wherein the position of the vehicle is corrected by 3D correlation between a 3D scene constructed from reference data and a 3D scene captured by vehicle sensors.

[0317] Any method according to the present invention may be implemented at least in part using software (e.g., a computer program). Accordingly, the present invention also extends to a computer program comprising computer-readable instructions executable to perform or cause a navigation device to perform a method according to any aspect or embodiment of the present invention. Accordingly, the present invention encompasses a computer program product which, when executed by one or more processors, causes the one or more processors to generate a suitable image (or other graphical information) for display on a display screen. The present invention correspondingly extends to a computer software carrier comprising such software which, when used to operate a system or device comprising data processing means, together with the data processing means causes the device or system to perform the steps of the method of the present invention. Such a computer software carrier may be a non-transitory physical storage medium, such as a ROM chip, a CDROM or a disk; or it may be a signal, such as an electrical signal via a wire, an optical signal or a radio signal (e.g., to a satellite) or the like. The present invention provides a machine-readable medium containing instructions which, when read by a machine, cause the machine to operate according to a method according to any aspect or embodiment of the present invention.

[0318] Without explicit statement, it should be understood that the present invention in any of its aspects may include any or all of the features described in connection with other aspects or embodiments of the present invention, provided they are not mutually exclusive. In particular, although various embodiments of operations that may be performed in the method and by the device have been described, it should be understood that any one or more or all of these operations may be performed in the method and by the device in any combination as required.

[0319] The following are certain examples of the present disclosure.

[0320] According to an example of the present disclosure, there is provided a method of generating positioning reference data associated with a digital map, the positioning reference data providing a compressed representation of the environment around at least one navigable element of a navigable network represented by the digital map, the method comprising:

[0321] For at least one navigable element represented by the digital map, obtaining a set of data points in a three-dimensional coordinate system, wherein each data point represents the surface of an object in the environment around the at least one navigable element of the navigable network;

[0322] Generate positioning reference data from the set of data points, the positioning reference data including at least one depth map indicative of the environment around the navigable element projected onto a reference plane, the reference plane being defined by a reference line associated with the navigable element, each pixel in the at least one depth map being associated with a position in the reference plane associated with the navigable element, and the pixel including a depth channel representing a distance from the associated position of the pixel in the reference plane to the surface of an object in the environment along a pre-determined direction, wherein the distance to the surface of the object represented by the depth channel of each pixel is determined based on a set of a plurality of sensed data points, each sensed data point indicative of a sensed distance from the position of the pixel along the pre-determined direction to the surface of the object, and wherein the distance to the surface of the object represented by the depth channel of the pixel is based on the closest distance, or closest mode distance, of the set of sensed data points; and

[0323] Associate the generated positioning reference data with the digital map data.

[0324] At least some of the sensed data points of the set of a plurality of sensed data points for a particular pixel can be associated with the surfaces of different objects.

[0325] The different objects can be located at different depths relative to the reference plane.

[0326] The distance to the surface of the object represented by the depth channel of a particular pixel can not be based on the average of the set of a plurality of sensed data points for the particular pixel.

[0327] According to another example of the present disclosure, there is provided a method of generating positioning reference data associated with a digital map, the positioning reference data providing a compressed representation of the environment around at least one navigable element of a navigable network represented by the digital map, the method comprising:

[0328] For at least one navigable element represented by the digital map, obtain a set of data points in a three-dimensional coordinate system, wherein each data point represents the surface of an object in the environment around the at least one navigable element of the navigable network;

[0329] Generate positioning reference data from the set of data points, the positioning reference data including at least one depth map indicative of the environment around the navigable element projected onto a reference plane, the reference plane being defined by a longitudinal reference line parallel to the navigable element and perpendicular to the surface of the navigable element, each pixel in the at least one depth map being associated with a position in the reference plane associated with the navigable element, and the pixel including a depth channel representing a lateral distance along a predetermined direction from the associated position of the pixel in the reference plane to the surface of an object in the environment, wherein the at least one depth map has a fixed longitudinal resolution and a variable vertical and / or depth resolution; and

[0330] Associate the generated positioning reference data with the digital map data.

[0331] The variable vertical and / or depth resolution may be non-linear.

[0332] The portion of the depth map closer to the ground may be presented at a higher resolution than the portion of the depth map above the ground.

[0333] The portion of the depth map closer to the navigable element may be presented at a higher resolution than the portion of the depth map further from the navigable element.

[0334] According to another example of the present disclosure, there is provided a method of generating positioning reference data associated with a digital map, the positioning reference data providing a compressed representation of the environment around at least one navigable element of a navigable network represented by the digital map, the method comprising:

[0335] For at least one navigable element represented by the digital map, obtain a set of data points in a three-dimensional coordinate system, wherein each data point represents the surface of an object in the environment around the at least one navigable element of the navigable network;

[0336] Generate positioning reference data from the set of data points, the positioning reference data including at least one depth map indicative of the environment around the navigable element projected onto a reference plane, the reference plane being defined by a reference line parallel to the navigable element, each pixel in the at least one depth map being associated with a position in the reference plane associated with the navigable element, and the pixel including a depth channel representing a distance along a predetermined direction from the associated position of the pixel in the reference plane to the surface of an object in the environment, wherein the predetermined direction is not perpendicular to the reference plane; and

[0337] Associate the generated positioning reference data with the digital map data indicating the navigable element.

[0338] The projection of the environment onto the reference plane may be a non-orthogonal projection.

[0339] The predetermined direction may be along a direction that is substantially 45 degrees relative to the reference plane.

[0340] The navigable element may include a road, and the navigable network includes a road network.

[0341] Positioning reference data for a plurality of navigable elements of the navigable network represented by the digital map may be generated.

[0342] The reference line associated with the navigable element may be defined by one or more points associated with the navigable element.

[0343] The reference line associated with the navigable element is an edge, boundary, lane, or centerline of the navigable element.

[0344] The positioning reference data may provide a representation of the environment on one or more sides of the navigable element.

[0345] The depth map may be in the form of a raster image.

[0346] Each pixel of the depth map may be associated with a specific longitudinal position and elevation in the depth map.

[0347] Associating the generated positioning reference data with the digital map data may include storing the positioning reference data in association with the navigable element to which it pertains.

[0348] The positioning reference data may include a representation of the environment on the left side and the right side of the navigable element.

[0349] The positioning reference data for each side of the navigable element may be stored in a combined dataset.

[0350] According to another example of the present disclosure, there is provided a method of generating positioning reference data associated with a digital map representing an element of a navigable network, the positioning reference data providing a compressed representation of the environment around at least one junction of the navigable network represented by the digital map, the method comprising:

[0351] For at least one junction represented by the digital map, obtain a set of data points in a three-dimensional coordinate system, where each data point represents the surface of an object in the environment around the at least one junction of the navigable network;

[0352] From the set of data points, generate positioning reference data, the positioning reference data including at least one depth map indicating the environment around the junction projected onto a reference plane, the reference plane being defined by a reference line bounded by a radius centered on a reference point associated with the junction, each pixel in the at least one depth map being associated with a position in the reference plane associated with the junction, and the pixel including a depth channel representing the distance from the associated position of the pixel in the reference plane to the surface of an object in the environment along a predetermined direction; and

[0353] Associate the generated positioning reference data with digital map data indicating the junction.

[0354] The depth map may extend approximately 360 degrees to provide a 360-degree representation of the environment around the junction.

[0355] The depth map may extend less than approximately 360 degrees.

[0356] The reference point may be located at the center of the junction.

[0357] The reference point may be associated with a node of the digital map representing the junction or a navigable element at the junction.

[0358] The junction may be an intersection.

[0359] The set of data points may be obtained using at least one rangefinder sensor on a mobile mapping vehicle that has previously traveled along the at least one navigable element.

[0360] The at least one rangefinder sensor may include one or more of the following: a laser scanner; a radar scanner; and a pair of stereo cameras.

[0361] According to another example of the present disclosure, provided is a method for determining the position of a vehicle relative to a digital map, the digital map including data representing junctions through which the vehicle travels, the method including:

[0362] Obtain positioning reference data associated with the digital map for a perceived current position of the vehicle in the navigable network, wherein the positioning reference data includes at least one depth map indicative of the environment around the vehicle projected onto a reference plane, the reference plane being defined by a reference line bounded by a radius centered on a reference point associated with the junction point, each pixel in the at least one depth map being associated with a position in the reference plane associated with the junction point through which the vehicle travels, and the pixel including a depth channel representing the distance along a pre-determined direction from the associated position of the pixel in the reference plane to the surface of an object in the environment;

[0363] Determine real-time scan data by scanning the environment around the vehicle using at least one sensor, wherein the real-time scan data includes at least one depth map indicative of the environment around the vehicle, each pixel in the at least one depth map being associated with a position in the reference plane associated with the junction point, and the pixel including a depth channel representing the distance along the pre-determined direction from the associated position of the pixel in the reference plane to the surface of an object in the environment as determined using the at least one sensor;

[0364] Calculate a correlation between the positioning reference data and the real-time scan data to determine an alignment offset between the depth maps; and

[0365] Adjust the perceived current position using the determined alignment offset to determine the position of the vehicle relative to the digital map.

[0366] According to another example of the present disclosure, there is provided a computer program product including computer-readable instructions executable to cause a system to perform the method as described above, optionally stored on a non-transitory computer-readable medium.

[0367] According to another example of the present disclosure, there is provided a system for generating positioning reference data associated with a digital map, the positioning reference data providing a compressed representation of the environment around at least one navigable element of a navigable network represented by the digital map, the system including processing circuitry configured to perform the following operations for at least one navigable element represented by the digital map:

[0368] Obtain a set of data points in a three-dimensional coordinate system, wherein each data point represents the surface of an object in the environment around the at least one navigable element of the navigable network;

[0369] Generate positioning reference data from the set of data points, the positioning reference data including at least one depth map indicative of the environment around the navigable element projected onto a reference plane, the reference plane being defined by a reference line associated with the navigable element, each pixel of the at least one depth map being associated with a position in the reference plane associated with the navigable element, and the pixel including a depth channel representing a distance along a predetermined direction from the associated position of the pixel in the reference plane to the surface of an object in the environment, wherein the distance to the surface of the object represented by the depth channel of each pixel is determined based on a set of a plurality of sensed data points, each sensed data point indicative of a sensed distance along the predetermined direction from the position of the pixel to the surface of the object, and wherein the distance to the surface of the object represented by the depth channel of the pixel is based on the closest distance or the closest mode distance of the set of sensed data points; and

[0370] Associate the generated positioning reference data with the digital map data.

[0371] According to another example of the present disclosure, there is provided a system for generating positioning reference data associated with a digital map, the positioning reference data providing a compressed representation of the environment around at least one navigable element of a navigable network represented by the digital map, the system including processing circuitry configured to perform the following operations for at least one navigable element represented by the digital map:

[0372] Obtain a set of data points in a three-dimensional coordinate system, wherein each data point represents the surface of an object in the environment around the at least one navigable element of the navigable network;

[0373] Generate positioning reference data from the set of data points, the positioning reference data including at least one depth map indicative of the environment around the navigable element projected onto a reference plane, the reference plane being defined by a longitudinal reference line parallel to and perpendicularly oriented to the surface of the navigable element, each pixel of the at least one depth map being associated with a position in the reference plane associated with the navigable element, and the pixel including a depth channel representing a lateral distance along a predetermined direction from the associated position of the pixel in the reference plane to the surface of an object in the environment, wherein the at least one depth map has a fixed longitudinal resolution and a variable vertical and / or depth resolution; and

[0374] Associate the generated positioning reference data with the digital map data.

[0375] According to another example of the present disclosure, there is provided a system for generating location reference data associated with a digital map, the location reference data providing a compressed representation of an environment around at least one navigable element of a navigable network represented by the digital map. The system includes processing circuitry configured to perform the following operations for at least one navigable element represented by the digital map:

[0376] Obtain a set of data points in a three-dimensional coordinate system, where each data point represents the surface of an object in the environment around the at least one navigable element of the navigable network;

[0377] Generate, from the set of data points, location reference data including at least one depth map indicating the environment around the navigable element projected onto a reference plane, the reference plane being defined by a reference line parallel to the navigable element. Each pixel in the at least one depth map is associated with a position in the reference plane associated with the navigable element, and the pixel includes a depth channel representing the distance along a predetermined direction from the associated position of the pixel in the reference plane to the surface of an object in the environment, where the predetermined direction is not perpendicular to the reference plane; and

[0378] Associate the generated location reference data with digital map data indicating the navigable element.

[0379] According to another example of the present disclosure, there is provided a system for generating location reference data associated with a digital map representing an element of a navigable network, the location reference data providing a compressed representation of an environment around at least one junction point of the navigable network represented by the digital map. The system includes processing circuitry configured to perform the following operations for at least one junction point represented by the digital map:

[0380] Obtain a set of data points in a three-dimensional coordinate system, where each data point represents the surface of an object in the environment around the at least one junction point of the navigable network;

[0381] Generate, from the set of data points, location reference data including at least one depth map indicating the environment around the junction point projected onto a reference plane, the reference plane being defined by a reference line bounded by a radius centered at a reference point associated with the junction point. Each pixel in the at least one depth map is associated with a position in the reference plane associated with the junction point, and the pixel includes a depth channel representing the distance along a predetermined direction from the associated position of the pixel in the reference plane to the surface of an object in the environment; and

[0382] Associate the generated positioning reference data with digital map data indicating the junction points.

[0383] According to another example of the present disclosure, there is provided a system for determining the position of a vehicle relative to a digital map, the digital map including data representing junction points through which the vehicle travels, the system including a processing circuit configured to:

[0384] Obtain positioning reference data associated with the digital map for a presumed current position of the vehicle in the navigable network, wherein the positioning reference data includes at least one depth map indicating the environment around the vehicle projected onto a reference plane, the reference plane being defined by a reference line bounded by a radius centered on a reference point associated with the junction point, each pixel in the at least one depth map being associated with a position in the reference plane associated with the junction point through which the vehicle travels, and the pixel including a depth channel representing the distance from the associated position of the pixel in the reference plane to the surface of an object in the environment along a pre-determined direction;

[0385] Determine real-time scan data by scanning the environment around the vehicle using at least one sensor, wherein the real-time scan data includes at least one depth map indicating the environment around the vehicle, each pixel in the at least one depth map being associated with a position in the reference plane associated with the junction point, and the pixel including a depth channel representing the distance from the associated position of the pixel in the reference plane to the surface of an object in the environment along the pre-determined direction determined using the at least one sensor;

[0386] Calculate the correlation between the positioning reference data and the real-time scan data to determine an alignment offset between the depth maps; and

[0387] Use the determined alignment offset to adjust the presumed current position to determine the position of the vehicle relative to the digital map.

Claims

1. A method for generating positioning reference data associated with a digital map representing elements of a navigable network, the positioning reference data providing a compressed representation of the environment around at least one junction of the navigable network represented by the digital map, the method comprising: For at least one junction represented by the digital map, obtaining a set of data points in a three-dimensional coordinate system, wherein each data point represents the surface of an object in the environment around the at least one junction of the navigable network; Generating positioning reference data from the set of data points, the positioning reference data including at least one depth map indicating the environment around the junction projected onto a reference plane, the reference plane being defined by a reference line bounded by a radius centered on a reference point associated with the junction, each pixel in the at least one depth map being associated with a position in the reference plane associated with the junction, and the pixel including a depth channel representing the distance from the associated position of the pixel in the reference plane to the surface of an object in the environment along a predetermined direction; and Associating the generated positioning reference data with digital map data indicating the junction.

2. The method according to claim 1, wherein the depth map extends 360 degrees to provide a 360-degree representation of the environment around the junction.

3. The method according to claim 1, wherein the depth map extends less than 360 degrees.

4. The method according to any one of claims 1 to 3, wherein the reference point is located at the center of the junction.

5. The method according to any one of claims 1 to 3, wherein the reference point is associated with a node of the digital map representing the junction or a navigable element at the junction.

6. The method according to any one of claims 1 to 3, wherein the junction is an intersection.

7. The method according to any one of claims 1 to 3, wherein the set of data points is obtained using at least one rangefinder sensor on a mobile mapping vehicle that has previously traveled along the at least one navigable element.

8. The method according to claim 7, wherein the at least one rangefinder sensor includes one or more of the following: a laser scanner; a radar scanner; and a pair of stereo cameras.

9. The method according to any one of claims 1 to 3, wherein the reference line is defined as a circle, and the circle is bounded by the radius.

10. The method according to any one of claims 1 to 3, wherein: The reference point is defined relative to the junction, and / or The reference point is defined by the center of the junction.

11. The method according to any one of claims 1 to 3, wherein the depth map includes a depth map image and / or a raster image.

12. A method for determining the position of a vehicle relative to a digital map, the digital map including data representing junctions through which the vehicle travels, the method comprising: Obtain positioning reference data associated with the digital map for a supposed current position of the vehicle in a navigable network, wherein the positioning reference data includes at least one depth map indicating the environment around the vehicle projected onto a reference plane, the reference plane being defined by a reference line bounded by a radius centered on a reference point associated with the junction point, each pixel in the at least one depth map being associated with a position in the reference plane associated with the junction point through which the vehicle travels, and the pixel containing a depth channel representing the distance along a predetermined direction from the associated position of the pixel in the reference plane to the surface of an object in the environment; Determine real-time scan data by scanning the environment around the vehicle using at least one sensor, wherein the real-time scan data includes at least one depth map indicating the environment around the vehicle, each pixel in the at least one depth map being associated with a position in the reference plane associated with the junction point, and the pixel containing a depth channel representing the distance along the predetermined direction determined using the at least one sensor from the associated position of the pixel in the reference plane to the surface of an object in the environment; Calculate a correlation between the positioning reference data and the real-time scan data to determine an alignment offset between the depth maps; and Adjust the supposed current position using the determined alignment offset to determine the position of the vehicle relative to the digital map.

13. The method according to claim 12, wherein the reference line is defined as circular, the circle being bounded by the radius.

14. The method according to claim 12 or 13, wherein: the reference point is defined relative to the junction point, and / or the reference point is defined by the center of the junction point.

15. The method according to claim 12 or 13, wherein the depth map includes a depth map image and / or a raster image.

16. A computer program comprising computer-readable instructions executable to cause a system to perform the method according to any one of the preceding claims, optionally stored on a non-transitory computer-readable medium.

17. A system for generating positioning reference data associated with a digital map representing elements of a navigable network, the positioning reference data providing a compressed representation of the environment around at least one junction point of the navigable network represented by the digital map, the system including processing circuitry configured to perform the following operations for at least one junction point represented by the digital map: Obtain a set of data points in a three-dimensional coordinate system, wherein each data point represents the surface of an object in the environment around the at least one junction point of the navigable network; Generate positioning reference data from the set of data points, the positioning reference data including at least one depth map indicative of the environment around the junction point projected onto a reference plane, the reference plane being defined by a reference line bounded by a radius centered on a reference point associated with the junction point, each pixel in the at least one depth map being associated with a position in the reference plane associated with the junction point, and the pixel including a depth channel representing the distance along a predetermined direction from the associated position of the pixel in the reference plane to the surface of an object in the environment; and Associate the generated positioning reference data with digital map data indicative of the junction point.

18. A system for determining the position of a vehicle relative to a digital map, the digital map including data representing junction points through which the vehicle travels, the system including processing circuitry configured to: Obtain positioning reference data associated with the digital map for a supposed current position of the vehicle in a navigable network, wherein the positioning reference data includes at least one depth map indicative of the environment around the vehicle projected onto a reference plane, the reference plane being defined by a reference line bounded by a radius centered on a reference point associated with the junction point, each pixel in the at least one depth map being associated with a position in the reference plane associated with the junction point through which the vehicle travels, and the pixel including a depth channel representing the distance along a predetermined direction from the associated position of the pixel in the reference plane to the surface of an object in the environment; Determine real-time scan data by scanning the environment around the vehicle using at least one sensor, wherein the real-time scan data includes at least one depth map indicative of the environment around the vehicle, each pixel in the at least one depth map being associated with a position in the reference plane associated with the junction point, and the pixel including a depth channel representing the distance along the predetermined direction from the associated position of the pixel in the reference plane to the surface of an object in the environment determined using the at least one sensor; Calculate a correlation between the positioning reference data and the real-time scan data to determine an alignment offset between the depth maps; and Adjust the supposed current position using the determined alignment offset to determine the position of the vehicle relative to the digital map.

Images