Positioning of the device using multilateration

Through the terminal user device collecting depth maps and using multilateral measurement calculations, combined with position reference information in the fingerprint database, the problem of insufficient vehicle positioning accuracy in the prior art is solved, higher precision vehicle positioning is achieved, and calculation costs are reduced.

CN113029125BActive Publication Date: 2024-11-19HERE GLOBAL BV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110240691.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2014-12-26
Filing Date
2015-12-21
Publication Date
2024-11-19
Estimated Expiration
2035-12-21

AI Technical Summary

Technical Problem

Existing GPS, WiFi and Bluetooth technologies have problems with insufficient accuracy in vehicle positioning, especially in the case of multi-path interference and occlusion in three-dimensional space, resulting in inaccurate lane or road horizontal positioning.

Method used

The end user device collects depth maps in the path network, extracts two-dimensional feature geometry, recognizes control points, calculates the distance from the control points, and uses multilateral measurement calculation to determine the geographical location, and combines the position reference information in the fingerprint database for precise positioning. .

Benefits of technology

Improves the accuracy of vehicle positioning, can position in the absence of conventional geolocation technology, and reduces computational costs, and is suitable for more and more vehicles with built-in complex inspection technologies.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113029125B_ABST
    Figure CN113029125B_ABST
Patent Text Reader

Abstract

A method for positioning a device using multilateral measurement includes: collecting a depth map at a location in a path network by a terminal user device (s101); extracting two-dimensional feature geometry from the depth map using a processor of the terminal user device (s103); identifying a number of control points in the extracted feature geometry (s105); calculating the distance between the terminal user device and the identified control points (s107); receiving location reference information of each identified control point from an external database (s109); and determining the geographic location of the terminal user device in the path network via multilateral measurement calculations using the location reference information of the identified control points and the calculated distance between the terminal user device and each identified control point (s111).
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The following disclosure relates to developing a fingerprint database and using multilateration to determine the geographic location of an end-user device (e.g., a vehicle, mobile phone, smart watch, etc.). Background Art

[0002] Vehicle positioning using the Global Positioning System (GPS), local area wireless technologies (e.g., WiFi), and short wavelength radio waves (e.g., Bluetooth) can be imprecise. In the case of GPS, multipath causes timing variations in the signals received by the vehicle. In the case of WiFi and Bluetooth, signal strength is an unreliable means for positioning, primarily due to obstruction and lack of precision in the location of the transmitter in three-dimensional (3D) space. In these cases, the reference against which multilateration is performed is not accurate enough to produce a lane level (or in some cases, road level) positioning. Summary of the invention

[0003] The present invention provides systems, devices and methods for developing a fingerprint database and determining the geographic location of a terminal user device using multilateration calculations. In one embodiment, the method includes collecting a depth map at a location in a path network by a terminal user device. The method further includes extracting a two-dimensional feature geometry from the depth map using a processor of the terminal user device. The method further includes identifying a number of control points in the extracted feature geometry. The method further includes calculating the distance between the terminal user device and the identified control points. The method further includes receiving location reference information for each identified control point from an external database. The method further includes determining the geographic location of the terminal user device in the path network via multilateration calculations using the location reference information of the identified control points and the calculated distance between the terminal user device and each identified control point.

[0004] In another embodiment, the method includes receiving an extracted two-dimensional feature geometry of a depth map of a location in a path network from an end-user device. The method further includes matching the extracted feature geometry with a plurality of feature geometries for the path network in a database using a processor. The method further includes retrieving location reference information of points of the matched feature geometry from the database. The method further includes transmitting the location reference information to the end-user device. In yet another embodiment, the apparatus includes at least one processor and at least one memory containing computer program code for one or more programs; the at least one memory and the computer program code are configured to cause the apparatus to perform at least the following operations using the at least one processor: (1) collect a depth map at a location in the path network; (2) extract a two-dimensional feature geometry from the depth map; (3) identify a plurality of control points in the extracted feature geometry; (4) calculate a distance between the end-user device and the identified control points; (5) receive location reference information for each identified control point from an external database; and (6) determine a geographic location of the end-user device in the path network via multilateration calculations using the location reference information of the identified control points and the calculated distance between the end-user device and each identified control point. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Exemplary embodiments are described herein with reference to the following drawings.

[0006] Figure 1 An example of a depth map image is illustrated having extracted horizontal slices at multiple elevations and two-dimensional images identified from the extracted slices.

[0007] Figure 2 An example of an encoded fingerprint in a fingerprint database is illustrated.

[0008] Figure 3 illustrate Figure 2 Examples of line feature geometries of encoded fingerprints.

[0009] Figure 4 illustrate Figure 2 An example of arc feature geometry of an encoded fingerprint.

[0010] Figure 5 illustrate Figure 2 Example of spline feature geometry of an encoded fingerprint.

[0011] Figure 6 An example of a vehicle is described in which feature geometry surrounding the vehicle is identified and extracted to determine a geographic location of the vehicle.

[0012] Fig. 7A and 7BAdditional examples of devices are described in which characteristic geometries and control points surrounding the device are identified and extracted to determine a geographic location of the device.

[0013] Figure 8 Another example of the multilateral measurement process based on three control points is described.

[0014] Fig. 9 An example flow diagram for determining a location of a device via multilateration techniques is illustrated.

[0015] Fig.10 Additional example flow diagrams for determining a location of a device are illustrated.

[0016] Fig.11 A diagram of an exemplary geographic and / or navigational data system.

[0017] Fig.12 illustrate Fig.11 Embodiments of components of a device used in an exemplary system. DETAILED DESCRIPTION

[0018] The position or location of an end-user device may be determined by deriving coordinates from two-dimensional (2D) feature geometries representing real objects in a path (e.g., road) network. An end-user device may refer to a device operated or used by a consumer. As used herein, non-limiting examples of end-user devices include vehicles (e.g., cars, trucks, buses, trains, motorcycles, boats, ships), and portable electronic devices such as mobile phones, laptops with wireless capabilities, video recording devices, scanning devices, messaging devices, personal digital assistants, and wearable computers (e.g., smart watches). Feature geometries may refer to two-dimensional shapes such as lines, arcs, and splines (e.g., B-splines) extracted from three-dimensional structures around the device. For example, 2D feature geometries may be extracted from three-dimensional structures such as building facades, road signs, polls, plants (e.g., trees), or other non-temporary structures present in a path network. Multiple feature geometries at a particular road segment may be combined to provide one or more unique fingerprints that identify a particular road segment from a surrounding area.

[0019] In other words, the location of a device (e.g., a vehicle) may be determined based on identification of fingerprints of various features surrounding the device. The location may be determined in real time, where features identified by the device are compared to a database of feature geometries / fingerprints. The accuracy of the determined location may be based on the location of the device in the path network and the method by which the device is located.

[0020] This positioning process may allow for improved positioning of the device relative to conventional geo-positioning techniques such as GPS, WiFi, or Bluetooth. Furthermore, feature-based visual positioning techniques may allow for positioning of the device in the absence of any available conventional geo-positioning techniques (e.g., GPS). Furthermore, feature-based positioning techniques using simple 2D feature geometry may reduce the overall computational cost associated with determining the position of the device in real-time. This eliminates the need for expensive graphics processing units (GPUs) to process images in real-time and allows vehicles utilizing low-cost visualization techniques to utilize the proposed feature-based positioning techniques described herein. Furthermore, more and more vehicles are having sophisticated viewing technologies built in and may utilize the proposed geo-referencing process.

[0021] As discussed herein, a database of two-dimensional feature geometries may be developed via a collection of depth maps generated by a depth sensing device (e.g., a high-precision light detection and ranging (LIDAR) device). Feature geometries, such as buildings, traffic lights, stop signs, vegetation (e.g., trees), and road properties (e.g., width, number of lanes) may be extracted from the collected depth maps. The extracted features may be represented or encoded as fingerprints of specific locations in a separate database. An end-user device (e.g., a vehicle) may determine its location based on the device's own recognition of the feature geometries and comparison to a fingerprint database of feature geometries, where the location is determined based on a match of the features or fingerprints.

[0022] Collection of Data

[0023] In some embodiments, a database containing various characteristic geometries of a path network may be developed through the collection and analysis of data of the path network. The path network may be a road network including a selected number of road segments within a metropolitan area or city. In some embodiments, a database of multiple road segments in multiple metropolitan areas or cities may be developed.

[0024] As used herein, a "road" or "road segment" may refer to any lane of travel or path that can or may become capable of monitoring in the future (e.g., highways, city streets, bus routes, train routes, walking / biking paths, waterways).

[0025] In certain embodiments, a database is developed through the collection of: (1) three-dimensional data about selected road segments within a road network, and (2) location data (e.g., GPS data) associated with the 3D data. The 3D data may be a depth map or point cloud data acquired using a 3D optical ranging system or an intensity-based scanning technique. In certain embodiments, the depth map or point cloud data is collected using a depth sensing device. The depth sensing device may be any depth sensing stereo camera (e.g., a LIDAR camera), a radio detection and ranging (RADAR) device, an ultrasonic device, or a structured light camera. The depth sensing device may include a mobile single camera configured to recognize three-dimensional structure from motion.

[0026] LIDAR (also referred to as LiDAR, Lidar or other similar expressions) may also be referred to as a three-dimensional laser scanning or optical ranging system that employs one or more lasers or cameras to collect data points representing an area (e.g., an area regarding a road or sidewalk).

[0027] LIDAR cameras in data acquisition systems can use ultraviolet, visible or near-infrared light to image objects, targeting a wide range of materials such as non-metallic objects, stones, rain, chemical compounds, aerosols, clouds and even single molecules. Narrow laser beams can map physical features with high resolution.

[0028] The software can generate a depth map or point cloud based on the measured distance, the position of an optical ranging system that may be on a mobile platform such as a car, and the angle of the laser. Other optical ranging systems include stereo cameras, time-of-flight infrared cameras, and structured light devices. LIDAR cameras collect and aggregate data points in a point cloud, where each data point corresponds to a local coordinate, such as (x, y, z), and is either on or off. Alternatively, the LIDAR data may be a grayscale point cloud that includes the intensity (indicating reflectivity) of each data point in a predetermined range of values ​​(e.g., 0 to 255, 0 to 65536), with black at one end of the range and white at the other end. The point cloud may be stored in ASCII or a LIDAR exchange format (e.g., the American Society for Photogrammetry and Remote Sensing (ASPRS). LAS file format). The one or more lasers may be in the near infrared spectrum (e.g., about 700 nm to about 5000 nm or about 800 nm to about 2500 nm) or other spectrums.

[0029] In certain embodiments, the LIDAR camera may be attached or otherwise integrated with a data collection device (e.g., a vehicle). There may also be a positioning system, such as GPS, integrated with the LIDAR vehicle system to provide a base reference for the data acquired by the LIDAR system. The mobility of the vehicle system may facilitate the acquisition of data across a large geographic area using an established location for reference, and the integration of the acquired data into a point cloud representing the geographic area.

[0030] In the development of the fingerprint database, the LIDAR data acquisition system can be combined with a geographic positioning system (GPS) to provide a base reference for the data acquired by the LIDAR system. Through the combination of the LIDAR system and GPS, the collected 3D data can be saved or encoded into the database along with the latitude, longitude, altitude and elevation measurements.

[0031] The mobility of the vehicle system can facilitate the acquisition of data across large geographic areas using the established locations for reference, and the integration of the acquired data into a point cloud representing the geographic area. In some embodiments, the depth map or point cloud data can be collected continuously as the data collection vehicle travels along selected road segments within the path network. In other embodiments, the data can be collected at predefined intervals (e.g., every 10 to 30 meters) along the road segment.

[0032] In certain embodiments, data from one or more data collection vehicles may be uploaded to a map developer database for future analysis. In some embodiments, the upload process may occur after the data collection vehicle has completed its collection of depth map or point cloud data. In other embodiments, the upload process may include transmitting the data to the map developer database via a connected network as the data is collected.

[0033] Feature extraction

[0034] After the collection of the depth map or point cloud data and upload to the map developer database, the data can be analyzed to obtain various feature geometries within the depth map or point cloud data. In other words, a computer algorithm can be run to search for specific two-dimensional features in the depth map data and extract those features. The two-dimensional feature geometries may include lines, collections of connected lines or curves, arcs, and splines within the physical structures of the depth map. In certain embodiments, the physical structures include building facades, road signs, street lights, and plants (e.g., trees) of the point cloud data.

[0035] Two-dimensional feature geometry can be extracted from a depth map by identifying the physical structure of the depth map and dividing or cutting the depth map into one or more horizontal planes at different elevations of the physical structure. The depth map data within each horizontal slice / segment can then be analyzed to obtain a specific geometry. During the dividing or cutting process, the depth map data is extracted from the data existing in the range above and below the defined elevation and the defined elevation. For example, the extraction includes the aggregation of data within 0.1 meters, 0.2 meters, 0.5 meters, 1 meter, or 2 meters of the defined elevation. In other words, the horizontal slice or segment has a thickness of less than 0.1 meters, less than 0.2 meters, less than 0.5 meters, less than 1 meter, less than 2 meters, 0.1-2 meters, 0.1-1 meters, 0.1-0.5 meters, or 0.1-0.2 meters. In some embodiments, the depth map data in the range above and below the defined elevation is merged or averaged together.

[0036] The analysis of the depth map data may be a dynamic or iterative process. In some embodiments, more than one horizontal plane is analyzed in order to find important or useful feature geometries within the depth map. For example, in some embodiments, a first elevation is selected and the depth map is cut to the horizontal plane at the first elevation. This cutting process and analysis alleviates the need to perform complex three-dimensional feature recognition or classification for the entire building or structure being viewed.

[0037] The horizontal plane is analyzed for important feature geometries, such as straight lines, sets of connected lines or curves, arcs, or splines. In some embodiments, the horizontal plane is analyzed for important feature geometries, such as sets of connected lines. For example, two-dimensional shapes may be included in the set of connected lines, including quadrilaterals such as squares and rectangles; or other polygons such as triangles, pentagons, hexagons, heptagons, octagons, nonagons, decagons, etc. In other embodiments, the horizontal plane may be analyzed for important connected curve shape geometries such as circles or ellipses. This cutting process and analysis alleviates the need to perform complex three-dimensional feature recognition or classification for the entire building or structure being viewed.

[0038] In the event that no useful features are identified in the extracted horizontal plane at the first elevation, a second elevation may be selected where data is extracted and analyzed at the second elevation. The process may be repeated at multiple elevations until at least one important or useful feature geometry is identified. In some embodiments, the process begins at a higher elevation from the road level and proceeds downward toward the road in elevation increments (e.g., every 5 meters). The starting elevation may be determined based on provided mapping data for a particular road segment area, where the approximate heights of buildings along the road segment are known. The mapping data may be provided to the map database from a separate source and is not necessarily compiled simultaneously with the data collection vehicle compiling and reporting the LIDAR data to the database.

[0039] In some embodiments, extraction and analysis may be performed at several predefined elevations (e.g., every 5 meters from the road level). The highest elevation of each segment in the path network from the road level may be a fixed value (e.g., 50 meters or 100 meters above the road level), or it may be a variable value based on known mapping data for each segment location (e.g., the highest elevation analyzed for a segment may be 20 meters above the road level because the highest building or structure at the segment location is 20 meters high). After extraction and analysis at multiple elevations, at least a portion of the extracted geometric features may be selected for storage in a fingerprint database. In some embodiments, the extracted features are classified or graded based on their shape, size, elevation, and variation / invariance. Determination of the grading or classification of features is discussed in more detail below.

[0040] The extraction of geometric shapes for each horizontal slice or plane may be performed using various algorithms, wherein the depth map data in the slice is converted to one or more two-dimensional geometric shapes. In certain embodiments, the algorithm is a linear regression algorithm, a curve regression algorithm, a machine learning algorithm, or a combination thereof. For example, where linear geometric features are observed in the depth map slice, the linear features are extracted from the depth map slice using a linear regression algorithm, and the depth map data is converted to a set of 2D lines or connected lines. Where arcs or splines are observed in the depth map slice, arc or spline features are extracted from the depth map slice using a curve regression algorithm, and the depth map data is converted to 2D arcs or splines. In some embodiments, more than one geometric feature may be observed within a slice. In such cases, both linear regression and curve regression algorithms may be performed to extract linear and curve features in the slice, and the data is converted to a 2D representation comprising a set of lines, connected lines or curves, arcs, splines, and the like.

[0041] Figure 1 A non-limiting example of a depth map image 10 with horizontal slices at different elevations is provided. In this figure, two elevations have been analyzed to obtain feature geometry, depicted as a first elevation 12 and a second elevation 14. Via a linear regression algorithm, a curve regression algorithm, and / or a machine learning algorithm, features in the depth map data at the first elevation 12 are extracted and represented as a first 2D representation 16 comprising a combination of connected lines and arcs. At the second elevation 14, the extraction process has produced a second 2D representation 18 at a second elevation comprising two lines.

[0042] Encoding feature geometry in fingerprint database

[0043] In certain embodiments, the extracted feature geometry may be geo-referenced and encoded in a fingerprint database for future use. That is, a fingerprint database with encoded feature geometry may be accessed to assist in communicating with the database to determine the location of a data collection device (e.g., a vehicle) (as discussed in more detail below).

[0044] In certain embodiments, the extracted feature geometries may be geo-referenced using specific information, such as the geographic location (e.g., latitude, longitude, elevation, altitude) of specific features or points of the geometry. These two-dimensional feature geometries may be geo-referenced to a map or geographic location using an algorithm that matches each point of the 2D geometry to location reference information or values ​​of one or more of: latitude, longitude, elevation from road level, and altitude. This process is possible when the depth sensing camera (e.g., LIDAR camera) performing the analysis and extraction is communicating with a data collection vehicle with a known geographic location (e.g., using GPS or another geo-referencing device). Using the known vehicle location and the known distance of the vehicle from the geometry extracted via the depth sensing system, geographic values ​​may be assigned or referenced to the extracted geometry.

[0045] For example, when the feature geometry is a line or a collection of connected lines, points along the line or collection of connected lines (including the endpoints of the line or collection of connected lines) can be georeferenced using location reference information about the latitude, longitude, elevation above the road, and altitude of the line points. For arc geometry, points along the arc can be georeferenced, providing information about the radius, latitude, longitude, elevation above the road, and altitude of the arc points. For splines, points along the spline can be georeferenced to provide information about the nodes, latitude, longitude, elevation above the road, and altitude of the spline points.

[0046] After geo-referencing of the extracted feature geometry, the geo-referenced 2D geometry may be encoded in a fingerprint database. In certain embodiments, the fingerprint database may store the feature geometry based on the extracted location. For example, the feature may be linked to a vehicle reference (e.g., the geographic location of the vehicle when the feature geometry was identified and extracted).

[0047] Figure 2-5 Describe an example of an encoding system for a fingerprint database. Figure 2 As shown in , each fingerprint encoded in the database contains a list of feature geometries (e.g., line features, arc features, and spline features). In some embodiments, the fingerprint includes a map reference attribute that specifies a link in a reference map that corresponds to a vehicle reference from which the fingerprint was derived. The map reference may include a link identifier and a longitudinal distance on the link.

[0048] In addition, each fingerprint includes a vehicle reference, which provides a three-dimensional reference point for the data collection device or vehicle during the localization process (described below). The reference point anchors all feature geometry extracted at that location and facilitates the feature comparison process during localization to account for differences in focus between the data collection device (e.g., vehicle) and the vehicle that captured / extracted the data. The vehicle reference describes the latitude, longitude, and altitude location from which the feature geometry was extracted (e.g., the location of the contact point of the rear tire on the ground when extracted). The latitude, longitude, and altitude measurements may be described using a World Geodetic Coordinate System (WGS), such as WGS 84.

[0049] Each fingerprint also contains multiple feature geometries, such as line features, arc features, and spline features ( Figures 3 to 5 ). The feature geometries may be sorted or ranked in order of importance of the factors, with feature geometries at the top of the list having more important or identifiable characteristics of the geographic location than features at the bottom of the list. Methods for ranking feature geometries are described in detail below and may also be applied in the process of encoding feature geometries in a fingerprint database. In some embodiments, the list of feature geometries includes at least 5, at least 10, or at least 15 different feature geometries per location.

[0050] Figure 3 Detailed description of the encoded line feature. As shown in the figure, the line feature represents a 2D line geometry with two control points (i.e., identifiable extraction points on the 2D geometry). In one embodiment, the two control points of the line may be opposite ends of the line. The control points of the line share the same elevation and altitude because the points have been extracted from the depth map on the same horizontal plane. Thus, the attributes of the line feature geometry include common values ​​for both elevation and altitude. The elevation can be expressed in meters above the vehicle reference. The individual longitude and latitude of each control point of the line feature geometry is also encoded and can be expressed in WGS 84 coordinates.

[0051] Figure 4 The encoded arc feature is depicted in detail. As shown in the figure, the arc feature represents a 2D arc geometry with two control points and a radius. In one embodiment, the two control points may be opposite ends of the arc. The control points of the arc share the same elevation and altitude because the points have been extracted from the depth map on the same horizontal plane. Thus, the attributes of the arc feature geometry include common values ​​for both elevation and altitude. The individual longitude and latitude of each control point of the arc feature geometry is also encoded and can be expressed in WGS 84 coordinates.

[0052] Figure 5The encoded spline feature is described in detail. As shown in the figure, the spline feature represents a 2D spline geometry (e.g., a B-spline) with multiple control points and knots. The control points of the spline share the same elevation and altitude because the points have been extracted from the depth map on the same horizontal plane. Thus, the attributes of the spline feature geometry include common values ​​for both elevation and altitude. The individual longitude and latitude of each control point of the spline feature geometry is also encoded and can be expressed in WGS 84 coordinates.

[0053] Determine end-user device location using fingerprint database

[0054] Using an established database of encoded fingerprints of a path network, an end-user device (e.g., a vehicle, mobile phone, smart watch, etc.) equipped with depth sensing camera technology (e.g., a LIDAR camera) may be able to determine its location within the path network. The end-user device may determine its location based on the device's own identification of feature geometries surrounding the end-user device / vehicle and comparison of specific feature geometries with an established fingerprint database (discussed above), where the location is determined based on a match of features or fingerprints to the database. In some embodiments, the location of the device may be determined via a combination of identification of feature geometries and GPS data. In other embodiments, in the absence of any GPS data, the location of the device is determined solely via identification of feature geometries.

[0055] An end-user device (e.g., a vehicle, mobile phone, smart watch, etc.) equipped with a depth sensing device (e.g., a LIDAR camera) can extract feature geometries in a manner similar to that described above for a data collection device constructing a fingerprint database to determine the location of the device. Several extracted feature geometries can be selected to define a fingerprint, and the defined fingerprint can be matched to the fingerprint database to determine the location. In some embodiments, the end-user device can calculate various parameters prior to or in conjunction with the extraction process to orient the collected depth map of the device using the fingerprint database. This can assist in determining not only the geographic location of the end-user device, but also the orientation of the device at that location.

[0056] In order to identify the location of a device in real time, the determination of the device's fingerprint may be a simplified process (compared to the construction of a fingerprint database) where only a few feature geometries are selected and compared to the fingerprint database. For example, each geographic location may have thousands of feature geometries stored in the fingerprint database. In other words, it may be inefficient for an end-user device (e.g., a vehicle) to analyze or consider all 2D feature geometries that may be used for positioning.

[0057] The geometry is selected based on two main factors (described in more detail below). The first factor is the ranking of the feature geometry, and the second factor is the geometric dilution of precision (GDOP) of the feature geometry. In certain embodiments, the process of locating a device in a path network includes (1) identifying feature geometries around the device, (2) determining the properties (e.g., shape, size, elevation, variation) and location (GDOP) of each feature geometry, (3) ranking the feature geometries according to their properties, and (4) selecting fingerprints of feature geometries that have a very high degree of distribution (high GDOP) and have high ranking values.

[0058] Factor 1: GDOP of feature geometry

[0059] The GDOP represents the geographic distribution of structures or feature geometries around the vehicle. In other words, the relationship between the location of one selected feature geometry and each additional feature geometry is a factor. In certain embodiments, the best or highest GDOP is provided by the most evenly distributed structures or feature geometries around the device. In some embodiments, the best or highest GDOP is determined based on the distribution of structures as plotted in the xy plane (i.e., ignoring the elevation of each structure in the z-axis).

[0060] For example, an optimal GDOP may occur if there are three structures around a vehicle, where the structures are separated or spaced 120 degrees from each other. In other words, a first structure may be located at 0 degrees from the vehicle in the xy plane (e.g., as measured from the direction of travel or direction of travel of the road), a second structure may be located at 120 degrees from the vehicle, and a third structure may be located at 240 degrees from the vehicle.

[0061] The best GDOP will occur if there are four structures around the vehicle, where each structure is 90 degrees apart from each other. The best GDOP will occur if there are five structures, where each structure is 72 degrees apart from each other, and so on.

[0062] Factor 2: Grading of feature geometry

[0063] Regarding the second factor, feature geometry (either within each sector or within the overall depth map) may be compared or ranked based on one or more of the following properties: (1) the shape of the feature, (2) the size of the feature, (3) the elevation of the feature, and (4) the variation (or invariance) of the feature (i.e., how much the shape of the feature changes over time, or whether the feature is absolutely present at a particular time). In some embodiments, additional factors may be considered, such as the color of the feature geometry or fingerprint. Recognition of color may require additional equipment attached or otherwise connected to the data collection or end-user device (e.g., a vehicle) because LIDAR cameras do not recognize color.

[0064] Regarding shape properties, feature geometries with extremely simple shapes, such as lines, arcs, and splines, can be ranked higher than other more complex shapes because their simplicity allows for more efficient processing. Figure 6 , the feature geometry representing the straight lines on the buildings at 20 meters and 50 meters elevations may be simpler than the feature geometry capturing the branches of the trees. Therefore, the feature geometry represented by the lines on the buildings may be ranked higher than the feature geometry associated with the branches of the trees.

[0065] With respect to dimensional properties, feature geometries that are larger in size may be ranked higher than feature geometries that are smaller in size. For example, Figure 6 , the feature geometry on the left building at 20 meters and 50 meters will be graded higher than the feature geometry on the right building at 20 meters.

[0066] With respect to elevation properties, feature geometries at higher elevations from road level may be ranked higher than feature geometries at lower elevations from road level. In other words, the height of a building may assist in identifying a particular location, and thus higher elevation feature geometries at each location may be ranked higher. For example, Figure 6 In the example above, a feature geometry at an elevation of 50 meters may be ranked higher than a feature geometry at an elevation of 20 meters. In some embodiments, locations within the path network may not contain important features at higher elevations, where features at lower elevations are then selected. This may include trees or other seasonal features.

[0067] Regarding the changing nature, less changing feature geometries are given high ratings. The variation of feature geometry may take into account the seasonality of the feature. For example, in the fall or winter, leaves may disappear and expose bare branches to the LIDAR camera. The leaves may then appear again in the spring or summer. Therefore, the branches, leaves, etc. of the tree vary based on the seasonality of the geometry, and therefore may be assigned less weight than the feature geometry that is constant (such as the straight lines of a building or the trunk of a tree). In some embodiments, multiple fingerprints may be stored in a fingerprint database to accurately represent the changing features of different seasons. The variation of feature geometry may also take into account whether the feature geometry exists at a specific time and does not exist at other times (i.e., a non-stationary object). For example, a LIDAR camera mounted to a vehicle may identify another vehicle (such as a large truck) traveling along a road segment in an adjacent lane. This feature geometry may be given a very low rating due to its non-stationary nature.

[0068] With respect to color properties, characteristic geometric shapes that include bright or non-neutral colors (e.g., red, pink, purple, orange, green, blue, or yellow) may be given a higher ranking than neutral colors (e.g., gray, black, brown, beige, or white). For example, the bright or non-neutral colors of a city or metropolitan area may represent the color of a unique building or structure, as compared to the majority of steel construction found within the city or metropolitan area. Through the identification of this unique building color, a device may be able to identify its location within the city's path network much more easily.

[0069] In certain embodiments, feature geometries are ranked based on a combination of the properties discussed above. The combination may include a weighted approach, where each property is given a weight that is a function of location and time. In other words, the weight of each property may vary in space and time (i.e., the weight may be dynamic).

[0070] In one embodiment, the overall ranking value of a particular feature geometry is calculated using the ranking value of the shape of the feature (“Shape”), the ranking value of the size of the feature (“Size”), the ranking value of the elevation of the feature (“Elevation”), and the ranking value of the variation of the feature (“Variation”). The overall ranking value of each feature geometry can be calculated using the following equation:

[0071] Grading value = W1×shape+W2×size+W3×elevation+W4×change

[0072] Among them, W1+W2+W3+W4=1.

[0073] In some embodiments, W1, W2, W3, and W4 have equal weights, indicating that each property is of equal importance. In other embodiments, W2 (associated with the size of the feature geometry) and / or W3 (associated with the elevation of the geometry) are given a higher weight than W1 (shape) and / or W4 (variation). In some embodiments, W2 is given the highest weight of the four factors. In other embodiments, W3 is given the highest weight of the four factors. In a particular embodiment, the weights are ranked from highest to lowest: W3, W2, W1, and W4.

[0074] In one example, four feature geometries are compared to each other based on four factors. In this example, each factor is given the same weight (W1=W2=W3=W4=0.25). The factors are defined with Boolean values ​​of true or false, where true=1 and false=0. A true value may represent a high-classification property, such as a geometry with a dimension greater than 10 meters in length, an elevation greater than 10 meters from road level, a shape of a line, a collection of connected lines or curves, an arc, or a spline, or a constant geometry. A false value may represent a low-classification property, such as a geometry with a dimension less than 10 meters in length, an elevation less than 10 meters from road level, a shape more complex than a collection of connected lines or curves, an arc, or a spline, or a varying geometry. A classification value is determined for each feature, and an overall classification is determined based on the classification values. The results of the predictive example are shown in the following table.

[0075] Table: Hierarchical feature geometry

[0076] name shape size Elevation change Grading value Final classification Feature 1 real real real real 1.0 1 Feature 2 Fake real Fake Fake 0.25 4 Feature 3 Fake real real Fake 0.50 3 Feature 4 real real Fake real 0.75 2

[0077] In other embodiments, feature geometry may be ranked using values ​​combined with or derived from Boolean values. For example, the values ​​used to categorize features may include the actual length or size of each feature, or the elevation of each feature from road level, while the shape of each feature or the variation of each structure may be defined using a Boolean true or false property.

[0078] Combination of GDOP and feature binning

[0079] In certain embodiments, a fingerprint comprising a plurality of characteristic geometries may be obtained through a combination of characteristic geometries from various locations or zones around the vehicle. In some embodiments, the process includes selecting characteristic geometries that are extremely highly distributed (i.e., have a high GDOP) and also have a high ranking value. In certain embodiments, the process may include dividing the depth map into a plurality of segments or zones (e.g., at least 3, 4, or 5 segments or zones), wherein structures in each segment of the depth map are identified, and the characteristic geometries within those structures are ranked relative to each other. In certain embodiments, each segment or zone includes an equal volume of the depth map. The overall fingerprint includes selecting the highest ranked characteristic geometries within each segment of the depth map. In some embodiments, where a structure is not identified within a segment of the depth map, additional structures (i.e., more than one structure) in adjacent segments or additional characteristic geometries within a single structure may be identified.

[0080] In some embodiments, the process selects the highest number of characteristic geometries to consider for determining the position of the device. In some embodiments, the selection includes at least 2, at least 3, at least 5, at least 10, or at least 15 characteristic geometries.

[0081] Determine location in path network

[0082] After an end-user device or vehicle selects a fingerprint comprising a plurality of feature geometries, the device may transmit the fingerprint data to a fingerprint database via a connected network. The fingerprint database may then compare the transmitted feature geometries with its stored feature geometry data.

[0083] In the comparison process, the database is able to match the selected and transmitted feature geometry with the feature geometry stored in the fingerprint database based on the uniqueness of the selected and transmitted feature geometry. Through this matching process, the fingerprint database transmits the location data of the feature geometry back to the device. This location data may include the encoded data of the feature geometry (discussed above), such as control point data of the feature geometry. In other words, the transmission of the encoded data for the control point includes the transmission of the latitude, longitude, and altitude of the control point.

[0084] Using the transmitted control point data, and knowing the measured distance of the device from each control point, the geographic location of the device can be determined. In addition, in some embodiments, the geographic orientation of the device (e.g., direction of travel) can be determined. That is, when matching feature geometries and knowing the location of the feature geometries relative to the location of the depth sensing device, the orientation of the best aligned feature geometries can be determined to indicate the orientation of the device and the depth sensing device (e.g., driving direction).

[0085] In certain embodiments, the geographic location is determined via a multilateration process using the transmitted control point data and the measured distances from the control points (ie, radii from the control points).

[0086] In some embodiments, the geographic location of the device is determined via a multilateration process utilizing at least 3 control points (and associated latitude, longitude, and altitude data associated with each control point). In other embodiments, at least 4 control points are used. In still other embodiments, 5 or more control points are used to calculate the geographic location of the vehicle. In alternative embodiments, only 2 control points may be used in conjunction with GPS data to determine the geographic location of the end-user device or vehicle.

[0087] The device's geographic location (i.e., the device's actual latitude / longitude, and optionally altitude, coordinates) may be calculated within 1 meter of its actual or true location on the ground. In some embodiments, the geographic location may be calculated within 0.5 meters, 0.1 meters, 50 centimeters, 20 centimeters, 10 centimeters, 5 centimeters, 3 centimeters, 2 centimeters, or 1 centimeter of the actual or true location on the ground.

[0088] Figure 6A non-limiting example of a multilateration process is provided. In this example, vehicle 20 (i.e., the vehicle's processor) has identified and selected four feature geometries around the vehicle to represent the fingerprint of the location. That is, vehicle 20 has identified and selected two straight feature geometries on a first building 24 that occur at elevations of 20 meters and 50 meters above road level. Vehicle 20 has also identified and selected an additional straight feature geometry that occurs on a second building 26 at an elevation of 20 meters. In addition, vehicle 20 has identified and selected a feature geometry of a traffic light pole 28 at an elevation of 2 meters.

[0089] After transmitting these characteristic geometries and matching them with the fingerprint database, the fingerprint database has returned the control point data for each of these geometries. Figure 6 , control points are identified as the endpoints of lines 22a, 22b, 22c, 22d, 22f, and 22g, and a single point 22e representing a traffic light pole. Each of the control points 22a-22g includes latitude, longitude, and altitude information. Combined with the distance identified between the vehicle 20 and each control point, this control point information can be used to determine the position of the vehicle 20 via a multilateration calculation.

[0090] Fig. 7A and 7B Another example of this process is provided, where the position of the vehicle is determined via extraction of feature geometry, matching of the geometry to a fingerprint database, and a multilateration process. Fig. 7A In Figure 1, four straight line features are extracted from three buildings 32, 34 and 36 around the vehicle. Through a matching process with the fingerprint database, the control point information of the feature geometry is relayed back to the vehicle for processing. Figure 7B As shown in , these four line features contain 7 control points, where d1-d7 depict the identified distance of the vehicle to each control point. Using these distances and control point information (eg, latitude, longitude, and altitude), the geographic location of the vehicle is determined.

[0091] Figure 8 Another example of a multilateral measurement process based on three control points P1, P2, P3 is provided. The geographic coordinates of the three control points are identified as (0,0), (d,0), and (i,j). The distance of the vehicle to the control points is identified by the radius r1, r2, r3. Based on these known distances and the geographic locations of the control points from the fingerprint database, the approximate geographic location of the vehicle is calculated. For example, in this case, the x, y, and z coordinates of the vehicle's location can be calculated using the following equations:

[0092]

[0093]

[0094]

[0095] like Figure 8 As depicted in , the approximate position is within the intersection of three spheres.

[0096] In some embodiments, such as in the case of an autonomous or HAD vehicle, after processing and determining the geographic location or orientation of the vehicle, the vehicle may process the reported data and make a decision as to whether to alert the operator or take action. In certain embodiments, a navigation device in the vehicle or another computer system in communication with the navigation device may include instructions for routing the vehicle or generate driving commands for steering the vehicle, shifting gears, increasing and decreasing throttle, and braking based on the reported location or orientation data.

[0097] Flowchart Example

[0098] Fig. 9 An example flow chart is provided for determining the location of a device using multilateration techniques. The process of the flow chart may be performed by Fig.10 and 11 Alternatively, another device may be configured to perform one or more of the following actions. Additional, fewer, or different actions may be included.

[0099] At action S101, a terminal user device collects a depth map at a location in a path network. At action S103, a processor of the terminal user device extracts a two-dimensional feature geometry from the depth map. At action S105, a number of control points in the extracted feature geometry are identified. At action S107, a distance between the terminal user device and the identified control points is calculated. At action S109, location reference information for each identified control point is received from an external database. At action S111, a geographic location of the terminal user device in the path network is determined via a multilateration calculation using the location reference information for the identified control points and the calculated distance between the terminal user device and each identified control point.

[0100] Fig.10 An additional example flow chart is provided for assisting in determining the location of a device in a path network. The process of the flow chart may be performed by Fig.11 and 12 Alternatively, another device may be configured to perform one or more of the following actions. Additional, fewer, or different actions may be included.

[0101] At action S201, an extracted two-dimensional feature geometry of a depth map of a location in a path network is received from an end-user device. At action S203, the extracted feature geometry from the end-user device is matched with a number of feature geometries in a database for the path network. At action S205, location reference information of points of the matched feature geometry is retrieved from the database. At action S207, the location reference information is transmitted to the end-user device via a connected network, where the end-user device and its associated processor can determine its geographic location via multilateration calculations.

[0102] Navigation and Network Systems

[0103] Fig.11 One embodiment of a geographic and / or navigation data system 100 that may be used to collect depth map data or determine the location of a device in a path network is depicted. The geographic data or navigation system 100 includes, but is not limited to, a data collection or end-user device 104 traveling on a path network 108 in a geographic area 112, a device 116, a processing device 128, a workstation or computer 132, and / or a database 136. Additional, fewer, or different components may be provided. For example, a proxy server, a name server, a map server, a cache server or cache network, a router, a switch or intelligent switch, additional computers or workstations, a user interface, a management component (e.g., a management workstation), a gateway device, a backbone, a port, a network connection, and / or a network interface may be provided. Although Fig.11 The components in are shown as separate from each other, but one or more of these components may be combined.

[0104] The data collection or end-user device 104 is a vehicle, such as a car, truck, motorcycle, bicycle, Segway, or other transportation device or mechanical device for moving on the path network 108. Alternatively, the data collection or end-user device 104 may represent a pedestrian, such as a human or animal, walking or moving along the path network 108. The path network 108 is a collection of road networks and / or other paths. For example, the path network 108 is a road network composed of various roads. The roads can be used to drive vehicles on local and adjacent regional streets and highways, for example. And, as an alternative or supplement to traditional streets, the path network 108 may include bicycle lanes or paths, walking paths or other travel paths. Alternatively, the path network 108 may be an open area space without a specific path or travel constraints. The path network 108 is in a geographic area 112, such as a city, suburb, state, country, and / or other geographic area.

[0105] In certain embodiments, the data collection or end-user device 104 is an autonomous vehicle or a highly automated driving (HAD) vehicle. As described herein, an "autonomous vehicle" may refer to a self-driving or unmanned mode in which no passenger is required to operate the vehicle. An autonomous vehicle may be referred to as a robotic vehicle or an automated vehicle. An autonomous vehicle may include passengers, but does not require a driver. These autonomous vehicles may park themselves or move cargo between locations without a human operator. An autonomous vehicle may include multiple modes and transition between the modes.

[0106] As described herein, a "highly automated driving (HAD) vehicle" may refer to a vehicle that does not completely replace a human operator. In fact, in a highly automated driving mode, the vehicle may perform some driving functions, and a human operator may perform some driving functions. The vehicle may also be driven in a manual mode, where the human operator assumes a degree of control over the movement of the vehicle. The vehicle may also include a fully unmanned driving mode. Other levels of automation are possible.

[0107] The data collection or end-user device 104 travels along or moves near the path network 108 to collect data. A support device or housing 116 is attached or connected to or carried by the device 104. The support housing 116 may be or may include equipment to collect data representing an area near or around a road or path or other area. For example, the data collected may be laser, LIDAR, image, and / or video / camera data (e.g., in the visible or other light spectrum). Other data may be collected, such as location data, GPS data, and / or other geographic data.

[0108] In one embodiment, the support housing 116 is a housing or container attached to or carried by a pedestrian (data collection or end-user device 104), and as the pedestrian is on and / or moving on a path, sidewalk, or other area, equipment such as a LIDAR system and / or camera in the support housing 116 collects data. In another embodiment, the support housing 116 is a housing or container attached to a vehicle, and as the vehicle is on and / or moving on a road or path, equipment such as a LIDAR device and / or camera in the support housing 116 collects or gathers data corresponding to the surrounding area. Alternatively, the support housing 116 may be the LIDAR device or camera device itself or a part thereof. The support housing 116 may be positioned at the rear end of a device (e.g., a vehicle) and may be angled to enhance collection. In other embodiments, the support housing 116 may be positioned anywhere and in any direction on a vehicle, pedestrian, or device.

[0109] The collected data is stored on one or more non-transitory computer-readable media 120, such as a CD-ROM, DVD, flash drive, hard drive, or other tangible media suitable for storing data. Different types of data may be stored on the same media 120. Alternatively, separate non-transitory media 120 may be used to store separate or different types of data. In one embodiment, LIDAR or laser data, photos (such as digital or electronic photos), video images, and / or other image data collected by the device 104 and / or the supporting housing 116 are stored in one or more non-transitory media 120. Alternatively, the non-transitory media 120 may be a signal carrying or having data. The collected LIDAR or image data may represent an area or region near or around a path, road, or other area. The collected laser or LIDAR data and / or images may include geographic features, such as sky features, terrain or surrounding features, roads or paths (such as sidewalks), road or path markings (such as pedestrian crossings or lane markings), road or path signs, points of interest ("POI") (such as buildings, parks, museums, etc.), and / or other features or objects.

[0110] The collected data is sent to a map, geographic, and / or navigation data developer 124, such as HERE or Nokia, via one or more non-transitory media 120. For example, the non-transitory media 120 may be mailed or carried to the map developer 124. Alternatively, some or all of the collected data may be transmitted to the map developer 124 via a wireless and / or wired network. For example, the network may include the Internet, an intranet, a local area network ("LAN"), a wide area network ("WAN"), a virtual private network ("VPN"), a server network, a cellular network, a satellite network, a broadcast network, a wireless or wired connection, and / or any known or future network or connection. The term "map developer" may also include third-party contractors.

[0111] The collected data is received by a processing device 128 maintained by the map developer 124. The processing device 128 is one or more servers, computers (such as desktop or laptop processing units), processors, and / or other electronic processing systems or devices. The processing device 128 includes, but is not limited to, a processor 140, a memory 144, and an image software application 148. Additional, fewer, or different components may be provided.

[0112] Processing device 128 may include or communicate with a workstation or computer 132. For example, workstation 132 is a user interface, electronic console, and / or computer having one or more input devices that can be used to access, control, and / or communicate with processing device 128 or components thereof.

[0113] Processor 140 is a general purpose processor, an application specific integrated circuit ("ASIC"), a digital signal processor, a field programmable gate array ("FPGA"), a digital circuit, an analog circuit, or a combination thereof. In one embodiment, processor 140 is one or more processors operable to control and / or communicate with various electronics and logic of processing device 128. Memory 144 is any known or future storage device. Memory 144 is a non-volatile and / or volatile memory, such as random access memory "RAM" (electronic), read only memory "ROM" (electronic), or erasable programmable read only memory (EPROM or flash memory). A memory network may be provided. Memory 144 may be part of processor 140. Memory 144 may be operable or configured to store laser, LIDAR and / or image data or other data.

[0114] The processor 140 or other processor is configured or adapted to execute an image software application 148 that may be stored in the memory 144 or other memory. For example, the image software application 148 generates a multi-layer image (e.g., a two-dimensional scalable vector graphic ("SVG") multi-layer image) based on model data (e.g., three-dimensional model data corresponding to a geographic area). In one embodiment, the application 148 identifies and / or receives three-dimensional model data, such as collected three-dimensional LIDAR point cloud data, and performs a process or rendering sequence discussed in detail below to generate a multi-layer graphic image of the geographic area. The generated multi-layer graphic image can be used in a navigation or geographic system and in a variety of user interfaces, devices or systems. Moreover, the generated multi-layer graphic image can be provided to developers of such systems and / or devices for configuration purposes.

[0115] In one embodiment, one or more generated multi-layered graphic images are stored, linked, indexed and / or associated in a database 136, which communicates with the processing device 128. Alternatively, one or more generated multi-layered graphic images are stored in one or more separate or different databases that may or may not be associated with the database 136 or its data. In one embodiment, the geographic database 136 contains data for navigation-related and / or geographic-related services. For example, the geographic database 136 contains segment / link and node data records representing a road network (e.g., the path network 108). Nodes represent endpoints and / or intersections of segments. Segments and nodes may be associated with attributes such as geographic coordinates, street names, address ranges, speed limits, turn restrictions at intersections, and other vehicle navigation attributes, as well as POIs such as gas stations, hotels, restaurants, museums, stadiums, offices, car dealerships, auto repair shops, buildings, shops, etc. Alternatively, in addition to or in lieu of vehicle road record data, the geographic database 136 contains segment / link and node data records representing pedestrian paths or other data.

[0116] The geographic database 136 may be a master geographic database stored in a format that facilitates updates, maintenance, and development. For example, the master geographic database 136 or the data in the master geographic database 136 is in Oracle spatial format, such as for development or production purposes. The Oracle spatial format or development / production database may be compiled into a delivery format, such as GDF format.

[0117] The data in the production and / or delivery format may be compiled or further compiled to form a geographic database product or database 152, which may be used in an end-user navigation device or system or other system or device. For example, a customer of a map developer 124 (e.g., a developer of a navigation device or system) may receive the generated multi-layered graphic image and other geographic data in a delivery format, and then compile and / or configure this data for an end-user device. In one embodiment, the geographic data is compiled (e.g., compiled into a PSF format) to organize and / or configure data (e.g., segments / links, nodes, multi-layered graphic images, and / or other data or attributes) for use in performing navigation-related functions and / or services by a navigation device, such as route calculation, route guidance, map display, speed calculation, distance and travel time functions, and other functions. The navigation-related functions may correspond to vehicle navigation, pedestrian navigation, or other types of navigation.

[0118] Fig.12 Embodiments of components of device 201 are described. For example, device 201 may be an embodiment of data collection or end-user device 104, such as a vehicle, or may be similar to support device 116. In one embodiment, device 201 is a support or housing that contains equipment for collecting data. For example, device 201 is removably or integrally attached or connected to a vehicle. Device 201 is positioned at the top rear end of the vehicle. Alternatively, device 201 may be positioned on or in any portion of the vehicle at any angle. In another embodiment, device 201 is attached to or carried by a pedestrian. For example, device 201 may be a LIDAR device or other device for collecting three-dimensional data.

[0119] Device 201 includes, but is not limited to, a processor 205, a memory 209, a positioning system 213, a data collection or end-user device 217, and a camera or camera device 221. Additional, fewer, or different components may be provided. For example, an input device may be provided. The input device may be one or more buttons, a keypad, a keyboard, a mouse, a stylus, a trackball, a rocker switch, a touchpad, a voice recognition circuit, or other device or component for controlling or inputting data in one or more of the components of device 201. Although Fig.12Components in the device 201 are shown as being separate from one another, but one or more of these components may be combined. Also, some components may be provided external to the device 201. For example, if the device 201 is a housing attached to a vehicle, the positioning system 213, processor 205, memory 209, input devices, and / or other components may be in the vehicle or on another part of the vehicle, while the data collection or end-user device 217 and camera 221 are in the device 201. The data collection or end-user device 217 and camera 221 may also be separated into different enclosures or supports.

[0120] The processor 205 is similar to or different from the processor 140. The processor 205 is configured to operate collection equipment, such as the positioning system 213, the data collection or end-user device 217, and the camera system 221. For example, the processor 205 sends commands to various collection devices to collect data, and synchronizes or manages different components. Also, the processor 205 is configured to store data from the collection devices in the memory 209. The memory 209 is similar to or different from the memory 144. The memory 209 can be operated or configured to store the collected light, laser, LIDAR, image and / or location data or other data. The memory 209 can be part of the non-transitory computer-readable medium 120 or can be a separate memory.

[0121] Positioning system 213 is a GPS system, one or more mechanical and / or electrical sensors, one or more gyroscopes, a local positioning system, one or more orientation sensors, or other system or device for providing location data of device 201, its components, or an attached device (e.g., a vehicle), such as positioning data (e.g., longitude, latitude, and / or altitude) and / or orientation data.

[0122] Camera 221 is one or more cameras used to take pictures or videos of the surrounding area. For example, camera system 221 includes a video camera that records video data (e.g., in the visible light spectrum) representing the road or path and geographic features near the road or path as the vehicle travels along the road or path. Camera 221 may also capture still photos separate from the video data. Camera 221 is capable of capturing different colors and associated text of geographic features.

[0123] The data collection or end-user device 217 includes an optical data collection device. For example, the device or system 217 may include a light source and a light receiver. In one embodiment, the data collection or end-user device 217 includes a LIDAR device or sensor, a laser device, and / or other device that collects data points (e.g., three-dimensional data) by emitting and receiving light.

[0124] In the embodiments described above, the communication network may include a wired network, a wireless network, or a combination thereof. The wireless network may be a cellular telephone network, an 802.11, 802.16, 802.20, or WiMax network. In addition, the network may be a public network (such as the Internet), a private network (such as an intranet), or a combination thereof, and may utilize a variety of networking protocols currently available or later developed, including but not limited to TCP / IP-based networking protocols.

[0125] Although the non-transitory computer-readable medium is described as a single medium, the term "computer-readable medium" includes a single medium or multiple media, such as a centralized or distributed database and / or associated cache memory and servers storing one or more sets of instructions. The term "computer-readable medium" will also include any medium that is capable of storing, encoding, or carrying a set of instructions for execution by a processor, or causing a computer system to perform any one or more of the methods or operations disclosed herein.

[0126] In a specific non-limiting exemplary embodiment, the computer readable medium may include a solid-state memory (e.g., a memory card), or other package that houses one or more non-volatile read-only memories. In addition, the computer readable medium may be a random access memory or other volatile rewritable memory. In addition, the computer readable medium may include magneto-optical or optical media, such as a disk or tape or other storage device to capture carrier signals such as signals transmitted on a transmission medium. A digital file attachment to an email or other self-contained information archive or archive collection may be considered a distribution medium, which is a tangible storage medium. Therefore, it is contemplated that the present invention includes any one or more of a computer readable medium or a distribution medium and other equivalents and successor media, in which data or instructions may be stored.

[0127] In alternative embodiments, specialized hardware implementations such as application specific integrated circuits, programmable logic arrays, and other hardware devices may be constructed to implement one or more of the methods described herein. Applications that may include the apparatus and systems of the various embodiments may generally include a variety of electronic and computer systems. One or more embodiments described herein may implement functionality using two or more specific interconnected hardware modules or devices that utilize related control and data signals that may be transmitted between modules and via modules or as part of an application specific integrated circuit. Thus, the present system encompasses software, firmware, and hardware implementations.

[0128] According to various embodiments of the present invention, the methods described herein may be implemented by a software program executable by a computer system. In addition, in exemplary non-limiting embodiments, implementations may include distributed processing, component / object distributed processing, and parallel processing. Alternatively, a virtual computer system process may be constructed to implement one or more methods or functionality as described herein.

[0129] Although this specification describes components and functions that may be implemented in specific embodiments with reference to specific standards and protocols, the present invention is not limited to such standards and protocols. For example, the standards used for Internet and other packet switching network transmissions (e.g., TCP / IP, UDP / IP, HTML, HTTP, HTTPS) represent examples of the state of the art. Such standards are periodically replaced by faster or more efficient equivalents having substantially the same functions. Therefore, replacement standards and protocols having the same or similar functions as those disclosed herein are considered equivalents thereof.

[0130] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language, including compiled or interpreted languages, and it may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store portions of one or more modules, subroutines, or code). A computer program may be deployed to execute on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communication network.

[0131] The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and the apparatus can also be implemented as, special purpose logic circuits such as FPGAs (field programmable gate arrays) or ASICs (application specific integrated circuits).

[0132] As used in this application, the term "circuitry" refers to all of the following: (a) hardware circuit implementations only (e.g., implementations in analog and / or digital circuits only); and (b) combinations of circuitry and software (and / or firmware), such as (where appropriate): a combination of (i) a processor, or (ii) portions of a processor / software (including a digital signal processor), software, and memory that work together to enable a device such as a mobile phone or server to perform various functions, and (c) circuitry, such as a microprocessor or portion of a microprocessor, that requires software or firmware for operation, even if the software or firmware is not actually present.

[0133] This definition of "circuitry" applies to all uses of this term in this application, including in any claims. As another example, as used in this application, the term "circuitry" would also cover an implementation of only a processor (or multiple processors) or portion of a processor and its accompanying software and / or firmware. The term "circuitry" would also cover (for example, and if applicable to a particular claim element) a baseband integrated circuit or an application processor integrated circuit for a mobile phone, or a similar integrated circuit in a server, cellular network device, or other network device.

[0134] Processors suitable for the execution of computer programs include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. In general, the processor receives instructions and data from a read-only memory or a random access memory, or both. The essential elements of a computer are a processor for executing instructions, and one or more memory devices for storing instructions and data. In general, a computer also includes one or more mass storage devices for storing data, or is operatively coupled to receive data from or transfer data to the mass storage device, or both, such as magnetic, magneto-optical, or optical disks. However, a computer need not necessarily have such devices. In addition, a computer may be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio player, a global positioning system (GPS) receiver, to name a few. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example: semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0135] To provide interaction with a user, embodiments of the subject matter described in this specification may be implemented on a device having a display (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse or trackball) by which the user can provide input to the computer. Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and the input from the user may be received in any form, including acoustic, voice, or tactile input.

[0136] Embodiments of the subject matter described in this specification may be implemented in a computing system that includes: a back-end component, such as a data server; or includes a middleware component, such as an application server; or includes a front-end component, such as a client computer with a graphical user interface or a web browser, through which a user can interact with the implementation of the subject matter described in this specification; or any combination of one or more of such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or media of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LANs") and wide area networks ("WANs"), such as the Internet.

[0137] A computing system may include clients and servers. Clients and servers are generally remote from each other and generally interact via a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0138] The illustrations of the embodiments described herein are intended to provide a general understanding of the structures of the various embodiments. The illustrations are not intended to serve as a complete description of all elements and features of the devices and systems utilizing the structures or methods described herein. Many other embodiments may be understood by those skilled in the art upon review of the present invention. Other embodiments may be utilized, and the other embodiments are derived from the present invention so that structural and logical substitutions and changes may be made without departing from the scope of the present invention. In addition, the illustrations are representative only and may not be drawn to scale. Specific proportions within the illustrations may be exaggerated, while other proportions may be minimized. Therefore, the present invention and the drawings should be considered illustrative rather than restrictive.

[0139] Although this specification contains many details, these should not be interpreted as limitations on the scope of the present invention or what content can be claimed, but rather as descriptions of features specific to a particular embodiment of the present invention. Certain features described in this specification in the context of a separate embodiment may also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment may also be implemented individually in multiple embodiments or in any suitable sub-combination. In addition, although the features may be described above as acting in certain combinations or even initially claimed, in some cases, one or more features from the claimed combination may be deleted from the combination, and the claimed combination may be directed to a sub-combination or a variation of the sub-combination.

[0140] Likewise, while operations are depicted in the drawings and described herein in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in a sequential order, or that all illustrated operations be performed, to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems may generally be integrated together in a single software product or packaged into multiple software products.

[0141] One or more embodiments of the present invention may be referred to herein individually and / or collectively by the term "the present invention", which is only for convenience and does not intend to automatically limit the scope of the present application to any particular invention or inventive concept. In addition, although specific embodiments have been illustrated and described herein, it should be understood that any subsequent arrangement designed to achieve the same or similar purpose may replace the specific embodiment shown. The present invention is intended to cover any and all adaptations or variations of the various embodiments. Combinations of the above embodiments and other embodiments not specifically described herein will be apparent to those skilled in the art after reviewing the description.

[0142] The Abstract of the present invention is provided to comply with 37 CFR § 1.72 (b), with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the above detailed description, various features may be grouped together or described in a single embodiment for the purpose of streamlining the present invention. The present invention should not be interpreted as reflecting the following intention: the claimed embodiments require more features than are explicitly stated in each claim. In fact, as reflected in the appended claims, the subject matter of the present invention may be directed to less than all the features of any of the disclosed embodiments. Therefore, the appended claims are incorporated into the detailed description, with each claim independently defining the subject matter of the individual claims.

[0143] It is intended that the above detailed description be regarded as illustrative rather than restrictive, and that it be understood that the appended claims, including all equivalents, are intended to define the scope of the invention. The claims should not be considered limited to the described order or elements unless so stated. Therefore, all embodiments within the scope and spirit of the appended claims and their equivalents are claimed as the present invention.

Claims

1. A method wherein include: collecting, by an end-user device, a depth map at a location in a path network, wherein the path network further comprises a road network; Cutting the depth map into a plurality of horizontal segments, each horizontal segment corresponding to a different elevation; extracting, using a processor of the end-user device, a two-dimensional feature geometry from at least one of the plurality of horizontal segments of the depth map, wherein the two-dimensional feature geometry comprises one or more of: an arc, a spline, a line, or a set of connected lines or curves; identifying a plurality of control points in the extracted feature geometries, wherein at least one of the feature geometries has at least two control points; calculating a distance between the end-user device and the identified control point; receiving location reference information for each identified control point from a database external to the end-user device; as well as A geographic location of the end-user device in the path network is determined via multilateration calculations using the location reference information of the identified control points and the calculated distance between the end-user device and each identified control point. 2 . The method of claim 1 , wherein the number of the identified control points is at least three.

3. The method of claim 1, wherein the determination of the geographic location comprises a combination of the multilateral measurement calculation and Global Positioning System (GPS) data of the location. The method of claim 1 , wherein the geographic location of the end-user device is determined to be within 1 meter of a ground truth location of the end-user device. 5 . The method of claim 1 , wherein the geographic location of the end-user device is determined to within 10 centimeters of a ground truth location of the end-user device.

6. The method of claim 1, wherein the location reference information comprises latitude and longitude values ​​and altitude, or latitude and longitude values ​​and altitude and elevation above a road or path level, for each of the identified control points.

7. The method of claim 1, wherein the depth map is collected using a depth sensing device of the end-user device, the depth sensing device being selected from the group consisting of: a LIDAR camera, a RADAR device, an ultrasonic device, a structure light camera, and a mobile single camera. The method of claim 7 , wherein the end-user device is a vehicle.

9. The method according to claim 1, further comprising: include: Said extracted feature geometry is transmitted to an external database prior to said receiving.

10. A method, wherein include: Receiving, by a processing device having a processor, from an end-user device, a two-dimensional feature geometry of a depth map of locations in a path network, wherein the path network further comprises a road network, wherein the two-dimensional feature geometry (a) is extracted from at least one horizontal segment of a slice corresponding to the depth map at a particular elevation, and (b) comprises one or more of: an arc, a spline, a line, or a set of connected lines or curves; matching, using the processor, the extracted feature geometry with a plurality of feature geometries in a database for the path network; Retrieving position reference information of points of the matching characteristic geometric shape from the database; as well as The location reference information is transmitted by the processing device to the end user device.

11. The method of claim 10, wherein the location reference information comprises latitude and longitude values ​​and altitude, or latitude and longitude values ​​and altitude and elevation above a road or path, for each of the points of the matched feature geometry.

12. A device, wherein include: at least one processor; as well as at least one memory containing computer program code for one or more programs; The at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus to perform at least the method of any one of the preceding claims.

13. The apparatus of claim 12, wherein the end-user device is a vehicle, and wherein the depth map is collected using a depth sensing device of the end-user device, the depth sensing device selected from the group consisting of: a LIDAR camera, a RADAR device, an ultrasonic device, a structure light camera, and a mobile single camera.

14. The apparatus of claim 12, wherein the at least one memory and the computer program code are configured to cause the apparatus to further perform: The extracted feature geometry is transmitted to an external database prior to receiving the position reference information.

15. A computer-readable medium comprising computer program code for one or more programs; the computer program code being configured to cause at least one processor to perform at least the method of any one of the preceding claims 1 to 11.