Method and computing system for processing digital map data and method for providing or updating digital map data of navigation device

Abstract

A map data processing system (30) processes the first digital map (41) and the second digital map (42) to identify a matching object. A first digital map (41) is used to generate a first array of ordered pixel values. A second digital map (42) is used to generate a second array of ordered pixel values. A machine learning (ML) model (31) is used to determine at least one score indicating a likelihood that a first digital map object (52) corresponds to a second digital map object (62).

Description

Technical Field

[0001] Embodiments of the present invention relate to techniques associated with digital maps. Embodiments of the present invention particularly relate to methods and computing systems operable to determine whether a first digital map object of a first digital map corresponds to a second digital map object of a second digital map. Embodiments of the present invention particularly relate to methods, systems, and devices operable to process a first digital map and a second digital map for purposes such as map merging, using attributes of one map in association with another map, performing map data updates, autonomous vehicle navigation and control tasks, and / or providing map data for a navigation device, etc. Background Art

[0002] Navigation devices are widely used and very popular, for example, because of their ability to provide assistance in navigation tasks and their ability to utilize traffic condition change information.

[0003] Navigation device functions (such as dynamic route search and / or traffic warning generation) or other vehicle functions (such as autonomous or semi-autonomous control operations) are all based on digital map data. It may be necessary to identify objects corresponding to each other in different digital maps. This identification of corresponding or matching objects is necessary for performing tasks such as map merging, but is not limited thereto.

[0004] Identifying matching objects in different digital maps can be a daunting task. Different digital maps can have different formats, different numbers of objects, different levels of accuracy, etc.

[0005] Traditional techniques for identifying matching objects in such digital maps require domain-specific knowledge. This particularly applies to the feature engineering step, where expertise is typically required to determine which features are relevant to assessing the likelihood that a first digital map object in a first digital map corresponds to a second digital map object in a second digital map.

[0006] Creating a matching algorithm requires domain-specific knowledge (usually specific to two maps), which makes the matching process complex, cumbersome, and error-prone. This particularly applies to feature engineering performed based on domain-specific knowledge. These features are typically used in handcrafted rules, which then form a road network matcher.

[0007] US11 231 283 B2 discloses a technique used in association with map data.

[0008] Traditional processes have various drawbacks. Traditional processes are time-consuming and require expertise for feature engineering. Traditional processes are not general-purpose. For illustrative purposes, features are typically a combination of general features (e.g., geometry) and map-specific features (e.g., road type). Traditional processes may be of low quality because they cannot cover all cases, resulting in blind spots or rule overlaps, leading to incorrect matching results. Summary of the Invention

[0009] Embodiments of the present invention aim to provide the following methods, systems, and devices: The methods, systems, and devices provide or utilize enhanced techniques for identifying corresponding objects relative to each other in different digital maps. In particular, it is desirable to provide methods, systems, and devices that allow for the identification of corresponding objects in different digital maps without relying on expertise in digital map data formats for feature engineering. Alternatively or additionally, it is desirable to provide methods, systems, and devices that allow for the identification of matching objects in different digital maps in a manner suitable for application to a wide variety (optionally all) possible map data formats.

[0010] According to an embodiment, there are provided the methods, systems, and devices described in the independent claims. The dependent claims define preferred embodiments.

[0011] According to an embodiment, there are provided a method and a computing system for transforming a road network matching problem into an image classification task. Machine learning techniques (such as deep learning machine learning (ML) techniques) can then be applied to identify corresponding roads within different networks of different digital maps.

[0012] According to one aspect of the present invention, a method of processing a first digital map and a second digital map includes the following steps:

[0013] Generating a first ordered pixel value array using the first digital map, the first ordered pixel value array representing a first digital map object in a region included in both the first digital map and the second digital map, the first digital map object being included in the first digital map;

[0014] Generating a second ordered pixel value array using the second digital map, the second ordered pixel value array representing a second digital map object in the region, the second digital map object being included in the second digital map;

[0015] Using a machine learning ML model to determine at least one score indicating the likelihood that the first digital map object corresponds to the second digital map object.

[0016] The ML model has an input layer that receives at least the first ordered pixel value array and the second ordered pixel value array. The ML model has an output layer that outputs the at least one score.

[0017] In the method, ordered pixel value arrays are generated from a first digital map and a second digital map. These ordered pixel value arrays can represent a two-dimensional image that indicates whether an object, such as a road or other navigable element, is present at each pixel of the ordered array.

[0018] An ML model is used to perform processing of the ordered arrays, and the ML model does not need to be separately trained for different formats of the first digital map and the second digital map. The ML model can be trained for a general task of matching objects represented by various ordered pixel value arrays in an image.

[0019] Thus, the method according to this aspect of the invention eliminates the need for expertise in feature engineering related to the corresponding field and is less error-prone compared to traditional techniques.

[0020] The method can be implemented by or using one or more integrated circuits.

[0021] All pixel values of the first ordered array and the second ordered array can be selected from a first set including one or more first pixel values and a second set including one or more second pixel values or a group consisting of the first set and the second set.

[0022] If a first digital map object overlaps with a pixel, the pixel value in the first ordered pixel value array provided for the pixel can be a pixel value included in the first set of one or more first pixel values. If the first digital map object does not overlap with the pixel, the pixel value in the first ordered pixel value array provided for the pixel can have a pixel value included in the second set of one or more second pixel values.

[0023] If an object overlaps with a pixel, the pixel value in the second ordered pixel value array provided for the pixel can be a pixel value included in the first set of one or more first pixel values. If the object does not overlap with the pixel, the pixel value in the second ordered pixel array provided for the pixel can be a pixel value included in the second set of one or more second pixel values.

[0024] The second set can include only a single pixel value or consist of only a single pixel value (e.g., a logical zero for indicating a non-overlapping arrangement with the pixel).

[0025] The first set may include or consist of only a single pixel value (e.g., a logical one to indicate an overlapping arrangement with a pixel), or may include or consist of a set of at least two second pixel values. The set of at least two second pixel values ​​may include second pixel values ​​indicating a hierarchy of infrastructure objects (such as, but not limited to, road categories).

[0026] The first ordered array of pixel values ​​and the second ordered array of pixel values ​​may be binary arrays of pixel values.

[0027] The ML model may be an ML model that has been trained using labeled training image data generated from at least one pair of different maps. The labels may indicate whether the objects depicted in the images correspond to each other, i.e., whether they represent the same real-world object.

[0028] The labeled training image data may include pixel value arrays generated from at least two pairs of different maps having different map formats and / or data structures, thereby enhancing the robustness of the ML model performance.

[0029] All binary pixel values ​​may be selected from the group consisting of a first pixel value and a second pixel value.

[0030] The pixel values ​​in the first ordered array, the second ordered array, and the optional additional ordered array need not be binary. For illustration purposes, different pixel values ​​may indicate different road categories. The pixel values ​​may be selected from a finite set of values ​​representing different hierarchical levels of path elements. Path elements may include roads where motor vehicles such as cars are allowed to travel, and roads where motor vehicles are prohibited from traveling and are reserved for pedestrians or cyclists.

[0031] If the first digital map object overlaps with the pixel, the pixel value in the first ordered pixel value array (the pixel value provided for the pixel) may have a first pixel value. If the first digital map object does not overlap with the pixel, the pixel value in the first ordered pixel value array (the pixel value provided for the pixel) may have a second pixel value. If the object overlaps with the pixel, the pixel value in the second ordered pixel value array (the pixel value provided for the pixel) may have a first pixel value. If the object does not overlap with the pixel, the pixel value in the second ordered pixel value array (the pixel value provided for the pixel) may have a second pixel value.

[0032] The input layer may also receive a third ordered array of pixel values ​​representing one or several additional first digital map objects in the area, the one or several additional first digital map objects being included in the first digital map and being different from the first digital map object.

[0033] The input layer may further receive a fourth ordered pixel value array that represents one or more additional second digital map objects in the region, the one or more additional second digital map objects being included in the second digital map and different from the second digital map object.

[0034] The third ordered pixel value array and the fourth ordered pixel value array may represent objects adjacent to the first digital map object and the second digital map object, such as roads, sidewalks, or bike lanes. By considering these adjacent objects, the matching accuracy can be improved.

[0035] The third ordered pixel value array and the fourth ordered pixel value array may have binary pixel values, which may have only two different values. This makes the implementation and training of the ML model particularly simple.

[0036] The third ordered pixel value array and the fourth ordered pixel value array may be non-binary pixel values. This increases the generality. The non-binary pixel values may be or may include pixel values representing a hierarchy of road classes.

[0037] The first digital map may include first nodes and first edges interconnecting the first nodes. The second digital map may include second nodes and second edges interconnecting the second nodes. The first digital map object may be one of the first edges. The second digital map object may be one of the second edges.

[0038] The first edge and the second edge may represent infrastructure objects on which motor vehicles are allowed to travel.

[0039] The first edge and the second edge may represent infrastructure objects on which cars are allowed to travel.

[0040] The first digital map may further include additional first edges that represent infrastructure objects prohibited for use by motor vehicles. The second digital map may further include additional second edges that represent infrastructure objects prohibited for use by motor vehicles.

[0041] The input layer may further receive: one or more first additional ordered pixel value arrays that represent at least one of the additional first edges; and one or more second additional ordered pixel value arrays that represent at least one of the additional second edges.

[0042] In the method, when determining at least one score for a first digital map object and a second digital map object representing a road on which a motor vehicle (such as a car) is allowed to travel, objects that are prohibited for vehicle use, such as sidewalks and bike lanes, can be considered.

[0043] The additional first side can include a bike lane and / or a sidewalk in the first digital map. The additional second side can include a bike lane and / or a sidewalk in the second digital map.

[0044] Each of the first ordered pixel value array, the second ordered pixel value array, each of the one or more first additional ordered pixel value arrays, and each of the one or more second additional ordered pixel value arrays can respectively consist of X×Y pixel values, where X and Y are integers greater than 1, and where the region is a rectangular region including X×Y pixels.

[0045] Y can be equal to X. This choice may be particularly useful for accommodating various relative rotations between objects in different maps. This choice may also be particularly useful for accommodating various different shapes of the first object and the second object and their neighborhoods, which are taken into account when determining the at least one score.

[0046] The method may further include generating a three-dimensional tensor that at least includes the first ordered array and the second ordered array, where the input layer receives the three-dimensional tensor.

[0047] The size of the three-dimensional tensor can be X×Y×Z, where X, Y, and Z are integers, where X and Y are greater than 1, and where Z is equal to or greater than 2. Z can be equal to or greater than 4 (for example, to consider roads adjacent to the first digital map object and the second digital map object). Z can be equal to or greater than 8 (for example, to consider sidewalks and / or bike lanes near the first digital map object and the second digital map object during the matching process).

[0048] The method may further include repeating the steps of generating the ordered arrays and processing the ordered arrays for multiple pairs of first digital map objects and second digital map objects. Thereby, different matching candidates can be examined by quantifying the likelihood that a candidate second digital map object corresponds to a first digital map object (i.e., represents the same real-world object as the first digital map object).

[0049] The ML model can be or can include an artificial neural network (ANN).

[0050] The ML model can be or can include a deep learning model.

[0051] The ML model can be or can include a residual neural network (RNN).

[0052] The ML model can be or can include a Convolutional Neural Network (CNN).

[0053] The ML model can be or can include a quantum ML model.

[0054] The method can further include using training data to train the ML model.

[0055] The training data can include labels, where for each of a plurality of sets of a first side and a second side of a training map for two different numbers, the label indicates whether the first side corresponds to the second side.

[0056] The ML model can be trained using supervised training.

[0057] The ML model can be trained using unsupervised training.

[0058] The ML model can be trained using semi-supervised training.

[0059] Training the ML model can include techniques such as gradient descent to adjust the parameters of the ML model. The parameters of the ML model adjusted during training can depend on the specific implementation of the ML model.

[0060] The labeled training data can include arrays of pixel values generated from at least two different pairs of maps.

[0061] Training the ML model can include generating a plurality of first training pixel value arrays from a first digital training map. Training the ML model can include generating a plurality of second training pixel value arrays from a first digital training map. The label assigned to any pair of a first training pixel value array and a second training pixel value array can indicate whether the objects (such as roads) in the first and second training pixel value arrays of the pair correspond to each other, i.e., whether they represent the same real-world object (e.g., road).

[0062] The first digital training map and the second digital training map are different digital maps. The first digital training map can have a first data format for storing map objects, and the second digital training map can have a second data format for storing map objects, the second data format being different from the first data format.

[0063] The first digital training map and the second digital training map can have a map format different from the map format of one or both of the first digital map and the second digital map on which road network matching is subsequently performed using the trained ML model.

[0064] Training an ML model may include generating additional multiple arrays of additional training pixel values from at least one additional digital training map. The at least one additional digital training map may have a map format, in particular a data structure of map objects, which may be different from both the first digital training map and the second digital training map. By performing training using various different digital training maps with various different map data formats, the robustness of the ML model performance can be enhanced.

[0065] It should be understood that the various digital training maps overlap to generate a sufficient number of arrays of training pixel values from different digital training maps showing corresponding objects.

[0066] The first digital map and / or the second digital map may include data generated using probe trajectory data. The probe trajectory data may be or may include a position trajectory. The position trajectory may be a Global Navigation Satellite System (GNSS) trajectory or floating car data, but is not limited thereto. The probe trajectory data may (optionally in a time-related manner) identify a series of positions.

[0067] The method may further include using the at least one score to determine whether to perform a map update. This may be particularly advantageous if one of the first digital map and the second digital map is obtained from probe data. For illustration purposes, if there are increasingly more vehicle trajectories that turn into objects in the first digital map, but there are no corresponding objects in the second digital map used in the server and / or navigation device and / or vehicle control device, this indicates that a map update may be needed.

[0068] The method may further include using at least one score to initiate a map merge of the first digital map and the second digital map.

[0069] During the map merge process, at least one score may be used to identify the second digital map object that best corresponds to any given first digital map object.

[0070] The method may further include performing a map merge to generate a third digital map based on the first digital map and the second digital map.

[0071] The map merge may use the score to determine whether a first digital map object in the first digital map corresponds to a second digital map object in the second digital map.

[0072] The map merge may include: if the score indicates that the first digital map object corresponds to the second digital map object, combining the attributes of the first digital map object in the first digital map with the attributes of the second digital map object in the second digital map.

[0073] The method may further include outputting a third digital map to at least one navigation device for vehicle navigation operations or vehicle control operations.

[0074] Data of the third digital map may be added to an additional map data layer of the navigation device.

[0075] The method may further include performing vehicle navigation operations by at least one server using the third digital map.

[0076] Data of the third digital map may be added to an additional map data layer of the server.

[0077] The method may further include providing the third digital map for storage in a map data repository.

[0078] Data of the third digital map may be added to an additional map data layer of the map data repository.

[0079] The method may further include performing vehicle control operations by at least one navigation device or control circuit on-board the vehicle using data of the third digital map.

[0080] Vehicle control operations may include autonomous driving operations.

[0081] The method may further include performing vehicle navigation operations by at least one navigation device or control circuit on-board the vehicle using data of the third digital map.

[0082] Vehicle navigation operations may include one or several of route search, route guidance, generating warnings or alerts.

[0083] According to another aspect of the present invention, there is provided a machine-readable instruction code which, when executed by a computer system or a server, causes the computer system or the server to execute the method according to any aspect or embodiment disclosed herein.

[0084] The machine-readable instruction code may be embodied in a non-transitory medium.

[0085] The machine-readable instruction code may be embodied in a signal that can be transmitted through an air interface.

[0086] According to another aspect of the present invention, there is provided a non-transitory storage medium storing machine-readable instruction code, wherein the machine-readable instruction code, when executed by a computer system or a server, causes the computer system or the server to execute the method according to any aspect or embodiment disclosed herein.

[0087] According to another aspect of the present invention, there is provided a method for providing or updating digital map data of a navigation device, the method comprising: performing a method for processing a first digital map and a second digital map, and providing or updating at least a part of the digital map data of the navigation device using at least one score.

[0088] Additional digital map layers may be stored in the digital map data of the navigation device.

[0089] Providing or updating at least a part of the digital map data of the navigation device using at least one score may include transmitting data obtained by a map merging operation using the at least one score. The data may be transmitted via a wireless or wired interface.

[0090] The additional digital map layer may include at least some data elements of a third digital map determined using at least one score.

[0091] According to another aspect of the present invention, there is provided a computing system for processing a first digital map and a second digital map, the computing system comprising:

[0092] A storage system operable to store the first digital map and the second digital map; and

[0093] At least one integrated circuit coupled to the storage system and operable to:

[0094] Generate a first ordered pixel value array using the first digital map, the first ordered pixel value array representing a first digital map object in an area included in both the first digital map and the second digital map, the first digital map object being included in the first digital map;

[0095] Generate a second ordered pixel value array using the second digital map, the second ordered pixel value array representing a second digital map object in the area, the second digital map object being included in the second digital map;

[0096] Use a machine learning ML model to determine at least one score indicating the likelihood that the first digital map object corresponds to the second digital map object.

[0097] The ML model has an input layer that receives at least the first ordered pixel value array and the second ordered pixel value array.

[0098] The ML model has an output layer that outputs the at least one score.

[0099] In the operation of the processing system, an ordered pixel value array is generated from a first digital map and a second digital map. These ordered pixel value arrays can represent a two-dimensional image that indicates whether an object, such as a road or other navigable element, exists at each pixel providing the ordered array.

[0100] The processing system is operable to cause an ML model to be used to perform processing of the ordered array, and the ML model does not need to be separately trained for different formats of the first digital map and the second digital map. The ML model can be trained for a general task of matching objects represented by various ordered pixel value arrays in an image.

[0101] The processing system according to this aspect of the invention thus eliminates the need for expertise in feature engineering related to the corresponding field and is less error-prone compared to conventional techniques.

[0102] The at least one integrated circuit can be operable to execute the method of any one aspect or embodiment disclosed herein.

[0103] According to another aspect of the invention, a system is disclosed that includes a computing system and at least one navigation device or vehicle control device, and the at least one navigation device or vehicle control device is operable to receive and use the result of processing performed by the computing system.

[0104] The navigation device or vehicle control device can be operable to perform at least one navigation operation or vehicle control operation based at least on the result of processing performed by the computing system.

[0105] According to another aspect of the invention, a navigation device or vehicle control device is disclosed that can be operable to perform at least one navigation operation or vehicle control operation using the result of a map processing method.

[0106] The navigation device or vehicle control device can be operable to perform at least one navigation operation or vehicle control operation based at least on the result of processing performed by the computing system.

[0107] According to a further embodiment, the use of the result of map merging and / or object matching by a server, navigation device, or vehicle controller for map data maintenance, map data update, navigation operations, driver assistance functions, semi-autonomous vehicle control operations, and / or autonomous vehicle control operations is disclosed.

[0108] The use can include processing the result of map merging and / or object matching to perform at least one navigation task (such as route search or route guidance) and / or at least one vehicle control operation (such as automatic steering or engine control operation).

[0109] Embodiments of the present invention achieve various effects and advantages. The method, system, and device allow for performing road network matching without the need for domain - specific knowledge. The method, system, and device are operable to predict the likelihood that any given second digital map object in a second digital map corresponds to the same real - world object as any given first digital map object in a first digital map. Although the method, system, and device are operable to perform road network matching for map merging, the method, system, and device are not limited thereto.

[0110] Embodiments of the present invention can achieve various additional effects and advantages. For illustrative purposes, in the case where an ordered pixel value array represents the presence or absence of an object (e.g., a road class) at corresponding pixel or object hierarchies, it is convenient to process the ordered pixel array through an ML model. Robust and particularly reliable results are obtained.

[0111] For further illustration, in the case of generating additional ordered pixel value arrays and using them for road network matching tasks, the accuracy can be further improved. The additional ordered pixel value arrays can include one or more ordered arrays corresponding to roads accessible to motor vehicles in the vicinity of the first digital map object and the second digital map object. Alternatively or additionally, the additional ordered pixel value arrays can include one or more ordered arrays corresponding to infrastructure elements accessible to pedestrians or cyclists but prohibited for motor vehicle use.

[0112] Although the techniques disclosed herein can be used for map merging, they can also be used, for example, to merge point - of - interest (POI) data from one digital map into another digital map.

[0113] The method, system, and device do not require but still allow the use of first and second digital maps in a specific format to train the ML model. By generating ordered pixel value arrays, the trained ML model can be applicable to a wide variety of different digital map data formats. Retraining can be optionally performed for a specific map data format, but it is generally not mandatory. BRIEF DESCRIPTION OF THE DRAWINGS

[0114] Embodiments of the present invention will be described with reference to the accompanying drawings, where like or corresponding reference numerals denote elements having like or corresponding configurations and / or functions.

[0115] Figure 1 is a block diagram of a map data processing system.

[0116] Figure 2 is a flowchart of a map data processing method.

[0117] Figure 3Graphical representations of a first digital map and a second digital map are shown.

[0118] Figure 4 A machine learning (ML) model that may be used in a map data processing system and method is shown.

[0119] Figure 5 A machine learning (ML) model that may be used in a map data processing system and method is shown.

[0120] Figure 6 A schematic block diagram illustrating an object matching process is shown.

[0121] Figure 7 , Figure 8 , Figure 9 , Figure 10 , Figure 11 and Figure 12 The operation of the map data processing system method is illustrated.

[0122] Figure 13 It is a flow chart of the map data processing method.

[0123] Figure 14 and Figure 15 The operation of the map data processing system and method is illustrated.

[0124] Figure 16 It is a flow chart of the map data processing method.

[0125] Figure 17 and Figure 18 The operation of the map data processing system and method is illustrated.

[0126] Figure 19 is a flow chart of a method according to an embodiment.

[0127] Figure 20 is a flow chart of a method according to an embodiment.

[0128] Figure 21 is a diagram of a system including a map data processing system.

[0129] Figure 22 is a block diagram of the vehicle.

[0130] Figure 23 is a graph showing a representative comparison of processing results obtained using a method according to an embodiment compared to a conventional method. DETAILED DESCRIPTION

[0131] Embodiments of the present invention will be described in detail.Although the embodiments will be described in conjunction with the merging of the first digital map and the second digital map, the embodiments are not limited thereto.

[0132] Unless otherwise specifically stated, the features of the embodiments may be combined with each other.

[0133] The techniques disclosed in detail herein can be used to perform automatic feature calculations in, for example, road network matching without performing expertise-dependent feature engineering to develop dedicated matching algorithms specific to the respective formats of the first digital map and the second digital map.

[0134] Matching objects in different digital maps is also referred to as road network matching in the art. As used herein, matching different objects in different digital maps or identifying corresponding objects refers to identifying which object in one digital map corresponds to an object in another digital map, i.e., they represent the same real-world object. The objects may represent roads and may be modeled as edges of a graph, particularly edges of a sequential graph.

[0135] As used herein, a first digital map object corresponds to a second digital map object if the first digital map object and the second digital map object correspond to the same real-world element (e.g., a road).

[0136] Therefore, the likelihood that a first digital map object matches a second digital map object is the likelihood that the first digital map object and the second digital map object correspond to the same real-world object (e.g., the same road).

[0137] In an embodiment, tensors may be employed. A tensor may combine different arrays of rectangular pixel values. For regions where different digital maps overlap, different arrays of rectangular pixel values may depict the presence or absence of different objects at corresponding pixels. Examples of different objects include: objects to be matched (e.g., roads) to determine the likelihood that they represent the same real-world object; roads adjacent to the objects to be matched; and paths adjacent to the objects to be matched that are prohibited for vehicle use.

[0138] The number of different arrays of rectangular pixel values combined into a tensor is also referred to as the number of channels.

[0139] The techniques disclosed herein can utilize a trained machine learning (ML) model. The ML model may be an artificial neural network (ANN). The ML model may include three or more layers, such as three or more hidden layers. The ML model may be a deep learning model. The ML model may be or may include a convolutional neural network (CNN) or a residual neural network (Resnet). The ML model may include a CNN and a Resnet.

[0140] Such ML models are particularly suitable for processing ordered arrays of pixel values representing two-dimensional images. The present invention is not limited to these specific types of ML models, but any processing capable of performing image classification can be used. This is because, in the technology of the embodiments, the matching task of digital map data is transformed into an image classification task, where image classification determines whether two ordered arrays of pixel values represent the same real-world object.

[0141] The method, system, and device according to the embodiments can be used for road network matching. Road network matching is a common problem whose goal is to determine which roads in one network correspond to which roads in another network. As disclosed in more detail herein, the embodiments can solve this problem by transforming the road network matching problem into an image classification problem. Then, deep learning techniques or other ML techniques can be applied to predict the matching of roads within a separate network. The result is a high-quality general matching technique that does not require the time typically needed for feature engineering and has better performance than traditional techniques.

[0142] The method, system, and device disclosed herein also cover the use of the results of road network matching. Thus, the results of road network matching can be used for various functions, such as map merging, and ultimately using the data of the merged map for vehicle navigation, route guidance, autonomous driving, map updating, but not limited thereto.

[0143] By knowing which roads in one map correspond to which roads in another map, information can be imported from one map to another. This information can be related to road geometry, such as which roads are unique in one of the maps. But it can also be related to bringing attributes from one map to another. For illustrative purposes, once the matching roads are identified, the point-of-interest (POI) information contained only in one of the first digital map and the second digital map can be imported into the other digital map.

[0144] Figure 1 is a block diagram of a map data processing system 20. The map data processing system 20 can be part of a system that further includes map data sources 41, 42 and / or a system or device that uses the output 49 of the map data processing system 20.

[0145] The map data processing system 20 includes a first interface 21 that is operable to receive a first digital map 41 and a second digital map 42. Different digital maps do not need to have the same format. The first digital map 41 and the second digital map 42 can be stored in a storage system 23.

[0146] The map data processing system 20 is operable to process a first digital map 41 and a second digital map 42 to identify pairs of objects that correspond to each other (i.e., represent the same real-world object).

[0147] The map data processing system 20 includes one or more circuits 30. The (multiple) circuits may include an integrated circuit, an integrated semiconductor circuit, a processor, a controller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), any one or any combination of (multiple) circuits including qubits and / or quantum gates, but is not limited thereto.

[0148] The (multiple) circuits 30 may be operable to process the digital maps 41, 42. The (multiple) circuits 30 may execute instruction code to implement a matching object recognition module 32 that identifies matching objects in the first digital map and the second digital map. The matching objects may be roads, but are not limited thereto. The matching object recognition module 32 may be operable to generate a plurality of ordered pixel value arrays from the first digital map and the second digital map. The matching object recognition module 32 may be operable to process the plurality of ordered arrays to determine one or more scores that indicate the likelihood that a first digital map object of the first digital map corresponds to a second digital map object of the second digital map.

[0149] The ordered pixel value arrays include at least a first ordered array representing a first digital map object in the first digital map and a second ordered pixel value array representing a second digital map object in the second digital map. Classification techniques may be applied to determine whether the first digital map object corresponds to the second digital map object. The classification techniques may employ one or more trained machine learning (ML) models, particularly at least one deep learning model 31. The plurality of trained ML models may be stored in the map data processing system 20. The plurality of trained ML models may be used in parallel or sequentially, e.g., to further improve accuracy and / or cross-validate the obtained results. The plurality of trained ML models may include ML models having different architectures (e.g., different numbers of hidden layers).

[0150] In addition to the first ordered array and the second ordered array, the matching object recognition module 32 may also generate additional ordered arrays (for any pair of candidate objects for which at least one score is to be determined). The additional ordered arrays may include an ordered array representing roads near the first digital map object and the second digital map object. The additional ordered arrays may include an ordered array representing paths for pedestrians and / or cyclists near the first digital map object and the second digital map object, which paths may be prohibited for motor vehicles. The additional ordered arrays may be input to a classifier (e.g., an ML model) for processing to improve matching accuracy.

[0151] For any pair of a first digital map object and a second digital map object, the result of the matching can be at least one score that indicates the likelihood that the first digital map object corresponds to the second digital map object (i.e., the first digital map object and the second digital map object represent the same real-world object).

[0152] The map data processing system 20 can execute a map merging module 33. The map merging module 33 can use at least one score determined by object matching recognition. Map merging can include generating third digital map data. The third digital map data can include a first data element obtained from the first map data and a second data element obtained from the second map data. For any object, information can be obtained from the digital map that provides better (e.g., more detailed and / or more up-to-date and / or more general) data from among the first digital map and the second digital map.

[0153] The map data processing system 20 can execute an output generation module 34. The output generation module 34 can generate an output that directly or indirectly depends on the result of object matching recognition. By way of illustration, the output generation module 34 can use the scores determined by the object matching recognition module 32 to prepare and transmit a map update that includes updated map data from the third digital map generated by the map merging module 33.

[0154] An output 49 generated by the detector data processing system 20 can be output via a second interface 22. The second interface 22 can be operable to communicatively interface with, for example, a navigation device (including but not limited to a mobile terminal of a cellular network), a vehicle control device, a map server, a map data repository, a navigation server, a traffic control server.

[0155] The operation of the map data processing system 20 will be described in more detail hereinafter. The map data processing system 20 can be operable to execute any of the processing techniques described in detail below.

[0156] Figure 2 is a flowchart of a method 100. The method 100 can be automatically executed by the map data processing system 20.

[0157] At step 101, a first ordered pixel value array is generated. The first ordered pixel value array is generated from a first digital map. For any pixel in the area where the first digital map and the second digital map overlap, the first ordered pixel value array can indicate whether a specific first digital map object (e.g., a first road on which a motor vehicle is allowed to travel) exists at the corresponding pixel in the first digital map. The first ordered pixel value array can have other formats. For illustrative purposes, it can indicate a functional road class or other indicators of an infrastructure object hierarchy according to the pixel positions determined from the first digital map.

[0158] At step 102, a second ordered pixel value array is generated. The second ordered pixel value array is generated from a second digital map. For any pixel in the area where the first digital map and the second digital map overlap, the second ordered pixel value array can indicate whether a specific second digital map object (e.g., a second road on which a motor vehicle is allowed to travel) exists at the corresponding pixel in the second digital map. The second ordered pixel value array can have other formats. For illustrative purposes, it can indicate a functional road class or other indicators of an infrastructure object hierarchy according to the pixel positions determined from the second digital map.

[0159] At step 103, image classification is performed. An ML model, particularly a trained deep learning model, can be used to determine one or more scores that indicate whether the first digital map object corresponds to the second digital map object.

[0160] The results of the matching can be used in various ways, including but not limited to map merging, navigation, map updating, and autonomous driving.

[0161] The essence of digital maps is that they allow a processing system to determine which positions within an area with X×Y pixels contain any given map object (such as a first map object or a second map object), and which pixels do not contain the given map object. To determine the first ordered array (e.g., at step 101) and the second ordered array (e.g., at step 102), and / or any other array mentioned herein for inference or training purposes, the map data processing system and method only need to know which map object is present at which pixel and accordingly set the pixel values. For illustrative purposes, the map data processing system and method can operate to determine, based on the map data for the pixels within the area, which object(s) is / are present at the corresponding pixel, set the pixel value of that pixel according to the determination result (e.g., as a binary value indicating the presence or absence of an object of interest at the corresponding pixel, or as a non-binary value that can indicate, for example, road class information), and repeat these operations until all X×Y pixels have been considered. For further illustration, the map data processing system and method can operate to identify all pixels of the area that overlap with any object of interest (e.g., a first map object or a second map object) defined by a first digital map or a second digital map, optionally repeat this determination until all objects of interest (e.g., map objects adjacent to the first map object or the second map object that can be optionally considered) have been considered, and set the pixel values based on the identification result. As described above, the essence of digital maps is to define the positions of map objects and the way they extend. Obviously, regardless of whether the (multiple) digital maps define map objects by the coordinates of the start and end points of the map objects, by a combination of the coordinates of object points and extension vectors, by a combination of both, or by yet other techniques, it is possible to easily determine the pixels that overlap with the map objects.

[0162] Figure 3 A portion 51 of a first digital map and a portion 61 of a second digital map are illustrated. The map data processing system and method can operate to determine the likelihood that a first digital map object 52 in the first digital map and a second digital map object in the second digital map 61 represent the same real-world object.

[0163] To solve this problem, a two-dimensional array of pixel values is generated from the digital map data, the two-dimensional array either including the selected objects 52, 62 for which the likelihood is to be determined or including objects 63, 64, 65 adjacent to the selected objects 52, 62. Preferably, none of the generated arrays of pixel values includes both the object and the adjacent objects to be matched in the corresponding processing step. However, it is acceptable for several adjacent objects (e.g., several adjacent roads) to be present in the ordered array of pixel values simultaneously.

[0164] It should be understood that, as Figure 3 shown, different digital maps can represent the road network and other paths at different levels of detail. For illustrative purposes, additional roads or sidewalks or bike lanes 64, 65 may be present in the second digital map 61.

[0165] Although various image classifiers can be used, ML models such as trained deep learning models are particularly suitable for performing image classification tasks. The processing includes processing a first ordered array of pixel values and a second ordered array of pixel values to determine whether objects 52, 62 correspond to each other.

[0166] Figure 4 and Figure 5 are schematic representations showing ML models 120, 125 that can be used. The ML model includes an input layer 121. The input layer 121 is operable to receive a number of ordered arrays of pixel values, the ordered arrays of pixel values including an array of pixel values generated only from the first digital map and an array of pixel values generated only from the second digital map. The input layer 121 can be operable to receive a plurality of pixel values generated only from the first digital map and the same number of pixel values generated only from the second digital map.

[0167] The ML models 120, 125 have an output layer 122. The output player 122 is operable to output one or a number of probabilities that indicate the probability that the first digital map object and the second digital map object match each other. There may be a single node 124 at the output, which indicates a single probability value, or indicates a binary result (match or no match).

[0168] There may be a number of nodes 126, 127 at the output of the ML model, which provide a number of probabilities. For illustrative purposes, one of the nodes 126 can indicate the degree of certainty of the classifier that the first digital map object and the second digital map object correspond to each other. Another of the nodes 127 can indicate the degree of certainty of the classifier that the first digital map object and the second digital map object do not correspond to each other.

[0169] Other ML model configurations can be used. Figure 4 and Figure 5 are only schematic. More complex hidden layer structures can be used, including convolutional layers and / or residual network structures.

[0170] As described above, the matching techniques disclosed herein can be used in various processes. Figure 6 One implementation is shown in

[0171] Figure 6A block diagram of a process according to an exemplary embodiment is shown, in which the matching techniques disclosed herein can be used for feature calculation 113 and prediction 115 of the process, in which data elements from a first digital map 41 and from a second digital map 42 are matched, for example, to combine the data elements in a merged third digital map.

[0172] In the linking step 111, potential matching candidates are assigned to the first road of the first digital map 41. The potential matching candidates can be all the roads in the second digital map 42 that are within a certain distance range of the first road.

[0173] The matching condition 112 can include a pair of the first road of the first digital map 41 and the second road of the second digital map 42.

[0174] The feature calculation step 113 includes generating an ordered pixel value array. At least two, four, six, eight or more ordered pixel value arrays can be generated. The ordered pixel value arrays can be regarded as representing the first road (the first ordered pixel value array) and the second road (the second ordered pixel value array) for which the corresponding possibility is to be determined. Additional ordered pixel value arrays can represent adjacent roads or adjacent paths for pedestrians and / or cyclists.

[0175] In the feature calculation step 113, each pair of candidate roads can be transformed into an associated tensor. The tensor can have Z channels, where Z is 2 or an even number greater than 2. Z / 2 channels can be filled with pixel values determined from the first map data. The remaining Z / 2 channels can be filled with pixel values determined from the second map data. Different channels can be used for the first road and the second road to be matched, as well as adjacent roads and / or paths that are not passable by motor vehicles.

[0176] The resulting column of pixel values is provided as a feature 114 to the prediction step 115. In the prediction step 115, various ordered pixel value arrays are input into the input layer of the ML model.

[0177] The ML model can be a deep learning model or other classifier that has been trained using, for example, at least one hundred thousand tensors or more tensors. Supervised learning can be employed to train the deep learning model or other classifier. The output layer of the deep learning model or other classifier can provide a score indicating the confidence level that the first road and the second road match each other. The score is not particularly limited, but can be selected from, for example, the range from 0 to 1. A score close to 1 indicates a high confidence level of matching. A score close to 0 indicates a high confidence level of non - matching.

[0178] The score indicating the possibility that the first road corresponds to the second road can be stored in the matching result 116 for subsequent use.

[0179] The process of matching objects is as Figures 7 to 12 shown, and will be explained in more detail with reference to these drawings.

[0180] Figure 7 Shows roads 52, 53, 54 of the first digital map and roads 62, 63, 64, 65, 66, 67 of the second digital map.

[0181] To identify the match of the first digital map object 52 (e.g., the first road) of the first digital map, the second digital map object 62 within the neighborhood 70 of the first digital map object 52 is considered. Roads 63, 64 are also located within or overlap with the neighborhood, and will be similarly processed as subsequent candidate matches.

[0182] To determine the likelihood that the first road 52 corresponds to the second road 62, all other roads 53, 54, 63 - 67 are considered adjacent objects (including those second roads 63, 64 for which matching is also to be performed).

[0183] The map data processing system and method generate a first ordered array 71, which has pixel values representing the first digital map object 52. The pixel values of the first ordered array 71 can depend only on the first digital map object 52, for example, by indicating in binary whether the first digital map object exists at the corresponding pixel. An array of non - binary pixel values can also be used.

[0184] The map data processing system and method generate a second ordered array 72, which has pixel values representing the second digital map object 62. The pixel values of the second ordered array 72 can depend only on the second digital map object 62, for example, by indicating in binary whether the second digital map object exists at the corresponding pixel. An array of non - binary pixel values can also be used.

[0185] At least one additional first ordered array 73 can be generated from the first digital map. The at least one additional first ordered array 73 can include an array of pixel values indicating whether there is an adjacent road 59 in the first digital map. The at least one additional first ordered array 73 can include an array of pixel values indicating whether there is a bicycle lane and / or a sidewalk within a region of the first digital map.

[0186] At least one additional second ordered array 74 can be generated from the second digital map. The at least one additional second ordered array 74 can include an array of pixel values indicating whether there is an adjacent road 69 in the second digital map. The at least one additional second ordered array 74 can include an array of pixel values indicating whether there is a bicycle lane and / or a sidewalk within a region of the second digital map.

[0187] Although the ordered pixel value arrays 71 to 74 are shown as binary pixel value arrays (where each pixel has one pixel value for indicating the presence of a map object at the pixel location and another pixel value for indicating the absence of the corresponding map object at the pixel location), more complex pixel value arrays can be generated. For illustrative purposes, the pixel value arrays can include functional road classes or other indicators of the road hierarchy as pixel values.

[0188] The pixel values of the ordered pixel value arrays 71 to 74 are input into the input layer of the ML model, and the ML model outputs a score indicating the likelihood that the first digital map object 52 matches the second digital map object 62.

[0189] Various ordered pixel value arrays 71 to 74 are combined to form a three-dimensional tensor of size X×Y×Z. Here, X and Y are integers representing the size (in pixels) of each of the ordered pixel value arrays 71 to 74. Z represents the number of ordered pixel value arrays generated for a pair of any given first digital map object and second digital map object for which the matching is to be performed.

[0190] Figure 13 is a flowchart of method 130. Method 130 can be automatically executed by the map data processing system 20.

[0191] At step 131, a first ordered pixel value array and at least one additional first ordered pixel value array are generated. The first ordered pixel value array and the at least one additional first ordered pixel value array are generated from the first digital map. For any pixel in the area where the first digital map and the second digital map overlap, the first ordered pixel value array can indicate whether a specific first digital map object (e.g., a first road on which motor vehicles are allowed to travel) exists at the corresponding pixel in the first digital map. For any pixel in the area where the first digital map and the second digital map overlap, the additional first ordered pixel value arrays can indicate additional objects that can be used to enhance the matching accuracy. These additional objects can include roads adjacent to the first ordered array, and / or bicycle lanes and / or sidewalks that are not allowed for motor vehicles but can still be used to enhance the matching accuracy. The additional first ordered pixel value arrays can indicate whether there are objects (e.g., adjacent roads) adjacent to the first digital map object at the corresponding pixel in the first digital map.

[0192] Other formats can be used. For illustrative purposes, the pixel values can indicate the functional road class according to the pixel positions determined from the first digital map. Other values indicating the hierarchy of infrastructure objects can be adopted in non-binary pixel value arrays.

[0193] At step 132, a second ordered pixel value array and at least one additional second ordered pixel value array are generated. The second ordered pixel value array and the at least one additional second ordered pixel value array are generated from a second digital map. For any pixel in the area where the second digital map and the second digital map overlap, the second ordered pixel value array can indicate whether a specific second digital map object (e.g., a second road on which a motor vehicle is allowed to travel) exists at the corresponding pixel in the second digital map. For any pixel in the area where the second digital map and the second digital map overlap, the additional second ordered pixel value array(s) can indicate additional objects that can be used to enhance the matching accuracy. These additional objects can include roads adjacent to the second ordered array, and / or bicycle lanes and / or sidewalks that are not allowed for motor vehicle use but can still be used to enhance the matching accuracy. The additional second ordered pixel value array(s) can indicate whether an object adjacent to the second digital map object (e.g., an adjacent road) exists at the corresponding pixel in the second digital map.

[0194] Other formats can be used. For illustration purposes, the pixel values can indicate functional road classes based on the pixel positions determined from the second digital map. Other values indicating the hierarchy of infrastructure objects can be employed in the non-binary pixel value array.

[0195] At step 133, a tensor is formed. The tensor includes the first ordered array, the additional first ordered array(s), the second ordered array, and the additional second ordered array(s) in different channels.

[0196] At step 134, the tensor is processed to determine the likelihood that the first digital map object matches the second digital map object. Step 134 includes processing all pixel values by an ML model. The input layer of the ML model can have X·Y·Z input nodes to receive the pixel values of the tensor.

[0197] The processing explained is typically performed for more than one pair of objects to find the best match for the first digital map object 52 in the second digital map. For illustration purposes, as Figure 14 and Figure 15 schematically shown, it can be performed for additional candidate second digital map objects 64 (shown as solid lines in Figure 14 and 63 (shown in Figure 15Matches are performed sequentially (shown as solid lines in the figure). It should be understood that in the case where the second digital map object 63 is processed as a candidate for matching with the first digital map object 52, the second digital map object 62 must be regarded as an "adjacent" section that needs to be included in the additional second array 74. Similarly, in the case where the second digital map object 64 is processed as a candidate for matching with the first digital map object 52, the second digital map object 62 must be regarded as an "adjacent" section that needs to be included in the additional second array 74.

[0198] Figure 16 is a flowchart of method 140. Method 140 can be automatically executed by the map data processing system 20.

[0199] At step 141, a first digital map object is selected from the first digital map. The first digital map object can be the first edge of the first graph representing the road network in the first digital map.

[0200] At step 142, a candidate second digital map object is selected from the second digital map. The candidate second digital map object can be the second edge of the second graph representing the same road network in the second digital map.

[0201] At step 143, scaling and / or normalization can be optionally performed. Scaling and / or normalization can ensure that the relevant parts of the first digital map and the second digital map are included in a pixel array of size X×Y, preferably a square.

[0202] At step 144, a number of ordered pixel value arrays are generated. The number of ordered arrays can include one or a number of ordered arrays generated from the first digital map and the same number of ordered arrays generated from the second digital map.

[0203] At step 145, image classification is performed to determine the likelihood that the candidate second digital map object matches the first digital map object.

[0204] At step 146, it is determined whether there are any additional candidate second digital map objects. Referring to Figure 7 、 Figure 14 and Figure 15 , the second digital map objects 63, 64 are identified as candidate second digital map objects because they are located within a certain distance 70 around the first digital map object 52. Therefore, the method returns to step 142, and steps 142 to 146 are repeated. In each iteration, at least one determined score can be stored for later use. Additional ordered arrays indicating adjacent roads (such as Figure 11 and Figure 12The arrays 73, 74 (shown in) need to be modified in each iteration to take into account the fact that another candidate second digital map object is being processed in the corresponding iteration.

[0205] At step 147, the scores determined for each pair of first digital map objects and respective candidate second digital map objects are used to identify the candidate second digital map object among the candidate second digital map objects that is most likely to correspond to the same real-world object as the first digital map object.

[0206] The scaling and / or normalization step 143 is an optional but advantageous step that has the effect of magnifying the region most relevant to the match. The processing is shown in Figure 17 and Figure 18 shown.

[0207] A first map buffer 81 is defined around the first digital map object, and a second map buffer 82 is defined around the candidate second digital map object. The overlapping or common portion of these buffers is the shared buffer 83.

[0208] The shared buffer 83 can have any size and initially does not need to be square. To enable the classifier to work properly, the geometry of the shared buffer 83 is scaled and translated to fit a rectangular pixel array of size X×Y. For illustrative purposes, Figure 17 the scaling factor applied to the shared buffer 83 in Figure 18 is different from the scaling factor applied to the shared buffer 83 in

[0209] which is smaller in size and can thus more easily fit a rectangular (preferably square) pixel array of size X×Y. When generating an ordered pixel value array, the scaling and translation (if applied) are taken into account. That is, the objects in the first digital map and the second digital map are scaled and / or translated according to the normalization applied to the shared buffer. Visual examples of various ordered pixel value arrays and their processing have been provided with reference to Figures 9 to 12 The results of object matching can be used for various purposes, including but not limited to map merging.

[0210] The result of object matching can be used for various purposes, including but not limited to map merging.

[0211] Figure 19 is a flowchart of method 150. Method 150 can be automatically executed by the map data processing system 20.

[0212] At step 151, matching objects are identified in the first digital map and the second digital map. Step 151 can be performed using any of the techniques described above.

[0213] At step 152, the matching results are used to perform map merging. Map merging may include using at least some data elements of a first digital map in combination with at least some data elements of a second digital map. Attributes, geometries, or other information associated with first digital map objects and second digital map objects may be combined. Map merging may involve determining which of the two digital maps provides better information for any object (e.g., in terms of accuracy, level of detail, and / or timeliness). Based on the determination result, information may be selected from one of the two digital maps for any object. Once matching objects (e.g., roads) are identified, information elements included in different digital maps may be aggregated.

[0214] At step 153, the results of map merging may be used. Using the results of map merging may include providing, storing, and / or otherwise using at least a portion of the data elements of the third digital map obtained by merging in step 152. The merged map data obtained at merging step 152 may be added to an additional map data layer. This applies regardless of whether the results of map merging are stored in a server, a map data repository, a navigation device (including but not limited to a mobile communication terminal such as a smart phone), or a vehicle control system.

[0215] The results of object matching (such as the results of map merging) may be received and used by a navigation device or a vehicle control system.

[0216] A navigation device (which may be but is not limited to a mobile communication terminal such as a smart phone) or a vehicle control system may not only use the results of the process, but may also contribute to at least one of the first digital map and the second digital map. For illustration purposes, at least one of the first digital map and the second digital map may be a map including edges determined based on probe track data transmitted by a probe such as a navigation device or other in-vehicle systems.

[0217] "Probe track data" may be or may include a position track. The position track may be a GPS track, a Galileo track, another Global Navigation Satellite System (GNSS) track, or floating car data, but is not limited thereto. Probe track data may be or may include information derived from the position track. Probe track data may indicate a position using a zone identifier designating different zones or using a more refined position indication (such as coordinates referring to a reference system).

[0218] As used herein, the term "detector" encompasses navigation devices, but is not necessarily limited thereto. Detector trajectory data may also indicate the detector trajectory of a device that does not perform navigation-related functions. The detector may be an in-dash navigation device, a portable navigation device (PND), a mobile terminal (MT) of a cellular communication network (e.g., a smart phone) that may (but does not need to) perform navigation-related operations, but is not limited thereto.

[0219] Accordingly, at least one of the first digital map and the second digital map may include digital map data determined based on detector trajectory data. The methods disclosed herein may include generating at least one of the first digital map and the second digital map using detector trajectory data. Techniques such as those disclosed in EP 3 977 051 A1 or EP 2 923 177 A1 may be used to generate the corresponding digital map data.

[0220] Figure 20 is a flowchart of method 160. Method 160 may be performed by or using a navigation device or a vehicle control system. The navigation device may be a portable device (such as a cellular phone or other communication terminal), a dedicated navigation device that may be removably disposed in a vehicle, or a navigation device that may be fixedly installed in a vehicle.

[0221] At optional step 161, detector trajectory data is transmitted by the navigation device or the vehicle control system. The detector trajectory data may be determined using a global navigation satellite system (GNSS) (such as a global positioning system (GPS) position sensor). The transmission of the detector trajectory data allows for generating or modifying at least one of the first digital map and the second digital map based on the actual trajectory of the vehicle recorded during on-site operation of the vehicle.

[0222] At step 162, map update data is received by the navigation device or the vehicle control system. The map update data includes data elements of a map obtained using the map merging techniques described in detail herein. In particular, the map update data includes data elements of the merged map that have been determined using at least one score of an object matching program.

[0223] At step 163, the received map update data may be used for local storage and local update of the navigation device or the vehicle control system. The map update data may be stored as an additional layer of an existing digital map of the navigation device or the vehicle control system. This enables the user of the navigation device or the vehicle control system to select the additional map layer while still providing the option to continue using the data elements of the previous map.

[0224] At step 164, the stored map update data can be used for navigation operations and / or vehicle control operations. Navigation operations can include route search, route guidance (including control of the human-machine interface (HMI)), outputting alerts and warnings by controlling the HMI. Vehicle control operations can include automatic steering, engine control, and / or braking operations.

[0225] Figure 21 FIG. is of system 10 including processing system 20. System 10 can include a detector located in vehicle 171 that agrees to share detector trajectory data for determining a map updated according to the detector trajectory. System 10 can include a detector located in vehicle 172 that has not agreed to share detector trajectory data for determining a map updated according to the detector trajectory.

[0226] The detector trajectory data can be collected by server 91 via wide area network 90 (such as via a wireless communication path). Server 91 can aggregate the detector trajectory data. The detector trajectory data or the digital map generated therefrom can be provided to map data processing system 20. As described above, it is optional to determine at least a portion of the first digital map or the second digital map according to the detector trajectory data, and detectors 171 and server 91 can be omitted.

[0227] Map data processing system 20 can receive map data of the first digital map and / or the second digital map from map server 95. Map data processing system 20 can process the first digital map and the second digital map as described above to determine matching objects in the first digital map and the second digital map.

[0228] The result of object matching can be used for map merging and determining map data update information 94, and the map data update information can be provided to a navigation device. The map data update information 94 can be transmitted via a push or pull mechanism for use by navigation devices operatively arranged in vehicles 171, 172, transmitted to a vehicle control system, and / or transmitted to communication terminal 173 that uses the map update information (e.g., for performing map updates, such as by storing the map update information in an additional layer of map data).

[0229] The map data update information can be used for various purposes, such as performing navigation operations or vehicle control operations, or providing merged map data for use by a terminal.

[0230] Figure 22 FIG. is of vehicle 180 that can be included in system 10. Vehicle 180 can be operated to perform autonomous driving operations.

[0231] Vehicle 180 includes one or more control circuits 181 that are operable to automatically control one or more actuators of vehicle 180. The control circuit(s) 181 is / are operable to control at least some of the actuator(s) in response to map update data 184. The control circuit(s) 181 may be operable to control at least the actuator 182 that affects the driving direction and the actuator 183 that affects the vehicle speed in response to map update data 184. For illustration purposes, the control circuit(s) 181 may control the steering wheel angle and / or the vehicle speed (e.g., engine output) in a time-dependent manner so that the vehicle performs a driving maneuver taking into account the map update data 184. The map update data 184 may be received via a wireless interface of vehicle 180 (e.g., via a cellular communication interface and / or a vehicle-to-vehicle (V2V) interface).

[0232] Although exemplary use cases in which the object matching process may be deployed have been described in detail, object matching may be used in a variety of additional scenarios.

[0233] The techniques disclosed herein provide various technical effects.

[0234] The systems, methods, and devices require little to no domain knowledge. In particular, feature engineering requires little or no domain knowledge. This is a significant improvement over traditional techniques that rely heavily on domain knowledge to perform feature engineering for device-customized and map data format-dependent object matching techniques. Thus, the systems, methods, and devices disclosed herein mitigate errors that may result from using domain knowledge to design and implement feature engineering in a context-dependent manner by eliminating or reducing the need for domain knowledge.

[0235] The systems, methods, and devices may be used for any two maps and any type of context as long as the ML model (e.g., deep learning model) has sufficient example images for training. Different ML models may or may not be used for different digital map types. The generation of the ordered pixel array enables the disclosed systems, methods, and devices to operate without specific retraining of the ML model for the respective formats of the first and second digital maps.

[0236] The systems, methods, and devices are suitable for visualization. Different channels of the tensor may be color-coded and output via a human-machine interface (HMI). When the tensor has more than three channels, the tensor may be flattened into a 3-channel tensor, which may be visualized as an RGB image, for example.

[0237] The systems, methods, and devices may use techniques known from tensor deep learning, where there are many architectures and techniques to create high-quality, scalable predictions from images. By transforming the road network matching problem into an image classification problem, these techniques can be adopted to determine the likelihood that two objects match each other.

[0238] Importantly, compared to traditional matching techniques, the systems, methods, and devices provide higher quality. The systems, methods, and devices can achieve a high precision / recall balance. For illustrative purposes, an average measurement over at least 50,000 labels distributed across 50 countries can achieve a recall rate of 96% and a precision rate of 99%.

[0239] Figure 23 is a graph showing the precision-recall curve of a method according to an embodiment (curve 191) compared to a traditional method (curve 192). The precision of the method according to the embodiment is better than that of the traditional method.

[0240] Although the embodiments have been described with reference to the accompanying drawings, modifications and variations may be implemented in other embodiments.

[0241] For illustrative purposes, although embodiments have been described that can generate and use an array of eight ordered pixel values for object matching for any pair of objects, fewer or greater numbers of ordered pixel value arrays can be generated and input into the ML model for classification.

[0242] For further illustration, although embodiments have been described that can process GPS or other location tracks to generate at least one of the edges in the first digital map and the second digital map, probe track data is not required when generating any digital map.

[0243] Embodiments of the present invention achieve various effects and advantages. For illustrative purposes, the embodiments provide methods, systems, and devices that provide enhanced accuracy for matching objects in different digital maps. The methods, systems, and devices thus provide enhanced techniques that can be used for map merging, navigation operations, driving assistance, or autonomous driving, but are not limited thereto.

Claims

1. A method for processing a first digital map (41; 51) and a second digital map (42; 61), the method comprising the following steps performed by at least one integrated circuit (30): Generating a first ordered pixel value array (71) using the first digital map (41; 51), the first ordered pixel value array (71) representing a first digital map object (52) in a region included in both the first digital map (41; 51) and the second digital map (42; 61), the first digital map object (52) being included in the first digital map (41; 51); Generating a second ordered pixel value array (72) using the second digital map (42; 61), the second ordered pixel value array (72) representing a second digital map object (62) in the region, the second digital map object (62) being included in the second digital map (42; 61); Using a machine learning ML model (31; 120) to determine at least one score indicating the likelihood that the first digital map object (52) corresponds to the second digital map object (62), Among them, The ML model (31; 120) having an input layer (121) that receives at least the first ordered pixel value array (71) and the second ordered pixel value array (72), and Wherein the ML model (31; 120) has an output layer (122) that outputs the at least one score.

2. The method according to claim 1, Among them, All pixel values of the first ordered pixel value array (71) and the second ordered pixel value array (72) are selected from a group including a first set of one or more first pixel values and a second set of one or more second pixel values, wherein the first ordered pixel value array (71) and the second ordered pixel value array (72) are generated such that at a pixel: If the first digital map object (52) overlaps with the pixel, the pixel value in the first ordered pixel value array (71) is a pixel value included in the first set of one or more first pixel values, If the first digital map object (52) does not overlap with the pixel, the pixel value in the first ordered pixel value array (71) is a pixel value included in the second set of one or more second pixel values, If the object overlaps with the pixel, the pixel value in the second ordered pixel value array (72) is a pixel value included in the first set of one or more first pixel values, and If the object does not overlap with the pixel, the pixel value in the second ordered pixel value array (72) is a pixel value included in the second set of one or more second pixel values.

3. The method according to claim 1 or claim 2, Among them, Both the first ordered pixel value array (71) and the second ordered pixel value array (72) are binary pixel value arrays.

4. The method according to any one of the preceding claims, wherein, The input layer (121) also receives A third ordered pixel value array (73) representing one or more additional first digital map objects (59) in the area, the one or more additional first digital map objects (59) being included in the first digital map (41; 51) and being different from the first digital map object (52), and A fourth ordered pixel value array (74) representing one or more additional second digital map objects (69) in the area, the one or more additional second digital map objects (69) being included in the second digital map (42; 61) and being different from the second digital map object (62).

5. The method according to any one of the preceding claims, Among them, The first digital map (41; 51) includes first nodes and first edges (52 - 54) interconnecting the first nodes, wherein the second digital map (42; 61) includes second nodes and second edges (62 - 67) interconnecting the second nodes, wherein the first digital map object (52) is one of the first edges (52 - 54), and wherein the second digital map object (62) is one of the second edges (62 - 67).

6. The method according to claim 5, Among them, The first edges (52 - 54) and the second edges (62 - 67) represent infrastructure objects on which a motor vehicle is allowed to travel, Optionally, wherein the first edges (52 - 54) and the second edges (62 - 67) represent infrastructure objects on which a car is allowed to travel.

7. The method according to claim 5 or claim 6, Among them, The first digital map (41; 51) further includes additional first edges representing infrastructure objects prohibited for use by motor vehicles, wherein the second digital map (42; 61) further includes additional second edges representing infrastructure objects prohibited for use by motor vehicles, wherein the input layer (121) further receives One or more first additional ordered pixel value arrays representing at least one of the additional first edges, and One or more second additional ordered pixel value arrays representing at least one of the additional second edges, Optionally, wherein the additional first edges include bicycle lanes and / or sidewalks in the first digital map (41; 51), and optionally, wherein the additional second edges include bicycle lanes and / or sidewalks in the second digital map (42; 61).

8. The method according to claim 7, Among them, Each of the one or more first additional ordered pixel value arrays, each of the one or more second additional ordered pixel value arrays, the first ordered pixel value array (71), and the second ordered pixel value array (72) respectively consists of X × Y pixel values, wherein X and Y are integers greater than 1, and Wherein, the region is a rectangular region including X×Y pixels, Optionally, wherein Y is equal to X.

9. The method according to any one of the preceding claims, further comprising generating a three-dimensional tensor including at least the first ordered pixel value array (71) and the second ordered pixel value array (72), Among them, the input layer (121) receiving the three-dimensional tensor, Optionally, wherein the size of the three-dimensional tensor is X×Y×Z, where X, Y, and Z are integers, where X and Y are greater than 1, and where Z is equal to or greater than 2, equal to or greater than 4, or equal to or greater than 8.

10. The method according to any one of the preceding claims, Among them, the ML model (31; 120) is a deep learning model (31; 120), and / or wherein the method further comprises: using training data to train the ML model (31; 120), wherein the training data includes labels, wherein, for each of a plurality of sets of the first side and the second side of two different digital training maps, the label indicates whether the first side corresponds to the second side.

11. The method according to any one of the preceding claims, Among them, the first digital map (41; 51) and / or the second digital map (42; 61) includes data generated using detector track data, Optionally, wherein the method further comprises using the at least one score for at least one of the following: determining whether to perform a map update; initiating map merging of the first digital map (41; 51) and the second digital map (42; 61).

12. The method according to any one of the preceding claims, further comprising performing map merging to generate a third digital map from the first digital map (41; 51) and the second digital map (42; 61), Among them, the map merging using the score to determine whether a first digital map object (52) in the first digital map (41; 51) corresponds to a second digital map object (62) in the second digital map (42; 61).

13. The method according to claim 12, wherein, The map merging includes: if the score indicates that the first digital map object (52) corresponds to the second digital map object (62), then combining the attributes of the first digital map object (52) in the first digital map (41; 51) with the attributes of the second digital map object (62) in the second digital map (42; 61).

14. The method according to claim 12 or claim 13, further comprising at least one of the following: Output the third digital map to at least one navigation device (173) for vehicle navigation operations or vehicle control operations, and optionally, wherein, the data of the third digital map is added in an additional map data layer; using the third digital map by at least one server (95) to perform vehicle navigation operations; providing the third digital map for storage in a map data repository (95).

15. The method according to any one of claims 12 to 14, further comprising using the data of the third digital map by at least one navigation device (173) or control circuit (181) on-board the vehicle to perform at least one of the following: Vehicle control operation, optionally, wherein, The vehicle control operation includes an autonomous driving operation or a driving assistance operation; A vehicle navigation operation, optionally, wherein the vehicle navigation operation includes one or several of route search, route guidance, generating a warning or an alarm.

16. A method for providing or updating digital map data of a navigation device, the method comprising: Performing the method according to any one of the preceding claims; Using the at least one score to provide or update at least a part of the digital map data of the vehicle control device (181) or the navigation device (173), Optionally, wherein additional digital map layers are stored in the digital map data of the navigation device, and further optionally, wherein the additional digital map layers include at least some data elements of the third digital map determined by using the method according to any one of claims 12 to 14.

17. A method for processing a first digital map (41; 51) and the second digital map (42; 61), the computing system comprising: A storage system (23) that is operable to store the first digital map (41; 51) and the second digital map (42; 61); and At least one integrated circuit (30) coupled to the storage system (23) and operable to: Generate a first ordered pixel value array (71) using the first digital map (41; 51), the first ordered pixel value array (71) representing a first digital map object (52) in an area included in both the first digital map (41; 51) and the second digital map (42; 61), the first digital map object (52) being included in the first digital map (41; 51); Generate a second ordered pixel value array (72) using the second digital map (42; 61), the second ordered pixel value array (72) representing a second digital map object (62) in the area, the second digital map object (62) being included in the second digital map (42; 61); Using a machine learning ML model (31; 120) to determine at least one score indicating the likelihood that the first digital map object (52) corresponds to the second digital map object (62), Wherein the ML model (31; 120) has an input layer (121) that at least receives the first ordered pixel value array (71) and the second ordered pixel value array (72), and Wherein the ML model (31; 120) has an output layer (122) that outputs the at least one score.

18. The computing system according to claim 17, Among them, The at least one integrated circuit (30) is operable to perform the method according to any one of claims 2 to 16.

19. A machine-readable instruction code that, when executed by a computer system or a server, causes the computer system or the server to perform the method according to any one of claims 1 to 16.

20. A non-transitory storage medium storing machine-readable instruction codes, wherein, When the machine-readable instruction code is executed by a computer system or a server, the computer system or the server is caused to execute the method according to any one of claims 1 to 16.

Images