System and method for correcting map

Abstract

The invention relates to a system and a method for correcting a map. There is provided a method for identifying a change in a road geometry, comprising: obtaining an image of an initial road geometry of a geographic area; obtaining an image of the movement data of the geographic area; forming a composite image from at least the image of the initial road geometry and the image of the movement data; and generating a road geometry corrected image by applying a trained road geometry correction model to the composite image, wherein the road geometry corrected image identifies one or more differences between an actual road geometry of the geographic region and the initial road geometry. A method of training a suitable road geometry correction model is also described.

Description

Technical Field

[0001] The present invention relates to systems and methods for correcting map data, and in particular for identifying discrepancies in stored road geometry. Background Art

[0002] Recent years have seen a dramatic increase in the adoption of electronic navigation aids. Many road vehicles now have integrated navigation and location services from the factory. Such services can provide accurate turn-by-turn navigation assistance, as well as increasingly specific guidance regarding lane changes and specific road layouts. The use of such navigation guidance systems has expanded beyond personal motor vehicles, and such systems are now commonly used by users of many other modes of transportation, including commercial vehicles (e.g., large trucks, commercial passenger vehicles). With advances in the field of autonomous vehicles, such navigation systems are a key technology to enable automated point-to-point driving.

[0003] In addition, as a large number of personal smart devices integrate navigation systems and positioning technologies, users are increasingly relying on such navigation services while using other forms of personal transportation (such as bicycles or even walking).

[0004] In all of these cases, accurate, detailed digital mapping information is essential, especially as the level of guidance provided by these systems becomes more sophisticated. As a result, it is desirable to have accurate road geometry and road layout over large geographic areas. The consequences of incorrect road geometry can lead to significant problems. For example, missing roads cause individual journeys to be suboptimal and not use the most efficient route. On an aggregate level, this can lead to wider issues such as unnecessary traffic or bottlenecks, as incorrect road geometry can affect a large number of people in the same geographic area. Similarly, incorrect road layouts can lead to confusion for vehicle users as they attempt to follow navigation guidance.

[0005] Since roadways and roads are frequently changed and modified, even initially accurate mapping data may become inaccurate within a short period of time. As a result, mapping companies expend a great deal of effort in updating existing mapping data, and in particular road geometry. This may involve manual inspection of flagged errors in road geometry, with cartographers updating mapping data based on several sources of information. This may result in delays in updating mapping data, as errors must first be reported. Likewise, automated (or semi-automated) mapping vehicles are used, which traverse the road network at different times, but this may also result in delays in error identification. Summary of the invention

[0006] The object of the present invention is to provide a system and method for identifying changes (or errors) in road geometry that solves the above-mentioned problems. In particular, it has been recognized in the present invention that position and / or movement data generated by various navigation systems / services themselves can be used to automatically identify errors in stored road geometry. To this end, the present invention provides a novel machine learning system for identifying changes in stored road geometry in a given geographic area by combining the stored road geometry with historical movement data. An image segmentation method is proposed that enables automatic identification of such changes.

[0007] According to a first aspect of the invention, there is provided a method for identifying changes in road geometry, wherein the method comprises: obtaining an image (or pixel map, or bitmap or other raster image) of an initial (or estimated or stored) road geometry of a geographic area; obtaining an image (or pixel map, or bitmap or other raster image) of movement data of the geographic area; forming a composite image (or pixel map, or bitmap or other raster image) from at least the image of the initial road geometry and the image of the movement data; generating a road geometry corrected image (or pixel map, or bitmap or other raster image) by applying a trained road geometry correction model to the composite image, wherein the road geometry corrected image identifies one or more differences between the actual road geometry of the geographic area and the initial road geometry. The movement data may be regarded as vehicle movement data of the geographic area. The trained road geometry correction model is typically or includes a convolutional encoder-decoder neural network. In some embodiments, the image of the initial road geometry may be an image mask.

[0008] It should be understood that the one or more differences may be or include any of the following: a road segment that exists in the actual road geometry and does not exist in the initial road geometry; a road segment that exists in the initial road geometry and does not exist in the actual road geometry; a road segment in the actual road geometry that is displaced relative to the initial road geometry.

[0009] The road geometry correction may further be used to update the initial road geometry. In this manner, in some embodiments, the method may further comprise updating the initial road geometry according to the road geometry corrected image to form an updated road geometry for the geographical area.

[0010] In some embodiments, the composite image includes at least one channel corresponding to the initial road geometry and at least another channel corresponding to movement data.

[0011] In some embodiments, the composite image is formed from the image of initial road geometry, the image of mobility data, and a satellite image of the geographic area.

[0012] The method may further comprise forming another synthetic image from the image of initial road geometry and a satellite image of the area, wherein the trained road geometry correction model is applied to the synthetic image and the another synthetic image to generate the road geometry corrected image. In this way, the accuracy of the road geometry correction may be enhanced by additional information obtainable from the satellite image. More specifically, the trained road geometry correction model may comprise: a first encoder arranged to receive as input a synthetic image comprising at least one channel corresponding to initial road geometry and at least another channel corresponding to movement data; a second encoder arranged to receive as input another synthetic image comprising at least one channel corresponding to initial road geometry and at least another channel corresponding to a satellite image.

[0013] According to a second aspect of the present invention, there is provided a method for training a neural network for identifying road geometry corrections, for example, a neural network embodying a road geometry correction model of the first aspect described above. The method comprises: obtaining a set of known road geometries for a plurality of geographic regions; obtaining a set of images (or pixel maps, or bitmaps or other raster images) of movement data for the plurality of geographic regions; modifying the known road geometries to form a set of modified road geometries; forming a plurality of synthetic images (or pixel maps, or bitmaps or other raster images) from at least the set of modified road geometries and the set of images of movement data, each synthetic image being formed by an image of a corresponding modified road geometry for a corresponding geographic region and the image of movement data for the corresponding geographic region; labeling each synthetic image based on the difference between the modified road geometry of the synthetic image and the corresponding known road geometry to form a set of labeled synthetic images; training a road geometry correction model based on the set of labeled synthetic images, so that the trained road geometry correction model is configured to generate road geometry corrected images as output from input synthetic images of initial road geometry and movement data.

[0014] It should be understood that modifying the known road geometry includes any of: adding road segments to the known road geometry; deleting road segments from the known road geometry; translating road segments of the known road geometry.

[0015] In some embodiments, each synthetic image is formed by an image of a corresponding modified road geometry of a corresponding geographical area, the image of the mobility data of the corresponding geographical area, and a satellite image of the corresponding geographical area. Alternatively, the method comprises, for each synthetic image, forming a corresponding further synthetic image from the image of the corresponding modified road geometry of the corresponding geographical area and a satellite image of the corresponding geographical area. Here, the road geometry correction model may comprise: a first encoder arranged to receive as input a synthetic image comprising at least one channel corresponding to an initial road geometry and at least another channel corresponding to mobility data; a second encoder arranged to receive as input another synthetic image comprising at least one channel corresponding to an initial road geometry and at least another channel corresponding to a satellite image.

[0016] It should be understood that in the embodiments of the first aspect or the second aspect, the trained road geometry correction model may be an image segmentation model. In particular, the trained road geometry correction model may be any one of the following: Unet; Segformer 80; Unet++.

[0017] The movement data used in the above embodiments may include a plurality of historical trips in the geographic area. Here, each element (or pixel) of the image of the movement data may indicate the amount (or number) of historical trips that traverse the corresponding portion of the geographic area corresponding to the element. The image of the vehicle movement data may be a heat map of historical trips in the geographic area.

[0018] It will be appreciated that the above method may further comprise updating (retraining) the trained road geometry correction model based on a plurality of artificially generated road geometry corrections. The plurality of artificially generated road geometry corrections may be further prompted by corresponding road geometry corrections generated using the trained road geometry correction model.

[0019] According to a third aspect of the present invention, there is provided a system suitable for executing the above first aspect or any embodiment thereof.

[0020] To this end, a system for identifying changes in road geometry is provided, the system comprising a memory and one or more processors configured to perform the following steps: obtaining an image (or pixel map, or bitmap or other raster image) of an initial (or estimated or stored) road geometry of a geographic area; obtaining an image (or pixel map, or bitmap or other raster image) of movement data of the geographic area; forming a composite image (or pixel map, or bitmap or other raster image) from at least the image of the initial road geometry and the image of the movement data; generating a road geometry-corrected image (or pixel map, or bitmap or other raster image) by applying a trained road geometry correction model to the composite image, wherein the road geometry-corrected image identifies one or more differences between the actual road geometry of the geographic area and the initial road geometry.

[0021] According to a fourth aspect of the present invention, there is provided a system suitable for executing the above second aspect or any embodiment thereof.

[0022] To this end, a system for training a neural network for identifying road geometry corrections is provided, the system comprising a memory and one or more processors configured to perform the following steps: obtaining a set of known road geometries for multiple geographic areas; obtaining a set of images (or pixel maps, or bitmaps or other raster images) of movement data for the multiple geographic areas; modifying the known road geometries to form a set of modified road geometries; forming a plurality of synthetic images (or pixel maps, or bitmaps or other raster images) from at least the set of modified road geometries and the set of images of movement data, each synthetic image being formed by an image of a corresponding modified road geometry for a corresponding geographic area and the image of movement data for the corresponding geographic area; labeling each synthetic image based on the difference between the modified road geometry of the synthetic image and the corresponding known road geometry to form a set of labeled synthetic images; training a road geometry correction model based on the set of labeled synthetic images, such that the trained road geometry correction model is configured to generate road geometry-corrected images as output from input synthetic images of initial road geometry and movement data.

[0023] According to a fifth aspect of the present invention, a computer program is provided, which, when executed by one or more processors, causes the one or more processors to perform the first aspect or any embodiment thereof. The computer program may be stored on a computer readable medium.

[0024] According to a sixth aspect of the present invention, there is provided a computer program, which when executed by one or more processors causes the one or more processors to perform the second aspect or any embodiment thereof. The computer program may be stored on a computer readable medium. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0026] Figure 1a Describe the geographical area (area or region);

[0027] Figure 1b Schematically illustrate a vehicle and associated navigation client;

[0028] Figure 1c Displaying stored road geometries for a geographic area;

[0029] Figure 2 A system for identifying changes or updates to road geometry is schematically illustrated;

[0030] Figure 3 This indicates that Figure 2 A flowchart of a method performed by a system;

[0031] Figure 4 Schematic illustration of the method used to train a road geometry correction model, such as Figure 2 A training system for the described road geometry correction model;

[0032] Figure 5 This indicates that Figure 4 A flowchart of a method performed by a training system;

[0033] Figure 6 A variant system for identifying changes to road geometry is schematically illustrated;

[0034] Figure 7a and 7b Schematic illustration of the method used to train a road geometry correction model, e.g. Figure 6 A variant system of a variant road geometry correction model of the variant system described in;

[0035] Figure 8 Showing several examples of images that can be used in or generated by embodiments of the present invention;

[0036] Fig. 9 shows other example images used and generated by embodiments of the present invention;

[0037] Fig.10 An example of a computer system is schematically illustrated. DETAILED DESCRIPTION

[0038] In the following description and in the drawings, certain embodiments of the present invention are described. However, it should be understood that the present invention is not limited to the described embodiments, and some embodiments may not include all of the features described below. However, it is apparent that various modifications and variations may be made herein without departing from the broader spirit and scope of the present invention as set forth in the appended claims.

[0039] Figure 1a A geographic area (area or region) 100 is illustrated. The geographic area 100 may be of any size and may include (or represent) part or all of the following: one or more continents and / or one or more countries and / or one or more states and / or one or more counties and / or one or more cities, etc. Figure 1a As shown in FIG. 1 , a geographic area 100 includes a network (or portion thereof) 105 of navigable elements that are located in a Figure 1a 100 is illustrated as corresponding solid lines connecting corresponding locations illustrated as nodes / circles. As discussed above, the navigable elements may be viewed as portions of a transportation network 105 along which mobile units (where the mobile units may be, for example, vehicles, people, etc.) may travel, so that, for example, the navigable elements forming the network 105 may include: one or more roads or portions thereof; and / or one or more routes taken by ferries or trains or portions thereof; and / or one or more paths for pedestrians or portions thereof; and / or one or more bicycle paths or portions thereof; and so on.

[0040] In the following description, for ease of discussion, it will be assumed that the navigable elements are segments of road along which road vehicles can travel. Here, the network 105 will be a road network (or a portion thereof). However, it should be understood that the systems and methods described herein are not limited to road networks and can be applied to bicycle path networks, pedestrian networks, waterways, flight routes, etc., or any combination thereof.

[0041] Figure 1a Also illustrated is an example mobile unit 110, which in this example is a road vehicle. It should be understood that the vehicle 110 can navigate (or travel) on the road network 105 using a suitable navigation system. The navigation system can provide real-time (or turn-by-turn) navigation guidance to allow the vehicle 110 to follow a predefined route. Additionally or alternatively, the predefined route may have been generated using a route generation tool (e.g., which may be present in the navigation system). In some cases, the navigation system may display a visual representation of the road network 105 and an indication of the location (or position) of the vehicle 110 on the road network 105.

[0042] Figure 1b Schematically illustrated is a vehicle 110 and an associated navigation client (or system) 215. The navigation client 215 is connected to a navigation data processing system 240 via the network 210. As illustrated, there may be multiple vehicles 110, each with its own navigation client 215. Although Figure 1b Three such navigation clients 215 are illustrated, but it should be understood that this is merely an example number of navigation clients 215 and that the number, identity, and location of navigation clients 215 may change over time.

[0043] It should be understood that to implement the various functions of the navigation system 215, the map data 241 is generally stored in one or more map databases 244. In addition or alternatively, the map data may be stored locally on the navigation client 215. The navigation client 215 may access or obtain map data from one or more map databases 244. The map data 241 may be in one or more forms or formats, may be one or more corresponding types, and may be suitable for one or more corresponding purposes. The map data 241 generally provides a model or representation of the geospatial reality of the geographic area 100 (or part thereof) including the road network 105. In particular, the map data 241 includes (or represents or encodes) the road geometry of the geographic area 100. The road geometry will be understood as the physical (or geographical) layout of the road segments of the road network 105 in the area 100. In this way, the road geometry in the map data 241 specifies (or represents or otherwise indicates) the shape and position of each road segment. Typically, the road geometry is stored (or represented) as a plurality of center lines of the road segment. The centerline may be (or include or be represented as) a sequence of geospatial coordinates (or points) along the length of the corresponding road segment. This road geometry may be visualized as a plan (or top-down) view of the road, as discussed briefly below. Figure 1b As shown in .

[0044] The map data 241 may also include one or more logical (or schematic) representations of the road network 105, such as in the form of one or more graphics. An example of map data is "SD" map data or "standard definition" map data. This map data represents fixed features (such as road infrastructure features of the road network 105) that are relevant to the vehicle navigation system. The SD map data may include a schematic representation of the road network 105 in the form of a graphical representation, wherein the graph includes arcs and nodes. The graph may be directed or undirected. The arcs are connected by nodes. The arcs and nodes of the SD map are associated or corresponded to the connections of the road segments and the road segments, respectively. For the nodes, the SD map data includes data for the associated point geometry, such as the coordinates corresponding to the location of the node or connection in the geographic area. The arc has an associated road trajectory line, commonly referred to as the road centerline, which represents the geometry of the road segment. In this way, the SD map data stores the road geometry discussed above in the form of the road trajectory line, which can be regarded as equivalent to the centerline of the road segment as discussed above. Thus, the SD map data includes data for the geometry of the arc. The SD map data may store other data associated with the nodes and / or arcs (eg, the speed limit applicable to the road segment).

[0045] Other examples of map data will be known to the skilled person and include:

[0046] “ADAS” map data or “Advanced Driver Assistance System” map data. ADAS map data may be considered a specific type of SD map data. ADAS map data represents fixed features (e.g. road infrastructure features of the road network 105, for example road curvature and road slope) that are relevant for ADAS functions (e.g. predictive cruise control).

[0047] "HD" map data or "High Definition" map data. HD map data may include a graphical representation of a road network, wherein the graph comprises arcs and nodes. The graph may be directed or undirected. Arcs are connected by nodes. The arcs and nodes of an HD map are associated or correspond to road regions and connections of road regions, respectively.

[0048] For some arcs, referred to as road arcs, the associated road region is one or more lanes of the road segment.

[0049] For some arcs, referred to as intersection arcs, the associated area is a road intersection. For nodes, the HD map data includes data of associated point geometry, for example, coordinates corresponding to the location of the node or connection in a geographic area. A road arc has an associated road trajectory (or road centerline) representing the geometry of the road segment, and may have one or more lane trajectory (or lane centerline) representing the geometry of the corresponding lanes of the road segment. The geometry of the intersection area may be specified by one or more intersection arcs. The intersection area may also have one or more intersection arcs corresponding to the geometry of the corresponding permitted maneuvers at the intersection, for example, for each entry lane into the intersection, there may be an intersection arc corresponding to the maneuver to each exit lane from the intersection that can be entered from the entry lane.

[0050] The HD map data includes data on the geometry of the arcs. In this way, it should be understood that the HD map data stores the road geometry discussed above in the form of the centerlines of these arcs.

[0051] The navigation system of the vehicle 110 typically generates probe (or position) data. The probe data 243 generated by the navigation system typically contains location information (e.g., GNSS information including latitude, longitude, and altitude) indicating the geographic location of the vehicle 110 and a time associated with the geographic location. The probe data 243 may contain additional information, such as one or more of the following: an identifier of the navigation system; a speed at which the vehicle 115 is traveling; an indication of the type of vehicle; etc.

[0052] The nature of navigation clients and the probe data they generate and provide are well known and therefore not described in further detail herein.

[0053] The navigation system may be arranged to determine along which road segment the vehicle 110 is currently positioned / traveling, for example in order to display the position of the vehicle 110 along the road segment to a user. It will be appreciated that this may be accomplished by comparing the position information in the probe data 243 with the known road geometry in the map data 241 for the relevant geographic area 100. For example, the navigation system may place (or identify or display) the vehicle 115 at the nearest point on the centerline of the nearest road segment. Doing so may enable the navigation system to provide appropriate turn-by-turn navigation guidance.

[0054] The navigation client 215 is typically arranged to transmit or provide the probe data 243 to the navigation data processing system 240. The navigation data processing system 240 is responsible for obtaining or acquiring the probe data 243 that has been generated and provided by the navigation client 215. The navigation data processing system 240 may store the probe data 243 in a probe data database 242. This stored probe data provides an indication of where a vehicle has been present at a particular location in the geographic area 100. Thus, this stored probe data 243 may be considered an example of movement data for the geographic area 100.

[0055] Figure 1c Stored road geometry 280 for a geographic area is shown. Stored road geometry 280 includes a plurality of road segments 282. Stored road geometry 280 is shown as a top-down (or plan) view, with solid lines indicating centerlines of the road segments. It should be understood that stored road geometry 280 may have been automatically (or semi-automatically) generated based on various data (or information) collected about the road network in the geographic area. This data may include aerial images, satellite images, images generated from a mapping vehicle traversing the road network, existing road maps (e.g., from geographic surveys and highway management agencies), etc. Stored road geometry 280 may have been generated and / or refined by a cartographer based on the data.

[0056] Figure 1c Also illustrated in FIG. 1 is a current (or actual) road geometry 290 for the geographic area 100. The current road geometry 290 includes a plurality of road segments 292. The current road geometry 290 is shown as a top-down (or plan) view, with solid lines indicating center lines of the road segments 292.

[0057] like Figure 1c As can be seen in FIG. 2 , there are several differences between the stored road geometry 280 and the actual road geometry 290. Figure 1c, a road segment 291 present in the actual road geometry 290 is missing in the stored road geometry 280. In addition, several road segments 282 in the stored road geometry 280 are shown as forming a four-way intersection 285, whereas in the actual road geometry 290, these road segments form a staggered intersection 295. It should be understood that the discrepancies or differences between the stored (or initial) road geometry and the actual road geometry may include any of the following:

[0058] Road segments that exist in the actual road geometry and do not exist in the stored road geometry;

[0059] Road segments that exist in the stored road geometry but not in the actual road geometry;

[0060] • Road segments that are displaced in actual road geometry relative to stored road geometry; etc.

[0061] Such differences may be due to errors in generating the stored road geometry. For example, the layout of an intersection may be misidentified by a cartographer. Likewise, such differences may be due to changes in the actual road geometry after the stored geometry was generated. For example, a four-way intersection may be changed to a staggered intersection for safety or traffic flow reasons. New roads (e.g., bypasses) may be laid, and / or existing road segments may be rerouted.

[0062] Figure 2 A system 300 for identifying changes (or updates) to road geometry is schematically illustrated. The system 300 includes a road geometry module 310, a movement data module 320, a synthesis module 330, and a road geometry analysis module 340.

[0063] The road geometry module 310 is arranged to obtain an image 312 (e.g., a raster image) of the initial road geometry 280 of the geographic area 100. As discussed above, the initial road geometry may be considered a representation of the geospatial reality of the road network in the geographic area 100. In particular, the road geometry represents the physical layout of road segments in the geographic area 100. However, as discussed above, the initial road geometry may include one or more differences compared to (or related to) the actual road geometry of the geographic area 100.

[0064] In the context of the present disclosure, the image 312 of the initial road geometry 280 is typically provided to the road geometry module in the form of a pixel map (or bitmap). In other words, the image 312 of the initial road geometry 280 is typically a raster image. The image of the initial road geometry 280 corresponds to the geographic area 100, such that locations in the geographic area correspond to (or are mapped to) locations in the image 312. The image 312 here typically corresponds to an overhead (or plan) representation of the road network and therefore the geographic area 100. It should be understood that the image 312 need not be a true photographic image of the geographic area, nor need the image be displayed or displayable. The image 312 typically represents road segments as lines (e.g., as discussed previously). Figure 1c ). Each line generally follows the path of the road segment. For example, the line may follow the center line of the road segment. To some extent, it should be understood that the image 312 may only include lines representing the road segment.

[0065] Thus, the image may be a single channel image. In other words, the image may include a single value for each pixel. In some cases, the image may be a binary image, i.e., where each pixel is represented (or stored) as a single bit. The image may include (or be in the form of) an array of values, each value indicating whether a road segment exists at a corresponding position in a geographic area.

[0066] The road geometry module 310 may be arranged to obtain an image 312 of initial road geometry from a data storage device, such as one or more map databases 244. Alternatively, the road geometry module may be arranged to generate an image 312 of initial road geometry from map data, such as any of the map data described above. The road geometry module 310 may be arranged to obtain such map data from a data storage device, such as one or more map databases 244. Generating an image of road geometry from map data is a process that should be well known to those skilled in the art, and therefore will not be discussed further herein.

[0067] The mobility data module 320 is arranged to obtain an image 322 (e.g., a raster image) of mobility data for a geographic area. The mobility data may include a plurality of historical trips in the geographic area. Additionally or alternatively, the mobility data may include probe data as described above. Thus, the mobility data may be viewed as providing an indication of the geographic locations of vehicles that have previously traveled in the geographic area during their respective trips in the geographic area. The mobility data may be aggregated data. In particular, the mobility data may indicate the number (or relative number) of vehicles that have traversed a particular geographic location.

[0068] The image 322 of mobile data is typically in the form of a pixel map (or bitmap). In other words, the image 322 of mobile data is typically a raster image. The image 322 of mobile data corresponds to the geographic area 100, so that locations in the geographic area correspond to (or are mapped to) locations in the image 322 of mobile data. To facilitate the synthesis operation briefly described below, the correspondence (or mapping) between locations in the geographic area and locations in the image 322 of mobile data is typically the same as the correspondence (or mapping) between locations in the geographic area and locations in the image 312 of road geometry. In other words, a given pixel location in the image 312 of road geometry and the image 322 of mobile data corresponds to the same location in the geographic area 100.

[0069] Therefore, the image 322 of mobile data is generally a top-down (or plan) representation of the geographic area 100. It should be understood that the image 322 of mobile data does not need to be a real photo image of the geographic area, and the image does not need to be displayed or can be displayed. The image 322 of mobile data indicates the movement of vehicles on the geographic area. In particular, the image 322 of mobile data indicates the historical amount (or flow or number or existence) of vehicles crossing a specific location of the geographic area. Typically, each element (or pixel) of the image 322 of mobile data indicates the amount of historical trips (or vehicles) crossing the corresponding part of the geographic area corresponding to the element. Here, the value of each pixel indicates the amount of vehicles. For example, for a single-channel image, a single value of each pixel can provide vehicle density (or amount). An example of this image would be a grayscale image. Similarly, it should be understood that the image 322 of mobile data may include several channels. For example, the image 322 of mobile data may be a three-channel image (e.g., an RGB image). Thus, the image 322 of mobile data may be in the form of a heat map with a color spectrum representing the range of vehicle amounts. In a specific example, the image 322 of movement data may be a three-channel image, where each channel is a single value. In some cases, the value of each channel may be encoded as a single bit. Here, a value of 1 in the first channel may indicate a "high" relative amount of vehicles traversing the location. A value of 1 in the second channel may indicate a "medium" relative amount of vehicles traversing the location. A value of 1 in the third channel may indicate a "low" relative amount of vehicles traversing the location. Zero-valued pixels (where all channels are 0, or alternatively where all channels are 1) may indicate that no vehicles (or vehicles below a threshold number) have traversed the location.

[0070] Of course, it will be appreciated that more than a single bit may be used for each channel. Typically, in an RGB image, each channel is represented as an 8-bit value (ranging between 0 and 255). In this example, for each corresponding RGB channel, a higher value indicates a high intensity and a lower value indicates a low intensity. A path through the available color spectrum of this image may be mapped to a vehicle intensity. The construction of such a heat map is well known and will not be further described herein.

[0071] Thus, it will be appreciated that an image may comprise a single value for each pixel (e.g., in the grayscale image or single channel image mentioned above). Thus, an image may comprise an array of values ​​(or be in the form of an array of values). Alternatively, an image may comprise multiple values ​​(or channels) for each pixel. Thus, an image may comprise an array of vectors (or be in the form of an array of vectors). The components of each vector are the channels of the image.

[0072] The mobility data module 320 may be arranged to obtain an image 322 of mobility data from a data storage device, such as the probe data database 242. Alternatively, the mobility data module 320 may be arranged to generate an image 322 of mobility data from the mobility data. The road geometry module 310 may be arranged to obtain such mobility data from a data storage device, such as the probe data database 242. Generating a heat map of mobility data or the like from trip and / or probe data is a process that should be well known to those skilled in the art and therefore will not be discussed further herein, see, for example, European Patent 2679956 - Method and Apparatus for Detecting Deviations from MapData.

[0073] The synthesis module 330 is arranged to form a synthesized image 332 from the image 312 of the initial road geometry and the image 322 of the mobile data. As described above, typically, the image 312 of the initial road geometry and the image 322 of the mobile data have a common mapping (or correspondence) to the location of the geographic area. In such cases, the image 312 of the initial road geometry and the image 322 of the mobile data can be directly synthesized to form a synthesized image. Therefore, the synthesized image may include a channel (or multiple channels) of the image 312 of the initial road geometry and a channel (or multiple channels) of the image 322 of the mobile data. In other words, each pixel of the synthesized image includes a channel of the same pixel of the image 312 of the initial road geometry and a channel of the same pixel of the image 322 of the mobile data. For example, when the image 312 of the initial road geometry is a single-channel image and the image 322 of the mobile data is a three-channel image, the resulting synthesized image is a four-channel image.

[0074] Alternatively, one of the image 312 of initial road geometry and the image 322 of movement data may be superimposed on the other using standard image processing techniques to form a composite image. Examples of suitable image processing techniques include any of the following: alpha blending, multiply blending, filter blending, overlay blending, etc.

[0075] Where the image 312 of initial road geometry and the image 322 of movement data do not have a common mapping (or correspondence) to locations of the geographic area 100, the synthesis module 330 is arranged to transform one or both of the images so that the resulting image has a common mapping (or correspondence) to locations of the geographic area 100. This allows the synthesis module 330 to use the resulting images to form a synthesized image in the manner described above.

[0076] Thus, it should be understood that each element (or pixel) of the composite image is based on the corresponding pixel of the image 312 of the initial road geometry and the image 322 of the movement data. In addition, each element (or pixel) of the composite image 332 corresponds to (or is mapped to) a corresponding location in the geographic area 100. As described above with respect to the image 322 of the movement data, the composite image 332 may include (or be in the form of) an array of vectors. The components of each vector are channels of the composite image, which include the channels of the image 312 of the initial road geometry and the image 322 of the movement data.

[0077] The road geometry analysis module 340 is arranged to generate a road geometry correction (or potential or candidate road geometry correction) by applying a trained road geometry correction model 345 to the synthetic image 332. The trained road geometry correction model 345 is arranged to receive the synthetic image 332 as input and produce (or generate) an indication of an error in the initial road geometry as output. For example, the output 342 of the trained road geometry correction model 345 may indicate, for each element (or pixel) of the synthetic image, whether the presence of a road in the initial geometry at the position corresponding to the element is correct.

[0078] The output of the trained road geometry correction model 345 is typically in the form of an output image 342 corresponding to the composite image 332. The value at each pixel of the output image indicates whether the presence of the road in the initial geometry at the location corresponding to the pixel is correct. Thus, the trained road geometry correction model 345 can be considered to perform an image segmentation function (or analysis). In this manner, it should be understood that the output image can take the form of a labeled image or mask.

[0079] The trained road geometry correction model 345 may be or include a convolutional encoder-decoder neural network. As briefly described below, many image segmentation architectures based on convolutional encoder-decoder neural networks can be used for the trained road geometry correction model 345. Thus, the trained road geometry correction model 345 may be or include any of the following: a Unet network; a Segformer 80 network; a Unet++ network; etc. It should be understood that many semantic segmentation deep neural network architectures should be known to the skilled person and can be adopted here.

[0080] In effect, the trained geometry correction model 345 can be viewed as identifying differences (or errors) in the initial road geometry based on information available in the mobility data. For example, where the mobility data shows a large number of vehicles moving along the roads indicated in the initial road geometry, it can be inferred that the location of the roads is incorrect in the initial road geometry and should be shifted to be consistent with the volume of traffic. Similarly, where there is a large amount of traffic in an area where no roads are indicated in the initial road geometry, it can be inferred that roads are missing from the initial road geometry. As part of the training process (described briefly below), relevant patterns in the synthetic images that indicate such scenes are effectively recognized and encoded as part of the trained network so that the trained network can identify misaligned or missing roads or other errors when presented with new (or unseen) synthetic images.

[0081] Typically, the output image 342 includes one or more labeled road segments. The road segments may be labeled according to the values ​​of the pixels that make up the road segments. Thus, the output image may include one or more color-coded road segments. The labels of the road segments may indicate any of the following: a road segment that exists in the actual road geometry and does not exist in the initial road geometry; a road segment that exists in the initial road geometry and does not exist in the actual road geometry; a road segment that is displaced in the actual road geometry relative to the initial road geometry; a road segment that exists in the actual road geometry and exists in the initial road geometry; and the like.

[0082] It should be understood that the output image 342 represents (or encodes or otherwise includes) corrections to the initial road geometry. Therefore, the output image 342 can be used by a subsequent system (or module) to correct (or update) the initial road geometry. In particular, the subsequent system (or module) can add, delete and / or move road segments in the initial geometry as indicated by the output image 342. Additionally or alternatively, the output image 342 can be provided to a user to manually update the road geometry according to the corrections in the output image 342. The updated road geometry can be stored in one or more map databases 244 for future use.

[0083] By using an initial (or existing estimate) of road geometry as one of the inputs to provide corrections to the existing geometry, it will be appreciated that a more accurate set of corrections may be obtained compared to an approach in which, for example, new geometry is obtained directly from the movement data and then compared to the existing estimate of road geometry. In particular, the method of the present invention allows information from an existing estimate of road geometry to be utilized to identify whether the existing estimate is in error. For example, certain errors in the estimated road geometry may be systematic due to, for example, the manner in which the existing estimate was generated. Patterns of such errors may be identified as part of training a road geometry correction model, such that other instances of such errors may be identified by the trained road geometry correction model, even in cases where the corresponding movement data itself may be ambiguous or uncertain.

[0084] Figure 3 This indicates that Figure 2 Flow chart of a method 350 performed by the system 300 .

[0085] At step 352, an image 312 of initial road geometry 280 of geographic area 100 is obtained. As discussed above, image 312 is typically a raster image and represents the geospatial reality of the road network in geographic area 100. Step 352 may include generating a raster image from map data in a particular area stored in a different manner, such as a vector graphics image.

[0086] At step 354, an image 322 of movement data for the geographic area 100 is obtained. As previously discussed, the movement data generally includes (or represents) data (e.g., probe data) indicating vehicle movement over the geographic area. Typically, each element (or pixel) of the image 322 of movement data indicates the amount of historical trips (or vehicles) that traversed the respective portion of the geographic area corresponding to the element.

[0087] At step 356, a composite image 332 is formed from at least the image 312 of initial road geometry and the image 322 of movement data. Typically, the composite image 332 is formed by superimposing (or merging) the image 312 of initial road geometry and the image 322 of movement data. It should be understood that each element (or pixel) of the composite image is based on a corresponding pixel of the image 312 of initial road geometry and the image 322 of movement data.

[0088] At step 358, a road geometry correction is generated by applying the trained road geometry correction model 345 to the synthetic image 332. The trained road geometry correction model 345 is arranged to receive the synthetic image 332 as input and produce (or generate) an indication of errors in the initial road geometry as output. Typically, the output of the trained road geometry correction model 345 indicates, for each element (or pixel) of the synthetic image, whether the presence of the road in the initial geometry at the location corresponding to that element is correct. Thus, the road geometry correction is typically in the form of an image.

[0089] In optional step 360 (which may or may not be performed by system 300), the initial road geometry is updated based on the road geometry correction to form an updated road geometry. In other words, the road geometry correction may be applied to the initial road geometry. It should be understood that in some cases, step 360 may be entirely computer-implemented. Alternatively, step 360 may be (at least partially) performed by a person (e.g., a cartographer). In particular, a manual review of the road geometry correction may be performed. The review may take into account other information to determine (or verify) whether the road geometry correction is correct. The road geometry correction may then be applied in response to (conditional on) the road geometry correction being verified as correct. As will be briefly described below, such verified road geometry corrections (and corresponding initial road geometries) may be stored for further training by the trained road geometry correction model 345.

[0090] Figure 4 Schematic diagram for training a road geometry correction model (e.g., Figure 2 The system 400 includes a training data module 410 and a training module 420.

[0091] The training module 420 is arranged to train a road geometry correction model 445 according to the training data set. The training of the road geometry correction model 445 causes the trained road geometry correction model 345 to be configured to generate a road geometry correction as output from the input synthetic image 332 of the initial road geometry and movement data, as described above with respect to Figure 2As will be appreciated, the road geometry correction model includes a plurality of trainable (or adjustable) parameters that are optimized during training. The specific training process depends on the architecture of the road geometry correction model 445, but as described above, architectures such as Unet; Segformer 80; Unet++, etc. may be used. Typically, for such architectures, adaptive optimizers such as Adam, AdamW, Adagrad, etc. may be used. However, non-adaptive methods such as gradient descent, stochastic gradient descent, etc. may be used. Such training processes are well known to those skilled in the art and are not further described herein.

[0092] The training dataset 415 includes a plurality of synthetic images 332 as described above. For each synthetic image 332, the training dataset 415 further includes a corresponding label. The label indicates an error (or difference) in the road geometry of the corresponding synthetic image 332. In particular, the label takes the form of a desired output of the trained road geometry correction model 345. Thus, the label is typically in the form of an output image, as described above with respect to Figure 2 described.

[0093] The training data module 410 is arranged to generate (or compile) a training data set 415. The training data module 410 may be arranged to receive (or obtain) a set of initial road geometries 411 for a plurality of geographic areas 100 and corresponding labels for each of the initial road geometries. Such initial road geometries 411 are road geometries that have previously had to be corrected. In this case, the labels indicate known corrections made to the road geometries. It will be appreciated that the initial road geometries 411 and their labels may therefore be artificially generated based on previously artificially generated corrections. This data, the initial road geometries 411 and their labels may be referred to as "real" or "real world" training data.

[0094] Additionally or alternatively, the training data module 410 may be arranged to receive (or obtain) a plurality of known road geometries 414 for a plurality of geographic areas 100. Figure 4, the training data module 410 includes an optional geometry modification module 412. The geometry modification module 412 is arranged to modify the known road geometry 414 to form a modified road geometry 418. The geometry modification module 412 may be arranged to modify the known road geometry 414 by any of the following: adding a road segment to the known road geometry; deleting a road segment from the known road geometry; translating a road segment of the known road geometry. Typically, the geometry modification model 412 is arranged to apply a modification (or modifications) to a graphical representation of the road geometry. This allows the geometry modification module 412 to modify the road geometry by adding or deleting or translating (or shifting) one or more edges of the graphic. As discussed above, the road geometry may already be in the form of a graphic. Alternatively, the geometry modification module 412 may be arranged to convert an image of the road geometry 414 into a graphical representation of the road geometry before modification. The geometry modification module 412 may be arranged to convert the graphical representation of the modified road geometry 418 into an image.

[0095] The geometry modification module 412 may be arranged to generate a corresponding label for the modified road geometry 418, which indicates the difference (or error) introduced in the modified road geometry compared to the known road geometry. It should be understood that this label may be generated based on the modification. Likewise, this label may be generated based on a comparison of the modified road geometry to the known road geometry.

[0096] Thus, the geometry modification module 412 may be arranged to generate a set of modified road geometries 418 from the plurality of known road geometries 414. It should be understood that a single known road geometry 414 may produce a plurality of different modified road geometries 418 in the set of modified geometries 418. The set of modified road geometries 418 may also be a set of labeled modified road geometries. This data, the modified road geometries and their labels, may be referred to as "synthetic" training data.

[0097] For ease of discussion in the following description, the term "a set of road geometries" will be used to encompass both the modified road geometries 418 (i.e., synthetic data) and the initial road geometries 411 (i.e., real-world data). It should be understood that the "a set of road geometries" may include exclusively real-world data, or exclusively synthetic data, or a mixture of both.

[0098] The training data module 410 includes the movement data module 320 . Figure 2 The above discussion of the mobile data module 320 shown in FIG. 1 is also applicable to Figure 4. In particular, the mobile data module 320 is arranged to obtain respective images 322 of mobile data for each of the geographic areas represented in the set of road geometries. These images of mobile data may be referred to as a set of images 322 of mobile data. Figure 2 The discussion of the image of mobile data 322 set forth is equally applicable to the images of mobile data in the set of images of mobile data herein.

[0099] The training data module 410 includes the synthesis module 330 . Figure 2 The above discussion of the synthesis module 330 shown in FIG. 1 is also applicable to Figure 4 . In particular, the synthesis module 330 is arranged to form a plurality of synthetic images 332 from at least the set of images 322 of the set of road geometry and movement data. Thus, each synthetic image 332 is formed by the synthesis module 330 from an image of modified road geometry for a given geographic area and an image of movement data for a given geographic area. Figure 2 The discussion set forth for composite image 332 applies equally to the composite images herein.

[0100] The training data module 410 is arranged to label the plurality of synthetic images 332. In particular, the training data module 410 is arranged to apply (or associate or otherwise link) each synthetic image 332 with a label of a corresponding road geometry. In other words, labels of the road geometry forming the synthetic image 332 are associated with the synthetic image. Each label is generally as described above with respect to Figure 2 The form of the output image described. It should be understood that the training data module 410 can be arranged to generate the labels and perform the association in a single step (or action). A plurality of labeled synthetic images form the training data set 415 (or at least a portion thereof).

[0101] In this manner, it will be appreciated that the training data module 410 is arranged to generate a training data set 415 comprising a plurality of synthetic images 332 of erroneous road geometry and movement data, and associated labels, typically in the form of images indicating errors (or discrepancies) in the road geometry depicted in the synthetic images. In other words, the training data set 415 comprises the expected inputs (e.g., synthetic images) and the expected (or desired) outputs (e.g., information about the road geometry correction model 345) of the trained road geometry correction model 345. Figure 2 ). Thus, the training module 420 may be arranged to train the road geometry correction model 445 using the training data set generated by the training data module 410 using standard supervised learning techniques.

[0102] It should be understood that the known road geometries discussed above may not be completely accurate and may themselves contain unknown or unidentified errors. However, since known errors are then incorporated into these geometries and these are marked in the training data set, their use will still benefit the training. In addition, the use of such "synthetic" training data enables initial training of the road geometry correction model 345 in situations where there may not be a sufficient amount of real training data available. It should be understood that during the use of the trained road geometry correction model 345, other real training data may be acquired. In particular, in the case where the errors identified by the trained road geometry correction model 345 are subject to manual inspection and further investigation, the errors in the road geometry subsequently identified (or verified) can be used as additional real training data. Then, the other collected real training data can be used to retrain (or refine or update) the trained road geometry correction model 345, thereby improving the accuracy of the trained road geometry correction model 345.

[0103] Figure 5 This indicates that Figure 4 Flowchart of method 500 performed by training system 400.

[0104] At step 510, a set of known road geometries for a plurality of geographic regions is obtained. Step 510 may include multiple iterations of step 352 described above.

[0105] At step 520, a set of images of mobility data for a plurality of geographic areas is obtained. Step 520 may include multiple iterations of step 354 described above. In particular, step 520 includes obtaining a respective image of mobility data for each of the geographic areas represented in the set of road geometries.

[0106] At step 530, the known road geometries are modified to form a set of modified road geometries. Specifically, step 530 may include modifying each known road geometry by any one or more of: adding a road segment to the known road geometry; deleting a road segment from the known road geometry; translating a road segment of the known road geometry. It should be understood that a single known road geometry may produce multiple different modified road geometries in the set of modified geometries.

[0107] At step 540, a plurality of composite images are formed from at least the set of images of the set of modified road geometry and movement data.Step 540 may include performing step 356 for each respective image of modified road geometry and movement data.

[0108] At step 550, a set of labeled synthetic images is formed. Step 550 includes, for each synthetic image, labeling the synthetic image based on differences between the modified road geometry of the synthetic image and the corresponding known road geometry to form a corresponding labeled synthetic image. Step 550 includes applying a label that generally includes (or associates or otherwise links) each synthetic image with the corresponding modified road geometry.

[0109] At step 560, a road geometry correction model is trained based on the set of labeled synthetic images. Thus, the labeled synthetic images can be considered as a training data set as discussed above. The training of step 560 causes the trained road geometry correction model to be configured to generate road geometry corrected images as output from the input synthetic images of initial road geometry and movement data.

[0110] Thus, the resulting trained road geometry correction model may be used to identify changes to the initial road geometry, such as as part of the system 300 or method 350 described above.

[0111] It should also be understood that the training method 500 may take the form of a continuous or continuous training method. In this manner, there may be an optional step 570 comprising identifying a plurality of road geometry corrections using the trained road geometry correction model for a plurality of initial road geometries. This identification may be performed, for example, in accordance with the method 350 described above. As set forth in optional step 360, the identified road geometry corrections may be subject to further manual verification. In this manner, a set of verified road geometry corrections may be generated. Such verified road geometry corrections will be road geometry corrections generated by the trained road geometry correction model and manually verified to be correct. Manual verification may also include manual modification of the road geometry corrections to include further (or updated) corrections.

[0112] At optional step 580, each initial road geometry may be labeled with a corresponding verified road geometry correction and added to (or included in) the training dataset. In this manner, step 580 may include updating the set of labeled synthetic images based on the verified road geometry corrections. Method 500 may then return to step 560, where the road geometry correction model is retrained based on the updated set of labeled synthetic images (i.e., updated training dataset). It should also be understood that additional labeled road geometries (e.g., the real-world data described above) may also be included in the updated training dataset as part of step 570.

[0113] Figure 6 A system 600 for identifying changes (or updates) to road geometry is schematically illustrated. The system 600 is a system described above with respect to Figure 2The system 600 includes a road geometry module 310, a mobility data module 320, a synthesis module 330, and a road geometry analysis module 340. Figure 2 The description set forth applies equally to system 600 and its components, except as noted below.

[0114] The variant system 600 further includes a photo image module 650. The photo image module is arranged to obtain a photo image 652 of the geographic area 100. The photo image 652 will typically be a top view (or aerial view) of the geographic area 100. It should be understood that although visible light photography will typically be used to generate the photo image 652, other types of photography, such as infrared photography or thermal photography, can be used. The photo image 652 can be any of the following: an aerial photograph; a satellite photograph (or image); and an elevated (or high-rise or mast) photograph. It should be understood that the photo image 652 may include multiple images that have been stitched together to form a single image. Various other types of (pre) processing may have been applied to the image before the image is obtained and / or may be applied to the image once the image is obtained. The photo image 652 is a raster image. Thus, the photo image 652 typically includes one or more channels as previously discussed with respect to other images.

[0115] The photo image module 650 is typically arranged to obtain a photo image 652 of the geographic area 100 from a suitable image storage device. It should be understood that there may be multiple photo images 652 of the geographic area 100 available to the photo image module 650. Thus, the photo image module 650 may be arranged to select the photo image 652 based on one or more criteria. The one or more criteria may include: the time at which the photo image was obtained (or taken); the resolution of the photo image; the time of day at which the photo image was obtained (or taken); a measure of the image quality of the photo image 652 (e.g., a measure of focus, a measure of exposure, etc.).

[0116] A criterion that may be particularly important is the level of occlusion in the photographic image 652. It will be appreciated that for some photographic images (e.g., high altitude aerial images or satellite images), atmospheric phenomena (e.g., clouds) may obscure some or all geographic areas in the image. Thus, the photographic image module 650 may be arranged to discard (or not select) photographic images 652 that have an occlusion level above a threshold.

[0117] It should be appreciated that the photo image available to photo image module 650 may include (or capture) an area larger than geographic area 100. Thus, photo image module 650 may be arranged to crop the photo image to geographic area 100 when obtaining photo image 652.

[0118] As with the system 300 described above, the synthesis module 330 is arranged to form the synthesized image 332 from the image 312 of the initial road geometry and the image 322 of the movement data. However, in the variant system 600, the synthesized image is also formed from the photographic image 652. Thus, in the variant system 600, the synthesis module 330 is arranged to form the synthesized image 332 from the image 312 of the initial road geometry, the image 322 of the movement data, and the photographic image 652.

[0119] As described above with respect to the image of mobile data, typically, the image 312 of initial road geometry and the photo image 652 have a common mapping (or correspondence) to the location of the geographic area. In such cases, the image 312 of initial road geometry, the image 322 of mobile data, and the photo image 652 can be directly synthesized to form a composite image.

[0120] In the event that the image 312 of the initial road geometry and the photographic image 652 do not have a common mapping (or correspondence) to the location of the geographic area 100, the synthesis module 330 is arranged to transform one or both of the images so that the resulting image has a common mapping (or correspondence) to the location of the geographic area 100. This allows the synthesis module 330 to use the resulting image to form the synthesized image 632.

[0121] Thus, the composite image 632 may include a channel (or channels) of the image 312 of the original road geometry, a channel (or channels) of the image 322 of the movement data, and a channel (or channels) of the photo image 652. In other words, each pixel of the composite image includes a channel of the same pixel of the image 312 of the original road geometry, a channel of the same pixel of the image 322 of the movement data, and a channel of the same pixel of the photo image 652.

[0122] Alternatively, the image 312 of initial road geometry, the image 322 of motion data, and the photographic image 652 may be superimposed on one another using standard image processing techniques to form the composite image 632. Examples of suitable image processing techniques include any of: alpha blending, multiply blending, filter blending, overlay blending, etc.

[0123] As with system 300, road geometry analysis module 340 is arranged to generate road geometry corrections by applying trained road geometry correction model 345 to synthetic image 332. Trained road geometry correction model 345 is arranged to receive synthetic image 332 as input and produce (or generate) indications of errors in initial road geometry as output. However, in this variation, the synthetic image is also formed from photographic image 652, and road geometry correction model 345 is trained accordingly, as briefly described below.

[0124] Thus, in this variation, the trained geometry correction model 345 can actually be viewed as identifying differences (or errors) in the initial road geometry based on the information available in the mobile data and the photo image. For example, where the photo image shows a road-like structure where no road exists in the road geometry, the trained geometry correction model 345 can identify this as a missing road. As part of the training process (described briefly below), relevant patterns in the synthetic image indicating such scenes are effectively identified and encoded as part of the trained network, so that the trained network can identify misaligned or missing roads or other errors when presented with new (or unseen) synthetic images.

[0125] It will be appreciated that by including satellite imagery data in the input to the road geometry correction model, the corrective power of the model may be improved in areas where minimal mobile data exists (e.g., seldom used roads or low population areas). Likewise, satellite imagery data may aid correction identification for certain categories of corrections, such as roads to be deleted. Specifically, in the case of a road to be deleted, the mobile data will provide an indication due to the lack of mobile data in the location of the road, however, this may also only indicate a road that is seldom used (but still exists). Thus, satellite imagery showing the presence (or absence) of such a road provides further certainty as to whether a correction is needed.

[0126] I'm here too Figure 6 In another variant system 600a shown in FIG. 1 , the synthesis module 330 is arranged to form a first synthesized image 332 (eg, as described above with respect to FIG. 1 ) from the image 312 of the initial road geometry and the image 322 of the movement data. Figure 3 The synthesis module 330 is also arranged to form a second synthetic image 634 from the image 312 of initial road geometry and the photographic image 652. The second synthetic image 632 may be formed in the same manner as the first synthetic image, but using the photographic image 652 instead of the image 322 of movement data.

[0127] In this further variation, the road geometry analysis module 340 is arranged to provide both the first and second synthetic images as separate but simultaneous inputs to the variant trained road geometry correction model 645. In other words, in this further variation, the variant trained road geometry correction model 645 is arranged to receive two inputs, namely the first synthetic image and the second synthetic image. As previously described, the trained road geometry correction model 345 is typically (or includes) a convolutional encoder-decoder neural network. Thus, the inputs of the trained road geometry correction model 345 are typically received (or processed) by an encoder (or multiple encoder layers). In the trained road geometry correction model 645 of this variation, there are two encoders 647; 648. The first encoder 647 is arranged to receive a synthetic image (e.g., the first synthetic image 332) formed by an image of the initial road geometry of the geographic area and an image of the mobile data of the geographic area. The second encoder 648 is arranged to receive a synthetic image (e.g., the second synthetic image 634) formed by an image of the initial road geometry of the geographic area and an image of a photograph of the geographic area. Thus, in this other variation, the variant trained road geometry correction model 645 may be (or include or be implemented as) a multi-branch N-encoder 1-decoder convolutional neural network. In this example, N is 2, but it should be understood that in some cases, the above-mentioned other variation may generate other composite images in which the image of the road geometry is superimposed on (or on) other photographic images of the geographic area. For example, if there are multiple sources of photographic images (such as multiple sources of satellite images), this may be useful. In such cases, the variant trained road geometry correction model 645 will accordingly include a corresponding another encoder for each other composite image. It should be understood that, except for these differences, the discussion of the trained road geometry correction model 345 outlined above is equally applicable to the variant trained road geometry correction model 645.

[0128] In this multi-branch N-encoder 1-decoder convolutional neural network, the output of the encoder is typically convolved according to a predetermined algorithm before being operated on by the decoder layer. An example of this multi-branch N-encoder 1-decoder convolutional neural network is ME-Net, described in "ME-Net: Multi-Encoder Net Framework for Brain Tumor Segmentation", Zhang et al., arXiv:2203.11213, https: / / doi.org / 10.48550 / arxiv.2203.11213.

[0129] It will be appreciated that by using this multiple encoder type architecture, one or more satellite images (or other input data) may be provided to the road geometry correction module while more easily allowing for situations where one (or more, or all) of the satellite images (or other data) are not available. In particular, with respect to satellite images, sometimes satellite imagery for a particular area is not available, or is very old or outdated. In these cases, the input to that particular encoder may simply be omitted, and corrections may be generated based on the available data. This is in contrast to a single encoder that is trained to receive a certain number of satellite images as part of the input, where it may be necessary to generate a suitable virtual input in place of the available images.

[0130] Figure 7a A system 700 for training a road geometry correction model (e.g., a modified road geometry correction model of the modified system 600) is schematically illustrated. The system 700 is a system described above with respect to Figure 4 A variation of the system 400 described above. The variation training system 400 includes a training data module 410 and a training module 420. Figure 4 The description set forth applies equally to the variation training system 700 and its components, except as noted below.

[0131] The training data module 410 of the variant training system 700 further includes a photo image module 650 . Figure 6 The above discussion of the photo image module 650 shown in FIG. 1 is equally applicable to the photo image module 650 shown here. The photo image module 650 is arranged to obtain a corresponding photo image 652 of each of the geographic areas represented in the set of road geometries. These photo images 652 may be referred to as a set of photo images 652. Figure 6 The discussion of the photographic images set forth herein also applies to the photographic images 652 in the set of photographic images herein.

[0132] The training data module 410 includes the synthesis module 330 . Figure 6 The above discussion of the synthesis module 330 shown in also applies to the synthesis module 330 shown here. In particular, the synthesis module 330 is arranged to form a plurality of synthetic images 632 from at least the set of road geometries 418; 411, the set of images 322 of mobile data, and the set of photographic images 652. As with system 400, module 412 is optional, and the "set of road geometries" may be modified road geometries 418 (i.e., synthetic data) or initial road geometries 411 (i.e., real-world data), or both. Thus, each synthetic image 632 is formed by the synthesis module 330 from an image of road geometry for a given geographic area, an image of mobile data for a given geographic area, and a photographic image for a given geographic area. The above discussion of Figure 6The discussion of composite image 632 described above applies equally to composite image 632 herein.

[0133] Thus, training data module 410 of variant training system 700 is arranged to generate training data set 715 in the same manner as training data module 410 of training system 400, except that synthetic images 632 in training data set 715 are synthetic images formed from images of road geometry, images of mobility data, and photographic images.

[0134] In this manner, it should be understood that by using this variant training dataset 715, the training module 420 is arranged to train the road geometry correction model 445, such that the trained road geometry correction model 345 is configured to generate a road geometry correction as output from the input composite image 635 of initial road geometry, movement data, and photographic images. It should be understood that the training techniques described previously are equally applicable to training using the variant training dataset 715.

[0135] exist Figure 7b In another variant training system 700a shown in , the training module 420 is arranged to train another variant road geometry correction model 645 of another variant system 600a. As described above, the another variant road geometry correction model 445 includes two or more encoders 647;648.

[0136] In the training data module 410 of another variant training system 700a, the synthesis module 330 is arranged to form a first synthesized image 332 (eg, as described above with respect to the training data module 410) from the image 312 of the initial road geometry and the image 322 of the movement data. Figure 3 The synthesis module 330 is also arranged to form a second synthesized image 634 from the image 312 of the initial road geometry and the photographic image 652. The second synthesized image 634 may be formed in the same manner as the first synthesized image, but using the photographic image 652 instead of the image 322 of the movement data.

[0137] Thus, the training data module 410 is arranged to generate another variant training data set 795 in the same manner as the training data module 410 of the training system 400, with the difference that for each synthetic image 332 formed from an image 312 of road geometry and an image of movement data, the variant training data set comprises a corresponding second synthetic image formed from said image 312 of road geometry and a corresponding photographic image 652.

[0138] In this further variation, the training module 420 is arranged to train a variant road geometry correction model 745 using a variant training data set 795. The training module is thereby arranged to form a variant trained road geometry correction model 645 as discussed above. As will be appreciated, the variant road geometry correction model 745 comprises two encoders 647; 648. The first encoder 647 is used to receive a composite image (e.g., the first composite image 332) formed by an image of the initial road geometry of the geographic area and an image of the movement data of the geographic area. The second encoder 648 is used to receive a composite image (e.g., the second composite image 652) formed by an image of the initial road geometry of the geographic area and an image of the photograph of the geographic area. It will be appreciated that the previously described training techniques are equally applicable to training using another variant training data set 795.

[0139] Figure 8 Several examples of images that may be used in or generated by embodiments of the systems and methods described above are shown.

[0140] Figure 8 Four images 822-1; 822-2; 822-3; 822-4 of mobile data are shown in FIG, each for a different geographic region. The images 822-1; 822-2; 822-3; 822-4 of mobile data have been generated from detection data collected by a navigation system in a vehicle traversing a road network in each geographic region. The images 822-1; 822-2; 822-3; 822-4 of mobile data are heat maps and are 512×512 (although it should be understood that larger or smaller image sizes may be used) pixel raster RGB images. The images 822-1; 822-2; 822-3; 822-4 of mobile data include three one-bit binary channels corresponding to red, green, and blue, respectively. The color of each pixel in the images 822-1; 822-2; 822-3; 822-4 of mobile data represents the density of detections at a given location. Here red represents high density, green represents medium density, and blue represents low density. White areas are where no detections were recorded. It should be understood that for the reproduction here, the RGB images are shown as grayscale.

[0141] Four corresponding images 812-1; 812-2; 812-3; 812-4 of the initial road geometry are also shown. Each image 812-1; 812-2; 812-3; 812-4 of the initial road geometry shows the same geographic area as the corresponding image 822-1; 822-2; 822-3; 822-4 of the movement data. Thus, the images 812-1; 812-2; 812-3; 812-4 of the initial road geometry are 512×512 pixel raster images, each image comprising a single one-bit channel. In the images 812-1; 812-2; 812-3; 812-4 of the initial road geometry, black pixels are where road segments exist, and white pixels are where road segments do not exist.

[0142] Four corresponding synthetic images 832-1; 832-2; 832-3; 832-4 are shown. These synthetic images are generated from pairs of images 822-1; 822-2; 822-3; 822-4 of movement data and images 812-1; 812-2; 812-3; 812-4 of initial road geometry as previously described. For ease of graphical reproduction, the synthetic images are shown as alpha blends of the corresponding images of movement data and the image of initial road geometry. However, it should be understood that in this particular example, the synthetic images are actually four channel 512×512 pixel raster images. In other words, each synthetic image includes three channels of images of movement data and a single channel of images of initial road geometry.

[0143] Output images 842-1; 842-2; 842-3; 842-4 are generated from each synthetic image 832-1; 832-2; 832-3; 832-4 using the implementation of the system 300 described above. In this particular implementation, the trained road geometry correction model 345 is a Unet++ image segmentation convolutional neural network trained using the Pytorch deep learning framework. The output images 842-1; 842-2; 842-3; 842-4 are 512×512 pixel RGB raster images, each image including three one-bit channels. Red lines (or pixels thereof) 843 indicate locations where road segments are missing from the corresponding initial road geometry. Green lines (or pixels thereof) 844 indicate new locations where existing roads in the initial geometry should be moved, i.e., because the roads were incorrectly positioned in the initial road geometry.

[0144] From the visual comparison between the images 812-1; 812-2; 812-3; 812-4 of the initial road geometry and the images 822-1; 822-2; 822-3; 822-4 of the mobile data, it can be seen that the output image accurately identifies errors in the road geometry based on the large detection density that occurs when there is no corresponding road segment. The output image also correctly ignores stray tracks or signals in the mobile data that may be caused by GPS errors and / or occasional use of vehicles that are not on the designated road.

[0145] Fig. 9 Other example images used and generated by other implementations of the above-described systems and methods are shown.

[0146] Fig. 9 8 shows several images 822 of movement data, corresponding images 812 of initial road geometry, and corresponding composite images 832. The discussion of images of movement data, corresponding images of initial road geometry, and corresponding composite images also applies to the images shown here.

[0147] In this case, three different trained road geometry correction models were tested. These are:

[0148] Unet image segmentation convolutional neural network; it generates the first set of output images 942-1

[0149] Segformer B0 image segmentation convolutional neural network; it generates the second set of output images 942-2

[0150] Unet++ image segmentation convolutional neural network; which generates a third set of output images 942-3.

[0151] as Figure 8 , the output images 942-1; 942-2; 942-3 shown here are generated by applying a particular trained road geometry correction model to the corresponding synthetic image 832. The output images 942-1; 942-2; 942-3 are 512×512 pixel RGB raster images, each image including three one-bit channels. Red lines (or pixels thereof) 843 indicate locations where road segments are missing from the corresponding initial road geometry. Green lines (or pixels thereof) 844 indicate new locations where existing roads in the initial geometry should be moved, i.e., because the roads were incorrectly positioned in the initial road geometry.

[0152] Again as can be seen from the visual comparison between the images, the output image accurately identifies the errors in the road geometry.

[0153] The road geometry correction models are trained using a training set of synthetic modified road geometries. Another validation set of synthetic modified road geometries is also used to validate each trained geometry correction model. The F1 score (i.e., the harmonic mean of precision and recall) is calculated for each trained model. The Jaccard index (or intersection over union score, IoU) is also calculated for each trained model.

[0154]

[0155] Table 1

[0156] As can be seen from Table 1, each of the tested image segmentation models achieves acceptable validation scores, with Unet++ being the preferred image segmentation model.

[0157] Fig.10 An example of a computer system 1000 is schematically illustrated. System 1000 includes a computer 1020. Computer 1020 includes storage media 1040, memory 1060, processor 1080, interface 1100, user output interface 1120, user input interface 1140, and network interface 1160, all linked together by one or more communication buses 1180.

[0158] The storage medium 1040 may be any form of non-volatile data storage device, such as one or more of a hard drive, a magnetic disk, an optical disk, a ROM, etc. The storage medium 1040 may store an operating system for execution by the processor 1080 to enable the computer 1020 to operate. The storage medium 1040 may also store one or more computer programs (or software or instructions or codes).

[0159] The memory 1060 may be any random access memory (storage unit or volatile storage medium) suitable for storing data and / or computer programs (or software or instructions or code).

[0160] The processor 1080 may be any data processing unit suitable for executing one or more computer programs (e.g., computer programs stored on the storage medium 1040 and / or in the memory 1060), some of which may be computer programs according to embodiments of the present invention or computer programs that, when executed by the processor 1080, cause the processor 1080 to perform a method according to embodiments of the present invention and configure the system 1000 as a system according to embodiments of the present invention. The processor 1080 may include a single data processing unit or a plurality of data processing units operating in parallel or in cooperation with each other. When performing data processing operations for embodiments of the present invention, the processor 1080 may store data to the storage medium 1040 and / or the memory 1060 and / or read data from the storage medium 1040 and / or the memory 1060.

[0161] The interface 1100 may be any unit for providing an interface to a device 1220 that is external to or removable from the computer 1020. The device 1220 may be a data storage device, such as one or more of an optical disk, a magnetic disk, a solid-state storage device, etc. The device 1220 may have processing capabilities, for example, the device may be a smart card. Therefore, the interface 1100 may access data from the device 1220, provide data to the device 1220, or interface with the device 1220 according to one or more commands it receives from the processor 1080.

[0162] The user input interface 1140 is arranged to receive input from a user or operator of the system 1000. The user may provide such input via one or more input devices of the system 1000, such as a mouse (or other pointing device) 1260 and / or a keyboard 1240, connected to or in communication with the user input interface 1140. However, it should be understood that the user may provide input to the computer 102 via one or more additional or alternative input devices, such as a touch screen. The computer 1020 may store input received from the input devices via the user input interface 1140 in the memory 1060 for subsequent access and processing by the processor 1080, or may pass it directly to the processor 1080 so that the processor 1080 can respond to the user input accordingly.

[0163] The user output interface 1120 is arranged to provide graphical / visual and / or audio output to a user or operator of the system 1000. Thus, the processor 1080 may be arranged to instruct the user output interface 1120 to form an image / video signal representing the desired graphical output, and to provide this signal to a monitor (or screen or display unit) 1200 of the system 1000 connected to the user output interface 1120. Additionally or alternatively, the processor 1080 may be arranged to instruct the user output interface 1120 to form an audio signal representing the desired audio output, and to provide this signal to one or more speakers 1210 of the system 1000 connected to the user output interface 1120.

[0164] Finally, network interface 1160 provides computer 1020 with the ability to download data from and / or upload data to one or more data communications networks.

[0165] It should be understood that Fig.10 The architecture of the system 1000 illustrated in FIG. 1 and described above is merely exemplary, and systems with different architectures (e.g., having a larger Fig.10 Fewer components or features shown in Fig.10) with additional and / or alternative components shown in the . As an example, the computer system 1000 may include one or more of the following: a personal computer; a server computer; a mobile phone; a tablet computer; a laptop computer; a television; a set-top box; a game console; other mobile devices or consumer electronic devices; an in-car entertainment system; an in-car navigation system, etc.

[0166] about Figure 2 , 4 The systems described in , 6, 7a and 7b may each be implemented as one or more computer systems (or executed together with) such as the system 1000 described above. Similarly, the navigation client mentioned above may be implemented as one or more computer systems (or executed together with) such as the system 1000.

[0167] It should be understood that the described methods have been shown as individual steps performed in a particular order. However, those skilled in the art will appreciate that these steps may be combined or performed in a different order while still achieving the desired results.

[0168] It should be understood that embodiments of the present invention can be implemented using various information processing systems. Specifically, although the figures and their discussion provide exemplary computing systems and methods, these are only presented to provide useful references when discussing various aspects of the present invention. Embodiments of the present invention can be performed on any suitable data processing device, such as a personal computer, a laptop computer, a personal digital assistant, a mobile phone, a set-top box, a television, a server computer, etc. Of course, for the purpose of discussion, the description of the system and method has been simplified, and they are only one of the many different types of systems and methods that can be used for embodiments of the present invention. It should be understood that the boundaries between the logic blocks are only illustrative, and alternative embodiments can merge logic blocks or elements, or can apply alternative decompositions of functions to various logic blocks or elements.

[0169] It should be understood that the above functions may be implemented as one or more corresponding modules as hardware and / or software. For example, the above functions may be implemented as one or more software components for execution by a processor of the system. Alternatively, the above functions may be implemented as hardware, such as on one or more field programmable gate arrays (FPGAs), and / or one or more application specific integrated circuits (ASICs), and / or one or more digital signal processors (DSPs), and / or other hardware arrangements. The method steps implemented in the flowcharts contained herein or as described above may each be implemented by a corresponding corresponding module; multiple method steps implemented in the flowcharts contained herein or as described above may be implemented together by a single module.

[0170] It should be understood that as long as the embodiments of the present invention are implemented by a computer program, the storage media and transmission media carrying the computer program form aspects of the present invention. A computer program may have one or more program instructions or program codes that, when executed by a computer, perform embodiments of the present invention. As used herein, the term "program" may be a sequence of instructions designed to be executed on a computer system, and may include subroutines, functions, procedures, modules, object methods, object implementations, executable applications, applets, servlets, source code, object code, shared libraries, dynamic link libraries, and / or other sequences of instructions designed to be executed on a computer system. The storage medium may be a disk (such as a hard drive or floppy disk), an optical disk (such as a CD-ROM, DVD-ROM, or a Blu-ray disk), or a memory (such as a ROM, RAM, EEPROM, EPROM, flash memory, or a portable / removable storage device), etc. The transmission medium may be a communication signal, a data broadcast, a communication link between two or more computers, etc.

Claims

1. A method for identifying changes in road geometry, the method comprising: obtaining an image of initial road geometry for a geographic area; obtaining an image of the mobile data for the geographic area; forming a composite image from said image of at least initial road geometry and said image of movement data; A road geometry-corrected image is generated by applying a trained road geometry correction model to the synthetic image, wherein the road geometry-corrected image identifies one or more differences between an actual road geometry of the geographic area and the initial road geometry.

2. The method of claim 1, wherein the trained road geometry correction model comprises a convolutional encoder-decoder neural network.

3. The method of claim 1 or 2, wherein the composite image comprises at least one channel corresponding to the initial road geometry and at least another channel corresponding to movement data.

4. The method of claim 1 or 2, wherein the composite image is formed from the image of initial road geometry, the image of mobility data, and a satellite image of the geographical area.

5. The method according to claim 1 or 2, wherein the method comprises forming a further composite image from the image of initial road geometry and a satellite image of the area, wherein the trained road geometry correction model is applied to the composite image and the further composite image to generate the road geometry corrected image, optionally wherein the trained road geometry correction model comprises: a first encoder arranged to receive as input a composite image comprising at least one channel corresponding to an initial road geometry and at least another channel corresponding to movement data; A second encoder is arranged to receive as input another composite image comprising at least one channel corresponding to the initial road geometry and at least another channel corresponding to the satellite image.

6. The method of claim 1 or 2, wherein the image of initial road geometry is an image mask.

7. The method of claim 1 or 2, wherein the one or more differences include any of the following: a road segment that exists in the actual road geometry and does not exist in the initial road geometry; a road segment that exists in the initial road geometry and does not exist in the actual road geometry; A road segment in the actual road geometry that is displaced relative to the initial road geometry.

8. The method of claim 1 or 2, wherein the method further comprises updating the initial road geometry according to the road geometry corrected image to form an updated road geometry for the geographic area.

9. A method of training a neural network for identifying road geometry corrections, the method comprising: obtaining a set of known road geometries for a plurality of geographic areas; obtaining a set of images of mobility data for the plurality of geographic areas; modifying the known road geometries to form a set of modified road geometries, forming a plurality of composite images from at least the set of modified road geometry and the set of images of the movement data, each composite image being formed from an image of a respective modified road geometry of a respective geographic area and the image of the movement data of the respective geographic area; labeling each synthetic image based on a difference between the modified road geometry of the synthetic image and the corresponding known road geometry to form a set of labeled synthetic images; A road geometry correction model is trained from the set of labeled synthetic images such that the trained road geometry correction model is configured to generate as output a road geometry corrected image from an input synthetic image of initial road geometry and movement data, optionally wherein modifying the known road geometry comprises any of: adding a road segment to the known road geometry; deleting road segments from the known road geometry; The road segment of the known road geometry is translated.

10. The method of claim 9, wherein each composite image is formed from an image of a respective modified road geometry of a respective geographic area, the image of mobility data for the respective geographic area, and a satellite image of the respective geographic area.

11. The method according to claim 9, wherein the method comprises, for each synthetic image, forming a respective further synthetic image from the image of the respective modified road geometry of the respective geographical area and a satellite image of the respective geographical area, optionally wherein the road geometry correction model comprises: a first encoder arranged to receive as input a composite image comprising at least one channel corresponding to an initial road geometry and at least another channel corresponding to movement data; A second encoder is arranged to receive as input another composite image comprising at least one channel corresponding to the initial road geometry and at least another channel corresponding to the satellite image.

12. The method according to any one of claims 9 to 11, wherein the trained road geometry correction model is an image segmentation model, optionally wherein the trained road geometry correction model is any one of the following: Unet; Segformer 80; Unet++.

13. A method according to any one of claims 9 to 11, wherein the movement data comprises a plurality of historical trips in the geographic area, optionally wherein (a) each element of the image of the movement data indicates an amount of historical trips traversing a respective portion of the geographic area corresponding to the element, or (b) the image of vehicle movement data is a heat map of historical trips in the geographic area.

14. The method of any one of claims 9 to 11, further comprising updating the trained road geometry correction model based on a plurality of artificially generated road geometry corrections, optionally wherein the plurality of artificially generated road geometry corrections are prompted by corresponding road geometry corrections generated using the trained road geometry correction model.

15. Apparatus arranged to perform a method according to any one of claims 1 to 14.

16. A computer readable medium storing a computer program which, when executed by a processor, causes the processor to perform the method according to any one of claims 1 to 14.

Images