Machine-learned models for identifying and refining transportation networks

Abstract

Segment identifying information is obtained for a portion of a transportation network, wherein the segment identifying information identifies a set of transportation network segments located within the portion of the transportation network. The segment identifying information and an image depicting the portion of a transportation network is processed with a machine-learned network identification model to obtain refinement information, wherein the refinement information is descriptive of one or more refinement updates for the segment identifying information. The segment identifying information is modified based on the one or more refinement updates.

Description

MACHINE-LEARNED MODELS FOR IDENTIFYING AND REFININGTRANSPORTATION NETWORKSPRIORITY CLAIM

[0001] The present application is based on and claims priority to United States Provisional Application Number 63 / 564,263 having a filing date of March 12, 2024. Application claims priority to and the benefit of each of such applications and incorporates all such applications herein by reference in their entirety.FIELD

[0002] The present disclosure relates generally to transportation network identification and refinement. More particularly, the present disclosure relates to machine-learned models for identifying and / or refining segments of a transportation network.BACKGROUND

[0003] Conventional navigation / mapping applications provide navigation services to users. Generally, navigation services first receive a queiy from a user. The query from the user identifies a Point of Interest (POI) that the user wishes to navigate to (e.g., a business, park, transportation network segment, building, geographic area, etc.). In response, the navigation system generates a sequence of navigation instructions. The navigation instructions describe a sequence of transportation network segments to traverse (e.g., roads, highways, crosswalks, public transportation networks, etc.). A transportation network, as described herein, generally refers to a network of physical or logical transportation infrastructure, such as vehicular transportation infrastructure (e.g., roads, streets, highways, etc.), public transportation infrastructure (e.g., subways, trains, buses, shuttles, etc.), commercial transportation infrastructure (e.g., commercial flights, ferries, rideshare services, etc.), etc.

[0004] Many conventional navigation / mapping applications can modify the suggested sequence of transportation network segments based on contextual events. For example, if a navigation system determines that a user has begun to traverse a transportation network segment that was not included in the sequence of transportation network segments, the navigation system can dynamically modify the sequence in real-time to adjust for the alternative segment traversed by the user.SUMMARY

[0005] Aspects and advantages of embodiments of the present disclosure will be set forth in part in the following description, or can be learned from the description, or can be learned through practice of the embodiments.

[0006] One example aspect of the present disclosure is directed to a computer- implemented method. The method includes obtaining, by a computing system comprising one or more computing devices, segment identifying information for a portion of a transportation network, wherein the segment identifying information identifies a set of transportation network segments located within the portion of the transportation network. The method includes processing, by the computing system, the segment identifying information and an image depicting the portion of a transportation network with a machine-learned network identification model to obtain refinement information, wherein the refinement information is descriptive of one or more refinement updates for the segment identifying information. The method includes modifying, by the computing system, the segment identifying information based on the one or more refinement updates.

[0007] Another example aspect of the present disclosure is directed to a computing system. The computing system includes one or more processor devices. The computing system includes a machine-learned network identification model trained to process imagery to sequentially identify segments of a transportation network depicted by the imagery. The computing system includes one or more tangible, non-transitory computer readable media storing computer-readable instructions that when executed by the one or more processor devices cause the one or more processor devices to perform operations. The operations include obtaining a first image depicting a portion of a transportation network, wherein the portion of the transportation network comprises a set of transportation network segments. The operations include processing the first image with the machine-learned network identification model to obtain segment identifying information, wherein the segment identifying information comprises a sequence of coordinate pairs, each of the sequence of coordinate pairs corresponding to a transportation network segment of the set of transportation network segments. The operations include adding the segment identifying information to a set of segment identifying information stored to a data store, wherein the set of segment identifying information comprises a plurality of sequences of coordinate pairs for a respective plurality of portions of the transportation network.

[0008] Another example aspect of the present disclosure is directed to aor more tangible, non-transitory computer readable media storing computer-readable instructions that whenexecuted by one or more processor devices cause the one or more processor devices to perform operations. The operations include obtaining segment identifying information for a portion of a transportation network, wherein the segment identifying information identifies a set of transportation network segments located within the portion of the transportation network. The operations include processing the segment identifying information and an image depicting the portion of a transportation network with a machine-learned network identification model to obtain refinement information, wherein the refinement information is descriptive of one or more refinement updates for the segment identifying information. The operations include modifying the segment identifying information based on the one or more refinement updates.

[0009] Other aspects of the present disclosure are directed to various systems, apparatuses, non-transitory computer-readable media, user interfaces, and electronic devices.

[0010] These and other features, aspects, and advantages of various embodiments of the present disclosure will become better understood with reference to the following description and appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate example embodiments of the present disclosure and, together with the description, serve to explain the related principles.BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Detailed discussion of embodiments directed to one of ordinary skill in the art is set forth in the specification, which makes reference to the appended figures, in which:

[0012] Figure 1A depicts a block diagram of an example computing system that performs training and utilization of machine-learned models for optimizing transportation network identification and / or refinement via sequence prediction according to some implementations of the present disclosure.

[0013] Figure IB depicts a block diagram of an example computing device that performs training of a machine-learned network identification model according to example embodiments of the present disclosure.

[0014] Figure 1C depicts a block diagram of an example computing device 50 that performs transportation network identification and / or refinement according to example embodiments of the present disclosure.

[0015] Figure 2 is a block diagram for refining existing segment identifying information stored to a data store according to some implementations of the present disclosure.

[0016] Figure 3 is an example illustration for identifying transportation network segments within a previously unidentified portion of a transportation network depicted by an image according to some implementations of the present disclosure.

[0017] Figure 4 is an example illustration for performing a deletion refinement update by generating refinement information with the machine-learned network identification model according to some implementations of the present disclosure.

[0018] Figure 5 A is an example illustration for performing a split refinement update by generating refinement information with the machine-learned network identification model according to some implementations of the present disclosure.

[0019] Figure 5B depicts an example illustration 506 of application of a split refinement update to a graph representation of a transportation network according to some implementations of the present disclosure.

[0020] Figure 6A is an example illustration for performing a shift refinement update by generating refinement information with the machine-learned network identification model according to some implementations of the present disclosure.

[0021] Figure 6B depicts an example illustration of application of a shift refinement update to a graph representation of a transportation network according to some implementations of the present disclosure.

[0022] Figure 7 depicts a flow chart diagram of an example method to perform refinement of segment identifying information with a machine-learned network identifying model according to some implementations of the present disclosure.

[0023] Figure 8 depicts a flow chart diagram of an example method to perform generation of segment identifying information with a machine-learned network identifying model to identify a portion of a transportation network according to some implementations of the present disclosure.

[0024] Reference numerals that are repeated across plural figures are intended to identify the same features in various implementations.DETAILED DESCRIPTIONOverview

[0025] Generally, the present disclosure is directed to transportation network identification and refinement. More particularly, the present disclosure relates to machine- learned models for identifying and / or refining segments of a transportation network. In recent times, transportation networks are often traversed with the assistance of navigation / mappingapplications. Conventional navigation / mapping applications provide routing services to users. To do so, a navigation system implemented by a navigation service provider can first receive a query from a user that specifies a Point of Interest (POI) that the user wishes to navigate to (e.g., a business, park, transportation network segment, building, geographic area, etc.). In response, the navigation system can generate a sequence of navigation instructions that describe a sequence of transportation network segments for the user to traverse (e.g., roads, highways, crosswalks, public transportation infrastructure, etc.).

[0026] Transportation network segments are generally selected from some manner of data store that catalogues identified and segmented transportation networks. For example, assume that a user requests navigation instructions to navigate between two locations in a major city. In response, the navigation system can retrieve information from a data store that identifies transportation network segments within the city. Using a routing algorithm, the navigation system can select a sequence of transportation network segments for the user to traverse based on the retrieved information.

[0027] Transportation networks must first be identified and segmented before segments of the network can be selected for inclusion in navigation instructions. Conventional approaches for identifying transportation networks collect satellite imagery depicting a top- down view of a portion of a transportation network (e.g., a three mile by three mile or three hundred meter by three hundred meter region of the transportation network, etc.). A series of filters and other image processing techniques are applied to the imagery to generate refined images (e.g., raster images, etc.) that isolate visible segments of the transportation network. A probability metric can then be predicted for each pixel of the refined images indicating a probability that the pixel corresponds to the transportation network. Based on the probability values, the transportation network can be identified, segmented, and then vectorized (e.g., using a vectorization algorithm, etc.). The segments of the transportation network can be stored to a data store for subsequent retrieval by navigation systems for route generation.

[0028] However, conventional transportation network identification approaches have generally proven to be computationally expensive - especially with regards to utilization of storage resources. For example, each of the multiple image processing steps used by conventional approaches can require non-trivial quantities of computing resources (e.g., memory, compute cycles, etc.). Further, some of the outputs of these processes (e.g., masks, filters, raster images, etc.) must be stored for analysis and subsequent application of heuristics. When these outputs are stored for each portion of a transportation network, the storage resources requirements can be substantial. In addition, many conventional approachesexhibit poor accuracy when compared to known (i.e., “ground-truth”) measurements for transportation networks. For example, transportation networks identified using conventional approaches can exhibit significant margins of error with regards to metrics which affect routing quality (e.g., geometric and topographic similarity metrics, average path length similarity metrics). Finally, conventional approaches generally lack the capability to efficiently refine portions of previously identified transportation networks. Instead, conventional approaches can only refine existing information by repeating the image processing steps described above to replace the existing information entirely.

[0029] Accordingly, implementations of the present disclosure propose machine-learned models for generating segment identifying information (i.e., information that identifies transportation segments) for transportation networks, and / or for refining existing segment identifying information. For example, a computing system can obtain an image depicting a portion of a transportation network (e.g., a 1 mile by 1 mile region, etc.), and segment identifying information that was previously generated for the portion of the transportation network. The imagery can be any type or manner of image information that depicts at least some of the portion of the transportation network, such as satellite imagery that depicts a top- down view of the portion of the transportation network. The images can be processed with a machine-learned network identification model alongside identified segment information generated when the transportation network was previously identified (e.g., using a conventional approach, etc.).

[0030] The machine-learned network identification model can be trained to generate refinement updates to refine the segment identifying information. Examples of such refinement updates include addition updates to add a new transportation segment, deletion updates to delete an identified transportation segment, split updates to split a previously identified transportation segment to multiple segments, shift updates to shift an identified transportation segment, etc. For transportation networks (or portions of networks) that have not previously been identified, the model can process the image depicting the portion of the transportation network to generate new segment identifying information (e.g., by exclusively performing addition refinement updates). The segment identifying information can identify the set of transportation network segments within the portion of the transportation network. For example, the segment identifying information may represent the set of transportation network segments as a graph structure, with each segment represented as an edge between two vertices.

[0031] Conversely, for portions of the transportation network that were identified previously, the model can process the image and the segment identifying information for that portion to generate refinement information. The refinement information can describe refinement update(s) to apply to the segment identifying information generated previously. For example, if the image processed by the model depicts a new transportation network segment recently constructed within the portion of the network, the refinement information can describe a refinement update to add the new transportation network segment to the set of transportation network segments. For another example, if the image also depicts that a previously identified segment has recently been removed, the refinement information can describe a refinement update to delete that transportation network segment from the set of transportation network segments.

[0032] The refinement updates can be applied to the segment identifying information. In particular, the refinement updates can be utilized to update the segment identifying information stored to the data store implemented by the navigation system. For example, the refinement information can structure the refinement update(s) as STATEful refinement updates structure for ingestion by an Application Programming Interface (API) implemented by the navigation system for refinement of the segment identifying information. In this manner, implementations described herein can be used to iteratively refine segment identifying information for a transportation network over time.

[0033] Aspects of the present disclosure provide a number of technical effects and benefits. As one example technical effect and benefit, implementations described herein substantially reduce computational resource utilization by providing the capability to refine segment identifying information. Specifically, segment identifying information (i.e., information that identifies segments of a transportation network) can be refined (e.g., to increase precision, segmentation, etc.) as certain technologies improve (e.g., image capture technologies, image processing technologies, etc.). However, the image processing techniques leveraged by conventional approaches cannot also be applied to refine segment identifying information (which is generally not stored as image data). Instead, conventional approaches can only refine segment identifying information by repeating the same computationally expensive processes to replace the existing segment identifying information. As such, the quantify of computing resources required to refine segment identifying information under a conventional approach is similar to the quantity required to generate the segment identifying information in the first place.

[0034] Accordingly, implementations described herein provide for a machine-learned network identification model trained to efficiently refine existing segment identifying information. The machine- learned network identification model can be trained to process segment identifying information and corresponding imagery to generate specific refinement updates on an as-needed basis. Because the machine-learned network identification model can generate refinement updates as-needed, the amount of computing resources utilized by the machine-learned network identification model is reduced substantially for segment identifying information that requires a minimal degree of refinement. In contrast, conventional approaches must repeat the same image processing techniques to refine the segment identifying information as were used to generate the segment identifying information, and as such, must utilize a substantial amount of computing resources regardless of the degree of refinement required.

[0035] For example, assume that segment identifying information must be refined to account for updated imagery depicting a newly constructed segment of a portion of a transportation network. To add the newly constructed segment to existing segment identifying information, a conventional approach can perform the same image processing and vectorization techniques to generate replacement segment identifying information that identifies and segments the entire portion of the transportation network (including the new segment). The replacement information can replace the existing information entirely. Conversely, the machine-learned network identification model can process both the existing segment identifying information and the updated imagery as inputs, and can generate a specific refinement update (e.g., an “ADD” refinement update) which modifies the segment identifying information to include the newly constructed segment without needing to identify or modify any other segments of the network. In such fashion, the machine-learned network identification model, using refinement information descriptive of one or more refinement updates for the segment identifying information, can substantially reduce the quantify of computing resources required to refine the segment identifying information.

[0036] As another example technical effect and benefit, implementations described herein can substantially reduce memory and storage resource consumption. Specifically, conventional approaches to identifying transportation networks generally include a step in which image information is generated (e.g., raster images, image masks, etc.). The image information is used to isolate pixels predicted to depict portions of a transportation network. Upon identification of the transportation network, segment identifying information for the transportation network can be stored in a data store for future use (e.g., for routing andnavigation purposes, etc.). However, conventional approaches generally require the image information generated previously to be stored to the data store along with the identifying information. Furthermore, the image information generated at each step can require substantial storage resources, with some image information being similar in size to the images depicting the transportation network. As such, when identifying transportation networks at scale, conventional approaches can require large quantities of memory and storage resources to implement.

[0037] Conversely, some implementations described herein obviate the need to store image information to the data store of the navigation system by generating segment identifying information that represents the set of transportation network segments as a graph structure (e.g., representing segments as edges between two vertices, etc.). Specifically, implementations described herein can process images depicting a transportation network with a machine-learned model to generate segment identifying information, and the segment identifying information can be stored to a data store for future use. However, unlike the information generated via conventional approaches, the segment identifying information can represent transportation networks without storing image information (e.g., rasters). In turn, eliminating the need to store image information can reduce storage and memory resource utilization by orders of magnitude. Accordingly, implementations described herein can substantially reduce utilization of storage resources (e.g., hard disk(s), data stores, data repositories, etc.) and memory resources (e.g., random access memory, cache, etc.).

[0038] As yet another example technical effect and benefit, implementations described herein exhibit substantial accuracy improvements over conventional approaches. Specifically, many conventional approaches utilize a combination of image processing and vectorization algorithms to generate segment identifying information. However, even after multiple refinements, approaches based on image processing and vectorization algorithms have failed to surpass sub-optimal scores across various accuracy metrics, such as an Average Path Length Similarity (APLS) score of 0.400, and a topographic recall score of 0.582. Conversely, implementations described herein have successfully exhibited more optimal scores across the same accuracy metrics, with an APLS score of 0.598 and a topographic recall score of .831.

[0039] With reference now to the Figures, example embodiments of the present disclosure will be discussed in further detail.Example Devices and Systems

[0040] Figure 1A depicts a block diagram of an example computing system 100 that performs training and utilization of machine-learned models for optimizing transportation network identification and / or refinement via sequence prediction according to some implementations of the present disclosure. The system 100 includes a user computing device 102, a server computing system 130, and a training computing system 150 that are communicatively coupled over a network 180.

[0041] The user computing device 102 can be any type of computing device, such as, for example, a personal computing device (e.g., laptop or desktop), a mobile computing device (e.g., smartphone or tablet), a gaming console or controller, a wearable computing device, an embedded computing device, or any other type of computing device.

[0042] The user computing device 102 includes one or more processors 112 and a memory 114. The one or more processors 112 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. The memory 114 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 114 can store data 116 and instructions 118 which are executed by the processor 112 to cause the user computing device 102 to perform operations.

[0043] In some implementations, the user computing device 102 can store or include one or more machine-learned network identification models 120. For example, the machine- learned network identification models 120 can be or can otherwise include various machine- learned models such as neural networks (e.g., deep neural networks) or other types of machine-learned models, including non-linear models and / or linear models. Neural networks can include feed-forward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks or other forms of neural networks. Some example machine-learned models can leverage an attention mechanism such as self-attention. For example, some example machine-learned models can include multiheaded self-attention models (e.g., transformer models). Example machine-learned network identification models 120 are discussed with reference to Figures 2-6A.

[0044] In some implementations, the one or more machine-learned network identification models 120 can be received from the server computing system 130 over network 180, stored in the user computing device memory 114, and then used or otherwise implemented by the one or more processors 112. In some implementations, the user computing device 102 canimplement multiple parallel instances of a single machine-learned network identification model 120 (e.g., to perform parallel transportation network sequencing across multiple instances of the machine-learned network identification model 120).

[0045] More particularly, the machine-learned network identification model 120 can process image(s) depicting a transportation network within a geographic area to identify and sequentially segment the transportation network. Specifically, in some implementations, the machine-learned network identification model 120 can be a personalized model that is trained to optimize segment identifying information in a user-specific manner. For example, assume that a user of the user computing device 102 resides in a subdivision actively being developed that includes a previously identified transportation network. The machine-learned network identification model 120 can process image information (e.g.. captured via a camera device of the user computing device 102), or other information (e.g., geolocation information for the user as the user navigates the transportation network) to determine that the transportation network has been developed further since it was last identified. In response, the user computing device 102 can inform the server computing system 130.

[0046] Additionally or alternatively, one or more machine-learned network identification models 140 can be included in or otherwise stored and implemented by the server computing system 130 that communicates with the user computing device 102 according to a clientserver relationship. For example, the machine-learned network identification models 140 can be implemented by the server computing system 130 as a portion of a web service (e.g., a navigation service, a mapping service, etc ). Thus, one or more models 120 can be stored and implemented at the user computing device 102 and / or one or more models 140 can be stored and implemented at the server computing system 130.

[0047] The user computing device 102 can also include one or more user input components 122 that receives user input. For example, the user input component 122 can be a touch-sensitive component (e.g., a touch-sensitive display screen or a touch pad) that is sensitive to the touch of a user input object (e.g., a finger or a stylus). The touch-sensitive component can serve to implement a virtual keyboard. Other example user input components include a microphone, a traditional keyboard, or other means by which a user can provide user input.

[0048] The server computing system 130 includes one or more processors 132 and a memory 134. The one or more processors 132 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. Thememory' 134 can include one or more non-transi lory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 134 can store data 136 and instructions 138 which are executed by the processor 132 to cause the server computing system 130 to perform operations.

[0049] In some implementations, the server computing system 130 includes or is otherwise implemented by one or more server computing devices. In instances in which the server computing system 130 includes plural server computing devices, such server computing devices can operate according to sequential computing architectures, parallel computing architectures, or some combination thereof.

[0050] As described above, the server computing system 130 can store or otherwise include one or more machine-learned network identification models 140. For example, the models 140 can be or can otherwise include various machine-learned models. Example machine-learned models include neural networks or other multi-layer non-linear models. Example neural networks include feed forward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks. Some example machine- learned models can leverage an attention mechanism such as self-attention. For example, some example machine-learned models can include multi-headed self-attention models (e.g., transformer models). Example models 140 are discussed with reference to Figures 2-6A.

[0051] More particularly, the machine-learned network identification model 140 can process image(s) depicting a portion of a transportation network. A transportation network, as described herein, generally refers to a network of physical or logical transportation infrastructure, such as vehicular transportation infrastructure (e.g., roads, streets, highways, etc.), public transportation infrastructure (e.g.. subways, trains, buses, shuttles, etc.), commercial transportation infrastructure (e.g., commercial flights, ferries, rideshare services, etc.), etc. For example, the image(s) may be satellite imagery that depicts a top-down view of the portion of the transportation network.

[0052] The machine-learned network identification model 140 can process the image depicting the portion of the transportation network to generate segment identifying information. The segment identifying information can identify a set of transportation network segments located within the portion of the transportation network. For example, assume that the transportation network (or a portion of the network depicted in the satellite imagery ) consists of a single street. Based on the length of the street, the street can be identified and segmented into multiple transportation network segments. The server computing system 130can process the imagery with the machine-learned network identification model 140 to obtain segment identifying information. The segment identifying information can identify each segment of the street. For example, assume that the machine-learned network identification model 140 segments the street is segmented into two segments. The segment identify ing information can represent the two segments as a graph structure, where the two segments are represented by two sequential edges formed by three vertices (e.g., a segment represented by the edge between a vertex A and a vertex B. and another segment represented by the edge between the vertex B and a vertex C).

[0053] In some implementations, the machine-learned network identification model 140 can process existing segment identifying information, or information derived therefrom, alongside the imagery depicting the portion of the transportation network. For example, assume that the machine-learned network identification model 140 processes imagery depicting the portion of the transportation network to generate segment identifying information that identifies segments of the transportation network as a sequence of coordinate pairs (e.g., two x / y coordinates per segment, etc.). Further assume that updated imagery is acquired that depicts a newly constructed portion of the network. The machine-learned network identification model 140 can process the vector of coordinate pairs and the updated imagery7to generate refinement updates for the segment identify ing information. The refinement updates can include multiple refinement update types e.g., adding new segments, deleting segments, splitting segments, shifting segments, etc.) to refine the segment identify ing information.

[0054] Specifically, in some implementations, the machine-learned network identification model 140 can be trained to perform a “DELETE"’ refinement update type, or similar, to delete or remove an existing transportation network segment from segment identifying information. For example, assume that the server computing system 130 is associated with a navigation service, and the navigation service implements a data store to which the server computing system 130 has access. The data store can store information that identifies segments of transportation networks. The server computing system 130 can acquire updated satellite imagery depicting that a segment of a previously identified transportation network has been removed. In response, the server computing system 130 can retrieve segment identify ing information generated for the transportation network, and can process the segment identify ing information alongside the updated imagery7with the machine-learned network identification model 140. The machine-learned network identification model 140 can generate a DELETE refinement update to delete the portion of the segment identifyinginformation that identifies the removed segment of the transportation network (e.g., two x / y coordinate pairs that form the removed segment, etc.)

[0055] Additionally, or alternatively, in some implementations, the machine-learned network identification model 140 can be trained to perform a “SHIFT” refinement update ty pe, or similar, to modify information indicating the location of certain transportation network segments. For example, the server computing system 130 can acquire updated high- fidelity satellite imagery depicting a previously identified transportation network. In response, the server computing system 130 can retrieve segment identifying information generated for the transportation network, and can process the segment identifying information alongside the high-fidelity imagery' with the machine-learned network identification model 140. If the high-fidelity imagery indicates that the location of a transportation segment in the segment identifying information is inaccurate, the machine-learned network identification model 140 can perform a SHIFT refinement update to shift the location of the portion of the segment identifying information that identifies the removed segment of the transportation network. For example, assume that a transportation network segment is defined as an edge between two vertices. The SHIFT refinement update can modify the location of one of the vertices of the edge, and by modifying the location of the vertex, can “move” all edges connected to the vertex.

[0056] Additionally, or alternatively, in some implementations, the machine-learned network identification model 140 can be trained to perform a “SPLIT” refinement update ty pe, or similar, to split certain transportation network segments into multiple segments. For example, the server computing system 130 can acquire updated high-fidelity satellite imagery depicting a previously identified transportation network. In response, the server computing system 130 can retrieve segment identifying information generated for the transportation network, and can process the segment identifying information alongside the high-fidelity imagery with the machine-learned network identification model 140. If the high-fidelity imagery' enables more granular segmentation of an existing segment, the machine-learned network identification model 140 can perform a SPLIT refinement update to split the segment into two segments. The machine-learned network identification model 140 can modify the segment identifying information to include the two segments. For example, the refinement update can add an intermediate coordinate between the coordinate pair that forms the segment to split the segment.

[0057] Additionally, or alternatively, in some implementations, the machine-learned network identification model 140 can be trained to perform a “ADD,” or addition, refinementupdate type to split certain transportation network segments into multiple segments. For example, the sen' er computing system 130 can acquire updated high-fidelity satellite imagery depicting a previously identified transportation network with a newly constructed portion. In response, the server computing system 130 can retrieve segment identifying information generated for the transportation network, and can process the segment identifying information alongside the high-fidelity imagery with the machine-learned network identification model 140. The machine-learned network identification model 140 can perform the addition refinement update to modify the segment identifying information to add a coordinate corresponding to the newly constructed portion of the transportation network. For example, assume that the final segment identified in the segment identifying information is formed between a vertex A and a vertex B. The addition refinement update can modify the segment identifying information to add a vertex C, and the newly added segment can be represented as an edge between the vertices B and C.

[0058] In some implementations, the machine-learned network identification model 140 can be a multimodal model that is trained to process multiple inputs. For example, the machine-learned network identification model 140 may include an image encoder portion, audio encoder portion, map-data encoder portion, etc.

[0059] The user computing device 102 and / or the server computing system 130 can train the models 120 and / or 140 via interaction with the training computing system 150 that is communicatively coupled over the network 180. The training computing system 150 can be separate from the server computing system 130 or can be a portion of the server computing system 130.

[0060] The training computing system 150 includes one or more processors 152 and a memory 154. The one or more processors 152 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. The memory 154 can include one or more non-transi lory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 154 can store data 156 and instructions 158 which are executed by the processor 152 to cause the training computing system 150 to perform operations. In some implementations, the training computing system 150 includes or is otherwise implemented by one or more server computing devices.

[0061] The training computing system 150 can include a model trainer 160 that trains the machine-learned models 120 and / or 140 stored at the user computing device 102 and / or theserver computing system 130 using various training or learning techniques, such as, for example, backwards propagation of errors. For example, a loss function can be backpropagated through the model(s) to update one or more parameters of the model(s) (e.g., based on a gradient of the loss function). Various loss functions can be used such as mean squared error, likelihood loss, cross entropy loss, hinge loss, and / or various other loss functions. Gradient descent techniques can be used to iteratively update the parameters over a number of training iterations.

[0062] In some implementations, performing backwards propagation of errors can include performing truncated backpropagation through time. The model trainer 160 can perform a number of generalization techniques (e.g., weight decays, dropouts, etc.) to improve the generalization capability of the models being trained.

[0063] In particular, the model trainer 160 can train the models 120 and / or 140 based on a set of training data 162. The training data 162 can include, for example, images depicting transportation networks, or portions of transportation networks, and corresponding groundtruth information. The ground-truth information can include measurements, metrics, etc. describing accurate dimensions and locations for the transportation network. Additionally, in some implementations, the ground-truth information can include segmentation information for the transportation network that optimally segments the transportation network. The model trainer 160 can train the model(s) based on difference(s) between outputs of the model(s) and the corresponding ground-truth information.

[0064] In some implementations, the model trainer 160 can generate the training data 162. Specifically, the training data 162 can be generated by the model trainer 160 by generating information indicative of incomplete road networks. For example, the model trainer 160 can iteratively corrupt ground-truth representations of transportation networks with a series of edge deletions, vertex shifts, edge merges, and then edge additions that can be reversed and backtracked as a series of edge deletions, edge splits, vertex shifts, and edge additions that recover the original transportation network. In such instances, it should be noted that the transportation network after the cormption phase becomes the conditional road network for the model 140 and the series of backtracks form the ground-truth sequence the model 140 is trained to predict. By corrupting in the order of deletions, shifts, merges, and then additions, the backtracked sequence is in the correct order to train the model 140 with deletion, split, shift, and then addition refinement update types. To ensure a deterministic order, each of the deletions, modifications, and additions can be sorted by (y. x). For example, a deletion cormption operation can be backtracked to train the model 140 toperform an addition refinement update ty pe. For another example, a shift corruption operation can be backtracked to train the model 140 to perform a shift refinement update type. For another example, a merge corruption operation can be backtracked to train the model 140 to perform a split refinement update type. For yet another example, an addition corruption operation can be backtracked to train the model 140 to perform a deletion refinement update type.

[0065] In some implementations, the model trainer 160 can generate a ‘"golden datasef’ for validation of the model 140. To do so, in some implementations, the model trainer 160 can utilize edits to historical segment identifying information to produce conditional road networks and the stateful updates to them.

[0066] In some implementations, if the user has provided consent, the training examples can be provided by the user computing device 102. Thus, in such implementations, the model 120 provided to the user computing device 102 can be trained by the training computing system 150 on user-specific data received from the user computing device 102. In some instances, this process can be referred to as personalizing the model.

[0067] The model trainer 160 includes computer logic utilized to provide desired functionality. The model trainer 160 can be implemented in hardware, firmware, and / or software controlling a general purpose processor. For example, in some implementations, the model trainer 160 includes program files stored on a storage device, loaded into a memory and executed by one or more processors. In other implementations, the model trainer 160 includes one or more sets of computer-executable instructions that are stored in a tangible computer-readable storage medium such as RAM, hard disk, or optical or magnetic media.

[0068] The network 180 can be any type of communications network, such as a local area network (e.g., intranet), wide area network (e.g., Internet), or some combination thereof and can include any number of wired or wireless links. In general, communication over the network 180 can be carried via any type of wired and / or wireless connection, using a wide variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or protection schemes (e.g., VPN, secure HTTP, SSL).

[0069] The machine-learned models described in this specification may be used in a variety of tasks, applications, and / or use cases.

[0070] Figure 1 A illustrates one example computing system that can be used to implement the present disclosure. Other computing systems can be used as well. For example, in some implementations, the user computing device 102 can include the model trainer 160 and the training dataset 162. In such implementations, the models 120 can be bothtrained and used locally at the user computing device 102. In some of such implementations, the user computing device 102 can implement the model trainer 160 to personalize the models 120 based on user-specific data.

[0071] Figure IB depicts a block diagram of an example computing device 10 that performs training of a machine-learned network identification model according to example embodiments of the present disclosure. The computing device 10 can be a user computing device or a server computing device.

[0072] The computing device 10 includes a number of applications (e.g., applications 1 through N). Each application contains its own machine learning library' and machine-learned model(s). For example, each application can include a machine-learned model. Example applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, etc.

[0073] As illustrated in Figure IB, each application can communicate with a number of other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, each application can communicate with each device component using an API (e.g., a public API). In some implementations, the API used by each application is specific to that application.

[0074] Figure 1C depicts a block diagram of an example computing device 50 that performs transportation network identification and / or refinement according to example embodiments of the present disclosure. The computing device 50 can be a user computing device or a server computing device.

[0075] The computing device 50 includes a number of applications (e.g., applications 1 through N). Each application is in communication with a central intelligence layer. Example applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, etc. In some implementations, each application can communicate with the central intelligence layer (and model(s) stored therein) using an API (e.g., a common API across all applications).

[0076] The central intelligence layer includes a number of machine-learned models. For example, as illustrated in Figure 1C, a respective machine-learned model can be provided for each application and managed by the central intelligence layer. In other implementations, two or more applications can share a single machine-learned model. For example, in some implementations, the central intelligence layer can provide a single model for all of theapplications. In some implementations, the central intelligence layer is included within or otherwise implemented by an operating system of the computing device 50.

[0077] The central intelligence layer can communicate with a central device data layer. The central device data layer can be a centralized repository of data for the computing device 50. As illustrated in Figure 1C, the central device data layer can communicate with a number of other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, the central device data layer can communicate with each device component using an API (e.g., a private API).

[0078] Figure 2 is a block diagram for refining existing segment identifying information stored to a data store according to some implementations of the present disclosure. More specifically, a computing system 200 (e.g., the server computing system 130 of Figure 1A, etc.) can include processor device(s) 202 and memory' 204 as described with regards to the processor(s) 112 and memory' 114 of Figure 1A. The memory 204 can include a network identification and refinement module 206. The computing system 200 can utilize the network identification and refinement module 206 to refine segment identifying information.

[0079] More specifically, in some implementations, the computing system 200 can include a navigation service system 208. Alternatively, the navigation service system 208 can be implemented remotely, with different computing devices of the same computing environment, etc. It should be noted that the navigation service system 208 is illustrated as being both separate from the computing system 200 and included in the computing system 200 to reflect that either instance is possible. The navigation service system 208 can be a system implemented by a navigation service provider. The navigation service system 208 can implement various navigation services and functions, such as routing functions, data storage, etc.

[0080] The navigation service system 208 can include a routing module 210. The routing module 210 can generate routes in response to navigation requests received from users. For example, a user of the navigation service implemented with the navigation service system 208 can provide a navigation request from a current location to a particular POI. In response, the routing module 210 can retrieve information indicating traversable transportation segments between the current location of the user and the particular POI from a data store 212. Specifically, the routing module 210 can retrieve segment identifying information 214 from the data store 212. The retrieved segment identifying information can indicate a seriesof roads, highways, streets, sidewalks, public transportation, etc. that a user can traverse to arrive at the POI.

[0081] The data store 212 can store and index segment identifying information for multiple portions of a transportation network. To follow the depicted example, the data store 212 can include the segment identifying information 214 for a particular portion of a transportation network identified as “TNP_N” (e.g.. the Nthportion of the transportation network). The data store 212 can also include segment identifying information for the other portions of the transportation network identified as “TNP_1” through “TNP_N”. In this manner, the data store 212 can sequentially index segment identifying information so that the information is easily retrievable in response to navigation requests.

[0082] In some instances, refinements must be made to the segment identifying information stored to the data store 212. To do so, the network identification and refinement module 206 can include a segment identifying information obtainer 216. The segment identifying information obtainer 216 can identify and obtain segment identifying information that requires refinement. In some implementations, the segment identifying information obtainer 216 can routinely obtain segment identifying information for a portion of the transportation network after a certain period of time has passed since the information was last refined. Additionally, or alternatively, in some implementations, the segment identifying information obtainer 216 can obtain segment identifying information in response to new or updated imagery being obtained for the portion of the transportation network corresponding to the segment identifying information.

[0083] To follow the depicted example, the segment identifying information obtainer 216 can obtain the segment identifying information 214. The network identification and refinement module 206 can further obtain an updated image 218. The updated image 218 can depict the portion of the transportation network that corresponds to the segment identifying information 214. For example, assume that the segment identifying information 214 was generated based on an original image depicting the portion of the transportation network. The updated image 218 can be a image that was captured more recently than the original image. Additionally, or alternatively, the updated image 218 can be an image that is captured with higher fidelity than the original image or with different camera parameters (e.g., focal length, aperture, field-of-view, resolution, etc. Additionally, or alternatively, the updated image 218 can be captured via a different type of technology (e.g., satellite imagery, drone imagery, ground-based vehicle imagery, LIDAR imagery, etc.). Additionally, or alternatively, theupdated image 218 can be an image that is captured with a different perspective than the original image.

[0084] The network identification and refinement module 206 can include a machine- learned network identification model 220. The machine-learned network identification model 220 can process the segment identifying information 214 and the updated image 218 to generate refinement information 222. Specifically, the machine-learned network identification model 220 can process an overhead RGB image and a prompt that encodes the existing road network via a sequence of edges. The machine-learned network identification model 220 can produce a sequence of pairs of (y, x) coordinates specifying an edge in the transportation network. For example, take the following sequence:[y 1, xl, y2. x2], [y2, x2, y3, x3], [y4, x4, y5, x5], PAD. PAD, .... EOS Each pair of brackets in the sequence can indicate each edge in the road network. So, (yl, xl) is connected to (y2, x2), and (y2, x2) is connected to (y3, x3), and so on. The “PAD’’ tokens can be included to ensure a static sequence length across examples — and to implicitly prevent early sequence termination — and “EOS"’ can indicate the end of the sequence. To ensure a deterministic ordering of edges, ground-truth sequences can be generated by sorting the edges by (y, x) so that edges that originate from the top-left comer of the image are ordered first.

[0085] The vocabulary' of the machine-learned network identification model 220 can be then generalized with four refinement update types (or more / less): ‘‘DELETE’". “SPLIT’", “SHIFT", and “ADD”, which delete edges, split edges into two with intermediate — not necessarily collinear — vertices, shift the location of vertices, and add edges, respectively. The predicted output sequence thus encodes stateful updates to the conditional road network encoded in the prompt. The machine-learned network identification model 220 can be trained to provide stateful updates that improve the existing road network (e.g., via realignment to imagery or improved precision / recall).

[0086] The refinement information 222 can include refinement updates for the segment identifying information 214. The refinement updates determined for the segment identifying information 214 can refine the segment identifying information 214 based on differences identified between the updated image 218 and the segment identifying information 214.

[0087] Refinement updates, as described herein, can refer to any type or manner of operation that adjusts some aspect or portion of the segment identifying information 214. Specifically, the refinement updates can include a number of different refinement update ty pes. In some implementations, the refinement update types can include a “DELETE” refinement update type, or similar, to delete or remove an existing transportation networksegment from segment identifying information. Additionally, or alternatively, in some implementations, the refinement update types can include a “SHIFT” refinement update type, or similar, to modify information indicating the location of certain transportation network segments. Additionally, or alternatively, in some implementations, the refinement update types can include a “SPLIT” refinement update type, or similar, to split certain transportation network segments into two segments. Additionally, or alternatively, in some implementations, the refinement update types can include a “ADD.” or addition, refinement update type to add a new transportation segment. Different types of refinement updates will be discussed in greater detail subsequently.

[0088] For example, the deletion refinement update type removes edges from the network with edges specified as a pair of (y, x) coordinates (e.g., “0. 0, 1, 2” deletes the edge between (0, 0) and (1, 2)). It should be noted that deleting an edge can be more axiomatic than deleting a vertex, since deleting a vertex functionally deletes all edges connected to the vertex. For another example, an addition refinement update type can create an edge in a transportation network with a format specified identically to deletion update refinement types (e.g.. “6, 7. 10. 10” adds an edge between (6, 7) and (10, 10)). It should be noted that the added edge may exist between two existing vertices, two new vertices, or one existing vertex and one new vertex.

[0089] In some implementations, the refinement information 222 can be structured as a sequence. For example, assume that deletion, split, shift, and addition refinement updates are all performed. The output sequence can be structured as DELETE, yl , xl, y2, x2, yl , xl , y3, x3, where the DELETE token indicates the start of a series of deletions, similar for SHIFT, SPLIT, and ADD. So, the edge from (yl, xl) to (y2, x2) will be deleted along with the edge from (yl. xl) to (y3, x3). The deletions can be followed by one split, two shifts, and one addition, for example. Alternatively, the output sequence from the model can be structured as:DELETE, y L xL y 2, x2,DELETE, yl, xl. y3. x3,SPLIT, yl, xl, y4, x4, y5, x5,SHIFT, y5, x5, y6, x6, SHIFT, y2, x2, y7, x7, ADD, y4. x4, y8, x8, PAD. PAD, ... , EOS

[0090] To follow the depicted example, assume that the updated image 218 depicts a new transportation segment added to the portion of the transportation network associated with the segment identifying information 214. Further assume that the segment identifying information 214 was generated prior to construction of the new transportation network segment. The network identification and refinement module 206 can process the updated image 218 and the segment identifying information 214 with the machine-learned network identification model 220. The machine-learned network identification model 220 can determine which refinement update(s) (if any) to include in the refinement information 222.

[0091] In some implementations, the machine-learned network identification model 220 can be trained to perform or select refinement update types in a particular sequence. For example, the machine-learned network identification model 220 can be trained to perform DELETE refinement updates first, then the SPLIT refinement updates, then the SHIFT refinement updates, and then the ADD refinement updates. In some implementations, deletion refinement updates can be restricted to the prompted road network, so they occur first. Similarly, split refinement updates can occur before shifts / additions, since the split refinement updates generally only apply to the remaining prompted road network and the splits will introduce vertices that could then be shifted. Finally, shifts and additions can fill any part of the road network missing (e.g., via stitching). In some implementations, for a specific refinement update type (e.g., DELETE), the sequence of coordinates can be deterministically ordered by (y, x) (i.e., from the top-left comer of the image). Thus, edges / vertices near the top-left comer of the image will be deleted / split / shifted / added first, progressing to the bottom-right comer.

[0092] In some implementations, the machine-learned network identification model 220 can utilize a pre-determined sequence of refinement update types to generate the refinement information 222. For example, the machine-learned network identification model 220 can first determine whether a deletion refinement update type is required. The machine-learned network identification model 220 can next determine whether a split refinement update type is required. The machine-learned network identification model 220 can next determine whether a shift refinement update type is required. The machine-learned network identification model 220 can next determine whether an addition refinement update type is required. Alternatively, in some implementations, the machine-learned network identification model 220 can determine whether particular refinement update types are necessary in some other order, or without any order.

[0093] The machine-learned network identification model 220 can generate the refinement information 222 to include an addition refinement update to add the newly constructed segment to the segment refinement information. To follow the depicted example, assume that the transportation network segment SEG_05 of the segment identifying information 214 is the final transportation network segment to be identified in the segment identifying information 214. Further assume that the newly constructed segment originates from the end of the segment SEG_05. The refinement information 222 can include an addition type refinement update that adds a new segment SEG_06 to the segment identifying information 214. The new segment SEG_06 can begin at the end coordinate 2,3 of the SEG_05, and can end at a new coordinate 4.5.

[0094] For another example, assume that updated image 218 identifies the ending location for the transportation segment SEG_02 of the segment identify ing information 214 as being inaccurate. In response, the refinement information 222 can include a shift type refinement update that shifts the current ending location of (0,2) for SEG_02 to a new ending location (0.3). In such fashion, the refinement information 222 can be utilized to iteratively update and refine existing segment identifying information 214 as additional information becomes available (e.g., the updated image 218, etc ).

[0095] The network identification and refinement module 206 can include an information modifier 224. The information modifier 224 can modify the segment identifying information 214 directly, or can otherwise cause the segment identifying information 214 to be modified. In some implementations, the information modifier 224 can access the data store 212 to apply the refinement updates of the refinement information 222. For example, the information modifier 224 can apply the refinement information 222 to the segment identifying information 214 local to the computing system 200. and use it to replace the segment identifying information 214 local to the data store 212.

[0096] Alternatively, in some implementations, the information modifier 224 can provide the refinement information 222 to the navigation service system 208. Specifically, the memory 204 can include a communication module 226. The communication module 226 can perform various functions to implement communications and / or the exchange of data between the computing system 200 and other device(s) / system(s), such as the navigation sendee system 208. Similarly, the navigation service system 208 can include an Application Programming Interface (API) 228.

[0097] The API 228 can be a STATEful API that can ingest and parse structured refinement information to extract the refinement updates, and can apply the refinementupdates to the segment identifying information with a data store modifier 230. For example, the communication module 226 can generate a structured STATEful refinement update 232 based on the refinement information 222 that is structured for ingestion by the API 228 (e.g., based on a pre-determined format specified by the API 228). The communication module 226 can transmit the STATEful refinement update 232 via the API 228 to the navigation service system 208. If successfully ingested and parsed by the API 228, the data store modifier 230 can apply the refinement updates to the data store 212.

[0098] Figure 3 is an example illustration for identifying transportation network segments within a previously unidentified portion of a transportation network depicted by an image according to some implementations of the present disclosure. Figure 3 will be discussed in conjunction with Figure 2. Specifically, as depicted, the image 302 depicts a portion of a transportation network that has been previously unidentified. The machine-learned network identification model 220 can process the image 302 to obtain the segment identifying information 214. The segment identifying information 214 can identify the transportation network depicted by the image 302.

[0099] The segment identifying information 214 can also segment the identified transportation network depicted by the image 302. To follow the depicted example, the segment identifying information 214 can include a sequence of coordinates that correspond to the identified segments of the transportation network. In some implementations, the segment identifying information 214 can represent the identified segments of the transportation network as a graph structure. As an example, the visual representation 304 provides a visual representation of the graph structure included in the segment identifying information 214. For example, the segment SEG_01, with starting coordinates of -5,-5 and ending coordinates of - 2,-3, is illustrated as an edge formed between vertices placed at -5,-5 and -2,-3 within a graph space.

[0100] Figure 4 is an example illustration for performing a deletion refinement update by generating refinement information with the machine-learned network identification model according to some implementations of the present disclosure. Figure 4 will be discussed in conjunction with Figures 2 and 3. Specifically, Figure 4 includes an updated image 402. The updated image 402 can depict the same portion of the transportation network as depicted by the image 302. For example, the updated image 402 can be an image captured using higher- fidelity capture technologies, etc. The visual representation 304 of the segment identifying information 214 is illustrated overlaid atop the updated image 402 to demonstrate that the transportation network segment SEG_05 (e.g., the segment from coordinates 1,1 to 2,3) hasbeen identified erroneously. However, it should be noted that this visual representation is only illustrated to more clearly illustrate various implementations of the present disclosure, and is not necessarily included in the updated image 402.

[0101] The machine-learned network identification model 220 can process the updated image 402 alongside the segment identifying information 214 to obtain the refinement information 222. Specifically, the machine-learned network identification model 220 can identify differences between the segment identifying information 214 and the updated image 402. As depicted, the segment SEG_05 (e.g., the segment between coordinates 1,1 and 2,3) does not correspond to a visible transportation segment. This may occur because a transportation segment previously existed in that location but was removed after the segment identifying information 214 was generated, or because the segment was mistakenly identified when generating the segment identifying information 214.

[0102] The machine-learned network identification model 220 can generate the refinement information 222 in response to identifying the erroneous transportation network segment. As illustrated, the refinement information 222 can include a deletion refinement update to delete the erroneous transportation network segment. The refinement information 222 can be applied to the segment identifying information 214 within the data store 212 to generate refined segment identifying information 404.

[0103] The refined visual representation 406 is a visual representation of the refined segment identifying information 406. As depicted, the refinement information 222. once applied to the segment identifying information 214, can remove the erroneously identified transportation segment SEG_05 between the coordinates 1,1 and 2,3. As depicted, the coordinate 2,3 is only connected to one other coordinate (e.g., 1,1) by the edge representing the transportation network segment to be deleted. Thus, when the edge between the coordinates 1,1 and 2,3 is removed, the coordinate 2,3 can also be removed. Alternatively, if the coordinate 2,3 is connected to two or more other coordinates, the edge between the coordinates 1,1 and 2,3 can be removed without removing the coordinate 2,3. In such fashion, the machine-learned network identification model 220 can be leveraged to iteratively refine previously identified transportation network portions as updated imagery, such as the updated image 402, is acquired.

[0104] Figure 5 A is an example illustration for performing a split refinement update by generating refinement information with the machine-learned network identification model according to some implementations of the present disclosure. Figure 5A will be discussed in conjunction with Figures 2-4. Specifically, Figure 5A includes the updated image 402. Theupdated image 402 can depict the same portion of the transportation network as depicted by the image 302. The SPLIT refinement update type illustrated in Figure 5A can be performed following performance of the DELETE refinement update type illustrated in Figure 4. As such, the visual representation 406 of the refined segment identifying information 404 is illustrated overlaid atop the updated image 402 to demonstrate the effect of the split refinement update. However, it should be noted that this visual representation is only illustrated to more clearly illustrate vanous implementations of the present disclosure, and is not necessarily included or depicted in the updated image 402.

[0105] The machine-learned network identification model 220 can process the updated image 402 alongside the segment identifying information 214 to obtain the refinement information 222. Specifically, the machine-learned network identification model 220 can identify differences between the segment identifying information 214 and the updated image 402. As depicted, the segment SEG_02 (e.g., the segment between -2,3 and 0,2) is longer than other transportation network segments, and is thus a candidate for splitting into two (or more) segments. This may occur to maintain a uniform segment length, or to match transportation network infrastructure for routing purposes (e.g.. traffic lights, stop signs, roundabouts, highway exits, etc.).

[0106] The machine-learned network identification model 220 can generate the refinement information 222 in response to identifying the erroneous transportation network segment. As illustrated, the refinement information 222 can include a split refinement update to split the transportation network segment. The refinement information 222 can be applied to the segment identifying information 214 within the data store 212 to generate refined segment identifying information 502. The refined segment identifying information 502 can include anew intermediate added to split an existing transportation network segment.

[0107] Refined visual representation 504 is a visual representation of the refined segment identify ing information 502. As depicted, the refinement information 222, once applied to the segment identifying information 214, can split the identified transportation segment between the coordinates -2,-3 and 0,2 by inserting an intermediate coordinate -1,0. As depicted in the refined visual representation 504, the intermediate coordinate -1,0 inserted between the coordinates -2,-3 and 0,2 can respectively serve as an end coordinate and start coordinate for the newly split transportation segments. For example, one newly split transportation segment can be formed between coordinates -2,-3 and -1,0, while the other newly split transportation segment can be formed between coordinates -1,0 and 0.2.

[0108] Figure 5B depicts an example illustration 506 of application of a split refinement update to a graph representation of a transportation network according to some implementations of the present disclosure. Specifically, the split refinement update type breaks an edge into two by placing an intermediate vertex (e.g., “5, 6, 7, 8, 6, 7” splits the edge between (5, 6) and (7, 8) into two edges by placing a new vertex at (6, 7) and connecting (5, 6) to (6. 7) and (6, 7) to (7, 8)). It should be noted that the intermediate vertex, or coordinate, need not be collinear. That is, a valid split could be ”5. 6. 7. 8, 6, 4’?, where (6. 4) is not collinear with the edge between (5, 6) and (7, 8). Non-collinearity can add flexibility' to the model and can save sequence length compared to combined split and shift refinement update types. It should also be noted that the split refinement update can be utilized to stitch images, representations, or segment identifying information for two portions of a transportation network together (alongside the addition refinement update type).

[0109] Figure 6A is an example illustration for performing a shift refinement update by generating refinement information with the machine-learned network identification model according to some implementations of the present disclosure. Figure 6A will be discussed in conjunction with Figures 2-5B. Specifically, Figure 6A includes the updated image 402. The updated image 402 can depict the same portion of the transportation network as depicted by the image 302. The SHIFT refinement update ty pe illustrated in Figure 6A can be performed following performance of the DELETE refinement update type illustrated in Figure 4 and the SPLIT refinement update illustrated in Figure 5A. As such, the refined visual representation 504 of the refined segment identifying information 502 is illustrated overlaid atop the updated image 402 to demonstrate the effect of the shift refinement update. However, it should be noted that this visual representation is only illustrated to more clearly illustrate various implementations of the present disclosure, and is not necessarily included or depicted in the updated image 402.

[0110] The machine-learned network identification model 220 can process the updated image 402 alongside the segment identifying information 214 to obtain the refinement information 222. Specifically, the machine-learned network identification model 220 can identify’ differences between the segment identifying information 214 and the updated image 402. As depicted, the segment SEG_01 (e.g., the segment between -5,-5 and -2,-3) is slightly misaligned with the underlying transportation network segment depicted in the updated image 402, and is thus a candidate for the shifting refinement update. This may occur to more accurately align the segment identifying information 214 with the underlying transportation network.

[0111] The machine-learned network identification model 220 can generate the refinement information 222 in response to identifying the erroneous transportation network segment. As illustrated, the refinement information 222 can include a shift refinement update to shift the transportation network segment. The refinement information 222 can be applied to the segment identifying information 214 within the data store 212 to generate refined segment identifying information 602. The refined segment identifying information 602 can include anew location for one of the coordinates of the segment SEG_01. More specifically, the shift refinement update included in the refinement information 222 can modify the location of the coordinate of the transportation network segment SEG_01 from -5,-5 to -6,-5 to more accurately correspond to the underlying transportation network segment.

[0112] Refined visual representation 604 is a visual representation of the refined segment identify ing information 602. As depicted, the refinement information 222, once applied to the segment identifying information 214, can shift one coordinate of the transportation network segment SEG_01 from coordinates -5,-5 to coordinates -6,-5. As depicted in the refined visual representation 604, because the placement of the coordinate -6,-5 affects how the segment maps to the underlying transportation network segment, shifting of a single coordinate can substantially increase the accuracy with which the refined segment identifying information 602 tracks the transportation network depicted in the updated image 402.

[0113] Figure 6B depicts an example illustration 606 of application of a shift refinement update to a graph representation of a transportation network according to some implementations of the present disclosure. Specifically, the shift refinement update type moves vertices and thus all edges connected to those vertices (e.g., “1, 2, 3, 4” moves the vertex at (1, 2) to (3, 4)). Shifting a vertex is more axiomatic than shifting an edge, which functionally must shift at least one vertex. Additionally, shifting vertices preserves the graph’s topology and connectivity. SHIFT operations should be local, targeting realignment of the road network.Example Methods

[0114] Figure 7 depicts a flow chart diagram of an example method 700 to perform refinement of segment identify ing information with a machine-learned network identifying model according to some implementations of the present disclosure. Although Figure 7 depicts steps performed in a particular order for purposes of illustration and discussion, the methods of the present disclosure are not limited to the particularly illustrated order orarrangement. The various steps of the method 700 can be omitted, rearranged, combined, and / or adapted in various ways without deviating from the scope of the present disclosure.

[0115] At 702, a computing system can obtain segment identifying information for a portion of a transportation network. The segment identifying information can identify a set of transportation network segments located within the portion of the transportation network. In some implementations, the computing system can access a data store to obtain the segment identifying information. The data store can store the segment identifying information for the portion of the transportation network and additional segment identifying information a plurality of additional portions of the transportation network. The data store can be implemented by a navigation service.

[0116] In some implementations, the segment identifying information for the portion of the transportation network and the additional segment identifying information the plurality of additional portions of the transportation network can include a segment identification graph that represents the transportation network as a graph structure. Processing segment identifying information and the image depicting the portion of the transportation network can include processing the segment identifying information and the image depicting the portion of the transportation network with the machine-learned network identification model to obtain the refinement information. The refinement information can include a STATEful refinement update structured for ingestion by an API implemented by the navigation service for refinement of the graph structure stored to the data store.

[0117] In some implementations, to obtain the segment identifying information, the computing system can obtain the segment identifying information for the portion of the transportation network. The segment identifying information can include a sequence of coordinate pairs corresponding to the set of transportation network segments. Each of the sequence of coordinate pairs can include start coordinates and end coordinates for a corresponding transportation network segment of the set of transportation network segments.

[0118] At 704, the computing system can process the segment identifying information and an image depicting the portion of a transportation network with a machine-learned network identification model to obtain refinement information. The refinement information can be descriptive of one or more refinement updates for the segment identifying information.

[0119] In some implementations, for a first refinement update of the one or more refinement updates with the machine-learned network identification model, the computing system can identify a first transportation network segment to be updated from the set of transportation network segments. The computing system can select a first refinement updatetype for the first transportation network segment from a plurality of refinement update types comprising one or more of a deletion update type that removes a transportation network segment of the set of transportation network segments, a split update type that identifies two or more transportation network segments from a transportation network segment of the set of transportation network segments, a shift update type that modifies dimensions and / or a location of a transportation network segment of the set of transportation network segments, or an addition update type that adds a new transportation segment not previously identified by the segment identifying information. The computing system can generate the first refinement update for the first transportation network segment. The first refinement update can include the first refinement update ty pe.

[0120] In some implementations, to select the first refinement update t pe for the first transportation network segment, the computing system can select the first refinement update ty pe for the first transportation network segment from the plurality of refinement update ty pes. The first refinement update type can include the deletion update type. The first refinement update can be configured to remove the first transportation network segment from the set of network transportation network segments. In some implementations, the set of transportation network segments can include the first transportation network segment and a second transportation network segment. The first transportation network segment can include a first start coordinate and a first end coordinate. The second transportation network segment can include a second start coordinate and a second end coordinate, and the second start coordinate can include the first end coordinate.

[0121] In some implementations, the segment identifying information represents the portion of the transportation network as a graph structure. The structure can include a first edge between the first start coordinate and the first end coordinate, and the first edge can be representative of the first transportation network segment. To generate the first refinement update for the first transportation network segment, the computing system can generate the first refinement update for the first transportation network segment. The first refinement update can be configured to remove the edge between the first start coordinate and the first end coordinate. For example, assume that an edge exists between two vertices located at (0,0) and (2,2). If both vertices are connected to multiple edges, the first refinement update can remove the edge between the two vertices without deleting either vertex. Alternatively, if the vertex at (2,2) is only connected to one edge (e.g.. the edge that connects to the vertex at (0,0)), the first refinement update cna remove the edge and can further remove the vertex at (2,2).

[0122] In some implementations, to generate the first refinement update for the first transportation network segment, the computing system can determine an intermediate coordinate for the first transportation network segment. The computing system can generate the first refinement update for the first transportation network segment. The first refinement update can be configured to modify the segment identify ing information to include a first sub-segment coordinate pair and a second sub-segment coordinate pair for the first subsegment and the second sub-segment respectively. A start coordinate of the first sub-segment coordinate pair can include the first start coordinate of the first transportation network segment. An end coordinate of the first sub-segment coordinate pair can include the intermediate coordinate. A start coordinate of the second sub-segment coordinate pair can include the intermediate coordinate. An end coordinate of the second sub-segment coordinate pair can include the first end coordinate of the first transportation network segment.

[0123] In some implementations, selecting the first refinement update type for the first transportation network segment can include selecting the first refinement update type for the first transportation network segment from the plurality of refinement update types. The first refinement update type can include the split update type. The first refinement update can be configured to split the first transportation network segment into a first sub-segment and a second sub-segment.

[0124] In some implementations, selecting the first refinement update type for the first transportation network segment can include selecting the first refinement update type for the first transportation network segment from the plurality of refinement update types. The first refinement update type can include the shift update type. The first refinement update can be configured to modify the coordinate pair of the sequence of coordinate pairs that corresponds to first transportation network segment.

[0125] In some implementations, selecting the first refinement update type for the first transportation network segment can include selecting the first refinement update type for the first transportation network segment from the plurality' of refinement update types. The first refinement update type can include the addition update type. The first refinement update can be configured to add an additional coordinate pair for a newly identified transportation segment within the portion of the transportation network. The start coordinate of the additional coordinate pair can include a coordinate of some other coordinate pair of the sequence of coordinate pairs.

[0126] In some implementations, the addition update type can add coordinate pairs for a newly identified transportation segment within the portion of the transportation network. Forexample, in some instances, the model can identify a segment that cannot be defined based on a previously identified transportation segment. In response, the model can identify the segment as an edge between two coordinate pairs (i.e., two vertices). The model can connect one (or both) of the newly added coordinate pairs to existing and / or subsequent coordinate pair(s) as needed. Alternatively, based on the positioning of the newly added coordinate pairs, the model may refrain from connecting one (or both) of the newly added coordinate pairs to existing and / or subsequent coordinate pair(s). For example, if one of the coordinate pairs is positioned along the edge of the image processed by the model, it is unlikely that the model will connect that coordinate to another coordinate.

[0127] At 706, the computing system can modify the segment identifying information based on the one or more refinement updates. In some implementations, the computing system can access the data store implemented by the navigation service and apply the one or more refinement updates to the segment identifying information stored to the data store for the portion of the transportation network. In some implementations, to modify the segment identifying information based on the one or more refinement updates, the computing system can send the STATEful refinement update via the API implemented by the navigation service for refinement of the graph structure.

[0128] Figure 8 depicts a flow chart diagram of an example method 800 to perform generation segment identifying information with a machine-learned network identifying model to identify a portion of a transportation network according to some implementations of the present disclosure. Although Figure 8 depicts steps performed in a particular order for purposes of illustration and discussion, the methods of the present disclosure are not limited to the particularly illustrated order or arrangement. The various steps of the method 800 can be omitted, rearranged, combined, and / or adapted in various ways without deviating from the scope of the present disclosure.

[0129] At 802, a computing system can obtain a first image depicting a portion of a transportation network. The portion of the transportation network can include a set of transportation network segments.

[0130] At 804. the computing system can process the first image with the machine- learned network identification model to obtain segment identify ing information. The segment identifying information can include a sequence of coordinate pairs, each of the sequence of coordinate pairs corresponding to a transportation network segment of the set of transportation network segments.

[0131] At 806, the computing system can add the segment identifying information to a set of segment identifying information stored to a data store. The set of segment identifying information can include a plurality of sequences of coordinate pairs for a respective plurality of portions of the transportation network.Additional Disclosure

[0132] The technology discussed herein makes reference to servers, databases, software applications, and other computer-based systems, as well as actions taken and information sent to and from such systems. The inherent flexibility7of computer-based systems allows for a great variety of possible configurations, combinations, and divisions of tasks and functionality between and among components. For instance, processes discussed herein can be implemented using a single device or component or multiple devices or components working in combination. Databases and applications can be implemented on a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0133] While the present subject matter has been described in detail with respect to various specific example embodiments thereof, each example is provided by way of explanation, not limitation of the disclosure. Those skilled in the art, upon attaining an understanding of the foregoing, can readily produce alterations to, variations of, and equivalents to such embodiments. Accordingly, the subject disclosure does not preclude inclusion of such modifications, variations and / or additions to the present subject matter as would be readily apparent to one of ordinary skill in the art. For instance, features illustrated or described as part of one embodiment can be used with another embodiment to yield a still further embodiment. Thus, it is intended that the present disclosure covers such alterations, variations, and equivalents.

Claims

WHAT IS CLAIMED IS:

1. A computer-implemented method comprising: obtaining, by a computing system comprising one or more computing devices, segment identifying information for a portion of a transportation network, wherein the segment identifying information identifies a set of transportation network segments located within the portion of the transportation network; processing, by the computing system, the segment identifying information and an image depicting the portion of a transportation network with a machine-learned network identification model to obtain refinement information, wherein the refinement information is descriptive of one or more refinement updates for the segment identifying information; and modifying, by the computing system, the segment identifying information based on the one or more refinement updates.

2. The method of claim 1, wherein obtaining the segment identifying information comprises: accessing, by the computing system, a data store to obtain the segment identifying information, wherein the data store stores the segment identifying information for the portion of the transportation network and additional segment identifying information a plurality of additional portions of the transportation network, and wherein the data store is implemented by a navigation service.

3. The method of claim 2, wherein modifying the segment identifying information based on the one or more refinement updates comprises: accessing, by the computing system, the data store implemented by the navigation service; and applying, by the computing system, the one or more refinement updates to the segment identifying information stored to the data store for the portion of the transportation network.

4. The method of claim 2, wherein the segment identifying information for the portion of the transportation network and the additional segment identify ing information the plurality of additional portions of the transportation network comprise a segment identification graph that represents the transportation network as a graph structure; andwherein processing the segment identifying information and the image depicting the portion of the transportation network comprises: processing, by the computing system, the segment identifying information and the image depicting the portion of the transportation network with the machine-learned network identification model to obtain the refinement information, wherein the refinement information comprises a STATEful refinement update structured for ingestion by an Application Programming Interface (API), and wherein the API is implemented by the navigation service for refinement of the graph structure stored to the data store.

5. The method of claim 4, wherein modifying the segment identifying information based on the one or more refinement updates comprises: sending, by the computing system, the STATEful refinement update via the API implemented by the navigation service for refinement of the graph structure.

6. The computer-implemented method of claim 1, wherein processing the segment identification information and the image depicting the portion of the transportation network comprises: for a first refinement update of the one or more refinement updates with the machine- learned network identification model: identifying, by the computing system, a first transportation network segment to be updated from the set of transportation network segments; selecting, by the computing system, a first refinement update type for the first transportation network segment from a plurality of refinement update types comprising one or more of: a deletion update type that removes a transportation network segment of the set of transportation network segments; a split update type that identifies two or more transportation network segments from a transportation network segment of the set of transportation network segments; a shift update type that modifies dimensions and / or a location of a transportation network segment of the set of transportation network segments; or an addition update type that adds a new transportation segment not previously identified by the segment identifying information; andgenerating, by the computing system, the first refinement update for the first transportation network segment, wherein the first refinement update comprises the first refinement update type.

7. The computer-implemented method of claim 6, wherein obtaining the segment identifying information comprises: obtaining, by the computing system, the segment identifying information for the portion of the transportation network, wherein the segment identifying information comprises a sequence of coordinate pairs corresponding to the set of transportation network segments, and wherein each of the sequence of coordinate pairs comprises start coordinates and end coordinates for a corresponding transportation network segment of the set of transportation network segments.

8. The computer-implemented method of claim 7, wherein selecting the first refinement update type for the first transportation network segment comprises: selecting, by the computing system, the first refinement update type for the first transportation network segment from the plurality of refinement update types, wherein the first refinement update type comprises the deletion update type; and wherein the first refinement update is configured to remove the first transportation network segment from the set of network transportation network segments.

9. The computer-implemented method of claim 8, wherein the set of transportation network segments comprises the first transportation network segment and a second transportation network segment, wherein the first transportation network segment comprises a first start coordinate and a first end coordinate, wherein the second transportation network segment comprises a second start coordinate and a second end coordinate, and wherein the second start coordinate comprises the first end coordinate.

10. The computer-implemented method of claim 9, wherein the segment identifying information represents the portion of the transportation network as a graph structure, and wherein the structure comprises a first edge between the first start coordinate and the first end coordinate, and wherein the first edge is representative of the first transportation network segment; andwherein generating the first refinement update for the first transportation network segment comprises: generating, by the computing system, the first refinement update for the first transportation network segment, wherein the first refinement update is configured to remove the edge between the first start coordinate and the first end coordinate.

11. The computer-implemented method of claim 6. wherein selecting the first refinement update type for the first transportation network segment comprises: selecting, by the computing system, the first refinement update type for the first transportation network segment from the plurality of refinement update types, wherein the first refinement update type comprises the split update type; and wherein the first refinement update is configured to split the first transportation network segment into a first sub-segment and a second sub-segment.

12. The computer-implemented method of claim 11, wherein generating the first refinement update for the first transportation network segment comprises: determining, by the computing system, an intermediate coordinate for the first transportation network segment; and generating, by the computing system, the first refinement update for the first transportation network segment, wherein the first refinement update is configured to modify the segment identifying information to include a first sub-segment coordinate pair and a second sub-segment coordinate pair for the first sub-segment and the second sub-segment respectively, wherein: a start coordinate of the first sub-segment coordinate pair comprises the first start coordinate of the first transportation network segment; an end coordinate of the first sub-segment coordinate pair comprises the intermediate coordinate; a start coordinate of the second sub-segment coordinate pair comprises the intermediate coordinate; and an end coordinate of the second sub-segment coordinate pair comprises the first end coordinate of the first transportation network segment.

13. The computer-implemented method of claim 6, wherein selecting the first refinement update type for the first transportation network segment comprises:selecting, by the computing system, the first refinement update type for the first transportation network segment from the plurality of refinement update types, wherein the first refinement update type comprises the shift update type; and wherein the first refinement update is configured to modify a coordinate pair of the sequence of coordinate pairs that corresponds to the first transportation network segment.

14. The computer-implemented method of claim 6, wherein selecting the first refinement update type for the first transportation network segment comprises: selecting, by the computing system, the first refinement update type for the first transportation network segment from the plurality of refinement update types, wherein the first refinement update type comprises the addition update type; and wherein the first refinement update is configured to add an additional coordinate pair for a newly identified transportation segment within the portion of the transportation network, wherein the start coordinate of the additional coordinate pair comprises a coordinate of some other coordinate pair of the sequence of coordinate pairs.

15. A computing system, comprising: one or more processor devices; a machine-learned network identification model trained to process imagery to sequentially identify segments of a transportation network depicted by the imagery; and one or more tangible, non-transitory computer readable media storing computer- readable instructions that when executed by the one or more processor devices cause the one or more processor devices to perform operations, the operations comprising: obtaining a first image depicting a portion of a transportation network, wherein the portion of the transportation network comprises a set of transportation network segments; processing the first image with the machine-learned network identification model to obtain segment identifying information, wherein the segment identifying information comprises a sequence of coordinate pairs, each of the sequence of coordinate pairs corresponding to a transportation netw ork segment of the set of transportation netw ork segments; and adding the segment identifying information to a set of segment identifying information stored to a data store, wherein the set of segment identifying informationcomprises a plurality of sequences of coordinate pairs for a respective plurality of portions of the transportation network.

16. The computing system of claim 15, wherein the operations further comprise: obtaining a second image different than the first image, wherein the second image depicts the portion of the transportation network comprising the set of transportation network segments; processing the second image and the segment identifying information with the machine-learned network identification model to obtain refinement information, wherein the refinement information is descriptive of one or more refinement updates for the segment identifying information; and applying the one or more refinement updates to the segment identifying information stored to the data store for the portion of the transportation network.

17. The computing system of claim 16, wherein processing the segment identification information and the second image compnses: for a first refinement update of the one or more refinement updates with the machine- learned network identification model: identifying a first transportation network segment to be updated from the set of transportation network segments of the segment identifying information; selecting a first refinement update type for the first transportation network segment from a plurality of refinement update types comprising one or more of: a deletion update type that removes a transportation network segment of the set of transportation network segments; a split update type that identifies two or more transportation network segments from a transportation network segment of the set of transportation network segments; a shift update type that modifies dimensions and / or a location of a transportation network segment of the set of transportation network segments; or an addition update type that adds a new transportation segment not previously identified by the segment identifying information; and generating the first refinement update for the first transportation network segment.

18. The computing system of claim 17, wherein the segment identifying information comprises a sequence of coordinate pairs corresponding to the set of transportation network segments, and wherein each of the sequence of coordinate pairs comprises start coordinates and end coordinates for a corresponding transportation network segment of the set of transportation network segments.

19. The computing system of claim 17, wherein processing the first image with the machine-learned network identification model comprises: for each transportation network segment of the set of transportation network segments: processing the first image with the machine-learned network identification model to identify the transportation network segment of the set of transportation network segments; determining a coordinate pair that represents the transportation network segment; selecting the addition update type for the transportation network segment; and appending the coordinate pair to the sequence of coordinate pairs of the segment identifying information.

20. One or more tangible, non-transitory computer readable media storing computer- readable instructions that when executed by one or more processor devices cause the one or more processor devices to perform operations, the operations comprising: obtaining segment identifying information for a portion of a transportation network, wherein the segment identifying information identifies a set of transportation network segments located within the portion of the transportation network; processing the segment identifying information and an image depicting the portion of a transportation network with a machine-learned network identification model to obtain refinement information, wherein the refinement information is descriptive of one or more refinement updates for the segment identifying information; and modifying the segment identifying information based on the one or more refinement updates.

Images