Generating customized digital maps via generating machine learning models

Abstract

A computing device for generating a customized digital map includes one or more memories for storing instructions and one or more processors for executing the instructions to perform operations including: receiving, from a user, an input related to customizing a feature associated with a location viewable on the digital map; in response to receiving the input, implementing a generating machine learning model to generate the customized digital map depicting the location having one or more customized features generated via the generating machine learning model based on the input; and providing the customized digital map for presentation via a display device.

Description

Technical Field

[0001] The present disclosure as a whole relates to providing a customized digital map at a computing device via a generated machine learning model. For example, the present disclosure relates to a method and computing device for providing a customized digital map related to a location via a generated machine learning model provided at a computing device associated with a user in response to a user input related to the location. Background Art

[0002] According to current methods, user computing devices download maps and satellite images / tiles from server computing systems, such as for navigation applications. In addition, there is an application programming interface (API) for users to customize these maps, wherein the server computing system renders the customized tiles, and the user computing device downloads the customized tiles from the server computing system. Summary of the invention

[0003] Aspects and advantages of embodiments of the disclosure will be set forth in part in the description which follows, or may be learned from the description, or may be learned by practicing example embodiments.

[0004] In one or more example embodiments, a computing device for generating a customized digital map is provided. For example, the computing device for generating a customized digital map includes: a display device; one or more memories configured to store instructions; and one or more processors configured to execute the instructions to perform operations, the operations including: receiving input from a user related to customizing features associated with a location viewable on a digital map; in response to receiving the input, implementing a generation machine learning model to generate the customized digital map, the customized digital map depicting the location with one or more customized features generated via the generation machine learning model based on the input; and providing the customized digital map for presentation via the display device.

[0005] In some implementations, the generative machine learning model is provided at the computing device.

[0006] In some implementations, receiving the input from the user related to customizing features associated with the location viewable on the digital map includes receiving a semantic description of a tile associated with the digital map to be generated.

[0007] In some implementations, the generative machine learning model is configured to generate the customized digital map by rendering a portion of the customized digital map depicting the location having the one or more customized features, and the operations further include receiving a remaining portion of the customized digital map rendered by the server computing system.

[0008] In some implementations, the operations further include implementing one or more machine learning models to smooth and blend the portion of the customized digital map rendered by the computing device and the remaining portion of the customized digital map rendered by the server computing system.

[0009] In some implementations, the operations also include receiving one or more default tiles associated with the location before receiving the input from the user, and the generative machine learning model is configured to generate the customized digital map by referencing the one or more default tiles and the input to render the customized digital map depicting the location with the one or more customized features.

[0010] In some implementations, the one or more default tiles associated with the location are received at a predetermined time according to a predetermined condition or according to a predetermined event.

[0011] In some implementations, the predetermined time occurs periodically, the predetermined condition is associated with a network condition of a network through which the one or more default tiles are received from the server computing system, and the predetermined event is associated with a state of the computing device.

[0012] In some implementations, the input indicates specific content to be omitted or reduced when generating the customized digital map, and implementing the generation of the machine learning model to generate the customized digital map depicting the location with the one or more customized features includes omitting or reducing the specific content when generating the customized digital map.

[0013] In some implementations, receiving the input from the user related to customizing features associated with the location viewable on the digital map includes receiving a selection of a user interface element corresponding to a custom map layer, and in response to the selection of the user interface element, implementing the generative machine learning model to generate the customized digital map, which depicts the location with one or more customized features generated via the generative machine learning model based on the input.

[0014] In some implementations, the operations further include, in response to the selection of the user interface element, generating the custom map layer based on at least one of information associated with the user or contextual information associated with the location.

[0015] In some implementations, the computing device includes one or more databases configured to store multiple generated machine learning models that are respectively associated with multiple different locations, and the operations also include retrieving the generated machine learning model associated with the location from the multiple generated machine learning models.

[0016] In some implementations, the input includes a text query specifying one or more objects to be associated with the location, and the digital map depicts the location including the one or more objects.

[0017] In some implementations, the generative machine learning model has been fine-tuned based on a large-parameter generative machine learning model having a greater number of parameters than the generative machine learning model.

[0018] In one or more example embodiments, a computer-implemented method for generating a customized digital map is provided. The computer-implemented method includes: receiving, by a computing device, input from a user related to customizing features associated with a location viewable on a digital map; in response to receiving the input, implementing, by the computing device, a generation machine learning model to generate the customized digital map, the customized digital map depicting the location with one or more customized features generated by the generation machine learning model based on the input; and providing, by the computing device, the customized digital map for presentation via a display device.

[0019] In some implementations, the generative machine learning model is provided at the computing device.

[0020] In some implementations, receiving the input from the user related to customizing features associated with the location viewable on the digital map includes receiving a semantic description of a tile associated with the digital map to be generated.

[0021] In some implementations, implementing the generating the machine learning model includes rendering a portion of the customized digital map depicting the location having the one or more customized features, and receiving a remaining portion of the customized digital map rendered by a server computing system.

[0022] In some implementations, the input indicates specific content to be omitted or reduced when generating the customized digital map, and implementing the generative machine learning model includes omitting or reducing the specific content when generating the customized digital map.

[0023] In one or more example embodiments, a computer-readable medium (e.g., a non-transitory computer-readable medium) storing instructions is provided, which can be executed by one or more processors of a computing system. In some implementations, the computer-readable medium stores instructions that may include instructions to cause the one or more processors to perform one or more operations associated with any of the methods described herein (e.g., operations of the server computing system and / or operations of the computing device). For example, these operations may include: receiving input from a user related to customizing features associated with a location viewable on a digital map; in response to receiving the input, implementing a generated machine learning model to generate a customized digital map, the customized digital map depicting the location with one or more customized features generated via the generated machine learning model based on the input; and providing the customized digital map for presentation via a display device. The computer-readable medium may store additional instructions to perform other aspects of the server computing system and computing device and corresponding operating methods, as described herein.

[0024] These and other features, aspects and advantages of various embodiments of the present disclosure will be better understood with reference to the following description, drawings and appended claims.The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate example embodiments of the present disclosure and, together with the description, serve to explain the relevant principles. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] A detailed discussion of example embodiments for persons of ordinary skill in the art is set forth in this specification with reference to the accompanying drawings, in which:

[0026] Figure 1A to Figure 1B Depicting an example system according to one or more example embodiments of the present disclosure;

[0027] Figure 2 A flowchart illustrating an example non-limiting computer-implemented method according to one or more example embodiments of the present disclosure;

[0028] Figure 3 An example block diagram of a computing device according to one or more example embodiments of the present disclosure is depicted;

[0029] FIG. 4A to FIG. 4B shows an example user interface screen of a mapping or navigation application according to one or more example embodiments of the present disclosure;

[0030] FIG. 5A to FIG. 5B shows an example user interface screen of a mapping or navigation application according to one or more example embodiments of the present disclosure;

[0031] Figure 6shows a block diagram including an example of a map data layer according to an example embodiment of the present disclosure;

[0032] Fig. 7A depicts a block diagram of an example computing system for generating a customized digital map via generating a machine learning model in response to receiving an input, according to one or more example embodiments of the present disclosure;

[0033] Figure 7B depicts a block diagram of an example computing device for generating a customized digital map via generating a machine learning model in response to receiving an input, according to one or more example embodiments of the present disclosure;

[0034] Figure 7C A block diagram of an example computing device for generating a customized digital map via generating a machine learning model in response to receiving an input is depicted in accordance with one or more example embodiments of the present disclosure. DETAILED DESCRIPTION

[0035] Reference will now be made to embodiments of the present disclosure, one or more examples of which are illustrated in the accompanying drawings, wherein like reference numerals represent like elements. Each example is provided by way of explanation of the present disclosure and is not intended to limit the present disclosure. In fact, it will be apparent to those skilled in the art that various modifications and changes may be made to the present disclosure without departing from the scope or spirit of the present disclosure. For example, a feature illustrated or described as part of one embodiment may be used together with another embodiment to produce yet another embodiment. Therefore, the present disclosure is intended to encompass modifications and changes within the scope of the appended claims and their equivalents.

[0036] The terms used herein are used to describe example embodiments and are not intended to limit and / or constrain the present disclosure. As used herein, the singular forms "a", "an", and "the" are also intended to include the plural forms unless the context clearly indicates otherwise. In the present disclosure, terms such as "including", "having", "comprising", etc. are used to specify features, numbers, steps, operations, elements, components, or combinations thereof, but do not exclude the presence or addition of one or more of the features, elements, steps, operations, elements, components, or combinations thereof.

[0037] It will be understood that although the terms first, second, third, etc. may be used herein to describe various elements, these elements should not be limited to these terms. Instead, these terms are used to distinguish one element from another element. For example, without departing from the scope of the present disclosure, a first element may be referred to as a second element, and a second element may be referred to as a first element.

[0038] The term "and / or" includes a combination of multiple related listed items or any of the multiple related listed items. For example, the scope of the expression or phrase "A and / or B" includes the item "A", the item "B" and the combination of the items "A and B".

[0039] In addition, the scope of the expression or phrase "at least one of A or B" is intended to include all of the following: (1) at least one of A; (2) at least one of B; and (3) at least one of A and at least one of B. Likewise, the scope of the expression or phrase "at least one of A, B, or C" is intended to include all of the following: (1) at least one of A; (2) at least one of B; (3) at least one of C; (4) at least one of A and at least one of B; (5) at least one of A and at least one of C; (6) at least one of B and at least one of C; and (7) at least one of A, at least one of B, and at least one of C.

[0040] According to current methods, a large amount of bandwidth and computing resources are required to render and download maps and satellite images / tiles from a server computing system to a user computing device. In addition, an application programming interface (API) exists for users to customize these maps, where the server computing system renders the customized tiles and the user computing device downloads the customized tiles from the server computing system. This method requires network resources (e.g., bandwidth) to exchange data between the user computing device and the server computing system, and may be problematic in low bandwidth or low network coverage areas, especially when each map may be rendered and downloaded multiple times to create different variations.

[0041] According to an example embodiment of the present disclosure, a user input that may include a semantic description of a map / tile to be rendered may be provided to a user computing device, and then a generative machine learning model provided at the user computing device may render a customized map (digital map) directly at the user computing device, thereby saving bandwidth and enabling high-fidelity map usage in low network coverage areas. By rendering a map on a device via a generative machine learning model, customized or personalized content may be implemented for a digital map while reducing bandwidth usage, including making more useful, task-specific map tiles that can help users, thereby eliminating constraints that may today require universal map tiles for all users across all tasks.

[0042] According to an example embodiment of the present disclosure, customized digital maps may be generated in real time via on-device machine learning models, thereby saving bandwidth, achieving personalization, and improving customer experience.

[0043] In some implementations, an existing generative machine learning model may be fine-tuned or refined to improve its performance on specific tasks, including rendering map tiles. Thus, a generative machine learning model for customizing digital maps may have improved speed and reduced size (thereby saving storage space) compared to existing models, and may be more easily deployed on user computing devices.

[0044] For example, a computing device (e.g., a server computing system, a training computer system, etc.) may be configured to fine-tune an existing generative machine learning model on a dataset of map tiles to train the generative machine learning model to generate realistic and accurate maps from text descriptions. The generative machine learning model may also be refined to remove unnecessary parameters and make it faster and smaller. For example, a computing device may be configured to fine-tune an existing model to be task-specific, such that a large parameter model (e.g., a 55 billion parameter model such as PaLI) may be refined to a smaller parameter model (e.g., a model with a billion parameters or less).

[0045] Once trained, the generative machine learning model can be used to render a map in real time on a user computing device with limited bandwidth. For example, a user can provide input including a text description related to a location, and the generative machine learning model can generate a detailed map of the location customized to the user's input, such as by showing a particular type of business or landmark or by depicting a business or landmark in a particular way.

[0046] An example semantic description may include prompts such as "Generate a map tile of Boston Long Wharf near the Aquarium. Make the water deep blue with waves.". The semantic description may include various other constraints or conditions, such as "Make a coffee shop appear on the corner. The coffee shop's name is Tatte's Bakery." The user may also provide further context or input to the generative machine learning model to shape the output, such as providing an image of the coffee shop's logo to render the sign and providing size information (e.g., "Make the sign somewhat large relative to the route. Render the start pin here and the destination pin here. Make the pins look nautical themed like a boat anchor.").

[0047] According to the disclosed embodiments, computing resource usage and network resource usage are also improved. For example, delegating the implementation of generating machine learning models to user computing devices can reduce network traffic and reduce the use of server computing system (cloud) resources.

[0048] In some implementations, the generative machine learning model may include a generative adversarial network, a variational autoencoder, a stable diffusion machine learning model, a visual transformer, etc. In addition, methods including singular value decomposition, LU decomposition, and principal component analysis may be used to achieve dimensionality reduction through a user computing device to obtain a generative machine learning model that can be implemented by a user computing device.

[0049] In some implementations, the user computing device and the server computing system may be configured to perform hybrid rendering, where the user computing device renders a portion of the map and the remaining portion is rendered by the server computing system. For example, a machine learning model (e.g., at the user computing device) may be configured to smooth and blend portions of the map rendered at the server computing system and portions of the map rendered at the user computing device so that the entire map has a consistent and similar appearance.

[0050] In some implementations, the default tile or starting tile may be used as seed data by the user computing device, wherein the generation machine learning model is configured to reference the default tile or starting tile when rendering the map. In some implementations, the user computing device may be configured to download or store a local template including the default tile or starting tile received from the server computing system. For example, the default tile or starting tile may be downloaded by the user computing device under specific conditions (e.g., when the available bandwidth is greater than a threshold, when the network traffic is less than a threshold, when the user computing device is connected to a WiFi network or a cellular network, etc.), in association with a specific event (e.g., during initial application installation, during application updates, etc.) or at a specific time (e.g., weekly, monthly, etc.).

[0051] According to examples of the present disclosure, each user can personalize their map experience (e.g., according to the user's tastes or needs, such as the destination establishment, the category or type of routes divided by task, etc.). For example, if a user wants to go grocery shopping and view the grocery store, other content that might otherwise appear on the map may be visually cluttered. The additional content also needs to be rendered (at the server computing system or the user computing device). Therefore, the user can provide input to the generative machine learning model to omit irrelevant content from the map, minimize irrelevant content, display irrelevant content in less detail, etc., thereby saving computing resources.

[0052] As another example, if the user is interested in bicycle paths, the generative machine learning model may be configured to focus on bicycle paths when generating a customized digital map. According to existing methods, when a user wishes to view bicycle paths on a map, the server computing system may render a map with bicycle paths at relevant locations, for example, by re-rendering map tiles or by utilizing a map stored in a memory (cache) that may have been previously rendered and satisfies the user's query. However, according to examples of the present disclosure, the generative machine learning model may be configured to customize map tiles in a manner specific to the user. For example, based on a user query or input or other conditions, the generative machine learning model may be configured to not render portions of the map that are not related to cycling, so that the map only shows routes that can be used for cycling, and may only show stores or facilities that are relevant to cyclists (e.g., showing bicycle stores without rendering fast food restaurants). Therefore, computing resources can be efficiently utilized by rendering only portions of the map that are relevant to the user input.

[0053] In some implementations, the generated machine learning model may be configured to render locations or points of interest that may be of interest to the user. For example, the generated machine learning model may be configured to determine that a location or point of interest may be of interest to the user based on user input, based on contextual information associated with the user (e.g., user preferences, past purchases by the user, historical information associated with the user, interests based on the user's friends, etc.), based on external information (e.g., current events, local alerts, traffic alerts, etc.). The generated machine learning model may be configured to render the location or point of interest in a manner that prominently displays the location or point of interest relative to other locations or points of interest (e.g., in a visually unique manner, such as in an enlarged manner, using bright colors, or in a highlighted manner). In addition, the digital map may highlight the location or point of interest in a stylized or themed manner. For example, the icon or symbol representing the location or point of interest may match or correspond to the user input or query, or may match or correspond to a scene associated with the user (e.g., for a user who skateboards and often drinks coffee, the shape of the icon representing a coffee shop displayed on the map may resemble a skateboard).

[0054] According to examples of the present disclosure, the generated machine learning model can be configured to render tiles at the user computing device to satisfy a user's request to customize the map and / or to conform to the task the user is performing (e.g., requesting directions to a point of interest). Compared to the current method of providing a universal map shared by all users in all tasks, according to the examples disclosed herein, customization of the map can be achieved in real time, thereby improving the user experience and facilitating navigation (e.g., making it easier to add routes, make changes to routes, identify relevant locations or points of interest in a user-specific manner, etc.).

[0055] One or more technical benefits of the present disclosure also include generating a customized digital map associated with a location via one or more machine learning models (e.g., a generative machine learning model) in response to receiving a query. For example, a generative machine learning model may be provided at a user computing device to render a map on the user computing device, wherein the map is customized or has personalized content. Rendering a map at a user computing device via a generative machine learning model reduces bandwidth usage compared to methods that require a user computing device to download tiles from a server computing system, and produces more useful, task-specific map tiles that can assist users, thereby eliminating constraints that may today require universal map tiles for all users across all tasks.

[0056] Another technical benefit of the present disclosure includes providing a fine-tuned or refined generative machine learning model for customizing digital maps. The fine-tuned or refined generative machine learning model can have improved speed and reduced size (thereby saving storage space) compared to existing models, and can be more easily deployed on user computing devices.

[0057] Another technical benefit of the present disclosure includes increasing device uptime by enabling map rendering when map rendering is not possible due to low bandwidth or reduced network availability.

[0058] Another technical benefit of the present disclosure includes embodiments in which a user computing device is configured to omit (or minimize or display in less detail) irrelevant content displayed on a digital map generated via a generative machine learning model, thereby conserving computing resources (e.g., processing power) and resulting in faster performance.

[0059] Another technical benefit of the present disclosure includes embodiments in which navigation may be performed more accurately or in an easier manner by displaying points of interest in a manner that is beneficial or personalized to the user, wherein the user may easily identify waypoints, routes, points of interest, etc. along a route during navigation.

[0060] Now referring to the accompanying drawings, Figure 1A is an example system according to one or more example embodiments of the present disclosure. Figure 1A An example of a system 1000 is shown, which includes a computing device 100, an external computing device 200, a server computing system 300, and external content 500, which can communicate with each other through a network 400. For example, the computing device 100 and the external computing device 200 may include any one of a personal computer, a smart phone, a tablet computer, a global positioning service device, a smart watch, etc. The network 400 may include any type of communication network, including a wired or wireless network or a combination thereof. The network 400 may include a local area network (LAN), a wireless local area network (WLAN), a wide area network (WAN), a personal area network (PAN), a virtual private network (VPN), etc. For example, wireless communication between elements of the example embodiment may be performed via wireless LAN, Wi-Fi, Bluetooth, ZigBee, Wi-Fi Direct (WFD), ultra-wideband (UWB), infrared data association (IIrDA), Bluetooth low energy (BLE), near field communication (NFC), radio frequency (RF) signals, etc. For example, wired communication between elements of the example embodiment may be performed via twisted pair, coaxial cable, fiber optic cable, Ethernet cable, etc. Communications over network 400 may use a variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or protection schemes (e.g., VPN, Secure HTTP, SSL).

[0061] As will be explained in more detail below, in some implementations, computing device 100 and / or server computing system 300 may form part of a navigation and mapping system that can provide a customized digital map of a location to a user of computing device 100 via generating a machine learning model.

[0062] In some example embodiments, the server computing system 300 may obtain data from one or more of the POI data repository 350, the navigation data repository 360, the user data repository 370, and the machine learning model data repository 380 to implement various operations and aspects of the navigation and mapping system as disclosed herein. The POI data repository 350, the navigation data repository 360, the user data repository 370, and the machine learning model data repository 380 may be provided integrally with the server computing system 300 (e.g., as part of one or more memory devices 320 of the server computing system 300) or may be provided separately (e.g., remotely). In addition, the POI data repository 350, the navigation data repository 360, the user data repository 370, and the machine learning model data repository 380 may be combined into a single data repository (database), or may include multiple corresponding data repositories. The data stored in one data repository (e.g., the POI data repository 350) may overlap with some of the data stored in another data repository (e.g., the navigation data repository 360). In some implementations, one data repository (e.g., machine learning model data repository 380) may reference data stored in another data repository (e.g., user data repository 370).

[0063] The POI data repository 350 may store information about locations or points of interest, for example, information about points of interest in an area or region associated with one or more geographic areas. Points of interest may include any destination or location. For example, points of interest may include restaurants, museums, stadiums, concert halls, amusement parks, schools, places of business, grocery stores, gas stations, theaters, shopping centers, lodging, etc. The point of interest data stored in the POI data repository 350 may include any information associated with the POI. For example, the POI data repository 350 may include location information of the POI, business hours of the POI, phone numbers of the POI, reviews about the POI, financial information associated with the POI (e.g., average cost of services provided and / or goods sold at the POI (such as meals, tickets, rooms, etc.)), environmental information about the POI (e.g., noise levels, environmental descriptions, traffic levels, etc. that may be provided or obtained in real time by various sensors located at the POI), descriptions of the types of services provided and / or goods sold, languages ​​spoken at the POI, URLs of the POI, image content associated with the POI, etc. For example, information about the POI may be obtained from external content 500 (eg, from a web page associated with the POI or from a sensor provided at the POI).

[0064] The navigation data repository 360 may store or provide map data / geospatial data to be used by the server computing system 300. Example geospatial data includes geographic imagery (e.g., digital maps, satellite images, aerial photographs, street-level photographs, synthetic models, etc.), tables, vector data (e.g., vector representations of roads, land parcels, buildings, etc.), point of interest data, or other suitable geospatial data associated with one or more geographic areas. In some examples, the map data may include a series of sub-maps, each sub-map including data for a geographic area including objects (e.g., buildings or other static features), travel paths (e.g., roads, highways, public transportation routes, walking paths, etc.), and other features of interest. The navigation data repository 360 may be used by the server computing system 300 to provide navigation directions, perform point of interest searches, provide point of interest location or classification data, determine distances, routes, or travel times between locations, or any other suitable purpose or task necessary or beneficial to perform the operation of the example embodiments disclosed herein.

[0065] In some examples, user data repository 370 may include current user location and heading data. In some examples, user data repository 370 may include information about one or more user profiles, including a variety of user data, such as user preference data, user demographic data, user calendar data, user social network data, user historical travel data, etc. For example, user data repository 370 may include, but is not limited to, email data, including text content, images, calendar information or contact information associated with emails; social media data, including comments, reviews, check-ins, likes, invitations, contacts, or reservations; calendar application data, including dates, times, events, descriptions, or other content; virtual wallet data, including purchases, e-tickets, coupons, or transactions; scheduling data; location data; SMS data; or other suitable data associated with a user account. According to one or more examples of the present disclosure, data may be analyzed to determine user preferences regarding POIs, such as to automatically suggest or automatically provide customized features regarding attributes representing locations or representing user preferences (e.g., displaying POIs using user favorite colors, displaying POIs using logos or symbols associated with user favorite artists or favorite animals, etc.), wherein a machine learning model is generated to generate a customized digital map with locations annotated or represented with customized features associated with the user. Data may be analyzed to determine user preferences regarding POIs, such as to determine user preferences regarding travel (e.g., mode of transportation, allowed travel time, etc.), determine possible recommendations for POIs for users, determine possible travel routes and modes of transportation for users to POIs, etc.

[0066] The user data repository 370 is provided to illustrate potential data that can be analyzed by the server computing system 300 in some embodiments to identify user preferences, recommend POIs, determine possible travel routes to POIs, determine transportation methods for going to POIs, determine videos of locations to be provided to computing devices associated with users, generate customized digital maps, etc. However, unless the user consents after being informed of what data is collected and how the data is used, such user data may not be collected, used, or analyzed. In addition, in some embodiments, tools may be provided to the user (e.g., in a navigation application or via a user account) to revoke or modify the scope of permissions. In addition, certain information or data may be processed in one or more ways before being stored or used so that personally identifiable information is removed or stored in an encrypted manner. Therefore, specific user information stored in the user data repository 370 may or may not be accessible by the server computing system 300 based on the permissions given by the user, or such data may not be stored in the user data repository 370 at all.

[0067] The machine learning model data repository 380 may store machine learning models that may be retrieved and implemented by the server computing system 300 for use in generating a refined or fine-tuned machine learning model (e.g., a refined or fine-tuned generated machine learning model) that may be provided to the computing device 100. The machine learning model data repository 380 may also store refined or fine-tuned machine learning models (e.g., a refined or fine-tuned generated machine learning model) that may be retrieved and implemented by the computing device 100. In some implementations, the computing device 100 may retrieve and implement machine learning models that are large parameter models that have not yet been fine-tuned or refined. The machine learning models (including large parameter models and refined or fine-tuned models) stored at the machine learning model data repository 380 may include multiple generative machine learning models respectively associated with multiple different locations. In some implementations, the machine learning model includes multiple generative machine learning models respectively associated with specific objects or structures provided at multiple different locations. The machine learning model may include a large language model (e.g., a Bidirectional Encoder Representations from Transformers (BERT) large language model). The machine learning model may include a generative artificial intelligence (AI) model (e.g., Bard), which may implement a generative adversarial network (GAN), a Transformer, a variational autoencoder (VAE), a neural radiance field (NeRF), etc. NeRF can be trained to learn a continuous volume scene function that can assign color and volume density to any voxel in space. The weights of the NeRF network can be optimized to encode a representation of the scene so that the model can render a new view seen from any point in space.

[0068] External content 500 may be any form of external content, including news articles, web pages, video files, audio files, written descriptions, ratings, game content, social media content, photos, commercial offers, transportation methods, weather conditions, sensor data obtained by various sensors, or other suitable external content. Computing device 100, external computing device 200, and server computing system 300 may access external content 500 via network 400. External content 500 may be searched by computing device 100, external computing device 200, and server computing system 300 according to known search methods, and search results may be ranked according to relevance, popularity, or other suitable attributes (including location-specific filtering or promotion).

[0069] Reference now Figure 1B , an example block diagram of a computing device and a server computing system according to one or more example embodiments of the present disclosure will now be described. Figure 1B, a computing device 100 is represented in FIG. 2 , but the features of the computing device 100 described herein are also applicable to the external computing device 200 .

[0070] Computing device 100 may include one or more processors 110, one or more memory devices 120, a navigation and mapping system 130, a position determination device 140, an input device 150, a display device 160, an output device 170, and a capture device 180. Server computing system 300 may include one or more processors 310, one or more memory devices 320, and a navigation and mapping system 330.

[0071] For example, the one or more processors 110, 310 may be any suitable processing device that may be included in the computing device 100 or the server computing system 300. For example, the one or more processors 110, 310 may include one or more of a processor, a processor core, a controller and an arithmetic logic unit, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an image processor, a microcomputer, a field programmable array, a programmable logic unit, an application specific integrated circuit (ASIC), a microprocessor, a microcontroller, etc. and combinations thereof, including any other device that can respond and execute instructions in a defined manner. The one or more processors 110, 310 may be a single processor or a plurality of processors that are operably connected (e.g., in parallel).

[0072] The one or more memory devices 120, 320 may include one or more non-transitory computer-readable storage media, including read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM) and flash memory, USB drive; volatile memory devices, including random access memory (RAM), hard disk, floppy disk, Blu-ray disc or optical media such as CD ROM disc and DVD; and combinations thereof. However, examples of the one or more memory devices 120, 320 are not limited to the above description, and the one or more memory devices 120, 320 may be implemented by other various devices and structures as will be understood by those skilled in the art.

[0073] For example, one or more memory devices 120 may store instructions that, when executed, cause one or more processors 110 to execute a generated map application 132 and execute the instructions to perform operations including: receiving input from a user related to customizing features associated with a location viewable on a digital map; in response to receiving the input, implementing a generated machine learning model to generate a customized digital map that depicts the location with one or more customized features generated via the generated machine learning model based on the input; and providing the customized digital map for presentation via a display, as described in an example according to the present disclosure.

[0074] The one or more memory devices 320 may also include data 322 and instructions 324 that may be retrieved, manipulated, created, or stored by the one or more processors 310. In some example embodiments, such data may be accessed and used as input to implement a generated map application 332 and execute the instructions to perform operations including: receiving input from a user related to customizing features associated with a location viewable on a digital map; in response to receiving the input, implementing a generated machine learning model to generate a customized digital map that depicts the location with one or more customized features generated via the generated machine learning model based on the input; and providing the customized digital map for presentation via a display, as described in accordance with examples of the present disclosure.

[0075] In some example embodiments, computing device 100 includes a navigation and mapping system 130. For example, navigation and mapping system 130 may include a map generation application 132 and a navigation application 134.

[0076] According to an example of the present disclosure, a map generation application 132 may be executed by a computing device 100 to provide a user of the computing device 100 with a way to customize one or more features or points of interest associated with a digital map presented to the user via a display device 160. The map generation application 132 may be a part of a navigation application 134 or a separate mapping application, or may be an independent application. The map generation application 132 may be configured to dynamically interact according to various user inputs. For example, the map generation application 132 may be configured to modify or customize the digital map in real time in response to user input or query by implementing a generated machine learning model (e.g., requesting a voice input that shows an object in a digital map with a user's favorite color and has a specific theme). The map generation application 132 may be configured to dynamically (e.g., in real time) generate a customized digital map related to a location according to user input. Other aspects of the map generation application 132 will be described herein.

[0077] In some examples, one or more aspects of the generated map application 132 may be implemented by a generated map application 332 of a server computing system 300, which may be located remotely, to generate and / or provide a customized digital map in response to receiving input from a user. In some examples, one or more aspects of the generated map application 332 may be implemented by a generated map application 132 of a computing device 100, to generate and / or provide a customized digital map in response to receiving input from a user.

[0078] According to an example of the present disclosure, the navigation application 134 may be executed by the computing device 100 to provide a way (route) of navigating to a location to the user of the computing device 100. The navigation application 134 may provide navigation services to the user. In some examples, the navigation application 134 may facilitate user access to the server computing system 300 that provides navigation services. In some example embodiments, the navigation service includes providing directions to a specific location such as a POI. For example, the user may enter a destination location (e.g., the address or name of the POI or the category of the POI). In response, the navigation application 134 may use the locally stored map data of a specific geographic area and / or the map data provided via the server computing system 300 to provide navigation information that allows the user to navigate to the destination location. For example, the navigation information may include turn-by-turn instructions from the current location (or the starting point or departure location provided) to the destination location. For example, the navigation information may include travel time (e.g., estimated or predicted travel time) from the current location (or the starting point or departure location provided) to the destination location.

[0079] Navigation application 134 may provide a visual depiction of a geographic area via display device 160 of computing device 100. The visual depiction of the geographic area may include one or more streets, one or more points of interest (including buildings, landmarks, etc.), and a highlighted depiction of a planned route. In some examples, navigation application 134 may also provide a location-based search option to identify one or more searchable points of interest within a given geographic area. In some examples, navigation application 134 may include a local copy of relevant map data. In other examples, navigation application 134 may access information that may be located at a remote server computing system 300 to provide the requested navigation service.

[0080] In some examples, navigation application 134 may be a dedicated application specifically designed to provide navigation services. In other examples, navigation application 134 may be a general purpose application (e.g., a web browser), and may provide access to a variety of services including navigation services via network 400.

[0081] For example, the navigation and mapping system 130 may store customized digital maps previously generated using one or more machine learning models (e.g., generating machine learning models) in one or more memory devices 120 (e.g., in a cache). The customized digital maps may be categorized or classified according to the location, the scene in which the customized digital map was generated (e.g., according to the type of POI, according to the activity associated with the user's input that caused the customized digital map to be generated, the time of day, the time of year, the weather conditions, the lighting conditions, etc.). For example, the navigation data repository 360 may be configured to store customized digital maps generated using one or more machine learning models stored at the machine learning model data repository 380. Example customized digital maps may include a customized digital map associated with a specific bicycle route that a user takes to work, a customized digital map associated with the user's favorite types of food served at restaurants in the area where the user lives, a customized digital map associated with the user's favorite colors (where buildings associated with the user input are displayed on the map using the favorite colors), etc. In some implementations, the computing device 100 may be configured to retrieve the customized digital map from the navigation data repository 360 (or from local memory) when the customized digital map matches or corresponds to a query or input from a user requesting the customized digital map.

[0082] For example, the navigation and mapping system 130 (e.g., the generating map application) may be configured to generate graphical representations of objects (e.g., buildings, roads, landmarks, etc.). For example, the navigation and mapping system 130 (e.g., the generating map application 132) may be configured to generate graphical representations of objects using one or more machine learning models (e.g., the generating machine learning models) in response to user input that results in generation of a customized digital map that includes graphical representations of objects (e.g., buildings, roads, landmarks, etc.). For example, the graphical representations may be generated based on images provided by a user or based on images from other external sources.

[0083] For example, the navigation and mapping system 130 (e.g., the map generating application 132) may be configured to convert user-generated media content into a universal form to anonymize the media content (e.g., by converting a real-world image of a person or animal located at a location into a two-dimensional or three-dimensional digital avatar representing the person or animal).

[0084] In some implementations, the navigation and mapping system 130 (e.g., the map generating application 132) may be configured to generate a customized digital map indicating the state of the location using sensor data obtained by one or more sensors (e.g., at the computing device 100 or elsewhere) in response to receiving input related to the location from a user. For example, the sensor data obtained by one or more sensors (e.g., at the computing device 100 or elsewhere) may indicate how many people are present at a location (e.g., based on the number of smart phones or other computing devices detected at the location). For example, the navigation and mapping system 130 (e.g., the map generating application 132) may generate a graphical representation of the location based on the number of people to accurately represent the location and depict the state of the location. For example, the navigation and mapping system 130 (e.g., the map generating application 132) may be configured to generate a graphical representation of the location using one or more machine learning models (e.g., the machine learning generating model) in response to user input that results in the generation of a customized digital map including a graphical representation indicating the state of the location. For example, the graphical representation of the location may include icons or graphical objects that are shaded in a manner that represents or indicates the status of the location (e.g., darker shades indicating busy or crowded locations and lighter shades indicating less busy or less crowded locations).

[0085] In some example embodiments, computing device 100 includes location determination device 140. Location determination device 140 may determine the current geographic location of computing device 100 and transmit such geographic location to server computing system 300 via network 400. Location determination device 140 may be any device or circuit system for analyzing the location of computing device 100. For example, location determination device 140 may determine the actual or relative location based on an IP address by using a satellite navigation positioning system (e.g., GPS system, Galileo positioning system, Global Navigation Satellite System (GLONASS), BeiDou satellite navigation positioning system), an inertial navigation system, a dead reckoning system, by using triangulation and / or proximity to a cellular tower or WiFi hotspot, and / or other suitable techniques for determining the location of computing device 100.

[0086] The computing device 100 may include an input device 150 configured to receive input from a user, and may include, for example, one or more of a keyboard (e.g., a physical keyboard, a virtual keyboard, etc.), a mouse, a joystick, a button, a switch, an electronic pen or stylus, a gesture recognition sensor (e.g., for recognizing a user's gestures, including movement of a body part), an input sound device or a voice recognition sensor (e.g., a microphone for receiving voice input such as a voice command or voice query), a trackball, a remote control, a portable (e.g., a cellular or smart) phone, a tablet PC, a pedal or foot switch, a virtual reality device, etc. The input device 150 may also include a tactile device for providing tactile feedback to the user. For example, the input device 150 may also be embodied by a touch-sensitive display with touch screen capabilities. For example, the input device 150 may be configured to receive input for customizing a digital map from a user associated with the input device 150.

[0087] Computing device 100 may include a display device 160 that displays information viewable by a user (e.g., a map, an immersive video of a location, a user interface screen, etc.). For example, display device 160 may be a non-touch-sensitive display or a touch-sensitive display. For example, display device 160 may include a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, an active matrix organic light emitting diode (AMOLED), a flexible display, a 3D display, a plasma display panel (PDP), a cathode ray tube (CRT) display, etc. However, the present disclosure is not limited to these example displays and may include other types of displays. Display device 160 may be used by a navigation and mapping system 130 provided at computing device 100 to display information related to input to a user (e.g., information related to a location of interest to the user, a user interface screen having user interface elements that can be selected by the user, etc.). The navigation information may include, but is not limited to, a digital map of the geographic area (e.g., an overhead view of the geographic area, a perspective view or street view of the geographic area, etc.), the location of the computing device 100 in the geographic area, a route through the geographic area specified on the map, one or more navigation directions (e.g., turn-by-turn directions through the geographic area), travel time for the route through the geographic area (e.g., from the location of the computing device 100 to the POI), and one or more of one or more points of interest in the geographic area.

[0088] The computing device 100 may include an output device 170 for providing output to a user, and may include, for example, one or more of an audio device (e.g., one or more speakers), a tactile device (e.g., a vibration device) for providing tactile feedback to the user, a light source (e.g., one or more light sources such as LEDs that provide visual feedback to the user), a thermal feedback system, etc. According to various examples of the present disclosure, the output device 170 may include a speaker that outputs a sound associated with a location in response to a user inputting a query or input related to the location, the query or input being provided as an input to a generative machine learning model that generates (renders) a customized digital map.

[0089] According to various examples of the present disclosure, the computing device 100 may include a capture device 180 capable of capturing media content. For example, the capture device 180 may include an image capturer 182 (e.g., a camera) configured to capture images (e.g., photos, videos, etc.) of a location. For example, the capture device 180 may include a sound capturer 184 (e.g., a microphone) configured to capture sounds or audio (e.g., audio recordings) of a location. The media content captured by the capture device 180 may be sent, for example, via the network 400 to one or more of the server computing system 300, the POI data repository 350, the navigation data repository 360, the user data repository 370, and the machine learning model data repository 380. For example, in some implementations, the image may be used to generate a customized digital map, and in some implementations, the media content may be provided as input to generate a machine learning model to generate a customized digital map related to a location, etc.

[0090] According to example embodiments described herein, server computing system 300 may include one or more processors 310 and one or more memory devices 320 as described herein. Server computing system 300 may also include a navigation and mapping system 330 similar to navigation and mapping system 130 described herein.

[0091] For example, the navigation and mapping system 330 may include a generated map application 332 that performs functions similar to those discussed above with respect to the generated map application 132. In some implementations, one or more machine learning models (e.g., generated machine learning models) associated with the navigation and mapping system 330 may be configured to generate content to be included in the customized digital map. For example, one or more machine learning models (e.g., generated machine learning models) associated with the navigation and mapping system 330 may be configured to render certain portions of the customized digital map that are sent to the computing device 100, where the computing device 100 is configured to render the remaining portions of the customized digital map via the one or more machine learning models (e.g., generated machine learning models) associated with the navigation and mapping system 130. For example, the size of the portion to be rendered by the navigation and mapping system 330 may vary depending on network conditions (e.g., available bandwidth, channel utilization conditions, latency conditions, throughput rates, etc.). In some implementations, one or more machine learning models associated with the navigation and mapping system 330 may be configured to process user input to generate information (e.g., semantic information), which may then be provided as input to one or more other machine learning models (e.g., generative machine learning models) associated with the navigation and mapping system 330 to generate content to be included in the customized digital map.

[0092] Examples of the present disclosure also relate to a computer-implemented method for generating a customized digital map in response to receiving a user query or input and using one or more generative machine learning models. Figure 2 A flowchart of an example non-limiting computer-implemented method is shown according to one or more example embodiments of the present disclosure. Figure 3 A block diagram of generating a map application according to one or more example embodiments of the present disclosure is shown.

[0093] Figure 2 The flowchart of shows a method 2000 for generating a customized digital map via generating a machine learning model in response to receiving an input. Although shown in a particular sequence or order, the order of these processes may be modified unless otherwise stated. Therefore, the embodiments shown should be understood as examples only, and the processes shown may be performed in a different order, and some processes may be performed in parallel. In addition, one or more processes may be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are also possible.

[0094] refer to Figure 2At operation 2100, method 2000 includes a computing device receiving input related to a location from a user. As described herein, the computing device may be embodied as computing device 100, server computing system 300, or a combination thereof. For example, the input may be provided by a user via input device 150. For example, the input may be in the form of a question, a command, or a description. For example, a user may provide a query related to the location to the computing device (e.g., input such as "Generate a map tile of Elliot Bay near downt own Seattle. Make the water emerald-colored with waves", or a request such as "show me the fastest route from Pike Place to the library with a yellow-brick road"). For example, the input may be provided or entered into navigation application 134, navigation application 334, map generation application 132, or map generation application 332.

[0095] In some implementations, the response to the input may be processed at the computing device 100 without involving the server computing system 300. In some implementations, the input may be sent from the computing device 100 to the server computing system 300, and at least a portion of the response to the input may be processed by the server computing system 300. For example, the input related to the location may be associated with various conditions (e.g., a particular time, lighting conditions, weather conditions, etc.). For example, the generate map application 132 may be configured to generate a customized digital map that reflects the current weather conditions of the location (e.g., fog coverage). For example, the generate map application 332 may be configured to generate a portion of a customized digital map.

[0096] At operation 2200, the computing device may be configured to generate adjustment parameters based at least in part on the input in response to receiving the input, wherein the adjustment parameters provide values ​​for one or more conditions associated with a customized digital map to be rendered related to the location. Figure 3 , the generated map application 3100 (which may correspond to the generated map application 132 and / or the generated map application 332) may include a tuning parameter generator 3110, one or more sequence processing models 3120, one or more large language models 3130, and one or more generated machine learning models 3140. The generated map application 3100 may receive input 3200 from a user, as described above with respect to Figure 2The adjustment parameter generator 3110 may be configured to generate adjustment parameters based at least in part on the input, wherein the adjustment parameters provide values ​​of one or more conditions associated with the customized digital map to be rendered related to the location.

[0097] In order to generate adjustment parameters, the adjustment parameter generator 3110 may be configured to retrieve the current value of one or more conditions at the location. In some implementations, the adjustment parameter generator 3110 may be configured to retrieve the current value of one or more conditions at the location based on sensor information 3300 that may correspond to data output by one or more sensors. One or more sensors may be provided at the computing device or provided externally (e.g., a sensor is provided at the location), and may provide information about the state of the location. For example, current values ​​associated with temperature data, lighting data, noise data, occupancy data, etc. may be retrieved by the adjustment parameter generator 3110 for various conditions at the location (e.g., temperature conditions, lighting conditions, noise conditions, occupancy conditions, etc.).

[0098] In some implementations, to generate adjustment parameters, the adjustment parameter generator 3110 may be configured to extract the value of one or more conditions from the input. The input may include information indicating the user's intention or requirement. In some implementations, the adjustment parameter generator 3110 (or one or more sequence processing models 3120 or one or more large language models 3130) may be configured to extract information from the input 3200 to identify the value of one or more conditions at the location, and the adjustment parameter generator 3110 may be configured to generate adjustment parameters based on the extracted values. For example, the input itself may identify a color (e.g., "emerald") or an attribute or feature (e.g., "brick road") that can be used to generate adjustment parameters.

[0099] In order to generate adjustment parameters, the adjustment parameter generator 3110 may be configured to infer the value of one or more conditions from the input. The input may include information indicating the user's intention or requirement. In some implementations, the adjustment parameter generator 3110 (or one or more sequence processing models 3120 or one or more large language models 3130) may be configured to infer information from the input 3200 to identify the value of one or more conditions at the location, and the adjustment parameter generator 3110 may be configured to generate adjustment parameters based on the inferred value. For example, the input may include a reference to "city center", and the adjustment parameter generator 3110 (or one or more sequence processing models 3120 or one or more large language models 3130) may be configured to infer a location in the city where the user is located that corresponds to the city center area of ​​the city.

[0100] In some implementations, the adjustment parameter generator 3110 may be configured to infer the value of one or more conditions from the input by providing the input to one or more sequence processing models 3120, wherein the one or more sequence processing models 3120 are configured to output the value of the one or more conditions in response to or based on the query. The one or more sequence processing models 3120 may include one or more machine learning models configured to process and analyze sequence data and process data that occurs in a specific order or sequence, including time series data, natural language text, or any other data with a time or sequence structure.

[0101] One or more sequence processing models 3120 may receive an input including text and tokenize the input by decomposing a sequence of text into small units (words) to provide a structured representation of the input sequence. One or more sequence processing models 3120 may represent word units as vectors in a continuous vector space by mapping each word unit to a high-dimensional vector, wherein the relationship between word units (words) is reflected in the geometric relationship between their corresponding vectors. For example, one or more sequence processing models 3120 may receive an input including the text "yellow brick road" and tokenize the input by decomposing a sequence of text into small units (word units) (e.g., "yellow", "brick" and "road"), thereby providing a structured representation of the input sequence. In word embedding, semantically similar words are closer in the vector space. For example, the vectors of "road" and "street" may be close to each other due to their semantic relationship, while the vectors of "brick" and "mud" may be farther apart than the vectors of "brick" and "asphalt".

[0102] For example, the input may include a request to provide a customized digital map relating to the status of various restaurants and depicting in a particular manner whether each restaurant is busy or not. One or more sequence processing models 3120 may be configured to tokenize and embed the input based on the query, based on semantic relationships with other vectors in the vector space, and based on other data represented as vectors in the vector space (e.g., input sequence data, which may include raw data relating to restaurants that may generally be considered busy) to infer that a restaurant that is at least 80% full compared to the restaurant's known capacity may be considered busy, or that a current wait time of more than 30 minutes is considered busy, or that no available reservations are considered busy, etc.

[0103] In order to generate adjustment parameters, the adjustment parameter generator 3110 may be configured to predict the future value of one or more conditions based on the current value of one or more conditions at the location and / or based on the historical value of one or more conditions at the location. The input may include information indicating the user's intention or requirement. In some implementations, the adjustment parameter generator 3110 (or one or more sequence processing models 3120 or one or more large language models 3130) may be configured to predict the future value of one or more conditions at the location, and the adjustment parameter generator 3110 may be configured to generate adjustment parameters based on the predicted future value. For example, the input may indicate that the user wants a representation of a certain location at a certain time or date in the future, and the adjustment parameter generator 3110 (or one or more sequence processing models 3120 or one or more large language models 3130) may be configured to predict the value of one or more conditions at the location based on the input. In some implementations, the adjustment parameter generator 3110 may be configured to retrieve the current value of one or more conditions at the location based on sensor information 3300 that may correspond to data output by one or more sensors or based on external content 3500 (e.g., information extracted from a website or other information source that may include map information). One or more sensors may be provided at the computing device or provided externally (e.g., the sensor is provided at an external computing device disposed at the location). For example, current values ​​associated with temperature data, lighting data, noise data, occupancy data, etc. may be retrieved by the adjustment parameter generator 3110 for various conditions at the location (e.g., temperature conditions, lighting conditions, noise conditions, occupancy data, etc.). In some implementations, the adjustment parameter generator 3110 may be configured to retrieve historical values ​​of one or more conditions at the location based on historical information 3400 that may be stored at various computing devices (e.g., one or more of the computing device 100, the external computing device 200, the server computing system 300, the external content 500, the POI data repository 350, the user data repository 370, etc.). For example, historical values ​​associated with temperature data, lighting data, noise data, occupancy data, etc. may be retrieved by the adjustment parameter generator 3110 for various conditions at the location (e.g., temperature conditions, lighting conditions, noise conditions, occupancy conditions, etc.).

[0104] For example, the map generation application 3100 (e.g., the adjustment parameter generator 3110) may be configured to implement one or more machine learning models to predict future values ​​of one or more conditions based on current values ​​and / or historical values ​​of one or more conditions. For example, the map generation application 3100 may be configured to use one or more prediction methods (e.g., linear regression, autoregressive integrated moving average model, exponential smoothing state space model) and / or neural networks (long short-term memory network, gated recurrent unit network, feedforward neural network, etc.) to predict the value of one or more conditions based on the current value and / or historical value of one or more conditions. Therefore, in order to generate a customized digital map showing specific conditions related to a location (e.g., the closing or opening status of a restaurant, the traffic conditions of a route, etc.) based on one or more predicted future values, the map generation application 3100 may be configured to generate a customized digital map based on the predicted future values ​​of one or more conditions (e.g., predicted values ​​of traffic conditions based on historical traffic information and current traffic information).

[0105] At operation 2300, the computing device may be configured to generate a customized digital map using one or more generative machine learning models, wherein the customized digital map depicts a location having one or more customized features generated via a generative machine learning model based on an input (e.g., a route displayed in a particular manner specified by a user, terrain displayed in a particular manner specified by a user, points of interest displayed in a particular manner specified by a user, etc.) and having values ​​for one or more conditions. For example, the generative map application 3100 may be configured to generate the customized digital map 3700 using one or more generative machine learning models 3140.

[0106] One or more generative machine learning models 3140 may include a deep neural network or a generative adversarial network (GAN), a variational autoencoder, a stable diffusion machine learning model, a visual transformer, a neural radiance field (NeRF), etc., to generate a customized digital map depicting a location with one or more customized features, wherein the one or more customized features have values ​​of conditions associated with the one or more customized features. For example, the computing device may include a database (e.g., a machine learning model data repository 380) configured to store multiple generative machine learning models respectively associated with multiple different locations. In some implementations, the computing device may be configured to retrieve a generative machine learning model associated with a specific location related to the input from the one or more generative machine learning models 3140.

[0107] In some implementations, one or more generative machine learning models 3140 may be trained on a large digital map dataset having corresponding information about conditions associated with each digital map. These conditions may include variables such as time of day, weather, lighting, object placement, occupancy, color, traffic, etc. During training, one or more generative machine learning models 3140 learn the relationship between visual elements in the digital map and the conditions that affect them. This may involve the computing device adjusting the internal parameters of each generative machine learning model to generate a realistic or accurate digital map based on the training data. The one or more generative machine learning models 3140 may be trained on one or more training datasets including multiple reference images of locations. The one or more training datasets may include values ​​of one or more conditions for at least some of the multiple reference images.

[0108] In some implementations, the one or more generated machine learning models 3140 are configured to generate a customized digital map 3700 based on the location as indicated in the input and the values ​​of one or more conditions associated with the features to be customized and rendered at the location. For example, if the query indicates a theme of a route (e.g., a nautical theme), the one or more generated machine learning models 3140 may be configured to generate a customized digital map 3700 by adjusting the one or more generated machine learning models 3140 with a tuning parameter. For example, the one or more generated machine learning models 3140 may be configured to take into account the tuning parameter (and the corresponding values ​​of the one or more conditions) to make a decision for rendering a customized digital map. For example, in some implementations, the one or more generated machine learning models 3140 may be configured to utilize an existing or pre-stored base digital map (default digital map) of the location (e.g., based on the map information 3600), and then render specific tiles of the existing or pre-stored base digital map related to the input so as to have a nautical theme of tiles associated with the route, while retaining tiles from the existing or pre-stored base digital map that are not associated or related to the route or the input. One or more generative machine learning models 3140 may be configured to output a customized digital map 3700 that depicts a location having one or more customized features having values ​​for conditions associated with the customized features that match the criteria provided in input 3200 and the tuning parameters generated by tuning parameter generator 3110. In some implementations, the computing device may be configured to implement post-processing operations to enhance the quality of the customized digital map (e.g., smoothing operations), add special effects, or fine-tune details.

[0109] At operation 2400, the computing device may be configured to provide a customized digital map 3700 that satisfies the input 3200. For example, the input may include a text query that specifies one or more objects to be included in the customized digital map, and the generated customized digital map 3700 may depict the one or more objects. For example, the customized digital map 3700 may be provided for presentation on the display device 160 of the computing device 100. In some implementations, the server computing system 300 may provide (send) the customized digital map 3700 or a portion of the customized digital map 3700 to the computing device 100, or the server computing system 300 may provide access to the customized digital map 3700 to the computing device 100. For example, the generated customized digital map 3700 may be stored at one or more computing devices (e.g., one or more of the computing device 100, the external computing device 200, the server computing system 300, the external content 500, the POI data repository 350, the user data repository 370, etc.).

[0110] In some implementations, after the customized digital map is generated, the user may provide feedback or further input to refine or change the customized digital map, and operations 2100 to 2400 may be repeated such that the computing device is configured to generate an adjusted (refined, modified, etc.) customized digital map using one or more generative machine learning models, e.g., in real time. The adjusted customized digital map depicts locations having one or more customized features as further specified by the user in further input.

[0111] In About Figure 2 and Figure 3 In the described example, the computing device may dynamically (e.g., in real time) generate a customized digital map 3700 related to the location in response to receiving input at operation 2100. However, in some implementations, the customized digital map that satisfies the request received at operation 2100 may be pre-stored or pre-existing and may be stored at one or more computing devices (e.g., one or more of the computing device 100, the external computing device 200, the server computing system 300, the external content 500, the POI data repository 350, the user data repository 370, etc.). Therefore, in this case, operations 2200 and 2300 may be omitted, and an operation of searching for a customized digital map that satisfies the input conditions may be performed as an intermediate operation between operations 2100 and 2400. Therefore, the computing device may respond to the request faster because fewer operations are performed or required.

[0112] Examples of the present disclosure also relate to user-facing aspects through which a user can request a customized digital map associated with a location. For example, Figure 4A and Figure 4BAn example of generating a customized digital map related to a location that can be presented on a display device associated with a user according to one or more example embodiments of the present disclosure is shown. For example, Figure 5A and Figure 5B Another example of generating another customized digital map related to a location that may be presented on a display device associated with a user according to one or more example embodiments of the present disclosure is shown. Figure 6 A block diagram including an example of a map data layer according to an example embodiment of the present disclosure is shown.

[0113] For example, Figure 4A A user interface screen of a mapping or navigation application according to one or more example embodiments of the present disclosure is shown. Figure 4A , user interface screen 4100 depicts a digital map of a location (geographic area) and includes various user interface elements for controlling or selecting options regarding the digital map. User interface screen 4100 includes a portion 4110 corresponding to a first map layer (e.g., a default map layer) that displays a map associated with the location, a first user interface element 4120 that provides input, and a plurality of user interface elements 4130 that, when selected, display corresponding features on the digital map (e.g., displaying restaurants on the map with corresponding icons when a user interface element corresponding to “restaurants” is selected, displaying hotels on the map with corresponding icons when a user interface element corresponding to “hotels” is selected, and so on). Other user interface elements (e.g., a zoom user interface element that zooms in and out of the map, an orientation user interface element that switches between different viewpoint orientations, and the like) may also be displayed (e.g., in an overlapping manner) on the digital map. For example, in Figure 4A , the digital map relates to the location of Seattle. First user interface element 4120 may be configured to enable a user to search for a particular location or point of interest, request directions between a starting point (origin) and a destination point, provide input requesting a customized digital map associated with a direction search or request, etc. For example, first user interface element 4120 may be in the form of a text box to enable a user to enter input (e.g., in text form). However, the user may provide input via other methods (e.g., via selection from a drop-down menu, via voice input through a microphone, etc.).

[0114] For example, a user may provide input via first user interface element 4120 that includes a text description related to a location, and one or more generative machine learning models 3140 may be configured to generate a detailed map of the location customized based on the user's input, such as by showing particular types of businesses or landmarks or by depicting businesses or landmarks in a particular manner.

[0115] In some implementations, the generate map application 3100 may be configured to receive a semantic description of a tile associated with a digital map to be generated. An example semantic description may include a prompt such as "Generate a map tile of Boston Long Wharf near the Aquarium. Make the water deep blue with waves." The semantic description may include various other constraints or conditions, such as “Make a coffee shop appear on the corner. The coffee shop's name is Tatte's Bakery.” The user may also provide further context or input to the generative machine learning model(s) 3140 to shape the output, such as by providing an image of a coffee shop's logo to render the sign and providing size information (e.g., “Make the sign somewhat large relative to the route. Render the start pin here and the destination pin here. Make the pins look nautical themed like a boat anchor.”).

[0116] According to embodiments described herein, the map displayed in portion 4110 may be generated in response to receiving input requesting a map of Seattle. Figure 4A The default map shown may be displayed using existing or default map data and may be rendered and provided to computing device 100 locally or at server computing system 300 .

[0117] Figure 4B Another user interface screen of a mapping or navigation application according to one or more example embodiments of the present disclosure is shown. Figure 4B , user interface screen 4200 depicts a digital map of a location (geographic area) and includes functions for controlling or selecting functions related to Figure 4A For example, the map generation application 3100 may be configured to generate a map based on the various user interface elements of the digital map described above. Figure 2 and Figure 3The described embodiments are used to generate Figure 4B Customized digital map shown.

[0118] As an example implementation, a user may provide input to user interface screen 4200 requesting that a map of nearby restaurants that serve sushi be displayed. The user may further specify that Elliot Bay be displayed to match the city's nickname, and that the map indicate in some manner whether the restaurant is open or closed. Figure 4B As shown, in response to user input, the map generation application 3100 may be configured to generate Figure 4B Customized digital maps shown. For example, the map generation application 3100 may be configured to display a water feature 4210 corresponding to Elliot Bay in emerald green according to the nickname of the city. For example, the map generation application 3100 may be configured to modify a default icon 4220 (e.g., a plate with accompanying cutlery) to indicate whether a sushi restaurant is open or closed by lighting the plate in bright yellow to indicate that the restaurant is open and graying the plate to indicate that the restaurant is closed. In some implementations, the map generation application 3100 may be configured to indicate how busy a restaurant is. For example, when a restaurant has a busy level exceeding a first threshold level, the map generation application 3100 may completely fill the plate with bright yellow, when the restaurant has a busy level exceeding a second threshold level but less than the first threshold level, the map generation application may fill half of the plate with bright yellow, and when the restaurant has a busy level less than the second threshold level, the map generation application may fill a quarter of the plate with bright yellow or less. For example, the map generation application 3100 may be configured to generate an icon to emphasize that the restaurant serves sushi. For example, the size of the icon may be too large compared to the default icon size. For example, the icon may include a graphical image 4230 of a sushi roll, such as a graphical image of a user's favorite type of sushi.

[0119] According to examples of the present disclosure, each user can personalize their map experience (e.g., according to the user's tastes or needs, such as the target institution, the category or type of routes divided by task, etc.). For example, if a user wants to go grocery shopping and view the grocery store, other content that may have appeared on the map may be visually confusing. Additional content also needs to be rendered (e.g., at the server computing system 300 or the user computing device 100). Therefore, the map generation application 3100 (e.g., one or more generation machine learning models 3140) may be configured to receive input from the user to omit irrelevant content from the map, minimize irrelevant content, display irrelevant content in less detail, etc., thereby saving computing resources. Therefore, the input indicates specific content to be omitted or reduced when generating a customized digital map. One or more generation machine learning models 3140 may be configured to generate a customized digital map describing a location with one or more customized features by omitting or reducing the specific content when generating a customized digital map.

[0120] As another example, if the user is interested in bicycle paths, one or more generative machine learning models 3140 may be configured to focus on bicycle paths when generating a customized digital map. For example, one or more generative machine learning models 3140 may be configured to customize map tiles in a manner specific to the user. For example, based on a user query or input or other conditions, one or more generative machine learning models 3140 may be configured to not render portions of the map that are not related to bicycling, such that the map only shows routes that can be used for bicycling, and may only show stores or facilities that are relevant to cyclists (e.g., showing bicycle stores without rendering fast food restaurants). Thus, computing resources may be efficiently utilized by rendering only portions of the map that are relevant to the user input.

[0121] In some implementations, one or more generated machine learning models 3140 may be configured to render locations or points of interest that may be of interest to the user. For example, one or more generated machine learning models 3140 may be configured to determine that a location or point of interest may be of interest to the user based on user input, based on contextual information associated with the user (e.g., user preferences, past purchases by the user, historical information associated with the user, interests based on the user's friends, etc.), based on external information (e.g., current events, local alerts, traffic alerts, etc.). One or more generated machine learning models 3140 may be configured to render the location or point of interest in a manner that prominently displays the location or point of interest relative to other locations or points of interest (e.g., in a visually unique manner, such as in an enlarged manner, using bright colors, or in a highlighted manner). In addition, the customized digital map may highlight the location or point of interest in a stylized or themed manner. For example, the icon or symbol representing the location or point of interest may match or correspond to the user input or query, or may match or correspond to a scene associated with the user (e.g., for a user who skateboards and often drinks coffee, the shape of the icon representing a coffee shop displayed on the map may resemble a skateboard).

[0122] In some implementations, the generated map application 3100 (e.g., one or more generated machine learning models 3140) may be configured to utilize default tiles or starting tiles as seed data by referencing the default tiles or starting tiles (e.g., based on the map information 3600) and input to render a customized digital map depicting a location with one or more customized features. For example, the computing device 100 may be configured to download or store a local template including the default tiles or starting tiles received from the server computing system 300. For example, the default tiles or starting tiles may be downloaded by the computing device 100 (before receiving input from the user) under specific conditions (e.g., when the available bandwidth is greater than a threshold, when the network traffic is less than a threshold, when the user computing device is connected to a WiFi network or a cellular network, etc.), in association with a specific event or state of the computing device (e.g., during initial map application or navigation application installation, during map application or navigation application update, etc.), or at a specific time (e.g., weekly, monthly, etc.).

[0123] In some implementations, computing device 100 and server computing system 300 may be configured to perform hybrid rendering, where computing device 100 renders a portion of the customized digital map and the remaining portion is rendered by server computing system 300. For example, one or more generative machine learning models (e.g., at computing device 100) may be configured to smooth and blend portions of the customized digital map rendered at server computing system 300 and portions of the customized digital map rendered at computing device 100 so that the entire map has a consistent and similar appearance. For example, one or more generative machine learning models 3140 may be provided at computing device 100 and configured to generate a customized digital map by rendering portions of the customized digital map that depict locations having one or more customized features (map tiles), and server computing system 300 may be configured to render portions of the customized digital map that do not include or belong to the customized features (other map tiles) (e.g., portions of the map that correspond to default map tiles).

[0124] For example, Figure 5A and Figure 5B Another example of generating a customized digital map related to a location that may be presented on a display device associated with a user according to one or more example embodiments of the present disclosure is shown.

[0125] For example, Figure 5A A user interface screen of a mapping or navigation application according to one or more example embodiments of the present disclosure is shown. Figure 5A, user interface screen 5100 depicts a digital map of a location (geographic area) and includes various user interface elements for controlling or selecting options regarding the digital map. For example, user interface screen 5100 includes a first portion 5110 corresponding to a first map layer (e.g., a default map layer corresponding to user interface element 5122) displaying a map associated with the location, and a second portion 5120 corresponding to a menu from which various map layers may be selected. For example, the menu may be displayed by selecting, for example, Figure 4A In some implementations, the menu includes multiple user interface elements, each of which corresponds to a different map layer (e.g., a first user interface element 5122 corresponds to a default map layer, a second user interface element 5124 corresponds to a bus layer that depicts various routes traveled by public transportation vehicles (such as bus routes, subway routes, etc.), and a third user interface element 5126 corresponds to a custom view layer, which, when selected, can cause a customized digital map to be displayed, the customized digital map including a custom map layer generated via one or more generative machine learning models). For example, Figure 5A As shown, other selectable map layers may include a satellite layer, a cycling layer, a terrain layer, a street view layer, a wildfire layer, and an air quality layer. The various map layers may be used as Figure 6 A portion of the map data layer 6000 is shown included and stored at the computing device 100 and / or the server computing system 300 .

[0126] In some implementations, the custom map layer may be pre-stored (e.g., stored at the computing device 100) based on a previously generated custom digital map. For example, the custom digital map may have been previously generated via one or more generative machine learning models based on the input, and may depict a location with one or more custom features having values ​​of one or more conditions associated with the custom features. In some implementations, the custom map layer may be generated (e.g., in real time) in response to a selection of the third user interface element 5126 by generating a custom digital map via one or more generative machine learning models based on the input, and may depict a location with one or more custom features having values ​​of one or more conditions associated with the custom features. In some implementations, the input may be actively provided by the user after selecting the third user interface element 5126. In some implementations, the input may be obtained or inferred based on various other inputs (e.g., by generating the map application 3100). For example, the input used to generate the custom map layer may include known user preferences, historical information about the user, contextual information associated with the user, contextual information associated with the location, and the like. The generate map application 3100 may be configured to generate a custom map layer suitable for the user based on one or more of these various other inputs not explicitly provided as input by the user.

[0127] Figure 5B A user interface screen 5200 of a mapping or navigation application according to one or more example embodiments of the present disclosure is depicted, the user interface screen depicting an example customized digital map related to a location (geographic area) that can be presented on a display device associated with a user. For example, the customized digital map can be presented for display to the user in response to selection of the third user interface element 5126. For example, the map generation application 3100 can be configured to generate a map layer 5220 that shows a route 5222 from a starting point 5224 to a destination point 5226, the route having one or more customized features personalized for a user associated with the computing device. Figure 5B As shown, the map generation application 3100 can be configured to customize the pins representing the starting point 5224 and the destination point 5226. For example, the pins can be generated based on user preferences (e.g., a coffee cup is used to represent the user's favorite morning beverage) or based on current events (e.g., using a coffee cup during National Coffee Week) or based on other factors. Figure 5BAs shown, the generated map application 3100 can be configured to customize other map features, such as a route 5222 between a starting point 5224 and a destination point 5226. For example, a route can be generated based on user preferences (e.g., a yellow brick road can be based on a user's favorite book) or based on contextual factors (e.g., a cobblestone road based on similar roads in a downtown area and colored based on the colors of a local sports team) or based on other factors. In some implementations, the intensity of the route color can be displayed (and dynamically changed) based on the amount of traffic along the route (e.g., darker intensities indicate high traffic and lighter intensities indicate low traffic). As shown in FIG. Figure 5B As shown, the map generation application 3100 may be configured to generate an object 5228 associated with the location. For example, the map generation application 3100 may be configured to generate the object 5228 based on the input. In some implementations, the input may include a text query that specifies one or more objects to be associated with the location. For example, the object may be in the form of a graphical object or an image or icon. Figure 5B As shown, object 5228 may be a killer whale, which may correspond to an animal that the user likes, may correspond to an event celebrating the killer whale migration season, etc. Thus, Figure 5B The customized digital map shown may reflect the user's preferences and be personalized to the user's tastes while reflecting an accurate representation of the route between navigation points using visually distinctive icons or graphical objects or images that the user can easily identify.

[0128] In some implementations, when information from a custom map layer is combined with information from another layer (e.g., Figure 6 When information from one or more of the layers shown in the digital map is conflicting or contradictory, the generated map application 3100 can be configured to prioritize the custom map layer over other map layers. For example, when multiple layers are called or stacked relative to the digital map being displayed, the custom map layer can be presented or overlaid on top of other information from other layers.

[0129] This disclosure is not limited to FIG. 4A to FIG. 5B , and other user interface elements may be provided by which a user may specify a request to obtain a customized digital map related to a location (or to directions or other navigation operations about a location) based on various conditions or inputs. For example, the user interface element may be in the form of a drop-down menu, a selectable user interface element, a text box, etc. Input may also be provided via voice prompts.

[0130] Figure 6A block diagram including an example map data layer according to an example embodiment of the present disclosure is depicted. One or more portions of the map data layer 6000 may be stored on one or more computing devices using any suitable memory. In some examples, the map data layer 6000 may be executed or implemented on one or more computing devices or computing systems including, for example, the computing device 100 and / or the server computing system 300.

[0131] like Figure 6 As shown, the map data layer 6000 includes multiple layers, examples of which may include a local coordinate system layer 6002, a location layer 6004, a plan layer 6006, a sign layer 6008, a traffic control device layer 6010, a landmark layer 6012, a hazard layer 6014, a default map layer 6016, a bus layer 6018, and a custom view layer 6020.

[0132] The multiple layers in the map data layer 6000 can be accessed by one or more computing systems and / or one or more computing devices including the computing device 100 and / or the server computing system 300. The generated map application 3100 (e.g., one or more generated machine learning models 3140) can be configured to reference information from the map data layer 6000 to generate a customized digital map in response to receiving input and / or in response to selection of the third user interface element 5126. As described herein, in some implementations, the generated map application 3100 (e.g., one or more generated machine learning models 3140) can be configured to prioritize data or information from the customized view layer 6020 over data or information from one of the other map data layers.

[0133] The local coordinate system layer 6002 may include data and / or information associated with a coordinate system (e.g., x, y, and z coordinates corresponding to latitude, longitude, and altitude, respectively), including the current coordinate system used by the computing device 100 and / or the server computing system 300.

[0134] The location layer 6004 may include data and / or information associated with a self-consistent description of the instantaneous location of the computing device 100 (e.g., the location of the computing device 100 at the current moment). The location may be associated with aspects including the physical location of the computing device 100 (e.g., pose, absolute velocity, and / or acceleration in a world or local coordinate system) or the semantic location of the computing device 100 (e.g., the road segment that the computing device 100 is currently on and the location of the computing device 100 on the road segment, the scalar velocity along the road segment, and / or the lane that the computing device 100 is in, etc.).

[0135] The planning layer 6006 can be used to annotate local road and lane maps based on the current plan or route. The planning layer 6006 can enable the computing device 100 to determine which roads or lanes ahead are to be followed by the current navigation route. The planning layer 6006 may include identification of planned roads and lanes ahead of the vehicle within the vehicle's field of view. The planning layer 6006 can be managed as part of the navigation interface. A route can be selected by searching for a destination, viewing possible routes, and initiating turn-by-turn navigation. Once the computing device 100 is in motion, the route can be automatically updated and recalculated as needed to adjust for driving behavior and road or traffic changes.

[0136] The sign layer 6008 may include data and / or information associated with physical signs visible from the road segment, including sign type, content, and three-dimensional positioning information. For example, a sign may include a speed limit sign observed while driving on a road, including the speed limit text on the sign. Signs may be classified into sign types that include a speed limit or a road name.

[0137] The traffic control device layer 6010 may be used to describe the location, properties, and status of traffic control devices (including stop lights and / or stop signs). The traffic control device layer may include an indication of the operational status of the traffic control device (e.g., operational, inoperative, or faulty).

[0138] The landmark layer 6012 can be used to describe landmarks visible from the road and can include a three-dimensional location of each landmark. In addition, the landmark layer 6012 can be used for positioning. The properties of the landmark layer 6012 include: positioning, which includes coordinates that can be used to determine the location of the landmark (e.g., x, y, z coordinates corresponding to latitude, longitude, and altitude); and visibility, which can be used to determine the distance at which the landmark is visible based on a vantage point (e.g., a vantage point of a vehicle at a particular location). The landmark layer 6012 can be updated in the event that a landmark is removed or relocated.

[0139] The hazard layer 6014 may include data and / or information associated with annotations of roads and lanes where traffic incidents, hazards, and / or obstructions occur. For example, hazards may include areas where construction is ongoing, natural disasters (e.g., floods, fires, heavy rain and / or snow), and other potentially harmful events that may impede or delay vehicle movement.

[0140] The default map layer 6016 may include data and / or information associated with a default map layer and may provide a digital map view in the form of a combination of light and earth-like tones to depict terrain and urbanized areas.

[0141] The transit map layer 6018 may include data and / or information indicating transit options including subways, subway routes, bus stops, and the like.

[0142] The custom view map layer 6020 may include data and / or information corresponding to a customized digital map generated via one or more generative machine learning models as described herein based on input from a user indicating various features to be customized (e.g., based on user preferences).

[0143] Various other map data layers can be provided as Figure 6 6000. For example, other map data layers may include a traffic layer, a satellite layer, an air quality layer, a bicycling layer, a terrain layer, a street view layer, a wildfire layer, a fog layer, a parking layer, a road layer, a lane layer, etc.

[0144] Fig. 7A A block diagram of an example computing system for generating a customized digital map via generating a machine learning model in response to receiving an input or query according to one or more example embodiments of the present disclosure is depicted. System 7100 includes a user computing device 7102, a server computing system 7130, and a training computing system 7150 communicatively coupled via a network 7180.

[0145] Figure 7B A block diagram of an example computing device for generating a customized digital map via generating a machine learning model in response to receiving an input or query is depicted in accordance with one or more example embodiments of the present disclosure.

[0146] Figure 7C A block diagram of an example computing device for generating a customized digital map via generating a machine learning model in response to receiving an input or query is depicted in accordance with one or more example embodiments of the present disclosure.

[0147] User computing device 7102 (which may correspond to computing device 100) may be any type of computing device, such as, for example, a personal computing device (e.g., a laptop or desktop computer), a mobile computing device (e.g., a smart phone or tablet), a gaming console or controller, a wearable computing device, an embedded computing device, or any other type of computing device.

[0148] The user computing device 7102 includes one or more processors 7112 and memory 7114. The one or more processors 7112 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.), and can be one processor or multiple processors operatively connected. The memory 7114 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc. and combinations thereof. The memory 7114 can store data 7116 and instructions 7118 executed by the processor 7112 to cause the user computing device 7102 to perform operations.

[0149] In some implementations, the user computing device 7102 may store or include one or more machine learning models 7120 (e.g., large language models, sequence processing models, generative machine learning models, etc.). For example, one or more machine learning models 7120 may be or may otherwise include various machine learning models, such as neural networks (e.g., deep neural networks) or other types of machine learning models, including nonlinear models and / or linear models. Example neural networks may include feedforward neural networks, recurrent neural networks (RNNs) (including recurrent neural networks based on long short-term memory (LSTM)), convolutional neural networks (CNNs), diffusion models, generative adversarial networks, or other forms of neural networks. The example neural network may be a deep neural network. Some example machine learning models may utilize attention mechanisms, such as self-attention. For example, some example machine learning models may include multi-head self-attention models (e.g., Transformer models). References herein Figures 1A to 6 Describes an example machine learning model.

[0150] In some implementations, one or more machine learning models 7120 may be received from a server computing system 7130 via a network 7180, stored in a memory 7114, and then used or otherwise implemented by one or more processors 7112. In some implementations, the user computing device 7102 may implement multiple parallel instances of a single machine learning model (e.g., to perform parallel tasks across multiple instances of a machine learning model). In some implementations, the task is a generation task, and one or more machine learning models may be implemented to output content (e.g., a customized digital map) in view of various inputs (e.g., queries, adjustment parameters, etc.). More specifically, the machine learning models disclosed herein (e.g., including large language models, sequence processing models, generation machine learning models, etc.) may be implemented to perform various tasks related to input queries.

[0151] According to an example of the present disclosure, a computing system may implement one or more sequence processing models 3120 as described herein to output the value of one or more conditions in response to or based on a query. One or more sequence processing models 3120 may include one or more machine learning models configured to process and analyze sequence data and process data that appears in a specific order or sequence, including time series data, natural language text, or any other data with a time or sequence structure.

[0152] According to an example of the present disclosure, a computing system may implement one or more large language models 3130 to determine multiple variables based on a query. For example, a large language model may include a bidirectional encoder representation from Transformers (BERT) large language model. For example, a large language model may be trained to understand and process natural language. A large language model may be configured to extract information from an input (query) to identify keywords, intents, and scenarios within the input to determine multiple variables for generating a customized digital map. Variables may include latent variables representing the underlying structure of a language.

[0153] According to an example of the present disclosure, a computing system may implement one or more generative machine learning models 3140 to generate a customized digital map depicting a location with one or more custom features, the one or more custom features having values ​​of one or more conditions associated with the custom features. One or more generative machine learning models 3140 may include a deep neural network or a generative adversarial network (GAN) to generate a customized digital map depicting a location with one or more custom features, the one or more custom features having values ​​of one or more conditions associated with the custom features. For example, one or more generative machine learning models 3140 may include a variational autoencoder, a stable diffusion machine learning model, a visual transformer, a neural radiant field (NeRF), etc., to generate a customized digital map depicting a location with one or more custom features, the one or more custom features having values ​​of conditions associated with the one or more custom features.

[0154] Additionally or alternatively, one or more machine learning models 7140 may be included in or otherwise stored and implemented by a server computing system 7130 that communicates with the user computing device 7102 according to a client-server relationship. For example, one or more machine learning models 7140 may be implemented by the server computing system 7130 as part of a web service (e.g., a navigation service, a mapping service, etc.). Thus, one or more machine learning models 7120 may be stored and implemented at the user computing device 7102, and / or one or more machine learning models 7140 may be stored and implemented at the server computing system 7130.

[0155] The user computing device 7102 may also include one or more user input components 7122 that receive user input. For example, the user input component 7122 may be a touch-sensitive component (e.g., a touch-sensitive display screen or a touchpad) that is sensitive to the touch of a user input object (e.g., a finger or a stylus). The touch-sensitive component may be used to implement a virtual keyboard. Other example user input components include a microphone, a traditional keyboard, or other devices and methods by which a user can provide user input.

[0156] The server computing system 7130 (which may correspond to the server computing system 300) includes one or more processors 7132 and memory 7134. The one or more processors 7132 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.), and may be one processor or a plurality of processors operatively connected. The memory 7134 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc., and combinations thereof. The memory 7134 may store data 7136 and instructions 7138 executed by the processor 7132 to cause the server computing system 7130 to perform operations.

[0157] In some implementations, the server computing system 7130 includes or is otherwise implemented by one or more server computing devices. Where the server computing system 7130 includes multiple server computing devices, such server computing devices may operate according to a sequential computing architecture, a parallel computing architecture, or some combination thereof.

[0158] As described above, the server computing system 7130 may store or otherwise include one or more machine learning models 7140. For example, the one or more machine learning models 7140 may be or may otherwise include various machine learning models. Example machine learning models include neural networks or other multi-layer nonlinear models. Example neural networks may include feedforward neural networks, recurrent neural networks (RNNs) (including recurrent neural networks based on long short-term memory (LSTM)), convolutional neural networks (CNNs), diffusion models, generative adversarial networks, or other forms of neural networks. Example neural networks may be deep neural networks. Some example machine learning models may utilize attention mechanisms, such as self-attention. For example, some example machine learning models may include multi-head self-attention models (e.g., Transformer models). References herein Figures 1A to 6 Describes an example machine learning model.

[0159] The user computing device 7102 and / or the server computing system 7130 may train one or more machine learning models 7120 and / or 7140 via interaction with a training computing system 7150 communicatively coupled via a network 7180. The training computing system 7150 may be separate from the server computing system 7130 or may be part of the server computing system 7130.

[0160] The training computing system 7150 includes one or more processors 7152 and memory 7154. The one or more processors 7152 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.), and can be one processor or multiple processors operatively connected. The memory 7154 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc. and combinations thereof. The memory 7154 can store data 7156 and instructions 7158 executed by the processor 7152 to cause the training computing system 7150 to perform operations. In some implementations, the training computing system 7150 includes one or more server computing devices or is otherwise implemented by one or more server computing devices.

[0161] The training computing system 7150 may include a model trainer 7160 that trains one or more machine learning models 7120 and / or 7140 stored at the user computing device 7102 and / or the server computing system 7130 using various training or learning techniques (e.g., such as error back propagation). For example, a loss function may be back propagated through the model to update one or more parameters of the model (e.g., based on the gradient of the loss function). Various loss functions may be used, such as mean squared error, likelihood loss, cross entropy loss, hinge loss, and / or various other loss functions. Gradient descent techniques may be used to iteratively update parameters over multiple training iterations.

[0162] In some implementations, performing error back-propagation may include performing truncated back-propagation through time. The model trainer 7160 may perform a variety of generalization techniques (eg, weight decay, backoff, etc.) to improve the generalization capabilities of the model being trained.

[0163] Specifically, the model trainer 7160 can train one or more machine learning models 7120 and / or 7140 based on a set of training data 7162. The training data 7162 may include, for example, various data sets that may be stored remotely or at the training computing system 7150. For example, in some implementations, example data sets for training include multiple digital maps related to a specific location, multiple images related to a specific location, and the like. However, other image and digital map data sets (e.g., images and digital maps from external websites) may be utilized. In some implementations, the data set may be limited to a specific category, type, terrain, landmark, time, and the like. In some implementations, the data set may contain different topics, including objects, terrain, individuals, crowds, landmarks, structures, and the like.

[0164] In some implementations, if the user has provided consent, the training examples may be provided by the user computing device 7102. Thus, in such implementations, one or more machine learning models 7120 provided to the user computing device 7102 may be trained by the training computing system 7150 on user-specific data received from the user computing device 7102. In some cases, this process may be referred to as personalizing the model.

[0165] The model trainer 7160 includes computer logic for providing the desired functionality. The model trainer 7160 can be implemented in hardware, firmware, and / or software that controls a general purpose processor. For example, in some implementations, the model trainer 7160 includes a program file stored on a storage device, loaded into a memory, and executed by one or more processors. In other implementations, the model trainer 7160 includes one or more sets of computer executable instructions stored in a tangible computer readable storage medium such as RAM, a hard disk, or an optical or magnetic medium.

[0166] The network 7180 can be any type of communication network, such as a local area network (e.g., an intranet), a wide area network (e.g., the Internet), or some combination thereof, and can include any number of wired or wireless links. In general, communications through the network 7180 can be conducted via any type of wired and / or wireless connection using a variety of communication protocols (e.g., TCP / IIP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or protection schemes (e.g., VPN, secure HTTP, SSL).

[0167] The machine learning models described in this specification can be used for a variety of tasks, applications, and / or use cases.

[0168] In some implementations, the input of the machine learning model of the present disclosure may be text or natural language data. The machine learning model may process the text or natural language data to generate an output. As an example, the machine learning model may process the natural language data to generate a language encoding output. As another example, the machine learning model may process the text or natural language data to generate a potential text embedding output. As another example, the machine learning model may process the text or natural language data to generate a translation output. As another example, the machine learning model may process the text or natural language data to generate a classification output. As another example, the machine learning model may process the text or natural language data to generate a text segmentation output. As another example, the machine learning model may process the text or natural language data to generate a semantic intent output. As another example, the machine learning model may process the text or natural language data to generate an amplified text or natural language output (e.g., text or natural language data with higher quality than the input text or natural language, etc.). As another example, the machine learning model may process the text or natural language data to generate a prediction output.

[0169] In some implementations, the input of the machine learning model of the present disclosure may be speech data. The machine learning model may process the speech data to generate an output. As an example, the machine learning model may process the speech data to generate a speech recognition output. As another example, the machine learning model may process the speech data to generate a speech translation output. As another example, the machine learning model may process the speech data to generate a potential embedding output. As another example, the machine learning model may process the speech data to generate an encoded speech output (e.g., an encoded and / or compressed representation of the speech data, etc.). As another example, the machine learning model may process the speech data to generate an amplified speech output (e.g., speech data of higher quality than the input speech data, etc.). As another example, the machine learning model may process the speech data to generate a text representation output (e.g., a text representation of the input speech data, etc.). As another example, the machine learning model may process the speech data to generate a predicted output.

[0170] In some implementations, the input of the machine learning model of the present disclosure may be sensor data. The machine learning model may process the sensor data to generate an output. As an example, the machine learning model may process the sensor data to generate a recognition output. As another example, the machine learning model may process the sensor data to generate a prediction output. As another example, the machine learning model may process the sensor data to generate a classification output. As another example, the machine learning model may process the sensor data to generate a segmentation output. As another example, the machine learning model may process the sensor data to generate a visualization output. As another example, the machine learning model may process the sensor data to generate a diagnostic output. As another example, the machine learning model may process the sensor data to generate a detection output.

[0171] Fig. 7A An example computing system is shown that can be used to implement various aspects of the present disclosure. Other computing systems may also be used. For example, in some implementations, the user computing device 7102 may include a model trainer 7160 and training data 7162. In such implementations, one or more machine learning models 7120 may be trained and used locally at the user computing device 7102. In some of such implementations, the user computing device 7102 may implement the model trainer 7160 to personalize one or more machine learning models 7120 based on user-specific data.

[0172] Figure 7B A block diagram of an example computing device for generating a customized digital map via generating a machine learning model in response to receiving a query according to one or more example embodiments of the present disclosure is depicted. The computing device 7200 may be a user computing device or a server computing device.

[0173] The computing device 7200 includes multiple applications (e.g., applications 1 to N). Each application contains its own machine learning library and machine learning model. For example, each application can include a machine learning model. Example applications include text messaging applications, email applications, dictation applications, virtual keyboard applications, browser applications, map applications, navigation applications, etc.

[0174] like Figure 7B As shown, each application can communicate with multiple other components of the computing device (such as, for example, one or more sensors, a context manager, a device state component, and / or additional components). In some implementations, each application can communicate with each device component using an API (e.g., a public API). In some implementations, the API used by each application is specific to the application.

[0175] Figure 7CA block diagram of an example computing device for generating a customized digital map via generating a machine learning model in response to receiving a query according to one or more example embodiments of the present disclosure is depicted. The computing device 70 may be a user computing device or a server computing device.

[0176] The computing device 7300 includes multiple applications (e.g., applications 1 to N). Each application communicates with a central intelligence layer. Example applications include text messaging applications, email applications, dictation applications, virtual keyboard applications, browser applications, map applications, navigation applications, etc. In some implementations, each application can use an API (e.g., a public API across all applications) to communicate with the central intelligence layer (and the models stored therein).

[0177] The central intelligence layer includes multiple machine learning models. For example, Figure 7C As shown, a corresponding machine learning model can be provided for each application, and the corresponding machine learning model can be managed by the central intelligence layer. In other implementations, two or more applications can share a single machine learning model. For example, in some implementations, the central intelligence layer can provide a single model for all applications. In some implementations, the central intelligence layer is included in the operating system of the computing device 7300 or is otherwise implemented by the operating system.

[0178] The central intelligence layer can communicate with the central device data layer. The central device data layer can be a centralized data repository for computing devices 7300. Figure 7C As shown, the central device data layer can communicate with multiple other components of the computing device (such as, for example, one or more sensors, a context manager, a device state component, and / or additional components). In some implementations, the central device data layer can communicate with each device component using an API (e.g., a private API).

[0179] To the extent that so-called general terms including "module" and "unit" are used herein, these terms may refer to, but are not limited to, software or hardware components or devices that perform certain tasks, such as field programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs). A module or unit may be configured to reside on an addressable storage medium and to be executed on one or more processors. Thus, by way of example, a module or unit may include components (such as software components, object-oriented software components, class components, and task components), processes, functions, attributes, procedures, subroutines, program code segments, drivers, firmware, microcodes, circuit systems, data, databases, data structures, tables, arrays, and variables. The functionality provided in components and modules / units may be combined into fewer components and modules / units or further divided into additional components and modules.

[0180] Aspects of the above-described example embodiments may be recorded in a non-transitory computer-readable medium including program instructions for implementing various operations embodied by a computer. The medium may also include data files, data structures, etc., alone or in combination with program instructions. Examples of non-transitory computer-readable media include magnetic media, such as hard disks, floppy disks, and tapes; optical media, such as CD ROMs, Blu-ray discs, and DVDs; magneto-optical media, such as optical discs; and other hardware devices specifically configured to store and execute program instructions, such as semiconductor memories, read-only memories (ROMs), random access memories (RAMs), flash memories, USB memories, etc. Examples of program instructions include machine codes (such as those generated by a compiler) and files containing higher-level codes that can be executed by a computer using an interpreter. Program instructions may be executed by one or more processors. The described hardware devices may be configured to act as one or more software modules in order to perform the operations of the above-described embodiments, and vice versa. In addition, non-transitory computer-readable storage media may be distributed between computer systems connected via a network, and computer-readable codes or program instructions may be stored and executed in a decentralized manner. Additionally, the non-transitory computer-readable storage medium may also be embodied in at least one application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0181] Each block of the flow chart may represent a unit, module, segment or portion of code, which includes one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions mentioned in the blocks may occur out of order. For example, two blocks shown in succession may actually be executed substantially concurrently (simultaneously), or the blocks may sometimes be executed in reverse order, depending on the functionality involved.

[0182] Although the present disclosure has been described with respect to various example embodiments, each example is provided by way of explanation, not limitation of the present disclosure. Those skilled in the art, after understanding the foregoing, may easily produce changes, modifications, and equivalents of such embodiments. Therefore, the present disclosure does not exclude such modifications, changes, and / or additions to the disclosed subject matter that would be readily understood by those of ordinary skill in the art. For example, a feature shown or described as part of one embodiment may be used together with another embodiment to produce yet another embodiment. Therefore, the present disclosure is intended to cover such changes, modifications, and equivalents.

Claims

1. A computing device for generating a customized digital map, comprising: Display device; one or more memories configured to store instructions; as well as One or more processors configured to execute the instructions to perform operations comprising: receiving input from a user related to customizing features associated with a location viewable on a digital map; In response to receiving the input, implementing a generative machine learning model to generate the customized digital map, the customized digital map depicting the location with one or more customized features generated via the generative machine learning model based on the input; and The customized digital map is provided for presentation via the display device.

2. A computing device as described in claim 1, wherein the generated machine learning model is provided at the computing device.

3. The computing device of claim 1 , wherein receiving the input from the user related to customizing features associated with the location viewable on the digital map comprises: A semantic description of a tile associated with the digital map to be generated is received.

4. The computing device of claim 2, wherein The generative machine learning model is configured to generate the customized digital map by rendering a portion of the customized digital map depicting the location having the one or more customized features, and The operations also include receiving a remaining portion of the customized digital map rendered by the server computing system.

5. The computing device of claim 4, wherein the operations further comprise implementing one or more machine learning models to smooth and blend the portion of the customized digital map rendered by the computing device and the remaining portion of the customized digital map rendered by the server computing system.

6. The computing device of claim 1, wherein The operations further include receiving one or more default tiles associated with the location prior to receiving the input from the user, and The generative machine learning model is configured to generate the customized digital map by referencing the one or more default tiles and the input to render the customized digital map depicting the location with the one or more customized features. 7 . The computing device of claim 6 , wherein the one or more default tiles associated with the location are received at a predetermined time according to a predetermined condition or according to a predetermined event.

8. The computing device of claim 7, wherein The predetermined time occurs periodically, The predetermined condition is associated with a network condition of a network through which the one or more default tiles are received from the server computing system, and The predetermined event is associated with a state of the computing device.

9. The computing device of claim 1, wherein the input indicates specific content to be omitted or reduced when generating the customized digital map, and Implementing the generating of the machine learning model to generate the customized digital map depicting the location having the one or more customized features comprises: The specific content is omitted or reduced when generating the customized digital map.

10. The computing device of claim 1, wherein Receiving the input from the user related to customizing features associated with the location viewable on the digital map includes: receiving a selection of a user interface element corresponding to a custom map layer, and In response to the selection of the user interface element, the generative machine learning model is implemented to generate the customized digital map, the customized digital map depicting the location with the one or more customized features generated via the generative machine learning model based on the input.

11. The computing device of claim 10, wherein the operations further comprise generating the custom map layer based on at least one of information associated with the user or contextual information associated with the location in response to the selection of the user interface element.

12. The computing device of claim 1, wherein The computing device includes one or more databases configured to store a plurality of generated machine learning models respectively associated with a plurality of different locations, and The operations also include retrieving the generated machine learning model associated with the location from the plurality of generated machine learning models.

13. The computing device of claim 1, wherein The input includes a text query specifying one or more objects to be associated with the location, and The digital map depiction includes the location of the one or more objects.

14. The computing device of claim 1, wherein The generative machine learning model has been fine-tuned based on a large-parameter generative machine learning model having a greater number of parameters than the generative machine learning model.

15. A computer-implemented method for generating a customized digital map, comprising: receiving, by a computing device, input from a user related to customizing features associated with a location viewable on a digital map; In response to receiving the input, implementing, by the computing device, a generative machine learning model to generate the customized digital map, the customized digital map depicting the location with one or more customized features generated via the generative machine learning model based on the input; as well as The customized digital map is provided by the computing device for presentation via a display device.

16. A computer-implemented method as described in claim 15, wherein the generated machine learning model is provided at the computing device.

17. The computer-implemented method of claim 15, wherein receiving the input from the user related to customizing features associated with the location viewable on the digital map comprises receiving a semantic description of a tile associated with the digital map to be generated.

18. The computer-implemented method of claim 15, wherein implementing said generating a machine learning model comprises: rendering a portion of the customized digital map depicting the location having the one or more customized features, and A remaining portion of the customized digital map rendered by the server computing system is received.

19. The computer-implemented method of claim 15, wherein the input indicates specific content to be omitted or reduced when generating the customized digital map, and Implementing the generation of the machine learning model includes omitting or reducing the specific content when generating the customized digital map.

20. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations for generating a customized digital map, the operations comprising: receiving input from a user related to customizing features associated with a location viewable on a digital map; In response to receiving the input, implementing a generative machine learning model to generate a customized digital map, the customized digital map depicting the location with one or more customized features generated via the generative machine learning model based on the input; as well as The customized digital map is provided for presentation via a display device.

Images