Generating customized digital map via generative machine-learned model

Abstract

To provide, generally, a customized digital map at a computing device via a generative machine-learned model.SOLUTION: A computing device for generating a customized digital map includes one or more memories to store instructions and one or more processors to execute the instructions to perform operations, the operations including: receiving an input from a user relating to customizing of features associated with a location viewable on a digital map; in response to receiving the input, implementing a generative machine-learned model to generate the customized digital map which depicts the location with one or more customized features which are generated via the generative machine-learned model based on the input; and providing the customized digital map for presentation via a display device.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present disclosure generally relates to providing a customized digital map on a computing device via a generative machine learning model. For example, the present disclosure relates to a method and a computing device for providing a customized digital map related to a location via a generative machine learning model provided on a computing device associated with a user in response to user input related to the location.

Background Art

[0002] According to current methods, a user's computing device downloads maps and satellite images / tiles from a server computing system, for example, for a navigation application. Further, there is an application programming interface (API) for the user to customize these maps, the server computing system renders custom tiles, and the user's computing device downloads the custom tiles from the server computing system.

Summary of the Invention

[0003] Aspects and advantages of embodiments of the present disclosure will be described in part in the following description, or will become apparent from the description, or may become apparent through practice of the exemplary embodiments.

[0004] In one or more exemplary embodiments, a computing device is provided for generating a customized digital map. For example, a computing device for generating a customized digital map includes a display device, one or more memories configured to store instructions, and one or more processors configured to execute the instructions to perform operations, the operations including receiving an input from a user related to customization of features associated with locations visible on the digital map, and in response to receiving the input, implementing a generative machine learning model to generate a customized digital map, wherein the customized digital map depicts locations with one or more customized features generated via the generative machine learning model based on the input, and providing the customized digital map for presentation via the display device, and one or more processors.

[0005] In some embodiments, the generative machine learning model is provided at the computing device.

[0006] In some embodiments, receiving an input from a user related to customization of features associated with locations visible on the digital map includes receiving a semantic description of tiles associated with the digital map to be generated.

[0007] In some embodiments, the generative machine learning model is configured to generate a customized digital map by rendering a portion of the customized digital map depicting locations with one or more customized features, and the operations further include receiving the remaining portion of the customized digital map rendered by a server computing system.

[0008] In some embodiments, the operation further includes implementing one or more machine learning models to smooth and blend a customized digital map portion rendered by a computing device and the remaining portion of the customized digital map rendered by a server computing system.

[0009] In some embodiments, the operation further includes receiving one or more default tiles associated with a location before receiving an input from a user, and a generative machine learning model is configured to generate a customized digital map by rendering a customized digital map that depicts the location with one or more customized features with reference to the one or more default tiles and the input.

[0010] In some embodiments, one or more default tiles associated with a location are received at a predetermined time according to a predetermined condition or in response to a predetermined event.

[0011] In some embodiments, the predetermined time occurs periodically, the predetermined condition is associated with network conditions of a network from which one or more default tiles are received by a server computing system, and the predetermined event is associated with the state of a computing device.

[0012] In some embodiments, the input indicates specific content to be omitted or reduced when generating a customized digital map, and implementing a generative machine learning model that generates a customized digital map indicating a location with one or more customized features when generating the customized digital map includes omitting or reducing the specific content.

[0013] In some embodiments, receiving input from a user regarding customizing features associated with locations visible on a digital map includes receiving a selection of a user interface element corresponding to a customized map layer, and in response to the selection of the user interface element, implementing a generative machine learning model to generate a customized digital map, wherein the customized digital map depicts locations with one or more customized features generated via the generative machine learning model based on the input, and generating.

[0014] In some embodiments, the operation further includes generating a customized map layer based on at least one of information related to the user or contextual information related to the location, in response to the selection of the user interface element.

[0015] In some embodiments, the computing device includes one or more databases configured to store a plurality of generative machine learning models respectively associated with a plurality of different locations, and the operation further includes obtaining a generative machine learning model associated with a location from among the plurality of generative machine learning models.

[0016] In some embodiments, the input includes a text query that specifies one or more objects associated with a location, and the digital map depicts a location that includes the one or more objects.

[0017] In some embodiments, the generative machine learning model is fine-tuned based on a large-scale parameter generative machine learning model that has more parameters than the generative machine learning model.

[0018] In one or more exemplary embodiments, a computer-implemented method for generating a customized digital map is provided. The computer-implemented method includes receiving, by a computing device, input from a user related to customization of features associated with locations visible on a digital map; and, in response to receiving the input, implementing, by the computing device, a generative machine learning model to generate a customized digital map, wherein the customized digital map depicts locations with one or more customized features generated via the generative machine learning model based on the input; and providing, by the computing device, the customized digital map for presentation via a display device.

[0019] In some embodiments, the generative machine learning model is provided on the computing device.

[0020] In some embodiments, receiving input from a user related to customizing features associated with locations visible on a digital map includes receiving a semantic description of tiles associated with the digital map to be generated.

[0021] In some embodiments, implementing the generative machine learning model includes rendering a portion of the customized digital map depicting locations with one or more customized features and receiving the remaining portion of the customized digital map rendered by a server computing system.

[0022] In some embodiments, the input indicates specific content to be omitted or reduced when the customized digital map is generated, and implementing the generative machine learning model when generating the customized digital map includes omitting or reducing the specific content.

[0023] In one or more exemplary embodiments, a computer-readable medium (e.g., a non-transitory computer-readable medium) storing instructions executable by one or more processors of a computing system is provided. In some embodiments, the computer-readable medium stores instructions that may cause one or more processors to perform one or more operations associated with any of the methods described herein (e.g., operations of a server computing system and / or operations of a computing device). For example, the operations may include receiving input from a user related to customizing features associated with locations visible on a digital map, and in response to receiving the input, implementing a generative machine learning model to generate a customized digital map, wherein the customized digital map depicts locations with one or more customized features generated via the generative machine learning model based on the input, and providing the customized digital map for presentation via a display device. The computer-readable medium may store additional instructions for implementing other aspects of server computing systems and computing devices, as well as corresponding methods of operation, as described herein.

[0024] These and other features, aspects, and advantages of the various embodiments of the present disclosure will become better understood with reference to the following description, drawings, and appended claims. The accompanying drawings, which are incorporated herein and constitute a part of this specification, illustrate exemplary embodiments of the present disclosure and, together with the description, serve to explain the relevant principles.

[0025] A detailed description of exemplary embodiments directed to those of ordinary skill in the art is set forth in this specification with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0026]

Figure 1A

Figure 1B

Figure 2

Figure 3

Figure 4A

Figure 4B

Figure 5A

Figure 5B

Figure 6

Figure 7A

Figure 7B

Figure 7C

Best Mode for Carrying Out the Invention

[0027] Reference is now made to embodiments of the present disclosure. One or more examples of the embodiments are shown in the drawings, and like reference numerals indicate like elements. Each example is provided as an illustration of the present disclosure and is not intended to limit the present disclosure. Indeed, it will be apparent to those skilled in the art that various modifications and variations can be made to the present disclosure without departing from the scope or spirit of the present disclosure. For example, features illustrated or described as part of one embodiment can be used with another embodiment to create yet another embodiment. Accordingly, it is intended that the present disclosure cover modifications and variations that come within the scope of the appended claims and their equivalents.

[0028] The terms used herein are for the purpose of describing exemplary embodiments and are not intended to limit and / or restrict the present disclosure. The singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. In the present disclosure, terms such as "including", "having", "comprising", etc. are used to specify features, numbers, steps, operations, elements, components, or combinations thereof, but do not exclude the presence or addition of one or more of features, elements, steps, operations, elements, components, or combinations thereof.

[0029] The terms first, second, third, etc. may be used herein to describe various elements, but it will be understood that the elements should not be limited by these terms. Instead, these terms are used only to distinguish one element from another. For example, without departing from the scope of the present disclosure, a first element may be referred to as a second element, and a second element may be referred to as a first element.

[0030] The term "and / or" includes combinations of multiple related listed items or any item of multiple related listed items. For example, the scope of the expression or phrase "A and / or B" includes the item "A", the item "B", and the combination of the item "A and B".

[0031] Furthermore, the scope of the expression or phrase "at least one of A or B" is intended to include all of (1) at least one of A, (2) at least one of B, and (3) at least one of A and at least one of B. Similarly, the scope of the expression or phrase "at least one of A, B, or C" is intended to include all of (1) at least one of A, (2) at least one of B, (3) at least one of C, (4) at least one of A and at least one of B, (5) at least one of A and at least one of C, (6) at least one of B and at least one of C, and (7) at least one of A, at least one of B, and at least one of C.

[0032] According to current methods, a significant amount of bandwidth and computing resources are used to render maps and satellite images / tiles and download them from a server computing system to a user's computing device. Further, there is an application programming interface (API) for users to customize these maps, the server computing system renders custom tiles, and the user's computing device downloads the custom tiles from the server computing system. This approach requires network resources (e.g., bandwidth) to exchange data between the user's computing device and the server computing system, and it can be problematic, especially in low-bandwidth or low-network-coverage areas, as each map may be rendered and downloaded multiple times to create different variations.

[0033] According to an exemplary embodiment of the present disclosure, user input that may include a semantic description of the map / tile to be rendered can be provided to the user's computing device, and then a generative machine learning model provided on the user's computing device can directly perform the rendering of a customized map (digital map) on the user's computing device, saving bandwidth and enabling the use of highly accurate maps in areas with low network coverage. By rendering the map on the device via the generative machine learning model, it is possible to implement customized or personalized content on the digital map, including forming more useful, task-specific map tiles for the user while reducing the amount of bandwidth used, and eliminating the conventional constraint that all users must use general-purpose map tiles for all tasks.

[0034] According to an exemplary embodiment of the present disclosure, a customized digital map can be generated in real time via a generative machine learning model on a device, thereby saving bandwidth, enabling personalization, and improving the customized experience.

[0035] In some embodiments, an existing generative machine learning model may be fine-tuned or distilled to improve its performance in specific tasks, including the rendering of map tiles. Thus, the generative machine learning model for customizing a digital map is faster, smaller (thereby saving storage space), and can be more easily deployed on a user's computing device compared to existing models.

[0036] For example, a computing device (e.g., a server computing system, a training computer system, etc.) may be configured to fine-tune an existing generative machine learning model on a dataset of map tiles to train the generative machine learning model to generate a realistic and accurate map from a text description. Also, the generative machine learning model can be distilled to remove unnecessary parameters and make the generative machine learning model faster and smaller. For example, the computing device may be configured to fine-tune an existing model to a task-specific one so that a large-scale parameter model (e.g., a 55 billion parameter large-scale model such as PaLI) can be distilled into a smaller parameter model (e.g., a model with 1 billion parameters or less).

[0037] After training, the generative machine learning model can be used to render maps in real time on a user's computing device with limited bandwidth. For example, a user can provide an input that includes a text description related to a location, and the generative machine learning model can generate a detailed map of that location customized according to the user's input, such as by indicating a particular type of business or landmark, or depicting the business or landmark in a particular manner.

[0038] Exemplary semantic descriptions can include prompts such as "Generate map tiles near the aquarium at Boston's Long Wharf. Make the water dark blue and wavy." The semantic description can include various other constraints or conditions, such as "Show a coffee shop at the corner. The name of the coffee shop is Tatte’s Bakery." The user can also provide additional context or input to shape the output, such as providing an image of the coffee shop's logo to the generative machine learning model to render a sign, and providing size information (e.g., "Make the sign larger than the route. Place the start point pin here and the destination pin here. Make the pins in the design of a nautical theme like an anchor of a ship.").

[0039] According to the disclosed embodiments, the usage of computing resources and network resources is also improved. For example, by delegating an embodiment of the generative machine learning model to the user's computing device, network traffic can be reduced, and the usage of server computing system (cloud) resources can be reduced.

[0040] In some embodiments, the generative machine learning model may include a generative adversarial network, a variational autoencoder, a stable diffusion machine learning model, a vision transformer, etc. Further, for dimensionality reduction, methods including singular value decomposition, LU decomposition, and principal component analysis may be performed by a computing device to obtain a generative machine learning model that can be implemented on the user's computing device.

[0041] In some embodiments, the user's computing device and the server computing system may be configured to perform hybrid rendering, in which the user's computing device renders a part of the map and the server computing system renders the remaining part. For example, a machine learning model (e.g., one on the user's computing device) can be configured to smooth and blend the part of the map rendered on the server computing system and the part of the map rendered on the user's computing device so that the entire map has a consistent and similar appearance.

[0042] In some embodiments, a default tile or starter tile may be utilized as seed data by a user's computing device, and the generative machine learning model may be configured to reference the default tile or starter tile when rendering a map. In some embodiments, the user's computing device may be configured to download or store a local template that includes the default tile or starter tile received from a server computing system. For example, the default tile or starter tile may be downloaded by the user's computing device under certain conditions (e.g., when the available bandwidth is greater than a threshold, when the network traffic is below a threshold, when the user's computing device is connected to a WiFi network or a cellular network, etc.), in relation to certain events (e.g., during the installation of an initial application, during an update of an application, etc.), or at certain times (e.g., weekly, monthly, etc.).

[0043] According to examples of the present disclosure, each user can personalize their map experience (e.g., according to the user's preferences or needs, such as the facility of interest, the category according to the task, or the type of route). For example, if a user wants to view a grocery store to go grocery shopping, if there is other content displayed on the map, it may become visually cluttered. The additional content also requires rendering (either on the server computing system 300 or the user's computing device 100). Thus, the user can provide inputs to the generative machine learning model to omit, minimize, display fewer details, etc. of the irrelevant content from the map, thereby saving computing resources.

[0044] As another example, if a user is interested in a bicycle route, the generative machine learning model may be configured to focus on the bicycle route when generating a customized digital map. According to existing methods, if a user wants to view a bicycle route on a map, the server computing system may render a map that includes the bicycle route at the relevant location, for example, by re-rendering map tiles or by utilizing a previously rendered map stored in memory (cache) that satisfies the user's query. However, according to an example of the present disclosure, the generative machine learning model may be configured to customize map tiles in a user-specific manner. For example, according to the user's query or input or other conditions, the generative machine learning model may be configured not to render portions of the map that are irrelevant to the bicycle route, such that the map can show only the routes available for the bicycle route and can show only stores or facilities relevant to bicycle users (e.g., display a cycle shop but not a fast food restaurant). Thus, computing resources can be efficiently utilized by rendering only the portions of the map relevant to the user input.

[0045] In some embodiments, the generative machine learning model may be configured to render locations or points of interest that are of interest to a user. For example, the generative machine learning model may be configured to determine, based on user input, based on context information related to the user (e.g., user preferences, past purchases of the user, historical information associated with the user, based on the interests of the user's friends, etc.), based on external information (e.g., current events, local alerts, traffic alerts, etc.), that a location or point of interest may be of interest to the user. The generative machine learning model may be configured to render the location or point of interest in a way that makes it stand out compared to other locations or points of interest (e.g., enlarged display, use of bright colors, or visually prominent methods such as highlighting). Further, the digital map may be able to display the location or point of interest in a stylized or themed way to make it stand out. For example, an icon or symbol representing the location or point of interest may match or correspond to the user input or query, or may match or correspond to the context related to the user (e.g., for a user who skateboards or habitually drinks coffee, the icon displayed on the map representing a coffee shop can be in the shape of a skateboard).

[0046] According to examples of the present disclosure, a generative machine learning model can be configured to render tiles on a user's computing device to meet the user's requirements for customizing a map and / or in response to a task being performed by the user (e.g., a request for route guidance to a point of interest). In contrast to current approaches that provide a general-purpose map shared by all users for all tasks, according to the examples disclosed herein, map customization is achieved in real time, improving the user experience and facilitating navigation (e.g., making it easier to add routes, change routes, identify relevant locations or points of interest in a user-specific manner, etc.).

[0047] One or more technical advantages of the present disclosure also include generating a customized digital map associated with a location via one or more machine learning models (e.g., a generative machine learning model) in response to receiving a query. For example, a generative machine learning model can be provided to a user's computing device to render a map that includes customized or personalized content on the user's computing device. Rendering a map on a user's computing device via a generative machine learning model leads to a reduction in the amount of bandwidth used compared to methods that require the user's computing device to download tiles from a server computing system, resulting in more useful task-specific map tiles that can assist the user and eliminating the constraint that all users must use general-purpose map tiles for all tasks as in the prior art.

[0048] Another technical advantage of the present disclosure includes providing a fine-tuned or distilled generative machine learning model for customizing digital maps. The fine-tuned or distilled generative machine learning model is faster, smaller in size (thereby saving storage space), and can be more easily deployed on a user's computing device compared to existing models.

[0049] Another technical advantage of the present disclosure includes increasing the uptime of a device by enabling the rendering of such maps even when map rendering is not possible due to low bandwidth or network availability limitations.

[0050] Another technical advantage of the present disclosure is that a user's computing device is configured to omit (or minimize or display less detail of) displaying irrelevant content on a digital map generated via a generative machine learning model, thereby saving computing resources (e.g., processing power) and leading to faster performance.

[0051] Another technical advantage of the present disclosure includes embodiments in which navigation can be performed more accurately or in an easier manner by displaying points of interest in a user-friendly or personalized manner such that a user can easily identify waypoints, routes, points of interest, etc. during route guidance.

[0052] Referring now to the drawings, FIG. 1A is a diagram of an exemplary system according to one or more exemplary embodiments of the present disclosure. FIG. 1A shows an example of a system 1000 that includes a computing device 100, an external computing device 200, a server computing system 300, and external content 500. These are communicating with each other via a network 400. For example, the computing device 100 and the external computing device 200 can include any of a personal computer, a smartphone, a tablet computer, a global positioning service device, a smartwatch, and the like. The network 400 can include any type of communication network including a wired or wireless network, or a combination thereof. The network 400 can include a local area network (LAN), a wireless local area network (WLAN), a wide area network (WAN), a personal area network (PAN), a virtual private network (VPN), and the like. For example, wireless communication between elements of the exemplary embodiments can be performed via a wireless LAN, Wi-Fi, Bluetooth, ZigBee, Wi-Fi Direct (WFD), ultra-wideband (UWB), Infrared Data Association (IrDA), Bluetooth Low Energy (BLE), Near Field Communication (NFC), radio frequency (RF) signals, and the like. For example, wired communication between elements of the exemplary embodiments can be performed via a pair cable, a coaxial cable, an optical fiber cable, an Ethernet cable, and the like. Communication via the network 400 can use a variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encoding or formatting (e.g., HTML, XML), and / or protection schemes (e.g., VPN, secure HTTP, SSL).

[0053] As will be described in more detail below, in some embodiments, the computing device 100 and / or the server computing system 300 may form part of a navigation and mapping system that can provide a location-customized digital map to a user of the computing device 100 via a generative machine learning model.

[0054] In some exemplary embodiments, the server computing system 300 may obtain data from one or more of a POI (Point-of-Interest) data store 350, a navigation data store 360, a user data store 370, and a machine learning model data store 380 to perform various operations and aspects of the navigation and mapping system disclosed herein. The POI data store 350, the navigation data store 360, the user data store 370, and the machine learning model data store 380 may be provided integrally with the server computing system 300 (e.g., as part of one or more memory devices 320 of the server computing system 300), or may be provided separately (e.g., remotely). Further, the POI data store 350, the navigation data store 360, the user data store 370, and the machine learning model data store 380 may be combined as a single data store (database), or may include a plurality of respective data stores. Data stored in one data store (e.g., the POI data store 350) may overlap with some data stored in another data store (e.g., the navigation data store 360). In some embodiments, one data store (e.g., the machine learning model data store 380) may reference data stored in another data store (e.g., the user data store 370).

[0055] The POI data store 350 can store information about places or points of interest, such as points of interest within an area or region related to one or more geographical areas. The point of interest can include any destination or location. For example, the point of interest can include restaurants, museums, sports venues, concert halls, amusement parks, schools, offices, grocery stores, gas stations, theaters, shopping malls, accommodation facilities, etc. The data of the points of interest stored in the POI data store 350 can include any information related to the POI. For example, the POI data store 350 can include the location information of the POI, the business hours of the POI, the phone number of the POI, reviews related to the POI, financial information related to the POI (e.g., the average cost of services and / or goods provided and / or sold at the POI, such as meals, tickets, rooms, etc.), environmental information related to the POI (e.g., noise level, atmosphere description, traffic level, etc., which can be provided in real time or be available by various sensors located at the POI), description of the types of services and / or goods provided, languages spoken at the POI, the URL of the POI, image content related to the POI, etc. For example, the information related to the POI may be retrievable from the external content 500 (e.g., from a web page related to the POI or from sensors placed at the POI).

[0056] The navigation data store 360 may store or provide map data / geospatial data used by the server computing system 300. Exemplary geospatial data includes geographic images (e.g., digital maps, satellite images, aerial photographs, street-level photographs, synthetic models, etc.), tables, vector data (e.g., vector representations of roads, plots, buildings, etc.), data of points of interest, or other suitable geospatial data associated with one or more geographic areas. In some examples, the map data can include a series of submaps, each submap including data of a geographic area that includes objects (e.g., buildings or other static features), routes of movement (e.g., roads, highways, public transportation lines, walking routes, etc.), and other features of interest. The navigation data store 360 is used by the server computing system 300 to provide route guidance, perform searches for points of interest, provide location or classification data of points of interest, determine distances, routes, or travel times between locations, or any other use or task necessary or useful for performing the operations of the exemplary embodiments disclosed herein.

[0057] In some examples, the user data store 370 can include the current user's location and heading data. In some examples, the user data store 370 can include information regarding one or more user profiles, including various user data such as user preference data, user demographic data, user calendar data, user social network data, user historical movement data, and the like. For example, the user data store 370 can include, without limitation, text content, images, email data including email-related calendar information or contact information, social media data including comments, reviews, check-ins, likes, invitations, contacts, or reservations, calendar application data including dates, times, events, descriptions, or other content, virtual wallet data including purchases, electronic tickets, coupons, or transactions, schedule data, location data, SMS data, or other suitable data related to the user account. According to one or more examples of the present disclosure, the data is analyzed to determine the user's preferences regarding POIs, and customized features are automatically proposed or provided, for example, with respect to representing a location or representing preferred attributes preferred by the user (e.g., displaying a POI using the user's favorite color, displaying a POI using a logo or symbol related to the user's favorite artist or favorite animal, etc.), and the generative machine learning model is implemented to generate a customized digital map including locations annotated or represented with customized features related to the user. For example, the data can be analyzed to determine the user's preferences regarding POIs, such as for determining the user's preferences for movement (e.g., means of movement, acceptable time for movement, etc.), for determining candidate recommendations of POIs for the user, for determining possible movement routes and means of movement for the user to a POI, and the like.

[0058] In some embodiments, the user data store 370 is provided to indicate potential data that can be analyzed by the server computing system 300 to identify user preferences, recommend POIs, determine possible travel routes to the POIs, determine the means of transportation used to travel to the POIs, determine the videos of locations to provide to the computing devices associated with the user, generate customized digital maps, etc. However, such user data may not be collected, used, or analyzed unless the user is notified in advance of what data will be collected and how they will be used and the user consents. Further, in some embodiments, the user may be provided with tools (e.g., within a navigation application or via a user account) to revoke or change the scope of permission. Additionally, certain information or data can be processed in one or more ways such that personally identifiable information is removed or stored in an encrypted manner before being stored or used. Thus, specific user information stored in the user data store 370 may or may not be accessible to the server computing system 300 based on the consent given by the user, or such data may not be stored in the user data store 370 at all.

[0059] The machine learning model data store 380 can store machine learning models that can be obtained and implemented by the server computing system 300 to generate distilled or fine-tuned machine learning models (e.g., distilled or fine-tuned generative machine learning models) that can be provided to the computing device 100. The machine learning model data store 380 can also store distilled or fine-tuned machine learning models (e.g., distilled or fine-tuned generative machine learning models) that can be obtained and implemented by the computing device 100. In some embodiments, the computing device 100 can obtain and implement a machine learning model that is a large-scale parameter model that has not been fine-tuned or distilled. The machine learning models stored in the machine learning model data store 380 (including large-scale parameter models and distilled or fine-tuned models) can include multiple generative machine learning models respectively associated with multiple different locations. In some embodiments, the machine learning model can include multiple generative machine learning models respectively associated with specific objects or structures provided at multiple different locations. The machine learning model can include a large-scale language model (e.g., a Bidirectional Encoder Representations from Transformers (BERT) large-scale language model). The machine learning model can include a generative artificial intelligence (AI) model (e.g., Bard) that can implement a Generative Adversarial Network (GAN), a Transformer, a Variational Autoencoder (VAE), a Neural Radiance Field (NeRF), etc. NeRF can be trained to learn a continuous volumetric scene function that can assign color and volume density to any voxel in space. The weights of the NeRF network can be optimized to encode the representation of the scene, whereby the model can render new views visible from any point in space.

[0060] External content 500 can be any form of external content, including news articles, web pages, video files, audio files, descriptive texts, evaluations, game content, social media content, photos, commercial offers, transportation means, weather conditions, sensor data obtained by various sensors, or other suitable external content. Computing device 100, external computing device 200, and server computing system 300 can access external content 500 via network 400. External content 500 can be searched by computing device 100, external computing device 200, and server computing system 300 according to known search methods, and the search results can be ranked according to other suitable attributes including relevance, popularity, or location-specific filtering or promotion.

[0061] Referring now to FIG. 1B, an exemplary block diagram of a computing device and a server computing system according to one or more exemplary embodiments of the present disclosure is described herein. Although computing device 100 is represented in FIG. 1B, the features of computing device 100 described herein are also applicable to external computing device 200.

[0062] Computing device 100 can include one or more processors 110, one or more memory devices 120, a navigation and mapping system 130, a position determination device 140, an input device 150, a display device 160, an output device 170, and a capture device 180. Server computing system 300 may include one or more processors 310, one or more memory devices 320, and a navigation and mapping system 330.

[0063] For example, one or more processors 110, 310 can be any suitable processing device that can be included in the computing device 100 or the server computing system 300. For example, one or more processors 110, 310 can include a processor, a processor core, a controller, and an arithmetic logic unit, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an image processor, a microcomputer, a field programmable array, a programmable logic unit, an application specific integrated circuit (ASIC), a microprocessor, a microcontroller, etc., and any other device capable of responding and executing instructions in a defined manner, and can include one or more of these combinations. One or more processors 110, 310 can be a single processor, or multiple processors operably connected, for example, in parallel.

[0064] One or more memory devices 120, 320 can include one or more non - transient computer - readable storage media including read - only memory (ROM), programmable read - only memory (PROM), erasable programmable read - only memory (EPROM), and flash memory, USB drives, volatile memory devices including random access memory (RAM), hard disks, floppy disks, Blu - ray disks, or optical media such as CD ROMs and DVDs, and combinations thereof. However, the examples of one or more memory devices 120, 320 are not limited to the above description, and one or more memory devices 120, 320 can be implemented by various other devices and structures as understood by those skilled in the art.

[0065] For example, when executed, one or more memory devices 120 cause one or more processors 110 to execute a generation map application 132, receive input from a user related to customization of features associated with locations visible on a digital map as described in examples of the present disclosure, and in response to receiving the input, implement a generative machine learning model to generate a customized digital map, wherein the customized digital map depicts locations with one or more customized features generated via the generative machine learning model based on the input, and provide the customized digital map for presentation via a display. The one or more memory devices 120 can store instructions for performing operations including causing the one or more processors 110 to execute the instructions for performing the operations described above.

[0066] One or more memory devices 320 can also include data 322 and instructions 324 obtained, operated on, created, or stored by one or more processors 310. In some exemplary embodiments, such data is accessed and used as input to implement a generation map application 332, and also receive input from a user related to customization of features associated with locations visible on a digital map as described in examples of the present disclosure, and in response to receiving the input, implement a generative machine learning model to generate a customized digital map, wherein the customized digital map depicts locations with one or more customized features generated via the generative machine learning model based on the input, and provide the customized digital map for presentation via a display. The one or more memory devices 320 can store instructions for performing operations including causing the one or more processors 310 to execute the instructions for performing the operations described above.

[0067] In some exemplary embodiments, computing device 100 includes a navigation and mapping system 130. For example, the navigation and mapping system 130 may include a map generation application 132 and a navigation application 134.

[0068] According to an example of the present disclosure, the map generation application 132 may be executed by the computing device 100 to provide a way for a user of the computing device 100 to customize one or more features or points of interest related to a digital map provided to the user via the display device 160. The map generation application 132 may be part of the navigation application 134 or a separate mapping application, or may be a stand-alone application. The map generation application 132 may be configured to be dynamically interactive according to various user inputs. For example, the map generation application 132 may be configured to modify or customize the digital map in real time in response to user input or queries by implementing a generative machine learning model (e.g., a voice input that requests that objects in the digital map be displayed in the user's favorite color or a specific theme). The map generation application 132 may be configured to dynamically generate (e.g., in real time) a customized digital map related to a location according to user input. Further aspects of the map generation application 132 are described herein.

[0069] In some examples, one or more aspects of the map generation application 132 can be implemented by the map generation application 332 of the server computing system 300 that can be remotely located to generate and / or provide a customized digital map in response to receiving an input from a user. In some examples, one or more aspects of the map generation application 332 can be implemented by the map generation application 132 of the computing device 100 to generate and / or provide a customized digital map in response to receiving an input from a user.

[0070] According to an example of the present disclosure, the navigation application 134 can be executed by the computing device 100 to provide a user of the computing device 100 with a way (route) to navigate to a location. The navigation application 134 can provide a navigation service to the user. In some examples, the navigation application 134 can facilitate the user's access to a server computing system 300 that provides a navigation service. In some exemplary embodiments, the navigation service includes providing route guidance to a specific location such as a POI. For example, a user can input a destination location (e.g., the address or name of a POI, or the category of a POI). In response, the navigation application 134 can use map data stored locally for a specific geographic area and / or map data provided via the server computing system 300 to provide navigation information for the user to move to the destination. For example, the navigation information can include turn-by-turn route guidance from the current location (or the provided starting point or departure location) to the destination. For example, the navigation information can include the travel time (e.g., estimated or predicted travel time) from the current location (or the provided starting point or departure location) to the destination.

[0071] The navigation application 134 can provide a visual depiction of a geographic area via the display device 160 of the computing device 100. The visual depiction of the geographic area can include one or more roads, one or more points of interest (including buildings, landmarks, etc.), and a highlighted display of a planned route. In some examples, the navigation application 134 can also provide a location-based search option that identifies one or more searchable points of interest within a given geographic area. In some examples, the navigation application 134 can include a local copy of the relevant map data. In other examples, the navigation application 134 can access information from the server computing system 300 that can be remotely located to provide the requested navigation services.

[0072] In some examples, the navigation application 134 can be a dedicated application specifically designed to provide navigation services. In other examples, the navigation application 134 can be a general-purpose application (e.g., a web browser) and can provide access to various different services including navigation services via the network 400.

[0073] For example, the navigation and mapping system 130 may store a customized digital map previously generated using one or more machine learning models (e.g., generative machine learning models) in one or more memory devices 120 (e.g., cache). The customized digital map may be categorized or classified according to the location and the context in which the customized digital map was generated (e.g., according to the type of POI, user input that caused the customized digital map to be generated, time of day, season, weather conditions, activities related to lighting conditions, etc.). For example, the navigation data store 360 may be configured to store a customized digital map generated using one or more machine learning models stored in the machine learning model data store 380. Exemplary customized digital maps may include a customized digital map associated with a specific bicycle route that a user uses for commuting, a customized digital map associated with the types of meals a user prefers that are offered at restaurants in the area where the user lives, a customized digital map associated with the user's favorite color where buildings related to user input are displayed on the map using the favorite color, and the like. In some embodiments, the computing device 100 may be configured to obtain a customized digital map from the navigation data store 360 (or local memory) when the customized digital map matches or corresponds to a query or input from a user requesting the customized digital map.

[0074] For example, a navigation and mapping system 130 (e.g., a generated map application) may be configured to generate a graphic representation of an object (e.g., a building, a road, a landmark, etc.). For example, the navigation and mapping system 130 (e.g., the generated map application 132) may be configured to use one or more machine learning models (e.g., a generated machine learning model, etc.) to generate a graphic representation of an object (e.g., a building, a road, a landmark, etc.) in response to user input that causes a customized digital map including the graphic representation of the object to be generated. For example, the graphic representation may be generated based on an image provided by the user or an image from another external source.

[0075] For example, the navigation and mapping system 130 (e.g., the generated map application 132) may be configured to convert media content generated by a user into a general format (e.g., by converting a real-world image of a person or an animal located at a certain place into a two-dimensional or three-dimensional digital avatar representing the person or the animal) to anonymize the media content.

[0076] In some embodiments, the navigation and mapping system 130 (e.g., the generated map application 132) may be configured to generate a customized digital map indicative of the state of a location in response to receiving input from a user related to that location, using sensor data obtained by one or more sensors (e.g., computing device 100 or installed at other locations). For example, the sensor data obtained by one or more sensors (e.g., at computing device 100 or elsewhere) may indicate how many people are present at a location (e.g., based on the number of smartphones or other computing devices detected at that location). For example, the navigation and mapping system 130 (e.g., the generated map application 132) may generate a graphical representation of the location according to the number of people, accurately represent the location, and depict the state of the location. For example, the navigation and mapping system 130 (e.g., the generated map application 132) may be configured to generate a graphical representation of the location using one or more machine learning models (e.g., a generated machine learning model, etc.) in response to user input that causes the generation of a customized digital map including a graphical representation indicative of the state of the location. For example, the graphical representation of the location may include icons or graphic objects shaded in a way that represents or indicates the state of the location (e.g., a dark shade indicates a congested or crowded location, and a light shade indicates a less congested or less populated location).

[0077] In some exemplary embodiments, computing device 100 includes a location determination device 140. The location determination device 140 can determine the current geographical location of the computing device 100 and transmit such geographical location to the server computing system 300 via network 400. The location determination device 140 can be any device or circuit for analyzing the location of the computing device 100. For example, the location determination device 140 can use a satellite navigation positioning system (e.g., GPS system, Galileo positioning system, Global Navigation Satellite System (GLONASS), BeiDou satellite navigation and positioning system), an inertial navigation system, a dead reckoning system, based on an IP address, triangulation and / or proximity to a cellular tower or WiFi hotspot, and / or other suitable techniques for determining the location of the computing device 100 to determine an actual or relative location.

[0078] The computing device 100 can include an input device 150 configured to receive input from a user, such as, for example, a keyboard (e.g., a physical keyboard, a virtual keyboard, etc.), a mouse, a joystick, a button, a switch, an electronic pen or stylus, a gesture recognition sensor (e.g., that recognizes a user's gesture including movement of a body part), an input sound device or a voice recognition sensor (e.g., a microphone that receives voice input such as voice commands or voice queries), a trackball, a remote controller, a portable phone (e.g., a mobile phone or a smartphone), a tablet PC, a pedal or a foot switch, a virtual reality device, or one or more of the like. The input device 150 can further include a haptic device that provides haptic feedback to the user. The input device 150 may also be embodied, for example, by a touch-sensitive display having a touch screen function. For example, the input device 150 may be configured to receive input from a user associated with the input device 150 to customize a digital map.

[0079] Computing device 100 may include a display device 160 that displays information visible to a user (e.g., a map, an immersive video of a location, a user interface screen, etc.). For example, display device 160 may be a non-touch sensitive display or a touch sensitive display. Display device 160 may include, for example, a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, an active matrix organic light emitting diode (AMOLED), a flexible display, a 3D display, a plasma display panel (PDP), a cathode ray tube (CRT) display, etc. However, the present disclosure is not limited to these exemplary displays and may include other types of displays. Display device 160 is used by a navigation and mapping system 130 provided in computing device 100 and can display information to the user in relation to an input (e.g., information related to a location of interest to the user, a user interface screen having interface elements selectable by the user, etc.). Navigation information may include, but is not limited to, a digital map of a geographic area (e.g., an aerial view of the geographic area, a perspective view of the geographic area, or a street view, etc.), the location of computing device 100 in the geographic area, a route through a geographic area specified on the map, one or more route guides (e.g., a turn-by-turn route guide through the geographic area), travel time for a route through the geographic area (e.g., from the location of computing device 100 to a POI), and one or more of one or more points of interest within the geographic area.

[0080] Computing device 100 may include an output device 170 for providing output to a user, for example, an audio device (e.g., one or more speakers), a haptic device (e.g., a vibration device) for providing haptic feedback to the user, a light source (e.g., one or more light sources such as an LED for providing visual feedback to the user), a thermal feedback system, or the like. According to various examples of the present disclosure, the output device 170 may include a speaker that outputs sound associated with a location in response to a user inputting a location-related query or input that is provided as an input to a generative machine learning model that generates (renders) a customized digital map.

[0081] According to various examples of the present disclosure, computing device 100 may include a capture device 180 capable of capturing media content. For example, capture device 180 may include an image capture device 182 (e.g., a camera) configured to capture an image (e.g., a photo, a video, etc.) of a location. For example, capture device 180 may include a sound capture device 184 (e.g., a microphone) configured to capture sound or audio (e.g., an audio recording) of a location. The media content captured by capture device 180 may be transmitted to one or more of server computing system 300, POI data store 350, navigation data store 360, user data store 370, and machine learning model data store 380, for example, via network 400. For example, in some embodiments, an image may be used to generate a customized digital map, and in some embodiments, media content may be provided as an input to a generative machine learning model to generate a customized digital map related to a location or the like.

[0082] According to the exemplary embodiments described herein, the server computing system 300 can include one or more processors 310 and one or more memory devices 320, as described herein. The server computing system 300 can also include a navigation and mapping system 330 similar to the navigation and mapping system 130 described herein.

[0083] For example, the navigation and mapping system 330 may include a generated map application 332, and this generated map application 332 performs functions similar to those described above with respect to the generated map application 132. In some embodiments, one or more machine learning models (e.g., a generative machine learning model) associated with the navigation and mapping system 330 may be configured to generate the content included in the customized digital map. For example, one or more machine learning models (e.g., a generative machine learning model) associated with the navigation and mapping system 330 may be configured to render a specific portion of the customized digital map that is transmitted to the computing device 100, and the computing device 100 is configured to render the remaining portion of the customized digital map via one or more machine learning models (e.g., a generative machine learning model) associated with the navigation and mapping system 130. For example, the size of the portion rendered by the navigation and mapping system 330 may vary according to the network status (e.g., available bandwidth, channel utilization, latency status, throughput rate, etc.). In some embodiments, one or more machine learning models associated with the navigation and mapping system 330 may be configured to process user input to generate information (e.g., semantic information), and this generated information is then provided as input to one or more other machine learning models (e.g., a generative machine learning model) associated with the navigation and mapping system 330 and may be used to generate the content included in the customized digital map.

[0084] Examples of the present disclosure also relate to a computer-implemented method for generating a customized digital map using one or more generative machine learning models in response to receiving a user query or input. FIG. 2 is a flowchart of an exemplary non-limiting computer-implemented method according to one or more exemplary embodiments of the present disclosure. FIG. 3 is a block diagram of a generation map application according to one or more exemplary embodiments of the present disclosure.

[0085] The flowchart of FIG. 2 shows a method 2000 for generating a customized digital map via a generative machine learning model in response to receiving an input. Although shown in a particular sequence or order, the order of the processes can be changed unless otherwise specified. Accordingly, the illustrated embodiments should be understood as examples only, and the processes shown can be performed in a different order and some processes can be performed in parallel. Further, in various embodiments, one or more processes can be omitted. Accordingly, not all processes are required in all embodiments. Other process flows are possible.

[0086] Referring to FIG. 2, method 2000 includes, at operation 2100, a computing device receiving input from a user related to a location. As described herein, the computing device may be embodied as computing device 100, server computing system 300, or a combination thereof. For example, the input may be provided by the user via input device 150. For example, the input may be in the form of a question, command, or description. For example, the user may provide a location-related query to the computing device (e.g., an input such as "Generate map tiles of Elliott Bay near downtown Seattle. Make the water emerald-colored and add waves." or a request such as "Show the shortest route from Pike Place to the library on a yellow brick road."). The input may be provided to, or entered into, for example, navigation application 134, navigation application 334, map generation application 132, or map generation application 332.

[0087] In some embodiments, the response to the input may be processed by computing device 100 without involving server computing system 300. In some embodiments, the input may be sent from computing device 100 to server computing system 300, and at least a portion of the response to the input may be processed by server computing system 300. For example, the location-related input may be associated with various conditions (e.g., a particular time, lighting conditions, weather conditions, etc.). For example, map generation application 132 may be configured to generate a customized digital map that reflects the current weather conditions of the location (e.g., apply a fog overlay). For example, map generation application 332 may be configured to generate a portion of a customized digital map.

[0088] In operation 2200, the computing device may be configured to generate adjustment parameters at least partially based on the input in response to receiving the input, and the adjustment parameters provide values for one or more conditions related to a customized digital map related to the location where rendering occurs. For example, referring to FIG. 3, the generation map application 3100 (which may correspond to the generation map application 132 and / or the generation map application 332) includes an adjustment parameter generator 3110, one or more sequence processing models 3120, one or more large language models 3130, and one or more generative machine learning models 3140. The generation map application 3100 may receive an input 3200 from the user as discussed above with respect to operation 2100 of FIG. 2. The adjustment parameter generator 3110 may be configured to generate adjustment parameters at least partially based on the input, and the adjustment parameters provide values for one or more conditions related to the rendering of a customized digital map related to the location.

[0089] To generate the adjustment parameters, the adjustment parameter generator 3110 may be configured to obtain current values for one or more conditions at the location. In some embodiments, the adjustment parameter generator 3110 may be configured to obtain current values for one or more conditions at the location based on sensor information 3300 that may correspond to data output by one or more sensors. The one or more sensors may be provided within the computing device or externally (e.g., sensors disposed at the location) and can provide information regarding the state of the location. For example, current values related to temperature data, lighting data, noise data, occupancy data, etc. may be obtained by the adjustment parameter generator 3110 with respect to various conditions at the location (e.g., temperature conditions, lighting conditions, noise conditions, occupancy conditions, etc.).

[0090] In some embodiments, to generate the adjustment parameters, the adjustment parameter generator 3110 may be configured to extract values for one or more conditions from the input. The input may include information indicating the user's intention or requirement. In some embodiments, the adjustment parameter generator 3110 (or one or more sequence processing models 3120, or one or more large language models 3130) may be configured to extract information from the input 3200 to identify values for one or more conditions at that location, and the adjustment parameter generator 3110 may be configured to generate the adjustment parameters based on the extracted values. For example, if the input itself includes a color (e.g., "emerald"), or an attribute or feature (e.g., "brick road"), they may be used to generate the adjustment parameters.

[0091] To generate the adjustment parameters, the adjustment parameter generator 3110 may be configured to estimate values for one or more conditions from the input. The input may include information indicating the user's intention or requirement. In some embodiments, the adjustment parameter generator 3110 (or one or more sequence processing models 3120 or one or more large language models 3130) may be configured to estimate information from the input 3200 to identify values for one or more conditions at that location, and the adjustment parameter generator 3110 may be configured to generate the adjustment parameters based on the estimated values. For example, the input may include a reference to "downtown", and the adjustment parameter generator 3110 (or one or more sequence processing models 3120 or one or more large language models 3130) may be configured to estimate the location corresponding to the downtown area within the city where the user is located.

[0092] In some embodiments, the adjustment parameter generator 3110 may be configured to estimate values for one or more conditions from an input by providing the input to one or more sequence processing models 3120, and the one or more sequence processing models 3120 may be configured to output values for one or more conditions in response to, or based on, a query. The one or more sequence processing models 3120 may include one or more machine learning models configured to process and analyze sequential data and handle data that occurs in a particular order or sequence, including time-sequential data, natural language text, or other data having a temporal or sequential structure.

[0093] The one or more sequence processing models 3120 can receive an input that includes text, tokenize the input by breaking the sequence of text into small units (tokens), and provide a structured representation of the input sequence. The one or more sequence processing models 3120 can represent tokens as vectors in a continuous vector space by mapping each token to a high-dimensional vector, where the relationship between tokens (words) is reflected in the geometric relationship between their corresponding vectors. For example, the one or more sequence processing models 3120 can receive an input that includes the text "yellow brick road", tokenize the input by breaking the sequence of text into small units (tokens) (e.g., "yellow", "brick", and "road"), thereby providing a structured representation of the input sequence. In word embedding, semantically similar words become closer in the vector space. For example, the vectors for "road" and "street" may be close to each other due to their semantic relationship, while the vectors for "brick" and "soil" may be farther apart compared to the vectors for "brick" and "asphalt".

[0094] For example, the input may include a requirement to provide a customized digital map related to the states of various restaurants and to depict in a specific way whether various restaurants are crowded. One or more sequence processing models 3120 may be configured to tokenize and embed the input, and based on a query, based on the semantic relationship with other vectors in the vector space, and based on other data represented as vectors in the vector space (e.g., input sequence data that may include raw data related to what can generally be considered a crowded restaurant), to infer a value of "crowded", and make inferences such as considering a restaurant that is at least 80% crowded compared to the known capacity of the restaurant as crowded, or considering a case where the current waiting time is 30 minutes or more as crowded, or considering a state where no reservations can be made as crowded.

[0095] To generate the adjustment parameters, the adjustment parameter generator 3110 may be configured to predict future values for one or more conditions at a location based on current values for one or more conditions at the location and / or based on historical values of one or more conditions. The input may include information indicating the user's intention or requirement. In some embodiments, the adjustment parameter generator 3110 (or one or more sequence processing models 3120 or one or more large language models 3130) may be configured to estimate future values for one or more conditions at the location, and the adjustment parameter generator 3110 may be configured to generate adjustment parameters based on the predicted future values. For example, the input may indicate that the user prefers a representation of the location at a future time or date, and the adjustment parameter generator 3110 (or one or more sequence processing models 3120 or one or more large language models 3130) may be configured to estimate values for one or more conditions at the location according to the input. In some embodiments, the adjustment parameter generator 3110 may be configured to obtain current values for one or more conditions at a location based on sensor information 3300 that may correspond to data output by one or more sensors, or based on external content 3500 (e.g., information extracted from a website or other information source that may include map information). The one or more sensors may be provided on a computing device or may be provided externally (e.g., sensors provided on an external computing device located at the location). For example, current values related to temperature data, lighting data, noise data, occupancy data, etc. may be obtained by the adjustment parameter generator 3110 and used for various conditions at the location (e.g., temperature conditions, lighting conditions, noise conditions, occupancy data, etc.).In some embodiments, the adjustment parameter generator 3110 may be configured to obtain historical values for one or more conditions at a location based on historical information 3400 that may be stored in various computing devices (e.g., one or more of computing device 100, external computing device 200, server computing system 300, external content 500, POI data store 350, user data store 370, etc.). For example, historical values related to temperature data, lighting data, noise data, occupancy data, etc. may be obtained by the adjustment parameter generator 3110 and used for various conditions (e.g., temperature conditions, lighting conditions, noise conditions, occupancy conditions, etc.) at that location.

[0096] For example, the generation map application 3100 (e.g., the adjustment parameter generator 3110) may be configured to implement one or more machine learning models to predict future values for one or more conditions based on current values and / or historical values for the one or more conditions. For example, the generation map application 3100 may utilize one or more prediction methods (e.g., linear regression, autoregressive integrated moving average model, exponential smoothing state space model) and / or neural networks (e.g., long short-term memory network, gated recurrent unit network, feedforward neural network, etc.) to predict the values of one or more conditions based on the current values and / or historical values of the one or more conditions. Thus, in order to generate a customized digital map indicating a specific state related to a location (e.g., the business state of a restaurant, the traffic situation of a route, etc.) according to one or more predicted future values, the generation map application 3100 may be configured to generate a customized digital map based on the predicted future values for one or more situations (e.g., predicted values for traffic situations based on past traffic information and current traffic information).

[0097] In operation 2300, the computing device may be configured to generate a customized digital map using one or more generative machine learning models, where the customized digital map depicts locations using one or more customized features (e.g., routes displayed in a particular manner specified by the user, terrain displayed in a particular manner specified by the user, points of interest displayed in a particular manner specified by the user, etc.) generated via a generative machine based on the input, and also using values of one or more conditions. For example, the generation map application 3100 may be configured to generate a customized digital map 3700 using one or more generative machine learning models 3140.

[0098] One or more generative machine learning models 3140 may include a deep neural network or a generative adversarial network (GAN), a variational autoencoder, a stable diffusion machine learning model, a visual transformer, a neural radiance field (NeRF), etc., to generate a customized digital map that depicts locations with one or more customized features having values for conditions related to the one or more customized features. For example, the computing device may include a database (e.g., the machine learning model data store 380) configured to store a plurality of generative machine learning models respectively associated with a plurality of different locations. In some embodiments, the computing device may be configured to obtain a generative machine learning model associated with a particular location related to the input from among one or more generative machine learning models 3140.

[0099] In some embodiments, one or more generative machine learning models 3140 may be trained on a large dataset of digital maps having corresponding information regarding conditions associated with each digital map. These conditions may include variables such as time, weather, lighting, object placement, occupancy status, color, traffic, and the like. During training, one or more generative machine learning models 3140 learn the relationships between visual elements within the digital map and the conditions that affect them. This may include a computing device adjusting the internal parameters of each generative machine learning model to generate a realistic or accurate digital map based on the training data. One or more generative machine learning models 3140 may be trained with one or more training datasets that include multiple reference images of a location. One or more training datasets may include values for one or more conditions for at least some of the multiple reference images.

[0100] In some embodiments, one or more generative machine learning models 3140 are configured to generate a customized digital map 3700 based on a location indicated in an input and values for one or more conditions associated with features that are customized and rendered at this location. For example, when a theme (e.g., a nautical theme) is specified for a route in a query, one or more generative machine learning models 3140 may be configured to generate a customized digital map 3700 by adjusting the one or more generative machine learning models 3140 with adjustment parameters. For example, one or more generative machine learning models 3140 may be configured to make a determination for rendering a customized digital map considering the adjustment parameters (and corresponding values for one or more conditions). For example, in some embodiments, one or more generative machine learning models 3140 may be configured to utilize an existing or pre-stored base digital map (default digital map), e.g., based on map information 3600, render specific tiles of the existing or pre-stored base digital map related to the input, apply a nautical theme to the tiles related to the route, while maintaining the existing or pre-stored base digital map for tiles not associated with or related to the route or input. One or more generative machine learning models 3140 may be configured to output a customized digital map 3700 depicting a location with one or more customized features having values for conditions related to the customized features that match the criteria provided in the input 3200 and the adjustment parameters generated by the adjustment parameter generator 3110. In some embodiments, the computing device may be configured to perform post-processing operations to improve the quality of the customized digital map (e.g., smoothing), add special effects, or fine-tune details.

[0101] In operation 2400, the computing device may be configured to provide a customized digital map 3700 that meets the input 3200. For example, the input may include a text query that specifies one or more objects included in the customized digital map, and the generated customized digital map 3700 may depict the one or more objects. For example, the customized digital map 3700 may be provided for presentation on the display device 160 of the computing device 100. In some embodiments, the server computing system 300 may provide (transmit) the customized digital map 3700 or a portion of the customized digital map 3700 to the computing device 100, or the server computing system 300 may provide access to the customized digital map 3700 to the computing device 100. For example, the generated customized digital map 3700 may be stored in one or more computing devices (e.g., one or more of the computing device 100, the external computing device 200, the server computing system 300, the external content 500, the POI data store 350, the user data store 370, etc.).

[0102] In some embodiments, after the customized digital map is generated, the user can provide feedback or further input to improve or modify the customized digital map, and operations 2100 to 2400 can be repeated so that the computing device can use one or more generative machine learning models to generate an adjusted (improved, modified, etc.) customized digital map, for example, in real time. This adjusted customized digital map depicts locations with one or more customized features further specified by the user with further input.

[0103] In the example described with respect to FIGS. 2 and 3, the computing device may dynamically (e.g., in real time) generate a customized digital map 3700 related to a location in response to receiving an input at operation 2100. However, in some embodiments, a customized digital map that satisfies the request received at operation 2100 may be pre-stored or may already exist and may be stored in one or more computing devices (e.g., one or more of computing device 100, external computing device 200, server computing system 300, external content 500, POI data store 350, user data store 370, etc.). Thus, in such cases, operations 2200 and 2300 may be omitted, but the operation of searching for a customized digital map that meets the conditions of the input may be performed as an intermediate operation between operations 2100 and 2400. Thus, the responsiveness of the computing device to the request can be faster the fewer operations that are performed or required.

[0104] Examples of the present disclosure also contemplate user-facing aspects in which a user can request a customized digital map related to a location. For example, FIGS. 4A and 4B show an example of generating a customized digital map related to a location that can be presented on a display device associated with a user, according to one or more exemplary embodiments of the present disclosure. For example, FIGS. 5A and 5B show another example of generating another customized digital map related to a location that can be presented on a display device associated with a user, according to one or more exemplary embodiments of the present disclosure. FIG. 6 is a block diagram including an example of a map data layer, according to an exemplary embodiment of the present disclosure.

[0105] For example, FIG. 4A is a diagram showing a user interface screen of a mapping or navigation application according to one or more exemplary embodiments of the present disclosure. In FIG. 4A, the user interface screen 4100 depicts a digital map of a location (geographical area) and includes various user interface elements for controlling or selecting options related to the digital map. For example, the user interface screen 4100 includes a portion 4110 corresponding to a first map layer (e.g., a default map layer) that displays a map associated with the location, a first user interface element 4120 for providing input, and a plurality of user interface elements 4130 that, when selected, display corresponding features on the digital map (e.g., when a user interface element corresponding to "restaurant" is selected, a restaurant is displayed on the map with a corresponding icon, and when a user interface element corresponding to "hotel" is selected, a hotel is displayed on the map with a corresponding icon, etc.). Other user interface elements may also be displayed on the digital map (e.g., in an overlay manner) (e.g., a zoom user interface element for zooming in and out of the map, an orientation user interface element for switching to different viewpoints, etc.). For example, in FIG. 4A, the digital map is related to the location of Seattle. The first user interface element 4120 may be configured to enable the user to search for a specific location or point of interest, request route guidance between a starting point and a destination, and provide input for requesting a customized digital map related to the search or route request, etc. For example, the first user interface element 4120 may be in the form of a text box that enables the user to enter input (e.g., in text form). However, the user may provide input via other means (e.g., via selection from a pull-down menu, via voice input through a microphone, etc.).

[0106] For example, a user can provide an input including a text description related to a location via a first user interface element 4120, and one or more generative machine learning models 3140 can be configured to generate a detailed map of the location customized according to the user's input, such as by indicating a particular type of business or landmark, or depicting the business or landmark in a particular manner.

[0107] In some embodiments, the generative map application 3100 may be configured to receive semantic descriptions of tiles associated with the generated digital map. Exemplary semantic descriptions may include prompts such as "Generate map tiles near the aquarium at the Long Wharf in Boston. Make the water dark blue and add waves." The semantic description can include various other constraints or conditions, such as "Display a coffee shop at the corner. The name of the coffee shop is Tatte’s Bakery." The user can also shape the output by providing additional context or input to one or more generative machine learning models 3140, such as providing an image of the coffee shop's logo to render a sign, or providing size information (e.g., "Make the sign larger than the route. Place the start point pin here and the destination pin here. Make the pins in the design of a nautical theme like a ship's anchor.").

[0108] According to the embodiments described herein, the map displayed in portion 4110 can be generated in response to receiving an input requesting a map of Seattle. The default map shown in FIG. 4A may be displayed using existing or default map data, for example, and may be rendered locally on the computing device 100 or rendered on the server computing system 300 and provided to the computing device 100.

[0109] FIG. 4B is a diagram showing another user interface screen of a mapping or navigation application according to one or more exemplary embodiments of the present disclosure. For example, in FIG. 4B, the user interface screen 4200 represents a digital map of a location (geographical area) and includes various user interface elements for controlling or selecting options regarding a digital map similar to that described with respect to FIG. 4A. For example, the generating map application 3100 may be configured to generate a customized digital map as shown in FIG. 4B according to the embodiments described with respect to FIGS. 2 and 3.

[0110] As an exemplary embodiment, the user may input to the user interface screen 4200 and request a map that displays nearby restaurants offering sushi. The user may further specify to display Elliott Bay according to the city's nickname and to indicate in some way whether the restaurant on the map is open or closed. As shown in FIG. 4B, in response to the user's input, the generated map application 3100 may be configured to generate the customized digital map shown in FIG. 4B. For example, the generated map application 3100 may be configured to display the water feature 4210 corresponding to Elliott Bay in the color of emerald green according to the city's nickname. For example, the generated map application 3100 may be configured to change the default icon 4220 (e.g., a plate and cutlery) to indicate whether the sushi restaurant is open or closed, and can indicate that the restaurant is open by making the plate glow bright yellow, and indicate that it is closed by making the plate gray. In some embodiments, the generated map application 3100 may be configured to indicate how crowded the restaurant is. For example, when the congestion level of the restaurant exceeds the first threshold level, the generated map application 3100 may completely fill the plate with bright yellow. When the congestion level of the restaurant exceeds the second threshold level but is less than the first threshold level, half of the plate may be filled with bright yellow. When the congestion level of the restaurant is less than the second threshold level, less than one-fourth of the plate may be filled with bright yellow. For example, the generated map application 3100 may be configured to generate an icon that emphasizes that the restaurant offers sushi. For example, the icon may be displayed larger than the default icon size. For example, the icon may include a graphic image 4230 of a sushi roll, which is, for example, the user's favorite type of sushi.

[0111] According to an example of the present disclosure, each user can personalize their map experience (e.g., according to the user's preferences or needs, such as the facility of interest, the category according to the task, or the type of route). For example, if a user wants to view a grocery store to go grocery shopping, if there is other content displayed on the map, it may become visually cluttered. The additional content also requires rendering (e.g., on either the server computing system 300 or the computing device 100). Thus, the generative map application 3100 (e.g., one or more generative machine learning models 3140) can be configured to receive input from the user to omit, minimize, display with less detail, etc., unrelated content from the map, thereby saving computing resources. For this reason, the input indicates specific content to be omitted or reduced when generating the customized digital map. One or more generative machine learning models 3140 can be configured to generate a customized digital map that depicts locations with one or more customized features by omitting or reducing specific content when generating the customized digital map.

[0112] As another example, if a user is interested in a bicycle route, one or more generative machine learning models 3140 may be configured to focus on the bicycle route when generating a customized digital map. For example, one or more generative machine learning models 3140 may be configured to customize map tiles in a user-specific manner. For example, according to a user's query or input or other conditions, one or more generative machine learning models 3140 may be configured not to render portions of the map that are irrelevant to the bicycle route, such that the map can show only routes that are bicycle-accessible and can show only stores or facilities relevant to bicycle users (e.g., display a cycle shop but not a fast-food restaurant). Thus, computing resources can be efficiently utilized by rendering only portions of the map relevant to the user input.

[0113] In some embodiments, one or more generative machine learning models 3140 may be configured to render locations or points of interest that are of interest to a user. For example, one or more generative machine learning models 3140 may, based on user input, based on context information related to the user (e.g., user preferences, the user's past purchases, historical information associated with the user, based on the interests of the user's friends, etc.), based on external information (e.g., current events, local alerts, traffic alerts, etc.), be configured to determine that a location or point of interest is of interest to the user. One or more generative machine learning models 3140 may be configured to render that location or point of interest in a way that makes it stand out by comparing it to other locations or points of interest (e.g., visually prominent ways such as zoomed-in display, use of bright colors, or highlighted display). Further, a customized digital map can display that location or point of interest in a stylized or themed way to make it stand out. For example, an icon or symbol representing a location or point of interest can match or correspond to a user input or query, or can match or correspond to the context related to the user (e.g., for a user who skateboards or habitually drinks coffee, the icon displayed on the map representing a coffee shop can be in the shape of a skateboard).

[0114] In some embodiments, the generation map application 3100 (e.g., one or more generative machine learning models 3140) may be configured to utilize a default tile or starter tile as seed data (e.g., based on map information 3600) and render a customized digital map that depicts a location with one or more customized features using the default tile or starter tile and the input. For example, the computing device 100 may be configured to download or store a local template that includes a default tile or starter tile received from the server computing system 300. For example, the default tile or starter tile may be downloaded by the computing device 100 (before receiving input from the user) in relation to a particular event or state of the computing device (e.g., during installation of an initial map application or navigation application, during an update of a map application or navigation application, etc.), under particular conditions (e.g., when the available bandwidth is greater than a threshold, when the network traffic is below a threshold, when the user's computing device is connected to a WiFi network or a cellular network, etc.), or at a particular time (e.g., weekly, monthly, etc.).

[0115] In some embodiments, computing device 100 and server computing system 300 may be configured to perform hybrid rendering, in which computing device 100 renders a portion of a customized digital map and the remaining portion is rendered by server computing system 300. For example, one or more generative machine learning models (e.g., on computing device 100) may be configured to smooth and blend a portion of the customized digital map rendered in server computing system 300 and a portion of the customized digital map rendered in computing device 100 such that the entire map has a consistent and similar appearance. For example, one or more generative machine learning models 3140 may be provided on computing device 100 and may be configured to generate a customized digital map depicting locations with one or more customized features by rendering a portion (map tile) of the customized digital map, and server computing system 300 may be configured to render portions (other map tiles) of the customized digital map that do not include or are not related to the customized features (e.g., portions of the map corresponding to default map tiles).

[0116] For example, FIGS. 5A and 5B show another example of generating a customized digital map related to a location that can be presented on a display device associated with a user, according to one or more exemplary embodiments of the present disclosure.

[0117] For example, FIG. 5A is a diagram showing a user interface screen of a mapping or navigation application according to one or more exemplary embodiments of the present disclosure. In FIG. 5A, the user interface screen 5100 represents a digital map of a location (geographical area) and includes various user interface elements for controlling or selecting options regarding the digital map. For example, the user interface screen 5100 includes a first portion 5110 corresponding to a first map layer (e.g., a default map layer corresponding to the user interface element 5122) that displays a map associated with the location, and a second portion 5120 corresponding to a menu by which various map layers can be selected. For example, the menu can be opened by selecting the user interface element 4140 as shown in FIG. 4A. In some embodiments, the menu includes a plurality of user interface elements, and each of the plurality of user interface elements corresponds to a different map layer (e.g., the first user interface element 5122 corresponds to the default map layer, the second user interface element 5124 corresponds to a traffic layer depicting various routes traveled by transportation vehicles such as bus routes and subway routes, the third user interface element 5126 corresponds to a custom view layer, and when this custom view layer is selected, it can display a customized digital map including a customized map layer generated via one or more generative machine learning models). For example, as shown in FIG. 5A, other selectable map layers can include a satellite layer, a bicycle route layer, a terrain layer, a street view layer, a forest fire layer, and an air quality layer. The various map layers are included as part of the map data layer 6000 as shown in FIG. 6 and can be stored in the computing device 100 and / or the server computing system 300.

[0118] In some embodiments, the customized map layer can be pre-stored (e.g., in computing device 100) based on a previously generated customized digital map. For example, the customized digital map may have been previously generated via one or more generative machine learning models based on an input, and can depict locations with one or more customized features having values for one or more conditions related to the customized features. In some embodiments, the customized map layer can be generated (e.g., in real time) in response to a selection of a third user interface element 5126 by generating a customized digital map via one or more generative machine learning models based on an input, and can depict locations with one or more customized features having values for one or more conditions related to the customized features. In some embodiments, the input can be actively provided by the user after selection of the third user interface element 5126. In some embodiments, the input can be obtained or inferred based on various other inputs (e.g., by the generative map application 3100). For example, the input for generating the customized map layer can include the user's known preferences, historical information related to the user, context information related to the user, context information related to the location, and the like. The generative map application 3100 can be configured to generate a customized map layer suitable for the user based on one or more of these various other inputs not explicitly provided by the user as an input.

[0119] Figure 5B shows a user interface screen 5200 of a mapping or navigation application that displays an exemplary customized digital map related to a location (geographical area) that can be presented on a display device associated with a user, according to one or more exemplary embodiments of the present disclosure. For example, the customized digital map can be presented for display to the user in response to the selection of a third user interface element 5126. For example, the generating map application 3100 may be configured to generate a map layer 5220 that shows a route 5222 from a starting point 5224 to a destination 5226 with one or more customized features personalized for a user associated with the computing device. As shown in Figure 5B, the generating map application 3100 may be configured to customize the pins representing the starting point 5224 and the destination 5226. For example, the pins may be generated according to the user's preference (e.g., a coffee cup may be used to represent the drink the user prefers in the morning), or according to the current event (e.g., a coffee cup may be used during National Coffee Week), or based on other factors. As shown in Figure 5B, the generating map application 3100 may be configured to customize other features of the map, such as the route 5222 between the starting point 5224 and the destination 5226. For example, the route may be generated according to the user's preference (e.g., a yellow brick road may be based on the user's favorite book), or according to contextual factors (e.g., a cobblestone road is used based on a similar road in the downtown area and is colored based on the color of the local sports team), or according to other factors. In some embodiments, the intensity of the color of the route can be displayed (and dynamically changed) based on the degree of traffic along the route (e.g., the color is darker when the traffic is heavy and lighter when the traffic is light). As shown in Figure 5B, the generating map application 3100 may be configured to generate an object 5228 associated with the location.For example, the generated map application 3100 may be configured to generate an object 5228 based on an input. In some embodiments, the input may include a text query that specifies one or more objects associated with a location. The object may be in the form of, for example, a graphic object, or an image, or an icon. As shown in FIG. 5B, the object 5228 may be a killer whale that corresponds to the user's favorite animal, or may correspond to an event celebrating the killer whale's migration season. Thus, the customized digital map shown in FIG. 5B reflects the user's preferences and can be personalized according to the user's preferences, while using visually distinctive icons, graphic objects, or images that are easily distinguishable by the user to reflect an accurate representation of the route between navigation points.

[0120] In some embodiments, when information from a customized map layer conflicts or collides with information from another layer (e.g., one or more of the layers shown in FIG. 6), the generated map application 3100 may be configured to prioritize the customized map layer over other map layers. For example, when multiple layers are called or stacked on the displayed digital map, the customized map layer can be presented or overwritten on top of other information from other layers.

[0121] The present disclosure is not limited to the exemplary user interface elements shown in FIGS. 4A through 5B, and other user interface elements may be provided, whereby the user can specify a request to obtain a customized digital map related to a location (or a route guidance or other navigation operation related to a location) according to various conditions or inputs. For example, the user interface element may be in the form of a pull-down menu, a selectable user interface element, a text box, etc. The input may be provided via a voice prompt.

[0122] FIG. 6 is a block diagram including an exemplary map data layer according to an exemplary embodiment of the present disclosure. One or more portions of the map data layer 6000 may be stored in one or more computing devices using any suitable memory. In some examples, the map data layer 6000 may be executed or implemented in one or more computing devices or computing systems including, for example, the computing device 100 and / or the server computing system 300.

[0123] As shown in FIG. 6, the map data layer 6000 includes a plurality of layers, examples of which may include a local coordinate system layer 6002, a position layer 6004, a plan layer 6006, a sign layer 6008, a traffic control device layer 6010, a landmark layer 6012, a hazard layer 6014, a default map layer 6016, a traffic layer 6018, and a custom view layer 6020.

[0124] The plurality of layers within the map data layer 6000 may be accessed by one or more computing systems and / or one or more computing devices including the computing device 100 and / or the server computing system 300. The generation map application 3100 (e.g., one or more generation machine learning models 3140) may be configured to reference information from the map data layer 6000 to generate a customized digital map in response to receiving an input and / or in response to a selection of a third user interface element 5126. As described herein, in some embodiments, the generation map application 3100 (e.g., one or more generation machine learning models 3140) may be configured to prioritize data or information from the custom view layer 6020 over data or information from one of the other map data layers.

[0125] The local coordinate system layer 6002 may include data and / or information associated with a coordinate system (e.g., x, y, and z coordinates corresponding to latitude, longitude, and altitude, respectively) that includes the current coordinate system used by the computing device 100 and / or the server computing system 300.

[0126] The position layer 6004 can include data and / or information related to a self-consistent description of the instantaneous position of the computing device 100 (e.g., the position of the computing device 100 at the current instant). This position can be associated in a manner that includes the physical position of the computing device 100 (e.g., pose, absolute velocity, and / or acceleration within a world or local coordinate frame) or the semantic position of the computing device 100 (e.g., the road segment on which the computing device 100 is currently located, where on the road segment the computing device 100 is, scalar velocity along the segment, and / or which lane the computing device 100 is in, etc.).

[0127] The plan layer 6006 can be used to annotate the local road and lane graph according to the current plan or route. The plan layer 6006 can enable the computing device 100 to determine the road or lane to proceed along with respect to the current navigation route. The plan layer 6006 can include identification information of the roads and lanes planned in front of the vehicle within the vehicle's field of view. The plan layer 6006 can be managed as part of the navigation interface. The route can be selected by searching for a destination, displaying possible routes, and starting turn-by-turn navigation. As the computing device 100 is in motion, the route may be automatically updated and recalculated as needed to adjust to changes in driving behavior and road or traffic conditions.

[0128] The identification layer 6008 can include data and / or information related to physical signs visible from a road segment, including the type, content, and three-dimensional position information of the signs. For example, the signs can include speed limit signs seen while driving on the road, including the speed limit text on the signs. The signs can be classified into sign types including speed limits or road names.

[0129] The traffic control device layer 6010 can be used to describe the location, attributes, and status of traffic control devices including stop signals and / or yield signs. The traffic control device layer can include notifications of the operating status of traffic control devices (e.g., operating, non-operating, or malfunctioning).

[0130] The landmark layer 6012 can be used to describe landmarks visible from the road and can include the three-dimensional positions of each landmark. Further, the landmark layer 6012 can be used for location identification. The attributes of the landmark layer 6012 include a location including coordinates (e.g., x, y, and z coordinates corresponding to latitude, longitude, and altitude) that can be used to determine the location of the landmark, and visibility that can be used to determine the distance at which the landmark can be seen based on a viewpoint (e.g., the viewpoint of a vehicle at a specific location). The landmark layer 6012 can be updated when a landmark is removed or moved.

[0131] The hazard layer 6014 can include data and / or information related to annotations of traffic accidents, dangerous locations, and / or obstacles to the road and lanes. For example, the hazards can include areas under construction, natural disasters (e.g., floods, fires, heavy rains, and / or heavy snows), and other potentially harmful events that can interfere with or delay the movement of vehicles.

[0132] The default map layer 6016 can include data and / or information related to the default map layer and can provide a digital map view that depicts the topography and the form of urban areas by combining a light color tone and a natural color tone.

[0133] The transportation map layer 6018 can include data and / or information indicating transportation options including subways, metro routes, bus stops, etc.

[0134] The custom view map layer 6020 can include data and / or information corresponding to a customized digital map generated via one or more generative machine learning models described herein based on input from a user indicating various features to be customized (e.g., according to the user's preferences).

[0135] Although not shown in FIG. 6, as part of the map data layer 6000, various other map data layers can be provided. For example, the other map data layers may include a transportation layer, a satellite layer, an air quality layer, a bicycle route layer, a topography layer, a street view layer, a forest fire layer, a fog layer, a parking layer, a road layer, a lane layer, etc.

[0136] FIG. 7A is a block diagram of an exemplary computing system for generating a customized digital map via a generative machine learning model in response to receiving an input or query according to one or more exemplary embodiments of the present disclosure. System 7100 includes a user computing device 7102, a server computing system 7130, and a training computing system 7150 communicatively coupled via a network 7180.

[0137] FIG. 7B is a block diagram of an exemplary computing device for generating a digital map customized via a generative machine learning model in response to receiving an input or query, according to one or more exemplary embodiments of the present disclosure.

[0138] FIG. 7C is a block diagram of an exemplary computing device for generating a digital map customized via a generative machine learning model in response to receiving an input or query, according to one or more exemplary embodiments of the present disclosure.

[0139] (which may correspond to computing device 100) The user's computing device 7102 can be any type of computing device, such as, for example, a personal computing device (e.g., a laptop or desktop), a mobile computing device (e.g., a smartphone or tablet), a game console or controller, a wearable computing device, an embedded computing device, or any other type of computing device.

[0140] The user's computing device 7102 includes one or more processors 7112 and a memory 7114. The one or more processors 7112 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or multiple processors operably connected. The memory 7114 can include one or more non-transitory computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 7114 can store data 7116 and instructions 7118 that are executed by the processor 7112 to cause the user's computing device 7102 to perform operations.

[0141] In some embodiments, the user's computing device 7102 may store or include one or more machine learning models 7120 (e.g., large language models, sequence processing models, generative machine learning models, etc.). For example, the one or more machine learning models 7120 can be various machine learning models such as neural networks (e.g., deep neural networks), or other types of machine learning models including non-linear models and / or linear models, or can include them. Exemplary neural networks can include feedforward neural networks, recurrent neural networks (RNNs) including long short-term memory (LSTM)-based recurrent neural networks, convolutional neural networks (CNNs), diffusion models, adversarial generative networks, or other forms of neural networks. Exemplary neural networks can be deep neural networks. Some exemplary machine learning models can utilize attention mechanisms such as self-attention. For example, some exemplary machine learning models can include multi-head self-attention models (e.g., transformer models). Exemplary machine learning models are described herein with reference to FIGS. 1A through 6.

[0142] In some embodiments, one or more machine learning models 7120 can be received from a server computing system 7130 through a network 7180, can be stored in a memory 7114, and can then be used or implemented by one or more processors 7112. In some embodiments, a user's computing device 7102 can implement multiple parallel instances of a single machine learning model (e.g., to perform parallel tasks across multiple instances of the machine learning model). In some embodiments, the task is a generation task, and one or more machine learning models can be implemented to output content (e.g., a customized digital map) considering various inputs (e.g., queries, adjustment parameters, etc.). More specifically, the machine learning models disclosed herein (including, for example, large language models, sequence processing models, generative machine learning models, etc.) can be implemented to perform various tasks related to an input query.

[0143] According to an example of the present disclosure, a computing system can implement one or more sequence processing models 3120 described herein to output values for one or more conditions in response to or based on a query. The one or more sequence processing models 3120 may include one or more machine learning models configured to process and analyze sequential data and handle data that occurs in a particular order or sequence, including time-sequential data, natural language text, or other data of a temporal or sequential structure.

[0144] According to an example of the present disclosure, a computing system may implement one or more large language models 3130 to determine a plurality of variables based on a query. For example, the large language model may include a Bidirectional Encoder Representations from Transformers (BERT) large language model. The large language model may be trained, for example, to understand and process natural language. The large language model may be configured to extract information from an input (query) to identify keywords, intents, and context within the input and to determine a plurality of variables for generating a customized digital map. The variables may include potential variables representing the underlying structure of the language.

[0145] According to an example of the present disclosure, a computing system may implement one or more generative machine learning models 3140 to generate a customized digital map depicting a location with one or more customized features having values for one or more conditions related to the customized features. The one or more generative machine learning models 3140 can include a deep neural network or an adversarial generative network (GAN) to generate a customized digital map depicting a location with one or more customized features having values for one or more conditions related to the customized features. For example, the one or more generative machine learning models 3140 may include a variational autoencoder, a stable diffusion machine learning model, a visual transformer, a neural radiance field (NeRF), etc. to generate a customized digital map depicting a location with one or more customized features having values for conditions related to the one or more customized features.

[0146] Additionally, or alternatively, one or more machine learning models 7140 can be included in a server computing system 7130 that communicates with the user's computing device 7102 according to a client-server relationship, or can be stored and implemented in other ways. For example, one or more machine learning models 7140 can be implemented by the server computing system 7130 as part of a web service (e.g., a navigation service, a mapping service, etc.). Thus, one or more machine learning models 7120 can be stored and implemented on the user's computing device 7102, and / or one or more machine learning models 7140 can be stored and implemented on the server computing system 7130.

[0147] The user's computing device 7102 can also include one or more user input components 7122 that receive user input. For example, the user input component 7122 can be a touch-sensitive component (e.g., a touch-sensitive display screen or a touch pad) that is sensitive to the touch of a user input object (e.g., a finger or a stylus). The touch-sensitive component can serve to implement a virtual keyboard. Other exemplary user input components include microphones, conventional keyboards, or other devices and methods by which a user can provide user input.

[0148] A server computing system 7130 (which may correspond to server computing system 300) includes one or more processors 7132 and a memory 7134. The one or more processors 7132 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or multiple processors operably connected. The memory 7134 can include one or more non-transitory computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 7134 can store data 7136 and instructions 7138 that are executed by the processor 7132 to cause the server computing system 7130 to perform operations.

[0149] In some embodiments, the server computing system 7130 includes or is otherwise implemented as one or more server computing devices. In an example where the server computing system 7130 includes multiple server computing devices, such server computing devices can operate according to a sequential computing architecture, a parallel computing architecture, or some combination thereof.

[0150] As described above, the server computing system 7130 can store or include one or more machine learning models 7140. For example, the one or more machine learning models 7140 can be or include various machine learning models. Exemplary machine learning models include neural networks or other multi-layer non-linear models. Exemplary neural networks can include feed-forward neural networks, recurrent neural networks (RNNs) including long short-term memory (LSTM)-based recurrent neural networks, convolutional neural networks (CNNs), diffusion models, adversarial generative networks, or other forms of neural networks. Exemplary neural networks can be deep neural networks. Some exemplary machine learning models can utilize attention mechanisms such as self-attention. For example, some exemplary machine learning models can include multi-head self-attention models (e.g., Transformer models). Exemplary machine learning models are described herein with reference to FIGS. 1A through 6.

[0151] The user's computing device 7102 and / or the server computing system 7130 can train one or more machine learning models 7120 and / or 7140 via interaction with a training computing system 7150 communicatively coupled via a network 7180. The training computing system 7150 can be separate from the server computing system 7130 or can be part of the server computing system 7130.

[0152] The training computing system 7150 includes one or more processors 7152 and a memory 7154. The one or more processors 7152 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be a single processor or multiple processors operably connected. The memory 7154 can include one or more non-transitory computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 7154 can store data 7156 and instructions 7158 that are executed by the processor 7152 to cause the training computing system 7150 to perform operations. In some embodiments, the training computing system 7150 includes or is otherwise implemented with one or more server computing devices.

[0153] The training computing system 7150 can include a model trainer 7160 that trains one or more machine learning models 7120 and / or 7140 stored in the user's computing device 7102 and / or the server computing system 7130 using various training or learning techniques such as, for example, backpropagation of error. For example, a loss function can be backpropagated through the model(s) and one or more parameters of the model(s) can be updated (e.g., based on the gradient of the loss function). Various loss functions can be used, such as mean squared error, likelihood loss, cross-entropy loss, hinge loss, and / or various other loss functions. Gradient descent can be used to iteratively update the parameters over a number of training iterations.

[0154] In some embodiments, performing backpropagation may include performing backpropagation that is truncated over time. The model trainer 7160 can perform several generalization techniques (e.g., weight decay, dropout, etc.) to improve the generalization ability of the model being trained.

[0155] In particular, the model trainer 7160 can train the machine learning models 7120 and / or 7140 based on a set of training data 7162. The training data 7162 can include various data sets that can be stored, for example, remotely or in the training computing system 7150. For example, in some embodiments, exemplary data sets used for training include a plurality of digital maps related to a particular location, a plurality of images related to a particular location, etc. However, other data sets of images and digital maps may also be used (e.g., images and digital maps from external websites). In some embodiments, the data set can be limited to a particular category, genre, terrain, landmark, time, etc. In some embodiments, the data set can encompass a variety of subjects including objects, terrain, individuals, groups of people, landmarks, structures, etc.

[0156] In some embodiments, when the user provides consent, the user's computing device 7102 can provide examples for training. Thus, in such embodiments, one or more machine learning models 7120 provided to the user's computing device 7102 can be trained by the training computing system 7150 based on user-specific data received from the user's computing device 7102. In some embodiments, this process can be referred to as personalization of the model.

[0157] The model trainer 7160 includes computer logic utilized to provide desired functionality. The model trainer 7160 can be implemented in hardware, firmware, and / or software that controls a general-purpose processor. For example, in some embodiments, the model trainer 7160 includes a program file stored on a storage device, loaded into memory, and executed by one or more processors. In other embodiments, the model trainer 7160 includes one or more sets of computer-executable instructions stored on a tangible computer-readable storage medium such as RAM, a hard disk, or an optical or magnetic medium.

[0158] The network 7180 can be any type of communication network, such as a local area network (e.g., an intranet), a wide area network (e.g., the Internet), or some combination thereof, and can include any number of wired or wireless links. Generally, communication via the network 7180 can be carried out via any type of wired and / or wireless communication using a variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encoding or formatting (e.g., HTML, XML), and / or security schemes (e.g., VPN, secure HTTP, SSL).

[0159] The machine learning models described herein can be used in a variety of tasks, applications, and / or use cases.

[0160] In some embodiments, the input to the machine learning model(s) of the present disclosure can be text or natural language data. The machine learning model(s) can process the text or natural language data to generate an output. As an example, the machine learning model(s) can process natural language data to generate a language encoding output. As another example, the machine learning model(s) can process text or natural language data to generate a latent text embedding output. As another example, the machine learning model(s) can process text or natural language data to generate a translation output. As another example, the machine learning model(s) can process text or natural language data to generate a classification output. As another example, the machine learning model(s) can process text or natural language data to generate a text segmentation output. As another example, the machine learning model(s) can process text or natural language data to generate a semantic meaning output. As another example, the machine learning model(s) can process text or natural language data to generate an upscaled text or natural language output (e.g., text or natural language data of higher quality than the input text or natural language). As another example, the machine learning model(s) can process text or natural language data to generate a prediction output.

[0161] In some embodiments, the input to the machine learning model(s) of the present disclosure can be audio data. The machine learning model(s) can process the audio data to generate an output. As an example, the machine learning model(s) can process the audio data to generate an audio recognition output. As another example, the machine learning model(s) can process the audio data to generate an audio translation output. As another example, the machine learning model(s) can process the audio data to generate a latent embedding output. As another example, the machine learning model(s) can process the audio data to generate an encoded audio output (e.g., an encoded and / or compressed representation of the audio data, etc.). As another example, the machine learning model(s) can process the audio data to generate an upscaled audio output (e.g., audio data of higher quality than the input audio data, etc.). As another example, the machine learning model(s) can process the audio data to generate a text representation output (e.g., a text representation of the input audio data, etc.). As another example, the machine learning model(s) can process the audio data to generate a prediction output.

[0162] In some embodiments, the input to the machine learning model(s) of the present disclosure can be sensor data. The machine learning model(s) can process the sensor data to generate an output. As an example, the machine learning model(s) can process the sensor data to generate a recognition output. As another example, the machine learning model(s) can process the sensor data to generate a prediction output. As another example, the machine learning model(s) can process the sensor data to generate a classification output. As another example, the machine learning model(s) can process the sensor data to generate a segmentation output. As another example, the machine learning model(s) can process the sensor data to generate a visualization output. As another example, the machine learning model(s) can process the sensor data to generate a diagnostic output. As another example, the machine learning model(s) can process the sensor data to generate a detection output.

[0163] FIG. 7A illustrates an exemplary computing system that can be used to implement aspects of the present disclosure. Other computing systems can be used as well. For example, in some embodiments, the user's computing device 7102 can include a model trainer 7160 and training data 7162. In such embodiments, one or more machine learning models 7120 can be both trained locally and used on the user's computing device 7102. In some of such embodiments, the user's computing device 7102 can implement a model trainer 7160 that personalizes one or more machine learning models 7120 based on user-specific data.

[0164] FIG. 7B is a block diagram of an exemplary computing device for generating a customized digital map via a generative machine learning model in response to receiving a query, according to one or more exemplary embodiments of the present disclosure. The computing device 7200 can be a user's computing device or a server computing device.

[0165] The computing device 7200 includes a plurality of applications (e.g., applications 1 to N). Each application includes its own machine learning library and one or more machine learning models. For example, each application can include a machine learning model. Exemplary applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, a map application, a navigation application, and the like.

[0166] As illustrated in FIG. 7B, each application can communicate with a plurality of other components of a computing device, such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some embodiments, each application can communicate with each device component using an API (e.g., a public API). In some embodiments, the API used by each application is specific to that application.

[0167] FIG. 7C is a block diagram of an exemplary computing device for generating a customized digital map via a generative machine learning model in response to receiving a query, according to one or more exemplary embodiments of the present disclosure. The computing device 70 can be a user computing device or a server computing device.

[0168] The computing device 7300 includes a plurality of applications (e.g., applications 1 to N). Each application communicates with a central intelligence layer. Exemplary applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, a map application, a navigation application, and the like. In some embodiments, each application can communicate with the central intelligence layer (and the model(s) stored therein) using an API (e.g., an API common across all applications).

[0169] The central intelligence layer includes a plurality of machine learning models. For example, as shown in FIG. 7C, each machine learning model can be provided to each application and managed by the central intelligence layer. In other embodiments, two or more applications can share a single machine learning model. For example, in some embodiments, the central intelligence layer can provide a single model to all applications. In some embodiments, the central intelligence layer is included within or otherwise implemented by the operating system of the computing device 7300.

[0170] The central intelligence layer can communicate with the central device data layer. The central device data layer can be a centralized repository of data for the computing device 7300. As illustrated in FIG. 7C, the central device data layer can communicate with a plurality of other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some embodiments, the central device data layer can communicate with each device component using an API (e.g., a private API).

[0171] When general terms such as "module" and "unit" are used in this specification, these terms may refer to software or hardware components or devices such as, but not limited to, a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC) that performs a specific task. A module or unit may be configured to reside on an addressable storage medium or may be configured to execute on one or more processors. Thus, a module or unit may include, by way of example, components such as software components, object oriented software components, class components and task components, processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided by the components and modules / units may be combined into fewer components and modules / units or further separated into additional components and modules.

[0172] Aspects of the exemplary embodiments described above may be recorded on a non-transitory computer-readable medium including program instructions for performing various operations implemented by a computer. The medium may also include program instructions, data files, data structures, etc., alone or in combination. Examples of non-transitory computer-readable media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD ROM disks, Blue-Ray disks, and DVDs, magneto-optical media such as optical disks, and other hardware devices specially configured to store and execute program instructions, such as semiconductor memories, read-only memories (ROMs), random access memories (RAMs), flash memories, USB memories, etc. Examples of program instructions include both machine code such as that generated by a compiler and files containing higher-level code that can be executed by a computer using an interpreter. The program instructions may be executed by one or more processors. The hardware devices described may be configured to function as one or more software modules for performing the operations of the above-described embodiments, or vice versa. Further, the non-transitory computer-readable storage medium may be distributed among computer systems connected via a network, and the computer-readable code or program instructions may be stored and executed in a decentralized manner. Further, the non-transitory computer-readable storage medium may also be embodied in at least one application-specific integrated circuit (ASIC) or field-programmable gate array (FPGA).

[0173] Each block of the flowchart illustrations may represent a unit, module, segment, or portion of code including one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks may occur out of the order noted. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.

[0174] Although the present disclosure has been described with respect to various exemplary embodiments, each example is provided for illustrative purposes and is not intended to limit the present disclosure. Those skilled in the art will readily be able to make changes, modifications, and equivalents to such embodiments upon reaching the foregoing understanding. Accordingly, the present disclosure does not exclude including such modifications, variations, and / or additions to the subject matter of the disclosed invention as would readily be apparent to those skilled in the art. For example, features illustrated or described as part of one embodiment may be used in another embodiment to create yet another embodiment. Accordingly, the present disclosure is intended to cover such changes, modifications, and equivalents.

Claims

1. 1. A computing device for generating a customized digital map, comprising: A display device; one or more memories configured to store instructions; One or more processors configured to execute the instructions to perform operations, the operations including: receiving input from a user related to customizing features associated with locations viewable on a digital map; responsive to receiving the input, implementing a generative machine learning model to generate the customized digital map, the customized digital map depicting the location with one or more customized features generated via the generative machine learning model based on the input; providing the customized digital map for presentation via the display device; and and one or more processors,

2. The computing device of claim 1 , wherein the generative machine learning model is provided on the computing device.

3. 2. The computing device of claim 1, wherein receiving the input from the user regarding customizing features associated with the places viewable on the digital map includes receiving a semantic description of tiles associated with the digital map to be generated.

4. the generative machine learning model is configured to generate the customized digital map by rendering a portion of the customized digital map that depicts the location with the one or more customized features; The computing device of claim 2 , wherein the operations further include receiving a remaining portion of the customized digital map rendered by a server computing system.

5. 5. The computing device of claim 4, wherein the operations further include implementing one or more machine learning models to smooth and blend the portion of the customized digital map rendered by the computing device with the remaining portion of the customized digital map rendered by the server computing system.

6. The operations further include receiving one or more default tiles associated with the location before receiving the input from the user; 2. The computing device of claim 1, wherein the generative machine learning model is configured to generate the customized digital map by referencing the one or more default tiles and the input to render the customized digital map depicting the location with the one or more customized features.

7. The computing device of claim 6 , wherein the one or more default tiles associated with the location are received at a predetermined time, in response to a predetermined condition, or in response to a predetermined event.

8. The predetermined time occurs periodically; the predetermined conditions are associated with network conditions of a network over which the one or more default tiles are received from a server computing system; The computing device of claim 7 , wherein the predetermined event is related to a state of the computing device.

9. the input indicates certain content to be omitted or reduced when generating the customized digital map; 2. The computing device of claim 1, wherein when generating the customized digital map, implementing the generative machine learning model to generate the customized digital map depicting the place with the one or more customized features includes omitting or reducing the certain content.

10. Receiving the input from the user regarding customizing features associated with the places viewable on the digital map includes receiving a selection of a user interface element corresponding to a customized map layer; 2. The computing device of claim 1, further comprising: in response to the selection of the user interface element, implementing the generative machine learning model to generate the customized digital map, the customized digital map depicting the location with the one or more customized features generated via the generative machine learning model based on the input.

11. 11. The computing device of claim 10, wherein the operations further include, in response to the selection of the user interface element, generating the customized map layer based on at least one of information related to the user or contextual information related to the location.

12. the computing device includes one or more databases configured to store a plurality of generative machine learning models each associated with a plurality of distinct locations; The computing device of claim 1 , wherein the operations further comprise obtaining the generative machine learning model associated with the location from among the plurality of generative machine learning models.

13. the input includes a text query specifying one or more objects associated with the location; The computing device of claim 1 , wherein the digital map depicts the location including the one or more objects.

14. The computing device of claim 1 , wherein the generative machine learning model is fine-tuned based on a large parameter generative machine learning model having a larger number of parameters than the generative machine learning model.

15. 1. A computer-implemented method for generating a customized digital map, comprising: receiving, by the computing device, input from a user related to customizing features associated with locations viewable on the digital map; implementing, by the computing device, a generative machine learning model in response to receiving the input to generate the customized digital map, the customized digital map depicting the location with one or more customized features generated via the generative machine learning model based on the input; providing, by the computing device, the customized digital map for presentation via a display device; A computer-implemented method comprising:

16. The computer-implemented method of claim 15 , wherein the generative machine learning model is provided on the computing device.

17. 16. The computer-implemented method of claim 15, wherein receiving the input from the user regarding customizing features associated with the places viewable on the digital map includes receiving a semantic description of tiles associated with the digital map to be generated.

18. Implementing the generative machine learning model includes: rendering a portion of the customized digital map depicting the location with the one or more customized features; receiving a remaining portion of the customized digital map rendered by a server computing system; and 16. The computer-implemented method of claim 15, comprising:

19. the input indicates certain content to be omitted or reduced when the customized digital map is generated; 16. The computer-implemented method of claim 15, wherein implementing the generative machine learning model includes omitting or reducing the certain content when generating the customized digital map.

20. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations to generate a customized digital map, the operations including: receiving input from a user related to customizing features associated with locations viewable on a digital map; responsive to receiving the input, implementing a generative machine learning model to generate a customized digital map, the customized digital map depicting the location with one or more customized features generated via the generative machine learning model based on the input; providing the customized digital map for presentation via a display device; and A non-transitory computer readable medium comprising:

Images