Machine learning-based models used to simulate dialogue interactions between users and points of interest.

By simulating the dialogue between POIs and users through machine learning models, the problem of low efficiency in semantic feature search in navigation services is solved, enabling efficient evaluation of the semantic features of multiple POIs and reducing the utilization of computing resources.

CN123125590APending Publication Date: 2026-11-17GOOGLE LLC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202580028965.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2024-04-19
Filing Date
2025-04-15
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing navigation services struggle to effectively search and filter semantically complex point-of-interest (POI) characteristics. Users need to browse a large amount of information to determine whether a POI possesses specific characteristics, resulting in high computational resource utilization and low efficiency.

Method used

We use a POI-specific language model derived from machine learning to simulate conversations between POIs and users, generating text content sets to provide information, and efficiently evaluating the semantic characteristics of multiple POIs through a chat interface.

Benefits of technology

It reduces the utilization of computing resources on mobile devices, improves the discoverability and selection efficiency of POIs, and allows users to quickly evaluate the semantic characteristics of multiple POIs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123125590A_ABST
    Figure CN123125590A_ABST
Patent Text Reader

Abstract

Information indicating the location of a specific geographic area, including multiple Points of Interest (POIs), is obtained. For a first POI among the plurality of POIs, a first set of text content is generated using a first machine learning-based POI-specific language model, configured to generate text content from the perspective of the first POI. For a second POI among the plurality of POIs, a second set of text content is generated using a second machine learning-based POI-specific language model, configured to generate text content from the perspective of the second POI. At least some of the second set of text content is provided to the user computing device.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Priority requirements

[0002] This application is based on and claims priority to U.S. Application No. 18 / 640,952, filed April 19, 2024. The entire contents of this application are incorporated herein by reference. Technical Field

[0003] This disclosure generally relates to providing information through simulated dialogue. More specifically, this disclosure relates to machine learning models, such as large language models (LLMs), trained to simulate dialogue from the perspective of various points of interest (POIs). Background Technology

[0004] Standard navigation services include route planning functionality to guide users from their starting point to desired points of interest (POIs). In some cases, navigation services also offer supplementary features to assist users in discovering new accessible POIs. Specifically, some navigation services store "profiles" of POIs, which include information related to specific POIs. Examples of information stored in POI profiles include user-generated content (e.g., reviews, images, videos, etc.), POI-specific information (e.g., opening hours, location, average wait time, etc.). Users of these navigation services can browse POI profiles to select the POIs they wish to visit. In this way, navigation services and other related services (e.g., mapping services, visual search services, etc.) can help users discover new accessible POIs. Summary of the Invention

[0005] Aspects and advantages of embodiments of this disclosure will be set forth in part in the description which follows, or may be learned from the description or by practice of the embodiments.

[0006] One example aspect of this disclosure relates to a computer-implemented method. The method includes: acquiring information indicating a specific geographic area where a user computing device is located, wherein the specific geographic area includes a plurality of points of interest (POIs). The method includes: for a first POI among the plurality of POIs, generating a first set of text content by the computing system using a first machine learning-based POI-specific language model, the first machine learning-based POI-specific language model being configured to generate text content from the perspective of the first POI. The method includes: for a second POI among the plurality of POIs, generating a second set of text content by the computing system using a second machine learning-based POI-specific language model, the second machine learning-based POI-specific language model being configured to generate text content from the perspective of the second POI. The method includes: providing at least some of the second set of text content to the user computing device by the computing system.

[0007] Another exemplary aspect of this disclosure relates to one or more tangible non-transitory computer-readable media storing computer-readable instructions that, when executed by one or more processor devices, cause the one or more processor devices to perform operations. The operations include: obtaining information from a user computing device indicating a specific geographic area, wherein the specific geographic area comprises a plurality of Points of Interest (POIs). The operations include: for a first POI among the plurality of POIs, generating a first set of text content using a first machine learning-based POI-specific language model, the first machine learning-based POI-specific language model being configured to generate text content from the perspective of the first POI. The operations include: for a second POI among the plurality of POIs, generating a second set of text content using a second machine learning-based POI-specific language model, the second machine learning-based POI-specific language model being configured to generate text content from the perspective of the second POI. The operations include: providing at least some of the second set of text content to the user computing device.

[0008] Another exemplary aspect of this disclosure relates to a user computing device. The user computing device includes one or more processor devices; and one or more tangible non-transitory computer-readable media storing computer-readable instructions that, when executed by the one or more processor devices, cause the one or more processor devices to perform operations. The operations include: receiving user input through an application executed by the user computing device, wherein the user input indicates a specific geographic area, and wherein the specific geographic area includes a plurality of Points of Interest (POIs). The operations include: sending information indicating the specific geographic area to a computing system associated with the application. The operations include: in response to sending the information, acquiring a set of text content generated using a machine learning-optimized POI-specific language model to generate text content from the perspective of POIs among the plurality of POIs. The operation includes: displaying messages comprising at least some of the text content set within a chat interface of the application executed by the user computing device, wherein the chat interface includes a POI representation of the POI as a chat participant within the chat interface, wherein the POI representation includes a representative POI image that visually indicates the POI, and wherein the message is depicted within the chat interface as being sent by the POI representation.

[0009] Other aspects of this disclosure relate to various systems, devices, non-transitory computer-readable media, user interfaces, and electronic devices.

[0010] These and other features, aspects, and advantages of the various embodiments of this disclosure will be better understood with reference to the following description and the appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the disclosure and, together with the description, serve to explain the relevant principles. Attached Figure Description

[0011] Referring to the accompanying drawings, a detailed discussion of embodiments is set forth in this specification for those skilled in the art, in which:

[0012] Figure 1 This is a block diagram of an environment based on some implementations of this disclosure, in which machine learning-based models for simulating dialogue interactions between points of interest (POIs) can be implemented.

[0013] Figure 2 An example user computing device is shown according to some implementations of this disclosure, which displays an interface of an application instance filled with a set of text content to simulate a dialogue between multiple Points of Interest (POIs).

[0014] Figure 3 This is based on some implementation methods of this disclosure. Figure 1A block diagram of the application / service modules of the computing system.

[0015] Figure 4 This is based on some implementation methods of this disclosure. Figure 1 A block diagram of the model orchestration module of the computing system.

[0016] Figure 5 An example user computing device is shown according to some implementations of this disclosure, which displays an interface for application instances populated with sets of text content to simulate dialogue between POI groups.

[0017] Figure 6A A block diagram of an example computing system, which simulates dialogue interactions between Points of Interest (POIs), is depicted according to some implementations of this disclosure.

[0018] Figure 6B A block diagram of an example computing device according to an example embodiment of the present disclosure is depicted, which performs optimization on a machine learning model to generate text content from the perspective of a specific POI.

[0019] Figure 6C A block diagram of an example computing device according to an exemplary embodiment of the present disclosure is depicted, which simulates dialogue interaction between POIs.

[0020] Figure 7 A flowchart is depicted for an example method for performing a dialogue interaction between a POI and a user, according to an example embodiment of the present disclosure.

[0021] Figure 8 A flowchart is depicted for an example method for performing a dialogue interaction between a POI and a user, according to an example embodiment of the present disclosure.

[0022] The repeated reference numerals across multiple figures are intended to identify the same features in various implementations. Detailed Implementation

[0023] Overview

[0024] Generally, this disclosure relates to providing information through simulated dialogue. More specifically, this disclosure relates to machine learning models, such as large language models (LLMs), trained to simulate dialogue from the perspective of various points of interest (POIs). Specifically, conventional navigation services include route planning functionality to guide users from their starting location to a desired POI. In some cases, navigation services also provide a "profile" of the POI, which includes information related to the specific POI (e.g., user-generated content (e.g., reviews, images, videos, etc.), opening hours, location, average wait time, etc.).

[0025] Users can submit queries for certain POI characteristics (e.g., "food," "restaurant," "park," etc.). In response, the navigation service can search and retrieve POI profiles that possess those characteristics. When searching for POIs with certain characteristics, conventional navigation applications can typically filter POIs based on semantically simple characteristics. For example, when searching for restaurants, conventional navigation applications allow users to search based on cuisine, price range, and location.

[0026] However, conventional navigation services typically cannot search or filter for more semantically complex characteristics, such as the "atmosphere" of a restaurant or whether the dogs in a dog park are too noisy. Instead, users must navigate to the POI's profile within an application associated with the navigation service (e.g., a navigation app, a visual search app, etc.) to access user-generated content (UGC) and other information included in the POI's profile. Based on the profile, users can determine whether the POI possesses those characteristics. For example, if a user wants to determine whether a specific dish at a restaurant is spicy, that user must browse through numerous user-submitted reviews to find one that mentions the seasoning in that particular dish (which may not even exist).

[0027] Therefore, the implementation described in this paper proposes a machine learning model (e.g., a large language model (LLM)) trained to simulate dialogue from the perspective of various Points of Interest (POIs). More specifically, a machine learning generative text model can be trained to engage in dialogue with a user from the perspective of a specific POI. If a user submits a query related to a local POI (e.g., a local restaurant), the query can be provided to the machine learning model, which can then generate a response to that query. This response can be provided to the user in a chat interface to simulate a dialogue between the user and the POI (e.g., represented by the model). For example, suppose a user submits a query to a virtual assistant application, such as "where can I get fasttakeout?" Restaurants located within a certain distance of the user can be identified, and a machine learning model can be instantiated to represent that restaurant. This model can process the query to generate a response based on POI-specific information (e.g., user-generated content). For example, if users frequently submit reviews indicating that takeout preparation time is five minutes, the model could generate a response such as, "We can have takeout ready in as soon as 5 minutes." This response could then be provided to the user within a chat interface to simulate a "conversation" between the restaurant and the user.

[0028] In this way, the implementation described in this paper can efficiently deliver information to users to improve the discoverability of Points of Interest (POIs). Furthermore, the implementation allows users to request semantically complex features from multiple POI-specific model instances associated with multiple POIs. POI-specific model instances can conversationally identify their similarities and differences with each other regarding the requested features. By enabling POI-specific LLMs to engage in dialogue, users can efficiently evaluate a large number of POIs in the time required to evaluate a single POI using conventional techniques (e.g., browsing each POI's profile separately). In this way, the implementation described in this paper effectively "parallelizes" the discoverability of POIs.

[0029] Furthermore, in many cases, the dialogue framework described in this paper naturally enables POI-specific model instances to provide contextual or supplementary information, which can help users select POIs. For example, suppose a user wants to find a new restaurant suitable for an evening date with their partner. If their partner likes desserts, the user can ask, "Where can I get a great dessert around me?" In response, a first POI-specific model instance of the first POI can indicate that the first POI offers delicious tiramisu, while a second POI-specific model instance can indicate that the second POI has the best ice cream in town. If the user knows that their partner prefers ice cream to tiramisu, the user can make a more effective POI selection decision based on the contextual information provided by the POI-specific model instance (e.g., expertise on tiramisu and ice cream).

[0030] The various aspects of this disclosure provide several technical effects and benefits. As an example of a technical effect and benefit, the implementation described herein can significantly reduce the utilization of computational resources on mobile devices. More specifically, when using conventional technologies (such as POI archives for navigation services), users searching for new POIs to access are limited to evaluating one POI at a time via their mobile devices. Due to this limitation, users must evaluate POIs sequentially using their mobile devices, which can require users to interact with their mobile devices for extended periods and thus consume significant computational resources (e.g., computation cycles, memory, power, storage, wireless bandwidth, etc.). However, the implementation described herein allows users to evaluate a large number of POIs within a small amount of time using a conversational chat interface populated with models representing various POIs, thereby reducing the utilization of computational resources on the mobile device. In this way, the implementation described herein can significantly reduce the utilization of computational resources on mobile devices.

[0031] Exemplary embodiments of this disclosure will now be discussed in further detail with reference to the accompanying drawings.

[0032] Example devices and systems

[0033] Figure 1 This is a block diagram of an environment 100 according to some implementations of this disclosure, in which machine learning-based models for simulating conversational interactions between points of interest (POIs) can be implemented. Specifically, in some implementations, the computing environment 100 includes a computing system 102. The computing system 102 may be a system for executing, implementing, facilitating, etc., various applications and services for navigation, visual search, virtual assistants, etc. The computing system 102 may include a processor device 104 and a memory 106. In some implementations, the computing system 102 may be a computing system including multiple computing devices. Alternatively, in some implementations, the computing system 102 may be one or more computing devices within the computing environment 100, which includes multiple distributed devices and / or systems. Similarly, the processor device 104 may include any computing or electronic device capable of executing software instructions to implement the functions described herein.

[0034] Memory 106 may be, or otherwise include, any means capable of storing data, including but not limited to volatile memory (random access memory, etc.), non-volatile memory, and storage devices (e.g., hard disk drives, solid-state drives, etc.). Specifically, memory 106 may include containerized units of software instructions (i.e., “packaged containers”). Containerized units of software instructions may collectively form a container that has been packaged using any type or manner of containerization technology.

[0035] A containerized unit of software instructions may include one or more applications and may further implement any software or hardware necessary to execute the containerized unit of software instructions in any type or manner of computing environment. For example, a containerized unit of software instructions may include software instructions (e.g., applications, dependencies, configuration files, libraries, related binaries, etc.) containing or otherwise implementing all components necessary for process isolation in any environment.

[0036] In some implementations, computing system 102 may include computing resources 108. Alternatively, in some implementations, workload management computing system 12 may access computing resources 108 from another location within computing environment 10. As described herein, computing resources 18 can generally refer to any type, manner, or set of physical and / or virtualized devices, software, instructions, modules, etc. Computing resources 108 may include processing devices (e.g., central processing unit (CPU), graphics processing unit (GPU), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), virtualized processing devices, accelerator devices, qubits, etc.), network resources (e.g., bandwidth, etc.), storage devices or systems (e.g., memory, hard disk, database, etc.), software resources (e.g., machine learning model instances, application instances, etc.), etc.

[0037] The memory 106 may include an application / service module 110. The application / service module 110 can implement, facilitate, coordinate, etc., the applications and / or services described above. For example, the application / service module 110 can implement navigation services utilized by users through a corresponding navigation application. As another example, the application / service module 110 can implement various services (e.g., navigation, visual search, virtual assistant, etc.) through multi-service applications (i.e., "super" applications, "all-in-one" applications, etc.).

[0038] This disclosure, particularly application / service module 110, focuses primarily on navigation-related applications and / or services to more clearly illustrate the various implementations described herein. However, it is important to note that applications and / or services are not limited to navigation or navigation-related use cases. Rather, the implementations described herein are applicable to any context in which a user wishes to evaluate a POI, place of business, commercial or government entity, etc.

[0039] Computing environment 100 may include user computing device 112. Users can access applications and / or services associated with application / service module 110 through devices such as user computing device 112. User computing device 112 may include processor device 114 and memory 116, as described with respect to processor device 104 and memory 106 of computing system 102. Memory 116 may include client application / service module 118. Client application / service module 118 can instantiate, implement, execute, etc., applications to interact with services associated with computing system 102. Specifically, client application / service module 118 may instantiate application instance 120. Application instance 120 may be an application that enables users of user computing device 112 to utilize applications implemented through application / service module 110.

[0040] For example, suppose a user of user computing device 112 wishes to utilize a navigation service implemented by application / service module 110. Client application / service module 118 can instantiate application instance 120. Application instance 120 can be an instance of a navigation application that enables the user to interact with the navigation service. Client application / service module 118 can use interface handler 121 to display the interface of application instance 120. The user can interact with application instance 120 using the interface. The interaction between the user, application instance 120, and / or the service implemented by application / service module 110 will be discussed in more detail in later sections of the specification.

[0041] Returning to computing system 102, application / service module 110 may include navigation information 119. Navigation information 119 may include map mapping information, route planning information, geographic information, or any other type of information used to provide navigation services. For example, navigation information 119 may be, or otherwise include, a data repository containing a top-down map view of a specific geographic area. As another example, navigation information 119 may identify points of interest (POIs) within a specific geographic area. For example, for a specific city, navigation information 119 may include the names and coordinates of various types of POIs within that city (e.g., restaurants, businesses, public infrastructure, etc.).

[0042] As described in this article, a Point of Interest (POI) generally refers to a physical location, area, or entity of any type or manner accessible to a user, such as buildings, businesses, public infrastructure, private infrastructure, etc. Furthermore, a POI can refer to a location or area at any level of granularity. For example, for an amusement park, the following can all be considered different POIs: a specific roller coaster within the park, a specific area of ​​the park including that roller coaster, the park itself, and the geographical area where the park is located (e.g., neighborhood, city, county, state, country, continent, etc.). In some implementations, a POI may refer to a virtual, non-physical area or location within a virtualized space. For example, a POI can refer to a virtualized representation of a real-life location (e.g., a realistic virtual representation of an actual park). As another example, a POI can refer to a fictional location or a virtualized representation of a previously existing historical location.

[0043] In some implementations, a POI may refer to a service different from the service implemented by application / service module 110. For example, suppose application / service module 110 implements a navigation service. The POI may refer to a specific restaurant. Alternatively, a POI may refer to a separate service, such as a food delivery service or a ride-sharing service, that facilitates a user's journey to the POI or the delivery of goods / services from the POI to the user. The implementation of service POIs will be discussed in further detail in the description.

[0044] Application / service module 110 may include service information 122. Service information 122 may include information obtained through (or for) the services and / or applications of application / service module 110. Specifically, service information 122 may include user-generated content (UGC) submitted by a user through interaction with application / service module 110. For example, a user of user computing device 112 may access a specific restaurant and take a picture of the food they eat. The user can upload the image using application instance 120. Application / service module 110 may receive the image and store it in service information 122, associating the image with the restaurant. In this way, service information 122 can serve as a repository for information and / or user-generated content specific to a particular point of interest (POI).

[0045] Specifically, service information 122 may include UGC 124 and POI-specific information 126. UGC 124 may include user-generated content of any type or manner, such as images, videos, audio, text content (e.g., comments, etc.). POI-specific information 126 may include information of any type or manner related to the POI, such as location information (e.g., the precise location or size of a specific POI), historical information (e.g., the number of times a user has visited the POI, the number of users who have visited the POI, etc.), operational information (e.g., business hours, accessibility options availability, pricing information, contact information, product catalogs, etc.). In some implementations, POI-specific information 126 may be submitted by the POI or its representative. For example, the POI may provide POI-specific information 126 through an application programming interface (API) implemented by application / service module 110 (e.g., without using a corresponding application instance). Alternatively, in some implementations, POI-specific information 126 may be submitted by the user (e.g., through an application instance).

[0046] To follow the illustrated example, POI-specific information 126 may include information specific to a particular POI (e.g., a retail business) identified by the identifier POI_907836. POI-specific information 126 for POI_907836 may include values ​​for specific POI characteristics. For example, POI-specific information 126 may include a specific POI type (e.g., retail) for the POI. As another example, POI-specific information 126 may indicate the POI's "cost tier," thus revealing the cost of the products or services offered by the POI relative to similar products or services offered by other POIs of the same type. As yet another example, POI-specific information 126 may indicate semantic tags associated with the POI. Such tags can be extracted from UGC 124 (e.g., processing user-submitted comments using a large language model (LLM), processing user-submitted images using a computer vision model to obtain semantic output, etc.). For example, based on the semantic tags “quick” and “colorful” used for POI_907836, it can be inferred that comments on user-submitted POIs typically describe POI_907836 as quick and colorful, and / or that images or videos of user-submitted POIs depict a clean and colorful environment.

[0047] Returning to user computing device 112, the client application service module may include interface processing program 121. Interface processing program 121 can generate and display an interface 128 for user computing device 112. For example, interface processing program 121 can display an operating system interface, enabling the user of user computing device 112 to interact with various functions or services implemented by user computing device 112. As another example, interface processing program 121 can generate an interface for application instance 120, enabling the user to directly interact with application instance 120, thereby interacting with services implemented by application / service module 110 that communicate with application instance 120.

[0048] Client application / service module 118 can send query information 130 to application / service module 110. Query information 130 may include or otherwise describe a query 131 from a user. The user's query 131 may be a query for a specific type of POI, or a query for information related to certain POIs or POI types. For example, a user may provide a query for POIs that meet certain criteria, such as upscale restaurants within walking distance, local parks open at night, or local attractions suitable for children. As another example, a user may provide a query for POI-related information, such as the holiday opening hours of a user's local gym or whether the chef at their favorite restaurant has changed.

[0049] In some implementations, the user of user computing device 112 may submit query 131 through interface 128. For example, the user may submit query 131 through a text input device associated with user computing device 112 (e.g., via a touchscreen device, keyboard device, etc.). Alternatively, in some implementations, the user may submit query 131 through another input device. For example, the user may submit query 131 by using spoken words captured by an audio capture device. As another example, the user may submit a visual query through an image capture device. As yet another example, the user may submit a query by signaling intent via a multimodal combination of gestures or signals (e.g., gestures, spoken words, eye movements, predictive analytics, historical user behavior, etc.).

[0050] Alternatively, in some implementations, application instance 120 or client application / service module 118 can infer the user's query intent and generate query 131 without explicit user instruction. More specifically, in some cases, client application / service module 118 can proactively generate query information 130 without receiving a direct query from the user. For example, suppose a user requests directions to a hotel in a nearby major city. Client application / service module 118 can predict that the user will want to dine at a restaurant within a certain distance of the hotel. Client application / service module 118 can provide query information 130 instructing the user to search for restaurants near the user's hotel.

[0051] Furthermore, in some implementations, query information 130 may include contextual information provided along with query 131. In some implementations, the contextual information may include location information. For example, client application / service module 118 may include location identifier 133. Location identifier 133 may identify the current location of user computing device 112 and / or predict the future location of user computing device 112 (e.g., based on requested navigation instructions, etc.). Alternatively, in some implementations, contextual information may include user-specific information, such as user preferences (e.g., dining preferences, dietary restrictions, accessibility requirements, etc.) and user behavior (e.g., preferred POI type, preferred travel distance, etc.). User-specific information may be collected, stored, or implemented in a privacy-preserving manner that does not identify the user.

[0052] Query information 130 can be obtained through application / service module 110. Application / service module 110 may include POI selector 132. POI selector 132 can identify and select POIs based on query information 130. In some implementations, POI selector 132 can select POIs based on the distance between the user's current location and the POI location. Alternatively, in some implementations, POI selector 132 can select POIs based on other criteria, such as the relevance between query 131 and the POI, POI characteristics (e.g., POI type, POI duration, POI size, etc.), traffic characteristics (e.g., whether the POI is accessible by public transportation, etc.).

[0053] For example, suppose query information 130 includes a search for restaurants within a user's walking distance. Query information 130 may indicate the user's current location. POI selector 132 may first determine the maximum distance to represent the user's "walking distance" (e.g., based on user history information (e.g., the user's average walking distance), user preferences (e.g., whether the user needs accessibility options, whether the user wants to use public infrastructure, etc.). It should be noted that in some implementations, the distance between the POI and the user may refer to the "travel distance" the user expects to travel, rather than an objective distance.

[0054] Once the maximum distance is determined, the POI selector 132 can select POIs within that distance that match the characteristics requested by the user in the query information 130. Following the previous example, the POI selector 132 can select POIs of the "restaurant" POI type based on the user's request for a restaurant. Additionally, in some implementations, the POI selector 132 can filter out some POIs from the selected POIs based on information included in the service information 122 and / or the query information 130. For example, if the service information 122 indicates that the user has a severe shellfish allergy, the POI selector 132 can filter out restaurants that primarily serve shellfish from the selected POIs. As another example, if the service information 122 indicates that the user dislikes low-light environments, the POI selector can filter out dimly lit restaurants from the selected POIs.

[0055] Alternatively, in some implementations, the POI selector 132 can rank the selected POIs based on the same information. Following the previous example, suppose a restaurant serves shellfish, but doesn't primarily serve shellfish dishes. The POI selector 132 wouldn't exclude this POI, but would lower its ranking relative to other selected POIs. As another example, suppose user reviews of the same restaurant indicate that the service is very fast. If the user prefers fast service, the POI selector 132 could improve the restaurant's ranking. In this way, the POI selector 132 can select an optimal set of POIs for the user based on contextual information and user preferences.

[0056] POI selector 132 can generate POI selection information 134. POI selection information 134 can indicate a set of POIs, which may include some (or all) of the POIs selected by POI selector 132. For example, suppose application / service module 110 specifies a maximum number of POIs to select. If the number of POIs initially selected by POI selector 132 is greater than the maximum number, POI selector 132 can rank the selected POIs and remove the lowest-ranked POIs until the maximum number is reached. In some implementations, POI selection information 134 may include context information. Context information may include information extracted from POI-specific information 126, partial (or complete) UGC 124, partial (or complete) query information 130, etc.

[0057] Application / service module 110 may include model orchestration module 136. Model orchestration module 136 can perform various actions to orchestrate machine learning models, including model instantiation, training, inference, storage, updating, fine-tuning, etc. Specifically, model orchestration module 136 can orchestrate a machine learning-based POI-specific language model 138. As described herein, the machine learning-based POI-specific language model 138 can be a machine learning model (or an instance thereof) configured (e.g., through prompts, training, fine-tuning, etc.) to generate text content sets 139A-139N (typically text content set 139) from the perspective of a specific POI.

[0058] For example, suppose one of the POIs indicated by POI selection information 134 is a restaurant called "Steve's Hamburgers". A machine learning-based POI-specific language model 138 could include a model trained or prompted to generate text content from the perspective of "Steve's Hamburgers" or its representatives. If query information 130 includes the query "where can I get a vegetarian-friendly hamburger?", the model could generate a set of text content such as "Here at Steve's, we make all kinds of vegetarian-friendly hamburgers, like tuna burgers and black bean burgers!"

[0059] In some implementations, the machine learning-based POI-specific language model 138 can be an instance of the same model that is prompted to respond from the perspective of a specific POI. Following the preceding example, the model orchestration module 136 can instantiate a first LLM instance and use the first LLM instance along with a first prompt to process query information 130, which prompts the model to "actlike a representative of Steve's Burgers." The model orchestration module 136 can instantiate a second LLM instance and use the second LLM instance along with a second prompt to process query information 130, which prompts the model to act as a representative of another corresponding POI. Alternatively, in some implementations, the model orchestration module 136 can provide query information 130 and a second prompt to the first LLM instance to reduce the total number of model instances loaded into memory.

[0060] Alternatively or concurrently, in some implementations, the machine learning-based POI-specific language model 138 may include instances of different models. Following the preceding example, suppose the first POI (e.g., for “Steve’s Burger”) is a fast-food restaurant, and the second POI is a high-end seated restaurant. The first LLM instance could be an LLM instance trained or fine-tuned to generate text content from the perspective of the fast-food restaurant, while the second LLM instance could be an LLM instance trained or fine-tuned to generate text content from the perspective of the high-end seated restaurant. In some implementations, the model orchestration module 136 may prompt the model configured for a specific type of POI. Following the preceding example, prompts could be made for LLM instances trained to generate text content from the perspective of the fast-food restaurant to specifically generate text content from the perspective of “Steve’s Burger.” Thus, the model (and / or corresponding model instances) can be optimized and / or prompted to simulate the perspective of a specific POI, a specific POI type, etc.

[0061] In some implementations, the machine learning-based POI-specific language model 138 may include a model or part of a model trained to process non-textual content (such as image data, video data, audio data, etc.). For example, suppose query information 130 includes a query image and corresponding text content. The machine learning-based POI-specific language model 138 may include a multimodal model that generates multimodal intermediate representations (e.g., lexical representations, etc.) of the query image and corresponding text content. Similarly, the machine learning-based POI-specific language model 138 may also include a generative model or part of a model trained to generate content other than text content, such as images, videos, speech, application-specific data (e.g., map mapping data, route planning data, etc.). To follow the preceding example, in addition to text content, the LLM instance representing "Steve's Burgers" may also generate representative images of "Tuna Burger" or "Black Bean Burger," or select existing representative images of those items (e.g., from UGC 124, etc.).

[0062] The model orchestration module 136 can select, instantiate, or otherwise implement a machine learning-based POI-specific language model 138 in response to POI selection information 134. For example, if POI selection information 134 indicates five different POIs, the model orchestration module 136 can instantiate five instances of the machine learning-based POI-specific language model 138. Alternatively, the model orchestration module 136 can instantiate fewer than five instances. For example, if three POIs belong to a first POI type (e.g., fast food restaurant) and two POIs belong to another POI type (e.g., convenience store), the model orchestration module 136 can instantiate two machine learning-based POI-specific language models 138: one trained to generate text content from the perspective of POIs of the first POI type, and the other trained to generate text content from the perspective of POIs of the second POI type.

[0063] The model orchestration module 136 may include a model store 140. The model store 140 may store machine-learned POI-specific language models 138. Additionally, in some implementations, the model store 140 may store contextual information associated with the various machine-learned POI-specific language models 138, such as version control information and training datasets. Furthermore, in some implementations, the model orchestration module 136 may include a hint store 142 for storing hints that prompt the machine-learned POI-specific language models 138 to generate text content from certain perspectives as needed. The model orchestration module 136 may include a model optimizer 144. The model optimizer 144 may train, fine-tune, hint, or otherwise optimize the machine-learned POI-specific language models 138 to generate content from the perspective of certain POIs.

[0064] Specifically, in some implementations, model store 140 may include a single large language model. Model store 140 may instantiate multiple instances of an LLM and may perform zero-shot or few-shot prompts on these instances with data associated with the POI (e.g., descriptions, images, reviews, videos, etc.). The POI may also have specific metadata extracted from other sources (e.g., public information repositories), which may be provided as contextual information to the LLM instances. Alternatively, in some implementations, model store 140 may include a single LLM, but this single LLM has specific fine-tuned weights that can be combined by POI. Alternatively, in some implementations, model store 140 may include multiple LLMs, which may alternatively be fine-tuned according to the type of POI. For example, model store 140 may include an LLM trained to generally represent restaurants. If multiple restaurants are selected, this particular LLM may be instantiated and prompted with multiple prompts corresponding to the multiple restaurants respectively. Model store 140 may include additional LLMs for other types of POIs (e.g., parks, services, etc.).

[0065] Model orchestration module 136 may include inter-model communication handler 146. Inter-model communication handler 146 can facilitate communication between machine learning-based POI-specific language models 138 (or instances thereof). Specifically, inter-model communication handler 146 may select one of the text content sets 139 generated by one of the machine learning-based POI-specific language models 138 and use the selected text content as input to another machine learning-based POI-specific language model 138 in a conversational manner (e.g., prompting a second model to respond to a first model). For example, suppose text content set 139A asserts a specific fact about a POI. Inter-model communication handler 146 may use another machine learning-based POI-specific language model to process query information 130 and text content set 139A to generate text content set 139B. Text content set 139B may include a response to text content set 139A that states a fact about a POI.

[0066] For example, suppose query 130 indicates a query for the best pecan pie in the local area. One of the machine learning-based POI-specific language models 138 can process query 130 to generate a text content set 139A from the perspective of a first POI (e.g., “our customers here at John's pie emporium claim that we have the best pecan pie in the state”). Another of the machine learning-based POI-specific language models 138 can generate a second text content set 139B in response to the text content generated by the first model (e.g., “the customers at Ryan's pieshop say the same, but Ryan's Pie shop came in first place at the 2023 PecanPie contest!”).

[0067] Application / service module 110 may include chat implementation module 148. Chat implementation module 148 facilitates the provision of model output (e.g., text content set 139) from a machine learning-based POI-specific language model 138 to a user computing device within the chat interface. Specifically, application / service module 110 may generate response information 150. Response information 150 may include text content set 139 generated using the machine learning-based POI-specific language model 138 in response to query information 130. Chat implementation module 148 may generate response information 150 such that text content set 139 is formatted for display within the chat interface at user computing device 112.

[0068] The chat implementation module 148 may include a representation image acquirer 152. The representation image acquirer 152 can acquire a representation image 153 to represent a specific POI within the interface 128. More specifically, typical chat applications often use dialog boxes to represent messages from users, derived from a representative "avatar" representing a specific message source. Common examples of avatars used in messaging applications include logos, photos, "default" representations (e.g., circles containing the initials of a user's name), etc. The representation image acquirer 152 can acquire the representation image 153 to represent a specific POI within the chat interface of the interface 128. For example, the representation image acquirer 152 can generate one of the representation images 153 using a generative visual model. As another example, the representation image acquirer 152 can select one of the representation images 153 from UGC 124 or POI-specific information 126. As yet another example, the representation image acquirer 152 can generate instructions causing the user computing device 112 to use an image stored locally on the user computing device 112 as one of the representation images 153. The chat implementation module 148 can include the representation image 153 in the response information 150.

[0069] The chat implementation module 148 can format the response information 150 for display within the chat interface 128. The response information 150 can associate a set of text content or a "message" with a machine learning-based POI-specific language model 138 used to generate those text content sets. The response information 150 may also include sequence identifiers for the text content sets to indicate the order in which the sets should be displayed within the chat interface. In some implementations, the response information 150 may indicate whether a set of text content is a response to a previous set, allowing the response to be displayed in a manner that suggests it is a response (e.g., by providing a visual association between the response and the previous set of text content).

[0070] More specifically, returning to user computing device 112, client application / service module 118 can display interface 128 via interface handler 121. Once the user provides query 131 (e.g., described by query information 130), query 131 can be displayed as a message in interface 128. For example, suppose application instance 120 is a navigation application, and interface 128 is an interface implemented by (or is) the navigation application. The user can submit a query through the navigation application (e.g., submit it to a search element of the navigation application displayed within interface 128). After submission, interface 128 can become a typical “chat” interface of a messaging-based application, where messages are described as “conversations” originating from the user’s visual representation (e.g., including circles of the initials of the user’s name, images of the user’s frequently used profiles, etc.). Query 131 can be depicted as a “conversational” message originating from (i.e., sent by) the user.

[0071] The client application / service module 118 of the user computing device 112 may include an inter-application communication module 154. The inter-application communication module 154 facilitates communication between application instance 120 and other application instances executed by the user computing device 602. Alternatively, in some implementations, the inter-application communication module 154 may communicate with external services implemented by the computing system 102 or other computing systems. For example, suppose a user of the user computing device 112 selects a specific restaurant from a list of selected points of interest (POIs). Based on additional input from the user, the inter-application communication module 154 may interact with a food delivery application or service to deliver food from the restaurant to the user. Alternatively, the inter-application communication module 154 may select a web browser or document reading application to allow the user to view the restaurant's menu. Alternatively, the inter-application communication module 154 may select an augmented reality (AR) / virtual reality (VR) application or operating system functionality to explore the representation of the POI in an AR / VR environment.

[0072] return Figure 2 , Figure 2 An example user computing device is shown according to some implementations of the present disclosure, which displays an interface 128 for application instance 120 filled with text content set 139 to simulate a dialogue between multiple POIs. Figure 2 Combining Figure 1 Let's discuss this together. Specifically, to follow the illustrated example, the user computing device 112 could be a smartphone device used by the user. The user can provide a query 131 (e.g., via text input field 202, etc.). Upon receiving the query, interface 128 can be displayed as a chat interface, and query 131 can be displayed as a message sent from a representation image of the user. To follow the illustrated example, the user's representation image could be a "placeholder image" depicting a circle with the initials of the user's name.

[0073] Query information 130 (including query 131) can be sent to computing system 102. In response, user computing device 112 can receive response information 150. Response information 150 may include a text content set 139 and a representation image 153. The response information can be formatted so that it can be inserted into chat interface 128. Similar to query 131, text content set 139 can be depicted as a "dialogue" emanating from representation image 153 representing the corresponding POI.

[0074] To follow the depicted example, POI selector 132 may select three POIs (e.g., Sullievan's Steaks, Greg's Grill, and Fannie's Food Delivery) in response to query information 130. The text content set 139 is depicted as messages or "dialogues" derived from representation images 153 representing the POIs. Specifically, one of the machine learning-based POI-specific language models 138 may be trained to generate text content from the perspective of the first POI (e.g., Sullievan's Steaks). The machine learning-based POI-specific language model 138 can generate a first text content set 139A from the perspective of a first POI (e.g., "Sullivan Steakhouse") as a response to query 131. Another machine learning-based POI-specific language model 138 can be trained to generate text content from the perspective of a second POI (e.g., Gregg's Grill). This model can generate a second text content set 139B from the perspective of the second POI as a response to query 131 (and optionally, as a response to the first text content set 139A). Yet another machine learning-based POI-specific language model 138 can be trained to generate text content from the perspective of a third POI (e.g., Fanny Food Delivery). This model can generate a second text content set 139C from the perspective of the third POI as a response to query 131 and text content sets 139A and 139B. In this way, the implementation method described herein allows users to efficiently and effectively evaluate multiple POIs within a chat-based interface.

[0075] Return to Figure 1 In some implementations, the client application / service module 118 of the user computing device 112 may include a client model orchestration module 156. In some implementations, the client model orchestration module 156 may orchestrate local machine learning model inference to perform some (or all) of the operations described with respect to the model orchestration module 136. The client model orchestration module 156 may store a distillation model repository 158, which includes models that are typically “smaller” than the machine learning-based POI-specific language model 138, or otherwise require fewer computational resources for inference. These models can be trained by distillation from the machine learning-based POI-specific language model 138.

[0076] Figure 3 This is based on some implementation methods of this disclosure. Figure 1 Block diagram of application / service module 110 of computing system 102. Figure 3 Combining Figure 1Let's discuss this together. Specifically, the application / service module 110 may include service information 122. Service information may include UGC 124 and POI-specific information 126. POI-specific information 126 may describe various characteristics of the POI, such as POI type, cost level, etc. For example, POI type may refer to the general classification of the POI (e.g., restaurant, retail store, company headquarters, manufacturing site, park, airport, subway, etc.). As another example, "cost level" may typically indicate the expected price of a good or service relative to other good or services of the same type.

[0077] In some implementations, POI-specific information 126 may include semantic tags or markers for the POI. Semantic tags can be obtained using various techniques. For example, semantic tags can be obtained from the POI via an API implemented by the application / service module 110. As another example, semantic tags can be generated by processing UGC 124 for the POI using methods such as LLM. Semantic tags can be used to select and / or rank POIs via the POI selector 132. Semantic tags can also be used to prompt the machine learning-based POI-specific language model 138. To follow the illustrated example, the semantic tag “SLOW” can be used to prompt the POI model represented by POI_ID097632 to indicate that the POI is not known for its slow service. In this way, the implementations described herein can enforce accurate responses from the machine learning-based POI-specific language model 138.

[0078] POI-specific information 126 may include information beyond POI-specific characteristics. For example, POI-specific information 126 may include information provided by the POI, such as opening hours, accessibility information, maps (e.g., indoor maps or guides), logos, menus, etc. In some implementations, POI-specific information 126 may include performance metrics or performance-based statistics for the POI, such as health scores, average wait times, rating scores, etc. Alternatively, in some implementations, POI-specific information 126 may include time information or information related to an upcoming event. For example, suppose the POI identified by POI_ID590892 is a brewery planning to host a local music festival in the near future. The POI may provide POI-specific information 126 to the application / service module 110 describing the event. POI-specific information 126 can be used as contextual input to a model representing the POI until the event has occurred. Alternatively, time information may be obtained through some other means, such as analyzing social media content.

[0079] Service information 122 may include application information 302. Application information 302 may include information specific to the application or service implemented by application / service module 110. Application information 302 may include information describing previous interactions with the POI, the type of POI, or the representation of the POI within the application or service. For example, application information 302 may indicate whether a certain type of user prefers a certain type of POI. As another example, application information 302 may indicate the average number of visitors to the POI over time or at certain times.

[0080] In some implementations, application information 302 may include integration information for integrating the POI with other websites, applications, services (e.g., third-party services, other services provided by application / service module 110, etc.). Specifically, in some implementations, application information 302 may include integration information for integrating the POI with certain services. For example, the integration information may integrate a restaurant POI with a food delivery service (e.g., enabling users to order takeout directly from the restaurant within application instance 120). As another example, the integration information may integrate a destination POI (e.g., a park, attraction, cinema, etc.) with a carpooling service (e.g., enabling users to request carpooling directly to the POI within application instance 120).

[0081] Alternatively, in some implementations, the integration information may include web integration for integration with web-based services, such as social media services. For example, the integration information may include links to relevant social media accounts for a specific POI. In some implementations, if the integration information includes links to social media accounts or other sources of social media information, the application / service module 110 can extract information from those social media accounts, such as recent posts, updates, uploaded content, etc. In this way, the application / service module 110 can prompt the model with up-to-date information to ensure the timeliness of the model's output.

[0082] In some implementations, the service information may include user history information 304. User history information 304 may describe previous interactions between the user of the user computing device 112 and the POI. For example, user history information 304 may include predictions of certain POI preferences, such as preferred POI types, preferred business hours, preferred travel distances, etc. In some implementations, the prediction of POI preferences may be based on the analysis of interactions between a specific user and other POIs. Alternatively, in some implementations, the prediction of POI preferences may be based on the analysis of interactions between multiple users of the application or service and other POIs. In some implementations, user history information 304 may describe user preferences (e.g., dining preferences, dietary restrictions, accessibility requirements, etc.), user behaviors (e.g., preferred POI types, preferred travel distances, etc.). User-specific information may be collected, stored, or implemented in a privacy-preserving manner that does not identify the user.

[0083] In some implementations, application / service module 110 may include location information 306. Location information 306 may indicate the current location of user computing device 112. Alternatively, in some implementations, location information 306 may indicate the predicted location of user computing device at a future point in time. For example, user computing device 112 may submit a request for a route to a specific destination. Application / service module 110 may obtain a route generated for the user and use that route to predict the future location of user computing device at a future point in time. In some implementations, location information 306 may be generated by application / service module 110 (e.g., as described in the preceding examples). Alternatively, in some implementations, location information 306 may be provided by user computing device 112.

[0084] POI selector 132 can process location information 306 and service information 122 to obtain POI selection information 134. POI selection information 134 can indicate the selected set of POIs, represented using a machine learning-based POI-specific language model 138. In some implementations, to generate POI selection information 134, POI selector 132 may include POI exclusion criteria information 308. For example, POI selector 132 can generate POI exclusion criteria information 308, and / or may receive POI exclusion criteria information 308 from user computing device 112.

[0085] POI exclusion criteria information 308 may indicate minimum or maximum values ​​for certain criteria to determine whether a POI should be excluded from POI selection information 134. Following the illustrated example, POI exclusion criteria information 308 may include a highest cost level (e.g., level 4) and a lowest cost level (e.g., level 2). Therefore, POIs with a cost level of level 1 or level 5 may be excluded from POI selection information 134.

[0086] Alternatively, in some implementations, the POI exclusion criteria information 308 may include the user's maximum travel distance to the POI. In some implementations, the maximum travel distance may be determined based on an objective distance metric between the user's current location and the POI. Alternatively, in some implementations, the maximum travel distance may be determined more accurately based on an analysis that considers available transportation modes (e.g., private transportation, carpooling services, public transportation, etc.), current traffic levels, predicted traffic congestion, etc. The POI selector 132 may make such a determination based on information within the service information 122. For example, application information 302 may include information describing the current transportation mode, while user history information 304 may indicate whether the user is willing to use public transportation.

[0087] Alternatively or alternatively, in some implementations, POI exclusion criteria information 308 may include accessibility criteria, such as the availability of accessibility options. POI selector 132 may make such a determination based on service information 122. For example, a portion of POI-specific information 126 may indicate whether the POI complies with accessibility regulations (e.g., a portion provided by the POI, a portion provided by a government entity that periodically assesses POI accessibility compliance, etc.). As another example, UGC 124 may include images depicting accessibility features. Such accessibility features may be detected based on analysis of UGC images performed by POI selector 132 (e.g., using various computer vision techniques or machine learning models). As yet another example, user history information 304 may indicate whether a user has any specific accessibility requirements or preferences. Alternatively or alternatively, in some implementations, POI exclusion criteria information 308 may include dietary preference criteria.

[0088] It should be noted that failure to meet one (or more) exclusion criteria does not necessarily exclude a POI from the POI selection information 134. Instead, in some implementations, exclusion criteria can be weighted and used to rank the POIs indicated by service information 122. For example, a POI exceeding the maximum distance criterion is relatively more likely to receive negative user feedback, and therefore exceeding the maximum distance criterion may result in a correspondingly severe ranking penalty. Conversely, failing to meet the minimum cost level criterion is unlikely to receive negative user feedback, and therefore failing to meet the minimum cost level may result in a correspondingly minor ranking penalty.

[0089] Alternatively, in some implementations, in order to generate POI selection information 134, POI selector 132 may include POI inclusion criteria information 310. For example, POI selector 132 may generate POI inclusion criteria information 310, and / or may receive POI inclusion criteria information 310 from user computing device 112.

[0090] POI inclusion criteria information 310 may indicate a minimum or maximum value for a certain criterion to determine whether to include the POI in POI selection information 134. Specifically, in some implementations, POI inclusion criteria information 310 may include POI type criteria indicating the desired POI type, such as "food" (e.g., a POI that provides food in some way), "CONV" (e.g., a convenience store), "REST" (e.g., a restaurant), "PARK" (e.g., a park), etc. POI selector 132 may determine the types of POIs a user might want to search for. POI selector 132 may make this determination based on information within service information 122. For example, application information 302 may include information describing the desired POI type, while user history information 304 may indicate whether the user has historically preferred other types of POIs.

[0091] In some implementations, the POI inclusion criteria 310 may include a minimum score criterion. The minimum score criterion may indicate the minimum total score of the POI based on user-submitted reviews. The POI selector 132 may determine the minimum score that a user might prefer. The POI selector 132 may make this determination based on information within the service information 122. For example, UGC 124 may include the total score of the POI from user-submitted reviews, while user history information 304 may indicate whether the user has historically preferred POIs with a specific minimum score.

[0092] In some implementations, the POI inclusion criteria 310 may include a minimum number of comments criterion. This minimum number of comments criterion may be based on user-submitted comments, indicating the minimum number of comments required for the POI. The POI selector 132 may determine the minimum number of comments a user might prefer. The POI selector 132 may make this determination based on information within the service information 122. For example, UGC 124 may include scores from user-submitted comments for POI submissions, while user history information 304 may indicate whether a user has historically preferred POIs with a specific number of comments.

[0093] In some implementations, the POI inclusion criteria information 310 may include previous access criteria. Previous access criteria could be the number of times (or the range of accesses) a user has previously made to a POI based on their preferences. The POI selector 132 can determine the number of preferred accesses. To follow the illustrated example, the POI inclusion criteria information 310 indicates that the user prefers POIs they have visited more than once but less than ten times. The POI selector 132 can make this determination based on information within the service information 122. For example, user history information 304 could indicate the number of times a user has visited a POI, and the number of times the user has historically preferred a POI.

[0094] As stated in POI exclusion criteria information 308, it should be noted that successfully meeting one (or more) inclusion criteria does not necessarily result in a POI being included in POI selection information 134. Instead, in some implementations, inclusion criteria can be weighted and used to rank the POIs indicated by service information 122. For example, a POI that fails to meet a minimum score criterion is relatively more likely to receive negative user reviews, and therefore may be subject to a corresponding severe ranking penalty. Alternatively, in some implementations, successfully meeting one (or a number and / or one type of) inclusion criterion can automatically include a POI.

[0095] In some implementations, the POI selector 132 may rank the POIs and then select a subset of the ranked POIs to include in the POI selection information 134. To follow the depicted example, assume that the POI selector 132 arranges POI_IDs 590892, 392187, 907836, 985462, and 097632 in descending order (e.g., 1, 2, 3, 4, 5). Further assume that POI_IDs 907836 and 985462 are excluded based on POI exclusion criterion information 308. The POI_IDs selected for inclusion could be the first, second, and fifth ranked POIs.

[0096] In some implementations, the user computing device 112 may provide input information 312 to the application / service module 110. Input information 312 may indicate an input method from the user that explicitly or implicitly selects a specific geographic area, POI type, etc. The POI selector 132 may utilize input information 312 to generate an initial selection of POIs (e.g., reducing the number of selected POIs to POI selection information 134), and / or determine the area from which POIs are initially sampled for subsequent selection. For example, input information 312 may simply include a phrase or name of a specific geographic area (e.g., Zurich). As another example, input information 312 may indicate a specific route requested by the user.

[0097] In some implementations, input information 312 can be derived from query 131. For example, if a user submits a standard query, such as "Italian restaurants in Zurich," then "Zurich" can be extracted and included in input information 312. Alternatively, in some implementations, the input information can be derived from a route requested by the user. For example, if a user requests a route, input information 312 can indicate the areas the user will traverse while navigating the route.

[0098] Figure 4 This is based on some implementation methods of this disclosure. Figure 1 A block diagram of the model orchestration module 136 of the computing system 102. Figure 3 Combining Figure 1 and Figure 3 Let's discuss this together. Specifically, the model orchestration module 136 may include the query 131 described by the query information 130. The model orchestration module 136 may include a first machine learning-processed POI-specific language model 402 and a second machine learning-processed POI-specific language model 404. The second machine learning-processed POI-specific language models 402 and 404 may be two different models from the machine learning-processed POI-specific language model 138, or they may be two instances of the same machine learning-processed POI-specific language model 138.

[0099] Specifically, in some implementations, the first machine learning-based POI-specific language model 402 can be a model specifically trained or fine-tuned to generate text content from the perspective of a first POI (e.g., Sullivan's Steakhouse). For example, a model optimizer can train the model by modifying the parameter values ​​of the machine learning-based POI-specific language model to enable the model to generate text content from the perspective of the first POI. Alternatively, in some implementations, the first machine learning-based POI-specific language model 402 can be prompted to generate text content from the perspective of the first POI.

[0100] For example, model orchestration module 136 can retrieve cue information 406 (e.g., from cue store 142, etc.). Cue information 406 may include text content describing instructions that can prompt the first machine learning-based POI-specific language model 402 to generate text content from the perspective of the first POI. In some implementations, cue information 412 may be generated using a language model (e.g., LLM). Alternatively, in some implementations, cue information 412 may be generated using conventional parsing techniques (e.g., regular expressions, etc.) (e.g., replacing the placeholder name value "[POI name]" with "Sullivan Steakhouse").

[0101] In some implementations, the model orchestration module 136 can instantiate a first machine-trained POI-specific language model 402 and a second machine-trained POI-specific language model 404 in response to POI selection information 134. For example, suppose the POI selection information 134 indicates that a first POI and a second POI have been selected. The model orchestration module 136 can determine that no pre-trained model exists for either POI, and in response, can instantiate two default instances such as an LLM. Then, the model orchestration module 136 can obtain prompt information 406 to prompt the first POI-specific machine-trained language model 402. In this way, the model orchestration module 136 can dynamically instantiate and prompt pre-trained base models (e.g., LLM) to generate text content from the perspective of a specific POI without performing additional training or fine-tuning processes.

[0102] The first POI-specific language model 402, processed by machine learning, can handle query 131 and contextual input information 408. Contextual input information 408 may include information from service information 122, location information 306, etc. Following the illustrated example, contextual input information 408 may indicate the current visitor level at the POI, the average time spent at the POI, information received from the POI, keywords received from the POI, etc. Contextual input information 408 may be processed together with prompt information 406 and query 131 to obtain a first text content set 410. Text content 410 may include a response to query 131.

[0103] To provide information to the user in a conversational manner, the model orchestration module 136 can enable a second machine learning-based POI-specific language model 404 to generate a response based on query 131 and the first text content set 410. In some implementations, the model orchestration module 136 can acquire a second prompt 412 and a second contextual input 414, as described regarding prompt 406 and contextual input 408.

[0104] In some implementations, a second machine learning-based POI-specific language model 404 can generate a response that directly addresses the first text content set 410. For example, the second machine learning-based POI-specific language model 404 can process the first text content set 410, query 131, and second contextual input information 414, and use the second machine learning-based POI-specific language model 404 to obtain a second text content set 416 that directly or indirectly addresses a portion of the first text content set 410. For example, the second text content set 416 may simply imply or indicate that the response follows a previous response. Alternatively, in some implementations, a second prompt message 412 can instruct the second machine learning-based POI-specific language model 404 to acknowledge the first text content set 410 in a conversational manner.

[0105] Figure 5 An example user computing device is shown according to some implementations of the present disclosure, which displays an interface 128 for application instance 120 filled with text content set 139 to simulate a dialogue between POI groups. Figure 5 Combining Figure 1 Let's discuss this together. Specifically, to follow the illustrated example, the user computing device 112 could be a smartphone device used by the user. The user can provide a query 131. Upon receiving a query, the interface 128 can be displayed as a chat interface, and the query 131 can be displayed as a message sent from a representation image of the user. To follow the illustrated example, the user's representation image could be a "placeholder image" depicting a circle with the initials of the user's name.

[0106] Based on the scope of the user's query, POI selector 132 can initially select a geographic area as the POI, in this case, New York City. More generally, POI selector 132, used in conjunction with prompt repository 142, can select POIs of any type or manner, including geographic areas rather than discrete business or government entities. In this case, the "New York City" (NYC) POI can be used to extract additional details from the user to answer the user's query. The NYC POI can be represented by representation image 501, which is selected to indicate that the represented POI is a specific geographic area.

[0107] More specifically, the NYC POI can generate a first text content set 502, which can be provided to the user within the chat interface 128. Based on a clarification question from the NYC POI, the user can provide a response 504 in the same manner. Response 504 can specify that the user wants to eat pizza. However, the density of pizza shops is relatively high, so due to the number of possible choices for the user, the POI selector 132 is unlikely to select a specific pizza shop POI as a representative.

[0108] Instead, POI selector 132 can select a specific POI type (e.g., a pizza shop) and form a group of nearby POIs of the same type that can collectively engage with the user. This group of POIs can be represented by a representation image 505 (e.g., a slice of pizza), which visually indicates to the user that they are engaging with the group of POIs, rather than with a specific POI. The model representing the POI group can process response 504, query 131, and a first text content set 502 to generate a second text content set 506. The second text content set 506 can be further used to clarify the user's intent. Specifically, the second text content set 506 can ask the user whether they wish to receive additional information.

[0109] Alternatively, the user can provide a third response 508. The third response 508 may request a specific type of information—in this example, map mapping information. In some implementations, a machine learning-based POI-specific language model representing the POI group can be trained to generate navigation-specific information (e.g., data compatible with map mapping or navigation applications), thereby processing the third response 508, the second text content set 506, the second response 504, the first text content set 502, and the query 131 to generate an output 510 including route planning information 512. The route planning information 512 can be formatted to be compatible with application instance 120. In other words, the route planning information 512 can be received by application instance 120, and application instance 120 can implement the route described by the route planning information 512.

[0110] Alternatively, in some implementations, output 510 may include text content generated using the same model. For example, the machine learning-based POI-specific language model representing POI groups could be a multimodal model that generates both text content and navigation-specific content. Alternatively, in some implementations, the machine learning-based POI-specific language model representing POI groups may generate instructions that cause the generation of route planning information 512. For example, the machine learning-based POI-specific language model representing POI groups may generate instructions that instruct application instance 120 to generate a route from the user's current location to the desired destination.

[0111] Alternatively, in some implementations, the POI selector 132 can select another application, and application integration can be provided within the interface 128. For example, the POI selector 132 can select a food delivery application, and the interface 128 can provide a message including element 514, which, when selected, causes the user computing device 112 to display an interface associated with the food delivery application. Additionally, the application instance 120 can (e.g., via an API, etc.) provide relevant information to the delivery application (e.g., the selected pizza restaurant).

[0112] Figure 6A A block diagram of an example computing system 600, according to some implementations of this disclosure, is depicted, which simulates dialogue interaction between Points of Interest (POIs). The computing system 600 includes a user computing device 602, a server computing system 630, and a training computing system 650 communicatively coupled via a network 680.

[0113] User computing device 602 can be any type of computing device, such as, for example, a personal computing device (e.g., a laptop computer or desktop computer), a mobile computing device (e.g., a smartphone or tablet computer), a game console or controller, a wearable computing device, an embedded computing device, or any other type of computing device.

[0114] User computing device 602 includes one or more processors 612 and memory 614. The one or more processors 612 can be any suitable processing device (e.g., processor core, microprocessor, ASIC, FPGA, controller, microcontroller, etc.) and can be a single processor or multiple processors operatively connected. Memory 614 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc., and combinations thereof. Memory 614 can store data 616 and instructions 618, which are executed by processor 612 to cause user computing device 602 to perform operations.

[0115] In some implementations, the user computing device 602 may store or include one or more machine learning-based POI-specific language models 620. For example, the machine learning-based POI-specific language model 620 may be, or may otherwise include, various machine learning models, such as neural networks (e.g., deep neural networks) or other types of machine learning models, including nonlinear and / or linear models. Neural networks may include feedforward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks, or other forms of neural networks. Some example machine learning models may utilize attention mechanisms, such as self-attention. For example, some example machine learning models may include multi-head self-attention models (e.g., transformer models).

[0116] In some implementations, one or more machine learning-based POI-specific language models 620 may be received from a server computing system 630 via a network 680, stored in a user computing device memory 614, and then used or otherwise implemented by one or more processors 612. In some implementations, the user computing device 602 may implement multiple parallel instances of a single machine learning-based POI-specific language model 620.

[0117] Alternatively, one or more machine learning-based POI-specific language models 640 may be included in or otherwise stored and implemented by a server computing system 630 that communicates with the user computing device 602 according to a client-server relationship. For example, the machine learning-based POI-specific language model 640 may be implemented by the server computing system 630 as part of a web service (e.g., a navigation service, a visual search service, etc.). Thus, one or more models 620 may be stored and implemented at the user computing device 602, and / or one or more models 640 may be stored and implemented at the server computing system 630.

[0118] User computing device 602 may also include one or more user input components 622 for receiving user input. For example, user input component 622 may be a touch-sensitive component (e.g., a touch-sensitive display or touchpad) sensitive to the touch of a user input object (e.g., a finger or stylus). The touch-sensitive component may be used to implement a virtual keyboard. Other example user input components include a microphone, a traditional keyboard, or other means by which the user can provide input.

[0119] Server computing system 630 includes one or more processors 632 and memory 634. The one or more processors 632 can be any suitable processing device (e.g., processor core, microprocessor, ASIC, FPGA, controller, microcontroller, etc.) and can be a single processor or multiple processors operatively connected. Memory 634 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc., and combinations thereof. Memory 634 can store data 636 and instructions 638, which are executed by processor 632 to cause server computing system 630 to perform operations.

[0120] In some implementations, the server computing system 630 includes one or more server computing devices or is otherwise implemented using one or more server computing devices. Where the server computing system 630 includes multiple server computing devices, such server computing devices may operate according to a sequential computing architecture, a parallel computing architecture, or some combination thereof.

[0121] As described above, the server computing system 630 may store or otherwise include one or more machine learning-based POI-specific language models 640. For example, model 640 may be, or may otherwise include, various machine learning models. Example machine learning models include neural networks or other multi-layered nonlinear models. Example neural networks include feedforward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks. Some example machine learning models may utilize attention mechanisms, such as self-attention. For example, some example machine learning models may include multi-head self-attention models (e.g., transformer models).

[0122] User computing device 602 and / or server computing system 630 can train models 620 and / or 640 via interaction with training computing system 650, which is communicatively coupled via network 680. Training computing system 650 may be separate from server computing system 630 or may be part of server computing system 630.

[0123] The training computing system 650 includes one or more processors 652 and memory 654. The one or more processors 652 can be any suitable processing device (e.g., processor core, microprocessor, ASIC, FPGA, controller, microcontroller, etc.) and can be a single processor or multiple processors operatively connected. The memory 654 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc., and combinations thereof. The memory 654 can store data 656 and instructions 658, which are executed by the processor 652 to cause the training computing system 650 to perform operations. In some implementations, the training computing system 650 includes one or more server computing devices or is otherwise implemented by one or more server computing devices.

[0124] The training computation system 650 may include a model trainer 660 that uses various training or learning techniques, such as, for example, backpropagation of error, to train machine learning models 620 and / or 640 stored at the user computing device 602 and / or the server computing system 630. For example, a loss function may be backpropagated through the model to update one or more parameters of the model (e.g., based on the gradient of the loss function). Various loss functions may be used, such as mean squared error, likelihood loss, cross-entropy loss, hinge loss, and / or various other loss functions. Gradient descent techniques may be used to iteratively update the parameters across multiple training iterations.

[0125] In some implementations, backpropagation of execution errors may include truncated backpropagation through time. Model trainer 660 may perform various generalization techniques (e.g., weight decay, dropout, etc.) to improve the generalization ability of the model being trained. Specifically, model trainer 660 may train machine learning-based POI-specific language models 620 and / or 640 based on a set of training data 662.

[0126] In some implementations, training examples can be provided by the user computing device 602 if the user has given consent. Therefore, in such implementations, the model 620 provided to the user computing device 602 can be trained by the training computing system 650 on user-specific data received from the user computing device 602. In some cases, this process can be referred to as model personalization.

[0127] Model trainer 660 includes computer logic for providing the desired functionality. Model trainer 660 can be implemented using hardware, firmware, and / or software that controls a general-purpose processor. For example, in some implementations, model trainer 660 includes a program file stored on a storage device, loaded into memory, and executed by one or more processors. In other implementations, model trainer 660 includes one or more computer-executable instruction sets stored in a tangible computer-readable storage medium (such as RAM, hard disk, or optical or magnetic media).

[0128] Network 680 can be any type of communication network, such as a local area network (e.g., intranet), a wide area network (e.g., the Internet), or some combination thereof, and can include any number of wired or wireless links. Generally, communication over Network 680 can be conducted using various communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or protection schemes (e.g., VPN, Secure HTTP, SSL) over any type of wired and / or wireless connection.

[0129] In some implementations, the input to the machine learning model of this disclosure can be image data. The machine learning model can process the image data to generate output. As an example, the machine learning model can process image data to generate image recognition output (e.g., image data identification, latent embedding of image data, encoded representation of image data, hashing of image data, etc.). As another example, the machine learning model can process image data to generate image segmentation output. As another example, the machine learning model can process image data to generate image classification output. As another example, the machine learning model can process image data to generate image data modification output (e.g., image data alteration, etc.). As another example, the machine learning model can process image data to generate encoded image data output (e.g., encoded and / or compressed representation of image data, etc.). As another example, the machine learning model can process image data to generate upgraded image data output. As another example, the machine learning model can process image data to generate predictive output.

[0130] In some implementations, the input to the machine learning model of this disclosure can be text or natural language data. The machine learning model can process the text or natural language data to generate output. As an example, the machine learning model can process natural language data to generate a language-encoded output. As another example, the machine learning model can process text or natural language data to generate a latent text embedding output. As another example, the machine learning model can process text or natural language data to generate a translation output. As another example, the machine learning model can process text or natural language data to generate a classification output. As another example, the machine learning model can process text or natural language data to generate a text segmentation output. As another example, the machine learning model can process text or natural language data to generate a semantic intent output. As another example, the machine learning model can process text or natural language data to generate an upgraded text or natural language output (e.g., text or natural language data of higher quality than the input text or natural language). As another example, the machine learning model can process text or natural language data to generate a predictive output.

[0131] In some implementations, the input to the machine learning model of this disclosure can be speech data. The machine learning model can process the speech data to generate output. As an example, the machine learning model can process speech data to generate speech recognition output. As another example, the machine learning model can process speech data to generate speech translation output. As another example, the machine learning model can process speech data to generate latent embedding output. As another example, the machine learning model can process speech data to generate encoded speech output (e.g., encoded and / or compressed representations of speech data, etc.). As another example, the machine learning model can process speech data to generate upgraded speech output (e.g., speech data of higher quality than the input speech data, etc.). As another example, the machine learning model can process speech data to generate text representation output (e.g., a text representation of the input speech data, etc.). As another example, the machine learning model can process speech data to generate predictive output.

[0132] In some implementations, the input to the machine learning model of this disclosure can be statistical data. Statistical data can be, represent, or otherwise include data calculated and / or computed from another data source. The machine learning model can process the statistical data to generate output. As an example, the machine learning model can process the statistical data to generate an identification output. As another example, the machine learning model can process the statistical data to generate a prediction output. As another example, the machine learning model can process the statistical data to generate a classification output. As another example, the machine learning model can process the statistical data to generate a segmentation output. As another example, the machine learning model can process the statistical data to generate a visualization output. As another example, the machine learning model can process the statistical data to generate a diagnostic output.

[0133] In some cases, the input includes visual data, and the task is a computer vision task. In other cases, the input includes pixel data from one or more images, and the task is an image processing task. For example, an image processing task could be image classification, where the output is a set of scores, each score corresponding to a different object class and representing the probability that one or more images depict an object belonging to that object class. An image processing task could be object detection, where the image processing output identifies one or more regions in one or more images, and for each region, identifies the probability that the identified region depicts an object of interest. As another example, an image processing task could be image segmentation, where the image processing output defines the probability of each category in a predetermined set of categories for each pixel in one or more images. For example, the set of categories could be foreground and background. As another example, the set of categories could be object classes. As another example, an image processing task could be depth estimation, where the image processing output defines a corresponding depth value for each pixel in one or more images. As another example, an image processing task could be motion estimation, where the network input includes multiple images, and the image processing output defines the motion of the scene depicted at that pixel between the images in the network input for each pixel in one of the input images.

[0134] In some cases, the input includes audio data representing spoken utterance, and the task is speech recognition. The output may include text output mapped to spoken utterance. In some cases, the task involves encrypting or decrypting the input data. In some cases, the task involves microprocessor performance tasks such as branch prediction or memory address translation.

[0135] Figure 6AAn example computing system that can be used to implement this disclosure is shown. Other computing systems may also be used. For example, in some implementations, user computing device 602 may include model trainer 660 and training dataset 662. In such implementations, model 620 may be trained locally at user computing device 602 and both may be used. In some implementations of this type, user computing device 602 may implement model trainer 660 to personalize model 620 based on user-specific data.

[0136] Figure 6B A block diagram of an example computing device 609 according to an exemplary embodiment of the present disclosure is depicted, which performs optimization via a machine learning model to generate text content from the perspective of a specific Point of Interest (POI). The computing device 609 may be a user computing device or a server computing device.

[0137] The computing device 609 includes multiple applications (e.g., applications 1 to N). Each application contains its own machine learning library and machine learning model. For example, each application may include a machine learning model. Example applications include text messaging applications, email applications, dictation applications, virtual keyboard applications, browser applications, etc.

[0138] like Figure 6B As shown, each application can communicate with multiple other components of the computing device, such as one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, each application can communicate with each device component using an API (e.g., a public API). In some implementations, the API used by each application is application-specific.

[0139] Figure 6C A block diagram of an example computing device 651 according to an exemplary embodiment of the present disclosure is depicted, which simulates a dialogue interaction between Points of Interest (POIs). The computing device 651 may be a user computing device or a server computing device.

[0140] The computing device 651 includes multiple applications (e.g., applications 1 to N). Each application communicates with a central intelligence layer. Example applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, etc. In some implementations, each application may use an API (e.g., a common API across all applications) to communicate with the central intelligence layer (and the models stored therein).

[0141] The central intelligence layer comprises multiple machine learning models. For example, such as... Figure 6CAs shown, a corresponding machine learning model can be provided for each application, and the corresponding machine learning model can be managed by a central intelligent layer. In other implementations, two or more applications can share a single machine learning model. For example, in some implementations, the central intelligent layer can provide a single model for all applications. In some implementations, the central intelligent layer is included within the operating system of computing device 651 or otherwise implemented by said operating system.

[0142] The central intelligence layer can communicate with the central device data layer. The central device data layer can be a centralized data storage repository for the computing device 651. For example... Figure 6C As shown, the central device data layer can communicate with multiple other components of the computing device, such as one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, the central device data layer may use an API (e.g., a private API) to communicate with each device component.

[0143] Example Method

[0144] Figure 7 A flowchart depicts an example method for performing a dialogue interaction between a POI and a user, according to an example embodiment of this disclosure. Although Figure 7 The steps are described in a particular order for illustrative and discussion purposes, but the method of this disclosure is not limited to the order or arrangement specifically shown. The various steps of method 700 may be omitted, rearranged, combined and / or adapted in various ways without departing from the scope of this disclosure.

[0145] At 702, the computing system can acquire information indicating a specific geographic area where the user's computing device is located, wherein the specific geographic area includes multiple points of interest (POIs). In some implementations, the computing system can acquire information indicating first user input received at the user's computing device. The first input can be selected from a specific geographic area, a first POI, a second POI, etc. The first input set may include information indicating the first user input. In some implementations, the first user input indicates a query, and a first text content set describes a first response to the query.

[0146] At 704, for the first POI among the plurality of POIs, the computing system can generate a first set of text content using a first machine learning-based POI-specific language model, which is configured to generate text content from the perspective of the first POI.

[0147] In some implementations, the information indicating a specific geographic area further includes information indicating the current location of the user's computing device. Before generating the first set of text content, the computing system may select a first POI from a plurality of POIs based at least in part on the distance between the user's current location and the location of the POI.

[0148] In some implementations, the first machine-learned POI-specific language model may be, or otherwise include, a first instance of a large base model (LFM) that has been at least partially trained to perform multiple types of language tasks. Before generating the first set of text content, the computational system may apply a tuning process to the first instance of the LFM, wherein the tuning process is configured to cause the first instance of the LFM to generate text content from the perspective of the first POI.

[0149] In some implementations, generating the first set of text content includes retrieving POI-specific contextual information indicating one or more characteristics of the first POI. In some implementations, one or more characteristics of the first POI may include one or more operating hours of the POI. In some implementations, one or more characteristics of the first POI may include user-generated content submitted for that POI. In some implementations, one or more characteristics of the first POI may include regional differences in the services or goods offered by the POI. For example, one or more characteristics of the first POI may indicate that the POI is located in a geographic region using the phrase "soda," while characteristics of the second POI may indicate that the POI is located in a geographic region using the phrase "pop." Other regional characteristics may be language-related (e.g., language patterns, proverbs, phrases, etc.), service-related (e.g., preferred types of goods or services within a region), or geographic or weather-related (e.g., typical climate of the region, predicted weather patterns for the region, etc.). In some implementations, one or more characteristics of the first POI may include the location of the POI. In some implementations, one or more characteristics of the first POI may include a semantic description of the POI.

[0150] The computing system can process a first input set using a first machine learning-based POI-specific language model to obtain a first set of text content. The first input set may include first POI-specific contextual information. In some implementations, the first input set may include prompts that describe instructions for generating text content from the perspective of the first POI.

[0151] In some implementations, generating a second text content set using a second machine learning-based POI-specific language model includes: a computing system processing a first user input and a first text content set using the second machine learning-based POI-specific language model to obtain a second text content set. The second text content set may describe a second response corresponding to one or more of a query or a first response. In some implementations, the computing system may process the second text content set using the first machine learning-based POI-specific language model to obtain a third text content set. The third text content set may describe a response to the second text content set. The computing system may provide at least some of the second text content set to a user computing device.

[0152] In some implementations, before generating the second set of text content, the computing system can select a second POI from a plurality of POIs based on one or more selection criteria. In some implementations, one or more selection criteria may include the location of the second POI, the operating hours of the second POI, the type of POI associated with the second POI, user-generated content submitted for the second POI, historical user POI preferences, etc.

[0153] In some implementations, the computing system may provide at least some of the first text content set to the user computing device. For this purpose, the computing system may acquire a first representative POI image representing the first POI. The computing system may provide first display information to the user computing device. The first display information may include at least some of the first text content set, the first representative POI image, an instruction to represent the first POI with the first representative POI image within a chat interface of an application executed by the user computing device, an instruction to display at least some of the first text content set as messages sent by the first POI within the application's chat interface, etc. In some implementations, the computing system may use a machine learning-based generative image model to process user-generated content submitted for the POI to obtain the first representative POI image.

[0154] In some implementations, the computing system may receive information from the user's computing device instructing user input in response to a first set of text content. The computing system may process the second input set using a second machine learning-based POI-specific language model to obtain the second set of text content, thereby generating the second set of text content using the second machine learning-based POI-specific language model. The second input set may include one or more of the following: at least some of the first set of text content; and user input. In some implementations, user input may include text content entered through a chat interface of an application executed by the user's computing device. The text content entered through the chat interface may be displayed as a message sent by the user within the application's chat interface. In some implementations, user input may include visual content and / or audio content.

[0155] In 706, for a second POI among multiple POIs, the computing system can generate a second set of text content using a second machine learning-based POI-specific language model, which is configured to generate text content from the perspective of the second POI. In some implementations, the computing system can use the second machine learning-based POI-specific language model to generate a second set of text content for a subset of POIs from multiple POIs. This subset of POIs may include a second POI and a third POI. The second POI and the third POI may share a specific POI type among multiple POI types. The second machine learning-based POI-specific language model can be configured to generate content from the perspective of a set of POIs sharing the specific POI type. In some implementations, the instruction to represent the second POI with a second representative POI image may include: an instruction to represent the subset of POIs with an avatar image based on the second representative POI image within a chat interface of an application executed by a user computing device.

[0156] In some implementations, the computing system can use a second POI-specific language model derived through machine learning to generate a second set of text content. Specifically, for a third POI among multiple POIs, the computing system can use a third POI-specific language model derived through machine learning to generate additional text content, which is configured to generate text content from the perspective of the third POI. The computing system can then append the additional text content to the second set of text content.

[0157] At 708, the computing system may provide at least some of the second text content set to the user computing device. In some implementations, providing at least some of the second text content set to the user computing device may include acquiring a second representative POI image representing the second POI. The computing system may provide second display information to the user computing device. The second display information may include at least some of the second text content set, the second representative POI image, instructions to represent the second POI with the second representative POI image in a chat interface of an application executed by the user computing device, instructions to display at least some of the second text content set as messages sent by the second POI in a chat interface of a navigation application, etc.

[0158] In some implementations, providing at least some of the second text content set to the user computing device may include providing navigation information to the user computing device. The navigation information may include route planning information indicating a route from the user computing device's current location to the second POI.

[0159] Figure 8 A flowchart depicts an example method for performing a dialogue interaction between a POI and a user, according to an example embodiment of this disclosure. Although Figure 8 The steps are described in a particular order for illustrative and discussion purposes, but the method of this disclosure is not limited to the order or arrangement specifically shown. The various steps of method 800 may be omitted, rearranged, combined and / or adapted in various ways without departing from the scope of this disclosure.

[0160] In 802, the user computing device can receive user input through an application executed by the user computing device. The user input can indicate a specific geographic area. The specific geographic area can include multiple Points of Interest (POIs).

[0161] In step 804, the user computing device may send information indicating a specific geographic area to a computing system associated with the application. In some implementations, sending the information indicating the specific geographic area to the computing system associated with the application may include: determining the current location of the user computing device, and sending the information indicating the current location of the user computing device to the computing system.

[0162] In some implementations, the application executed by the user's computing device may be, or otherwise include, a navigation application. Sending information indicating a specific geographic area to the computing system associated with the application may include receiving user selection input for that specific geographic area via the navigation application.

[0163] At 806, in response to sending information, the user computing device can acquire a set of text content generated using a machine learning-based POI-specific language model, which is optimized to generate text content from the perspective of POIs among a plurality of POIs. In some implementations, the user computing device can receive from a computing system a set of text content generated by the computing system using a machine learning-based POI-specific language model configured to generate text content from the perspective of POIs among a plurality of POIs.

[0164] In some implementations, to obtain a set of text content, the user computing device may use a specific machine learning model to generate a first set of text content, which is optimized to generate text content from the perspective of POIs among a plurality of POIs. The specific machine learning model may be, or otherwise include, a machine learning-based POI-specific language model, a machine learning-based POI-specific distillation model trained by distillation from a machine learning-based POI-specific language model, etc.

[0165] At 808, the user computing device can display messages, including at least some of a set of text content, within a chat interface of an application executed by the user computing device. The chat interface can be a representation of POIs (Points of Interest) for chat participants within the chat interface. The POI representation can include a representative POI image that visually indicates the POI. Messages can be depicted within the chat interface as being sent by a POI representation.

[0166] In some implementations, the user computing device may receive a second set of text content generated using a second machine learning-based POI-specific language model, which is optimized to generate text content from the perspective of a second POI among a plurality of POIs. The user computing device may display a second message comprising at least some of the second text content set within a chat interface of an application executed by the user computing device. The chat interface may include a second POI representation of a chat participant within the chat interface, representing the second POI. The second POI representation may include a second representative POI image visually indicating the second POI. The second message may be depicted within the chat interface as being sent by the second POI representation.

[0167] In some implementations, before receiving the second text content set, the computing system may receive second user input via a chat interface of an application executed by the user's computing device. In some implementations, the second user input may include text content instructing a second Point of Interest (POI). In some implementations, at least some of the text content set describes goods or services offered by the first POI. The second user input may include text content describing a request for goods or services offered by a POI other than the first POI. At least some of the second text content set may describe goods or services offered by the second POI.

[0168] Implementation

[0169] The following provides various implementation methods for the above:

[0170] Implementation method 1: A user computing device, comprising:

[0171] - One or more processor devices; and

[0172] - One or more tangible non-transitory computer-readable media storing computer-readable instructions that, when executed by the one or more processor devices, cause the one or more processor devices to perform operations, including:

[0173] ○ Receive user input via an application executed by the user computing device, wherein the user input indicates a specific geographic area, and wherein the specific geographic area includes multiple points of interest (POIs);

[0174] ○ Send information indicating the specific geographic area to the computing system associated with the application;

[0175] ○ In response to sending this information, obtain a set of text content generated using a machine learning-optimized POI-specific language model, which is optimized to generate text content from the perspective of POIs among the plurality of POIs; and

[0176] ○ Displaying messages comprising at least some of the text content set within the chat interface of the application executed by the user's computing device, wherein the chat interface includes a POI representation of the POI as a chat participant within the chat interface, wherein the POI representation includes a representative POI image that visually indicates the POI, and wherein the message is depicted within the chat interface as being sent by the POI representation.

[0177] Implementation 2: The user computing device as described in implementation 24, wherein the operation further includes:

[0178] - Receive from the computing system a second set of text content generated using a second machine learning-optimized POI-specific language model, the second machine learning-optimized POI-specific language model being optimized to generate text content from the perspective of a second POI among the plurality of POIs; and

[0179] - Displaying a second message comprising at least some of the second text content set within the chat interface of the application executed by the user's computing device, wherein the chat interface includes a second POI representation of the second POI as a chat participant within the chat interface, wherein the second POI representation includes a second representative POI image that visually indicates the second POI, and wherein the second message is depicted within the chat interface as being sent by the second POI representation.

[0180] Implementation method 3: The user computing device as described in implementation method 2, wherein at least some of the second text content set responds to at least some of the text content set.

[0181] Implementation 4: The user computing device as described in Implementation 2, wherein before receiving the second text content set, the operation includes:

[0182] - Receive second user input through the chat interface of the application executed by the user's computing device.

[0183] Implementation method 5: The user computing device as described in implementation method 4, wherein the second user input includes text content indicating the second POI.

[0184] Implementation method 6: The user computing device as described in implementation method 4, wherein:

[0185] - At least some of the descriptions in the text content set are of goods or services provided by the POI;

[0186] - Wherein the second user input includes text content describing a request for goods or services offered by POIs other than the POI itself; and

[0187] - Wherein at least some of the descriptions in the second text content set are goods or services provided by the second POI.

[0188] Implementation method 7: The user computing device as described in implementation method 1, wherein obtaining the text content set includes:

[0189] - Receive from the computing system the set of text content generated by the computing system using a machine learning-optimized POI-specific language model, which is optimized to generate text content from the perspective of POIs among the plurality of POIs.

[0190] Implementation method 8: The user computing device as described in implementation method 1, wherein obtaining the text content set includes:

[0191] - The text content set is generated using a specific machine learning model configured to generate text content from that perspective of the POIs among the plurality of POIs, wherein the specific machine learning model includes:

[0192] ○ The POI-specific language model processed by machine learning; or

[0193] ○ A machine learning-based POI-specific distillation model trained by distillation from a machine learning-based POI-specific language model.

[0194] Implementation 9: The user computing device as described in Implementation 1, wherein sending the information indicating the specific geographic area to the computing system associated with the application includes:

[0195] - Determine the current location of the user's computing device; and

[0196] - Send information to the computing system indicating the current location of the user's computing device.

[0197] Implementation method 10: The computing system as described in implementation method 1, wherein:

[0198] - The application executed by the user's computing device includes a navigation application; and the information indicating the specific geographic area sent to the computing system associated with the application includes:

[0199] ○ This navigation application receives user selection input for a specific geographic area.

[0200] Additional Public Content

[0201] This paper discusses technologies related to servers, databases, software applications, and other computer-based systems, as well as the actions taken and the information sent to and from such systems. The inherent flexibility of computer-based systems allows for a wide variety of possible configurations, combinations, and partitions of tasks and functionality between and within components. For example, the processes discussed in this paper can be implemented using a single device or component, or multiple devices or components working in combination. Databases and applications can be implemented on a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0202] While the subject matter has been described in detail with respect to various specific exemplary embodiments, each example is provided by way of explanation and not limitation. Modifications, alterations, and equivalents of such embodiments will readily arise for those skilled in the art upon acquiring an understanding of the foregoing. Therefore, this disclosure does not exclude such modifications, alterations, and / or additions to the subject matter that will be readily understood by those of ordinary skill in the art. For example, features shown or described as part of an embodiment may be used with another embodiment to produce yet another embodiment. Therefore, this disclosure is intended to cover such modifications, alterations, and equivalents.

Claims

1. A computer-implemented method, comprising: A computing system including one or more processor devices acquires information indicating a specific geographic area where a user's computing device is located, wherein the specific geographic area includes multiple points of interest (POIs); For the first POI among the plurality of POIs: The computing system generates a first set of text content using a first machine learning-based POI-specific language model, wherein the first machine learning-based POI-specific language model is configured to generate text content from the perspective of the first POI. For the second POI among the plurality of POIs: The computing system generates a second set of text content using a second machine learning-based POI-specific language model, wherein the second machine learning-based POI-specific language model is configured to generate text content from the perspective of the second POI. as well as The computing system provides at least some of the second text content set to the user computing device.

2. The computer-implemented method of claim 1, wherein generating the first text content set using the first machine learning-processed POI-specific language model comprises: The computing system retrieves first POI-specific contextual information indicating one or more characteristics of the first POI; as well as The computing system processes the first input set using the first machine learning-based POI-specific language model to obtain the first text content set, wherein the first input set includes the first POI-specific context information.

3. The computer-implemented method of claim 2, wherein the one or more characteristics of the first POI include one or more of the following: The operating hours of the POI; User-generated content submitted for the POI; Regional differences in the services or goods provided by the POI; The location of the POI; or The text description of the POI.

4. The computer-implemented method of claim 2, wherein the first input set includes prompts, and wherein the prompts describe instructions for generating text content from the perspective of the first POI.

5. The computer-implemented method of claim 2, wherein obtaining the information indicating the specific geographic area comprises: The computing system acquires information indicating first user input received at the user computing device, wherein the first user input selects at least one of the following: The specific geographical area; The first POI; or The second POI; and The first input set further includes the information indicating input from the first user.

6. The computer-implemented method as described in claim 5, wherein: The first user input indicates a query; and The first text content set describes the first response to the query.

7. The computer-implemented method of claim 6, wherein generating the second text content set using the second machine learning-processed POI-specific language model comprises: The computing system processes the first user input and the first text content set using the second machine learning-based POI-specific language model to obtain the second text content set, wherein the second text content set describes a second response to one or more of the query or the first response.

8. The computer-implemented method of claim 7, wherein the method further comprises: The computing system processes the second text content set using the first machine learning-based POI-specific language model to obtain a third text content set, wherein the third text content set describes the response to the second text content set; as well as The computing system provides at least some of the second text content set to the user computing device.

9. The computer-implemented method of claim 8, wherein generating the first text content set using the first machine learning-processed POI-specific language further comprises: The computing system provides at least some of the first text content set to the user computing device; The computing system acquires a first representative POI image representing the first POI; as well as The computing system provides first display information to the user computing device, wherein the first display information includes: At least some of the above in the first text content set; The first representative POI image; Instructions that represent the first POI using the first representative POI image within the chat interface of an application executed by the user computing device; and In the chat interface of the application, an instruction is given to display at least some of the first text content set as messages sent by the first POI.

10. The computer-implemented method of claim 9, wherein providing the first display information to the user computing device further comprises: The computing system receives information from the user computing device indicating user input in response to the first text content set; and The generation of the second text content set using the second machine learning-based POI-specific language model includes: The computing system processes the second input set using the second machine learning-processed POI-specific language model to obtain the second text content set, wherein the second input set includes one or more of the following: At least some of the above in the first text content set; and The user input.

11. The computer-implemented method of claim 9, wherein providing the user computing device with the at least some of the second text content set comprises: The computing system acquires a second representative POI image representing the second POI; as well as The computing system provides second display information to the user computing device, wherein the second display information includes: At least some of the text content in the second text content set; The second representative POI image; Instructions to represent the second POI with the second representative POI image within the chat interface of the application executed by the user computing device; and An instruction to display at least some of the second text content set as messages sent by the second POI within the chat interface of the application executed by the user computing device.

12. The computer-implemented method of claim 11, wherein generating the second text content set for the second POI comprises: The computing system uses the second machine learning-based POI-specific language model to generate the second text content set for a subset of POIs from the plurality of POIs, wherein the subset of POIs includes the second POI and the third POI, wherein the second POI and the third POI share a specific POI type among the plurality of POI types, and wherein the second machine learning-based POI-specific language model is configured to generate content from the perspective of a set of POIs sharing the specific POI type.

13. The computer-implemented method of claim 12, wherein the instruction to represent the second POI with the second representative POI image comprises: Instructions to represent the subset of POIs with avatar images based on the second representative POI image within the chat interface of the application executed by the user computing device.

14. The computer-implemented method of claim 9, wherein acquiring the first representative POI image representing the first POI comprises: The computing system processes user-generated content submitted for the POI using a machine learning-based generative image model to obtain the first representative POI image.

15. The computer-implemented method of claim 1, wherein generating the second text content set using the second machine learning-based POI-specific language model further comprises: For the third POI among the plurality of POIs, the computing system generates additional text content using a third machine learning-based POI-specific language model, which is configured to generate text content from the perspective of the third POI. as well as The computing system appends the additional text content to the second text content set.

16. The computer-implemented method of claim 1, wherein the information indicating the specific geographic area further includes information indicating the current location of the user computing device; Before generating the first text content set, the method includes: The computing system selects the first POI from the plurality of POIs based at least in part on the distance between the current location of the user's computing device and the location of the POI; and Before generating the second text content set, the method includes: The computing system selects the second POI from the plurality of POIs based on one or more selection criteria, wherein the one or more selection criteria include at least one of the following: The location of the second POI; The operating hours of the second POI; The POI type associated with the second POI; User-generated content submitted for the second POI; or Historical user POI preferences.

17. The computer-implemented method of claim 1, wherein the first machine learning-based POI-specific language model comprises a first instance of a large base model LFM, the LFM being at least partially trained to perform multiple types of language tasks, and wherein the method comprises, prior to generating the first set of text content: The computing system applies a tuning process to the first instance of the LFM, wherein the tuning process is configured to cause the first instance of the LFM to generate text content from the angle of the first POI.

18. The computer-implemented method of claim 1, wherein providing the user computing device with the at least some of the second text content set comprises: The computing system provides navigation information to the user computing device, wherein the navigation information includes: Route planning information indicating the route from the current location of the user's computing device to the second POI.

19. One or more tangible non-transitory computer-readable media storing computer-readable instructions that, when executed by one or more processor devices, cause the one or more processor devices to perform operations, the operations including: Information indicating a specific geographic region is obtained from a user's computing device, wherein the specific geographic region includes multiple points of interest (POIs); For the first POI among the plurality of POIs: A first set of text content is generated using a first machine learning POI-specific language model, wherein the first machine learning POI-specific language model is configured to generate text content from the perspective of the first POI. For the second POI among the plurality of POIs: A second set of text content is generated using a second machine learning-based POI-specific language model, which is configured to generate text content from the perspective of the second POI. as well as Provide at least some of the second text content set to the user computing device.

20. A user computing device, comprising: One or more processor devices; as well as One or more tangible non-transitory computer-readable media storing computer-readable instructions that, when executed by the one or more processor devices, cause the one or more processor devices to perform operations, the operations including: User input is received by an application executed by the user computing device, wherein the user input indicates a specific geographic area, and wherein the specific geographic area includes multiple points of interest (POIs); Send information indicating the specific geographic area to the computing system associated with the application; In response to sending the information, a set of text content generated using a machine learning-based POI-specific language model is obtained, the machine learning-based POI-specific language model being optimized to generate text content from the perspective of POIs among the plurality of POIs; as well as The message includes at least some of the text content set displayed within the chat interface of the application executed by the user computing device, wherein the chat interface includes a POI representation of the POI as a chat participant within the chat interface, wherein the POI representation includes a representative POI image that visually indicates the POI, and wherein the message is depicted within the chat interface as being sent by the POI representation.