User assistance system based on instructive generative thread

Abstract

Embodiments of the disclosed techniques include generating a first thread classification cue based on a first thread portion of an online conversation involving a user of a computing device; sending the first thread classification prompt word to a first large language model; receiving a first thread classification generated and output by the first large language model based on the first thread classification prompt word; formulating a plan execution cue word based on the first thread classification; sending the plan execution cue word to a second large language model; receiving a second thread part generated and output by the second large language model based on the plan execution cue word and the online dialogue; and generating a tag for a third thread portion of the online conversation.

Description

Technical Field

[0001] The technical fields covered by this disclosure include computer programs that use artificial intelligence to understand user requests for assistance and automate responses to those requests in a manner that simulates human conversation. Another technical field covered by this disclosure is generative artificial intelligence.

[0002] Copyright Notice This patent document, including the accompanying drawings, contains copyrighted material. The copyright holder does not object to the facsimile reproduction of this patent document, as it appears in the publicly accessible records of the United States Patent and Trademark Office, consistent with the fair use principle of U.S. copyright law, but otherwise reserves all copyright rights. Background Technology

[0003] A search engine is a software system designed to find and retrieve stored information that matches a search query. A chatbot (or chatbot) is a software application that can retrieve information and answer questions by simulating natural language conversations with human users. Attached Figure Description

[0004] This disclosure will be more fully understood from the detailed description given below and from the accompanying drawings, which illustrate various embodiments of this disclosure. The drawings are for explanation and understanding only and should not be construed as limiting this disclosure to the specific embodiments shown.

[0005] Figure 1A This is a flowchart of an example method for guided generative thread-based user assistance using components of a user assistance system based on guided generative threads, according to some embodiments of this disclosure.

[0006] Figure 1B This is a flowchart of an example method for generating thread classification prompts using components of a user assistance system based on guided generative threads, according to some embodiments of this disclosure.

[0007] Figure 1C This is a flowchart of an example method for generating planned execution prompts using components of a user-assisted system based on a guided generative thread, according to some embodiments of the present disclosure.

[0008] Figure 1D This is a block diagram of an example architecture for a computing system according to some embodiments of the present disclosure.

[0009] Figure 2A This is a timing diagram illustrating an example of communication between components of a thread-based user assistance interface and a user assistance system based on a guided generative thread, according to some embodiments of the present disclosure.

[0010] Figure 2BThis is a timing diagram illustrating an example of using multiple threads to generate scheduled execution prompts according to some embodiments of the present disclosure.

[0011] Figure 2C This is a flowchart illustrating examples of contextual content generated by a generative model according to some embodiments of the present disclosure.

[0012] Figure 3A , Figure 3B , Figure 3C , Figure 3D , Figure 3E , Figure 3F , Figure 3G , Figure 3H , Figure 3I , Figure 3J , Figure 3K , Figure 3L , Figure 3M , Figure 3N , Figure 3O , Figure 3P , Figure 3Q , Figure 3R , Figure 3S , Figure 3T , Figure 3U and Figure 3V An example of a screen capture comprising at least one stream configured to provide a user interface screen based on a guided generative thread, according to some embodiments of the present invention, is illustrated.

[0013] Figure 4A and Figure 4B The illustration shows an example of at least one stream of screen capture, which includes a user interface screen configured to provide user assistance based on a guided generative thread, according to some embodiments of the present disclosure.

[0014] Figure 5 This is a block diagram of a computing system including a user-assisted system based on a guided generative thread, according to some embodiments of the present disclosure.

[0015] Figure 6 These are examples of entity diagrams based on some embodiments of this disclosure.

[0016] Figure 7 This is a flowchart of an example method for using a guided generative thread-based user assistance component in a system, according to some embodiments of the present disclosure.

[0017] Figure 8 This is a block diagram of an example computer system including components of a user-assisted system based on a guided generative thread, according to some embodiments of the present disclosure.

[0018] Specific implementation method People often turn to their computing devices when they need answers to questions or other types of assistance, such as help with job searches. Conventional search engines require users to explicitly provide or select search terms that identify the type of information they are looking for. While search engines are ubiquitous, designing them to accurately interpret user queries on a personalized basis remains an ongoing challenge because each user has a unique communication style and expresses themselves differently in different contexts. Using search engines, users often need to rewrite or restate their queries multiple times through trial and error to ultimately obtain the desired set of search results.

[0019] Conventional chatbots operate in a manner similar to search engines, but in a way that mimics human conversation. Therefore, chatbots replace multiple iterations of search queries with multi-turn conversational dialogues with human users.

[0020] Regular chatbots work best when users have a clear idea of ​​the kind of information or assistance they are seeking (e.g., a well-defined goal or intent) and the chatbot has been designed to handle that particular type of query. For example, chatbots have been used to provide technical support, where users can tell the chatbot exactly what is wrong with their device or software. However, regular chatbots are often frustrating for users whose intent is less clear, as they may still not achieve their goal even after multiple rounds of conversation with the chatbot. In other scenarios, the type of assistance a user is seeking may lead to lengthy, multi-threaded conversations that regular chatbots cannot handle. An example illustrating the application of the disclosed technology is job search. Prior to the disclosed technology, chat-style software had not been widely and successfully implemented to assist users in finding jobs that match their specific preferences and abilities.

[0021] Generative models use artificial intelligence techniques (e.g., neural networks) to machine-generate new digital content based on model inputs and previously existing data that the model has used for training. Discriminative models, however, are based on conditional probabilities P(…). y | x That is, given input x (For example, is this a picture of a dog?) Output in the case of y The probability of the generative model capturing the joint probability P( x, y ),that is, x and y The probability of them happening together (e.g., given the photo of a dog and an unknown person, is it possible that the person is Sam, the dog's owner?).

[0022] Generative language models are a specific type of generative model that generates new text in response to model input. Model input includes a task description, also known as cue words. The task description can include examples of instructions and / or digital content. The task description can be in the form of natural language text, such as questions or statements, and can also include non-textual content, such as digital images and / or digital audio.

[0023] Given a task description, a generative model can generate a set of task description-output pairs, where each pair contains a distinct output. In some implementations, the generative model assigns a score to each of the generated task description-output pairs. The outputs in a given task description-output pair contain text generated by the model itself, rather than text provided to the model as input. The score associated with a given task description-output pair represents the probabilistic or statistical likelihood of a relationship between the output in that pair and the corresponding task description. The score for a given task description-output pair depends on how the generative model has been trained and the data used to perform the model training. The generative model can sort the task description-output pairs by score and output only one or more pairs with the highest scores. For example, the generative model can discard lower-scoring pairs and output only the highest-scoring pairs as its final output.

[0024] Large Language Models (LLMs) are a type of generative language model that uses deep learning techniques to be trained in an unsupervised manner on large amounts of unlabeled data, such as publicly available text extracted from the Internet. LLMs can be configured to perform one or more Natural Language Processing (NLP) tasks, such as generating text, classifying text, answering questions in a conversational manner, and translating text from one language to another.

[0025] Large Language Models (LLMs) are capable of answering questions in a conversational manner. Having been trained on vast amounts of data, LLMs are also able to operate on a wide range of topics in online conversations. Therefore, LLMs have the potential to improve the performance of chatbots. However, LLMs suffer from the technical problem of hallucination. In artificial intelligence, hallucination is generally defined as generated content that is meaningless or unfaithful to the provided source content. Because chatbots typically involve long or multi-threaded conversations, the risk of AI hallucination increases with each round or thread of conversation provided to the LLM. For example, the risk of AI hallucination may increase when a user switches between multiple different topics within the same conversational session or returns to a topic from an earlier thread. As a result, AI hallucination is an obstacle to using LLMs in chatbots.

[0026] As a result of these and other issues, the technical challenge is to incorporate LLM into chatbot-style user assistance systems while mitigating the risks of AI illusion.

[0027] Another technical challenge is how to machine-generate digital images, videos, and / or audio, and incorporate these into user assistance. Yet another challenge is how to reduce the burden of user input when processing and responding to requests for user assistance. Yet another challenge is how to scale session-based user assistance systems to a large number of users (e.g., hundreds of thousands to millions or more) without linearly increasing the size of the user assistance system. Further technical challenges include how to efficiently generate and distribute user assistance across a wide variety of user devices, such as adapting user assistance to different screen sizes, different device types, etc. A further technical challenge is how to respond to latency issues while providing session-based user assistance, for example, how to respond when an increase or decrease in latency is detected.

[0028] To address these and other technical challenges of conventional user assistance systems, the disclosed techniques provide a user assistance system based on guided generative threads. The disclosed techniques are thread-based because they are designed to enable verbose and / or multi-threaded dialogue between the user and the user assistance system, resulting in improved efficiency, scalability, and reduced latency. The disclosed techniques are generative because one or more generative models (e.g., LLM) are used to machine-generate and output responses to user requests in a conversational natural language manner.

[0029] As described in more detail below, the disclosed techniques are directive because the prompts provided as input to one or more LLMs are configured to constrain the operation of one or more LLMs to a well-defined set of input parameters in order to avoid AI illusions. For example, if the most recent turn of user input in a lengthy conversation refers to “third position,” an AI illusion could occur in a conventional scenario without the disclosed techniques because the LLMs might not have the context to determine which position is the “third” position. However, using the disclosed techniques, the ambiguity of “third position” is eliminated before the prompts are submitted to the LLMs to avoid AI illusions.

[0030] In some implementations, the disclosed techniques utilize one or more contextual sources, such as entity graphs, graph-based networks, recommender systems, domain applications, and / or external data sources, to identify parameters to be used to constrain the operations of one or more LLMs. For example, if a user-assistance system has identified five job positions that match a user's criteria, the system can generate prompts that include instructions to: search the entity graph to determine companies associated with the five positions; search the user connection graph to determine if the user has any connections to any of those companies; rank job positions at companies the user has connections to above job positions at companies the user has no connections to; and restate the user-assistance system's natural language output to refer the user's connections as potential referral sources. In this and other examples, the disclosed techniques overcome the technical problem of AI illusion in the context of dialogue-based, generative, thread-based user-assistance systems.

[0031] Job search is one example of how the disclosed technologies can be used to enhance user assistance. For instance, the disclosed technologies can be configured as job search or career development assistants, helping users improve their online job searches and manage various online tasks required for job seeking. For example, a user-assistance system configured with the disclosed technologies can: automatically generate job recommendations based on the user's goals, skills, experience, and preferences; automatically generate comparative insights across multiple jobs based on the user's preferences; automatically generate suggestions for new skills for the user to develop to advance their career; automatically create personalized resumes and cover letters based on the specific job the user is applying for; and automatically generate cue words for the user's upcoming interviews. Other example use cases include education, learning, and other domain-specific applications, as well as more general or domain-independent user-assistance environments.

[0032] Specific aspects of the disclosed techniques are described within the context of generative models that output written fragments (i.e., natural language text). However, the disclosed techniques are not limited to generative models that produce text output. For example, aspects of the disclosed techniques can be used to generate user-assisted outputs including non-textual machine-generated outputs such as digital images, videos, multimedia, audio, hyperlinks, and / or platform-independent file formats.

[0033] The specific aspects of the disclosed technology are described in the context of electronic dialogue conducted via a network, user connection network, or application software system (such as instant messaging services, chatbots, or social networking services). However, the aspects of the disclosed technology are not limited to such a context and can be used to improve user-assisted machine generation using other types of software applications. Any network-based application software system can be used as an application software system capable of applying the disclosed technology. For example, news, entertainment, and e-commerce applications installed on mobile devices, enterprise systems, messaging systems, search engines, job flow management systems, collaboration tools, and social graph-based applications can all be used as application software systems capable of using the disclosed technology.

[0034] This disclosure will be more fully understood from the following detailed description given with reference to the accompanying drawings. The detailed description of the drawings is for explanation and understanding purposes only and should not be construed as limiting this disclosure to the specific embodiments described.

[0035] In the accompanying drawings and the following description, reference may be made to components that have the same name but different reference numerals in different drawings. The use of different reference numerals in different drawings indicates that components with the same name may represent the same embodiment or different embodiments of the same component. For example, in some embodiments, components with the same name but different reference numerals in different drawings can have the same or similar functions, such that the description of one component with respect to one drawing can be applied to other components with the same name in other drawings.

[0036] Similarly, the components shown and described in conjunction with some embodiments in the accompanying drawings and the following description can be used with or incorporated into other embodiments. For example, components illustrated in a particular drawing are not limited to use with respect to the embodiments described in conjunction with the drawings, but can be used with or incorporated into other embodiments, including those shown in other drawings.

[0037] As used herein, a dialogue or session can refer to one or more digital threads involving a user and a user-assisted system of a computing device. For example, a dialogue or session can have associated user identifiers, session identifiers, session or dialogue identifiers, and timestamps. A thread, as used herein, can refer to one or more rounds of dialogue involving a user and a user-assisted system. A dialogue round, as used herein, can refer to user input and associated system-generated responses, such as system-generated replies to user input. For example, a thread can include a first thread portion and a second thread portion, the first thread portion being, for example, a question received from a user of the computing device, and the second thread portion being, for example, machine-generated natural language text, audio, video, and / or images generated by a user-assisted system in response to the user's question.

[0038] A thread can have an associated thread identifier. A thread can consist of non-contiguous thread segments. For example, a thread can include thread segments related to a common topic, even if these thread segments are temporally separated by other threads or thread segments. Any dialogue, thread, or thread segment can include one or more different types of digital content, including natural language text, audio, video, digital images, hyperlinks, and / or multimodal content such as web pages. A thread segment can have an associated source identifier (e.g., user or system) and a timestamp that identifies the source of the thread segment.

[0039] Figure 1 is a flowchart of an example method for user assistance based on guided generative threads for using components of a computing system according to some embodiments of the present disclosure, the computing system including a guided generative thread-based user assistance system, a thread-based user assistance interface, one or more entity graphs, and one or more data sources.

[0040] The method is executed by processing logic, which includes hardware (e.g., processing device, circuitry, dedicated logic, programmable logic, microcode, device hardware, integrated circuits, etc.), software (e.g., instructions that run or execute on the processing device), or a combination thereof. In some embodiments, the method is executed by a component of a user assistance system 102 based on guided generative threads. In some embodiments, the user assistance system 102 based on guided generative threads includes... Figure 1A The one shown may not be present Figure 5 The components specifically shown in the text, or those made by Figure 5 The user assistance system 580, based on a guided generative thread, is executed by components thereof. In some embodiments, the guided generative thread-based user assistance system 580 includes... Figure 5 The one shown may not be present Figure 1AThe components specifically shown, or those executed by components of computing system 140, in some embodiments, computing system 140 includes Figure 1B The one shown may not be present Figure 1A or Figure 5 The components specifically shown, or those executed by components of computing system 170, in some embodiments, computing system 170 includes Figure 1C The one shown may not be present Figure 1A , Figure 1B or Figure 5 The components specifically shown, or those executed by components of computing system 194, in some embodiments, computing system 194 includes Figure 1D The one shown may not be present Figure 1A , Figure 1B , Figure 1C or Figure 5 The components are specifically illustrated. Although shown in a particular sequence or order, the order of processes can be modified unless otherwise stated. Therefore, the illustrated embodiments should be understood as examples only, and the illustrated processes can be performed in different orders, and some processes can be performed in parallel. In addition, at least one process can be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are also possible.

[0041] exist Figure 1A The example illustrates an exemplary computing system 100, which includes an exemplary guided generative thread-based user assistance system 102 and an exemplary thread-based user assistance interface 118. The guided generative thread-based user assistance system 102 of Figure 1 includes a thread classification prompt generator 104, a first large language model 108, a thread label generator 109, a plan execution prompt generator 112, and a second large language model 116, as described in more detail below.

[0042] exist Figure 1A In the example, components of the guided generative thread-based user assistance system 102 are implemented using an application server or server cluster, which can include a secure environment (e.g., a secure enclave, encryption system, etc.) for processing thread data. In other implementations, one or more components of the guided generative thread-based user assistance system 102 are implemented using, for example, those referenced herein. Figure 5 The described user system 510 is implemented on a client device. For example, in some implementations, some or all of the user assistance system 102 based on guided generative threads is implemented directly on the user's client device, thereby avoiding the need to communicate with a server via a network such as the Internet.

[0043] In some implementations, the guided generative thread-based user assistance system 102 communicates bidirectionally with a thread-based user assistance interface 118 via a computer network. The thread-based user assistance interface 118 includes front-end user interface functionality, which in some embodiments is considered part of the guided generative thread-based user assistance system 102.

[0044] As described in more detail below, the thread classification prompt generator 104 and the plan execution prompt generator 112 are each configured to formulate and output specific types of prompts that can be used as input to one or more large language models. As used herein, prompts include, for example, one or more machine-readable questions, statements, instructions, and / or examples combined with a set of parameter values ​​that constrain the large language model's operation in generating and outputting responses to the prompts. For example, a classification prompt, as used herein, can include instructions that cause the large language model to output a classification (e.g., the large language model operates in a discriminative manner), while a plan execution prompt, as used herein, can include instructions that cause the large language model to execute a plan (e.g., a multi-step prompt) to machine-generate and output one or more thread components (e.g., the large language model operates in a generative manner).

[0045] The way cue words are organized and the wording used to express those elements significantly affect the output generated by the large language model in response to the cue words. For example, small changes in the content or structure of the cue words can lead to very different outputs from the large language model. Thus, the thread classification cue word generator 104 and the planned execution cue word generator 112 are each specifically configured to cause one or more large language models to generate and output thread portions that respond to user-generated thread portions based on specific parameters, instructions, and constraints applicable to a specific task, such as thread classification or planned execution, to be performed by one or more large language models.

[0046] In order to create and operate the various parts of the guided generative thread-based user assistance system 102 and / or the thread-based user assistance interface 118, the components of the guided generative thread-based user assistance system 102 and / or the thread-based user assistance interface 118 can access one or more context sources to, for example, obtain parameter values ​​that can be used to constrain the operations of one or more large language models. Figure 1A Examples of context sources shown include entity graph 103, knowledge graph 105, and data source 107.

[0047] Entity Graph 103 includes a graph-based representation of entity data. As used herein, an entity may refer to a user of the guided generative thread-based user assistance system 102 and / or the thread-based user assistance interface 118, a user of an application software system operating the guided generative thread-based user assistance system 102 and / or the thread-based user assistance interface 118, or another type of entity. Examples of other entity types include companies, organizations, institutions, attributes (e.g., job titles, skills), and digital content items (e.g., articles, posts, comments, shares, or job postings). For example, in an online user connection network such as a social networking service, an entity may include or reference web pages that users of the user connection network can interact with via a user system, wherein the web pages are configured to display digital content items, such as articles, posts, messages, another user's profile, or profile data related to a company, organization, institution, or job posting.

[0048] Entity graph 103 represents entities, such as users, organizations (e.g., companies, schools, institutions), and content items (e.g., user profiles, job postings, announcements, articles, comments, and shares), as nodes in the graph. Entity graph 103 represents relationships between entities as edges or combinations of edges between nodes in the graph, also referred to as mappings or links. In some implementations, one or more entity graphs represent mappings between or within different data blocks (e.g., relationships between job postings, skills, and job titles). In some implementations, the edges, mappings, or links in entity graph 103 indicate online interactions or activities related to the entities connected by the edges, mappings, or links. For example, if a user applies for a job, an edge can be created connecting the user entity to the job entity in the entity graph, where the edge can be labeled using a tag such as "apply".

[0049] The portions of entity diagram 103 can be automatically regenerated or updated from time to time based on changes and updates to the stored data, such as in response to updates to entity data and / or activity data. Similarly, entity diagram 103 can refer to an entire system-wide entity diagram or only a portion of a system-wide diagram, such as a sub-diagram. For example, entity diagram 103 can refer to a sub-diagram of a system-wide diagram, wherein the sub-diagram is associated with a specific entity or entity type.

[0050] Not all implementations have a knowledge graph. In some implementations, knowledge graph 105 is a subset or superset of entity graph 103. The superset of entity graph 103 also contains nodes and edges arranged in a similar manner to entity graph 103 and provides similar functionality. For example, in some implementations, knowledge graph 105 includes multiple different entity graphs 103 connected by cross-application or cross-domain edges or links. For example, knowledge graph 105 can incorporate entity graphs 103 that have been created across multiple different databases or multiple different software products. As an example, knowledge graph 105 can include links between job postings stored and managed by a first application software system and related company comments stored and managed by a second application software system different from the first application software system. Additional or alternative examples of entity graphs and knowledge graphs are discussed in... Figure 5 and Figure 6 As shown below.

[0051] As in Figure 1A As shown, entity graph 103 and / or knowledge graph 105 can supply entity data 122 and / or link data 124 to thread classification prompt generator 104. For example, thread classification prompt generator 104 can use entity data 122 and / or link data 124 to tag thread portions using entity tags, determine which thread classification prompt template to select from the prompt template library, and / or generate thread classification prompts once a thread classification prompt template has been selected. For example, thread classification prompt generator 104 can use entity graph 103 and / or knowledge graph 105 to obtain one or more parameter values ​​to be included in the thread classification prompts.

[0052] For example, the thread category prompt generator 104 can use entity identifiers in entity graphs 103 and 105 to label words or phrases in the thread section. For example, if the thread section includes the phrase "I am interested in becoming a software engineer," then the thread category prompt generator 104 can traverse entity graph 103 to find a title or skill entity that matches "software engineer," determine the identifier associated with that entity, and label the phrase using the identifier for "software engineer" extracted from entity graph 103.

[0053] As another example, the thread category prompt generator 104 can use entity graphs 103 and 105 to select a thread category prompt template. For example, the thread category prompt generator 104 can determine the industry associated with a specific type of job (e.g., technology, healthcare, sales, etc.) or the geographic region associated with the job posting based on a search of entity graph 103, and then select a thread category prompt template based on that industry or geographic region.

[0054] As another example, the thread classification prompt generator 104 can use entity graphs 103 and 105 to generate thread classification prompts. For example, if a user provides a thread portion containing the phrase "I want to work at company X", the thread classification prompt generator 104 can determine, based on a search of entity graph 103, that "company = company X" is included as a parameter value in the thread classification prompt, and based on a search of entity graph 103, determine that the user has five connections at company X, and include identifiers for these five connections, or indicate a flag indicating "connection = yes" in the thread classification prompt.

[0055] As in Figure 1A As shown, entity graph 103 and / or knowledge graph 105 can supply entity data 126 and / or link data 128 to the execution prompt generator 112. For example, the execution prompt generator 112 can use entity data 126 and / or link data 128 to map thread categories to plan types, determine which execution prompt template to select from the prompt template library, and / or generate execution prompts once a execution prompt template has been selected. For example, the execution prompt generator 112 can use entity graph 103 and / or knowledge graph 105 to obtain one or more parameter values ​​to be included in the execution prompts.

[0056] For example, the plan execution prompt generator 112 can use the link data 128 to determine the skills associated with the user who has provided the thread section, determine the skills associated with the job entity marked in the thread section, and map the thread category of "job search" to "draft resume" of the plan type based on the overlap between the user's skills and the skills associated with the job entity.

[0057] As another example, the plan execution prompt generator 112 can use entity graphs 103 and 105 to select a plan execution prompt template. For example, the plan execution prompt generator 112 can determine the industry associated with a specific type of job (e.g., technology, healthcare, sales, etc.) or the geographic region associated with the job posting based on a search of entity graph 103, and then select a plan execution prompt template based on that industry or geographic region (e.g., to draft a resume suitable for a specific industry or geographic region).

[0058] As another example, the execution prompt generator 112 can use entity diagrams 103 and 105 to generate execution prompts. For instance, if the execution prompt generator 112 selects an execution prompt template that includes instructions to generate a resume for the software industry, the execution prompt generator 112 can use entity diagrams 103 and 105 to extract relevant skills from the user's profile and include these skills in the execution prompts.

[0059] Data source 107 can be used to provide retrieved data 130 to thread classification prompt generator 104 and / or scheduled execution prompt generator 112 in a similar manner. Examples of retrieved data 130 include online conversation history 113, web content 115 (e.g., web pages, such as user profile pages, company pages, articles, and posts), data obtained from one or more recommendation systems, and data obtained from domain applications (such as software platforms external to the guided generative thread-based user assistance system 102 but accessible via, for example, one or more APIs (Application Programming Interfaces)).

[0060] Examples of recommender systems include machine learning models that have been trained on historical data to rate user-entity pairs, rank user-entity pairs based on their scores, and select one or more user-entity pairs from the highest-ranked pairs to formulate and output user recommendations. Examples of data obtained from recommender systems include user connection recommendations and job recommendations (e.g., people you may know, jobs you may be interested in).

[0061] Data retrieved from the recommender system can be used to constrain the operation of one or more large language models. For example, the recommender system output can be used by the thread classification prompt generator 104 and / or the plan execution prompt generator 112 to determine whether to include an entity in the prompt or exclude it. For instance, if a user indicates in the thread section that they are interested in working at a specific company, and the job recommendation system outputs job recommendations at that company, then the job recommendations from the recommender system can be included in the plan execution prompt. Similarly, the plan execution prompt can be configured to exclude jobs that are not highly recommended (e.g., those with recommendation scores below a threshold value) by the job recommendation system. As another example, if a user is connected to five people at the company, and the user recommender system outputs connection recommendations for a sixth person working at the same company, then the plan execution prompt can be configured to include the connection recommendation instead of the user's existing connections in the plan execution prompt, or the plan execution prompt can be configured to exclude the connection recommendation from the plan execution prompt and include one or more of the user's existing connections in the plan execution prompt.

[0062] Data retrieved from one or more external applications and / or platforms can be used to constrain the operation of one or more large language models. An example of data obtained from external applications or platforms that can be used by the thread classification prompt generator 104 and / or the plan execution prompt generator 112 to constrain the operation of one or more large language models is entity rating data. For example, job rating data and / or company rating data can be used to exclude job postings from low-rated companies from the plan execution prompts. For instance, if a company rating is less than a threshold rating value determined based on specific design or implementation requirements, job postings associated with that company are ignored from the plan execution prompts. Similarly, if a company rating is greater than or equal to a threshold rating value, where the threshold is determined based on specific design or implementation requirements, job postings associated with that company are included in the plan execution prompts.

[0063] The online conversation history 113 includes historical threads and thread segments associated with online conversations involving a specific user. That is, each user will have a separate online conversation history 113. For example, when an online conversation is initiated between a user and a user assistance system 102 based on guided generative threads (e.g., via a thread-based user assistance interface 118), the initial thread segment that begins the online conversation and all subsequent thread segments involving that user are stored in the online conversation history 113. For example, in some implementations, a text file is created to store the online conversation history 113 and is updated whenever a new thread or thread segment is added to the online conversation, such that the text file contains the entire conversation history involving the user up to the current timestamp.

[0064] Data retrieved from online dialogue history can be used to constrain the operation of one or more large language models. For example, online dialogue history 113 can be used by thread classification prompt generator 104 to disambiguate thread segments subsequently received from the same user or to enhance those subsequently received thread segments using additional contextual data. As another example, online dialogue history 113 can provide parameter values ​​that will be used by plan execution prompt generator 112 when generating plan execution prompts to constrain the plan execution of a second large language model 116. For example, although a recently submitted thread segment may not yet mention the company or industry the user is looking to hire, plan execution prompt generator 112 can extract company names or industry information previously provided in earlier rounds of dialogue and include those company names or industry names in the plan execution prompts.

[0065] Data retrieved from web content 115 can be used to constrain the operation of one or more large language models. Examples of web content 115 that can be extracted and used by thread classification prompt generator 104 and / or plan execution prompt generator 112 to constrain the operation of large language models include user experience, interests, professional fields, educational history, job titles, skills, job history, and similar information related to other types of entities, such as new articles related to companies associated with job postings. For example, negative news articles about a company, such as articles discussing recent layoffs, can be used to exclude that company from plan execution prompts. Similarly, if a company's webpage mentions that the company is involved in an emerging technology that matches the user's interests, the company can be included in the plan execution prompts.

[0066] Thread context data (such as entity data 122, 126, link data 124, 128, and retrieved data 130) can be provided to the user-assisted system 102 based on guided generative threads from potentially various applications, platforms, and data sources (including user interfaces, databases, and other types of data repositories, including online, real-time, and / or offline data sources). Figure 1A In the example, thread context data is received via one or more user devices or systems (such as portable user devices such as smartphones, wearable devices, tablet computers or laptop computers, one or more web servers, and / or one or more database servers); however, the user assistance system 102 based on guided generative threads can receive thread context data of any different type via any type of electronic machine, device or system.

[0067] In operation, the thread classification prompt generator 104 receives a user-generated thread segment 120 via a thread-based user assistance interface 118. In response to the user-generated thread segment 120, the thread classification prompt generator 104 formulates and outputs a thread classification prompt 106. For example, if the user-generated thread segment 120 is the first thread segment in an online conversation, then the thread classification prompt 106 is based on the user-generated thread segment 120 and one or more possible thread context data. If there have been previous rounds of conversation before the user-generated thread segment 120, then the thread classification prompt 106 is based on the user-generated thread segment 120, one or more previous conversation rounds, and one or more possible thread context data.

[0068] Thread classification prompt 106 contains one or more instructions for the first large language model 108 to generate and output classifications (e.g., task type, user intent, or goal) based on the user-generated thread portion 120 and any constraints contained in the prompt. For example, the thread classification prompt generator 104 selects a thread classification prompt template, combines the template with the user-generated thread portion 120 and optionally one or more thread context data blocks to formulate the thread classification prompt. An example of an operation that can be performed by the thread classification prompt generator 104 to generate thread classification prompt 106 is provided in... Figure 1B As shown below.

[0069] The first large language model 108 includes one or more neural network-based machine learning models. In some implementations, a neural network-based deep learning model architecture is used to construct the first large language model 108. In some implementations, the neural network-based architecture includes one or more input layers that receive model inputs, generate one or more embeddings based on the model inputs, and pass the one or more embeddings to one or more other layers of the neural network. In other implementations, the one or more embeddings are generated based on model inputs from a preprocessor, which are then input to the neural network model, and the neural network model generates an output based on the embeddings.

[0070] In some implementations, the neural network-based machine learning model architecture includes one or more self-attention layers that allow the model to assign different weights to different parts of the model input. Alternatively or additionally, the neural network architecture includes feedforward layers and residual connections that allow the model to machine complex data patterns including relationships between different parts of the model input in multiple different contexts. In some implementations, the neural network-based machine learning model architecture is constructed using a transformer-based architecture that includes self-attention layers, feedforward layers, and residual connections between layers. The exact number and arrangement of each type of layer, as well as the hyperparameter values ​​used to configure the model, are determined based on the specific design or implementation requirements of the guided generative thread-based user assistance system 102.

[0071] In some examples, the neural network-based machine learning model architecture includes or is based on one or more generative transformer models, one or more generative pre-trained transformer (GPT) models, one or more bidirectional encoder representations from a transformer (BERT) model, one or more large language models (LLM), one or more XLNet models, and / or one or more other natural language processing (NL) models. In some examples, the neural network-based machine learning model architecture includes or is based on one or more predictive text neural models capable of receiving text input and generating one or more outputs by processing the text using one or more neural network models. Examples of predictive neural models include, but are not limited to, generative pre-trained transformers (GPT), BERT, and / or recurrent neural networks (RNNs). In some examples, one or more types of neural network-based machine learning model architectures include or are based on one or more multimodal neural networks capable of outputting different modalities (e.g., text, images, sound, etc.) based on text input, individually and / or in combination. Thus, in some examples, a multimodal neural network implemented in a user-assisted system based on guided generative threads can output digital content including combinations of two or more of text, images, video, or audio.

[0072] In some implementations, a first large language model 108 is trained on a large dataset of digital content such as natural language text, images, videos, audio files, or multimodal datasets. For example, training samples of digital content, such as natural language text extracted from publicly available data sources, are used to train one or more generative models of a user-assistance system based on a guided generative thread. The size and composition of the dataset used to train one or more models of the user-assistance system based on a guided generative thread can vary depending on the specific design or implementation requirements of the user-assistance system based on a guided generative thread. In some implementations, one or more datasets in the dataset used to train one or more models of the user-assistance system based on a guided generative thread include hundreds of thousands to millions or more different training samples.

[0073] In some embodiments, one or more models of a guided generative thread-based user-assistance system comprise multiple generative models trained on datasets of varying sizes. For example, a guided generative thread-based user-assistance system can include a comprehensive but low-capacity generative model trained on a large dataset to generate thread portions in response to user input, and the same generative model can also include a less comprehensive but high-capacity model trained on a smaller dataset, wherein the high-capacity model is used to generate outputs based on examples obtained from the low-capacity model. In some implementations, reinforcement learning is used to further improve the outputs of one or more models of the guided generative thread-based user-assistance system. In reinforcement learning, ground truth examples of the desired model outputs are paired with corresponding inputs, and these input-example-output pairs are used to train or fine-tune one or more models of the guided generative thread-based user-assistance system.

[0074] In some implementations, graph neural networks are used to implement one or more models of a user-assisted system based on supervised generative threads. For example, in one model instance, a modified version of the bidirectional encoder representation with a transformer neural network (BERT) is specifically configured to generate and output thread classifications, and in another instance, to generate and output machine-generated thread portions. In some implementations, self-supervision is utilized to train the modified BERT, for example, by masking portions of the input data, allowing BERT to learn to predict the masked data. During scoring, the masked entity is associated with a portion of the input data, and the model generates an output at the location of the masked entity based on the input data.

[0075] In operation, the first language model 108 receives input including a thread classification prompt 106. The thread classification prompt 106 is transmitted to the first language model 108 via, for example, an application programming interface (API). In response to the thread classification prompt 106, the first language model 108 generates and outputs a thread classification 110. The thread classification 110 includes labels that clarify the task type, user intent, or goal of the thread portion 120, which is determined, generated, and output by the first language model 108 in response to the thread classification prompt 106.

[0076] Thread tag generator 109 receives thread category 110 (e.g., via an API). Thread tag generator 109 converts thread category 110 into thread tag 111, such that thread tag 111 is configured for display at a thread-based user assistance interface 118. For example, if thread category 110 represents the category as a numeric or alphanumeric code, thread tag generator 109 converts or transforms said value or code into a text tag (e.g., J123 is transformed into "Sales Manager Location at Acme"). To determine thread tag 111, thread tag generator 109 performs a lookup in a mapping table (e.g., a key-value store) that stores the relationship between thread category 110 and corresponding thread tag 111. Alternatively, thread tag 111 includes natural language text extracted from the thread.

[0077] Thread tag 111 is based on and associated with a thread that includes at least a user-generated thread portion 120. For example, thread tag generator 109 creates a message that includes tag 111, the thread identifier of its associated thread, and tag 111 (e.g., thread ID, thread_tag). The message containing thread tag 111 and the associated thread ID is transmitted to thread-based user assistance interface 118 for display in conjunction with the associated thread matching the thread ID. For example, in a vertical or horizontal scrolling message transmission paradigm, thread tag 111 is displayed at the top or bottom, or left or right, of the screen while the associated thread is being displayed. An example of thread tags dynamically created using the disclosed techniques is shown in the user interface screen capture diagram described below.

[0078] Thread classification 110 is also passed to execution prompt generator 112 (e.g., via an API). In response to thread classification 110, execution prompt generator 112 formulates and outputs execution prompt 114. Execution prompt 114 contains one or more instructions for the second large language model 116 to generate and output machine-generated thread portions 134 based on thread classification 110 and any constraints contained in the prompt. For example, execution prompt generator 112 selects an execution prompt template, combines the execution prompt template with thread classification 110 and optionally one or more thread context data blocks to formulate execution prompt 114. Examples of operations that can be performed by execution prompt generator 112 to generate execution prompt 114 are described below.

[0079] The second large language model 116 includes one or more neural network-based machine learning models, such as any type of model described above with reference to the first large language model 108. In some implementations, the second large language model 116 includes the first large language model 108. For example, the first large language model 108 and the second large language model 116 are included in the same large language model. In some implementations, the second large language model 116 and the first large language model 108 have the same model architecture but are trained differently. For example, in some implementations, the first large language model 108 is trained on a large dataset of digital content, while the second large language model 116 is pre-trained on the same large dataset but then tuned for a specific task type (such as job search, resume generation, interview preparation, etc.).

[0080] In operation, the second large language model 116 is machine-generated and outputs a machine-generated thread portion 134. Examples of the machine-generated thread portion 134 include natural language text and / or multimodal content, such as conversational questions, job recommendations including links to relevant job postings, personalized task lists tailored based on thread context data, personalized job evaluations tailored based on thread context data, push notifications, pull notifications, etc. Additional examples of the machine-generated thread portion 134 are shown in the user interface screen capture diagram described below.

[0081] In summary, user-generated thread segment 120 and machine-generated thread segment 134 constitute the thread of the online conversation. The thread can include other user-generated and / or machine-generated thread segments occurring before or after thread segments 120 and 134. For example, the online conversation can include several rounds of dialogue, including multiple user-generated and machine-generated thread segments, and the online conversation can continue intermittently over variable time intervals such as several minutes or over hours, days, or weeks. Whenever an additional thread segment (whether user-generated or system-generated) is added to the online conversation, it is added (e.g., appended) to the conversation history, allowing the thread classification prompt generator 104 and the scheduled execution prompt generator 112 to each access the entire conversation history to formulate their respective prompts.

[0082] The thread-based user assistance interface 118 includes a front-end component through which a user can interact with a guided, generative thread-based user assistance system 102 on the user's electronic device. For example, the thread-based user assistance interface 118 displays an online conversation including user-generated thread segments 120 and machine-generated thread segments 134. If the online conversation includes multiple threads, the thread-based user assistance interface 118 dynamically groups the thread segments by thread and labels the threads using associated thread tags 111.

[0083] In some implementations, the thread-based user-accessible interface 118 includes a "focusing" user interface mechanism. When selected by the user, the focusing mechanism allows the user to zoom in on a specific thread based on its associated thread label 111. For example, in a multi-threaded online conversation, the focusing mechanism allows the user to select a specific thread label, and by doing so, only the portion of the thread associated with the selected thread label is viewed (e.g., other threads not associated with the selected thread label are collapsed or hidden). Examples of user interface displays and mechanisms are shown in the user interface screen capture diagrams described below.

[0084] For illustrative purposes, the above text is provided. Figure 1A The examples shown in the accompanying drawings are not limited to the described examples. Additional or alternative details and implementations are described herein.

[0085] Figure 1B This is a flowchart of an example method for generating thread classification prompts using components of a user assistance system based on guided generative threads, according to some embodiments of this disclosure.

[0086] The method is executed by processing logic, which includes hardware (e.g., processing device, circuitry, dedicated logic, programmable logic, microcode, device hardware, integrated circuits, etc.), software (e.g., instructions that run or execute on the processing device), or a combination thereof. In some embodiments, the method is executed by a component of computing system 140, or by... Figure 1A The user assistance system 102, based on guided generative threads, is executed, in some embodiments including... Figure 1A The one shown may not be present Figure 1B The components specifically shown in the text, or those made by Figure 5 The components of the user assistance system 580, which is based on a guided generative thread, are used to execute this, and in some embodiments, include... Figure 5 The one shown may not be present Figure 1B The components are specifically illustrated. Although shown in a particular sequence or order, the order of processes can be modified unless otherwise stated. Therefore, the illustrated embodiments should be understood as examples only, and the illustrated processes can be performed in different orders, and some processes can be performed in parallel. In addition, at least one process can be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are also possible.

[0087] exist Figure 1BIn this system, computing system 140 includes: data storage, including thread history 144, tag dictionary 150, entity graph 154, and thread classification prompt dictionary 162; and subprocesses, including large language model 148, recommender system 156; and domain application 158. These data storages and subprocesses supply input to various functional components of computing system 140, including operations 142, 146, 152, 160, and 164. Operations 142, 146, 152, 160, and 164 provide input to functional component 166. The data storage, subprocesses, and functional components are embodied in one or more non-transitory machine-readable media, making them accessible to and executable by one or more processors of computing system 140.

[0088] Functional component 142 receives a user-generated thread portion. For example, functional component 142 obtains the user-generated thread portion, such as user-generated thread portion 120, via a thread-based user assistance interface (such as thread-based user assistance interface 118). The user-generated thread portion includes an associated user identifier (e.g., account identifier, session identifier, network address, or device identifier) ​​and an associated thread portion identifier. The user-generated thread portion also includes a timestamp indicating the date and time received at functional component 142 or the thread-based user assistance interface. Functional component 142 provides (e.g., transmits or sends) the user-generated thread portion and applicable identifier and timestamp data to functional component 166 and / or functional component 146.

[0089] Functional component 146 obtains (e.g., acquires) one or more stored threads associated with a received user-generated thread portion. For example, functional component 146 searches the thread history 144 for stored threads or thread portions that match a user identifier and have associated timestamps falling within time intervals including the timestamps of the received user-generated thread portions. In some implementations, the thread history 144 comprises a text file containing text of previous thread portions, such that functional component 146 performs a text search or string matching algorithm to determine whether the thread history 144 contains any thread or thread portion associated with the received user-generated thread portion. Although not specifically shown, in some implementations, a large language model 148 is used to segment the thread history 144 based on thread classification and / or entity labels.

[0090] As used in this article, "match" or "matching" can refer to an exact match or an approximate match, such as a match calculated based on the similarity between two data blocks. An example of similarity calculation is cosine similarity. Other schemes that can be used to determine the similarity between or within data blocks include clustering algorithms (e.g., k-means clustering), binary classifiers trained to determine whether two items in a pair are similar or dissimilar, and neural network-based vectorization techniques such as WORD2VEC. In some implementations, generative language models (such as large language models) are used to determine the similarity of data blocks.

[0091] Functional component 146 provides (e.g., transmits or sends) any retrieved stored thread and applicable identifier and timestamp data to functional component 166 and / or functional component 152.

[0092] Functional component 152 uses entity tags obtained from tag dictionary 150 to tag the received user-generated thread portion. In some implementations, functional component 152 uses a large language model 148 (e.g., a first large language model 108 or a second large language model 116) to perform entity tagging. For example, functional component 152 generates entity tagging prompts containing one or more machine-readable instructions that instruct the large language model 148 to "use the tag dictionary to tag entities in the received user-generated thread portion." In some implementations, the large language model 148 divides the dialogue into threads based on entity tagging. In other implementations, named entity recognition (NER) or another entity tagging scheme can be used.

[0093] Tag dictionary 150 contains stored canonical entity names and associated data values ​​(e.g., title: software engineer; skill: Python; company: Acme). In some implementations, tag dictionary 150 is personalized for the user, for example, dynamically customized based on thread history and / or thread context data. For instance, if functional component 152 identifies a previously unseen entity name in a received user-generated thread segment, functional component 152 adds the entity name to tag dictionary 150 for potential use in subsequent rounds of online conversation.

[0094] In some implementations, stored threads previously tagged by functional component 152, obtained by functional component 146 from thread history 144, are used to update the tag dictionary 150 or as input to the large language model 148 associated with the tagged cue words. For example, if a received user-generated thread portion contains ambiguous words or phrases, such as references to previously discussed entities or topics (e.g., "third job" or "the last job I viewed"), functional component 152 searches thread history 144 for entities that match the ambiguous phrase, and if a match is found, functional component 152 applies a tag to the ambiguous phrase based on the matching entity found in thread history 144. Functional component 152 provides (e.g., passes or sends) the tagged user-generated thread portion to functional components 166 and / or 160.

[0095] Functional component 160 obtains (e.g., acquires) thread-related context data from one or more context sources based on the tagged user-generated thread portion. Figure 1B Examples of context sources shown include entity graph 154, recommender system 156, and domain application 158. Examples of entity graph 154 include... Figure 1A Entity diagrams 103 and 105. Examples of recommender systems 156 include trained machine learning-based rating models, ranking models, and / or classification models, such as job recommender models and connection recommender models. Examples of domain applications 158 include web-based applications that provide user ratings (e.g., company ratings) and social networking services that provide user feedback, reactions, and / or comments about entities (e.g., social networking services that allow users to react to and / or comment on company posts).

[0096] As an example, functional component 160 traverses entity graph 154 to connect users with companies mentioned in the tagged user-generated thread section. As another example, functional component 160 obtains a set of job recommendations from recommendation system 156 based on skills mentioned in the tagged user-generated thread section. As yet another example, functional component 160 searches for rating data in a rating system based on company names mentioned in the tagged user-generated thread section. Functional component 160 provides (e.g., transmits or sends) the retrieved thread-related context data to functional component 166.

[0097] Functional component 166 generates thread classification prompts based on the output of one or more functional components 142, 146, 152, 160, and 164. Functional component 166 feeds the output of a large language model (e.g., Figure 1A The first major language model (108) provides (e.g., passing or sending) the generated thread classification prompt words.

[0098] Examples of thread category prompts that can be generated by functional component 166 are shown in Table 1 below.

[0099] Table 1. Examples of thread category prompts.

[0100] As shown in Table 1, the example thread classification prompt contains instructions for classifying user input (e.g., a received user-generated thread portion). The example thread classification prompt also constrains the large language model to a set number of possible categories into which the user input will be classified (e.g., requiring the large language model to select only one category), specifies the applicable thread context data (e.g., user profile, conversation history, previous user input, category and job recommendations), and specifies the output format (e.g., natural language text) to be generated by the large language model for thread classification. In some implementations, the design of prompts that provide specific instruction portions (e.g., general instructions, context, output indicators, etc.) improves the efficiency of communication with the large language model, which in turn improves classification accuracy.

[0101] Figure 1C This is a flowchart of an example method for generating planned execution prompts using components of a user-assisted system based on a guided generative thread, according to some embodiments of the present disclosure.

[0102] The method is executed by processing logic, which includes hardware (e.g., processing device, circuitry, dedicated logic, programmable logic, microcode, device hardware, integrated circuits, etc.), software (e.g., instructions that run or execute on the processing device), or a combination thereof. In some embodiments, the method is executed by a component of computing system 170, or by... Figure 1A The user assistance system 102, based on guided generative threads, is executed, in some embodiments including... Figure 1A The one shown may not be present Figure 1C The components specifically shown in the text, or those made by Figure 5 The components of the user assistance system 580, which is based on a guided generative thread, are used to execute this, and in some embodiments, include... Figure 5 The one shown may not be present Figure 1C The components are specifically illustrated. Although shown in a particular sequence or order, the order of processes can be modified unless otherwise stated. Therefore, the illustrated embodiments should be understood as examples only, and the illustrated processes can be performed in different orders, and some processes can be performed in parallel. In addition, at least one process may be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are also possible.

[0103] exist Figure 1C In this system, computing system 170 includes: data storage, including thread history 176, entity graph 180, and plan library 188; and subprocesses, including large language model 172, recommender system 182, and domain application 184. These data storages and subprocesses supply input to various functional components of computing system 170, including operations 174, 178, 186, and 190. Operations 174, 178, 186, and 190 provide input to functional component 192. The data storages, subprocesses, and functional components are embodied in one or more non-transitory machine-readable media, making them accessible to one or more processors of computing system 170 for execution by said one or more processors.

[0104] Functional component 174 receives the response from the large language model 172 (e.g., the first large language model 108) to thread classification prompts (e.g., by...). Figure 1B The computing system 140 generates and outputs a thread classification in response to a received user-generated thread portion (a thread classification prompt word). Functional component 174 provides (e.g., transmits or sends) the thread classification, along with applicable identifiers and timestamp data, to functional components 192 and / or 178.

[0105] Functional component 178 obtains (e.g., acquires) one or more stored threads associated with the received thread category. For example, functional component 178 searches in thread history 176 for stored threads or thread segments that match the received thread category. In some implementations, thread history 176 includes a text file containing text of previous thread segments, causing functional component 178 to perform a text search or string matching algorithm to determine whether thread history 176 contains any threads or thread segments that match the received thread category. Functional component 178 provides (e.g., transmits or sends) any retrieved stored threads, along with applicable identifiers and timestamp data, to functional component 192 and / or functional component 186.

[0106] Functional component 186 obtains (e.g., retrieves) thread-related context data from one or more context sources based on thread classification and / or the retrieved stored threads. Figure 1C Examples of context sources shown include entity graph 180, recommender system 182, and domain application 184. Examples of entity graph 180 include... Figure 1AEntity diagrams 103 and 105. Examples of recommender systems 182 include trained machine learning-based rating models, ranking models, and / or classification models, such as job recommender models and connection recommender models. Examples of domain applications 184 include web-based applications that provide user ratings (e.g., company ratings) and social networking services that provide user feedback, reactions, and / or comments about entities (e.g., social networking services that allow users to react to and / or comment on company posts).

[0107] As an example, functional component 186 traverses entity graph 180 based on the received thread classification for user connections to companies mentioned in the retrieved (or stored) threads. As another example, functional component 186 obtains a set of connection recommendations from recommendation system 182 based on the retrieved (or stored) threads. As yet another example, functional component 186 searches for rating data in a rating system based on the names of companies mentioned in the retrieved (or stored) threads. Functional component 186 provides (e.g., transmits or sends) the retrieved thread-related context data to functional component 192 and / or functional component 190.

[0108] Functional component 190 selects a plan (e.g., a multi-step prompt) and obtains (e.g., retrieves) an associated plan template from plan library 188. Plans can be domain-specific or domain-independent. Examples of job domain-specific plans include those for assisting users in job searches, resume writing, interview preparation, or referral requests. Examples of domain-independent plans include generic instructions, such as instructions to perform a search containing placeholders for parameter values ​​that can be obtained from, for example, thread context data.

[0109] Functional component 192 maps the received thread classification (alone or in combination with data extracted from one or more stored threads or retrieved thread contexts) to a plan identifier, and then retrieves a plan template that matches the plan identifier. As used herein, mapping can refer to an executable procedure such as a table lookup or database search. In some implementations, the functional component uses portions of one or more stored threads and / or the retrieved thread context 186 to select a plan. For example, if the retrieved thread context indicates that the user has a first-degree connection with a company hiring for a position of interest to the user, functional component 190 can select a plan to help the user request a referral rather than a plan to help the user write a resume or a plan to help the user prepare for an interview.

[0110] Functional component 192 generates a plan execution prompt based on the output of one or more functional components 174, 178, 186, and 190. In some implementations, functional component 193 includes rewriting the prompt or sub-prompts to simplify downstream jobs for the large language model. In some implementations, functional component 193 includes instructions for executing specific steps of the plan online or offline, for example, to conserve or optimize the use of computing resources. Functional component 192 provides (e.g., transmits or sends) the generated plan execution prompt to the large language model (e.g., ...). Figure 1A The second largest language model (116).

[0111] Examples of planned execution prompts that can be generated by functional component 192 are shown in Table 2 below.

[0112] Table 2. Examples of planned execution prompts.

[0113] As shown in Table 2, the example plan execution prompt instructs the generative model to generate job recommendations and includes several specific instructions that will be executed by the large language model. The example plan execution prompt also includes thread context data as constraints, causing the large language model to output job recommendations based on said thread context data.

[0114] Another example of a planned execution prompt is an instruction for a generative model to generate a job assessment. For instance, a job assessment prompt might contain multiple sections, each containing one or more instructions, such as: identifying the task the generative model should perform (e.g., “You are an assistant, career coach, and job search assistant”), retrieving user context (e.g., “Get user profile, user preferences, user conversation history, job and company information, and comparison with other applicants”), and generating a response (e.g., “Provide an assessment of the job based on the user context”).

[0115] Figure 1D This is a block diagram of an example architecture for a computing system according to some embodiments of the present disclosure. In example architecture 194, a user-assisted system based on guided generative threads can be implemented as supported by a cross-modal generative AI platform. N One of a series of vertical applications, where... N It is a positive integer. For example, a general-purpose thread-based user assistance system can be implemented as a vertical application supported by a cross-modal generative AI platform. A general-purpose user assistance system can be configured for a specific domain via one or more plans. For example, a set of plans can be provided to customize the general-purpose user assistance system for a job domain, a learning and education domain, an e-commerce domain, an entertainment or gaming domain, or another domain.

[0116] Each vertical application and related initiative is supported by a cross-modal generative AI platform, which can be implemented using components described herein, such as large language models. As a result, this cross-modal generative AI platform can seamlessly integrate thread-based interaction patterns with other interaction patterns, such as push notifications, pull notifications, feeds, and recommendations. To this end, the thread-based guided generative AI platform combines multiple different interaction patterns via the cross-modal generative AI platform by maintaining general application logic for managing the global state and maintaining logical connections between or within different interaction patterns. For example, this cross-modal generative AI platform tracks, updates, and maintains the interactions of thread-based interaction patterns with other interaction patterns, such as push notifications, pull notifications, feeds, and recommendations, and their corresponding states. For example, the global state includes both state information for a specific initiative and state information for a vertical application (e.g., a union).

[0117] In some implementations, cross-modal generative AI platforms are configured to passively push personalized content to users, understand user intent, emotions, or goals (whether through explicit or inactive actions), and proactively communicate with users to provide timely suggestions and tune recommendations accordingly. Cross-modal generative AI platforms potentially receive user feedback across multiple different interaction modes (e.g., impressions, views, reactions, etc.), all of which can be used to improve the generative AI platform through, for example, cue word engineering / refinement and / or model tuning.

[0118] In the specific context of job-related user assistance, the following scenario can be implemented using the disclosed technologies: A user clicks the "Jobs" tab of a vertical application. A user assistance system based on a guided generative thread is activated and asks the user about their job search goals. The user assistance system based on the guided generative thread creates a plan, executes the plan using a generative AI platform, and displays relevant jobs in a conversational format, with next steps highlighted, explained, and recommended in context provided by the generative AI platform. The generative AI output is based on past cross-modal interactions, interactions, and other context. The generative AI output includes, for example, user-personalized explanations of job recommendations. User-customized generative AI output provides explanations to help the user understand the reasons for the job recommendations (if they are not explicitly obvious to her). The wording and tone of the generative AI output encourage the user to provide feedback that can be used to improve future recommendations.

[0119] Figure 2A This is a timing diagram illustrating an example of communication between components of a thread-based user assistance interface and a user assistance system based on a guided generative thread, according to some embodiments of the present disclosure.

[0120] exist Figure 2AIn Figure 2, the communication indicated by the arrows occurs in a time sequence. For example, thread (1) communication from the thread-based user assistance interface 118 to the thread classification prompt generator 104 occurs at a first time point, and classification prompt (1) communication from the thread classification prompt generator 104 occurs at a second time point after the first time point. Communication between the components shown in Figure 2 includes, for example, network communication and / or on-device communication. For example, all or part of the thread-based user assistance interface 118, the thread classification prompt generator 104, the first large language model 108, the plan execution prompt generator 112, and the second large language model 116 can be implemented on a single device or across multiple devices.

[0121] exist Figure 2A In the example, one or more large language models are used to classify, label, and respond to multiple threads. A thread classification prompt generator 104 receives the thread (1) via a thread-based user-aided interface 118. In response to the thread (1), the thread classification prompt generator 104 generates and outputs a thread classification prompt (1). A first large language model 108 receives the thread classification prompt (1) via the thread classification prompt generator 104. The first large language model 108 generates and outputs the thread classification (1). A plan execution prompt generator 112 and the thread-based user-aided interface 118 receive the thread classification (1) via the first large language model 108.

[0122] In response to thread classification (1), the thread-based user assistance interface 118 can output thread labels in association with the display of thread (1) and based on thread classification (1). In response to thread classification (1), the execution plan prompt generator 112 generates and outputs the execution plan prompt (1). The second language model 116 receives the execution plan prompt (1) via the execution plan prompt generator 112. In response to the execution plan prompt (1), the second language model 116 generates and outputs a response (1). The thread-based user assistance interface 118 receives the response (1) via the second language model 116 and displays the response (1).

[0123] In response to the display of response (1), the thread-based user assistance interface 118 receives thread (2). The thread classification prompt generator 104 (e.g., from data storage) receives thread (1) and also receives thread (2) via the thread-based user assistance interface 118. Based on thread (1) and thread (2), the thread classification prompt generator generates and outputs the thread classification prompt (2). The first large language model 108 receives the thread classification prompt (2) via the thread classification prompt generator 104. In response to the thread classification prompt (2), the first large language model 108 generates and outputs the thread classification (2).

[0124] In response to thread classification (2), the thread-based user assistance interface 118 can output thread labels in association with the display of thread (2) and based on thread classification (2). In response to thread classification (2), the execution plan prompt generator 112 generates and outputs the execution plan prompt (2). The second language model 116 receives the execution plan prompt (2) via the execution plan prompt generator 112. In response to the execution plan prompt (2), the second language model 116 generates and outputs the response (2). The thread-based user assistance interface 118 receives the response (2) via the second language model 116 and displays the response (2).

[0125] For illustrative purposes, the above-mentioned examples are provided. Figure 2A The examples shown in the accompanying drawings are not limited to the described examples. Additional or alternative details and implementations are described herein.

[0126] Figure 2B This is a timing diagram illustrating an example of using multiple threads to generate scheduled execution prompts according to some embodiments of this disclosure. Figure 2B In the diagram, the communications indicated by the marked arrows occur in a time sequence; for example, dialogue D1 begins at the first moment, and dialogue D2 begins at the second moment after the first moment.

[0127] exist Figure 2B In the example, dialogue D1 includes three threads, for example, thread C1T1, thread C1T2, and thread C1T3, while dialogue D2 includes threads C2T1, thread C2T3, and thread C2T2. Each thread in each dialogue of D1 and D2 has been classified, for example, by a large language model as described herein. For example, thread D1T1 has been classified as having topic T1, thread D1T2 has been classified as having topic T2, and thread D1T3 has been classified as having topic T3, such that dialogue D1 contains three different threads covering three different topics. Similarly, dialogue D2 contains three different threads covering the same three topics but in a different order. In dialogue D2, thread D2T1 has been classified as having topic T1, thread D2T3 has been classified as having topic T3, and thread D2T2 has been classified as having topic T2.

[0128] All threads in each dialogue in D1 and D2 are associated with the same user and are therefore stored as part of the same thread history. Thus, when the execution prompt generator 112 generates execution prompts, it searches the thread history for previous threads that match the current thread's topic. For example, when the execution prompt generator 112 generates execution prompts for thread D2T1, it searches the thread history for previous threads that have been categorized with topic T1. Since thread D1T1 matches topic T1, both thread D1T1 and thread D2T1 are used to generate execution prompts for thread D2T1. In response to the execution prompts for thread D2T1 based on both thread D1T1 and thread D2T1, the second language model 116 generates and outputs a response for thread D2T1.

[0129] Similarly, when the execution plan prompt generator 112 generates an execution plan prompt for thread D2T3, it searches the thread history for previous threads that have been categorized with topic T3. Since thread D1T3 matches topic T3, both thread D1T3 and thread D2T3 are used to generate an execution plan prompt for thread D2T3. In response to the execution plan prompts for thread D2T3 based on both thread D1T3 and thread D2T3, the second language model 116 generates and outputs a response for thread D2T3.

[0130] Similarly, when the execution plan prompt generator 112 generates an execution plan prompt for thread D2T2, it searches the thread history for previous threads that have been categorized as having topic T2. Since thread D1T2 matches topic T2, both thread D1T2 and thread D2T2 are used to generate an execution plan prompt for thread D2T2. In response to the execution plan prompt for thread D2T2 based on both thread D1T2 and thread D2T2, the second large language model 116 generates and outputs a response for thread D2T2.

[0131] In this way, the disclosed technology can effectively manage long-running chats in chronological order, even when users reference topics they first mentioned several days ago. The disclosed technology dynamically organizes and tags threads as they occur, allowing users to see how topics change as they scroll through the conversation history, leveraging multi-threaded conversations. In some implementations, the disclosed technology intelligently updates the conversation header displaying the thread topic (e.g., title and company / organization). Thread topics or tags are not limited to text but can also include icons or other non-text output, or alternatively. In this way, the disclosed technology can intelligently group threads about specific topics, even if these threads are spaced apart in time (e.g., threads occur on multiple different login sessions, potentially separated by hours or days). The intelligent dynamic thread tagging of the disclosed technology also allows users to search and filter threads on specific topics within ongoing conversations without having to manage lists of individual conversations or chats.

[0132] In some implementations, the disclosed techniques create a user interface anchoring mechanism associated with thread topics, such that each thread topic has its own anchor point. Users can click any anchor point to return to the previous thread associated with that anchor point. For example, if a conversation has covered multiple different topics, but the user wants to return to a previous topic, they can click the anchor point to quickly jump to the relevant part of the conversation without having to scroll.

[0133] In some implementations, the disclosed techniques include dynamic context headers and a "focus" feature that removes or hides all parts of the thread that do not relate to the selected topic or the current topic. The focus feature provides a non-linear way to navigate the conversation while minimizing the need for scrolling, as anchors are dynamically created and displayed, and the focus feature allows members to use anchors to jump to any topic in the conversation. The ability to explicitly search by keyword or topic is also available at the top of the member's profile.

[0134] For illustrative purposes, the following is provided: Figure 2B The examples shown and the accompanying descriptions are provided. This disclosure is not limited to the examples described. Additional or alternative details and implementations are described herein.

[0135] Figure 2C This is a flowchart illustrating examples of contextual content generated by a generative model according to some embodiments of the present disclosure.

[0136] The method is executed by processing logic, which includes hardware (e.g., processing device, circuitry, dedicated logic, programmable logic, microcode, device hardware, integrated circuits, etc.), software (e.g., instructions that run or execute on the processing device), or a combination thereof. In some embodiments, the method is executed by a component of computing system 170, or by... Figure 1A The user assistance system 102, based on guided generative threads, is executed, in some embodiments including... Figure 1A The components shown may not be specifically shown in Figure 2C, or may be made of Figure 5 The components of the user assistance system 580, which is based on a guided generative thread, are used to execute this, and in some embodiments, include... Figure 5 The one shown may not be present Figure 2C The components are specifically illustrated. Although shown in a particular sequence or order, the order of processes can be modified unless otherwise stated. Therefore, the illustrated embodiments should be understood as examples only, and the illustrated processes can be performed in different orders, and some processes can be performed in parallel. In addition, at least one process can be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are also possible.

[0137] exist Figure 2C In this document, the techniques described herein are used to generate a plan execution prompt 208. A large language model 206 is applied to the plan execution prompt 208, such that the input of the plan execution prompt 208 to the large language model 206 causes the large language model 206 to execute a multi-step plan to generate contextual output, for example, to machine-generate a response to a user's request for information or assistance. The large language model 206 includes one or more generative models, such as one of the generative models described herein. The execution prompt 208 contains instructions that cause the large language model 206 to perform the following operations: obtain user context data 202; obtain target entity data 204; communicate bidirectionally with the search system 210 to obtain external data that may be relevant to the target entity and / or the requesting user; match potentially relevant external data with user context data 202 and / or target entity data 204 based on the matching of potentially relevant external data with user context data 202 and / or target entity data 204; determine whether any potentially relevant external data is included or excluded in the output to be generated by the large language model 206; and generate and output one or more contextual outputs, such as context-generated target entity evaluation and / or context-generated task list.

[0138] exist Figure 2CIn the example, user context data 202 includes, for example, user profile data, user activity data, user connection data, or other user-related information obtained from, for example, social networking services. Examples of target entities include entities that the user expects information about or that are helpful to them, such as job titles, job postings, business projects the user is considering purchasing, etc. Other examples of target entities are concepts that the user wants to explore. For example, if the user has not yet found a job that interests them, the target entity might be a specific job title, career goal, objective, or intent.

[0139] Search system 210 includes, for example, search engines, such as Internet search engines. Search engine 210 searches applications, services, and / or data sources connected to network 214 to obtain potentially relevant external data that matches one or more parameters specified by large language model 206. For example, large language model 206 executes instructions for planning execution of cue words 206, which cause large language model 206 to generate and output search queries based on user context data and / or target entity data 204.

[0140] The search system 210 executes search queries by returning potentially relevant external data from, for example, one or more web-connected data sources (such as one or more domain applications 212 and / or one or more content distribution services related to the target entity). Examples of domain applications are web applications or "apps" that provide data or services related to the target entity data 204. For example, if the target entity is a job posting, examples of web applications include job rating sites, newsboards, and social media sites where users post information and comments about jobs.

[0141] Examples of content distribution services 216 are content distribution services related to target entity data 204, such as news sites, news apps, or news feeds that distribute digital content related to target entity data 204. For example, if the target entity is a job, examples of content distribution services include apps, sites, and feeds (such as business news services, startup news services, etc.) that distribute content about jobs and / or companies.

[0142] As in Figure 2CAs shown, the generation of contextual outputs (e.g., context-generated target entity assessments 218 and / or context-generated task lists 220) by the large language model 206 effectively combines external data from network 214 with user contextual data 202 (e.g., user profile data, user intent, etc.) and / or target entity data 204. For example, job assessments, entity assessments, or task lists are contextualized based on both user contextual data 202 (e.g., profile data and / or user intent) and external data obtained from network 214 via search system 210. For example, a user can be matched with a set of different job postings based on their profile data and / or intent, and then the assessment of a particular job can be contextualized to include the obtained external data about the job or the hiring company, such as search data about typical starting salaries at the company, whether the company has recently received funding or the number of funding rounds the company has received, the company's stock price, basic information, ratings, and current leaderboards. External data can be used to modify the assessment. For example, if external data conflicts with the user's contextual data, the evaluation can be modified to infer that the job is not a good fit for the user, rather than inferring that the job is a good fit. Similarly, if external data does not conflict with the user's contextual data, the evaluation can be modified to infer that the job will be a good fit for the user, rather than inferring that the job is not a good fit.

[0143] Contextual task lists 220 can be generated based on target entities or on the user's more general intent, purpose, or goal. For example, in a job context, contextual task lists 220 can be configured as tools to help users plan their careers, search for jobs, or strategically manage the job-specific application process. A screenshot shows an example of a job-specific task list, such as a task list containing items like drafting a resume for the job, helping me prepare for an interview, etc. An example of an intent-based task list 220 that is not focused on a specific job is as follows: Suppose a user with the title of technical project manager matches a senior position at company X; the technical assessment described herein relates to entities relevant to the user (e.g., people in the user's network, skills the user has and / or does not have, a comparison of the user's resume with job requirements, etc.) and generates prompts that enable the large language model 206 to create a personalized, strategic task list to assist the user in the overall job search process, rather than the process of seeking a specific job opportunity. In this example, user context data 202 can include user preferences, such as explicit feedback provided by the user (e.g., "There are fewer jobs like this; I want to manage an AI team"). In other embodiments, examples of context task list 220 include task lists that help users accomplish another type of purpose or goal, such as planning a wedding, applying to college, managing a project, or organizing a to-do list.

[0144] The example shown in Figure 2 and the accompanying description are provided for illustrative purposes. This disclosure is not limited to the described example. Additional or alternative details and implementations are described herein.

[0145] Figure 3A , Figure 3B , Figure 3C , Figure 3D , Figure 3E , Figure 3F , Figure 3G , Figure 3H , Figure 3I , Figure 3J , Figure 3K , Figure 3L , Figure 3M , Figure 3N , Figure 3O , Figure 3P , Figure 3Q , Figure 3R , Figure 3S , Figure 3T , Figure 3U and Figure 3V The illustration depicts an instance of at least one stream comprising a user interface screen configured to provide user assistance based on a guided generative thread, according to some embodiments of the present invention. Figure 3A , Figure 3B , Figure 3C , Figure 3D , Figure 3E , Figure 3F , Figure 3G , Figure 3H , Figure 3I , Figure 3J , Figure 3K , Figure 3L , Figure 3M , Figure 3N , Figure 3O , Figure 3P , Figure 3Q , Figure 3R , Figure 3S , Figure 3T , Figure 3U and Figure 3V In the user interface shown, for the purposes of this disclosure, specific data that will typically be displayed may be anonymized. For example, in the live example, the actual data will be displayed instead of an anonymous version. For instance, the text “title” will be replaced with the actual title (e.g., software engineer), and “first name and last name” will be replaced with the user’s actual name.

[0146] exist Figure 3A , Figure 3B , Figure 3C , Figure 3D , Figure 3E , Figure 3F , Figure 3G , Figure 3H , Figure 3I , Figure 3J , Figure 3K , Figure 3L , Figure 3M , Figure 3N , Figure 3O , Figure 3P , Figure 3Q , Figure 3R , Figure 3S , Figure 3T , Figure 3U and Figure 3V The user interfaces shown are presented by application software systems, such as user assistance systems, to users who want their computing devices to assist them in performing tasks. In some implementations, the user interfaces are each implemented as web pages, for example, stored on a server or in the user's device's cache, and then loaded onto the user's device's display via the user's device sending a page load request to the server. The icons, selections, and arrangement of the elements shown in the user interface are copyrighted by 2023 LinkedIn Corporation. All rights reserved.

[0147] The graphical user interface control elements (e.g., fields, boxes, buttons, etc.) shown in the screen capture are implemented via software used to construct the user interface screen. Although the screen capture illustrates an example of a user interface screen, such as a visual display of numbers, for example, in an online format or webpage, this disclosure is not limited to online formats or webpage implementations, visual displays, or graphical user interfaces. In other implementations, for example, alternatively or in addition to a graphical user interface, an audio-based user interface is used, including an embedded audio system (e.g., a microphone, speech processing software, and a speaker).

[0148] exist Figure 3A , Figure 3B , Figure 3C , Figure 3D , Figure 3E , Figure 3F , Figure 3G , Figure 3H , Figure 3I , Figure 3J , Figure 3K , Figure 3L , Figure 3M , Figure 3N , Figure 3O , Figure 3P , Figure 3Q , Figure 3R , Figure 3S , Figure 3T , Figure 3U and Figure 3V The user interface shown provides an example of the capabilities of a user-assisted system based on guided generative threads, as described herein, including the ability to provide personalized recommendations to the user within the context of an online conversation. The multi-threaded online conversation mechanism described herein allows the user to further interact with the user-assisted system in the conversation on relevant or subsequently relevant topics. For example, once the user-assisted system has identified a position of potential interest to the user, the online conversation can move to the topic of whether the user's skill set is a good match for that position. Subsequently, within the same conversation, once the evaluation of the user's skill set against the job requirements has been completed, another thread can cover the topic of preparing a job application or updating the user's resume.

[0149] In some implementations, different context sources can be used in different ways across different threads of an online conversation. For example, in one thread, the recommender system can obtain a set of job recommendations included in prompts, such that a large language model can generalize the set of job recommendations based on the user's skills, etc. In another thread, a user profile (e.g., a webpage) can be obtained from, for example, a social networking service or other application software system, and then the user profile can be included in prompts, such that a large language model can generate recommendations based on the user profile.

[0150] In some implementations, the output generated by one or more generative models of a user-assisted system based on a guided generative thread is generated using a specific intonation, voice, or style. For example, some implementations maintain a library of intonations, voices, or styles accessible to the generative model, and planned execution prompts can include instructions that cause the generative model to select an intonation, voice, and / or style from a library that matches user context data. Examples of intonations that can be included in the intonation library include friendly, informative, perceptive, responsive, encouraging, collaborative, etc. For example, a prompt template can be developed for planned execution prompts, which includes instructions for causing the generative model to: obtain feedback on previously generated output; calculate an evaluation metric indicating how appropriate the intonation, voice, or style used to generate the previous output, based on user feedback; and then select a new intonation, voice, or style if the value of the evaluation metric falls below a threshold determined based on the requirements or design of the specific implementation. The generative model can be instructed to calculate similar evaluation metrics and use them in a similar manner for other aspects of the generative model output, such as relevance, accuracy, completeness, and personalization.

[0151] exist Figure 3A In the user interface 300, a system-generated thread portion 302 is displayed. This system-generated thread 302 is machine-generated using the techniques described herein to mention the user's name and invite the user to interact with the user-assistance system in a conversational tone. The user interface 300 also includes a user interface control mechanism 303. The choice of the user interface control mechanism 303 leads to… Figure 3B The user interface 304 conversion.

[0152] exist Figure 3B In response to the selection of user interface control mechanism 303, user interface 304 uses portions of the techniques described herein to display another system-generated thread portion that has already been generated. The system-generated thread portion of user interface 304 includes user profile data 305 associated with the user, which has been retrieved via a context source (e.g., an application software system) and incorporated into the system-generated thread portion.

[0153] exist Figure 3C In this document, user interface 306 shows another system-generated thread portion that has been generated using parts of the techniques described herein. For example, the system-generated thread portion of user interface 306 is generated by applying a large language model to prompt words that include instructions for generating and outputting natural language text that provides career development assistance using collaborative intonation.

[0154] exist Figure 3D In the diagram, user interface 307 displays another system-generated thread portion that has been generated using parts of the techniques described herein. For example, the system-generated thread portion of user interface 307 includes a question and a set 308 of selectable actions generated by applying a large language model to prompt words, which include instructions for generating and outputting natural language text that asks the user for information about their current goal. The selection of the "active search" selectable action 308 leads to… Figure 3E The user interface 309 conversion.

[0155] exist Figure 3E In the user interface 209, another system-generated thread section 310 is shown, which has been generated using the techniques described herein. For example, the system-generated thread section 310 includes a dialogue explanation and a question 311, generated by applying a large language model to prompts that include instructions for generating and outputting natural language text that asks the user about the type of job they are looking for. The question 311 includes user-specific details already obtained from one or more contextual sources, such as stored thread history or the user's job search history. The user interface 209 also includes a set 312 of selectable template user responses, input boxes 313 configured to receive user input (e.g., text input or speech input converted to text by the system), and user input devices such as a keyboard or keypad 314.

[0156] exist Figure 3F In the present invention, user interface 315 displays a user-generated thread portion 316 and another system-generated thread portion 317. In response to user-generated thread portion 316, system-generated thread portion 317 is partially generated using the techniques described herein. For example, system-generated thread portion 317 of user interface 315 includes a question 318 generated by applying a large language model to prompts that include the job entity (Tax Consulant) mentioned in the previous thread portion 311, since user-generated thread portion 316 includes an affirmative response to the previous thread portion (e.g., question 311). System-generated thread portion 317 of user interface 315 also includes a user-personalized job assessment 319. User-personalized job assessment 318 is machine-generated and output by the large language model based on execution prompts supplied to the large language model, the execution prompts containing instructions for comparing the user's experience with the job description associated with the job title mentioned in question 311.

[0157] exist Figure 3GIn the user interface 320, a user-generated thread portion 321 and another system-generated thread portion 322 are displayed. In response to the user-generated thread portion 321 and previous thread history, the system-generated thread portion 322 is generated using techniques described herein. For example, the system-generated thread portion 322 of the user interface 320 includes questions generated by applying a large language model to prompts requesting additional user preferences. The user interface 320 also includes a set 312 of user-selectable response options.

[0158] Figure 3H User interface 324 is shown. User interface 324 illustrates a scenario where the user has not made a selection. Figure 3G Any response option 312, but additional user-specific preferences 325 can be entered into the input box using an input mechanism such as a keypad or microphone.

[0159] exist Figure 3I In the user interface 326, a user-generated thread section 327 is displayed, which includes preferences 325, another system-generated thread section 328, and user-selected options 329. In response to the user-generated thread section 327 and previous thread history, a system-generated thread section 328 has been partially generated using the techniques described herein. For example, the system-generated thread section 328 of the user interface 326 includes questions generated by applying a large language model to prompts requesting additional user preferences, and also includes user preferences contained in the user-generated thread section 327, which can be retrieved from the thread history.

[0160] exist Figure 3J In the user interface 330, a system-generated thread portion is displayed, which includes a link to job posting 331, and system-generated thread portions 332 and 333 containing an LLM-generated user-personalized profile of job posting 331. System-generated thread portions 332 and 333 are generated by applying a large language model to cue words that instruct the large language model to summarize job posting 331 based on user preferences contained in the thread history. In generating system-generated thread portions 332 and 333, one or more contextual sources, such as social networking services and rating systems, are invoked. For example, a social networking service is invoked to determine how many connections the user has with the company that posted the job. User interface 330 also includes an evaluation user interface control mechanism 334 and a set of user-selectable options 312. The set of user-selectable options is dynamically updated as thread classification changes.

[0161] exist Figure 3KIn the interface 335, the user interface displays thread category labels 336, user-generated thread portions 337, system-generated thread portions 338, and a set of user-selectable options 312. The thread category labels 336 are dynamically generated using disclosed techniques, such as generating thread category prompts based on thread history and applying a large language model to the prompts. The system-generated thread portions 338 are generated by applying a large language model to prompts that instruct the large language model to respond to the user-generated thread portions 337.

[0162] exist Figure 3L In the user interface 339, a system-generated thread section is displayed, which includes a link to job posting 340 (a different job posting from the previously displayed one), and user-personalized summaries 341 and 342 of job posting 340. The user-personalized summary is dynamically generated using disclosed techniques, such as generating a plan execution prompt that includes parameter values ​​retrieved from one or more context sources and applying a large language model to the plan execution prompt. User interface 339 also includes a system-generated thread section 343, which is based on thread history (e.g., user-generated thread section 337). In response to the user's rejection of the first job recommendation (Senior Tax Consulant, too much travel), the user-selectable options 312 are dynamically updated.

[0163] exist Figure 3M In the interface 344, the user interface displays inputs for user-generated thread section 346, system-generated thread section 348 in response to user-generated thread section 346, and user-generated thread section 349. The user interface 344 also displays a thread category label 345. Thread category label 345 differs from thread category label 336 because the system has determined that the thread topic has changed from Senior Tax Consulant to Senior Tax Advisor.

[0164] exist Figure 3N In the interface 350, the user interface 350 displays a user-generated thread section 351, a system-generated thread section 352 that responds to the user-generated thread section 351, and a thread category label 345.

[0165] exist Figure 3O In the user interface 353, a system-generated thread portion is displayed, which includes a link to the job posting 354, and user-personalized summaries of the job posting 355 and 356. The system-generated thread portion of the user interface 353 is generated using disclosed techniques, such as applying a large language model to a plan execution prompt that includes one or more parameter values ​​obtained from one or more context sources.

[0166] exist Figure 3P In the user interface 357, the system-generated thread portion is displayed, which includes a link to the company profile, a user-personalized summary of the job posting 358, and a job description 359 obtained from the company profile page. The system-generated thread portion of the user interface 353 is generated using disclosed techniques, such as by applying a large language model to a planned execution prompt that includes one or more parameter values ​​obtained from one or more context sources.

[0167] exist Figure 3Q In the user interface 360, a system-generated thread portion 361 is displayed. The system-generated thread portion 361 is generated using disclosed techniques, such as by applying a large language model to a planned execution prompt, which instructs the large language model to generate natural language output asking the user whether they want the system to generate a personalized assessment based on thread history (e.g., including job postings 358 from job description 359 obtained from a company profile page).

[0168] exist Figure 3R In the interface 362, the user interface displays system-generated thread category labels 363, user-generated thread portions 364, and system-generated thread portions 365. In response to the user-generated thread portion 364, the system-generated thread portion 365 is generated using disclosed techniques, such as by applying a large language model to a planned execution prompt, which instructs the large language model to generate a personalized user assessment of a job posting 358, including information 359 obtained from a company profile page. The planned execution prompt includes, for example, instructions for the large language model to match job postings and company profiles with user preferences obtained from thread history, skills and experiences obtained from the user's profile page, etc.

[0169] exist Figure 3S In the user interface 366, the system-generated thread category label 367 and the system-generated thread portion 368 are displayed. The system-generated thread portion 368 is generated using disclosed techniques, such as by applying a large language model to a planned execution prompt word that instructs the large language model to generate the problem.

[0170] exist Figure 3TIn the user interface 369, a system-generated thread category label 370, a user-generated thread portion 371, a system-generated thread portion 372, and a selected user option 373 are displayed. In response to the user-generated thread portion 371, the system-generated thread portion 372 is generated using disclosed techniques, such as applying a large language model to a planned execution prompt, which instructs the large language model to summarize employee reviews for the company associated with job posting 358. For example, the planned execution prompt instructs the large language model to obtain employee reviews from an external application such as a job review site or search engine, and the machine generates and outputs a summary of the employee reviews retrieved from the external application.

[0171] exist Figure 3U In the interface 374, the user interface displays a set 312 of system-generated thread category labels 370, user-generated thread portions 376, system-generated thread portions 378, and user-selectable options. In response to the user-generated thread portion 376, the system-generated thread portion 378 is generated using disclosed techniques, such as by applying a large language model to instruct the large language model to generate a planned execution prompt for applying a personalized list of tasks to the user for job posting 358. For example, the planned execution prompt instructs the large language model to generate and output recommended next steps based on the user's current state (e.g., based on thread history), the match between the user's skills and job requirements, and other potential information from one or more contextual sources.

[0172] exist Figure 3V In the interface 380, the user interface displays a system-generated thread section 381 (e.g., a job posting) and a task list 382 generated in response to previous thread history (e.g., thread sections 376, 378). Task list 382 includes tasks 383 and 386. Each task 383 has a task description (e.g., task description 385) and a checkbox 384. Task list 382 is generated using disclosed techniques, such as by applying a large language model to a planned execution prompt that instructs the large language model to generate a personalized task list for the user applied to job posting 381. For example, the planned execution prompt instructs the large language model to generate, output, and rank or prioritize recommendations for the next step based on the user's current state (e.g., based on thread history), the match between the user's skills and job requirements, and other potential information obtained from one or more contextual sources. For example, the system has obtained information from a social networking service indicating that the user has a connection to the company that posted job 381. Therefore, the requirement for referral task 383 is ranked higher than the requirement for updating your profile task 386, which makes the requirement for referral task 383 appear higher than the requirement for updating your profile task 386 in task list 382.

[0173] exist Figure 3A , Figure 3B , Figure 3C , Figure 3D , Figure 3E , Figure 3F , Figure 3G , Figure 3H , Figure 3I , Figure 3J , Figure 3K , Figure 3L , Figure 3M , Figure 3N , Figure 3O , Figure 3P , Figure 3Q , Figure 3R , Figure 3S , Figure 3T , Figure 3U and Figure 3V The user interface flow diagram shown illustrates an example of a multithreaded online dialogue and how the disclosed techniques can constrain the operation of a large language model within the context of a multithreaded online dialogue, including performing dynamic thread classification and tagging.

[0174] exist Figure 3A , Figure 3B , Figure 3C , Figure 3D , Figure 3E , Figure 3F , Figure 3G , Figure 3H , Figure 3I , Figure 3J , Figure 3K , Figure 3L , Figure 3M , Figure 3N , Figure 3O , Figure 3P , Figure 3Q , Figure 3R , Figure 3S , Figure 3T , Figure 3U and Figure 3V The examples described above are provided for illustrative purposes. For example, although the examples are illustrated as user interface screens with a small form factor for devices such as smartphones, tablets, and wearable devices, the user interface can be configured for other forms of electronic devices, such as desktop computers and / or laptops. This disclosure is not limited to the described examples. Additional or alternative details and implementations are described herein.

[0175] Figure 4A and Figure 4B The illustration shows an example of at least one stream of screen capture, which includes a user interface screen configured to provide user assistance based on a guided generative thread, according to some embodiments of the present disclosure.

[0176] Figure 4A and Figure 4B The diagram illustrates a user interface flow or sequence that can be presented to the user to assist them by generating and outputting one or more custom task lists through a machine. Figure 4A and Figure 4B Each diagram in the document illustrates an example of a user interface screen that can be used to manage tasks associated with job searches using the guided generative threading technique described in this document.

[0177] exist Figure 4A and Figure 4B In the user interface shown, for the purposes of this disclosure, specific data that will typically be displayed may be anonymized. In the live example, the actual data will be displayed instead of the anonymized version. For example, the text “title” will be replaced with the actual title (e.g., software engineer), and “first name and last name” will be replaced with the user’s actual name.

[0178] exist Figure 4A and Figure 4B The user interfaces shown are presented by application software systems, such as user assistance systems, to users who want to use their computing devices for task assistance. In some implementations, the user interfaces are each implemented as web pages, for example, stored on a server or in the user's device cache, and then loaded onto the user's device display via the user's device sending a page load request to the server. The icons, selection, and arrangement of the elements shown in the user interface are copyrighted 2023 LinkedIn Corporation. All rights reserved.

[0179] The graphical user interface control elements (e.g., fields, boxes, buttons, etc.) shown in the screen capture are implemented via software used to construct the user interface screen. Although the screen capture shows examples of user interface screens, such as visual displays of numbers, e.g., online formats or web pages, this disclosure is not limited to online formats or web page implementations, visual displays, or graphical user interfaces. In other implementations, for example, automated chatbots are used instead of filler forms, where the chatbot requests user input of requested information via text and / or spoken audio received via a microphone embedded in a computing device, in a conversational, natural language dialogue, or message-based format.

[0180] Figure 4AThe illustration shows an example of a screen capture of a user interface 402 that displays elements of a machine-generated task list using one or more large language models. The task list of user interface 402 includes tasks 404, tasks 406, and user-selectable options 312. In some implementations, tasks are ranked or color-coded based on data obtained from one or more contextual sources. For example, in some implementations, the task list is generalized (e.g., not related to a specific job or other entity), while in others, the task list is entity-specific.

[0181] Figure 4B The illustration shows an example of a screen capture of a user interface 408 that displays elements of a machine-generated task list using one or more large language models. The task list of user interface 408 includes several different tasks, including tasks 409 and 410. In some implementations, tasks are ranked or color-coded based on data obtained from one or more context sources. For example, in some implementations, items in the task list are ranked or color-coded based on the user's current job search status, which the system determines based on thread history. For example, preparation for interview task 409 can be deactivated if the user has not yet applied for a job, and celebrating your new job task 410 can be deactivated if the user has not yet accepted a job offer.

[0182] Because the data obtained from contextual sources is dynamic, the job-specific task list generated using the disclosed technology is specific to each user-job pair. For example, if the same user applies for two different jobs, the user's job-specific task list will be different for each job because the tasks included in the list can be different and / or the order in which the tasks are ranked can be different. This is because the disclosed technology is able to determine how well a user's background, skills, experience, and preferences match each specific job based on data obtained from one or more contextual sources included in the plan execution prompts applied to it by a large language model.

[0183] For illustrative purposes, the following is provided: Figure 4A and Figure 4B The examples shown are consistent with the description appended above. For instance, although the examples are illustrated as user interface screens with a small form factor for devices such as smartphones, tablets, and wearables, the user interface can be configured for other forms of electronic devices, such as desktop computers and / or laptops. This disclosure is not limited to the described examples. Additional or alternative details and implementations are described herein.

[0184] Figure 5This is a block diagram of a computing system including a user-assisted system based on a guided generative thread, according to some embodiments of the present disclosure.

[0185] exist Figure 5 In one embodiment, the computing system 500 includes one or more user systems 510, a network 520, an application software system 530, a user assistance system 580 based on guided generative threads, a data storage system 550, and an event logging service 570.

[0186] In some implementations, all or at least some components of the guided generative thread-based user assistance system 580 are implemented at the user system 510. For example, the thread-based user assistance interface 514 and the guided generative thread-based user assistance system 580 are implemented directly on a single client device, such that communication between the thread-based user assistance interface 514 and the guided generative thread-based user assistance system 580 occurs on the device itself, without needing to communicate with, for example, one or more servers via the internet. (The dashed line indicates...) Figure 5 The term is used to indicate that all or part of the user-assistance system 580 based on guided generative threads can be implemented directly on the user system 510 (e.g., the user's client device). In other words, both the user system 510 and the user-assistance system 580 based on guided generative threads can be implemented on the same computing device.

[0187] The components of the computing system 500, including a user-assisted system 580 based on a guided generative thread, are described in more detail in this paper.

[0188] User system 510 includes at least one computing device, such as a personal computing device, server, mobile computing device, wearable electronic device, or smart device, and at least one software application that the computing device can execute, such as an operating system or front-end for an online system. Many different user systems 510 can connect to network 520 simultaneously or at different times. Different user systems 510 can contain components similar to those described in conjunction with the illustrated user systems 510. For example, many different end users of computing system 500 can interact simultaneously or at different times with many different instances of application software system 530 through their respective user systems 510.

[0189] User system 510 includes user interface 512. User interface 512 is installed on or accessible by user system 510 via network 520. Embodiments of user interface 512 include a thread-based user assistance interface 514. Thread-based user assistance interface 514 enables user interaction with guided generative thread-based user assistance system 580 and / or application software system 530, including the creation and continuation of online dialogues. For example, thread-based user assistance interface 514 provides a user input mechanism through which guided generative thread-based user assistance system 580 receives user-generated thread portions, and provides an output mechanism through which guided generative thread-based user assistance system 580 electronically transmits system-generated thread portions to the user.

[0190] The thread-based user assistance interface 514 includes, for example, a graphical display screen, which includes graphical user interface elements, such as at least one input box or other input mechanism and at least one slot. As used herein, a slot refers to space on a graphical display (such as a web page or mobile device screen) into which digital content (such as threads) can be loaded for display to the user. For example, the thread-based user assistance interface 514 may be configured with a scrollable arrangement of variable-length slots simulating online chat or instant messaging sessions. The position and size of specific graphical user interface elements on the screen are specified using a markup language such as HTML (Hypertext Markup Language). On a typical display screen, graphical user interface elements are defined by two-dimensional coordinates. In other implementations, such as virtual reality or augmented reality implementations, a three-dimensional coordinate system may be used to define the slots. Examples of user interface screens that can be included in the thread-based user assistance interface 514 are shown in the screen capture diagrams illustrated in the accompanying drawings and described herein.

[0191] User interface 512 can be used to create, edit, send, view, receive, process, and organize online conversations, including parts of multi-threaded conversations. In some implementations, user interface 512 enables users to upload, download, receive, send, or share other types of digital content items, including posts, articles, comments, and shares, to initiate user interface events, and to view or otherwise perceive output, such as data and / or digital content generated by application software system 530, guided generative thread-based user assistance system 580, and / or content distribution service 538. For example, user interface 512 can include a graphical user interface (GUI), a conversational voice / speech interface, a virtual reality, augmented reality, or mixed reality interface, and / or a haptic interface. User interface 512 includes mechanisms for logging into application software system 530, clicking or tapping GUI user input control elements, and interacting with thread-based user assistance interface 514 and digital content items, such as online conversations and machine-generated threaded parts. Examples of user interface 512 include web browsers, command-line interfaces, and mobile application front-ends. As used herein, user interface 512 can include an application programming interface (API).

[0192] exist Figure 5 In the example, user interface 512 includes a thread-based user assistance interface 514. The thread-based user assistance interface 514 includes a front-end user interface component of a guided generative thread-based user assistance system 580, an application software system 530, or a messaging component of the application software system 530. For example, the thread-based user assistance interface 514 can be directly integrated with other components of any user interface of the application software system 530, rather than as a standalone chatbot or other type of chat feature. For ease of discussion, the thread-based user assistance interface 514 is shown as a component of user interface 512, but access to the thread-based user assistance interface 514 can be limited to a specific user system 510. For example, in some implementations, access to the thread-based user assistance interface 514 is limited to registered users of the guided generative thread-based user assistance system 580 or the application software system 530.

[0193] Network 520 includes an electronic communication network. Network 520 can be implemented on any medium or mechanism that provides for the exchange of digital data, signals, and / or instructions between various components of computing system 500. Examples of network 520 include, but are not limited to: local area network (LAN), wide area network (WAN), Ethernet or Internet, or at least one terrestrial, satellite, or wireless link, or any combination of any number of different network and / or communication links.

[0194] Application software system 530 includes any type of application software system that provides or enables the creation, uploading, and / or distribution of at least one form of digital content (including machine-generated thread portions) between or among user systems, such as user system 510, via user interface 512. In some implementations, a portion of a user-assisted system 580 based on guided generative threads is a component of application software system 530. Components of application software system 530 may include entity graph 532 and / or knowledge graph 534, user connection network 536, content distribution service 538, and search engine 540.

[0195] exist Figure 5 In the example, application software system 530 includes entity graph 532 and / or knowledge graph 534. Entity graph 532 and / or knowledge graph 534 include data organized according to a graph-based data structure that can be traversed via queries and / or indexes to determine relationships between entities. An example of an entity graph is... Figure 6 As shown in the text, as described herein. For example, as referenced... Figure 6 In more detail, entity graph 532 and / or knowledge graph 534 can be used to calculate various types of relationship weights, affinity scores, similarity measures and / or statistical results between, in, or related to entities.

[0196] Entity graphs 532 and 534 include graph-based representations of data stored in the data storage system 550 described herein. For example, entity graphs 532 and 534 represent entities, such as users, organizations (e.g., companies, schools, institutions), and content items (e.g., job postings, announcements, articles, comments, and shares), as nodes in the graph. Entity graphs 532 and 534 represent relationships (also referred to as mappings or links) between or within entities as edges or combinations of edges between nodes in the graph. In some implementations, mappings between different data blocks used by the application software system 530 are represented by one or more entity graphs. In some implementations, edges, mappings, or links indicate online interactions or activities associated with the entities connected by the edges, mappings, or links. For example, if a user applies for a job, an edge connecting the user entity to the job entity in the entity graph can be created, where the edge can be labeled using a tag such as "apply".

[0197] Parts of entity diagrams 532 and 534 can be automatically regenerated or updated from time to time based on changes and updates to the stored data (e.g., updates to entity data and / or activity data). Similarly, entity diagrams 532 and 534 can refer to the entire system-wide entity diagram or only a portion of the system-wide diagram. For example, entity diagrams 532 and 534 can refer to a subset of the system-wide diagram, wherein the subset is related to a specific user or user group of application software system 530.

[0198] In some implementations, knowledge graph 534 is a subset or superset of entity graph 532. For example, in some implementations, knowledge graph 534 includes multiple different entity graphs 532 joined by edges across applications or domains. For example, knowledge graph 534 can incorporate entity graphs 532 that have been created across multiple different databases or across different software products. In some implementations, entity nodes in knowledge graph 534 represent concepts such as product surfaces, verticals, or application domains. In some implementations, knowledge graph 534 includes a platform for extracting and storing different concepts that can be used to establish links between data across multiple different software applications. Examples of concepts include topics, industries, and skills. Knowledge graph 534 can be used to generate and derive content and entity-level embeddings that can be used to discover or infer new interrelationships between entities and / or concepts, and then these new interrelationships can be used to identify related entities. Like other parts of entity graph 532, knowledge graph 534 can be used to calculate various types of relationship weights, affinity scores, similarity measures, and / or statistical relevance between or among entities and / or concepts.

[0199] Knowledge graph 534 comprises a graph-based representation of the data stored in the data storage system 550 described herein. Knowledge graph 534 represents relationships between entities or concepts as edges or combinations of edges between nodes in the graph, also referred to as links or mappings. In some implementations, mappings between different data blocks used by application software system 530 or across multiple different application software systems are represented by knowledge graph 534.

[0200] User connection network 536 includes, for example, social networking services, specialized social networking software, and / or other social graph-based applications. Content distribution service 538 includes, for example, chatbots or chat-like systems, messaging systems (such as peer-to-peer messaging systems that enable the creation and exchange of messages between users of application software system 530), or news feeds. Search engine 540 includes a search engine that enables users of application software system 530 to enter and execute search queries on user connection network 536 and / or entity graph 532 and / or knowledge graph 534. In some implementations, one or more parts of thread-based user assistance interface 514 and / or guided generative thread-based user assistance system 580 communicate bidirectionally with search engine 540. Application software system 530 can include, for example, online systems providing social networking services, general-purpose search engines, specialized search engines, messaging systems, content distribution platforms, e-commerce software, enterprise software, or any combination of the foregoing or other types of software.

[0201] In some implementations, the front-end of application software system 530 can operate within user system 510, for example, as a plugin or component in a graphical user interface of a web application or mobile application, or as a web browser executing user interface 512. In embodiments, the mobile application or web browser of user system 510 can send network communications (such as HTTP requests) on network 520 in response to user input received through a user interface (such as user interface 512) provided by the web application, mobile application, or web browser. A server running application software system 530 can receive input from the web application, mobile application, or browser executing user interface 512, use the input to perform at least one operation, and return output to user interface 512 using network communications such as HTTP responses, which the web application, mobile application, or browser receives and processes at user system 510.

[0202] exist Figure 5 In the example, application software system 530 includes a content delivery service 538. Content delivery service 538 may include data storage services, such as a web server storing digital content items, and send digital content items to users within an online conversation operated by a user assistance system 580 based on guided generative threads. Alternatively or additionally, the user assistance system 580 based on guided generative threads may interface with one or more components or services of content delivery service 538, such as one or more recommendation models (e.g., content you might be interested in, people you might know, etc.) to obtain information that can be included in a system-generated thread portion of the online conversation operated by the user assistance system 580 based on guided generative threads.

[0203] In some embodiments, content delivery service 538 processes requests from, for example, application software system 530 and / or user assistance system 580 based on guided generative threads, and distributes digital content items to user system 510 in response to the requests. Requests include, for example, network messages, such as HTTP (Hypertext Transfer Protocol) requests, for transferring data from an application frontend to an application backend, or from an application backend to a frontend, or more generally, requests for transferring data between two different devices or systems, such as data transfer between a server and a user system. Requests are made, for example, by a browser or mobile application on the user device in conjunction with user interface events such as login, clicking a graphical user interface element, or page load. In some implementations, content delivery service 538 is part of application software system 530 or user assistance system 580 based on guided generative threads. In other implementations, content delivery service 538 interfaces with application software system 530 and / or user assistance system 580, for example, via one or more application programming interfaces (APIs).

[0204] exist Figure 5 In the example, application software system 530 includes search engine 540. Search engine 540 is a software system designed to search and retrieve information by performing queries on data storage such as databases, connected networks, and / or graphs. The queries are designed to find information that matches specified criteria such as keywords and phrases. For example, search engine 540 is used to retrieve data 534 by performing queries on various data storage systems of data storage system 550 or by traversing entity graph 532.

[0205] Based on input received via the thread-based user assistance interface 514 and / or other data sources, the guided generative thread-based user assistance system 580 uses one or more large language models to operate online dialogues with the user of the application software system 530 and / or the user assistance system 580. In some implementations, the guided generative thread-based user assistance system 580 generates thread classification prompts, thread classifications, planned execution prompts, thread parts, and thread labels based on various forms of input data, including user-generated thread portions, thread history, and / or context sources. In various embodiments, additional or alternative features and functionalities of the guided generative thread-based user assistance systems described herein (such as guided generative thread-based user assistance system 102) are included in the guided generative thread-based user assistance system 580.

[0206] Event logging service 570 captures and records network activity data generated during the operation of application software system 530 and / or user assistance system 580 based on guided generative threads, including user interface events generated in real time at user system 510 via user interface 512, and formulates user interface events into data streams that can be consumed by, for example, a stream processing system. Examples of network activity data include thread creation, thread editing, thread viewing, page loading, clicking on messaging or graphical user interface control elements, creating, editing, sending, and viewing messages, and social action data such as likes, shares, comments, and social reactions (e.g., "insightful," "curious," etc.). For example, when a user starts a thread or clicks a user interface element (such as a message, link, or user interface control element, such as a view, comment, share, or response button) or uploads a file, or creates a message, loads a webpage, or scrolls through a feed via the application software system 530 of user system 510, the event logging service 570 triggers an event to capture identifiers such as a session identifier, event type, date / timestamp of the user interface event, and possibly other information about the user interface event, such as the impression portal and / or impression channel involved in the user interface event. Examples of impression portals and channels include, for example, device type, operating system, and software platform, such as a web or mobile platform.

[0207] For example, when a user creates a thread segment via the guided generative thread-based user assistance system 580 or reacts to a system-generated thread segment received from the guided generative thread-based user assistance system 580, the event logging service 570 stores the corresponding event data in a log. The event logging service 570 generates a data stream that includes records of real-time event data for each user interface event that has occurred. The event data recorded by the event logging service 570 can be preprocessed and anonymized as needed, making it usable for, for example, generating relationship weights, affinity scores, similarity metrics, and / or developing training data for artificial intelligence models.

[0208] Data storage system 550 includes data storage and / or data services for storing digital data received, used, manipulated, and generated by application software system 530 and / or user-assisted system 580 based on guided generative threads, including thread classification prompts, planned execution prompts, user-generated threads, system-generated threads, thread metadata, attribute data, activity data, machine learning model training data, machine learning model parameters, and machine learning model inputs and outputs, such as machine-generated classification and machine-generated score data.

[0209] exist Figure 5In the example, data storage system 550 includes entity data storage 552, activity data storage 554, cue word data storage 556, thread data storage 558, and large language model (LLM) data storage 560. Entity data storage 552 stores data, such as profile data, relating to users, companies, positions, and other entities used by the guided generative thread-based user assistance system 580 for example, generating cue words, generative thread portions, and / or calculating weights, statistics, similarity measurements, or scores. Activity data storage 554 stores data relating to network activity, such as user interface event data extracted from application software system 530, thread-based user assistance interface 514, and / or guided generative thread-based user assistance system 580 via event logging service 570, which is used by the guided generative thread-based user assistance system 580 for example, generating cue words, generating thread portions, and / or calculating weights, statistics, similarity measures, or scores.

[0210] The prompt word data store 556 stores prompt word templates and / or prompt words generated and output by one or more components of the guided generative thread-based user assistance system 580, including thread classification prompt words and planned execution prompt words. The thread data store 558 stores online dialogues, threads, or thread segments, including machine-generated thread segments generated by one or more large language models of the guided generative thread-based user assistance system 580, related metadata, and related data, such as thread context data obtained from one or more context sources. The LLM data store 560 stores data that can be used to configure, train, or tune one or more large language models of the guided generative thread-based user assistance system 580.

[0211] In some embodiments, data storage system 550 includes multiple different types of data storage and / or distributed data services. As used herein, a data service may refer to a physical, geographical grouping of machines, a logical grouping of machines, or a single machine. For example, a data service may be a data center, a cluster, a group of clusters, or a machine. The data storage of data storage system 550 can be configured to store data generated by real-time and / or offline (e.g., batch) data processing. A data storage configured for real-time data processing can be referred to as a real-time data storage. A data storage configured for offline or batch data processing can be referred to as an offline data storage. Data storage can be implemented using databases such as key-value stores, relational databases, and / or graph databases. Data can be written to and read from the data storage using query techniques such as SQL or NoSQL.

[0212] A key-value database or key-value store is a non-relational database that organizes and stores data records as key-value pairs. A key uniquely identifies a data record; that is, the value associated with the key. The value associated with a given key can be, for example, a single data value, a list of data values, or another key-value pair. For example, the value associated with a key can be data identified by the key or a pointer to that data. A relational database defines a data structure as a table or a set of tables in which data is stored in rows and columns, where each column of the table corresponds to a data field. Relational databases use keys to create relationships between data stored in different tables, and keys can be used to join data stored in different tables. Graph databases use graph data structures that include multiple interconnected graph primitives to organize data. Examples of graph primitives include nodes, edges, and predicates, where nodes store data, edges create relationships between two nodes, and predicates are assigned to edges. A predicate defines or describes the type of relationship that exists between nodes connected by an edge.

[0213] The data storage system 550 resides on at least one persistent and / or volatile storage device that can reside within the same local network as at least one other device of the computing system 500 and / or in a network remote relative to at least one other device of the computing system 500. Therefore, although depicted as being included in the computing system 500, a portion of the data storage system 550 can be part of the computing system 500 or accessed by the computing system 500 via a network such as network 520.

[0214] Although not specifically shown, it should be understood that any of the user system 510, application software system 530, guided generative thread-based user assistance system 580, data storage system 550, and event logging service 570 includes an interface embodied as computer program code stored in computer memory, which, when executed, enables the computing device to communicate bidirectionally with any other of the user system 510, application software system 530, guided generative thread-based user assistance system 580, data storage system 550, or event logging service 570 using a communication coupling mechanism. Examples of communication coupling mechanisms include network interfaces, inter-process communication (IPC) interfaces, and application programming interfaces (APIs).

[0215] Each of the user system 510, application software system 530, guided generative thread-based user assistance system 580, data storage system 550, and event logging service 570 is implemented using at least one computing device communicatively coupled to electronic communication network 520. Any one of the user system 510, application software system 530, guided generative thread-based user assistance system 580, data storage system 550, and event logging service 570 can be bidirectionally communicatively coupled to network 520. User system 510 and other different user systems (not shown) can be bidirectionally communicatively coupled to application software system 530 and / or guided generative thread-based user assistance system 580.

[0216] A typical user of user system 510 can be an administrator or end user of application software system 530 or user assistance system 580 based on guided generative threads. User system 510 is configured to communicate bidirectionally with any one of application software system 530, user assistance system 580 based on guided generative threads, data storage system 550, and event logging service 570 via network 520.

[0217] As used herein, terms such as component, system, and model refer to computer-implemented structures, such as combinations of software and hardware, including computer programming logic, data, and / or data structures implemented in circuits, stored in memory, and / or executed by one or more hardware processors.

[0218] The features and functions of user system 510, application software system 530, user assistance system 580 based on guided generative threads, data storage system 550, and event logging service 570 are implemented using computer software, hardware, or a combination of software and hardware, and are capable of including combinations of automated functions, data structures, and digital data schematically shown in the accompanying drawings. User system 510, application software system 530, user assistance system 580 based on guided generative threads, data storage system 550, and event logging service 570 are implemented using computer software, hardware, or a combination of software and hardware, and are capable of including combinations of automated functions, data structures, and digital data schematically shown in the accompanying drawings. Figure 5 The elements are shown as separate elements for ease of discussion, but unless otherwise stated, this illustration does not imply the need to separate these elements. The illustrated systems, services, and data storage (or their functions) of each of the user system 510, application software system 530, user assistance system 580 based on guided generative threads, data storage system 550, and event logging service 570 can be partitioned across any number of physical systems (including a single physical computer system) and can communicate with each other in any suitable manner.

[0219] exist Figure 8In the embodiments described, for ease of discussion, portions of the thread-based user assistance interface 514 and the guided generative thread-based user assistance system 580 are collectively referred to as the guided generative thread-based user assistance system 850. The thread-based user assistance interface 514 and the guided generative thread-based user assistance system 580 do not need to be entirely implemented on the same computing device, in the same memory, or loaded into the same memory simultaneously. For example, access to the thread-based user assistance interface 514 and / or the guided generative thread-based user assistance system 580 can be limited to different, mutually exclusive sets of user systems and / or servers. For example, in some implementations, a separate personalized version of the guided generative thread-based user assistance system 580 is created for each user of the system, such that data is not shared between or among the separate personalized versions of the system 580. Furthermore, the thread-based user assistance interface 514 can typically be implemented on a user system, while the guided generative thread-based user assistance system 580 can typically be implemented on a server computer or a group of servers. However, in some embodiments, one or more portions of the guided generative thread-based user assistance system 580 are implemented on the user system. For example, in some implementations, both the thread-based user assistance interface 514 and the guided generative thread-based user assistance system 580 are implemented on the user system (e.g., a client device). Further details regarding the operation of the guided generative thread-based user assistance system 850 are described herein.

[0220] Figure 6 These are examples of entity diagrams based on some embodiments of this disclosure.

[0221] According to some embodiments of this disclosure, entity graph 600 can be used by application software systems (e.g., social networking services) to support user network connections. Entity graph 600 can be used (e.g., queried or traversed) to obtain or generate thread context data, which can be used to formulate model inputs for large language models of user-assisted systems based on guided generative threads.

[0222] Entity graph 600 includes nodes, edges, and data associated with the nodes and / or edges (such as labels, weights, or scores). Nodes can be weighted based on, for example, similarity to other nodes, edge counts, or other types of calculations, and can be weighted based on, for example, affinity, relationship, activity, similarity, or commonalities between nodes connected by edges, such as common attribute values ​​(e.g., two users have the same job title or employer, or two users are in a user connection network). n Degree connection, where, n (where the integer is a positive integer) is used to weight the edges.

[0223] The drawing mechanism is used to create, update, and maintain entity diagrams. In some implementations, the drawing mechanism is a component of the database schema for implementing entity diagram 600. For example, the drawing mechanism can be... Figure 5 The data storage system 550 and / or application software system 530 shown are components, and the entity graph created by the drawing mechanism can be stored in one or more data storages of the data storage system 550.

[0224] Entity graph 600 is dynamic (e.g., continuously updated) because it is updated in response to the occurrence of interactions between entities in an online system (e.g., a user connection network) and / or the computation of new relationships between or within the nodes of the graph. These updates are achieved through real-time data ingestion and storage techniques, or through offline data extraction, computation, and storage techniques, or a combination of real-time and offline techniques. For example, entity graph 600 is updated in response to updates to user profiles, the creation or deletion of user connections with other users, and the creation and distribution of new content items such as messages, posts, articles, comments, and shares. As another example, entity graph 600 is updated when new computations are performed, such as when new relationships between nodes are created based on statistical correlation or machine learning model outputs.

[0225] Entity graph 600 includes a knowledge graph containing cross-application links. For example, thread context data obtained from one or more context sources can be linked to entities and / or edges in the entity graph.

[0226] exist Figure 6 In the example, entity graph 600 includes entity nodes that represent entities, such as content item nodes (e.g., post U21, article 1), user nodes (e.g., user 1, user 2, user 3, user 4), and job node (e.g., job 1, job 2). Entity graph 600 also includes attribute nodes that represent the attributes of entities (e.g., job title data, article title data, skill data, topic data). Examples of attribute nodes include title nodes (e.g., title U1, title A1), company nodes (e.g., company 1), topic nodes (topic 1, topic 2), and skill nodes (e.g., skill A1, skill U11, skill U31, skill U41).

[0227] Entity graph 600 also includes edges. These edges individually and / or collectively represent various types of relationships between or among nodes. Data can be linked to both nodes and edges. For example, when stored in a data store, each node is assigned a unique node identifier, and each edge is assigned a unique edge identifier. The edge identifier can be, for example, a combination of the node identifier of the node connected by the edge and a timestamp indicating the date and time the edge was created. For example, in graph 600, an edge between user nodes can represent an online social connection between users represented by nodes, such as a "friend" or "follower" connection between connected nodes. As an example, in entity graph 600, user 3 is a first-degree connection of user 1 via a connection edge between user 3 and user 1, while user 2 is a second-degree connection of user 3, although user 1 and user 2 have different types of connections (follows) than user 3.

[0228] In entity graph 600, edges can represent activities involving entities represented by nodes connected by the edges. For example, the POSTED edge between user node 2 and post node U21 indicates that the user represented by user node 2 publishes the digital content item represented by post node U21 to the application software system (e.g., as a job posting published to the user's network). As another example, the SHARED edge between user node 1 and post node U21 indicates that the user represented by user node 1 shares the content item represented by post node U21. Similarly, the CLICKED edge between user node 3 and article node 1 indicates that the user represented by user node 3 clicks on the article represented by article node 1, and the LIKED edge between user node 3 and comment node U1 indicates that the user represented by user node 3 likes the content item represented by comment node U1.

[0229] In some implementations, combinations of nodes and edges are used to calculate various scores, and these scores are used by various components of a user-assisted system based on guided generative threads, such as generating thread classification prompts, generating thread classifications, selecting execution plans, generating plan execution prompts, and / or generating thread components. For example, the following can be used to calculate a score measuring the affinity of a user (represented by the user 1 node) for a position (represented by the position 2 node): path p1 It includes the sequence of edges between node user1, post U21, and post 2; and / or paths. p2 It includes the sequence of edges between node user1, comment U1, and position 2; and / or paths. p3 It includes the sequence of edges between nodes user1, user2, postU21, and post2; and / or paths. p4It includes the sequence of edges between nodes User 1, User 3, Job 1, Company 2, and Job 2. Path p1 , p2 , p3 , p4 And / or any one or more of the other paths in Figure 600 can be used to calculate scores representing affinity, relationship, or statistical correlation between different nodes. For example, based on relative edge counts, the user-job affinity score calculated between user U1 and job 2 may be higher than the user-job affinity score calculated between user U4 and job 2. Similarly, the user-skill affinity score calculated between user 3 and skill U31 may be higher than the user-skill affinity score calculated between user 3 and skill U11. As another example, the job-skill affinity score calculated between job 1 and skill U31 may be higher than the job-skill affinity score calculated between job 1 and skill U41.

[0230] exist Figure 6 In this context, the entity graph includes multiple subgraphs, such as subgraph A, subgraph B, subgraph C, subgraph D, and subgraph E. One or more subgraphs within these subgraphs can be used as thread context data by one or more components of a user-assisted system based on guided generative threads. Figure 6 Each subgraph in the diagram involves a different entity type. For example, subgraph A includes job entities and links related to job entities. Subgraph B includes company entities and links related to company entities. Subgraph C includes content items (e.g., posts, comments, and articles) and links related to content items. Subgraph D includes user entities (e.g., job seekers, recruiters, etc.) and links related to users. Subgraph E includes skills (e.g., skills that can be associated with users and / or jobs) and links related to skills.

[0231] The subgraphs facilitate the efficient determination of relevant thread context data that can be used for thread classification and / or planned execution. For example, in a user-assisted system based on guided generative threads—if a user enters a thread mentioning "Job 1 I looked at yesterday"—the system can search for identifiers for the job in the user's thread history and then search subgraph A for more detailed information about the job, which can be used to generate thread classification prompts. As another example, subgraph B can be used to identify the companies that have posted the job, subgraph D can be used to determine if the user has any connections with that company, and this contextual data can be used to generate planned execution prompts.

[0232] For illustrative purposes, the following is provided: Figure 6 The examples shown and the description attached above. This disclosure is not limited to the examples described.

[0233] Figure 7 This is a flowchart of an example method for using a guided generative thread-based user assistance component in a system, according to some embodiments of the present disclosure.

[0234] Method 700 is executed by processing logic, which includes hardware (e.g., processing device, circuitry, special-purpose logic, programmable logic, microcode, device hardware, integrated circuits, etc.), software (e.g., instructions that run or execute on the processing device), or a combination thereof. In some embodiments, method 700 is executed by the user-assisted system 102 of FIG1 based on guided generative threads or Figure 5 The user-assisted system 580, based on guided generative threads, performs the operation through one or more components. For example, in some implementations, a portion of method 700 is executed by the components described herein. Figure 1A and / or Figure 5 The illustrated user-assisted system is executed using one or more components based on a guided generative thread. Although shown in a specific sequence or order, the order of processes can be modified unless otherwise stated. Therefore, the illustrated embodiments should be understood as examples only, and the illustrated processes can be executed in different orders, and some processes can be executed in parallel. Furthermore, at least one process can be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are also possible.

[0235] At operation 702, the processing device generates first-thread classification prompts based on a first-thread portion of an online conversation involving a user of the computing device. For example, operation 702 formulates classification prompts that can be used to classify the first-thread portion. Operation 702, for example, is performed by... Figure 1A The thread classification prompt generator 104 shown and described herein is used to perform this.

[0236] In some implementations, operation 702 includes sending a first thread portion to a first large language model and receiving a tagged version of the first thread portion. For example, operation 702 may include tagging individual words or phrases of the first thread portion using entity tags and / or activity tags. In some implementations, the tagged version of the first thread portion includes entity data associated with the first thread portion. In some implementations, in response to receiving the first thread portion, entity data is generated and output by the first large language model based on user-associated data retrieved from one or more stored threads, one or more data sources, one or more entity connection graphs, one or more domain applications, and / or one or more recommendation systems. In some implementations, the online dialogue includes multiple different natural language threads.

[0237] In some implementations, operation 702 includes retrieving a stored classification template based on a tagged version of the first thread portion. For example, operation 702 can use the tagged version of the first thread portion to identify thread classification templates that match one or more labels already applied to the first thread portion by the first language model. In some implementations, the retrieved classification template includes at least one instruction to be executed by the first language model. In some implementations, operation 702 also includes the retrieved classification template and the retrieved data in the first thread classification prompt.

[0238] In some implementations, operation 702 includes sending the online conversation to a first language model. For example, operation 702 could include sending an electronic document to the first language model, the electronic document comprising the entire history of all threads of the online conversation involving the user during a time period. As used herein, "sending" can refer to any mechanism by which an AI model (such as a first language model) can be applied to one or more inputs (such as one or more parts of the online conversation). For example, sending could include transmitting electronic communications via a network and / or an application programming interface (API).

[0239] In some implementations, operation 702 includes receiving a threaded version of an online dialogue, wherein the threaded version of the online dialogue includes a first thread portion, and the threaded version of the online dialogue is generated and output by a first large language model. For example, operation 702 could include the first large language model dividing the online dialogue into threads, where each thread has a different label. As used herein, "receive" can refer to any mechanism by which one or more outputs generated by an AI model (such as the first large language model) are obtained from the AI ​​model. For example, "receive" could include transmitting electronic communications via a network and / or an application programming interface (API).

[0240] At operation 704, the processing device sends the first thread classification prompt word to the first large language model. As used herein, "send" can refer to any mechanism by which an AI model (such as the first large language model) can be applied to one or more inputs (such as the first classification prompt word). In some implementations, operation 704 is performed, for example, by the thread classification prompt word generator 104 shown in Figure 1 and described herein.

[0241] At operation 706, the processing device receives a first thread classification, wherein the first thread classification is generated and output by a first large language model based on a first thread classification prompt. For example, operation 706 can include classifying the first thread portion based on the output of the first large language model, wherein the thread classification can include thread topics such as job, company, skill, or activity, for example, job (or other entity) search, resume generation, recommendation generation, etc. As used herein, "receive" can refer to any mechanism by which one or more outputs generated by an AI model (such as the first large language model) are obtained from the AI ​​model. In some implementations, operation 706 is, for example, generated by... Figure 1A The first large language model 108 shown and described in this paper is implemented.

[0242] At operation 708, the processing device formulates a plan execution prompt based at least on the first thread classification. For example, operation 708 may include selecting an execution plan based on the first thread classification of operation 706, and formulating a plan execution prompt based on the selected execution plan. In some implementations, operation 706, for example, is performed by... Figure 1A The plan execution prompt generator 112 shown and described herein is used to execute the plan.

[0243] In some implementations, operation 708 includes formulating a planned execution prompt based on the first thread classification received at operation 706. In some implementations, in response to determining that at least one stored thread involving the user matches the first thread classification, a planned execution prompt is formulated based on the first thread portion and the at least one stored thread that matches the first thread classification.

[0244] In some implementations, operation 708 includes: based on the first thread classification, in response to receiving the first thread portion, retrieving user-related data from one or more stored threads, one or more data sources, one or more entity connection graphs, one or more domain applications, and / or one or more recommendation systems, and including the retrieved data in a planned execution prompt. For example, operation 708 may include retrieving thread context data from one or more context sources, wherein the thread context data can be used to classify the first thread portion.

[0245] In some implementations, operation 708 includes: retrieving a stored plan template based on a first thread classification, wherein the retrieved stored plan template includes one or more instructions to be executed by a second language model, and including the retrieved plan template and the retrieved data in a plan execution prompt. For example, operation 708 can include selecting a stored plan template from a template library and / or applying the retrieved plan template to retrieved data obtained from one or more context sources.

[0246] At operation 710, the processing device sends the planned execution prompt to the second large language model. As used herein, "send" can refer to any mechanism by which an AI model (such as the second large language model) can be applied to one or more inputs (such as the planned execution prompt). In some implementations, operation 710 is performed, for example, by the planned execution prompt generator 112 shown in Figure 1 and described herein.

[0247] At operation 712, the processing device receives a second thread portion, wherein the second thread portion is generated and output by a second large language model at least based on a planned execution prompt. For example, operation 721 can include the second large language model generating and outputting a natural language or multimodal (e.g., text, video, audio, one or more images, etc.) response to the first thread portion, such as one or more job (or other entity) recommendations, job (or other entity) evaluations, or task recommendations. As used herein, "receive" can refer to any mechanism that obtains one or more outputs generated by an AI model (such as a second large language model). In some implementations, the second large language model is a large language model different from the first large language model. In some implementations, operation 712 is, for example, generated by a second large language model. Figure 1A The second largest language model 116 shown and described in this paper is implemented.

[0248] In some implementations, the second thread portion is generated and output by a second language model based on one or more parts of the planned execution prompt and online dialogue. In some implementations, the second thread portion includes one or more recommendation tasks selected, prioritized, and output by the second language model in response to receiving one or more of the first thread portion based on one or more of the planned execution prompt, online dialogue, or user-related data retrieved from one or more stored threads, one or more data sources, one or more entity connection graphs, one or more domain applications, and / or one or more recommendation systems.

[0249] In some implementations, the second thread portion includes an evaluation of the job (or other entity), which is summarized and output by a second language model in response to receiving one or more user-related data from the first thread portion based on planned execution prompts, online dialogues, or data retrieved from one or more stored threads, one or more data sources, one or more entity connection graphs, one or more domain applications, and / or one or more recommendation systems.

[0250] In some implementations, the second thread portion includes entity recommendations or activity recommendations, which are generated and output by a second language model in response to receiving one or more user-related data from the first thread portion based on planned execution prompts, online dialogues, or data retrieved from one or more stored threads, one or more data sources, one or more entity connection graphs, one or more domain applications, and / or one or more recommendation systems.

[0251] At operation 714, the processing device generates tags for third thread segments of the online conversation, wherein the tags are based at least on the first thread classification of operation 706. For example, operation 714 can include dynamically tagging thread segments based on thread classifications generated by a first large language model. In some implementations, operation 714 is, for example, generated by... Figure 1A The thread tag generator 109 shown and described herein is used to perform this.

[0252] In some implementations, the tags are based on one or more parts of a first thread classification and an online conversation. In some implementations, the tags are configured to be displayed on a computing device. In some implementations, the third thread portion includes the first thread portion of operation 702 and the second thread portion of operation 712.

[0253] In some implementations, method 700 also includes dynamically labeling different threads of the online conversation using different labels based on thread classifications generated and output by a large language model. For example, method 700 can include labeling a fourth thread portion of the online conversation based on a second thread classification generated and output by a first large language model.

[0254] In some implementations, one or more of the system-generated thread portions include, alone or in combination with natural language text, any one of video, audio, and / or one or more images. In some implementations, the processing device presents one or more of the system-generated thread portions to a user at a user assistance interface and receives user input in response to one or more of the system-generated thread portions, wherein the user input includes any one of the following: modifications to one or more system-generated thread portions, requests for subsequent system-generated thread portions, or actions in response to one or more of the system-generated thread portions. In some implementations, the processing device configures one or more prompts according to prompt templates selected from a library of prompt templates, wherein the library of prompt templates contains one or more orders of magnitude fewer templates than the user of the user assistance system. In some implementations, the processing device configures one or more prompts to convert one or more of the system-generated thread portions from a first size to a second size, wherein the second size is more efficient for presentation at one or more user devices, or for distribution to one or more user devices via a network. In some implementations, the processing device configures one or more prompt words to generate system-generated thread portions for distribution via a network based on user interaction with system-generated thread portions. These prompt words are configured to trigger one or more generative AI models to formulate the system-generated thread portions for presentation on end-user devices with different screen resolutions, thereby facilitating interaction between the user and the system-generated thread portions and resulting in an improved conversational user assistance system. In some implementations, the processing device detects an increase or decrease in latency in the output of one or more system-generated thread portions and, in response to detecting an increase in latency, reduces the number of input signals, uses one or more generative AI models with a reduced size (e.g., fewer model parameters), uses a more compact prompt word template (e.g., fewer prompt word portions, instructions, or examples), or reduces the size of the system-generated thread portions (e.g., specifies a shorter maximum string length).

[0255] For illustrative purposes, the following is provided: Figure 7 The examples shown and the accompanying descriptions are provided. This disclosure is not limited to the examples described.

[0256] Figure 8 This is a block diagram of an example computer system including components of a user-assisted system based on a guided generative thread, according to some embodiments of the present disclosure.

[0257] exist Figure 8The diagram illustrates an example machine of a computer system 800, within which a set of instructions is capable of executing to cause the machine to perform any of the methods discussed herein. In some embodiments, the computer system 800 may correspond to a component of a networked computer system (e.g., as...). Figure 1A The computing system 100 or Figure 5 The computer system 500 (components), including, coupled to, or utilizing the machine to execute an operating system to perform operations with Figure 1A A user assistance system 102 based on guided generative threads or Figure 5 The operation corresponds to one or more components of the guided generative thread-based user assistance system 580. For example, when the computing system is executing a portion of the guided generative thread-based user assistance system 102 or the guided generative thread-based user assistance system 580, the computer system 800 corresponds to a portion of the computing system 500.

[0258] The machine is connected (e.g., networked) to other machines in a network, such as a local area network (LAN), intranet, extranet, and / or the Internet. The machine is capable of operating within the capacity of a server or client machine in a client-server network environment, as a peer-to-peer (or distributed) network environment, or as a server or client machine in a cloud computing infrastructure or environment.

[0259] A machine is a personal computer (PC), smartphone, tablet PC, set-top box (STB), personal digital assistant (PDA), cellular phone, web device, wearable device, server, or any machine capable of executing a set of instructions (sequential or otherwise) specifying actions to be taken by that machine. Furthermore, although a single machine is illustrated, the term "machine" includes any set of machines that individually or jointly execute a set (or more) of instructions to perform any of the methods discussed herein.

[0260] Example computer system 800 includes processing device 802, main memory 804 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM), etc.), memory 803 (e.g., flash memory, static random access memory (SRAM), etc.), input / output system 810, and data storage system 840, which communicate with each other via bus 830.

[0261] Processing device 802 represents at least one general-purpose processing device, such as a microprocessor, central processing unit, etc. More specifically, the processing device can be a Complex Instruction Set Computing (CISC) microprocessor, a Reduced Instruction Set Computing (RISC) microprocessor, a Very Long Instruction Word (VLIW) microprocessor, or a processor implementing other instruction sets, or a combination of instruction sets. Processing device 802 can also be at least one special-purpose processing device, such as an Application-Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), a network processor, etc. Processing device 802 is configured to execute instructions 812 for performing the operations and steps discussed herein.

[0262] exist Figure 8 In some embodiments, when computer system 800 is executing those portions of guided generative thread-based user assistance system 580, guided generative thread-based user assistance system 850 refers to portions of guided generative thread-based user assistance system 580. When those portions of guided generative thread-based user assistance system 850 are being executed by processing device 802, instruction 812 includes portions of guided generative thread-based user assistance system 850. Therefore, guided generative thread-based user assistance system 850 is shown in dashed lines as part of instruction 812 to illustrate that portions of guided generative thread-based user assistance system 850 are sometimes executed by processing device 802. For example, when at least some portions of guided generative thread-based user assistance system 850 are embodied in instructions to cause processing device 802 to perform the methods described herein, some of these instructions may be read from main memory 804 and / or data storage system 840 into processing device 802 (e.g., read into internal cache or other memory). However, it is not required that all user assistance systems 850 based on guided generative threads be included in instruction 812 at the same time, and at other times, for example, when at least one part of the user assistance system 850 based on guided generative threads is not executed by processing device 802, the part of the user assistance system 850 based on guided generative threads is stored in at least one other component of computer system 800.

[0263] Computer system 800 also includes a network interface device 808 for communication via network 820. The network interface device 808 provides bidirectional data communication coupling to the network. For example, the network interface device 808 can be an Integrated Services Digital Network (ISDN) card, a cable modem, a satellite modem, or a modem to provide data communication connectivity to a corresponding type of telephone line. As another example, the network interface device 808 can be a Local Area Network (LAN) card to provide data communication connectivity to a compatible LAN. Wireless links can also be implemented. In any such implementation, the network interface device 808 can transmit and receive electrical, electromagnetic, or optical signals carrying digital data streams representing various types of information.

[0264] A network link enables data communication to other data devices via at least one network. For example, a network link can provide a connection to a worldwide packet data communication network commonly referred to as the "Internet," such as from a local network to a host computer or a data device operated by an Internet Service Provider (ISP). Local networks and the Internet use electrical, electromagnetic, or optical signals to carry digital data to and from computer systems.

[0265] Computer system 800 is capable of sending messages and receiving data, including program code, through one or more networks and network interface devices 808. In the Internet example, the server is capable of sending application request code through the Internet and network interface device 808. The received code can be executed by processing device 802 upon receipt and / or stored in data storage system 840 or other non-volatile storage device for later execution.

[0266] Input / output system 810 includes output devices, such as displays, like liquid crystal displays (LCDs) or touchscreen displays, for displaying information to a computer user or to speakers, haptic devices, or other forms of output devices. Input / output system 810 may include input devices, such as alphanumeric keys and other keys configured to transmit information and command selections to processing device 802. Alternatively or additionally, input devices may include cursor controls, such as a mouse, trackball, or arrow keys, for transmitting directional information and command selections to processing device 802 and for controlling cursor movement on the display. Alternatively or additionally, input devices may include microphones, sensors, or sensor arrays for transmitting sensed information to processing device 802. For example, sensed information may include voice commands, audio signals, geographic location information, haptic information, and / or digital images.

[0267] Data storage system 840 includes a machine-readable storage medium 842 (also referred to as a computer-readable medium) storing thereon at least one set of instructions 844 or software embodying any of the methods or functions described herein. The instructions 844 may also reside wholly or at least partially within main memory 804 and / or processing device 802 during execution by computer system 800, which also constitute machine-readable storage media. In one embodiment, the instructions 844 include implementations of a user-assisted system 850 corresponding to a guided generative thread-based system (e.g., Figure 1A User assistance system 102 or based on guided generative threads Figure 5 Instructions for the functionality of a user-assisted system (580) based on guided generative threads.

[0268] dotted line Figure 8 The instruction 812 is used to indicate that the user-assistance system based on guided generative threads is not required to be fully embodied in instructions 812, 814, and 844 simultaneously. In one example, a portion of the user-assistance system based on guided generative threads is embodied in instruction 814, which is read into main memory 804 as instruction 814, and a portion of instruction 812 is read into processing device 802 as instruction 812 for execution. In another example, some portions of the user-assistance system based on guided generative threads are embodied in instruction 844, while other portions are embodied in instruction 814, and still others in instruction 812.

[0269] Although the machine-readable storage medium 842 is shown as a single medium in the exemplary embodiment, the term "machine-readable storage medium" should be considered as including a single medium or multiple media comprising storing instructions. The term "machine-readable storage medium" should also be considered as including any medium capable of storing or encoding a set of instructions for machine execution and causing the machine to perform any method of this disclosure. Therefore, the term "machine-readable storage medium" should be considered as including, but not limited to, solid-state memories, optical media, and magnetic media. For illustrative purposes, the following is provided: Figure 8 The examples shown and the description attached above. This disclosure is not limited to the examples described.

[0270] Some parts of the detailed description above have been presented based on the algorithms and symbolic representations of operations on data bits within computer memory. These algorithmic descriptions and representations are the most effective way for those skilled in the art of data processing to communicate their material properties to those skilled in the art. Here, an algorithm is considered as a self-consistent sequence of operations that leads to a desired result. These operations are those that require physical manipulation of physical quantities. Typically, although not always necessary, these quantities take the form of electrical or magnetic signals that can be stored, combined, compared, and otherwise manipulated. It has proven convenient, primarily for common use, to refer to these signals as bits, values, elements, symbols, characters, items, numbers, etc.

[0271] However, it should be remembered that all these and similar terms will be associated with appropriate physical quantities and are merely convenient labels applied to those quantities. This disclosure may refer to the actions and processes of computer systems or similar electronic computing devices that manipulate data represented as physical (electronic) quantities within the registers and memories of the computer system and convert them into other data similarly represented as physical quantities within the computer system's memory or registers or other such information storage systems.

[0272] This disclosure also relates to an apparatus for performing the operations described herein. The apparatus can be specifically configured for the intended purpose, or it can include a general-purpose computer selectively activated or reconfigured by a computer program stored in a computer. For example, a computer system or other data processing system (such as computing system 100 or computing system 500) can perform the computer-implemented methods described above in response to its processor executing a computer program (e.g., a sequence of instructions) contained in memory or other non-transitory machine-readable storage medium. Such a computer program can be stored in a computer-readable storage medium, such as, but not limited to, any type of disk, including floppy disks, optical disks, CD-ROMs and magneto-optical disks, read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic cards or optical cards, or any type of medium suitable for storing electronic instructions, each coupled to a computer system bus.

[0273] The algorithms and displays presented herein are not inherently related to any particular computer or other device. Various general-purpose systems can be used with programs based on the teachings herein, or it may prove convenient to construct more specialized devices to perform the methods. The structures for various such systems will be described below. Furthermore, this disclosure is not directed to any particular programming language. It should be understood that the teachings of this disclosure as described herein can be implemented using various programming languages.

[0274] This disclosure can be provided as a computer program product or software, which can include a machine-readable medium having instructions stored thereon, the instructions being usable for programming a computer system (or other electronic device) to perform processes according to this disclosure. The machine-readable medium includes any mechanism for storing information in a machine-readable (e.g., computer-readable) form. In some embodiments, the machine-readable (e.g., computer-readable) medium includes machine-readable (e.g., computer-readable) storage media, such as read-only memory (“ROM”), random access memory (“RAM”), disk storage media, optical storage media, flash memory components, etc.

[0275] The following provides illustrative examples of the techniques disclosed herein. Embodiments of the techniques may include any of the instances described herein, any combination of any of the instances described herein, or any combination of any part of the instances described herein.

[0276] In Example 1, a method includes: generating a first thread category cue based on a first thread portion of an online conversation involving a user of a computing device; sending the first thread category cue to a first large language model; receiving a first thread category, wherein the first thread category is generated and output by the first large language model based on the first thread category cue; formulating a plan execution cue based on the first thread category, wherein, in response to determining that at least one stored thread involving the user matches the first thread category, the plan execution cue is formulated based on the first thread portion and the at least one stored thread that matches the first thread category; sending the plan execution cue to a second large language model; receiving a second thread portion, wherein the second thread portion is generated and output by the second large language model based on the plan execution cue and the online conversation; and generating a tag for a third thread portion of the online conversation, wherein the tag is configured to be displayed at the computing device, the tag being based on the first thread category, and the third thread portion comprising the first thread portion and the second thread portion.

[0277] Example 2 includes the subject of Example 1, and further includes: labeling a fourth thread portion of the online dialogue based on a second thread classification generated and output by the first large language model, wherein the online dialogue includes multiple natural language threads. Example 3 includes the subject of Example 1 or Example 2, wherein generating the first thread classification prompt includes: sending the first thread portion to the first large language model; and receiving a labeled version of the first thread portion, wherein the labeled version of the first thread portion includes entity data associated with the first thread portion, and the entity data is generated and output by the first large language model in response to receiving the first thread portion based on user-associated data retrieved from at least one of stored threads, data sources, entity connection graphs, domain applications, or recommendation systems. Example 4 includes the subject of Example 3, wherein generating the first thread classification prompt includes: retrieving a stored classification template based on the labeled version of the first thread portion, wherein the retrieved classification template includes at least one instruction to be executed by the first large language model; and including the retrieved classification template and the retrieved data in the first thread classification prompt. Example 5 includes the subject of any one of Examples 1-4, wherein generating the first thread classification prompt includes: sending the online dialogue to the first large language model; and receiving a threaded version of the online dialogue, wherein the threaded version of the online dialogue includes the first thread portion and the threaded version is generated and output by the first large language model. Example 6 includes the subject of any one of Examples 1-5, wherein formulating the plan execution prompt includes: based on the first thread classification, in response to receiving the first thread portion, retrieving data associated with the user from at least one of stored threads, data sources, entity connection graphs, domain applications, or recommendation systems; and including the retrieved data in the plan execution prompt. Example 7 includes the subject of Example 6, wherein formulating the plan execution prompt includes: retrieving a stored plan template based on the first thread classification, wherein the retrieved stored plan template includes multiple instructions to be executed by the second large language model; and including the retrieved plan template and the retrieved data in the plan execution prompt. Example 8 includes the subject of any one of Examples 1-7, wherein the second thread portion includes multiple tasks that are selected, prioritized, and output by the second large language model in response to receiving data associated with the user from the first thread portion based on the planned execution prompt, the online dialogue, and data retrieved from at least one of stored threads, data sources, entity connection graphs, domain applications, or recommendation systems.Example 9 includes the subject of any one of Examples 1-8, wherein the second thread portion includes an evaluation of the job, said evaluation being generated and output by the second language model in response to receiving data from the first thread portion that is generalized and output by the second language model based on the planned execution prompt, the online dialogue, and user-related data retrieved from at least one of stored threads, data sources, entity connection graphs, domain applications, or recommendation systems. Example 10 includes the subject of any one of Examples 1-9, wherein the second thread portion includes a recommendation being generated and output by the second language model in response to receiving data from the first thread portion that is generated and output by the second language model based on the planned execution prompt, the online dialogue, and user-related data retrieved from at least one of stored threads, data sources, entity connection graphs, domain applications, or recommendation systems.

[0278] In Example 11, a system includes: at least one processor; and at least one memory device coupled to the at least one processor, wherein the at least one memory device includes instructions that, when executed by the at least one processor, cause the at least one processor to perform at least one operation including: generating a first thread category prompt based on a first thread portion of an online conversation involving a user of a computing device; sending the first thread category prompt to a first large language model; receiving a first thread category, wherein the first thread category is generated and output by the first large language model based on the first thread category prompt; and formulating a plan to execute the prompt based on the first thread category, wherein, in response to a certain condition... The system determines that at least one stored thread involving the user matches the first thread category, the planned execution prompt is formulated based on the first thread portion and the at least one stored thread matching the first thread category; sends the planned execution prompt to a second large language model; receives a second thread portion, wherein the second thread portion is generated and output by the second large language model based on the planned execution prompt and the online dialogue; and generates a tag for a third thread portion of the online dialogue, wherein the tag is configured for display on the computing device, the tag is based on the first thread category, and the third thread portion includes the first thread portion and the second thread portion.

[0279] Example 12 includes the subject matter of Example 11, wherein, when executed by the at least one processor, the instructions cause the at least one processor to perform at least one of the following operations: sending the first thread portion to the first large language model; receiving a tagged version of the first thread portion, wherein the tagged version of the first thread portion includes entity data associated with the first thread portion, and the entity data is generated and output by the first large language model in response to receiving user-associated data retrieved from at least one of a stored thread, a data source, an entity connection graph, a domain application, or a recommendation system; retrieving a stored classification template based on the tagged version of the first thread portion, wherein the retrieved classification template includes at least one instruction to be executed by the first large language model; and including the retrieved classification template and the retrieved data in the first thread classification prompt. Example 13 includes the subject of Example 11 or Example 12, wherein the instructions, when executed by the at least one processor, cause the at least one processor to perform at least one of the following operations: based on the first thread classification, in response to receiving the first thread portion, retrieving data associated with the user from at least one of stored threads, data sources, entity connection graphs, domain applications, or recommendation systems; including the retrieved data in the plan execution prompt; retrieving a stored plan template based on the first thread classification, wherein the retrieved stored plan template includes multiple instructions to be executed by the second large language model; and including the retrieved plan template and the retrieved data in the plan execution prompt. Example 14 includes the subject of any one of Examples 11-13, wherein the second thread portion includes multiple tasks selected, prioritized, and output by the second large language model in response to receiving the first thread portion based on the plan execution prompt, the online dialogue, and user-associated data retrieved from at least one of stored threads, data sources, entity connection graphs, domain applications, or recommendation systems. Example 15 includes the subject matter of any one of Examples 11-14, wherein the second thread portion includes an evaluation of the position in response to receiving data from the first thread portion that is generalized and output by the second large language model based on a planned execution prompt, online dialogue, and data associated with the user retrieved from at least one of stored threads, data sources, entity connection graphs, domain applications, or recommendation systems. Example 16 includes the subject matter of any one of Examples 11-15, wherein the second thread portion includes a recommendation that is generated and output in response to receiving data from the first thread portion that is generated and output by the second large language model based on the planned execution prompt, the online dialogue, and data associated with the user retrieved from at least one of stored threads, data sources, entity connection graphs, domain applications, or recommendation systems.

[0280] In Example 17, at least one non-transitory machine-readable storage medium includes instructions that, when executed by the at least one processor, cause the at least one processor to perform at least one operation including: generating a first thread category cue based on a first thread portion of an online conversation involving a user of a computing device; sending the first thread category cue to a first large language model; receiving a first thread category, wherein the first thread category is generated and output by the first large language model based on the first thread category cue; formulating a planned execution cue based on the first thread category, wherein, in response to determining that at least one stored thread involving the user matches the first thread category, the planned execution cue is formulated based on the first thread portion and the at least one stored thread matching the first thread category; sending the planned execution cue to a second large language model; receiving a second thread portion, wherein the second thread portion is generated and output by the second large language model based on the planned execution cue and the online conversation; and generating a label for a third thread portion of the online conversation, wherein the label is configured for display at the computing device, the label being based on the first thread category, and the third thread portion comprising the first thread portion and the second thread portion.

[0281] Example 18 includes the subject of Example 17, wherein the second thread portion includes multiple tasks that are selected, prioritized, and output by the second large language model in response to receiving data from the first thread portion that is associated with the user based on the planned execution prompt, the online dialogue, and data retrieved from at least one of stored threads, data sources, entity connection graphs, domain applications, or recommendation systems. Example 19 includes the subject of Example 17 or Example 18, wherein the second thread portion includes job evaluation that is summarized and output by the second large language model in response to receiving data from the first thread portion that is associated with the user based on the planned execution prompt, the online dialogue, and data retrieved from at least one of stored threads, data sources, entity connection graphs, domain applications, or recommendation systems. Example 20 includes the topic of any of Examples 17-19, wherein the second thread portion includes a recommendation that is generated and output in response to receiving data associated with the user from the first thread portion generated by the second large language model based on the planned execution prompt, the online dialogue, and data retrieved from at least one of stored threads, data sources, entity connection graphs, domain applications, or recommendation systems.

[0282] Example 21 includes the subject matter of any of the other examples, wherein one or more system-generated thread portions in the system-generated thread portion include, alone or in combination with natural language text, any one of video, audio, and / or one or more images. Example 22 includes the subject matter of any of the other examples, wherein the processing device presents one or more system-generated thread portions in the system-generated thread portion to a user at a user assistance interface and receives user input in response to one or more system-generated thread portions, wherein the user input includes any one of the following: modifications to the one or more system-generated thread portions, requests for subsequent system-generated thread portions, or actions in response to one or more system-generated thread portions.

[0283] Example 23 includes the theme of any of the other examples, wherein the processing device configures one or more prompts from the prompts based on prompt templates selected from a library of prompt templates, wherein the library of prompt templates contains one or more orders of magnitude fewer templates than the user of the user assistance system. Example 24 includes the theme of any of the other examples, wherein the processing device configures one or more prompts to convert one or more system-generated thread portions from a first size to a second size, wherein the second size is more efficient for rendering at one or more user devices, or for distribution to one or more user devices via a network. Example 25 includes the theme of any of the other examples, wherein the processing device configures one or more prompts based on interactions between a user and a system-generated thread portion to generate a system-generated thread portion for distribution via a network, wherein the one or more prompts are configured to trigger one or more generative AI models to formulate the system-generated thread portion for rendering at end-user devices with different screen resolutions, thereby facilitating interaction between the user and the system-generated thread portion, resulting in an improved session user assistance system.

[0284] Example 26 includes the subject of any of the other examples, wherein the processing device detects an increase or decrease in latency of one or more system-generated thread portions in the output of the system-generated thread portions, and in response to detecting an increase in latency: reduces the number of input signals, or uses one or more generative AI models with a reduced size (e.g., fewer model parameters), or uses a more compact cue word template (e.g., fewer cue word portions, instructions, or examples), or reduces the size of the system-generated thread portions (e.g., specifies a shorter maximum string length).

[0285] In the foregoing specification, embodiments of the present disclosure have been described with reference to specific exemplary embodiments thereof. It will be apparent that various modifications can be made thereto without departing from the broader spirit and scope of the embodiments of the present disclosure as set forth in the appended claims. Therefore, the specification and drawings should be considered exemplary rather than restrictive.

Claims

1. A computer-implemented method, comprising: (702) First-thread category prompts are generated based on the first thread portion of an online conversation involving a user with a computing device; Send the first thread classification prompt (704) to the first large language model; Receive (706) a first thread classification, wherein the first thread classification is generated and output by the first large language model based on the first thread classification prompt words; (708) Plan execution prompt words are formulated based on the first thread category, wherein, in response to determining that at least one stored thread involving the user matches the first thread category, the plan execution prompt words are formulated based on the first thread portion and the at least one stored thread that matches the first thread category; Send the plan execution prompt (710) to the second largest language model; Receive (712) the second thread portion, wherein the second thread portion is generated and output by the second large language model based on the planned execution prompt and the online dialogue; and Generate (714) tags for a third thread portion of the online conversation, wherein the tags are configured to be displayed on the computing device, the tags are based on the first thread classification, and the third thread portion includes the first thread portion and the second thread portion.

2. The computer-implemented method according to claim 1 further includes: The fourth thread portion of the online dialogue is labeled based on the second thread classification generated and output by the first large language model, wherein the online dialogue includes multiple natural language threads.

3. The computer-implemented method according to claim 1, wherein, The generation of the first thread category prompt includes: Send the first thread portion to the first large language model; and Receive a tagged version of the first thread portion, wherein the tagged version of the first thread portion includes entity data associated with the first thread portion, and the entity data is generated and output by the first large language model in response to receiving data associated with the user retrieved from at least one of stored threads, data sources, entity connection graphs, domain applications, or recommendation systems.

4. The computer-implemented method according to claim 3, wherein, The generation of the first thread category prompt includes: Retrieve the stored classification template based on the tag version of the first thread portion, wherein the retrieved classification template includes at least one instruction to be executed by the first large language model; and The retrieved category templates and retrieved data are included in the first thread category suggestions.

5. The computer-implemented method according to claim 1, wherein, The generation of the first thread category prompt includes: Send the online dialogue to the first large language model; and Receive a threaded version of the online dialogue, wherein the threaded version of the online dialogue includes the first thread portion, and the threaded version is generated and output by the first large language model.

6. The computer-implemented method according to claim 1, wherein, The prompts for executing the plan include: Based on the first thread classification, in response to receiving the first thread portion, data associated with the user is retrieved from at least one of the stored threads, data sources, entity connection graphs, domain applications, or recommendation systems; and The retrieved data will be included in the planned execution prompt.

7. The computer-implemented method according to claim 6, wherein, The prompts for executing the plan include: Based on the first thread classification, a stored plan template is retrieved, wherein the retrieved stored plan template includes multiple instructions to be executed by the second large language model; and The retrieved plan template and retrieved data are included in the plan execution prompt.

8. The computer-implemented method according to claim 1, wherein, The second thread portion includes multiple tasks that are selected, prioritized, and output by the second large language model in response to receiving user-related data from the first thread portion, based on the planned execution prompt words, the online dialogue, and data retrieved from at least one of stored threads, data sources, entity connection graphs, domain applications, or recommendation systems.

9. The computer-implemented method according to claim 1, wherein, The second thread portion includes an evaluation of the job, which is in response to receiving data from the first thread portion that is generalized and output by the second large language model based on the planned execution prompt, the online dialogue, and user-related data retrieved from at least one of stored threads, data sources, entity connection graphs, domain applications, or recommendation systems.

10. The computer-implemented method according to claim 1, wherein, The second thread portion includes recommendations that are generated and output in response to receiving user-related data generated and output by the second large language model from the first thread portion based on the planned execution prompts, the online dialogue, and data retrieved from at least one of stored threads, data sources, entity connection graphs, domain applications, or recommendation systems.

11. A system comprising: At least one processor; as well as At least one memory device coupled to the at least one processor, wherein the at least one memory device includes instructions that, when executed by the at least one processor, cause the at least one processor to perform at least one operation including: (702) First-thread category prompts are generated based on the first thread portion of an online conversation involving a user with a computing device; Send the first thread classification prompt (704) to the first large language model; Receive (706) a first thread classification, wherein the first thread classification is generated and output by the first large language model based on the first thread classification prompt words; (708) Develop a planned execution prompt based on the first thread category, wherein, in response to determining that at least one stored thread involving the user matches the first thread category, the planned execution prompt is developed based on the first thread portion and the at least one stored thread that matches the first thread category; Send the plan execution prompt (710) to the second largest language model; Receive (712) the second thread portion, wherein the second thread portion is generated and output by the second large language model based on the planned execution prompt and the online dialogue; and Generate (714) tags for a third thread portion of the online conversation, wherein the tags are configured to be displayed on the computing device, the tags are based on the first thread classification, and the third thread portion includes the first thread portion and the second thread portion.

12. The system according to claim 11, wherein, When the instruction is executed by the at least one processor, it causes the at least one processor to perform at least one of the following operations: Send the first thread portion to the first large language model; Receive a tagged version of the first thread portion, wherein the tagged version of the first thread portion includes entity data associated with the first thread portion, and the entity data is generated and output by the first large language model in response to receiving data associated with the user retrieved from at least one of stored threads, data sources, entity connection graphs, domain applications, or recommendation systems. Retrieve the stored classification template based on the tag version of the first thread portion, wherein the retrieved classification template includes at least one instruction to be executed by the first large language model; and The retrieved category templates and retrieved data are included in the first thread category suggestions.

13. The system according to claim 11, wherein, When the instruction is executed by the at least one processor, it causes the at least one processor to perform at least one of the following operations: Based on the first thread classification, in response to receiving the first thread portion, data associated with the user is retrieved from at least one of the stored threads, data sources, entity connection graphs, domain applications, or recommendation systems; The retrieved data will be included in the plan execution prompt; The stored plan template is retrieved based on the first thread classification, wherein the retrieved stored plan template includes multiple instructions to be executed by the second large language model; as well as The retrieved plan template and retrieved data are included in the plan execution prompt.

14. The system according to claim 11, wherein, The second thread portion includes multiple tasks that are selected, prioritized, and output by the second large language model in response to receiving user-related data from the first thread portion, based on the planned execution prompt words, the online dialogue, and data retrieved from at least one of stored threads, data sources, entity connection graphs, domain applications, or recommendation systems.

15. The system according to claim 11, wherein, The processor detects an increase or decrease in the latency of the second thread portion of the output, and in response to detecting an increase in latency: reduces the number of input signals, or uses a large language model with a smaller size compared to the second large language model, or reduces the size of the second thread portion.

16. The system according to claim 11, wherein, The second thread portion includes: in response to receiving the first thread portion, a recommendation generated and output by the second language model based on the planned execution prompt, the online dialogue, and user-related data retrieved from at least one of stored threads, data sources, entity connection graphs, domain applications, or recommendation systems; or in response to receiving the first thread portion, an evaluation of the position summarized and output by the second language model based on the planned execution prompt, the online dialogue, and user-related data retrieved from at least one of stored threads, data sources, entity connection graphs, domain applications, or recommendation systems.

17. At least one non-transitory machine-readable storage medium comprising instructions that, when executed by the at least one processor, cause the at least one processor to perform at least one operation comprising: (702) First-thread category prompts are generated based on the first thread portion of an online conversation involving a user with a computing device; Send the first thread classification prompt (704) to the first large language model; Receive (706) first thread category, where, The first thread classification is generated and output by the first large language model based on the first thread classification prompt words; (708) Plan execution prompt words are formulated based on the first thread category, wherein, in response to determining that at least one stored thread involving the user matches the first thread category, the plan execution prompt words are formulated based on the first thread portion and the at least one stored thread that matches the first thread category; Send the plan execution prompt (710) to the second largest language model; Receive (712) the second thread portion, wherein the second thread portion is generated and output by the second large language model based on the planned execution prompt and the online dialogue; and Generate (714) tags for a third thread portion of the online conversation, wherein the tags are configured to be displayed on the computing device, the tags are based on the first thread classification, and the third thread portion includes the first thread portion and the second thread portion.

18. The at least one non-transitory machine-readable storage medium according to claim 17, wherein, The second thread portion includes multiple tasks that are selected, prioritized, and output by the second large language model in response to receiving user-related data from the first thread portion, based on the planned execution prompt words, the online dialogue, and data retrieved from at least one of stored threads, data sources, entity connection graphs, domain applications, or recommendation systems.

19. The at least one non-transitory machine-readable storage medium according to claim 17, wherein, The second thread portion includes an evaluation of the job, which is in response to receiving data from the first thread portion that is generalized and output by the second large language model based on the planned execution prompt, the online dialogue, and user-related data retrieved from at least one of stored threads, data sources, entity connection graphs, domain applications, or recommendation systems.

20. The at least one non-transitory machine-readable storage medium according to claim 17, wherein, The second thread portion includes recommendations that are generated and output in response to receiving user-related data generated and output by the second large language model from the first thread portion based on the planned execution prompts, the online dialogue, and data retrieved from at least one of stored threads, data sources, entity connection graphs, domain applications, or recommendation systems.

Images