Generating output from large-scale language models using data graphs

Abstract

This invention provides a system and method for generating output data based on a data graph. An output request is received that seeks output data based on the data graph. The output request includes either a natural language request from a target user or an application request from the target user's application. The data graph has nodes and edges between nodes. Nodes represent entities associated with an enterprise organization, and edges represent relationships between entities. The output request is used as the first input to a Large-Scale Language Model (LLM) to generate a graph data query using the LLM. The graph data query is executed against the data graph to obtain a graph data output representing a subpart of the data graph. The graph data output is used as the second input to the LLM to generate output data using the LLM.

Description

Background Art

[0001] Background

[0001] Corporate organizations such as companies with hundreds or thousands of employees may manage a large amount of data on entities associated with the organization, such as various users (e.g., employees), the emails sent by the users, the documents created by the users, the meetings attended by the users, etc. These entities may have relationships among them. For example, a first user (e.g., a first entity) may be in a copyright relationship with a document (e.g., a second entity) generated by that first user. When that document is shared with a second user in the organization, such as being included in an email message or being referenced in a meeting invitation, additional relationships may be generated or modified. Knowledge about these relationships can be utilized to recommend relevant entities to users when performing tasks such as sending an email (e.g., recommending documents to attach) or creating a meeting invitation (e.g., recommending users to invite). Data about entities and relationships may be stored as a data graph having nodes representing the entities and edges representing the relationships. However, creating appropriate queries to extract relevant information from the data graph can be difficult or time-consuming for some users. Further, the results of the queries (e.g., some documents and emails) may be too cumbersome or complex for a user to review when time is limited, such as during the preparation for a meeting that is about to start.

[0002]

[0002] The embodiments are described with respect to these and other general considerations. Also, although relatively specific problems have been considered, it should be understood that the embodiments should not be limited to solving the specific problems identified in the background.

Summary of the Invention

[0003] Summary

[0003] The aspects of this disclosure relate to the use of data graphs as input to neural network models such as large-scale language models (LLMs).

[0004]

[0004] According to at least one example of the present disclosure, a computer implementation method for generating output data based on a data graph is provided. The method includes receiving an output request for output data based on a data graph, the output request including either a natural language request from a target user or an application request from an application of the target user, the data graph having nodes and edges between nodes, the nodes representing entities associated with an enterprise organization, and the edges representing relationships between entities; using the output request as a first input to a Large Language Model (LLM) to generate a graph data query using the LLM; executing the graph data query on the data graph to obtain a graph data output representing a subpart of the data graph; and using the graph data output as a second input to the LLM to generate output data using the LLM.

[0005]

[0005] According to at least one example of the present disclosure, a computer implementation method for training a Large-Scale Language Model (LLM) is provided. The method includes: converting a training graph data output from a data graph into a text data format that the LLM can read, wherein the data graph has nodes and edges between nodes, the nodes representing entities associated with corporate organizations, and the edges representing relationships between entities; generating a training set containing the converted training graph data output; training the LLM to accept graph data as input prompts using the training set; and providing the LLM with an extraction prompt, wherein the extraction prompt includes a syntax example for the LLM to extract a second graph data output from the data graph.

[0006]

[0006] According to at least one example of the present disclosure, a system is provided for generating output data based on a data graph. The system includes at least one processor and at least one memory storing computer executable instructions, the computer executable instructions being executed by the at least one processor, the at least one processor receiving an output request for output data based on a data graph, the output request including either a natural language request from a target user or an application request from an application of a target user, the data graph having nodes and edges between nodes according to a graph schema, the nodes representing entities associated with an enterprise organization, and the edges representing relationships between entities, the system using the output request as a first input to a Large Language Model (LLM) to generate a graph data query using the LLM, the graph data query being based on a graph schema, the system executing the graph data query on the data graph to obtain a graph data output representing a subpart of the data graph, and the system using the graph data output as a second input to the LLM to generate output data using the LLM.

[0007]

[0007] This summary of the invention is provided in a simplified form to introduce the concepts that will be further described in the following detailed description. This summary is not intended to identify any major or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0008] Brief explanation of the drawing / diagram

[0008] The following diagram illustrates an example that is neither limited nor exhaustive. [Brief explanation of the drawing]

[0009] [Figure 1]

[0009] A block diagram of an example of a data graph processing system for a large-scale language model (LLM) according to an example embodiment is shown. [Figure 2]

[0010] An example of a data graph based on one embodiment is shown. [Figure 3]

[0011] An example block diagram of training a neural network model using graph data, according to one embodiment, is shown. [Figure 4]

[0012] An example block diagram of a system that generates output data based on a data graph, according to one embodiment, is shown. [Figure 5]

[0013] A diagram illustrating an example of a graphical user interface for providing output data from an LLM using graph data, according to one embodiment, is shown. [Figure 6]

[0014] A diagram illustrating an example of a method for training an LLM according to one embodiment is shown. [Figure 7]

[0015] A diagram illustrating an example of a method for generating output data based on a data graph, according to one embodiment, is shown. [Figure 8]

[0016] This block diagram shows examples of physical components of computer devices that can implement aspects of this disclosure. [Figure 9]

[0017] This is a simplified block diagram of a computer device that can implement aspects of this disclosure. [Modes for carrying out the invention]

[0010] Detailed explanation

[0018] The following detailed description refers to the accompanying drawings, which constitute part of this specification, and which illustrate specific embodiments or examples. These embodiments may be combined, other embodiments may be utilized, and structural modifications may be made without departing from the disclosure. Embodiments may be practiced as methods, systems, or devices. Accordingly, embodiments may take the form of hardware implementations, complete software implementations, or implementations combining software and hardware embodiments. Accordingly, the following detailed description should not be taken as limiting, and the scope of this disclosure shall be defined by the accompanying claims and equivalents.

[0011]

[0019] A data graph may be generated where nodes represent entities associated with an enterprise organization, and edges between nodes represent relationships between entities. Nodes may represent entities such as users, documents, emails, meetings, and conversations, while edges represent relationships such as document creation, document modification, document sharing, meeting invitations, inter-document link data, email sending, and email replies. While data graphs often contain information that improves search, prediction, recommendation, inter-entity lookup, clustering, and other processing scenarios, efficiently searching and consuming data from a data graph encompassing the entire organization is challenging. In some cases, enterprise-level data graphs can have millions of nodes and billions of relationships.

[0012]

[0020] To obtain useful information from a data graph, users may write queries to extract results from the data graph. For example, similar to creating queries using a structured query language (SQL), queries generally take an appropriate form depending on the structure of the data graph by referencing specific types or names of nodes and edges. However, this approach makes it difficult for users to manipulate graph data because they may not know what nodes or node types are available, or what relationships are represented by the data graph. Even after generating and executing appropriate queries to obtain results from the data graph, the results may include a large number of documents or contacts that are not easily accessible (e.g., reviewable or understandable) when time is limited.

[0013]

[0021] In the examples described herein, a data graph is used to provide input to a neural network model, such as a Large-Scale Language Model (LLM). In various examples, the data graph input may be a data structure representing a portion of the data graph, or data generated based on query results from the data graph. The data graph input may take the form of prompts to the neural network model, a training set for the neural network model, or a combination thereof. By training an LLM with the data graph input, the LLM may be configured to access information from the data graph to, for example, identify documents relevant to a user or extract portions of the data graph for further processing. Furthermore, the LLM may be configured to process the data based on priorities represented in the data graph. For example, the LLM can create a summary of documents or emails tailored to a specific user, highlighting technical features if the user is a software developer, or highlighting financial details if the user is a project manager. Advantageously, prioritization can be performed based on the data in the data graph without the user explicitly requesting that they perform the prioritization themselves.

[0014]

[0022] In some examples, a natural language request is received from a target user seeking output data based on a data graph. The data graph has nodes and edges between nodes according to a graph schema. Nodes represent entities associated with an organization, and edges represent relationships between entities. For example, a target user might request a meeting preparation summary for an upcoming meeting, where documents and emails related to the meeting, along with the meeting itself, are represented in the data graph. The natural language request is used as the first input to a Large-Scale Language Model (LLM), and the LLM generates a graph data query. The graph data query is based on a graph schema, allowing it to be executed against the data graph, resulting in a graph data output representing a sub-part of the data graph. This sub-part of the data graph might include nodes for the meeting agenda, nodes for users scheduled to attend the meeting, nodes for recent documents discussed at previous meetings, or other appropriate nodes. The output data is generated using the LLM, with the graph data output as the second input to the LLM. For example, the LLM might use information from the sub-part of the data graph to generate summaries of the agenda and recent documents.

[0015]

[0023] In accordance with embodiments of this disclosure, Figure 1 shows an example of a data graph processing system 100 for a large-scale language model. The data graph processing system 100 is configured to improve the processing of graph data using a neural network model, for example, to improve the extraction of graph data from a data graph and the processing of graph data to provide useful information to a user. To do so, the data graph processing system 100 includes one or more of a graph data source 110, a graph data processor 120, and a data store 160. A network 150 connects the graph data source 110, the graph data processor 120, and the data store 160 in a communicative manner. The network 150 may include one or more networks such as a local area network (LAN), a wide area network (WAN), a corporate network, and the Internet, and may include one or more of wired, wireless, and / or optical portions.

[0016]

[0024] The graph data source 110 may be any suitable type of computer device, including a desktop computer or PC (personal computer), a server, a distributed computing platform, or a cloud platform device. The graph data source 110 may be configured to run one or more software applications (or “Applications”) and / or services, and / or manage hardware resources (e.g., a processor, memory, etc.) that are available to users of the graph data source 110 and / or the graph data processor 120. The graph data processor 120 may be similar to the graph data source 110, or may include one or more server devices, a distributed computing platform, a cloud platform device, and / or other computer devices. In some examples, the graph data source 110 and the graph data processor 120 may together be a mobile computer device, a stationary computer device, a server device, a distributed computing platform, and / or a cloud platform device. For ease of consideration, the description herein refers to a single graph data source 110, but the characteristics and examples of the graph data source 110 also apply to two, three, or more instances of the graph data source 110. Similarly, although this specification refers to a single graph data processor 120, the characteristics and examples of the graph data processor 120 also apply to two, three, or more instances of the graph data processor 120.

[0017]

[0025] The graph data source 110 is configured to provide graph data for a graph representing entities (such as enterprise organizations or business organizations) and corresponding relationships between the entities. Thus, the graph data source 110 may be referred to herein as a source computing device or a data source. Various examples of entities include contacts (such as users), emails, meetings, documents, topics, messages, conversations, channels or news feeds, acronyms, definitions, key phrases, or other suitable entities. Examples of relationships include document creation, document modification, document sharing, meeting invitations, link data between documents, email sending, email replies, or other suitable relationships. The graph data source 110 is further configured to manage, modify, and / or store the graph data.

[0018]

[0026] The graph data source 110 includes a node processor 112 and a graph data store 114. The node processor 112 is configured to generate and / or process graph data, and the graph data store 114 is configured to store the graph data. In some examples, the graph data processor 120 includes a node processor 122 and a graph data store 124, which are generally similar to the node processor 112 and the graph data store 114, respectively. The graph data processor 120 further includes a conversion processor 163 and a language model 168.

[0019]

[0027] Generally, the language model 168 is a neural network model configured to generate text output based on a text input, and the output is generally in a natural language format, for example, written in a dialogue format that can be easily understood by a user without special training on a computer. The conversion processor 163 is configured to convert the graph data output from the data graph into a text data format that can be read by the LLM 168. Although the conversion processor 163 and the LLM 168 are shown as part of the graph data processor 120, one or both of the conversion processor 163 and the LLM 168 may be implemented on the graph data source 110, the data store 160, a stand-alone computer device (not shown), a distributed computer device (e.g., a cloud service), or other suitable processors.

[0020]

[0028] In various embodiments, the node processors 112 and 122 may be implemented as software modules, application-specific integrated circuits (ASICs), firmware modules, or other suitable implementations. The graph data stores 114 and 124 may be implemented as any type of one or more storage mechanisms including memory devices such as magnetic disks (e.g., within a hard disk drive), optical disks (e.g., within an optical disk drive), magnetic tapes (e.g., within a tape drive), random access memory (RAM) devices, read-only memory (ROM) devices, and / or other suitable types of storage media.

[0021]

[0029] In some examples, the graph data source 110 and / or graph data processor 120 correspond to a user application or user service for an enterprise organization, and the user application facilitates the use and / or storage of data for or about entities of the enterprise organization. Examples of user applications include email applications (e.g., Microsoft Outlook), messaging applications (e.g., Microsoft Teams), social or professional networks (e.g., LinkedIn), search applications (e.g., Bing), document repository applications (e.g., Microsoft SharePoint), or other appropriate applications and services. User applications are sometimes called graph data generators because they generate data about entities (e.g., documents, emails, contacts) and the relationships between entities.

[0022]

[0030] In some examples, the graph data source 110 and the graph data processor 120 generate graph data independently of each other, and then these are combined to form, for example, a universal graph. In other words, the graph data source 110 may provide an Outlook email service and generate graph data related to emails and contacts, while the graph data processor 120 may provide a SharePoint document service and generate graph data related to documents, in which case either or both of the graph data source 110 and the graph data processor 120 generate graph data independently of other computer devices. Because different user applications may have access to additional information about a given entity, combining graph data from different user applications (or different levels of access) improves knowledge about the entity. For example, a first user may create a text document and send it to a second user via email using the Outlook email application, and the second user may then share that text document with a third user using the Teams messaging application. By combining graph data from both the email application and the messaging application, knowledge about the relationship between a text document and a third user (e.g., the "viewed" relationship) becomes available in the Outlook email application, and knowledge about the relationship between a text document and a first user (e.g., the "author" relationship) becomes available in the messaging application. This knowledge, provided to both applications, enables more accurate recommendations and predictions in various scenarios, such as recommending files to attach to an email or recommending recipients of an email. Furthermore, additional relationships between the first and third users may be identified, and corresponding new edges may be generated for the data graph. These new edges may be added to or pushed to the graph data of the Outlook email application and / or the graph data of the messaging application.For example, the graph data source 110 may support an email application, and the graph data processor 120 may support a Teams messaging application.

[0023]

[0031] Node processor 112 is configured to generate graph data (e.g., data graph 200 in Figure 2) for a corresponding user application (e.g., Outlook) and store that graph data in graph data store 114. Similarly, node processor 122 is configured to generate graph data for a corresponding user application (e.g., Teams) and store that graph data in graph data store 124. As those skilled in the art will understand, graph data stores 114 and 124 store graph data about entities (e.g., documents, messages, etc.), while the entities themselves may be stored in their graph data stores, separate data stores such as content data store 165, or other appropriate locations.

[0024]

[0032] The data store 160 is configured to combine graph data generated by one or more graph data sources 110 and / or graph data processors 120 to produce an integrated data graph. In various embodiments, the data store 160 is a network server, a cloud server, a network-attached storage ("NAS") device, or other suitable computer device. The data store 160 may include one or more storage mechanisms of any type, including memory devices such as magnetic disks (e.g., in hard disk drives), optical disks (e.g., in optical disk drives), magnetic tapes (e.g., in tape drives), random access memory (RAM) devices, read-only memory (ROM) devices, and / or other suitable types of storage media. Although there is only one instance of the data store 160 shown in Figure 1, the data graph processing system 100 may include two, three, or more similar instances of the data store 160. Furthermore, in some embodiments, the network 150 may provide access to other data stores similar to the data store 160 located outside the data graph processing system 100.

[0025]

[0033] The data store 160 includes a node processor 162, integrated data 164, a content data store 165, and a neural network model 166. The node processor 162 is configured to combine graph data from different graph data sources 110 or graph data processors 120 into a data graph such as data graph 164 (also called integrated data 164). The node processor 162 may be configured to use a neural network model (or machine learning model), such as the neural network model 166, to generate integrated data 164 or embeddings for other appropriate analysis. In some examples, the node processor 162 may be similar to node processors 112 and 122. For example, the node processor 162 can generate graph data or combine graph data from other sources (e.g., graph data source 110). In some examples, the node processor 162 may be implemented as a standalone computer device or server, similar to the graph data source 110 or graph data processor 120.

[0026]

[0034] The integrated data 164 includes combined graph data from graph data stores 114 and 124, for example, as a centralized repository of graph data. For example, integrated data 164 may represent entities associated with an organization, along with the relationships between those entities. In some examples, integrated data 164 generally corresponds to a data graph 200 (Figure 2) and may be stored as one or more data structures, database entries, or other appropriate formats. In other examples, integrated data 164 may be omitted, and graph data may be retrieved dynamically from local graph data stores (e.g., graph data stores 114 and 124). The content data store 165 contains documents, files, user profiles, and / or signals representing user interactions between entities. Although a single content data store 165 is shown, the content data store 165 may be implemented distributed across several instances of data store 160. For example, the first data store may host an Exchange server for email and user accounts, the second data store may host a SharePoint server for files and documents, and the third data store may host an SQL database.

[0027]

[0035] In the examples of this disclosure, node processors 112, 122, or 162 are further configured to extract graph data from integrated data 164. To do so, node processors 112, 122, or 162 may receive requests for graph data based on integrated data 164 (or local graph data stores 114, 124). In various examples, this request may be one of many different types, such as a request for the generation of candidates (e.g., files to be attached to an email), a request for relevant entities for a search (e.g., files related to a topic), a request for automatic suggestions or recommendations of entities (e.g., users to be included in an email or meeting request), a request for the synthesis of entities, or other appropriate request types. The graph data provided in response to the request may include nodes of the data graph 200, edges of the data graph 200, embeddings, documents, or files corresponding to the nodes or edges, or identifiers corresponding to the nodes and / or edges (e.g., unique identifiers, links, file locations, etc.). In other words, this request may be called a request for embeds, nodes, edges, documents, files, users, and / or meetings related to the search query.

[0028]

[0036] As described above, the language model 168 may be a neural network model such as a Large-Scale Language Model (LLM) and may be configured to process prompts and provide outputs. The LLM168 may be implemented as, for example, a transformer model (e.g., Generative Pretrained Transformer, GPT) or other suitable model. Generally, the LLM168 may receive prompts from a user, an application programming interface (API), an application executed by a computer device (e.g., a graph data source 110 or a graph data processor 120), or other suitable input sources.

[0029]

[0037] Generally, the LLM168 is configured to process prompts written in natural language or an appropriate text data format, but it may also process prompts containing programming language code, scripting language code, text (formatted or plain text), pseudocode, XML, HTML, JSON, images, video, etc. In some scenarios, the text data format is compatible with the API of a software module or processor that can pass input data to the LLM168, and / or is compatible with a software module or processor that can provide output data to the LLM168.

[0030]

[0038] In some examples, LLM168 interacts with another neural network model (not shown), an executable file (not shown), or an API (not shown) that converts all or part of the received prompt or other input into a format suitable for processing by LLM168. For example, LLM168 may receive a prompt that includes an image and a natural language question related to that image. LLM168 provides the image to a neural network model, which converts the image into a text description of its contents, and LLM168 can then process that text description (as an extended prompt including the text description and the natural language question, or as a follow-up prompt including the text description).

[0031]

[0039] In some examples, input prompts for LLM168 are provided by the target user in the user interface (e.g., user interface 410 in Figure 4), and these prompts may include requests for information related to nodes in the data graph 200. Generally, LLM168 can process the input prompts and generate appropriate graph data queries, which can then be executed against the data graph 200 (e.g., by the node processor) to obtain graph data output. Examples of input prompts: "What is the planning document that Nikhil and Luci have been working on?", "I missed the name of the large-scale language model document that Chris created. Can you help me find it?", "Please help me prepare for the next meeting with Vipin." Advantageously, LLM168 generates the graph data queries rather than the user manually creating them.

[0032]

[0040] In another example, an LLM168 extraction prompt contains a syntax example for LLM168 to extract graph data output from data graph 200. Using this extraction prompt as a reference, LLM168 can generate queries to node processors to extract graph data or documents based on input prompts. Generally, an extraction prompt describes the structure of the data graph (e.g., nodes, edges, metadata, fields, etc.) and the semantics (meaning, semiotics) of how the request to the node processor should be formatted. An extraction prompt may be a single prompt or multiple separate prompts provided to LLM168 (e.g., during session startup, LLM initialization, after reset, etc.). Example of an extraction prompt: Extract entities from the provided question and arrange them in the format shown in the example. The keys are sourceType, sourceEntity, Relation, targetType, and targetEntity, and the values ​​are lists. You should select the appropriate Relation from only ["", "contains", "had shared", "To recipient", "CC recipient", "Authored by"]. sourceType and TargetType should be selected only from ["", "Meeting", "Email", "Document", "Contact", "Topic"]. example: Question: How many documents are created by Vipindeep? Final form: "[sourceType:Document;Relation(Relationship):"Authored by(Author)"; targetEntity:Vipindeep;targetType:Contact(contact)] Question: How many emails contain an attachment named CalendarFeature.pptx? Final form: "[sourceType:Email(email);Relation(relationship):Contains(contains);targetType:Document;targetEntity:“CalendarFeature.pptx” ]"

[0033]

[0041] In this example, the keys and relationships define at least part of the structure of the data graph 200. In other examples, the extraction prompt may include more complex queries with multiple targets, multiple sources, and so on.

[0034]

[0042] In further examples, the prioritization prompt may include example syntax for LLM168 to extract node weights from data graph 200 based on node edges. The prioritization prompt may also include examples of how to extract embeddings, metadata, or fields associated with nodes. In some examples, the prioritization prompt may include example API calls to node processor 162 to rank or prioritize groups of nodes based on their relevance to users, such as how recently a user has used or modified the corresponding entity (e.g., more recent accesses have higher priority), to whom an entity (e.g., a document) was sent (e.g., files sent to a manager have higher priority than those sent to an external contact), or how many times an entity (e.g., a user) has been invoked.

[0035]

[0043] In some examples, graph data stores 114 and / or 124 contain data in alternative formats, such as log files, telemetry files, and entities, which are stored / used in accordance with the company's privacy policy and compliance policy. In some examples, a node processor 112 processes data in alternative formats, such as log files or telemetry files, and generates new graph data (e.g., nodes and / or edges) to be stored in graph data store 114.

[0036]

[0044] Figure 2 shows an example of a data graph 200 according to one embodiment. The data graph 200 generally corresponds to a corporate organization, business, workgroup, or other appropriate domain in various examples. In some examples, the data graph 200 is provided as a combination of subgraphs from a graph data source 110 (e.g., associated with an email application) and a graph data processor 120 (e.g., associated with a messaging application). In other examples, the data graph 200 is provided as a centralized graph (e.g., integrated data 164).

[0037]

[0045] The data graph 200 has nodes representing entities associated with an enterprise organization and edges between nodes representing relationships between entities. In some examples, the data graph 200 is a graph of data and interactions, which contains information about interactions with entities, for example, where interactions are represented by edges between nodes. Examples of entities include documents (e.g., spreadsheets, text documents, videos, images, etc.), files, users (e.g., employees, clients, vendors), emails, messages, meetings, groups within an organization (e.g., accounting department, research and development department, etc.), topics, topic-based groups (e.g., users who searched for or created documents associated with a topic), or other appropriate entities. Relationships between entities may include the creation or modification of documents by users (or groups), the sharing of documents by users, the invitation or attendance of meetings by users, linked data between documents, comments and / or replies to comments, emails and email replies, or other appropriate relationships. In some scenarios, multiple different relationships exist between two or more nodes. For example, a user may modify the slideshow (modification relationship), present the slideshow (presenter relationship), or share the slideshow (sharing relationship).

[0038]

[0046] In the example shown in Figure 2, the data graph 200 includes user nodes 202, 204, 206, and 265, a slideshow node 230, a text document node 270, a spreadsheet node 275, and a message node 260. For example, user node 202 might represent the first employee of a corporate organization, while user node 204 might represent the second employee who is the manager of the first employee. In other words, user nodes 202 and 204 share a manager relationship, represented by the edge of the data graph 200. The slideshow node 230 might represent a PowerPoint presentation previously presented by the first employee, thereby causing user nodes 202 and 206 to share a presenter relationship. User node 206 might represent a third employee who attended a meeting with the first employee, thereby causing user nodes 202 and 206 to share a meeting relationship.

[0039]

[0047] User node 265 may represent a fourth employee who created a message (message node 260), which was viewed by the first employee (user node 202). The fourth employee may also share a text document (text document node 270). Therefore, the fourth employee (user node 265) shares the text document (node ​​270) created by the first employee (node ​​202). In some examples, node processor 162 is configured to combine data graphs from different graph data stores (e.g., graph data stores 114, 124) into data graph 200, combining corresponding nodes and relationships to provide additional insights into the entities of the corporate organization.

[0040]

[0048] Some nodes in the data graph 200 may not be directly related to other nodes, but rather through one, two, three, or more intermediate nodes. For example, message node 260 shares a view relationship with user node 202 (e.g., a first employee viewed a comment or message represented by message node 260), while user node 265 represents a fourth employee who created that message (e.g., the fourth employee has an author relationship with message node 260). Another example is text document node 270, which may represent a text document containing a link to data in a spreadsheet represented by spreadsheet node 275 (e.g., there is a link relationship between text document node 270 and spreadsheet node 275). For clarity, the number of nodes shown in Figure 2 is small, but please understand that a corporate organization with hundreds or thousands of employees and their associated documents, meeting calendars, etc., may have millions of nodes, along with billions of edges representing the relationships between those nodes. Furthermore, these nodes and relationships may originate from two, three, or more different data sources (for example, graph data source 110 and / or graph data processor 120).

[0041]

[0049] In various examples, nodes in data graph 200 may contain content, metadata, or both. For example, the content of slideshow node 230 might include text, images, and animations displayed within the corresponding slideshow. Metadata may include the number of times the slideshow has been presented, viewed, or modified, its file size or number of slides, the time the slideshow was accessed, and the time elapsed since the most recent access. Some nodes in data graph 200 may contain metadata that is not present in other nodes.

[0042]

[0050] In some examples, embeddings are generated for a data graph, where the embeddings represent the semantics of entities within an enterprise organization. Embeddings are implemented in a relatively low-dimensional vector space or feature space, and may be implemented as vectors with, for example, 10, 20, 100 elements, or another appropriate number of elements that allow for more efficient processing compared to a graph walk. Furthermore, in contrast to systems that use embeddings based solely on the content of entities (e.g., text), the examples described herein utilize node processors that generate embeddings based on content, relationships, and / or both content and relationships between entities.

[0043]

[0051] In some examples, each node in the data graph 200 is associated with a set of embeddings located at different levels of granularity or "slices" of the data graph. As a first example, a set of embeddings may include a first embedding based on a user-level slice representing all entity interactions and knowledge at the user level. These user-level embeddings are per-user and represent a deeper level of user personalization, but do not necessarily have a broader perspective context. As a second example, a set of embeddings may include a second embedding based on a group-level slice representing group-level entity relationships (e.g., relationships between departments rather than individuals). As a third example, a set of embeddings may include a third embedding based on an enterprise-level slice (e.g., the entire data graph 200). In general, a second embedding based on a group-level slice may be more scalable than a third embedding based on an enterprise-level slice. As described above, the node processor 112 may remove extraneous elements from the data graph 200 to obtain an instance of the data graph 200 specific to the desired level of granularity before generating embeddings corresponding to that desired level of granularity.

[0044]

[0052] A graph view or subset of the data graph 200 may be generated, for example, by filtering the data graph 200 by entity type, relationship type, values ​​of a specific field, data source (e.g., graph data source 110 or graph data processor 120), time range, or other appropriate criteria, and returned to the graph data source 110 or graph data processor 120 for further analysis and deep learning.

[0045]

[0053] Figure 3 shows an exemplary block diagram 300 of training a neural network model using graph data according to one embodiment. Block diagram 300 may be executed by a computer device 310 (corresponding to graph data source 110, graph data processor 120, or data store 160), a translation processor 340 (corresponding to translation processor 163), and an LLM 350 (corresponding to LLM 168). Computer device 310 includes a graph data store 326 configured to store graph data such as data graph 200, and node processors 324 (corresponding to node processors 112, 122, and 162). In some examples, graph data store 326 corresponds to graph data store 114, graph data store 124, and / or integrated data 164.

[0046]

[0054] Node processor 324 is configured to receive graph data queries and, in response to them, provide appropriate graph data output 330. As mentioned above, in response to a request from a target user, "How many documents have been created by Vipindeep?", an example of a graph data query might be "[sourceType:Document;Relation(Relationship):"Author";targetEntity:Vipindeep;targetType:Contact]". In some examples, graph data output 330 may take the appropriate data structure that LLM368 cannot easily understand. Translation processor 362 is configured to process graph data output 330 to generate training data 342 in a text data format that LLM368 can read. For example, a graph data query may correspond to a graph data output including nodes 202, 260, 265, and 270. To provide data in a format suitable for LLM368, translation processor 362 may transform the data structures of nodes 202, 260, 265, and 270 into training data 342 in the following text data format. Text document 270 ~ modified by user 202 Text document 270 shared by user 265 Message 260 ~ Created by user 265 Message 260 ~ viewed by User 202

[0047]

[0055] For clarity, only simple relationships are shown in text data format, but complex relationships (e.g., one-to-many, many-to-one, many-to-many), and / or multiple relationships can also be represented using appropriate text data formats, such as parenthetical lists, comma-separated lists, or multi-line items. In some examples, the training data 342 is spread across multiple separate prompts, for example, to reduce the size of the prompts or to speed up the prompt training. Advantageously, once LLM368 has been trained using different instances of the training data 342, LLM368 can more easily handle new entities generated or modified after training. In general, LLM368 may be trained using hundreds or thousands of instances of the training data 342 (for example, by taking slices of the data graph 200). In other examples, additional or fewer instances may be used for training.

[0048]

[0056] In some examples, LLM368 is retrained when a new entity type, metadata, field, or other feature is added to datagraph 200.

[0049]

[0057] Figure 4 shows an example block diagram of a system 400 that generates output data based on a data graph, according to one embodiment. The system 400 includes a user interface 410, a computer device 440 (generally corresponding to computer device 310), a conversion processor 462, and an LLM 468.

[0050]

[0058] The user interface 410 may be implemented as a software module, application, smartphone app, web portal, or other suitable user interface using the graph data source 110, graph data processor 120, data store 160, smartphone, or suitable computer device. Generally, the user interface 410 provides a text-based interface or chat interface to the target user. The target user can input or speak (using a suitable speech-to-text converter, not illustrated) natural language (NL) requests 412. For example, the target user may request or provide a prompt such as "Please help me prepare for my next meeting with Vipin" as an NL request 412. Advantageously, the NL request 412 allows the user to provide free-form requests without requiring direct knowledge of the structure of the underlying data graph (e.g., data graph 200) or the API for accessing the data graph.

[0051]

[0059] In some examples, system 400 further includes applications 411, such as user applications (e.g., document processing applications like Microsoft Word, email clients like Microsoft Outlook), server-based applications (e.g., Microsoft Exchange), or other appropriate applications. Application 411 provides application requests 413 to LLM 468. Application requests 413 typically correspond to NL requests 412, but are instead generated by application 411. Application requests 413 may correspond to the user context of the target user, or tasks at hand for the target user, such as preparing for an upcoming meeting, upcoming tasks to be completed, etc., as a follow-up notification after a meeting (e.g., a to-do list) or other appropriate scenario. In some examples, the user interface 410 is omitted, and only application 411 is used to generate output requests.

[0052]

[0060] LLM468 (shown in 468A as the first instance of calling LLM468) uses the output request (i.e., either the NL request 412 or the application request 413) as the first input to generate a graph data query 432. Based on the graph schema (e.g., the data structure of the data graph 200) according to the extraction prompt described above, LLM468 generates the graph data query 432. The graph data query 432 is executed against the data graph 200 to obtain a graph data output 448 representing a subpart of the data graph 200. For example, the graph data query 432 is provided to the node processor 444 of computer device 440, thereby identifying the relevant node corresponding to the output request. As an example, the graph data output 448 may include the node of the target user to whom Vipin has been invited, the node of the next meeting, the node of users who are scheduled to attend that meeting, and the node of the most recent document discussed at the previous meeting. Advantageously, the graph data output 448 may be customized to a specific user by obtaining slices or subgraphs around the target user. In this way, the graph data output 448 provides context relevant to the user, such as the current schedule, recently used files, and recent phone calls with contacts.

[0053]

[0061] The conversion processor 462 is configured to convert the graph data output 448 into a text data format (text data 470) that the LLM 468 can read. For example, the graph data output 448 may be converted into the following text data format. Conference - AAA ~ produced by Vipin Meeting - AAA Attendees Target Users Meeting-AAA ~ including Agenda-April-2023.pptx Email - Sent by BBB ~ to the target user Email - BBB recipient Vipin Email - BBB ~ containing Agenda-2023.pptx, Agenda-2022.pptx Agenda-2023.pptx document Agenda-2022.pptx document

[0054]

[0062] As another example, the text data format of conference-AAA may be converted as follows: Meeting-AAA;[Produced by ~; Attendees; Includes ~];[Vipin, Target Users, Agenda-April-2023.ptx]

[0055]

[0063] Text data 470 is provided to LLM468 (shown as 468B as a second instance calling LLM468), which then generates output data 480 based on text data 470. In some examples, LLM468 also receives entities from content data store 465 (corresponding to content data store 165) along with text data 470 (or as part of a follow-up request by LLM468) to generate output data 480. For example, LLM468B can generate an overview of changes in a meeting agenda, a task that is completed based on a comparison of Agenda-2023.pptx and Agenda-2022.pptx. Advantageously, LLM468B receives graph data (text data 470) specific to a particular user and their context, rather than simply providing summaries of individual documents from content data store 465, which LLM468B uses (after the training described above) to provide a customized summary tailored to a specific target user. For example, LLM can create document or email summaries tailored to specific users, emphasizing technical features if the target user is a software developer, or highlighting financial details if the target user is a project manager. Advantageously, prioritization can be performed based on data in a data graph without the target user explicitly requesting that they prioritize themselves.

[0056]

[0064] Figure 5 shows an example of a graphical user interface 500 for providing output data from LLM using graph data, according to one embodiment. Generally, node processor 112 (or node processors 122, 162, 444) may be configured to identify nodes that are similar to, related to, or adjacent to a given node or search query. In some examples, node processor 112 identifies nodes based on graph data queries generated by LLM as described above. In various examples, graph data queries may be generated in response to NL requests from the user, or they may be generated automatically based on appropriate triggers such as application requests 413 (e.g., opening a user interface menu item, receiving an email, saving a document, etc.).

[0057]

[0065] In the example shown in Figure 5, the graphical user interface 500 includes a meeting insights "tile" or popup for an email node corresponding to an email invitation to a quarterly sprint status meeting. The graphical user interface 500 may also include, for example, suggested emails 510, suggested files 520, and / or suggested users 530, corresponding to graph data output 448 and / or text data 470. The graphical user interface 500 also includes a summary 540. In one example, the summary 540 corresponds to output data 480 and includes summaries of documents Agenda-2023.pptx and Agenda-2022.pptx generated by LLM 468. As mentioned above, the summary 540 may be customized for a specific user. In the example shown in Figure 5, the summary 540 is generated by LLM to focus on software-related features (e.g., those relevant to software developers), with budgeting only briefly mentioned at the end. In contrast, an alternative instance of Summary 540 for project managers may provide further budget-related details, such as the remaining available budget and items awaiting approval before the meeting.

[0058]

[0066] Figure 6 shows a flowchart of an exemplary method 600 for training a large-scale language model, according to one embodiment. The technical processes shown in these figures are executed automatically unless otherwise specified. In any given example, some steps of the process may be repeated, possibly with different parameters or data to operate them. The steps in a particular example may be executed in a different order than the top-to-bottom order shown in Figure 6. The steps may be executed sequentially, partially overlapping, or entirely in parallel. Thus, the order in which the steps of method 600 are executed may vary between the processes of one execution example and another execution example of the process. Steps may be omitted, combined, renamed, regrouped, executed on one or more machines, or deviate from the illustrated flow, provided that the process being executed is operational and conforms to at least one claim. At least some of the steps in Figure 6 may be performed by a graph data source 110 (for example, via node processor 112), a graph data processor 120 (via node processor 122, translation processor 163, and language model 168), a data store 160 (via node processor 162, neural network model 166), or other suitable computer devices.

[0059]

[0067] Method 600 begins with step 602. In step 602, the training graph data output from the data graph is converted into a text data format that the LLM can read. In some examples, the data graph corresponds to data graph 200 and the LLM corresponds to LLM 368. The training graph data output generally corresponds to graph data output 330, which may be converted to training data 342 by the conversion processor 362. The data graph has nodes and edges between nodes, as shown in Figure 2 and described above, for example. Nodes represent entities associated with an enterprise organization, and edges represent relationships between entities.

[0060]

[0068] In step 604, a training set is generated that includes the transformed training graph data output. The training set generally corresponds to the training data 342 and may include, for example, multiple transformed graph data outputs 330 based on the graph data store 326.

[0061]

[0069] In step 606, the LLM is trained using the training set and receives graph data as input prompts.

[0062]

[0070] In step 608, an extraction prompt is provided to the LLM. The extraction prompt includes an example syntax for the LLM to extract a second graph data output from the data graph.

[0063]

[0071] Method 600 may further include providing the LLM with a prioritization prompt, which includes a syntax example for the LLM to extract weights for nodes in a data graph based on the node edges. For example, the LLM may extract a node rank or other prioritization (e.g., more recent accesses have a higher priority, files sent to a manager rather than an external contact have a higher priority, etc.).

[0064]

[0072] For example, the syntax example describes a graph walk of a data graph starting from a starting node, where the graph walk identifies adjacent nodes from the starting node with respect to a search query. The graph walk may also be based on embeddings of adjacent nodes, where the node weights correspond to the distance metric of those embeddings.

[0065]

[0073] In some examples, a data graph is a heterogeneous graph with different types of nodes, where entities include one or more of users, documents, emails, meetings, and conversations, and relationships include one or more of document creation, document modification, document sharing, meeting invitations, link data between documents, email sending, and email replies.

[0066]

[0074] Method 600 may further include extracting a first graph data output from a data graph such that it includes one or more of the following: a first output following a first subgraph of a data graph having user-level granularity; a second output following a second subgraph of a data graph having group-level granularity; a third output following a third subgraph of a data graph having enterprise-level granularity; and a fourth output following a fourth subgraph of a data graph having document-level granularity.

[0067]

[0075] Method 600 may further include receiving a natural language request from a target user for output data based on a data graph, using the natural language request as a first input to the LLM to generate a graph data query, the graph data query being generated by the LLM based on the graph schema and extraction prompt of the data graph, executing the graph data query on the data graph to obtain a graph data output representing a subpart of the data graph based on the natural language request, and using the graph data output as a second input to the LLM to generate output data.

[0068]

[0076] Figure 7 shows a flowchart of an example of a method 700 for generating output data based on a data graph, according to an example embodiment. The technical processes shown in these figures are executed automatically unless otherwise specified. In any given example, some steps of the process may be repeated, possibly with different parameters or data for operation. The steps of a particular example may be executed in a different order than the top-to-bottom order shown in Figure 7. The steps may be executed sequentially, partially overlapping, or entirely in parallel. Thus, the order in which the steps of method 700 are executed may vary between the processes of one execution example and another execution example of the process. Steps may be omitted, combined, renamed, regrouped, executed on one or more machines, or deviate from the illustrated flow, provided that the process being executed is operational and conforms to at least one claim. At least some of the steps in Figure 7 may be performed by a graph data source 110 (for example, via node processor 112), a graph data processor 120 (via node processor 122, translation processor 163, and language model 168), a data store 160 (via node processor 162, neural network model 166), or other suitable computer devices.

[0069]

[0077] Method 700 begins with step 702, in which an output request is received that requests output data based on a data graph. The output request may be either a natural language request from a target user or an application request from a target user's application. The data graph has nodes and edges between nodes, where nodes represent entities associated with an enterprise organization and edges represent relationships between entities. For example, the output request may correspond to NL request 412 (or application request 413), and the data graph may correspond to data graph 200. The graph data output represents one or more nodes from the data graph. The data graph is a heterogeneous graph with different types of nodes, where entities include one or more of users, documents, emails, meetings, and conversations, and relationships include one or more of document creation, document modification, document sharing, meeting invitations, link data between documents, email sending, and email replying. In some examples, an output request includes a request for nodes in the data graph related to the graph data query, and generating the graph data query using the LLM includes providing the LLM with an extraction prompt, which includes a syntax example for the LLM to extract graph data output from the data graph.

[0070]

[0078] In step 704, the output request is used as the first input to the Large Language Model (LLM) to generate a graph data query using the LLM. In some examples, the graph data query is based on a graph schema and may correspond to graph data query 432 in LLM 468, for example.

[0071]

[0079] In step 706, a graph data query is performed on the data graph to obtain graph data output representing a sub-part of the data graph. For example, node processor 444 may perform graph data query 432 using graph data store 446.

[0072]

[0080] In step 708, the graph data output is used as a second input to the LLM to generate output data using the LLM. For example, output data 480 may be generated by LLM 468. In some examples, step 708 includes converting the graph data output to a text data format that the LLM can read. For example, a conversion processor 462 may convert the graph data output 448 to text data 470.

[0073]

[0081] Generally, step 708 may further include providing the LLM with the converted graph data output as a second input and one or more documents corresponding to one or more nodes as a third input to generate output data. For example, LLM468B may receive documents from the content data store 465 along with the text data 470.

[0074]

[0082] In some examples, the transformed graph data output represents the weights of one or more documents depending on the target user, and step 708 includes generating weighted summaries of the content of that one or more documents according to the weights. The weights of one or more documents may include the number of LLM tokens that LLM should use to generate the weighted summaries. For example, LLM may have a limit of 1500 tokens for the sentence generation output, and the weighted summaries may be assigned 400 tokens for the first document, 800 tokens for the second document, and 300 tokens for the third document.

[0075]

[0083] In some examples, the graph data output represents multiple nodes from a data graph, where each node contains one or more of the previous nodes, and the transformed graph data output represents the user context of the target user. In one such example, step 708 further involves selecting one or more nodes as a subset of the multiple nodes in the output data according to the user context of the target user, and generating summaries of the contents of one or more documents, where the transformed graph data output represents the weights of one or more documents according to the target user.

[0076]

[0084] Figures 8 and 9 and related descriptions provide consideration of various operating environments in which embodiments of this disclosure can be put into practice. However, the devices and systems illustrated and discussed in relation to Figures 8 and 9 are for illustrative and illustrative purposes only and do not limit the vast number of computer device configurations that can be used to put into practice embodiments of this disclosure as described herein.

[0077]

[0085] Figure 8 is a block diagram showing the physical components (e.g., hardware) of a computer device 800 capable of implementing aspects of this disclosure. The computer device components described below may have computer executable instructions for running a node processor application 820 on a computer device (e.g., a graph data source 110), which include computer executable instructions for the node processor application 820 that can be executed to implement the methods disclosed herein. In a basic configuration, the computer device 800 may include at least one processing unit 802 and system memory 804. Depending on the configuration and type of the computer device, the system memory 804 may include, but is not limited to, volatile storage (e.g., random access memory), non-volatile storage (e.g., read-only memory), flash memory, or any combination of such memories. The system memory 804 may include an operating system 805 and one or more program modules 806 suitable for running node processor applications 820, such as one or more components relating to Figure 1 or Figure 5, and in particular a node processor 821 (for example, corresponding to node processors 112, 122, 514, or 524).

[0078]

[0086] For example, the operating system 805 may be suitable for controlling the operation of the computer device 800. Furthermore, embodiments of the present disclosure may be practiced in conjunction with graphics libraries, other operating systems, or other application programs, and are not limited to any particular application or system. This basic configuration is shown in Figure 8 by the internal components of the dashed line 808. The computer device 800 may have additional features or functions. For example, the computer device 800 may also include additional data storage devices (removable and / or non-removable) such as magnetic disks, optical disks, or tapes. Such additional storage is shown in Figure 8 by the removable storage device 809 and the non-removable storage device 810.

[0079]

[0087] As described above, several program modules and data files may be stored in system memory 804. While running on the processing unit 802, the program module 806 (for example, the node processor application 820) may execute processes including, but not limited to, those described herein. Other program modules that may be used in accordance with aspects of this disclosure, particularly those used to generate an integrated graph, may include the node processor 821.

[0080]

[0088] Furthermore, embodiments of the present disclosure may be implemented in electrical circuits including separate electronic elements, packaged or integrated electronic chips including logic gates, circuits utilizing microprocessors, or single chips including electronic elements or microprocessors. For example, embodiments of the present disclosure may be implemented via a system-on-a-chip (SOC) in which one or more of the components shown in Figure 8 can be integrated into a single integrated circuit. Such an SOC device may include one or more processing units, graphics units, communications units, system virtualization units, and various application functions, all of which are integrated (or "burned into") a chip substrate as a single integrated circuit. When operating via an SOC, with respect to the client's ability to switch protocols, the functions described herein may operate via application-specific logic integrated with other components of the computer device 800 on a single integrated circuit (chip). Embodiments of the present disclosure may be implemented using other techniques capable of performing logical operations such as AND, OR, and NOT, including but not limited to mechanical, optical, fluidic, and quantum techniques. Furthermore, embodiments of the present disclosure may be implemented in a general-purpose computer or any other circuit or system.

[0081]

[0089] Computer device 800 may also include one or more input devices 812, such as a keyboard, mouse, pen, sound or voice input device, or touch or swipe input device. Output devices 814, such as a display, speaker, or printer, may also be included. The devices described above are examples, and other devices may be used. Computer device 800 may also include one or more communication connections 816 that enable communication with other computer devices 850. Suitable communication connections 816 include, but are not limited to, radio frequency (RF) transmitters, receivers, and / or transceiver circuits, universal serial buses (USB), parallel and / or serial ports.

[0082]

[0090] As used herein, the term “computer-readable media” may include computer storage media. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technique for storing information, such as computer-readable instructions, data structures, or program modules. System memory 804, removable storage device 809, and non-removable storage device 810 are all examples of computer storage media (e.g., memory storage). Computer storage media may include RAM, ROM, electrically erasable read-only memory (EEPROM), flash memory, or other memory technologies, CD-ROM, digital versatile disk (DVD), or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage, or other magnetic storage devices, or other products that can be used to store information and that can be accessed by computer device 800. Such computer storage media may be part of computer device 800. Computer storage media does not include carrier waves or other propagating or modulated data signals.

[0083]

[0091] Communication media may be embodied by computer-readable instructions, data structures, program modules, or other data within modulated data signals (such as carrier waves or other carrier mechanisms), and include all information distribution media. The term “modulated data signal” may refer to a signal in which one or more characteristics are set or modified, such as by encoding the information within the signal. As an example, but not limited to, communication media may include wired media such as wired networks or direct wired connections, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0084]

[0092] Figure 9 is a block diagram showing the architecture of one embodiment of the computer device 900. That is, the computer device 900 can incorporate a system (e.g., architecture) 902 for implementing several embodiments. In one embodiment, system 902 is implemented as a “smartphone” capable of running one or more applications (e.g., a browser, email, calendar, contact manager, messaging client, game, and media client / player). In some embodiments, system 902 is integrated as a computer device such as an integrated personal digital assistive device (PDA) and a mobile phone. System 902 may include a display 905 (similar to display 905), such as a touchscreen display or other suitable user interface. System 902 may also include an optional keypad 935 (similar to keypad 935) and one or more peripheral device ports 930, such as input / output ports for audio, video, control signals, or other suitable signals.

[0085]

[0093] In some examples, system 902 may include a processor 960 coupled to memory 962. System 902 may also include a special-purpose processor 961, such as a neural network processor. One or more application programs 966 may be loaded into memory 962 and run on or in conjunction with the operating system 964. Examples of application programs include telephone dialer programs, email programs, personal information management (PIM) programs, word processing programs, spreadsheet programs, internet browser programs, and messaging programs. System 902 also includes a non-volatile storage area 968 within memory 962. The non-volatile storage area 968 can be used to store persistent information that is not lost even when the power to system 902 is turned off. Application programs 966 may use and store information such as emails or other messages used by email applications in the non-volatile storage area 968. A synchronization application (not shown) also exists on system 902 and is programmed to interact with a corresponding synchronization application residing on the host computer to synchronize the information stored in the non-volatile storage area 968 with the corresponding information stored on the host computer.

[0086]

[0094] System 902 has a power supply 970, which may be implemented as one or more batteries. Power supply 970 may further include an external power source, such as an AC adapter or a powered docking cradle for replenishing or recharging batteries.

[0087]

[0095] System 902 may also include a wireless interface layer 972 that performs radio frequency communication transmission and reception functions. The wireless interface layer 972 facilitates wireless connectivity between System 902 and the "outside world" via a communication carrier or service provider. Transmission and reception with the wireless interface layer 972 are performed under the control of the operating system 964. In other words, communications received by the wireless interface layer 972 may be transmitted to an application program 966 via the operating system 964, and vice versa.

[0088]

[0096] A visual indicator 920 may be used to provide visual notifications, and / or an audio interface 974 may be used to generate audible notifications via an audio transducer (not shown). In the illustrated embodiment, the visual indicator 920 may be a light-emitting diode (LED), and the audio transducer may be a speaker. These devices may be directly coupled to the power supply 970 so that, once activated, they remain on for a period indicated by the notification mechanism, even if the processor 960 and other components shut down to conserve battery power. The LED may be programmed to remain on indefinitely until the user takes action to indicate that the device is powered on. The audio interface 974 is used to provide audible signals to the user and to receive voice signals from the user. For example, the audio interface 974 may be coupled not only to an audio transducer but also to a microphone (e.g., to facilitate a phone conversation) to receive voice input. According to embodiments of this disclosure, the microphone may also function as an audio sensor to facilitate notification control, as described below. System 902 may further include a video interface 976, which can operate a peripheral device port 930 (for example, an onboard camera) to record still images and video streams, etc.

[0089]

[0097] A computer device 900 implementing system 902 may have additional features or functions. For example, the computer device 900 may also include additional data storage devices (removable and / or non-removable) such as magnetic disks, optical disks, or tapes. Such additional storage is shown in Figure 9 by a non-volatile storage area 968.

[0090]

[0098] The data / information generated or captured by system 902 may be stored locally, or the data may be stored on any number of storage media, which the device can access via the wireless interface layer 972, or via a wired connection between computer device 900 and another computer device associated with computer device 900 (for example, a server computer on a distributed computer network such as the Internet). It will be understood that such data / information can be accessed via computer device 900 via the wireless interface layer 972 or via a distributed computer network. Similarly, such data / information may be easily transferred between multiple computer devices for storage and use according to well-known data / information transfer and storage means, including email and collaborative data / information sharing systems.

[0091]

[0099] It should be understood that Figure 9 is described for illustrative purposes of the method and system, and is not intended to limit this disclosure to a specific sequence of steps or a specific combination of hardware or software components.

[0092]

[0100] The descriptions and illustrations of one or more embodiments provided in this application are not intended in any way to limit or restrict the scope of the claimed disclosure. The embodiments, examples, and details provided in this application are considered sufficient to transfer ownership and enable others to create and use the best form of the claimed disclosure. The claimed disclosure should not be construed as being limited to any embodiments, examples, or details provided in this application. Various features (structural and methodological), whether shown and described together or separately, are intended to be selectively included or omitted in order to create embodiments having a particular set of features. A person skilled in the art will be able to conceive of variations, modifications, and alternative embodiments that fall within the spirit of the broader embodiments of the overall concept of the invention embodied in this application and do not deviate from the broader scope of the claimed disclosure, given the descriptions and illustrations provided in this application.

Claims

1. A computer implementation method for generating output data based on a data graph, Receiving an output request for the output data based on the data graph, wherein the output request includes either a natural language request from a target user or an application request from the target user's application, the data graph having nodes and edges between the nodes, the nodes representing entities associated with an enterprise organization, and the edges representing relationships between the entities. The output request is used as a first input to a Large-Scale Language Model (LLM), and a graph data query is generated using the LLM. The process involves executing the graph data query on the data graph to obtain graph data output representing a sub-part of the data graph, A computer implementation method comprising using the graph data output as a second input to the LLM, and generating the output data using the LLM.

2. The aforementioned graph data output represents one or more nodes from the data graph, The aforementioned data graph is a heterogeneous graph having different types of nodes, The aforementioned entity includes one or more of the following: user, document, email, meeting, and conversation. The computer implementation method according to claim 1, wherein the aforementioned relationship includes one or more of the following: creating a document, modifying a document, sharing a document, inviting someone to a meeting, linking data between documents, sending an email, and replying to an email.

3. The computer implementation method according to claim 2, wherein generating the output data using the LLM includes converting the graph data output into a text data format readable by the LLM.

4. The computer implementation method according to claim 3, wherein generating the output data using the LLM includes providing the LLM with the converted graph data output as a second input and one or more documents corresponding to one or more nodes as a third input to generate the output data.

5. The converted graph data output represents the weight of one or more documents according to the target user, The computer implementation method according to claim 4, wherein generating the output data includes generating weighted summaries of the contents of one or more documents according to the weights.

6. The graph data output represents multiple nodes from the data graph, each of which includes one or more nodes, and the converted graph data output represents the user context of the target user. Generating the output data includes selecting one or more nodes as a subset of the multiple nodes of the output data according to the user context of the target user, The computer implementation method according to claim 4, wherein generating the output data includes generating summaries of the contents of one or more documents, and the converted graph data output represents the weights of the one or more documents according to the target user.

7. The output request includes a request to find the nodes of the data graph related to the graph data query, The computer implementation method according to claim 1, wherein generating the graph data query using the LLM includes providing the LLM with an extraction prompt, the extraction prompt including a syntax example for the LLM to extract graph data output from the data graph.

8. A computer implementation method for training a large-scale language model (LLM), The process involves converting the training graph data output from a data graph into a text data format readable by the LLM, wherein the data graph has nodes and edges between the nodes, the nodes represent entities associated with corporate organizations, and the edges represent relationships between the entities. To generate a training set that includes the converted training graph data output, The LLM is trained to accept graph data as an input prompt using the aforementioned training set, A computer implementation method comprising providing an extraction prompt to the LLM, wherein the extraction prompt includes a syntax example for the LLM to extract a second graph data output from the data graph.

9. The computer implementation method according to claim 8, further comprising providing the LLM with a prioritization prompt, the prioritization prompt including a syntax example for the LLM to extract the weights of the nodes in the data graph based on the edges of the nodes.

10. The computer implementation method according to claim 9, wherein the syntax example describes a graph walk of the data graph from a starting node, and the graph walk identifies adjacent nodes from the starting node with respect to a search query.

11. The computer implementation method according to claim 10, wherein the graph walk is based on the embedding of adjacent nodes, and the weights of the nodes correspond to the distance metric of the embedding.

12. The aforementioned data graph is a heterogeneous graph having different types of nodes, The aforementioned entity includes one or more of the following: user, document, email, meeting, and conversation. The computer implementation method according to claim 9, wherein the aforementioned relationship includes one or more of the following: creating a document, modifying a document, sharing a document, inviting someone to a meeting, linking data between documents, sending an email, and replying to an email.

13. A system for generating output data based on a data graph, At least one processor, and It includes at least one memory that stores computer executable instructions, and when the computer executable instructions are executed by the at least one processor, the at least one processor receives Receiving an output request for the output data based on the data graph, wherein the output request includes either a natural language request from a target user or an application request from the target user's application, the data graph having nodes and edges between the nodes, the nodes representing entities associated with an enterprise organization, and the edges representing relationships between the entities. The output request is used as a first input to a Large-Scale Language Model (LLM), and a graph data query is generated using the LLM. The process involves executing the graph data query on the data graph to obtain graph data output representing a sub-part of the data graph, A system that uses the aforementioned graph data output as a second input to the LLM, and uses the LLM to generate the aforementioned output data.

14. The aforementioned graph data output represents one or more nodes from the data graph, The aforementioned data graph is a heterogeneous graph having different types of nodes, The aforementioned entity includes one or more of the following: user, document, email, meeting, and conversation. The system according to claim 13, wherein the relationship includes one or more of the following: creating a document, modifying a document, sharing a document, inviting someone to a meeting, linking data between documents, sending an email, and replying to an email.

15. The system according to claim 14, wherein the computer executable instruction causing at least one processor to generate the output data using the LLM includes converting the graph data output into a text data format readable by the LLM.

Images