Computerized assistance using artificial intelligence knowledge base

Abstract

A computerized personal assistant includes a natural language user interface, a natural language processing mechanism, an identity mechanism, and a knowledge base update mechanism. The knowledge base update mechanism is configured to update, based on a computer-readable representation of the user input, a user-centric artificial intelligence knowledge base associated with a particular user to include new or updated user-centric facts, a knowledge base update mechanism updates the user-centric artificial intelligence knowledge base via an update protocol that can be used by a plurality of different computer services.

Description

This application is a divisional application of the invention patent application "Computerized assistance using artificial intelligence knowledge base" with application date of March 27, 2019 and application number 201980025282.1. Background Art

[0001] Artificial intelligence is an emerging field with nearly limitless applications. It is believed that as AI technology advances, user-centric AI applications will have enormous utility for personal computer users. However, the creation and maintenance of robust AI knowledge bases has been a significant obstacle to providing useful AI applications to personal computer users. Summary of the Invention

[0002] This summary is provided to introduce in simplified form a series of concepts that will be further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all disadvantages identified in any part of this disclosure.

[0003] The computerized personal assistant includes a natural language user interface, a natural language processing mechanism, an identity mechanism, and a knowledge base update mechanism. The knowledge base update mechanism is configured to provide user-centric facts to a user-centric artificial intelligence knowledge base associated with the user. The user-centric artificial intelligence knowledge base is updated via an update protocol usable by multiple different computer services and / or provides queries via a query protocol usable by multiple different computer services. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Figure 1A to Figure 1B A simplified graph data structure comprising a small number of multiple user-centric facts is shown.

[0005] Figures 2A to 6B Focusing on specific user-centric facts shows Figure 1A and 1B Graph data structure.

[0006] Figures 7A to 9B Focusing on the cross-references between the multiple component graph structures included in the graph data structure, Figure 1A and 1B Graph data structure.

[0007] FIG. 10A to FIG. 12B Show Figure 1A and 1B An exemplary implementation of a component graph structure of a graph data structure.

[0008] Figure 13An exemplary computing environment illustrating a user-centric artificial intelligence knowledge base.

[0009] Figure 14 A method for maintaining a user-centered artificial intelligence knowledge base is presented.

[0010] Figure 15 A method for querying a user-centric artificial intelligence knowledge base is presented.

[0011] Figure 16 An exemplary computerized personal assistant is shown.

[0012] Figure 17 A method for a computer service to provide one or more user-centric facts to a user-centric artificial intelligence knowledge base is shown.

[0013] Figure 18 A method for computer service querying a user-centered artificial intelligence knowledge base is presented.

[0014] Figure 19 An example computing system for maintaining and querying a user-centric artificial intelligence knowledge base is schematically illustrated. DETAILED DESCRIPTION

[0015] A computer service can improve its interaction with users by collecting and / or analyzing user-centric data. As used in this application, "computer service" broadly refers to any software and / or hardware process with which a user can interact, either directly or via interaction with another computer service (e.g., a software application, an interactive website, or a server that implements a communication protocol). As used in this application, "user-centric" broadly refers to any data associated with or related to a particular computer user (e.g., facts and / or computer data structures representing user interests, user relationships, and user interactions with a computer service).

[0016] User-centric data can be provided as input to an artificial intelligence (AI) program, which can be configured to provide enhanced interaction between a user and a computer based on analysis of the provided user-centric data. For example, providing enhanced interaction can include predicting an action a user is likely to take and facilitating that action. However, in order to meaningfully enhance the interaction between a user and a computer, the AI program requires very large amounts of data. Furthermore, this data must be centrally available in a suitable data format.

[0017] For example, a computerized personal assistant may include a natural language processing engine for processing natural language queries submitted by a user (e.g., through a natural user interface (NUI), such as a natural language user interface, configured to receive audio and / or text natural language queries). To serve natural language queries, the computerized personal assistant requires a knowledge base of facts. In some embodiments, a "knowledge base" may include a collection of facts represented as subject-predicate-object triples. The knowledge base may be generated using a combination of manually labeled data and data mining techniques to aggregate information from public sources (such as the Internet). Previous approaches for building knowledge bases have resulted in very large computational burdens (e.g., running data mining tasks for each of multiple different users), and / or a large amount of manual design and supervision (e.g., manually labeled data) to achieve good results.

[0018] Thus, previous methods for establishing and maintaining knowledge bases may not be suitable for utilizing AI to provide enhanced user interaction with computers. In contrast, a user-centric AI knowledge base as described in the present application extends the idea of a knowledge base to support a collection of user-centric facts related to one or more specific users (e.g., a personal computer user, or a group of users of an enterprise computer network). A user-centric AI knowledge base may include user-centric facts generated from a variety of different application-specific data providers (e.g., a data provider associated with a computerized personal assistant, and a data provider associated with an address book program). A user-centric AI knowledge base can efficiently store multiple user-centric facts by distributing the storage of the user-centric facts across multiple storage locations (e.g., associated with different applications). In addition, a user-centric AI knowledge base can include various application-independent enrichments for the user-centric facts that can assist in AI processing of the knowledge base, such as answering queries.

[0019] A single user can interact with multiple different computer services, each of which can identify and store information about the user. A computer service can aggregate data about interactions with the user, and multiple different computer services can collectively aggregate large amounts of data about the user. Each of the multiple computer services can store information about the user in a different, application-specific location. "Application-specific" is used in this application to mean specific to any computer service. In addition, each of the multiple computer services can identify and store only a limited subset of possible information about the user, distinct from the information stored by the different computer services of the multiple computer services. In this way, information about the user can be distributed across multiple different storage locations.

[0020] In addition, application-specific data associated with computer services may be stored in application-specific storage formats. Thus, even when two different computer services have related functionality, the computer services may not be able to share data. In some cases, a first-party provider provides a set of related computer services (e.g., a document editing suite) that can share storage formats. However, even if the computer services in this suite can share data with each other, utilizing this data to enhance interaction with users further relies on data from external knowledge sources (such as global knowledge sources (e.g., the Internet)), third-party software applications provided by different third-party software providers, and / or usage data from other users (e.g., in the context of enterprise software applications, or in the context of social network applications).

[0021] Software providers may wish to automatically aggregate information from diverse sources to build a user-centric AI knowledge base to facilitate improved user interactions. However, previous approaches to building AI knowledge bases have focused solely on global data, rather than user-centric data that can be specifically applied in the context of interactions with specific users. Consequently, previous approaches to building knowledge bases are not suitable for building user-centric AI knowledge bases for users of multiple computer services.

[0022] Figure 1A An exemplary graph data structure 100 is shown for representing a collection of user-centric facts that may be associated with application-specific data distributed across multiple different computer services. The graph data structure 100 allows for focused queries and is suitable for implementing a user-centric AI knowledge base.

[0023] Graph data structure 100 includes a plurality of different component graph structures 102, such as component graph A, component graph B, etc. Each component graph structure may be an application-specific component graph structure associated with a different computer service. For example, application-specific component graph structure A may be associated with a scheduling program, while application-specific component graph structure B may be associated with an email program.

[0024] Each component graph structure includes multiple user-centric facts 104, for example, the user-centric fact F stored in component graph A A.1 、F A.2 , and the user-centric facts F stored in the component graph B B.1 、F B.2. User-centric facts include a subject graph node 106, an object graph node 108, and an edge 110 connecting the subject graph node to the object graph node. Subject graph nodes and object graph nodes can be collectively referred to as nodes. A node can represent any noun, where "noun" is used to refer to any entity, event, or concept, or any suitable application-specific information (e.g., details of previous actions performed by a user using the computer service). Similarly, "subject noun" and "object noun" are used in this application to refer to a noun represented by a subject graph node or by an object graph node, respectively. Representing a collection of user-centric facts as a graph data structure facilitates manipulation and traversal of the graph data structure (e.g., to respond to queries).

[0025] Visualize the set of user-centric facts as Figure 1B Figure 150 in is helpful. Figure 1B In the graph 150 depicted in FIG, nodes 152 are depicted as solid circles and edges 154 are depicted as arrows. Solid circles with outgoing edges (where arrows point outward from the solid circles) depict subject graph nodes, while solid circles with incoming edges (where arrows point into the solid circles) depict object graph nodes. The multiple component graphs are treated as a single graph by treating the edges between the component graphs as edges in the larger graph. Therefore, Figure 1B A single composite graph 150 is shown that includes multiple component graph structures. To simplify the explanation, the example graph 150 includes only two component graphs with twelve nodes. In actual implementations, the user-centric graph will include many more nodes (e.g., hundreds, thousands, millions, or more) interspersed among many more component graphs.

[0026] Figure 2A to Figure 2B Focus on specific user-centric facts F A.1 Depicts Figure 1A to Figure 1B The graph data structure of user-centric facts F A.1 Depend on Figure 2A The thick rectangle in the Figure 2A Other user-centric facts of the graph data structure are not shown in detail in FIG. Bold lines are similarly used in the following figures to draw attention to specific facts. User-centric facts F A.1 Including subject graph node S A.1 , edge E A.1 and object graph node O A.1 For example, the subject graph node S A.1 can represent the user's employer, and the object graph node O A.1 can represent tasks assigned to the user by her employer. A.1 Can describe the subject graph node S A.1 and object graph node O A.1Any suitable relationship between them. In the example above, the edge E A.1 It can represent the relationship of “assigning a new task”. User-centric fact F A.1 The subject-edge-object triples of together represent the fact that the user's employer assigned her a new task.

[0027] The subject graph node of the first fact can represent the same noun as the object graph node of different second facts. For example, Figure 3A to Figure 3B Focus on user-centric facts A.3 Show Figures 1A to 2B The same graph data structure as the user-centric fact F A.3 Define the subject graph node S A.3 , edge E A.3 and object graph node O A.3 Object graph node O A.3 The same noun can be expressed as Figure 2B The subject graph node S A.1 Therefore, the graph data structure can transform the fact F A.3 The object graph node O A.3 and the fact that A.1 The subject graph node S A.1 Identified as a single node, this Figure 2B and 3B 150. By recognizing that certain object graph nodes and subject graph nodes represent the same noun, the graph data structure is able to represent user-centric facts as complex relationships between multiple different nouns, which can be visualized as paths on graph 150. For example, when a particular node is an object graph node for a first fact and a subject graph node for a different second fact, it is possible to derive an inference from the combination of these two facts, similar to a logical syllogism.

[0028] A subject graph node of a first user-centric fact may represent the same noun as a different subject graph node of a second user-centric fact. When two different subject graph nodes represent the same noun, the graph data structure may identify the two subject graph nodes as a single node. For example, Figures 4A to 4B Focus on user-centric facts A.4 Show Figures 1A to 3B The same graph data structure as the user-centric fact F A.4 Define the subject graph node S A.4 , edge E A.4 and object graph node O A.4 . Subject graph node S A.4 The same noun can be expressed as Figure 3B The subject graph node S A.3Therefore, the graph data structure can identify these two subject graph nodes as a single node, which is Figure 3B and 4B 150 in the same position. Although the subject graph node S A.4 and S A.3 can be identified as a single node, edge E A.4 Is with edge E A.3 Different, similarly, the object graph node O A.4 is the object graph node O A.3 Therefore, even if the subject graph node S A.4 and S A.3 Represents the same noun, but triples (S A.4 , E A.4 , O A.4 ) and (S A.3 , E A.3 , O A.3 ) represent two different facts.

[0029] Similarly, a subject graph node of a first user-centric fact may represent the same noun as an object graph node of a different second user-centric fact. In other words, the same noun may be the object of multiple different user-centric facts that have different subject graph nodes and possibly edges representing different relationship types. For example, FIG5A to FIG5B Focus on user-centric facts A.5 Show Figures 1A to 4B The same graph data structure as the user-centric fact F A.5 Define the subject graph node S A.5 , edge E A.5 and object graph node O A.5 Object graph node O A.5 The same noun can be expressed as Figure 2B The object graph node O A.1 Therefore, the graph data structure can identify these two object graph nodes as a single node, which is Figure 2B and 5B Depicted in the same position as in FIG. 150 .

[0030] Two or more different user-centric facts may involve a specific pair of subject nouns and object nouns. For example, the subject nouns and object nouns can be represented by a subject graph node S A.5 , edge E A.5 and object graph node O A.5 The first user-centric fact F A.5 To express. At the same time, as in FIG6A to FIG6B As depicted in the figure, the subject graph node S A.6can represent the same subject noun, and the object graph node O A.6 It can also express the same object noun, such as Figure 6B S in A.6 and O A.6 Location and Figure 5B S in A.5 and O A.5 Therefore, the subject graph node S A.5 It can be through the first edge E A.5 Connect to object graph node O A.5 , and the subject graph node S A.6 Via a second different edge E A.6 Connect to object graph node O A.6 Like the subject graph node and the object graph node, the edge E A.6 exist Figure 6B is depicted in the E A.5 exist Figure 5B However, the edge E A.5 and E A.6 are different edges, for example, representing different relationships between subject and object graph nodes. In one example, the subject graph node S A.5 and S A.6 Can correspond to a first user account (e.g., identified by an email address). In the same example, object graph node O A.5 and O A.6 Can correspond to different second user accounts. In this example, edge E A.5 can represent the relationship of "email sent to", and the edge E A.6 Relationships representing different “scheduled a meeting with…” Thus, the graph data structure includes two or more user-centric facts with the same subject and object nouns.

[0031] In other examples, two nouns may be involved in two different user-centric facts, but their roles are swapped between subject and object. In other words, the first noun is the subject noun of the first fact, and the second noun is the object noun of the first fact, and the first noun is the object noun of the second fact, and the second noun is the subject noun of the second fact. For example, a pair of nouns representing "Alice" and "Bob" is included in the first fact stating "Alice arranged a meeting with Bob", and is also included in the second fact stating "Bob arranged a meeting with Alice". In addition to swapping the roles of subject and object, two facts using these two nouns can have different types of edges, for example, "Alice" and "Bob" may also be included in a third fact stating "Bob sent an email to Alice".

[0032] As mentioned above and as Figure 1A 、 2A , 3A, 4A, 5A and 6A, the graph data structure includes multiple application-specific component graph structures (e.g., corresponding to different computer services). FIG. 7A to FIG. 7B Focusing on two different user-centric facts F A.6 and F B.1 Show Figures 1A to 6B The same graph data structure of the user-centric fact F A.6 The subject graph node S A.6 , edge E A.6 and object graph node O A.6 is defined as part of the component graph structure A. Similarly, the user-centric fact F B.1 The subject graph node S B.1 , edge E B.1 and object graph node O B.1 Defined as part of the component graph structure B.

[0033] The graph data structure can identify nouns from one component graph, and nouns from different component graphs as single nodes. For example, FIG8A to FIG8B Focus on user-centric facts B.3 Show Figures 1A to 7B The same graph data structure as the user-centric fact F B.3 Define the subject graph node S B.3 , edge E B.3 and object graph node O B.3 Object graph node O B.3 The same noun can be expressed as Figure 2B The object graph node O A.1 、 Figure 5B The object graph node O A.5 、 Figure 7B The object graph node O A.5 and Figure 8B The object graph node O B.3 Therefore, the graph data structure can identify these four object graph nodes as a single node, which is Figure 2B 、 5B , 7B and 8B are depicted in the same position in FIG150. It is worth noting that the object graph node O B.3 is in the component graph structure B, and the object graph node O A1 , O A5 and O A5is in component graph structure A. This identification of two or more different nodes corresponding to the same noun may be referred to as a "node cross-reference" between the two nodes in this application. A subject graph node or an object graph node representing a particular noun may store node cross-references to other nodes in the same component graph structure or any other component graph structure.

[0034] Similarly, a user-centric fact in a first component graph structure may define a subject graph node in the first component graph structure, along with an edge (referred to in this application as a "cross-reference edge") pointing to an object graph node in a different second component graph structure. For example, FIG. 9A to FIG. 9B Focus on user-centric facts A.2 Show Figures 1A to 8B The same graph data structure as the user-centric fact F A.2 Including subject graph node S A.2 and edge E AB.2 However, the edge E AB.2 Points to object graph node O B.2 Instead of pointing to different object graph nodes in the component graph structure A. Edge E AB.2 Connections between component graphs may be indicated in any suitable manner, for example, by storing a component graph identifier indicating a connection to an object graph node in component graph B, and an object graph node identifier indicating a particular object graph in component graph B.

[0035] Node cross-references and cross-reference edges connect multiple component graph structures. For example, node cross-references and cross-reference edges can be traversed in the same manner as edges, thereby allowing traversal of the graph data structure to traverse along paths that span multiple component graph structures. In other words, the graph data structure 100 facilitates an overall artificial intelligence knowledge base that includes facts from and / or spans different computer services. Node cross-references and cross-reference edges may be collectively referred to as cross-references in this application. Similarly, when a node is included in a cross-reference in multiple component graph structures, the node may be said to be cross-referenced across the component graph structures.

[0036] Furthermore, in addition to connecting two different component graph structures via cross-references, a component graph structure can include one or more user-centric facts relating to a subject graph node in the component graph structure and an object graph node in an external knowledge base (e.g., an internet-based, social network, or networked enterprise application). Due to the use of cross-reference edges, outgoing edges connected to a subject graph node can indicate a connection to an external object graph node in any suitable manner, such as by storing a pair of identifiers indicating the external graph and an object graph node in the external graph. In some cases, the external graph may not store any user-centric data, such as when the external graph is a global knowledge base derived from public knowledge on the internet.

[0037] By including cross-references between component graph structures, facts about a particular noun (e.g., an event or entity) can be distributed across multiple component graph structures while still supporting centralized reasoning about relationships between user-centric facts in different component graph structures and in external databases (e.g., by traversing the multiple component graph structures via cross-references and cross-reference edges).

[0038] Each user-centric fact can be stored in a predictable shared data format that stores user-centric facts including application-specific facts associated with a computer service without changing the format of the application-specific data of the computer service. Such a predictable shared data format is referred to as an "application-independent data format" in this application. This application-independent data format can store the information required to query the user-centric AI knowledge base while avoiding redundant storage of application-specific data. The graph data structure can be implemented using a complementary application programming interface (API) that allows read and write access to the graph data structure. The API can constrain access to the graph data structure to ensure that all data written to the graph data structure is in the application-independent data format. At the same time, the API can provide a mechanism that any computer service can use to add new user-centric facts to the graph data structure in an application-independent data format, thereby ensuring that all data stored in the graph data structure is predictably usable by other computer services using the API. In addition to providing read / write access to user-centric facts stored in the graph data structure, the API can provide data processing operations that include both read and write operations, such as query operations and caching of query results.

[0039] A graph data structure comprising user-centric facts about a plurality of different computer services may be implemented differently without departing from the spirit of the present disclosure, such as Figure 1A Graph data structure 100. Figure 10AOne such non-limiting embodiment of a node-centric data structure 200 is schematically shown, comprising a node record for each noun. Each node record represents one or more subject and / or object graph nodes associated with the noun. When a node record represents a subject graph node, the node record additionally represents outgoing edges connecting the subject graph node to one or more object graph nodes. For example, Figure 10A Two node records are shown, in order to fully represent Figures 1A to 9B The data structure 100 of all nodes requires multiple node records of node record NR A

[42] and node record NR A

[43] The more generalized graph data structure 100 is described above to generally introduce the features of the user-centric multi-service AI knowledge base. In practice, a more efficient data storage method (such as the node-centric data structure 200) can be used to implement the generalized features described above.

[0040] The node-centric data structure 200 stores each node of the constituent graph structure in a node record storage location defined by a consistent node record identifier associated with the node. The node record identifier can be a numeric and / or textual identifier, a reference to a computer storage location (e.g., an address in computer memory or a file name on a computer disk), or any other suitable identifier (e.g., a uniform resource locator (URL)). For example, a node record NR A

[42] may be identified by a node record identifier 'A

[42] ', which includes a node field identifier 'A' identifying the node record as part of a component graph structure A, and includes an additional numeric identifier 42. Thus, the graph data structure may identify the node record NR A

[42] Stored in the node record storage location defined by the identifier 'A

[42] '. For example, the graph data structure can store the node record NR A

[42] is stored in row #42 of the database table associated with the component graph structure A. It should be noted that the bracketed nomenclature (e.g., A

[42] ) is illustrative and serves to emphasize that while the node-centric data structure 200 ultimately defines the same node / graph as the broader graph data structure 100, the specific implementation of the node data structure 200 is different. However, the graph data structures described in this application may be implemented using any suitable nomenclature.

[0041] exist Figure 10A In the example, the node records NR A

[42] The subject graph node S representing the more general graph data structure 100 A.3 and S A.4 , and the node records NR A

[43] The subject graph node S representing the more general graph data structure 100 A.7and S A.8 and object graph node O A.4 Therefore, in Figure 10B In the node record NR A

[42] Depicted in the figure are Figure 3B and 4B The subject graph node S A.3 and S A.4 Same location. Return to Figure 10A , node record NR A

[42] Extended to display subject graph nodes S in application-independent data formats A.3 and S A.4 In one example, the user-centric facts stored in the graph data structure are application-specific facts. In addition to the application-specific facts, the user-centric facts may include one or more enrichments, which are additional data including application-independent facts associated with the application-specific facts.

[0042] In addition to the application-independent representation of connections in the graph data structure, the application-independent data format for user-centric facts allows for efficient storage of application-specific facts. For example, the subject graph node S A.3 The subject of an application-specific fact may be represented by P, for example, the fact that a new meeting is scheduled. Thus, the graph data structure stores, for an application-specific fact, aspect pointers indicating auxiliary application-specific data associated with the application-specific fact. In this example, the aspect pointer P A

[42] Is the instruction with S A.3 An identifier (e.g., a numeric identifier) of the storage location of the associated auxiliary application specific data (e.g., the storage location of the calendar entry representing the details of the meeting). A

[42] An aspect pointer may be associated with a particular type of data (e.g., calendar data) based on its inclusion in the application-specific graph structure A, because the application-specific graph structure A is associated with calendaring software. In other examples, the aspect pointer may include additional identifying information specifying the type of the auxiliary application-specific data, such that the aspect pointer may be used to represent different types of auxiliary application-specific data (e.g., multiple file types usable by a word processing application). The graph data structure may be used to locate the auxiliary application-specific data via the aspect pointer while avoiding redundant storage of the auxiliary application-specific data in the graph data structure.

[0043] In addition to the aspect pointers stored in the subject graph nodes, application-specific facts are further represented by edges connecting the subject graph node to one or more object graph nodes. Although a single subject graph node may be included in more than one user-centric fact, the graph data structure 200 efficiently stores only a single node record for the subject graph node. This node record includes a list of outgoing edges for all nodes, which reduces storage space requirements relative to storing a copy of the subject graph node for each user-centric fact in which the subject graph node appears. The list of outgoing edges may be empty for some nodes, for example, for nouns that are just object graph nodes. The list of outgoing edges for a subject graph node includes one or more edge records that define one or more edges. The one or more edges may be stored in an edge record, such as an ER node. A

[261] and ER A

[375] .although Figure 10A A node record is shown with two outgoing edges; a node record may include any number of edges to suitably represent relationships with respect to other nodes of the graph data structure, such as zero, one, or three or more edges.

[0044] An edge from a subject graph node to an object graph node specifies the object graph node record identifier associated with that object graph node, e.g., node record NR A

[42] The object graph node ID 'A

[55] '. The object graph node record identifier is a consistent node record identifier indicating the storage location of the node record defining the object graph node, such as NR A

[55] The edge may further specify the object domain identifier of the component graph structure storing the object graph node, for example, indicating NR A

[55] The node record NR stored in the component graph structure A A

[42] The object domain ID is 'A

[55] '. Therefore, Figure 10B Shows the node record NR A

[42] Record ER via node A

[261] and ER A

[375] Indicates that the edge connected to the node record NR A

[55] and NR A

[21] Return to Figure 10A , an edge from a subject graph node to an object graph node (e.g., as represented in an edge record) can further specify the type of relationship between the subject graph node and the object graph node. For example, an edge record ER A

[261] Define the relationship type R A

[261] , which may indicate a "scheduled meeting" relationship, while the edge records ER A

[375] Define the relation R A

[375] , which may indicate a "make a promise" relationship.

[0045] In addition to representing application-specific facts via aspect pointers and outgoing edge lists, subject graph nodes can represent one or more enrichments of the application-specific facts. In one example, the one or more enrichments include node confidence values, e.g., node record NR A

[42] The node confidence C A

[42] Node confidence C A

[42] Can instruct nodes to record NR A

[42] (and the subject graph nodes it represents) to the user. For example, the confidence value can be determined by a machine learning model trained to identify relevance to the user, by learning to distinguish labeled samples of relevant data from labeled samples of irrelevant data. For example, training the machine learning model can include supervised training using user-labeled samples (e.g., derived from direct user feedback during use using an application), and / or unsupervised training. Figure 10A In the example, the node records NR A

[42] The node confidence C A

[42] Can be compared to the node record NR A

[55] The node confidence C A

[55] Higher values indicate that the node records NR A

[42] Considered to be better than node record NR A

[55] More relevant to that user.

[0046] exist Figure 10A In the example shown in FIG, the one or more enrichments also include an edge confidence value associated with each edge, for example, an edge record ER A

[261] The edge confidence K A

[261] Like node confidence values, edge confidence values can indicate whether a particular edge (e.g., ER A

[261] ) with the user. Different edges between a pair of nodes can have different confidence values. For example, the edge record ER indicating the “scheduled meeting” relationship A

[261] Can have an edge record than indicates "make a commitment" A

[375] The edge confidence K of the edge confidence A

[375] Lower edge confidence K A

[261] , for example, if the scheduled meeting is considered more relevant to the user than the commitment.

[0047] In addition to the node confidence value and the edge confidence value, one or more enrichments of the application-specific fact may include other application-independent and / or application-specific data. A

[42] Including indication and access node record NR A

[42] associated with application-specific data (eg, by aspect pointer P A

[42] Access metadata M of information associated with the indicated data) A

[42] . Access metadata M A

[42] This may include a timestamp indicating the time and date of the most recent access, a delta value indicating changes resulting from the most recent access, or any other suitable metadata.

[0048] The graph data structure may additionally store one or more tags defining auxiliary data associated with the user-centric fact. A

[42] Also includes label T A

[42] , which may include any other appropriate assistance data associated with the node. For example, when the node NR A

[42] When representing a person (e.g., associated with a contact book entry), the tag T A

[42] You can include a nickname for the person and an alternative email address for the person.

[0049] User-centric facts and the nodes / edges of the user-centric facts can be enriched with additional semantics stored in the tags. For example, tags can be used to store one or more enrichments of user-centric facts. The tags stored in the node records can be associated with the user-centric facts in which the node record represents a subject graph node or in which the node record represents an object graph node. Alternatively or in addition, the tags stored in the node records can be associated with the node record itself (for example, with the subject graph node represented by the node record and / or with the object graph node represented by the node record), or with one or more edges connected to the node record. In other examples, tags can be used to store metadata of user-centric facts (for example, instead of or in addition to access metadata of user-centric facts). For example, when a user-centric fact is associated with a timestamp, the timestamp can optionally be stored between the tags of the user-centric facts.

[0050] In some examples, the one or more tags are searchable tags, and the graph data structure is configured to allow searching for user-centric facts by searching within the searchable tags (e.g., searching for a tag that stores a search string, or searching for a tag that stores a specific type of data).

[0051] The graph data structure may be represented as a plurality of application-specific component graph structures, wherein nodes of the component graph structures are cross-referenced across the component graph structures (e.g., as described above with reference to Figures 8A to 9B described, by cross-referencing edges and node cross-referencing). Figure 11A Focus on the subject graph node S A.7 、S A.8 Node NR A

[43] and object graph node O A.4 , shown in Figure 10AAnother example view of the same node-centric data structure 200 is shown in FIG. A

[43] The list of outgoing edges includes an edge record ER indicating the object graph node ID 'A

[61] ' A

[213] , represents a node in the same component graph structure A. A

[43] The list of outgoing edges also includes an edge record ER indicating the object domain ID 'B' and the object graph node ID 'B

[61] ' A

[435] , represents a node in the component graph structure B. Therefore, the edge record ER A

[435] is the cross-reference edge. Figure 11B Depicted with a diagram Figures 1A to 9B The same graph data structure, including NR A

[43] and its direction to NR A

[61] and NR B

[25] The exit edge.

[0052] Figure 12A Focus on the object graph node O A.1 , O A.5 , O A.6 and O A.7 Node NR A

[67] Shown Figure 10A Another exemplary view of a node-centric data structure 200 is shown in FIG. A

[67] The list of outgoing edges is empty because NR A

[67] represents only object graph nodes, which have only incoming edges. However, NR A

[67] Also includes a data structure pair representing the graph O A.1 , O A.5 , O A.6 and O A.7 Represent the same node as O of the component graph structure B B.3 Therefore, the cross-reference specifies the reference domain ID 'B' indicating the component graph structure B, and the cross-reference node O B.3 A node record NR is stored in a data structure centered on another node representing the component graph structure B. B

[67] The reference node ID in is 'B

[76] '. Figure 12B Graphically depicts the Figures 1A to 9B The same graph data structure as shown in . Note that the node records NR A

[67] Depicted with Figure 5B and 6B The object graph node O A.5 and O A.6 At the same location, Figure 8B The object graph node O A.5 Same location.

[0053] The node-centric data structure 200 is application-independent in that it can be used to track facts from two or more potentially unrelated computer services that may have different native data formats. The node-centric data structure 200 can store application-specific facts for any computer service by storing aspect pointers so that user-centric facts can include application-specific facts even when application-specific facts can be stored in an application-specific format. In addition, the node-centric data structure 200 validates the representation of user-centric facts defined in the context of two or more computer services by storing cross-references in the form of domain identifiers (e.g., object domain identifiers in the outgoing edge list of each subject graph node, or reference domain identifiers in a representation of node cross-references) and node identifiers (e.g., object graph node identifiers and reference node identifiers). In addition, the node-centric data structure 200 is user-centric in that a different node-centric data structure 200 can be defined for each user. The node-centric data structure 200 can be suitable for storing user-centric facts in the context of multiple different users interacting with a shared computer service (e.g., a web browser). Because the node-centric data structure 200 stores application-specific facts via aspect pointers and represents relationships to facts in other data structures via cross-references, it is able to store user-centric facts about a user in a user-centric graph data structure specific to that user without requiring write access to application-specific data of the shared computer service.

[0054] Figure 13 An exemplary computing environment 1300 for maintaining a user-centric AI knowledge base is shown. Computing environment 1300 includes a graph storage mechanism 1301 that is communicatively coupled to a plurality of application-specific data provider computers (e.g., application-specific data provider computers 1321, 1322, and 1329, etc.) via a network 1310. Graph storage mechanism 1301 and the plurality of application-specific data provider computers are further communicatively coupled to one or more user computers of a user (e.g., user computer 1340 and user computer 1341) via network 1310. For example, user computer 1340 may be a user's desktop computer, and user computer 1341 may be the user's mobile computing device. Network 1310 may be any suitable computer network (e.g., the Internet).

[0055] In some examples, computing environment 1300 may further include a computerized personal assistant 1600 that is communicatively coupled to graph storage mechanism 1301 via network 1310. Computerized personal assistant 1600 may be implemented in any suitable manner, such as as an all-in-one computing device or as a software application executable on any suitable computing device (such as a desktop computer or mobile phone). The computerized personal assistant may be a standalone computer service or an assistant component of another computer service (e.g., an email / calendar application, a search engine, an integrated development environment).

[0056] In some examples, computing environment 1300 may further include cloud services 1311. Cloud services 1311 may be communicatively coupled to other computing devices of computing environment 1300 via network 1310. Cloud services 1311 may include one or more computing devices configured to perform any suitable tasks. In some examples, cloud services 1311 may be used to offload functionality of another computing device to cloud services 1311. For example, functionality of graph storage mechanism 1301, application-specific data provider computer 1321, user computer 1340, and / or computerized personal assistant 1600 may be offloaded to cloud services 1311.

[0057] In one example, cloud service 1311 is configured to perform a natural language processing task. Graph storage mechanism 1301 can be configured to perform the natural language processing task by offloading the task to cloud service 1311. Thus, graph storage mechanism 1301 can offload input data for the natural language processing task to cloud service 1310 and receive output data from cloud service 1310 indicating the results of the natural language processing task. Alternatively or in addition, computerized personal assistant 1600 can be configured to perform the natural language processing task with remote processing assistance from cloud service 1311. In a similar manner, computing devices of computing environment 1300 can offload any suitable task to cloud service 1311, where cloud service 1311 is configured to perform the offloaded task.

[0058] The graph storage mechanism 1301 can be implemented as a single machine (e.g., a computer server). Alternatively, the functionality of the graph storage mechanism 1301 can be distributed across multiple different physical devices (e.g., by implementing the graph storage mechanism 1301 as a virtual service provided by a computer cluster). Each application-specific data provider computer (e.g., application-specific data provider computer 1321) can be associated with one or more computer services of a user, such as a social networking application, an email application, a calendaring application, an office suite, an IoT appliance function, a web search application, etc.

[0059] As a user interacts with one or more computer services via user computer 1340 and / or user computer 1341, an application-specific data provider computer may aggregate information related to the user's interactions with the one or more computer services. In the example of a user interacting with a social networking application, application-specific data provider computer 1322 may be communicatively coupled to a server that provides functionality for the social networking application. Thus, as the user interacts with the social networking application, the server may provide indications of the interactions to application-specific data provider computer 1322. In turn, application-specific data provider computer 1322 may provide one or more user-centric facts to graph storage mechanism 1301. In an example, user computer 1340 may execute one or more applications that may provide additional user-centric facts aggregated at user computer 1340 to graph storage mechanism 1301, enabling user computer 1340 to serve as an additional application-specific data provider. In an example, user computer 1340 may execute one or more applications that may request user-centric facts from graph storage mechanism 1301 (e.g., by sending queries). While the above examples include a user interacting with one or more computer services via user computer 1340, in other examples, the user may interact with the one or more computer services via user computer 1341 instead of or in addition to user computer 1340. For example, when user computer 1340 acts as an application-specific data provider by providing one or more facts to graph storage mechanism 1301, user computer 1341 may execute one or more application programs to request user-centric facts from graph storage mechanism 1301. In this manner, graph storage mechanism 1301 may aggregate user-centric facts from multiple different user computers of the user while also allowing each of the different user computers to request and use the user-centric facts.

[0060] In one example, a user computer 1340 may execute a computerized personal assistant configured to communicate with a graph storage mechanism 1301 via a network 1310. The computerized personal assistant may receive queries from the user via a NUI configured to receive audio and / or textual natural language queries. The computerized personal assistant may send one or more queries to the graph storage mechanism 1301 to receive one or more user-centric facts output by the graph storage mechanism 1301 in response to the queries. Alternatively or additionally, the computerized personal assistant may aggregate user-centric facts (e.g., user interests indicated in a conversation via the NUI), enabling the user computer 1340 to act as an application-specific data provider. In some cases, the graph storage mechanism 1301 and the application-specific data provider may be managed by a single entity or organization, in which case the application-specific data provider may be referred to as a "first-party" application-specific data provider. In other cases, the graph storage mechanism 1301 and the application-specific data provider may be managed by different entities or organizations, in which case the application-specific data provider may be referred to as a "third-party" application-specific data provider.

[0061] Figure 14 An exemplary method 1400 for maintaining a user-centric artificial intelligence knowledge base is shown. The user-centric AI knowledge base can be suitable for storing user-centric facts about a user's interactions with multiple different, potentially unrelated computer services. In addition, the user-centric AI knowledge base can be used to answer queries about the user-centric facts. Maintaining the user-centric AI knowledge base includes building the knowledge base by storing one or more user-centric facts, updating the user-centric AI knowledge base by adding additional facts, and updating the user-centric AI knowledge base as it is queried and used (e.g., storing cached answers to queries so that the queries can be quickly answered later).

[0062] At 1401, method 1400 includes maintaining a graph data structure comprising a plurality of user-centric facts associated with a user. Each user-centric fact may have an application-independent data format, for example, comprising a subject graph node, an object graph node, and an edge connecting the subject graph node to the object graph node. The graph data structure may be represented as a plurality of application-specific component graph structures, wherein the nodes of the component graph structures are cross-referenced across the component graph structures. The graph data structure may be stored using any suitable application-independent data format, as described above with reference to FIG. FIG. 10A to FIG. 12B A node-centric data structure 200 is described.

[0063] At 1402, method 1400 optionally includes automatically sending a request to an application-specific data provider to provide user-centric facts. Sending the request to the application-specific data provider can be accomplished in any suitable manner, such as over a computer network via an API of the application-specific data provider. The request can indicate that specific user-centric facts should be provided (e.g., user-centric facts from a specific range of time and / or date, user-centric facts not yet provided by the application-specific data provider, and / or user-centric facts with specific associated tags). Alternatively, the request can indicate that all available user-centric facts should be provided. Sending a request to the application-specific data provider and receiving user-centric facts in response to the request may be referred to herein as "pulling" data from the application-specific data provider. Additional user-centric facts may be requested according to any suitable schedule (e.g., periodically). In addition to providing additional user-centric facts in response to a request, one of the plurality of application-specific data providers may send user-centric facts in the absence of a request to do so, which may be referred to herein as "pushing" user-centric facts to the user-centric AI knowledge base.

[0064] At 1403, method 1400 includes receiving a first user-centric fact from a first application-specific data provider associated with a first computer service (e.g., due to pulling data from the application-specific data provider, or due to the application-specific data provider pushing the user-centric fact to a user-centric AI knowledge base). As described at 1404, the user-centric fact, such as the first user-centric fact, can be received via an update protocol (e.g., an update API) that constrains the storage format of the user-centric fact to an application-independent data format. For example, the update API can constrain the storage format to use a specific data storage format, such as an implementation of the node record format described above. In addition, the update API can constrain the maximum disk usage of the stored data and / or require that the data be stored in an encrypted data format.

[0065] At 1405, method 1400 includes adding a first user-centric fact to the graph data structure in an application-independent data format. For example, adding the first user-centric fact to the graph data structure may include translating the first user-centric fact into the above reference FIG. 10A to FIG. 12B The node record format described above is stored in a storage location associated with the node record identifier of the node record. The first user-centric fact can be an application-specific fact. Therefore, at 1406, the graph data structure can store an aspect pointer for indicating auxiliary application-specific data associated with the application-specific fact, as described above with respect to FIG. 10A to FIG. 12B described.

[0066] At 1406, the graph data structure may optionally store application-independent enrichments, e.g., application-independent facts associated with application-specific facts. The enrichments may be included in the user-centric facts received from the application-specific data provider. Alternatively or in addition, the user-centric facts provided by the application-specific data provider may be pre-processed via an enrichment pipeline including one or more enrichment adapters to include one or more enrichments. When the graph data structure stores labels associated with the user-centric facts (e.g., labels stored in node records), the one or more enrichments may be included in the labels.

[0067] In one example, an enrichment adapter includes a machine learning model configured to receive application-specific facts, identify the relevance of the application-specific facts to a user, and output a numerical confidence value indicating the relevance. The machine learning model can be any applicable model, such as a statistical model or a neural network. The machine learning model can be trained, for example, based on user feedback. For example, when the machine learning model is a neural network, the output of the neural network can be evaluated via an objective function that indicates the error level of the predicted relevance output by the neural network compared to the actual relevance indicated in the user feedback. The gradient of the objective function can be calculated in terms of the derivative of each function in a layer of the neural network using backpropagation. Accordingly, the weights of the neural network can be adjusted based on the gradient (e.g., via gradient descent) to minimize the error level indicated by the objective function. In some examples, the machine learning model can be trained for a specific user based on direct feedback provided by the user while interacting with the software application (e.g., indicating the relevance of search results in a search application). Thus, the trained machine learning model may be able to estimate the relevance of the user. In some examples, the machine learning model can be trained based on indirect feedback from the user (e.g., by estimating the similarity of relevant content to other content that the user has indicated as relevant in the past).

[0068] In another example, the enrichment adapter includes a natural language program for identifying natural language features of an application-specific fact. For example, the natural language program can determine a subject graph node and / or object graph node of the application-specific fact by identifying the natural language feature as associated with an existing subject and / or object graph node. In some examples, the natural language program can determine a relationship type for an edge of the application-specific fact. In some examples, the natural language program can determine one or more labels for the subject graph node and / or object graph node of the application-specific fact. The natural language program can be configured to recognize features including: 1) named entities (e.g., people, organizations, and / or objects), 2) intent (e.g., a sentiment or goal associated with the natural language feature), 3) events and tasks (e.g., tasks that a user intends to do at a later time), 4) topics (e.g., topics contained or represented by the user-centric fact), 5) locations (e.g., geographic locations referenced by the user-centric fact, or locations where the user-centric fact was generated), and / or 6) dates and times (e.g., timestamps indicating past events or future scheduled events associated with the user-centric fact).

[0069] The enrichment associated with a user-centric fact can provide enriched semantics for the user-centric fact (e.g., additional meaningful information, information beyond that provided by the connectivity structure formed by the edges between the subject graph node and the object graph node of the user-centric fact). The graph data structure can identify and include additional user-centric facts that can be derived from the enriched semantics (e.g., based on one or more enrichments added to the enrichment pipeline). Therefore, adding a user-centric fact including one or more enrichments can also include identifying additional user-centric facts based on the one or more enrichments and adding the additional user-centric facts to the graph data structure in an application-independent data format.

[0070] Identifying additional user-centric facts based on the one or more enrichments may include identifying that an enrichment of the one or more enrichments corresponds to another user-centric fact already included in the graph data structure (e.g., because the enrichment is associated with a subject noun or an object noun of the other user-centric fact). Alternatively or in addition, identifying the additional user-centric facts based on the one or more enrichments may include: identifying the additional user-centric facts based on the one or more enrichments may include identifying a first enrichment of the one or more enrichments that is associated with a subject noun not already included in any user-centric fact, identifying that a second enrichment of the one or more enrichments is associated with an object noun, identifying a relationship between the subject noun and the object noun, and adding a new user-centric fact containing the object noun and the subject noun to the graph data structure. In some examples, identifying the additional user-centric facts based on the one or more enrichments includes identifying any suitable relationships between the one or more enrichments and adding a user-centric fact representing the identified relationships.

[0071] In one example, each of the first node and the second node includes an enrichment for specifying an identified named entity, where the two enrichments specify the same named entity. Thus, the graph data structure can store an edge connecting the first node to the second node, and the relationship type of the edge can indicate that the two nodes are inferred to be associated with the same entity. Alternatively or additionally, the graph data structure can store node cross-references in each node, indicating that other nodes are associated with the same named entity. Alternatively, the graph data structure can modify the first node to include data for the second node and delete the second node, thereby avoiding redundant storage of data for the second node by collapsing the representation to include a single node instead of two nodes.

[0072] In another example, a first node includes an enrichment for specifying a named entity, and an edge can be added connecting the first node to a second node representing the same named entity. In another example, an edge can be added between the first node and a second node having the same associated topic. In another example, an edge can be added between a first node and a second node having the same associated time and / or location. For example, an edge can be added between a first node that refers to a particular calendar date (e.g., in a tag) and a second node that was created on the particular calendar date (e.g., as indicated by accessing metadata). In another example, an edge can be added between two nodes that were created at or refer to the same location.

[0073] At 1407, adding user-centric facts to the graph data structure in an application-independent data format optionally includes storing the user-centric facts as encrypted user-centric facts, wherein access to the encrypted user-centric facts is restricted by a certificate. For example, the certificate may be a user account certificate associated with the user, and the encrypted user-centric facts are readable only by the holder of the user account certificate. In another example, the certificate may be an enterprise account certificate associated with the user in an enterprise computer network, and the encrypted user-centric facts are readable only by the holder of the enterprise account certificate and by one or more administrators of the enterprise computer network. The user-centric facts received from the application-specific data provider may be received as encrypted user-centric facts, encrypted using the certificate. If so, the encrypted user-centric facts may be stored in the same encrypted form using the same certificate, without decrypting and re-encrypting the encrypted user-centric facts. Furthermore, the encrypted user-centric facts may be encrypted using a homogeneous encryption scheme, in which case the encrypted user-centric facts may be modified (e.g., to add enrichment) without decrypting and re-encrypting the encrypted user-centric facts and then storing the modified encrypted user-centric facts. Alternatively, the user-centric facts received from the application-specific data provider may be encrypted using a different certificate, in which case the received user-centric facts may be decrypted and re-encrypted using the certificate, and the resulting re-encrypted user-centric facts may then be stored. Alternatively, the user-centric facts received from the application-specific data provider may not be encrypted, in which case the received user-centric facts may be encrypted using the certificate, and the resulting encrypted user-centric facts may then be stored.

[0074] In addition to storing user-centric facts as encrypted user-centric facts, the graph data structure can also provide additional privacy and security by refreshing the graph data structure in response to a flush trigger to redact one or more user-centric facts. For example, a flush trigger can be a user command to redact one or more encrypted user-centric facts. The user command can indicate specific user-centric facts (e.g., user-centric facts responsive to a specific query, or user-centric facts from a specific range of times / dates). Alternatively or in addition, the user command can indicate that the entire user-centric AI knowledge base should be revised. In another example, the refresh trigger can be an automatically scheduled trigger, for example, occurring periodically, or once at a specific scheduled time in the future.

[0075] At 1408, method 1400 includes receiving a second user-centric fact from a second application-specific data provider associated with a second computer service different from the first computer service. The second user-centric fact can be received in any suitable manner (e.g., as described above with respect to receiving the first user-centric fact).

[0076] At 1409, method 1400 includes adding a second user-centric fact to the graph data structure in an application-independent data format. The second user-centric fact can be added to the graph data structure in any suitable manner (e.g., as described above with respect to adding the first user-centric fact to the graph data structure). Compared to a knowledge base that only includes user-centric facts obtained from a single computer service, the application-independent data format can assist the user-centric AI knowledge base with improved utility (e.g., in answering user queries). Although the first user-centric fact and the second user-centric fact can be received from two different computer services, which may have different, incompatible native data formats, both the first user-centric fact and the second user-centric fact can be stored in the same application-independent data format. Furthermore, although the above example includes user-centric facts from two different computer services, there is no limit to the number of different computer services that can contribute to the knowledge base. In addition, there is no requirement that the different computer services be related to each other or to the user-centric AI knowledge base in any particular manner (e.g., the different computer services and the user-centric AI knowledge base can be unrelated to each other and provided by different computer service providers). Therefore, a user-centric AI knowledge base can include user-centric facts from multiple different computer services, thereby including more user-centric facts from more diverse contexts. In addition, cross-referencing between application-specific component graph structures enables user-centric facts to express relationships between aspects of different computer services, which can further improve practicality compared to maintaining multiple different, decentralized knowledge bases without cross-referencing.

[0077] In some cases, a second user-centered fact may have the same subject noun as a first user-centered fact already stored in the graph data structure, while having different object nouns and edges than the first user-centered fact. Thus, adding the second user-centered fact may include recognizing that the second user-centered fact has the same subject noun as the first user-centered fact, and modifying the node record representing the first user-centered fact to include a new edge record representing a new outgoing edge from the subject noun to the object noun of the second user-centered fact. Thus, the node record may initially represent the first user-centered fact, and the node record may subsequently be updated to additionally represent the second user-centered fact.

[0078] At 1410, method 1400 optionally includes outputting a subset of user-centric facts included in the graph data structure in response to the query, wherein the subset of user-centric facts is selected to satisfy a set of constraints defined by the query. Responding to the query can be accomplished in any suitable manner, such as according to Figure 15 Method 1500.

[0079] Figure 15 An exemplary method 1500 for responding to a query is shown. The query defines a set of constraints that can be satisfied by a subset of user-centric facts in a user-centric AI knowledge base. The constraints can include any suitable characteristics of the subject graph nodes, object graph nodes, and edges that define the user-centric facts, such as any of the following: 1) the type of subject and / or object graph nodes included in the user-centric fact (e.g., a subject graph node represents a person); 2) the identity of the subject and / or object graph nodes included in the user-centric fact (e.g., an object graph node represents a specific email message); 3) the type of edge connecting the subject graph node and the object graph node; 4) the confidence value of the subject graph node, object graph node, and / or edge; 5) the range of dates and / or times of day associated with the subject graph node, object graph node, and / or edge; and / or 6) any other characteristics of the subject graph node, object graph node, and / or edge, such as access metadata and / or tags of the subject graph node. The answer to the query includes a subset of user-centric facts that satisfy the set of constraints, or an indication that the set of constraints cannot be satisfied.

[0080] The query may be received in any suitable manner. For example, the query may be received via a query protocol (e.g., a query API) that allows a client to programmatically define the set of constraints in a computer-readable query format. Alternatively, the query may be received as a natural language query and converted into a set of constraints by identifying the constraints using natural language processing techniques. For example, a user-centric AI knowledge base can train a semantic embedding model to represent a natural language query and a set of constraints as points in a latent space learned by the semantic embedding model, so as to translate the natural language query into a set of constraints by identifying a point in the latent space corresponding to a natural language query and outputting a set of constraints corresponding to the point in the latent space. Alternatively or additionally, the user-centric AI knowledge base can use a parsing model (e.g., dependency parsing) to match the grammatical structure of a natural language query to a template query and specify a set of constraints by filling in the details of the template query with the details of the natural language query.

[0081] At 1501, method 1500 includes identifying a query that has previously been used as a cached query, and outputting a cached response to serve the cached query by answering with the same subset of user-centric facts that satisfied the cached query when previously received. Thus, at 1507, method 1500 includes outputting the previously cached subset of user-centric facts. Such a cache can efficiently (e.g., immediately) retrieve the cached response if the cached query is received again in the future.

[0082] If the query has not been served, then at 1502, method 1500 includes selecting a subset of user-centric facts that satisfy a set of constraints defined by the query. The set of constraints can be satisfied by traversing a graph data structure to find user-centric facts that at least partially satisfy the constraints. The graph data structure represents structured relationships between user-centric facts (e.g., two facts with the same subject noun can be represented by a single node in a component graph and cross-referenced between component graphs). In this way, traversing the graph data structure to find user-centric facts that satisfy the query may be more efficient than exhaustively searching the set of user-centric facts. For example, if a user frequently interacts with a particular other person, the frequent interactions may indicate that the other person may be related to the user. Therefore, there may be more user-centric facts that have that person as a subject or object, and at the same time, it is more likely that traversing the edges of the graph data structure and encountering a node representing the other person will cause many edges to and from the node representing the person.

[0083] Traversing the graph data structure may include a "random walk" along the edges of the graph data structure. The random walk may begin in a current user context, which is used in this application to refer to any suitable starting point for answering a query. In an example, the current user context may be defined by the query (e.g., by including a contextual keyword indicating a subject graph node to use as a starting point). In other examples, the current user context may be an application-specific context suitable for determining a subject graph node to use as a starting point (e.g., "answer an email").

[0084] When a node is encountered during a random walk (e.g., at the starting point), the node can be checked to determine whether it meets the constraints of the query. If so, it can be output to a subset of user-centric facts that are responsive to the query. Then, after encountering the node, the random walk can continue, encountering more nodes. In order to find more nodes, the random walk can continue along the outgoing edges of the encountered nodes. Determining whether to continue along the outgoing edge can be a weighted random determination, including evaluating a weight representing the likelihood of following the edge, and sampling whether to follow the edge based on the weight and a random number (e.g., a "roulette wheel selection" algorithm implemented using a random number generator). The weight of the edge connecting the subject graph node to the object graph node can be determined based on the confidence value of the subject graph node, the edge node, and / or the object graph node. In one example, the confidence value can be interpreted as an indication of user relevance, so edges that are more relevant or connect more relevant nodes are more likely to be followed. The weight of the edge may be further determined based on other data of the subject graph node, object graph node, and edge, such as by evaluating the relevance of the edge to the query based on a natural language comparison of the edge type with one or more natural language features of the query.

[0085] By specifying constraints (e.g., characteristics of subject graph nodes, object graph nodes, and edges that define user-centric facts), users can formulate various questions that need to be answered using the user-centric AI knowledge base.

[0086] In addition to selecting a subset of user-centric facts that satisfy the constraints specified in the query, the user-centric AI knowledge base is able to respond to other specialized queries. Figure 15 Two exemplary specialized queries are shown: a slice query and a rank query.

[0087] In one example, at 1504, the query is a sharded query indicating a start node and a distance parameter. The answer to the sharded query is a subset of user-centric facts that includes user-centric facts reached by starting at the start node and traversing the edges of the graph data structure to form a path of length at most equal to the distance parameter from the start node. For example, if the distance parameter is set to 1, the answer to the query will include the start node and all once-removed nodes directly connected to the start node; while if the distance parameter is set to 2, the answer to the query will include the start node, all once-removed nodes, and all twice-removed nodes directly connected to at least one of the once-removed nodes. The sharded query can represent a set of user-centric facts that are potentially related to a particular user-centric fact of interest (e.g., user-centric facts related to the start node) by being connected to the start node via paths at most the distance parameter. By setting a small distance parameter, the answer to the query can represent a relatively small set of facts closely related to the start node, and similarly, by setting a large distance parameter, the answer can represent a larger set of facts indirectly related to the start node. As an alternative to specifying a start node, a query may also use the current user context as a start node, thereby representing a set of user-centric facts about the user's current context.

[0088] In another example, at 1505, the query is a ranking query for ranking a plurality of user-centric facts based at least in part on a confidence value associated with each user-centric fact, and a subset of the user-centric facts is ranked according to the confidence value of each user-centric fact. The ranking query can be interpreted as collecting user-centric facts that may be relevant to the user without imposing additional specific constraints on the query. In addition to ranking the plurality of user-centric facts based on confidence values, the plurality of user-centric facts can also be ranked based on other features. For example, if a user-centric fact is more recent (e.g., based on a timestamp associated with each fact), it can be weighted as more relevant. In another example, the ranking query can include a keyword, and if a user-centric fact includes at least one node with the keyword in its label, it can be weighted as more relevant.

[0089] although Figure 15 Depicts two examples of specialized queries, but does not Figure 15A user-centric AI knowledge base capable of other categories of specialized queries is depicted in . For example, a subset of user-centric facts in response to a sharded query can be sorted by confidence values as in a ranked query, thereby combining the functionality of both queries. In another example, the query is a pivot query that indicates a starting node. The answer to the pivot query is a subset of user-centric facts that includes user-centric facts reached by starting at the starting node and traversing the edges of the graph data structure to form a path of infinite (or arbitrary, large) length. A pivot query can be interpreted as a sharded query that does not limit the length of the path reached from the starting node (e.g., where the distance parameter is infinite). In some examples, the answer to the query may include visualizing a graph diagram (e.g., Figure 2A ), which may be annotated or animated to include any suitable information for that user-centric fact (e.g., auxiliary application-specific data indicated by an aspect pointer included in one of the user-centric facts).

[0090] A query can define a time constraint such that the subset of user-centric facts output as a response to the query is limited to user-centric facts associated with timestamps indicating times within the range defined by the query. In some examples, time is an inherent property of nodes and edges in the graph data structure (e.g., each user-centric fact included in a plurality of user-centric facts in the graph data structure is associated with one or more timestamps). The one or more timestamps associated with a node or edge can indicate a time when the user-centric fact was created, accessed, and / or modified (e.g., access metadata for a node included in the user-centric fact). Alternatively or in addition, the one or more timestamps can indicate a time referenced in the user-centric fact (e.g., the time a meeting was scheduled, or any other timestamp added by an enrichment adapter in an enrichment pipeline). The one or more timestamps can optionally be stored as a searchable tag.

[0091] In some examples, one or more constraints defined by a query include an answer type constraint, and thus, the subset of user-centric facts selected in response to the query may include only user-centric facts that satisfy the answer type constraint. For example, the answer type constraint may limit the characteristics of the subject graph nodes, object graph nodes, and / or edges of the user-centric fact. For example, the answer type constraint may indicate specific types of subject and / or object graph nodes, such as: 1) either the subject or the object is a person; 2) both the subject and the object are colleagues; 3) the subject is a location; 4) the object is a topic; or 5) the subject is a person and the object is a scheduled event. Alternatively or additionally, the answer type constraint may indicate one or more specific subjects and / or objects, such as 1) the subject is a user; 2) the subject is the user's boss, Alice; or 3) the object is any one of Alice, Bob, or Charlie. Alternatively or additionally, the answer type constraint may indicate one or more edges of a particular type, for example, by indicating a relationship type such as a "sent email" relationship, a "went to lunch" relationship, or a "research topic" relationship.

[0092] In some examples, one or more constraints defined by a query may include a graph context constraint, and thus, the subset of user-centric facts selected in response to the query may include only user-centric facts related to contextualized user-centric facts in the user-centric AI knowledge base that satisfy the graph context constraint. Two different user-centric facts may be described as related in this application based on any feature of the graph data structure that can indicate a possible relationship. For example, when a graph context constraint indicates that the user's boss is Alice, the contextualized user-centric fact may be any fact that has a node that represents Alice as a subject graph node or an object graph node. Thus, the subset of user-centric facts selected in response to the query may include other user-centric facts that have subject and / or object graph nodes that are directly connected to the node representing Alice via an edge. Alternatively or in addition, the subset of user-centric facts may include user-centric facts that are indirectly related to Alice, for example, user-centric facts that have subject and / or object graph nodes that are indirectly connected to the node representing Alice via a path of two or more edges. In some cases, the query including the graph context constraint can be a sharded query, and the subset of user-centric facts can include only user-centric facts that are related to the contextualized user-centric fact and reachable at most within a certain distance of the node of the contextualized user-centric fact. In other examples, the query including the graph context constraint can be a ranked query, and the subset of user-centric facts can include some user-centric facts that are most likely related to the contextualized user-centric fact, for example, user-centric facts that are connected to the contextualized user-centric fact via many different paths, or via paths that include edges with high confidence values.

[0093] Answering a query may include traversing a graph data structure based on one or more timestamps associated with each user-centric fact, which may be referred to herein as a traversal of a time dimension of the graph data structure. For example, answering a query may include starting at a node associated with a time defined by the query and traversing the graph by following any edges having timestamps indicating a later time, such that the timestamps increase in the same order as the traversal. In other examples, answering a query may include traversing the graph data structure by following any edges having timestamps prior to the date defined by the query. Furthermore, the inherent time properties of each node and edge in the graph data structure may form a timeline view of the graph data structure. In an example, an answer to a query may include a timeline view of the graph data structure, e.g., with the user-centric facts arranged in chronological order of occurrence based on the timestamps associated with each user-centric fact.

[0094] As another example of specialized queries, the graph data structure can be configured to allow searching for user-centric facts based on searchable tags stored by the graph data structure for the user-centric facts. Searching for user-centric facts based on searchable tags can include traversing the graph in any suitable manner (e.g., as described above with respect to shard queries or pivot queries) and, while traversing the graph, outputting any user-centric facts encountered during the traversal for which the graph structure stores searchable tags.

[0095] As another example of specialized queries, the graph data structure can be configured to serve user context queries by searching for user-centric facts that may be relevant to the user's current context. Thus, a user context query may include one or more constraints related to the user's current context. For example, the one or more constraints may include a time constraint based on the current time when the user context query is served. Alternatively or in addition, the one or more constraints may include a graph context constraint related to the user's current context, for example, a graph context constraint that specifies tasks in which the user can participate.

[0096] In some examples, one or more constraints of a user context query may be based on state data of a computer service that issues the user context query. In some examples, the state data of the computer service includes natural language features (e.g., intent, entity, or subject), and the one or more constraints include an indication of the natural language features. For example, when the computer service is an email program, the one or more constraints of the user context query may include: 1) a time constraint based on the time when the user begins composing the email; 2) a graph context constraint indicating the subject of the email; and 3) a graph context constraint indicating the recipient of the email. Thus, the subset of user-centric facts selected in response to the user context query may include user-centric facts that are current (based on timestamp) and potentially relevant to the user's task of composing the email (based on subject and recipient).

[0097] After selecting a subset of user-centric facts in response to the query, the subset of user-centric facts can be cached so that future responses to the same query can be immediately answered. Thus, at 1506, method 1500 optionally includes caching the query as a cached query and caching the subset of user-centric facts in response to the query as a cached response. Then, at 1507, method 1500 includes outputting the subset of user-centric facts in response to the query, which can include directly outputting the selected subset of user-centric facts, or after caching the selected subset of user-centric facts, outputting the resulting cached subset of user-centric facts.

[0098] When the graph data structure includes encrypted user-centric facts, the graph data structure can be filtered to create a filtered graph data structure that does not include one or more encrypted user-centric facts, but includes other user-centric facts without revealing that one or more encrypted user-centric facts are excluded. For example, the encrypted user-centric facts to be excluded can be selected by the user (e.g., by selecting encrypted user-centric facts in response to a query, or by selecting encrypted user-centric facts from a specific time / date range). The filtered graph data structure can still be used to answer the query, but does not include any encrypted user-centric facts in the answer. The filtered graph data structure omits the excluded user-centric facts without in any way indicating the absence of the excluded user-centric facts. For example, the answer will not indicate that one or more encrypted user-centric facts appear but have been revised. Instead, the answer will simply omit the one or more encrypted user-centric facts, while potentially including other user-centric facts.

[0099] A user-centric AI knowledge base utilizing the above-described graph data structure can support improved interaction between a user and one or more computer services. A computer service can serve as an application-specific data provider by providing data (e.g., user-centric facts) to the user-centric AI knowledge base. Alternatively or additionally, the computer service can use the user-centric AI knowledge base to answer queries. For example, a computer service can use the user-centric AI knowledge base to provide information to a user (e.g., in response to a user query). In some examples, a computer service can be configured to automatically perform actions to assist a user based on the user-centric facts in the user-centric AI knowledge base.

[0100] When each of a plurality of computer services contributes user-centric facts to a user-centric AI knowledge base, the user-centric AI knowledge base can assist in information sharing between the plurality of computer services. Thus, the user-centric AI knowledge base can enable one or more computer services to assist the user in a collaborative manner. For example, a first computer service can provide one or more user-centric facts to the user-centric AI knowledge base, and a second computer service can perform actions to assist the user based on the one or more user-centric facts. In this way, a second computer service can provide services related to the first computer service even when the data required to provide such functionality is not directly available in the second computer service, and when such functionality is not included in the first computer service.

[0101] Examples of computer services that can use a user-centric AI knowledge base include: 1) a computerized personal assistant; 2) an email client; 3) a calendar / scheduling program; 4) a word processing program; 5) a presentation editing program; 6) a spreadsheet program; 7) a programming / publishing program; 8) an integrated development environment (IDE) for computer programming; 9) a social networking service; 10) a workplace collaboration environment; and 11) a cloud data storage and file synchronization program. However, utilization of the user-centric AI knowledge base is not limited to the above examples of computer services, and any computer service can use the user-centric AI knowledge base in any suitable manner (e.g., by providing data and / or by issuing queries).

[0102] The computer service that utilizes the user-centric AI knowledge base can be a first-party computer service authorized and / or managed by the organization or entity that manages the user-centric AI knowledge base, or a third-party service authorized and / or managed by a different organization or entity. The computer service can utilize the giant user-centric AI knowledge base via one or more APIs (e.g., update APIs and query APIs) of the user-centric AI knowledge base, where each API can be used by multiple different computer services including first-party computer services and third-party services.

[0103] Figure 16 An exemplary computer service is shown in the form of a computerized personal assistant 1600. The computerized personal assistant 1600 can utilize the functionality of a user-centric AI knowledge base. For example, Figure 13 In the computing environment 1300 of FIG. 1 , a computerized personal assistant 1600 is communicatively coupled to a graph storage mechanism 1301 that implements a user-centric AI knowledge base. Thus, the computerized personal assistant can be configured to interact with the graph storage mechanism 1301 to provide user-centric facts to the user-centric AI knowledge base and to issue queries to be serviced by the user-centric AI knowledge base. The computerized personal assistant 1600 can be a standalone computer service or an auxiliary component of another computer service (e.g., an email / calendar application, a search engine, an integrated development environment).

[0104] The computerized personal assistant 1600 includes a natural language user interface 1610 configured to receive user input and / or user queries. The natural language user interface 1610 may include a keyboard or any other text input device configured to receive user input in text form. The natural language user interface 1610 may include a microphone 1611 configured to capture voice audio. Therefore, the user input and / or user query received by the natural language user interface 1610 may include voice audio captured by the microphone. In some examples, the natural language user interface 1610 is configured to receive user voice audio and output text representing the user voice audio. Alternatively or additionally, the natural language user interface 1610 may include an ink input device 1612, and the user input received by the natural language user receiving interface 1610 may include user handwriting and / or user gestures captured by the ink input device. "Ink input device" may be used in this application to refer to any device or combination of devices that can allow a user to provide ink-based input. For example, an ink input device may include any device that allows a user to indicate a range of two-dimensional or three-dimensional positions relative to a display or any other surface, such as 1) a capacitive touch screen controlled by a user's finger; 2) a capacitive touch screen controlled by a stylus; 3) a "hover" device that includes a stylus and a touch screen configured to detect the position of the stylus as it approaches the touch screen; 4) a mouse; or 5) a video game controller. In some examples, the ink input device may alternatively or additionally include a camera configured to detect user gestures. For example, a camera may be configured to detect gestures based on three-dimensional movements of a user's hand. Alternatively or additionally, a camera (e.g., a depth camera) may be configured to detect the movements of the user's hand as two-dimensional positions relative to a surface or plane, such as relative to a plane defined by the front side of the camera's viewing cone.

[0105] The computerized personal assistant 1600 also includes a natural language processing (NLP) mechanism 1620 configured to output a computer-readable representation of user input and / or user queries received at the natural language user interface 1610. Thus, when the natural language user interface 1610 is configured to receive user input, the NLP mechanism 1620 is configured to output a computer-readable representation of the user input; and when the natural language user interface 1610 is configured to receive a user query, the NLP mechanism 1620 is configured to output a computer-readable representation of a query based on the user query.

[0106] When the NLP mechanism 1620 is configured to output a computer-readable representation of the query based on the user query, the NLP mechanism 1620 may also be configured to output an identified user intent based on the user query. Thus, the one or more constraints defined by the computer-readable representation of the query may include constraints based on the identified user intent.

[0107] The NLP mechanism 1620 can be configured to parse a user's utterances to identify intents and / or entities defined by the utterances. An utterance is any user input, e.g., a sentence or sentence fragment, which may or may not be very grammatical (e.g., with respect to grammar, word usage, pronunciation, and / or spelling). An intent represents an action that a user may wish to perform, which may include questions or tasks, such as making a reservation at a restaurant, calling a taxi, displaying a reminder at a later time, and / or answering a question. Entities can include specific named entities (e.g., the user's boss Alice) or placeholders representing specific types of entities, such as people, animals, coworkers, places, or organizations.

[0108] The NLP mechanism 1620 can be configured to use any suitable natural language processing technology. For example, the NLP mechanism 1620 may include a dependency parser and / or a structural parser configured to identify the grammatical structure of an expression. The NLP mechanism 1620 may also be configured to identify key-value pairs representing the semantic content of an expression, such as a pairing of the type of entity of an expression with the name of a specific entity of that type. In some examples, the components of the NLP mechanism 1620 (e.g., a dependency parser) may utilize one or more machine learning techniques. Non-limiting examples of these machine learning techniques may include feedforward networks, recurrent neural networks (RNNs), long short-term memory (LSTMs), convolutional neural networks, support vector machines (SVMs), generative adversarial networks (GANs), variational autoencoding, Q learning, and decision trees. The various identifiers, engines, and other processing blocks described in this application may be trained via supervised and / or unsupervised learning using these, or any other appropriate machine learning techniques, in order to perform the described evaluations, decisions, recognitions, and the like. However, it should be understood that this specification is not intended to propose new technologies for performing these evaluations, decisions, recognitions, and the like. Rather, the present specification is directed to managing computing resources and, therefore, is intended to be compatible with any type of processing module, including processing modules that have not yet been developed.

[0109] In some examples, NLP mechanism 1620 can be configured to recognize a set of predefined intents and / or entities. Alternatively or additionally, NLP mechanism 1620 can be trained based on example expressions. In some examples, NLP mechanism 1620 can be configured to recognize new intents and / or entities by providing a number of labeled examples to NLP mechanism 162, wherein the labeled examples include the new intent to be recognized together with exemplary expressions annotated as representing the relevant entities. NLP mechanism 1620 can be trained for the specific purpose of parsing expressions in the context of computerized personal assistant 1600, so that the accuracy and / or performance of NLP mechanism 1620 is optimized for computerized personal assistant 1600. For example, when NLP mechanism 1620 is based on one or more neural networks, training NLP mechanism 1620 can include training the one or more neural networks via stochastic gradient descent using a backpropagation algorithm.

[0110] In some cases, NLP mechanism 1620 may be configured to recognize a specific human language, such as English. Alternatively or additionally, NLP mechanism 1620 may be configured to recognize multiple different human languages. When NLP mechanism 1620 is configured to recognize multiple different human languages, NLP mechanism 1620 may be able to process expressions that include text in multiple different languages.

[0111] In some examples, computerized personal assistant 1600 can be implemented as an all-in-one computing device contained within a single housing. For example, the all-in-one computing device can include one or more logic devices and one or more storage devices that store instructions executable by the logic device to provide the functionality of natural language user interface 1610, NLP mechanism 1620, identity mechanism 1630, enrichment adapter 1640, knowledge base update mechanism 1650, knowledge base query mechanism 1660, and output system 1670. In some examples, the all-in-one computing device can additionally include one or more input devices (e.g., microphone 1611 and / or ink input device 1612). In some examples, the all-in-one computing device can include one or more output devices (e.g., a display and / or speakers included in output subsystem 1670). Alternatively or additionally, the all-in-one computing device can include a communication subsystem configured to be communicatively coupled to other computer services (e.g., as part of output subsystem 1670).

[0112] In some examples, one or more components of computerized personal assistant 1600 (e.g., natural language user interface 1610, NLP mechanism 1620, identity mechanism 1630, enrichment adapter 1640, knowledge base update mechanism 1650, knowledge base query mechanism 1660, and / or output subsystem 1670) can be configured to collaborate with one or more other computer services (e.g., other computing devices) to perform computations, transfer data, and implement the functionality of one or more components described above.

[0113] In one example, the computerized personal assistant 1600 can be implemented across two or more different computing devices. For example, the NLP mechanism 1620 can offload one or more natural language processing tasks to one or more cloud services, such as Figure 13 1300. Thus, the cloud service 1311 can be configured to process natural language as described above and return a computer-readable representation of the user input to the computerized personal assistant 1600 as output at the NLP mechanism 1620. In some examples, the NLP mechanism 1620 can partially pre-process the user input before sending the pre-processed input for further processing by the cloud service 1311, and / or post-process the computer-readable representation of the pre-processed input received from the cloud service 1311 before outputting the post-processed computer-readable representation of the pre-processed input at the NLP mechanism 1620. For example, the remote computer service can be a cloud service that provides processing of natural language input, such as MICROSOFT LUIS. TM Language understanding services.

[0114] Alternatively or in addition to NLP mechanism 1620 offloading functionality to cloud service 1311 in the manner described above, any other components of computerized personal assistant 1600 can be similarly configured to offload tasks to cloud service 1311 or to any other computing device (e.g., graph storage mechanism 1310, application-specific data provider 1321, and / or user computer 1340). In this manner, the functionality of components of computerized personal assistant 1600 can utilize the hardware and / or software included in computerized personal assistant 1600 in addition to utilizing other computing devices and / or computer services.

[0115] For example, the knowledge base update mechanism 1650 can be configured to offload tasks, such as adding new user-centric facts to the user-centric knowledge base, to the graph storage mechanism 1310. Thus, to add new user-centric facts to the user-centric knowledge base, the knowledge base update mechanism 1650 can be configured to provide user-centric facts to the graph storage mechanism 1301 via the network 1310, and receive a subset of user-centric facts selected in response to a query, including a subset of user-centric facts in the user-centric AI knowledge base implemented by the graph storage mechanism 1301.

[0116] Alternatively or additionally, the knowledge base query mechanism 1660 can be configured to offload tasks, such as issuing queries to be served by the user-centric knowledge base, to the graph storage mechanism 1310. Thus, to issue a query to be served by the user-centric knowledge base, the knowledge base query mechanism 1660 can be configured to provide user-centric facts to the graph storage mechanism 1301 via the network 1301 and receive a response based on a subset of the user-centric facts (e.g., a textual answer to the query based on the subset of the user-centric facts).

[0117] Computerized personal assistant 1600 also includes an identity mechanism 1630 configured to associate user input with a particular user. Identity mechanism 1630 can use any appropriate information to determine the user's identity. For example, when natural language user interface 1610 includes a microphone, and when the user input includes voice audio, identity mechanism 1630 can include a speaker recognition engine configured to distinguish between users based on the voice audio. Alternatively or additionally, when computerized personal assistant 1600 includes a camera (e.g., as part of natural language user interface 1610), identity mechanism 1630 can include a facial recognition engine configured to distinguish between users based on a photograph of the user's face. Alternatively or additionally, identity mechanism 1630 can be configured to receive biometric information (e.g., a user's fingerprint) and distinguish between users based on the biometric information. Alternatively or additionally, identity mechanism 1630 can be configured to prompt the user to provide an identification (e.g., login information such as a username and password).

[0118] In an example where the user-centric AI knowledge base includes one or more encrypted user-centric facts, access to the one or more encrypted user-centric facts may be constrained by credentials associated with a particular user. Thus, a computerized personal assistant associated with a particular user may need to provide credentials to access (e.g., read and / or modify) the user-centric AI knowledge base. Accordingly, identity mechanism 1630 may be configured to identify the particular user and provide credentials associated with the particular user. For example, identity mechanism 1630 may store the credentials, such as a digital certificate, so that the credentials are provided whenever computerized personal assistant 1600 provides user-centric facts to the user-centric AI knowledge base or issues a query to be serviced by the user-centric AI knowledge base.

[0119] Optionally, in some examples, the computerized personal assistant 1600 further includes an enrichment adapter 1640 configured to output an enrichment based on the computer-readable representation of the user input, wherein the new or updated user-centric facts comprise the enrichment output by the enrichment adapter. For example, the enrichment adapter 1640 may be configured to add the above-mentioned Figure 14 Any enrichment of the description, e.g., 1) named entities, 2) intents, 3) events and tasks, 4) topics, 5) locations, and / or 6) dates and times. In some examples, enrichment adapter 1640 is an enrichment pipeline comprising a plurality of enrichment adapters each configured to output an enrichment, wherein the new or updated user-centric fact comprises all enrichments output by each enrichment adapter of the enrichment pipeline.

[0120] Optionally, in some examples, the computerized personal assistant 1600 further includes a knowledge base update mechanism 1650 configured to update the user-centric AI knowledge base associated with the particular user to include new or updated user-centric facts based on the computer-readable representation of the user input. The knowledge base update mechanism 1650 can update the user-centric AI knowledge base via an update protocol (e.g., an update API). The update protocol can be used by a plurality of different computer services. As described above with reference to Figure 14 As described, the update protocol may constrain the storage format of new or updated user-centric facts to an application-independent data format.

[0121] Optionally, in some examples, the computerized personal assistant 1600 further includes a knowledge base query mechanism 1660 configured to query a user-centric artificial intelligence knowledge base associated with the particular user and output a response based on a subset of user-centric facts in the user-centric artificial intelligence knowledge base that satisfy one or more constraints defined by the computer-readable representation of the user query. The knowledge base query mechanism can be configured to query the user-centric AI knowledge base via a query protocol (e.g., a query API) usable by a plurality of different computer services.

[0122] In some examples, one or more constraints defined by the computer-readable representation of the query may include an answer type constraint, and thus, the subset of user-centric facts may include only user-centric facts that satisfy the answer type constraint. Alternatively or additionally, one or more constraints defined by the computer-readable representation of the query may include a graph context constraint, and thus, the subset of user-centric facts may include only user-centric facts related to contextualized user-centric facts in the user-centric AI knowledge base that satisfy the graph context constraint. Thus, reference may be made to Figure 15 The subset of facts described that determine the user is centric.

[0123] In some examples, the query is a user context query for determining the current context of the user. Figure 15 As described, the subset of user-centric facts may include one or more user-centric facts related to the user's current context. When the query is a user-context query, the computer-readable representation of the query may be independent of any user query received at the natural language user interface 1610 and / or interpreted at the NLP mechanism 1620. Instead, the user-context query may be automatically issued based on any suitable data available to the computerized personal assistant (e.g., sensor data such as GPS data, time / date data, state data of the computerized personal assistant, and / or state data of any other cooperating computer service (e.g., a computer service configured to share data with the computerized personal assistant by providing user-centric facts to the user-centric AI knowledge base, and / or a computer service that can be controlled by the computerized personal assistant)).

[0124] Optionally, in some examples, the computerized personal assistant 1600 includes an output subsystem 1670 configured to output data (e.g., a response to a query output by the knowledge base query mechanism). For example, the output subsystem 1670 may include a speech synthesis engine configured to generate speech audio based on a subset of user-centric facts selected in response to the query, and a speaker configured to output the speech audio. In some examples, the output subsystem 1670 may include a display configured to visually present a text answer based on the subset of user-centric facts selected in response to the query. In some examples, the display may be configured to visually present the subset of user-centric facts directly in graphical form (e.g., as a graphical depiction of a graph data structure including the subset of user-centric facts, or as a timeline including user-centric facts arranged in chronological order of occurrence).

[0125] In some examples, the response output by the knowledge base query mechanism includes computer-readable instructions configured to cause the collaborative computer service to perform an action to assist the user based on a subset of user-centric facts in the user-centric AI knowledge base that satisfy one or more constraints defined by the computer-readable representation of the query. For example, the collaborative computer service can be a software application running on one or more devices that implements the computerized personal assistant. In other examples, the collaborative computer service can be a networked computer service accessible via a computer network. Thus, the output subsystem 1670 can include a communication device (e.g., a radio) configured to be communicatively coupled to the computer network so as to convey computer-readable instructions to the collaborative computer service.

[0126] In some examples, the actions performed by the collaborative computer service to assist the user include changing preference settings of the collaborative computer service based on a subset of user-centric facts in the user-centric AI knowledge base that satisfy one or more constraints defined by the computer-readable representation of the query (e.g., when the query includes a request to change a particular preference setting).

[0127] In some examples, the actions performed by the collaborative computer service to assist the user include visually presenting a depiction of state data of the collaborative service computer, wherein the state data of the collaborative computer service is related to user-centric facts in the user-centric artificial intelligence knowledge base that satisfy one or more constraints defined by the computer-readable representation of the query.

[0128] although Figure 16 A standalone computerized personal assistant is depicted, but any other computer service may include Figure 16Any subset of the components depicted in

[0015] (e.g., a natural language user interface, an NLP mechanism, an identity mechanism, an enrichment adapter, a knowledge base update mechanism, a knowledge base query mechanism, and / or an output subsystem). These components can facilitate the use of a user-centric AI knowledge base as described with respect to a computerized personal assistant (e.g., by providing data to the user-centric AI knowledge base and by servicing queries using the user-centric AI knowledge base). Any computer service that utilizes the user-centric AI knowledge base is a computerized personal assistant, regardless of whether such service is a stand-alone service or an auxiliary component of another computer service with a different primary function (e.g., an email / calendar application, a search engine, an integrated development environment).

[0129] In one example, a user composes an email in an email program, using the subject line "Weekly Report on Widget Development," and selects the user's boss, Alice, as the recipient. The user may click an "Analyze" button to receive suggestions for completing the email. Thus, the email program may issue a contextual query to be served by a user-centric AI knowledge base. The contextual query may indicate the state of the email program. Furthermore, the contextual query may include one or more natural language features based on the state of the application (e.g., the content in the subject line of the email). For example, the one or more natural language features may include an identified intent (e.g., "find information"), an identified subject ("widgets"), and an identified entity (e.g., the user's boss, Alice). Thus, in addition to user-centric facts about "widgets," the subset of user-centric facts selected in response to the query may include user-centric facts related to the user's exchanges with her boss, Alice.

[0130] Based on the subset of the user-centric facts selected in response to the query, the email program may display one or more suggestions based on the subset of the user-centric facts. For example, the email program may display one or more relevant files that the user may want to review when preparing the email and / or attaching to the email, such as a "Widget Report" spreadsheet file and a "Widget Development Notes" document file. Alternatively or additionally, the email program may display one or more links from the user's web search history that may be about "widgets" and / or about "widgets" more generally. Alternatively or additionally, the email program may display one or more related emails, such as an email from Alice saying "Please include estimated development costs for next month in this week's weekly report." Alternatively or additionally, the email program may display one or more email addresses of other users that may be relevant to the email, such as Alice's colleague Bob and resident widget expert Charlie. Alternatively or additionally, the email program may suggest that the user schedule a meeting with Alice. The email program may be configured to suggest a specific meeting time, for example, based on user-centric facts about the user's availability in the user-centric AI knowledge base, and additional user-centric facts about Alice's availability in the enterprise knowledge base.

[0131] Computer services configured to utilize the user-centric AI knowledge base may be implemented and / or organized in any suitable manner and are not limited to the components and organization shown in FIG6 . Figure 17 An exemplary method 1700 for providing a computer service with data to a user-centric AI knowledge base is shown. Figure 18 An exemplary method 1800 for a computer service using a user-centric AI knowledge base to service a query is shown. Methods 1700 and 1800 may be implemented by any suitable computing device and / or computer service, such as a computer service that cooperates with any computer service compatible with the AI knowledge base. Figure 16 Computerized Personal Assistant 1600.

[0132] At 1701, Figure 17 The method 1700 includes identifying a computer-readable representation of user input associated with a particular user of a computer service.

[0133] Optionally, in some examples, at 1702, via a natural language user interface (e.g., Figure 16 In some examples, the user input is received by an identity mechanism (e.g., a user) configured to associate the user input with a particular user. Figure 16The identity mechanism 1630) identifies the specific user of the computer service.

[0134] Optionally, in some examples, at 1703, a computer-readable representation of the user input is output by the NLP machine based on the user input. For example, Figure 16 The computerized personal assistant 1600 shown in FIG. 1 may be configured to output a computer-readable representation of the user input via an NLP mechanism 1620 .

[0135] At 1704, method 1700 includes updating a user-centric AI knowledge base associated with the particular user to include new or updated user-centric facts based on the computer-readable representation of the user input. "New user-centric facts" is used in this application to refer to any user-centric facts that are not already included in the user-centric AI knowledge base, such as user-centric facts that include subjects and / or objects that are not already included in any other user-centric facts in the user-centric AI knowledge base, or user-centric facts that include new edges between subjects and objects in the user-centric AI knowledge base. "Updated user-centric facts" is used in this application to refer to modifications of user-centric facts already defined in the user-centric AI knowledge base, such as modifications to user-centric facts to include one or more new tags and / or enrichments. In some examples, updates to user-centric facts by a knowledge base update mechanism such as Figure 16 The knowledge base update mechanism 1650) updates the user-centered AI knowledge base.

[0136] Optionally, in some examples, at 1705, the new or updated user-centric fact includes an enrichment based on the computer-readable representation of the user input. For example, Figure 16 The computerized graph personal assistant 1600 includes an enrichment adapter configured to output an enrichment based on the computer-readable representation of the user input. In other examples, the computer service may not include any enrichment adapter, but the new or updated user-centric fact may still include enrichment, for example, the enrichment may be added to the new or updated user-centric fact by an enrichment adapter of a graph storage mechanism included in an embodiment of the user-centric AI knowledge base.

[0137] At 1706, the update can be performed via an update protocol usable by a plurality of different computer services (e.g., as described above with reference to Figure 14 At 1707, the update protocol can constrain the storage format of the new or updated user-centric facts to an application-independent data format. For example, Figure 16The computerized personal assistant 1600 includes a knowledge base update mechanism 1650 configured to update the user-centric AI knowledge base via an update API.

[0138] Figure 18 An exemplary method 1800 is shown for a computer service serving queries using a user-centric AI knowledge base.

[0139] At 1801, method 1800 includes identifying a computer-readable representation of a query associated with a particular user of the computer service. In some examples, the query may be identified by an identity mechanism (e.g., Figure 16 In some examples, the specific user can be identified based on login information associated with the computer service and / or based on the user's ownership of a computer device (e.g., a mobile phone) that executes the computer service.

[0140] Optionally, at 1802, the query is sent via a natural language user interface (e.g., Figure 16 Thus, at 1803, the user query may be received by an NLP mechanism such as Figure 16 The NLP mechanism 1620 of the embodiment outputs a computer-readable representation of the query based on the user query. In some examples, the query includes constraints based on natural language features of the user query, such as based on the intent, subject, and / or entity identified from the user query. For example, if the user asks "Who is an expert in widgets?", the query may include a graph context constraint indicating that the answer should be about "widgets" and an answer type constraint indicating that the answer should include a user-centric fact that the subject and / or object is a person, based on recognizing that the user's intent is to find a specific person.

[0141] In some examples, the query may be a user context query for determining a current context of a user, and thus, the subset of user-centric facts may include one or more user-centric facts related to the current context. In some examples, the query may indicate a state of a computer service, and thus, the one or more user-centric facts related to the current context may include user-centric facts related to the state of the computer service. In some examples, method 1800 also includes identifying a computer-readable representation of a natural language feature defined by the state of the computer service, and thus, the one or more constraints include constraints based on the natural language feature. In some examples, identifying the computer-readable representation of the natural language feature may be performed by an NLP mechanism such as Figure 16 NLP mechanism 1620) is executed.

[0142] At 1804, method 1800 includes requesting a query via a query protocol usable by a plurality of different computer services (e.g., as described above with reference to Figure 15 In some examples, querying the user-centric AI knowledge base can be performed by a knowledge base query mechanism (such as a query API described in the preceding text). Figure 16 knowledge base query mechanism 1660) to execute.

[0143] At 1805, the user-centric AI knowledge base may be an update protocol that can be used via a plurality of different computer services (e.g., as described above with reference to Figure 14 The update protocol may be updated to include new or updated user-centric facts. At 1806, the update protocol may constrain the storage format of the new or updated user-centric facts to an application-independent data format.

[0144] At 1807 , method 1800 includes outputting a response based on a subset of user-centric facts in the user-centric AI knowledge base that satisfy one or more constraints defined by the computer-readable representation of the query.

[0145] At 1808, method 1800 optionally includes causing the computer service or a collaborating computer service to perform an action to assist the user based on a subset of user-centric facts in the user-centric AI knowledge base that satisfy one or more constraints defined by the computer-readable representation of the query. Causing the computer service to perform an action to assist the user may include outputting computer-readable instructions configured to cause the computer service or a collaborating computer service to perform the action to assist the user.

[0146] In some examples, a response based on a subset of the user-centric facts in the user-centric AI knowledge base is generated by an output subsystem (e.g., Figure 16 For example, the output subsystem 1670 may include a communication subsystem communicatively coupled to a collaborative computer device that implements the collaborative computer service via a computer network. Thus, outputting computer-readable instructions may include sending computer-readable instructions to the collaborative computer device via a computer network. In some examples, the output subsystem 1670 may include a display device, and the computer-readable instructions may be configured to cause the display device to visually present a depiction of state data of the computer service, the state data being related to a subset of user-centric facts in the user-centric AI knowledge base that satisfy one or more constraints defined by the query. Alternatively or in addition, the output subsystem 1670 may include a speaker, and the computer-readable instructions may be configured to generate voice audio describing the state data of the computer service, and cause the speaker to output the voice audio.

[0147] In some examples, a computer service may automatically provide facts to a user-centric AI knowledge base (e.g., according to method 1700 or in any other suitable manner using an update protocol for the user-centric AI knowledge base). For example, a computer service may continuously monitor sensor and / or state data of the computer service to provide facts about the context of a user of the computer service, such as GPS data indicating the user's location and clock data indicating the current time. Alternatively or additionally, in some examples, the computer service may automatically issue queries to be serviced by the user-centric AI knowledge base (e.g., according to method 1800 or in any other suitable manner using a query protocol for the user-centric AI knowledge base). For example, the computer service may repeatedly issue user context queries according to a schedule in order to monitor the user's context (e.g., in order to automatically perform actions related to the context).

[0148] The computerized personal assistant 1600 can provide a wide range of assistance to the user (e.g., by providing information to the user or by automatically performing tasks for the user). In one example, the computerized personal assistant 1600 can be configured to collaborate with multiple other computer services, including an email program and a sensor monitoring program configured to output global positioning system (GPS) data indicating the location of a user device (e.g., a mobile phone) belonging to user 1690. Each of the computerized personal assistant 1600, the email program, and the sensor monitoring program provides one or more user-centric facts to the user-centric AI knowledge base, for example, according to method 1700. For example, the email program can provide a new user-centric fact to the user-centric AI knowledge base each time the user sends an email. The new user-centric fact can include an object graph node indicating the identity of user 1690, a subject graph node indicating the recipient of the email, and an edge indicating a relationship of "sent email". The new user-centric fact can include one or more enrichments, for example, an enrichment indicating the identified subject of the email and a timestamp indicating the time the email was sent. The sensor monitoring program may also provide one or more new user-centric facts to the user-centric AI knowledge base, for example, by continuously monitoring the location of the user's mobile phone and providing a new user-centric fact indicating the location of the user's mobile phone and a corresponding timestamp whenever the location of the user's mobile phone changes. Thus, the user-centric AI knowledge base may contain multiple user-centric facts indicating the location of user 1690's mobile phone and multiple user-centric facts indicating each email sent by user 1690.

[0149] Furthermore, the user-centric AI knowledge base may include additional user-centric facts (e.g., added by the graph storage computer 1301) based on the enrichment of the new user-centric facts added by the email program and the new user-centric facts added by the sensor monitoring program. For example, the user-centric AI knowledge base may include an additional fact indicating that an email was likely sent from a particular location based on comparing a timestamp associated with the email with a timestamp associated with a fact provided by the sensor monitoring program.

[0150] The computerized personal assistant 1600 may later issue a query to be serviced by the user-centric AI knowledge base (e.g., via method 1800) to determine the location of the user's 1690 workplace. For example, the query may include a graph context constraint indicating "work-related emails" and an answer type constraint indicating "location." Thus, the subset of user-centric facts selected in response to the query may include locations from which the user 1690 may have sent one or more work-related emails. Based on the subset of user-centric facts, the computerized personal assistant 1600 may identify that a significant portion of work-related emails were sent from a particular location within a particular time range. Thus, the computerized personal assistant 1600 can identify the location of the user's 1690 workplace and the user's 1690 work schedule.

[0151] Subsequently, in a similar manner, the user-centric AI knowledge base may contain a user-centric fact identifying that user 1690 frequently sets an alarm (e.g., on her mobile phone) at night from a particular location and always silences the alarm the next morning. Accordingly, computerized personal assistant 1600 may issue a query regarding the location and time of the alarm. Based on the response to the query, computerized personal assistant 1600 may identify user 1690's home address and user 1690's sleep schedule.

[0152] Computerized personal assistant 1600 may be able to assist a user with a variety of different tasks based on identifying one or more aspects of user-centric facts in a user-centric AI knowledge base (e.g., by issuing queries according to method 1700). For example, when computerized personal assistant 1600 identifies user 1690's work and sleep schedule, computerized personal assistant 1600 may be able to set the user's alarm by default based on their typical scheduling preferences.

[0153] In some examples, such as Figure 16As depicted in FIG1 , the computerized personal assistant 1600 may be able to provide an enhanced response to a user query based on one or more aspects of the user-centric facts. For example, user 1690 may ask computerized personal assistant 1600, “Are there any good restaurants near my workplace?” as shown in speech bubble 1691. Accordingly, computerized personal assistant 1600 may recognize a computer-readable representation of the query (e.g., according to method 1800 at 1801). Computerized personal assistant 1600 may issue a query to be serviced by a user-centric AI knowledge base (e.g., via a query protocol usable by multiple different computer services, as described at 1804). The query may include a graph context constraint indicating “near my workplace” and an answer type constraint indicating “restaurant.” Accordingly, computerized personal assistant 1600 may output a response to the query (e.g., as described at 1807) based on a subset of the user-centric facts that satisfy the graph context constraint and the answer type constraint. For example, computerized personal assistant 1600 may suggest one or more restaurants within a convenient distance of user 1690’s workplace.

[0154] In some examples, the computerized personal assistant 1600 may request more information from the user 1690 in order to make a selection based on the user's 1960 preferences. For example, when the user 1690 asks to find good restaurants near work, the subset of user-centric facts selected in response to the query may indicate a number of different restaurants within a similar distance. Thus, the computerized personal assistant 1600 may ask a follow-up question indicating a specific restaurant, such as "How about 'Burrito' restaurant?" as shown in the speech bubble 1692.

[0155] In some examples, computerized personal assistant 1600 can provide one or more additional user-centric facts to the user-centric AI knowledge base in answering a user query (e.g., according to methods 1700 and 1800). For example, user 1690 can respond to the question "How is the 'Burrito' restaurant?" by saying "I don't like burritos," as shown in speech bubble 1693. Thus, computerized personal assistant 1600 can add a new fact to the user-centric AI knowledge base indicating that the user does not like burritos.

[0156] To determine a restaurant option that satisfies the user's preferences, the computerized personal assistant 1600 may ask an additional follow-up question, such as, "How about 'Sushi Restaurant'?" as shown in speech bubble 1694. If the user 1690 responds, "Okay, make a reservation after work," as shown in speech bubble 1695, the computerized personal assistant 1600 may infer when to schedule the reservation based on identifying the user's 1690 work schedule.

[0157] In addition to considering user 1690's work schedule, the user-centric AI knowledge base can enable computerized personal assistant 1600 to consider other potentially relevant factors, such as factors associated with one or more user-centric facts included in the user-centric AI knowledge base. For example, the user-centric AI knowledge base may include a user-centric fact that describes what time user 1690 typically prefers to eat. In some examples, the user-centric AI knowledge base may consider factors determined based on additional facts outside of the user-centric knowledge base, such as a predicted duration of a trip from user 1690's workplace to a "sushi restaurant" as determined based on GPS data associated with the user's location and an identified work schedule. For example, computerized personal assistant 1600 may be configured to determine the predicted trip duration via an API of a mapping service that provides geolocation and traffic planning functionality.

[0158] Based on when user 1690 typically leaves work, when user 1690 typically prefers to eat, and the predicted duration of the journey from user 1690's work to the "sushi restaurant," computerized personal assistant 1600 can determine a suitable time for the reservation, e.g., 6 PM. Thus, in response to a series of user queries and user input, computerized personal assistant 1600 can output a response, such as in speech bubble 1696, confirming that the reservation has been made (e.g., according to method 1800 at 1807). Furthermore, computerized personal assistant 1600 can output computer-readable instructions configured to cause computerized personal assistant 1600 and / or other cooperating computer services to perform actions to assist the user (e.g., according to method 1800 at 1808). For example, computerized personal assistant 1600 can output computer-readable instructions configured to schedule a reservation (e.g., via a restaurant reservation service that provides an API for making reservations). Additionally, computerized personal assistant 1600 can identify one or more new user-centric facts that can be added to the user-centric AI knowledge base, namely, that user 1690 does like sushi.

[0159] Based on interactions with and assistance from user 1690, computerized personal assistant 1600 and other computer services can continuously add new user-centric facts to the user-centric AI knowledge base and issue queries to make informed decisions based on the user-centric facts in the user-centric AI knowledge base. Thus, the user-centric AI knowledge base can enable computerized personal assistant 1600 and other computer services to continuously improve and provide assistance to user 1690 based on her preferences.

[0160] In some examples, the computerized personal assistant 1600 can automatically issue a series of queries (e.g., according to a schedule) to automatically perform actions to assist the user 1690 based on a subset of user-centric facts selected in response to each query. For example, when the user 1690 has a restaurant reservation at 'Sushi Restaurant' at 6:00 PM, the computerized personal assistant 1600 can be configured to repeatedly issue user-context queries at 2-minute intervals between 5:30 PM and 6:00 PM. The user-context query can include constraints related to the current time, the user 1690's current activity, and / or the user's location based on GPS data, and the subset of user-centric facts selected in response to the user-context query can include one or more user-centric facts related to the restaurant reservation, such as based on similarity in timestamp and location information. Later, the user 1690 can leave work at 5:50 PM and travel to 'Sushi Restaurant'. Thus, computerized personal assistant 1600 can recognize that user 1690 is approaching the restaurant and automatically assist user 1690 by visually presenting relevant information (e.g., confirmation of the reservation, the restaurant menu, and a map showing directions to the destination).

[0161] In some embodiments, the methods and processes described herein may rely on a computing system of one or more computing devices. Specifically, these methods and processes may be implemented as computer applications or services, application programming interfaces (APIs), libraries, and / or other computer program products.

[0162] Figure 19 A non-limiting embodiment of a computing system 1900 capable of specifying one or more of the above-described methods and processes is schematically illustrated. For example, computing system 1900 can function as graph storage mechanism 1301, application-specific data provider computer 1321, or user computer 1340. In some examples, computing system 1900 can provide the functionality of computerized personal assistant 1600 or any other computer service configured to utilize a user-centric AI knowledge base (e.g., according to method 1700 or method 1800, or in any other suitable manner utilizing an update protocol and / or query protocol for the user-centric AI knowledge base). Computing system 1900 is shown in simplified form. Computing system 1900 can take the form of one or more personal computers, server computers, tablet computers, home entertainment computers, network computing devices, gaming devices, mobile computing devices, mobile communication devices (e.g., smartphones), and / or other computing devices.

[0163] The computing system 1900 includes a logic mechanism 1901 and a storage mechanism 1902. The computing system 1900 may optionally include a display subsystem 1903, an input subsystem 1904, a communication subsystem 1905, and / or Figure 19 Other components not shown.

[0164] Logic mechanism 1901 includes one or more physical devices configured to execute instructions. For example, the logic mechanism can be configured to execute instructions that are part of one or more applications, services, programs, routines, libraries, objects, components, data structures, or other logical constructs. These instructions can be implemented to perform a task, implement a data type, transform the state of one or more components, achieve a technical effect, or otherwise achieve a desired structure.

[0165] The logic mechanism may include one or more processors configured to execute software instructions. Additionally or alternatively, the logic mechanism may include one or more hardware or firmware logic mechanisms configured to execute hardware or firmware instructions. The processors of the logic mechanism may be single-core or multi-core, and the instructions executed thereon may be configured for serial, parallel and / or distributed processing. The individual components of the logic mechanism may optionally be distributed between two or more separate devices, which may be located in remote locations and / or configured for coordinated processing. Aspects of the logic mechanism may be virtualized and executed by a networked computing device configured in a cloud computing configuration that is remotely accessible.

[0166] The storage mechanism 1902 includes one or more physical devices configured to hold executable instructions that can be executed by the logic mechanism to implement the methods and processes described in this application. When implementing these methods and processes, the state of the storage mechanism 1902 can be transformed—for example, to hold different data.

[0167] The storage mechanism 1902 may include removable and / or built-in devices. The storage mechanism 1902 may include optical storage (e.g., CD, DVD, HD-DVD, Blu-ray Disc, etc.), semiconductor memory (e.g., RAM, EPROM, EEPROM, etc.), and / or magnetic storage (e.g., hard drive, floppy drive, tape drive, MRAM, etc.), etc. The storage mechanism 1902 may include volatile, non-volatile, dynamic, static, read / write, read-only, random access, sequential access, location addressable, file addressable, and / or content addressable devices.

[0168] While storage mechanism 1902 comprises one or more physical devices, aspects of the instructions described herein may alternatively be propagated via a communication medium (eg, electromagnetic signals, optical signals, etc.) that is not retained by a physical device for a finite duration.

[0169] Aspects of the logic mechanism 1901 and the storage mechanism 1902 may be integrated together into one or more hardware logic components. These hardware logic components may include, for example, field programmable gate arrays (FPGAs), program and application specific integrated circuits (PASIC / ASICs), program and application specific standard products (PSSP / ASSPs), systems on chips (SOCs), and complex programmable logic devices (CPLDs).

[0170] The terms "module," "program," and "engine" may be used to describe several aspects of a computing system 1900 implemented to perform a particular function. In some cases, a module, program, or engine may be instantiated via a logic mechanism 1901 that executes instructions held by a storage mechanism 1902. The term "machine" may be used to describe one or more logic machines that instantiate these modules, programs, or engines. For example, the natural language processing mechanism described in this application may take the form of an ASIC or a general-purpose processor that runs software, firmware, or hardware instructions that translates raw user input (e.g., voice audio detected by a microphone) into a computer-readable representation of the input that is more suitable for downstream processing. It should be understood that different modules, programs, and / or engines may be instantiated on the same machine and / or from the same application, service, code block, object, library, routine, API, function, etc. Similarly, the same module, program, and / or engine may span two or more different machines and / or be instantiated by different applications, services, code blocks, objects, routines, APIs, functions, etc. The terms "module," "program," and "engine" may cover individual or grouped executable files, data files, libraries, drivers, scripts, database records, and the like.

[0171] When included, the display subsystem 1903 can be used to present a visual representation of the data held by the storage mechanism 1902. This visual representation can take the form of a graphical user interface (GUI). As the methods and processes described in this application change the data held by the storage mechanism and thereby transform the state of the storage mechanism, the state of the display subsystem 1903 can likewise be transformed to visually represent the changes in the underlying data. The display subsystem 1903 can include one or more display devices using virtually any type of technology. These display devices can be combined with the logic mechanism 1901 and / or storage mechanism 1902 in a shared package, or these display devices can be peripheral display devices.

[0172] When included, the input subsystem 1904 may include or interface with one or more user input devices, such as a keyboard, mouse, touch screen, or game controller. In some embodiments, the input subsystem may include or interface with selected natural user input (NUI) component portions. Such component portions may be integrated or peripheral devices, and the conversion and / or processing of input actions may be handled on-board or off-board. Example NUI component portions may include microphones for voice and / or sound recognition; infrared, color, stereo, and / or depth cameras for machine vision and / or gesture recognition; head trackers, eye trackers, accelerometers, and / or gyroscopes for motion detection and / or intent recognition; and electric field sensing component portions for assessing brain activity.

[0173] When included, the communication subsystem 1905 can be configured to communicatively couple the computing system 1900 to one or more other computing devices. The communication subsystem 1905 can include wired and / or wireless communication devices compatible with one or more different communication protocols. As non-limiting examples, the communication subsystem can be configured to communicate via a wireless telephone network, or a wired or wireless local area network or wide area network. In some embodiments, the communication subsystem can allow the computing system 1900 to send and / or receive messages to and / or from other devices via a network such as the Internet.

[0174] In one example, a computerized personal assistant includes: a natural language user interface configured to receive user input; a natural language processing mechanism configured to output a computer-readable representation of the user input; an identity mechanism configured to associate the user input with a specific user; and a knowledge base update mechanism configured to update a user-centric artificial intelligence knowledge base associated with the specific user to include new or updated user-centric facts based on the computer-readable representation of the user input, wherein the knowledge base update mechanism updates the user-centric artificial intelligence knowledge base via an update protocol usable by multiple different computer services, the update protocol constraining the storage format of the new or updated user-centric facts to an application-independent data format. In this or any other example, the computerized personal assistant further includes an enrichment adapter configured to output an enrichment based on the computer-readable representation of the user input, wherein the new or updated user-centric facts include the enrichment output by the enrichment adapter. In this or any other example, the user-centric artificial intelligence knowledge base includes one or more encrypted user-centric facts, wherein access to the one or more encrypted user-centric facts is constrained by credentials associated with the specific user. In this or any other example, the computerized personal assistant also includes a knowledge base query mechanism configured to query a user-centric artificial intelligence knowledge base associated with a particular user and output a response based on a subset of user-centric facts in the user-centric artificial intelligence knowledge base that satisfy one or more constraints defined by a computer-readable representation of the query. In this or any other example, the knowledge base query mechanism is configured to query the user-centric artificial intelligence knowledge base via a query protocol usable by multiple different computer services. In this or any other example, the natural language user interface is further configured to receive a user query; and the natural language processing mechanism is further configured to output a computer-readable representation of the query based on the user query. In this or any other example, the natural language processing mechanism is further configured to output a user intent identified based on the user query, wherein the one or more constraints defined by the computer-readable representation of the query include constraints based on the identified user intent. In this or any other example, the query is a user context query for determining a current context of the user, and the subset of user-centric facts includes one or more user-centric facts relevant to the current context. In this or any other example, the one or more constraints defined by the computer-readable representation of the query include an answer-type constraint; and the subset of user-centric facts includes only user-centric facts that satisfy the answer-type constraint.In this example or any other example, one or more constraints defined by the computer-readable representation of the query include graph context constraints; and the subset of user-centric facts includes only user-centric facts related to contextualized user-centric facts in the user-centric artificial intelligence knowledge base that satisfy the graph context constraints. In this example or any other example, the response output by the knowledge base query mechanism includes computer-readable instructions configured to cause the collaborative computer service to perform an action to assist the user based on the subset of user-centric facts in the user-centric artificial intelligence knowledge base that satisfy the one or more constraints defined by the computer-readable representation of the query. In this example or any other example, the action for assisting the user includes changing a preference setting of the collaborative computer service based on the subset of user-centric facts in the user-centric artificial intelligence knowledge base that satisfy the one or more constraints defined by the computer-readable representation of the query. In this example or any other example, the action for assisting the user includes visually presenting a depiction of state data of the collaborative computer service, the state data related to the subset of user-centric facts in the user-centric artificial intelligence knowledge base that satisfy the one or more constraints defined by the computer-readable representation of the query.

[0175] In one example, a computerized personal assistant includes: a natural language user interface configured to receive a user query; an identity mechanism configured to associate the user query with a specific user; a natural language processing mechanism configured to output a computer-readable representation of the user query; and a knowledge base query mechanism configured to query a user-centric artificial intelligence knowledge base associated with the specific user and output a response to the query based on a subset of user-centric facts in the user-centric artificial intelligence knowledge base that satisfy one or more constraints defined by the computer-readable representation of the query, wherein the user-centric artificial intelligence knowledge base can be updated to include new or updated user-centric facts via an update protocol that can be used by multiple different computer services, and the update protocol constrains the storage format of the new or updated user-centric facts to an application-independent data format.

[0176] In one example, a method for automatically responding to queries includes: identifying a computer-readable representation of a query associated with a particular user of a computer service; querying a user-centric artificial intelligence knowledge base associated with the particular user via a query protocol usable by multiple different computer services; and outputting a response based on a subset of user-centric facts in the user-centric artificial intelligence knowledge base that satisfy one or more constraints defined by the computer-readable representation of the query, wherein the user-centric artificial intelligence knowledge base can be updated to include new or updated user-centric facts via an update protocol usable by multiple different computer services, the update protocol constraining the storage format of the new or updated user-centric facts to an application-independent data format. In this or any other example, the computer service is a computerized personal assistant. In this or any other example, the query is a user-context query for determining a current context of the user, wherein the subset of user-centric facts includes one or more user-centric facts related to the current context. In this or any other example, the query indicates a state of the computer service, and the one or more user-centric facts related to the current context include user-centric facts related to the state of the computer service. In this or any other example, the method further includes identifying a computer-readable representation of a natural language feature defined by the state of the computer service, wherein the one or more constraints include constraints based on the natural language feature. In this or any other example, the method further includes causing the computer service to perform an action to assist the user based on a subset of user-centric facts in the user-centric artificial intelligence knowledge base that satisfy the one or more constraints defined by the computer-readable representation of the query.

[0177] It should be understood that the configuration and / or method described in this application are exemplary in nature, and these specific embodiments or examples should not be considered in a restrictive sense, because many variations are possible. Specific routines or methods described in this application can represent one or more of any number of processing strategies. Equally, the various actions shown and / or described can be according to the sequence shown and / or described, with other sequences or parallel execution or omitted. Equally, the order of the process described above can be changed.

[0178] The subject matter of the present disclosure includes all novel and non-obvious combinations and sub-combinations of the various processes, systems and configurations, and other features, functions, acts and / or properties disclosed in this application, and any and all equivalents thereof.

Claims

1. A computer system comprising: one or more computer processors; as well as A computer memory storing computer usable instructions that, when used by the one or more computer processors, cause the one or more computer processors to perform operations comprising: Receive inquiries; querying a user-centric artificial intelligence knowledge base, the user-centric artificial intelligence knowledge base comprising a graph data structure storing a plurality of user-centric facts, the user-centric artificial intelligence knowledge base further comprising an application-independent data format associated with a node record format, the node record format supporting aspect pointers to auxiliary application-specific data associated with application-specific facts in an application-specific context in which a user interacts with a plurality of different computer services; selecting a subset of user-centric facts that satisfy a set of constraints defined by the query, wherein the user-centric facts include one or more application-specific facts, the one or more application-specific facts being associated with corresponding auxiliary application-specific data, the corresponding auxiliary application-specific data having facet pointers that support retrieval and storage of the auxiliary application-specific data; as well as In response to the query, a subset of the user-centric facts is cached as a cached response.

2. The system according to claim 1, wherein: The query is a sharded query indicating a start node and a distance parameter, and wherein the subset of user-centric facts includes user-centric facts reached by starting at the start node and traversing a plurality of edges that are at most equal to the distance parameter from the start node.

3. The system according to claim 1, wherein: The query is a ranking query that ranks the plurality of user-centric facts based at least on a confidence value associated with each user-centric fact, and the subset of user-centric facts is ranked in order according to the confidence value of each user-centric fact.

4. The system of claim 1 , wherein the operations further comprise: The query is cached as a cached query.

5. The system of claim 4, wherein the operations further comprise: receiving a second query; determining that the second query corresponds to the cached query; as well as Based on the cache query, the cache response is output.

6. The system according to claim 1, wherein: The user-centric facts stored in the graph data structure are application-specific facts, and the graph data structure stores, for the application-specific facts, enrichments including application-independent facts associated with the application-specific facts.

7. The system according to claim 1, wherein: The aspect pointer supports multiple file types for storing different kinds of auxiliary application-specific data associated with auxiliary application-specific facts of the application-specific context.

8. One or more hardware computer storage media storing computer-usable instructions that, when used by one or more computing devices, cause the one or more computing devices to perform operations comprising: Receive inquiries; querying a user-centric artificial intelligence knowledge base, the user-centric artificial intelligence knowledge base comprising a graph data structure storing a plurality of user-centric facts, the user-centric artificial intelligence knowledge base further comprising an application-independent data format associated with a node record format, the node record format supporting aspect pointers to auxiliary application-specific data associated with application-specific facts in an application-specific context in which a user interacts with a plurality of different computer services; Select a subset of user-centric facts that satisfy the set of constraints defined by the query, where The user-centric facts include one or more application-specific facts, the one or more application-specific facts are associated with corresponding auxiliary application-specific data, the corresponding auxiliary application-specific data having an aspect pointer supporting retrieval and storage of the auxiliary application-specific data; as well as In response to the query, a subset of the user-centric facts is cached as a cached response.

9. The medium according to claim 8, wherein The query is a sharded query indicating a start node and a distance parameter, and wherein the subset of user-centric facts includes user-centric facts reached by starting at the start node and traversing a plurality of edges that are at most equal to the distance parameter from the start node.

10. The medium according to claim 8, wherein The query is a ranking query that ranks the plurality of user-centric facts based at least on a confidence value associated with each user-centric fact, and the subset of user-centric facts is ranked in order according to the confidence value of each user-centric fact.

11. The medium of claim 8, the operations further comprising: The query is cached as a cached query.

12. The medium of claim 11, the operations further comprising: receiving a second query; determining that the second query corresponds to the cached query; as well as Based on the cache query, the cache response is output.

13. The medium according to claim 8, wherein The user-centric facts stored in the graph data structure are application-specific facts, and the graph data structure stores, for the application-specific facts, enrichments including application-independent facts associated with the application-specific facts.

14. The medium according to claim 8, wherein The aspect pointer supports multiple file types for storing different kinds of auxiliary application-specific data associated with auxiliary application-specific facts of the application-specific context.

15. A method comprising: Receive inquiries; querying a user-centric artificial intelligence knowledge base, the user-centric artificial intelligence knowledge base comprising a graph data structure storing a plurality of user-centric facts, the user-centric artificial intelligence knowledge base further comprising an application-independent data format associated with a node record format, the node record format supporting aspect pointers to auxiliary application-specific data associated with application-specific facts in an application-specific context in which a user interacts with a plurality of different computer services; selecting a subset of user-centric facts that satisfy a set of constraints defined by the query, wherein the user-centric facts include one or more application-specific facts, the one or more application-specific facts being associated with corresponding auxiliary application-specific data, the corresponding auxiliary application-specific data having facet pointers that support retrieval and storage of the auxiliary application-specific data; as well as In response to the query, a subset of the user-centric facts is cached as a cached response.

16. The method according to claim 15, wherein The query is a sharded query indicating a start node and a distance parameter, and wherein the subset of user-centric facts includes user-centric facts reached by starting at the start node and traversing a plurality of edges that are at most equal to the distance parameter from the start node.

17. The method according to claim 15, wherein: The query is a ranking query that ranks the plurality of user-centric facts based at least on a confidence value associated with each user-centric fact, and the subset of user-centric facts is ranked in order according to the confidence value of each user-centric fact.

18. The method according to claim 15, further comprising: The query is cached as a cached query.

19. The method according to claim 15, further comprising: receiving a second query; determining that the second query corresponds to the cached query; as well as Based on the cache query, the cache response is output.

20. The method according to claim 15, wherein The user-centric facts stored in the graph data structure are application-specific facts, and the graph data structure stores an enrichment of the application-specific facts including application-independent facts associated with the application-specific facts; and wherein the aspect pointer supports multiple file types for storing different kinds of auxiliary application-specific data associated with the auxiliary application-specific facts of the application-specific context.

21. A computer system comprising: one or more computer processors; as well as A computer memory storing computer usable instructions that, when used by the one or more computer processors, cause the one or more computer processors to perform operations comprising: Receive inquiries; querying a user-centric artificial intelligence knowledge base, the user-centric artificial intelligence knowledge base comprising a graph data structure storing a plurality of user-centric facts, the user-centric artificial intelligence knowledge base further comprising an application-independent data format associated with a node record format, the node record format supporting aspect pointers to auxiliary application-specific data; generating a response to the query based on one or more aspects of the plurality of user-centric facts, wherein an aspect of the user-centric facts factors into information supporting generation of the response; and The response to the query is transmitted.

22. The system of claim 21, wherein: The auxiliary application-specific data is associated with application-specific facts of an application-specific context in which a user interacts with a plurality of different computer services.

23. The system of claim 21, wherein: The query is associated with a computerized personal assistant providing a natural language user interface.

24. The system of claim 21, wherein: The query is associated with a query protocol usable by a plurality of different computer services, or wherein the query includes a context constraint or an answer type constraint.

25. The system of claim 21, the operations further comprising: requesting additional information associated with the query; receiving additional information associated with the query; generating a subsequent response to the query based on the additional information; and The subsequent response is transmitted.

26. The system of claim 25, the operations further comprising: generating one or more additional user-centric facts based on the additional information; as well as The user-centric artificial intelligence knowledge base is updated with the one or more additional user-centric facts.

27. The system of claim 25, the operations further comprising: inferring a value of an aspect of the user-centric fact based on transmitting the subsequent response; as well as A secondary operation is performed based on the value of the one aspect of the user-centric fact.

28. One or more computer storage media storing computer-usable instructions that, when used by one or more computing devices, cause the one or more computing devices to perform operations comprising: Receive inquiries; querying a user-centric artificial intelligence knowledge base, the user-centric artificial intelligence knowledge base comprising a graph data structure storing a plurality of user-centric facts, the user-centric artificial intelligence knowledge base further comprising an application-independent data format associated with a node record format, the node record format supporting aspect pointers to auxiliary application-specific data; generating a response to the query based on one or more aspects of the plurality of user-centric facts, wherein An aspect of the user-centric fact is the factors that support the information that generated the response; and The response to the query is transmitted.

29. The medium according to claim 28, wherein The auxiliary application-specific data is associated with application-specific facts of an application-specific context in which a user interacts with a plurality of different computer services.

30. The medium of claim 28, wherein The query is associated with a computerized personal assistant providing a natural language user interface.

31. The medium of claim 28, wherein The query is associated with a query protocol usable by a plurality of different computer services, or wherein the query includes a context constraint or an answer type constraint.

32. The medium of claim 28, the operations further comprising: requesting additional information associated with the query; receiving additional information associated with the query; generating a subsequent response to the query based on the additional information; and The subsequent response is transmitted.

33. The medium of claim 32, the operations further comprising: generating one or more additional user-centric facts based on the additional information; as well as The user-centric artificial intelligence knowledge base is updated with the one or more additional user-centric facts.

34. The medium of claim 32, the operations further comprising: inferring a value of an aspect of the user-centric fact based on transmitting the subsequent response; as well as A secondary operation is performed based on the value of the one aspect of the user-centric fact.

35. A method comprising: Receive inquiries; querying a user-centric artificial intelligence knowledge base, the user-centric artificial intelligence knowledge base comprising a graph data structure storing a plurality of user-centric facts, the user-centric artificial intelligence knowledge base further comprising an application-independent data format associated with a node record format, the node record format supporting aspect pointers to auxiliary application-specific data; generating a response to the query based on one or more aspects of the plurality of user-centric facts, wherein an aspect of the user-centric facts factors into information supporting generation of the response; and The response to the query is transmitted.

36. The method according to claim 35, wherein The auxiliary application-specific data is associated with application-specific facts of an application-specific context in which a user interacts with a plurality of different computer services.

37. The method according to claim 35, wherein The query is associated with a query protocol usable by a plurality of different computer services, or wherein the query includes a context constraint or an answer type constraint.

38. The system of claim 35, the operations further comprising: requesting additional information associated with the query; receiving additional information associated with the query; generating a subsequent response to the query based on the additional information; and The subsequent response is transmitted.

39. The system of claim 35, wherein the operations further comprise: generating one or more additional user-centric facts based on the additional information; as well as The user-centric artificial intelligence knowledge base is updated with the one or more additional user-centric facts.

40. The system of claim 35, the operations further comprising: inferring a value of an aspect of the user-centric fact based on transmitting the subsequent response; as well as A secondary operation is performed based on the value of the one aspect of the user-centric fact.

Images