Computerized assistance using an artificial intelligence knowledge base

Abstract

A computerized personal assistant includes a natural language user interface, a natural language processing mechanism, an identity mechanism, and a knowledge base update mechanism. The knowledge base update mechanism is configured to update a user-centric artificial intelligence knowledge base associated with a particular user to include new or updated user-centric facts based on a computer-readable representation of a user input, wherein the knowledge base update mechanism updates the user-centric artificial intelligence knowledge base via an update protocol usable by a plurality of different computer services.

Description

Background Art

[0001] Artificial intelligence is an emerging field with almost limitless applications. It is believed that as AI technology advances, user-centric AI applications will have tremendous utility for personal computer users. The creation and maintenance of a robust AI knowledge base has been a serious obstacle to providing useful AI applications to personal computer users. Summary of the invention

[0002] This summary is provided to introduce a series of concepts that will be further described below in the detailed description in a simplified form. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. In addition, the claimed subject matter is not limited to implementations that address any or all of the shortcomings noted in any part of this disclosure.

[0003] The computerized personal assistant includes a natural language user interface, a natural language processing mechanism, an identity mechanism, and a knowledge base update mechanism. The knowledge base update mechanism is configured to provide user-centric facts to a user-centric artificial intelligence knowledge base associated with a user. The user-centric artificial intelligence knowledge base is updated via an update protocol usable by multiple different computer services and / or provides queries via a query protocol usable by multiple different computer services. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Figure 1A and Figure 1B A simplified graph data structure including a small number of multiple user-centric facts is shown.

[0005] Figure 2A , Figure 2B , Figure 3A , Figure 3B , Figure 4A , Figure 4B , Figure 5A , Figure 5B , Fig. 6A and Figure 6B Focusing on specific user-centric facts shows Figure 1A and 1B The graph data structure of .

[0006] Fig. 7A , Figure 7B , Fig. 8A , Figure 8B , Fig. 9A and Fig. 9B Focusing on the cross-references between the multiple component graph structures included in the graph data structure, Figure 1A and 1B The graph data structure of .

[0007] Fig. 10A , Fig. 10B , Fig.11A , Fig. 11B , Fig. 12A and Fig. 12B Show Figure 1A and 1B An exemplary implementation of a component graph structure of a graph data structure.

[0008] Fig.13 An exemplary computing environment for a user-centric artificial intelligence knowledge base is shown.

[0009] Fig.14 A method for maintaining a user-centric artificial intelligence knowledge base is presented.

[0010] Fig.15 A method for querying a user-centric artificial intelligence knowledge base is shown.

[0011] Fig.16 An exemplary computerized personal assistant is shown.

[0012] Fig.17 A method for a computer service to provide one or more user-centric facts to a user-centric artificial intelligence knowledge base is shown.

[0013] Fig.18 A method for computer service querying a user-centric artificial intelligence knowledge base is shown.

[0014] Fig.19 An example computing system for maintaining and querying a user-centric artificial intelligence knowledge base is schematically illustrated. DETAILED DESCRIPTION

[0015] A computer service can improve interaction with a user by collecting and / or analyzing user-centric data. As used in this application, "computer service" broadly refers to any software and / or hardware process with which a user can interact, either directly or via interaction with another computer service (e.g., a software application, an interactive website, or a server that implements a communication protocol). As used in this application, "user-centric" broadly refers to any data associated with or related to a particular computer user (e.g., facts and / or computer data structures representing user interests, user relationships, and user interactions with a computer service).

[0016] User-centric data can be provided as input to an artificial intelligence (AI) program, which can be configured to provide enhanced interaction between a user and a computer based on analysis of the provided user-centric data. For example, providing enhanced interaction can include predicting an action that a user is likely to take, and facilitating that action. However, in order to meaningfully enhance the interaction between a user and a computer, the AI ​​program requires very large amounts of data. Furthermore, this data must be centrally available in a suitable data format.

[0017] For example, a computerized personal assistant may include a natural language processing engine for processing natural language queries submitted by a user (e.g., via a natural user interface (NUI), such as a natural language user interface, configured to receive audio and / or text natural language queries). In order to serve natural language queries, the computerized personal assistant requires a knowledge base of facts. In some embodiments, a "knowledge base" may include a collection of facts represented as subject-predicate-object triples. A knowledge base may be generated using a combination of manually labeled data and data mining techniques to aggregate information from public sources (such as the Internet). Previous approaches for building knowledge bases have resulted in very large computational burdens (e.g., running data mining tasks for each of multiple different users), and / or a large amount of manual design and supervision (e.g., manually labeled data) to achieve good results.

[0018] Thus, previous methods for establishing and maintaining knowledge bases may not be suitable for utilizing AI to provide enhanced user interaction with computers. In contrast, a user-centric AI knowledge base as described in the present application extends the idea of ​​a knowledge base to support a collection of user-centric facts related to one or more specific users (e.g., a personal computer user, or a group of users of an enterprise computer network). A user-centric AI knowledge base may include user-centric facts generated from a variety of different application-specific data providers (e.g., a data provider associated with a computerized personal assistant, and a data provider associated with an address book program). A user-centric AI knowledge base can efficiently store multiple user-centric facts by distributing the storage of the user-centric facts across multiple storage locations (e.g., associated with different applications). In addition, a user-centric AI knowledge base may include various application-independent enrichments for the user-centric facts, which can assist AI processing of the knowledge base, such as answering queries.

[0019] A single user can interact with multiple different computer services, which can each identify and store information about the user. A computer service can gather data about the interaction with the user, and multiple different computer services can jointly gather a large amount of data about the user. Each computer service of these multiple computer services can store information about the user in different, application-specific locations. "Application-specific" is used in this application to mean specific to any computer service. In addition, each computer service of these multiple computer services can identify and only store a limited subset of possible information about the user, which is different from the information stored by the different computer services of these multiple computer services. Like this, information about the user can be distributed across multiple different storage locations.

[0020] In addition, application-specific data associated with computer services can be stored in application-specific storage formats. In this way, even when two different computer services have related functions, computer services may not be able to share data. In some cases, a first-party provider provides a set of related computer services (e.g., a document editing suite) that can share storage formats. However, even if the computer services in this suite can share data with each other, using this data to enhance interaction with users will further rely on data from external knowledge sources (such as global knowledge sources (e.g., the Internet), third-party software applications provided by different third-party software providers, and / or other users' usage data (e.g., in the context of enterprise software applications, or in the context of social network applications).

[0021] Software providers may wish to automatically aggregate information from different sources to build a user-centric AI knowledge base to facilitate improved user interactions. However, previous methods for building AI knowledge bases have focused only on global data, rather than user-centric data that can be specifically employed in the context of interactions with specific users. Therefore, previous methods for building knowledge bases are not suitable for building user-centric AI knowledge bases for users of multiple computer services.

[0022] Figure 1A An exemplary graph data structure 100 for representing a collection of user-centric facts that may be associated with application-specific data distributed across multiple different computer services is shown. The graph data structure 100 allows for focused queries and is suitable for implementing a user-centric AI knowledge base.

[0023] The graph data structure 100 includes a plurality of different component graph structures 102, for example, component graph A, component graph B, etc. Each component graph structure may be an application-specific component graph structure associated with a different computer service. For example, the application-specific component graph structure A may be associated with a scheduling program, while the application-specific component graph structure B may be associated with an email program.

[0024] Each component graph structure includes a plurality of user-centric facts 104, for example, a user-centric fact F stored in a component graph A A.1 、F A.2 , and the user-centric facts F stored in the component graph B B.1 、F B.2 . User-centric facts include a subject graph node 106, an object graph node 108, and an edge 110 connecting the subject graph node to the object graph node. Subject graph nodes and object graph nodes may be collectively referred to as nodes. Nodes may represent any noun, where "noun" is used to refer to any entity, event, or concept, or any suitable application-specific information (e.g., details of previous actions performed by a user using the computer service). Similarly, "subject noun" and "object noun" are used in this application to refer to a noun represented by a subject graph node or by an object graph node, respectively. Representing a collection of user-centric facts as a graph data structure facilitates manipulation and traversal of the graph data structure (e.g., in order to respond to queries).

[0025] Visualize the collection of user-centric facts as Figure 1B Figure 150 in is helpful. Figure 1B In the graph 150 depicted in FIG, nodes 152 are depicted as solid circles and edges 154 are depicted as arrows. Solid circles with outgoing edges (where the arrow points outward from the solid circle) depict subject graph nodes, while solid circles with incoming edges (where the arrow points into the solid circle) depict object graph nodes. The multiple component graphs are treated as a single graph by treating the edges between the component graphs as edges in a larger graph. Therefore, Figure 1B A single combined graph 150 is shown that includes multiple component graph structures. To simplify the explanation, the example graph 150 includes only two component graphs with twelve nodes. In actual implementations, the user-centric graph will include many more nodes (e.g., hundreds, thousands, millions, or more) interspersed among many more component graphs.

[0026] Figure 2A to Figure 2B Focus on specific user-centric facts A.1 Depicted Figure 1A to Figure 1B The graph data structure of user-centric facts F A.1 Depend on Figure 2A The thick rectangle in the Figure 2AOther user-centric facts of the graph data structure are not shown in detail in FIG. Bold line shapes are similarly used in the following figures to draw attention to specific facts. User-centric facts F A.1 Including the subject graph node S A.1 , Edge E A.1 and object graph node O A.1 For example, the subject graph node S A.1 can represent the user's employer, and the object graph node O A.1 can represent tasks assigned to the user by her employer. A.1 Can describe the subject graph node S A.1 and object graph node O A.1 Any suitable relationship between them. In the above example, the edge E A.1 It can represent the relationship of “assigning a new task”. User-centric fact F A.1 The subject-edge-object triplet of together represents the fact that the user's employer assigned her a new task.

[0027] The subject graph node of the first fact can represent the same noun as different object graph nodes of the second fact. For example, FIG. 3A to FIG. 3B Focus on user-centric facts A.3 Show Figure 1A to Figure 2B The same graph data structure as in Figure 2. User-centric facts F A.3 Define the subject graph node S A.3 , Edge E A.3 and object graph node O A.3 . Object graph node O A.3 The same noun can be expressed as Figure 2B The subject graph node S A.1 Therefore, the graph data structure can transform the fact F A.3 The object graph node O A.3 and the fact that A.1 The subject graph node S A.1 Identified as a single node, this Figure 2B and 3B 150. By recognizing that certain object graph nodes and subject graph nodes represent the same noun, the graph data structure is able to represent user-centric facts as complex relationships between multiple different nouns, which can be visualized as paths on graph 150. For example, when a particular node is an object graph node for a first fact and a subject graph node for a different second fact, it is possible to derive an inference from the combination of these two facts, similar to a logical syllogism.

[0028] A subject graph node of a first user-centric fact may represent the same noun as a different subject graph node of a second user-centric fact. When two different subject graph nodes represent the same noun, the graph data structure may identify the two subject graph nodes as a single node. For example, FIG. 4A to FIG. 4B Focus on user-centric facts A.4 Show Figures 1A to 3B The same graph data structure as in Figure 2. User-centric facts F A.4 Define the subject graph node S A.4 , Edge E A.4 and object graph node O A.4 . Subject graph node S A.4 The same noun can be expressed as Figure 3B The subject graph node S A.3 Therefore, the graph data structure can identify these two subject graph nodes as a single node, which is Figure 3B and 4B 150 in the same position. Although the subject graph node S A.4 and S A.3 can be identified as a single node, edge E A.4 Is with edge E A.3 Differently, similarly, the object graph node O A.4 is the object graph node O A.3 Therefore, even if the subject graph node S A.4 and S A.3 It means the same noun, but the triple (S A.4 , E A.4 , O A.4 ) and (S A.3 , E A.3 , O A.3 ) represent two different facts.

[0029] Similarly, a subject graph node of a first user-centric fact may represent the same noun as an object graph node of a different second user-centric fact. In other words, the same noun may be the object of multiple different user-centric facts having different subject graph nodes and possibly edges representing different relationship types. For example, FIG. 5A to FIG. 5B Focus on user-centric facts A.5 Show Figures 1A to 4B The same graph data structure as in Figure 2. User-centric facts F A.5 Define the subject graph node S A.5 , Edge E A.5 and object graph node O A.5 . Object graph node O A.5 The same noun can be expressed as Figure 2BThe object graph node O A.1 Therefore, the graph data structure can identify these two object graph nodes as a single node, which is Figure 2B and 5B Depicted in the same position as in Figure 150.

[0030] Two or more different user-centric facts may involve a specific pair of subject nouns and object nouns. For example, the subject nouns and object nouns may be represented by a subject graph node S A.5 , Edge E A.5 and object graph node O A.5 The first user-centric fact F A.5 At the same time, as in FIG. 6A to FIG. 6B As depicted in the figure, the subject graph node S A.6 can represent the same subject noun, while the object graph node O A.6 It can also represent the same object noun, such as Figure 6B S A.6 and O A.6 The location and Figure 5B S A.5 and O A.5 Therefore, the subject graph node S A.5 The first edge E A.5 Connect to object graph node O A.5 , and the subject graph node S A.6 Via a second different edge E A.6 Connect to object graph node O A.6 Like the subject graph node and the object graph node, the edge E A.6 exist Figure 6B is depicted in the E A.5 exist Figure 5B However, the edge E A.5 and E A.6 are different edges, for example, representing different relationships between subject and object graph nodes. In one example, the subject graph node S A.5 and S A.6 Can correspond to the first user account (e.g., identified by an email address). In the same example, the object graph node O A.5 and O A.6 can correspond to different second user accounts. A.5 can represent the relationship of "sent email to", and the edge E A.6 Relationships representing different “scheduled a meeting with…” Thus, the graph data structure includes two or more user-centric facts with the same subject and object nouns.

[0031] In other examples, two nouns may be involved in two different user-centric facts, but their roles are swapped between subject and object. In other words, the first noun is the subject noun of the first fact, and the second noun is the object noun of the first fact, while the first noun is the object noun of the second fact, and the second noun is the subject noun of the second fact. For example, a pair of nouns representing "Alice" and "Bob" is included in the first fact stating "Alice arranged a meeting with Bob", and is also included in the second fact stating "Bob arranged a meeting with Alice". In addition to swapping the roles of subject and object, two facts using these two nouns can have different types of edges, for example, "Alice" and "Bob" may also be included in the third fact stating "Bob sent an email to Alice".

[0032] As mentioned above and as Figure 1A , 2A , 3A, 4A, 5A and 6A, the graph data structure includes multiple application-specific component graph structures (e.g., corresponding to different computer services). For example, FIG. 7A to FIG. 7B Focusing on two different user-centric facts F A.6 and F B.1 Show Figures 1A to 6B The same graph data structure as the user-centric fact F A.6 The subject graph node S A.6 , Edge E A.6 and object graph node O A.6 is defined as part of the component graph structure A. Similarly, the user-centric fact F B.1 The subject graph node S B.1 , Edge E B.1 and object graph node O B.1 Defined as part of the component graph structure B.

[0033] The graph data structure can identify nouns from one component graph, and identify nouns from different component graphs as single nodes. For example, FIG. 8A to FIG. 8B Focus on user-centric facts B.3 Show Figures 1A to 7B The same graph data structure as in Figure 2. User-centric facts F B.3 Define the subject graph node S B.3 , Edge E B.3 and object graph node O B.3 . Object graph node O B.3 The same noun can be expressed as Figure 2B The object graph node O A.1 , Figure 5B The object graph node O A.5 , Figure 7BThe object graph node O A.5 and Figure 8B The object graph node O B.3 Therefore, the graph data structure can identify these four object graph nodes as a single node, which is Figure 2B , 5B , 7B and 8B in the same position depicted in Figure 150. It is worth noting that the object graph node O B.3 is in the component graph structure B, while the object graph node O A.1 , O A.5 and O A.5 is in component graph structure A. This identification of two or more different nodes corresponding to the same noun may be referred to in this application as a "node cross-reference" between two nodes. A subject graph node or an object graph node representing a particular noun may store node cross-references to other nodes of the same component graph structure or any other component graph structure.

[0034] Similarly, a user-centric fact in a first component graph structure may define a subject graph node in the first component graph structure, along with an edge (referred to herein as a "cross-reference edge") pointing to an object graph node in a different second component graph structure. For example, FIG. 9A to FIG. 9B Focus on user-centric facts A.2 Show Figures 1A to 8B The same graph data structure as in Figure 2. User-centric facts F A.2 Including the subject graph node S A.2 and edge E AB.2 However, the edge E AB.2 Points to object graph node O B.2 Instead of pointing to different object graph nodes in the component graph structure A. Edge E AB.2 Connections between component graphs may be indicated in any suitable manner, for example, by storing a component graph identifier indicating a connection to an object graph node in component graph B, and an object graph node identifier indicating a particular object graph in component graph B.

[0035] Node cross-references and cross-reference edges connect multiple component graph structures. For example, node cross-references and cross-reference edges can be traversed in the same manner as edges, allowing traversal of the graph data structure to traverse along a path that spans multiple component graph structures. In other words, the graph data structure 100 facilitates an overall artificial intelligence knowledge base that includes facts from and / or across different computer services. Node cross-references and cross-reference edges may be collectively referred to as cross-references in this application. Similarly, when a node is included in a cross-reference in multiple component graph structures, the node may be referred to as being cross-referenced across the component graph structure.

[0036] In addition, in addition to connecting two different component graph structures through cross-references, the component graph structure can include one or more user-centric facts involving subject graph nodes in the component graph structure and object graph nodes in an external knowledge base (e.g., an Internet-based, social network, or networked enterprise application software). Due to the use of cross-reference edges, outgoing edges connected to subject graph nodes can indicate connections to external object graph nodes in any suitable manner, such as by storing paired identifiers indicating the external graph and object graph nodes in the external graph. In some cases, the external graph may not store any user-centric data, such as when the external graph is a global knowledge base derived from public knowledge on the Internet.

[0037] By including cross-references between component graph structures, facts about a particular noun (e.g., an event or entity) can be distributed across multiple component graph structures while still supporting centralized reasoning about relationships between user-centric facts in different component graph structures and in external databases (e.g., by traversing the multiple component graph structures via cross-references and cross-reference edges).

[0038] Each user-centric fact can be stored in a predictable shared data format, which stores user-centric facts including application-specific facts associated with a computer service without changing the format of the application-specific data of the computer service. Such a predictable shared data format is referred to as an "application-independent data format" in this application. The application-independent data format can store the information required to query the user-centric AI knowledge base while avoiding redundant storage of application-specific data. The graph data structure can be implemented with a complementary application programming interface (API) that allows read and write access to the graph data structure. The API can constrain access to the graph data structure to ensure that all data written to the graph data structure is in the application-independent data format. At the same time, the API can provide a mechanism that any computer service can use to add new user-centric facts to the graph data structure in an application-independent data format, thereby ensuring that all data stored in the graph data structure is predictably usable by other computer services using the API. In addition to providing read / write access to user-centric facts stored in the graph data structure, the API can provide data processing operations that include both read and write, such as query operations and caches of query results.

[0039] A graph data structure including user-centric facts about a plurality of different computer services may be implemented differently without departing from the spirit of the present disclosure, such as Figure 1A A graph data structure 100. Fig. 10AOne such non-limiting embodiment of a node-centric data structure 200 is schematically illustrated, including a node record for each noun. Each node record represents one or more subject and / or object graph nodes associated with the noun. When a node record represents a subject graph node, the node record additionally represents an outgoing edge connecting the subject graph node to one or more object graph nodes. For example, Fig. 10A Two node records are shown to fully represent Figures 1A to 9B The data structure 100 of all nodes requires multiple node records of node records NR A

[42] and node record NR A

[43] The more generalized graph data structure 100 is described above to generally introduce the features of the user-centric multi-service AI knowledge base. In practice, a more efficient data storage method (such as the node-centric data structure 200) can be used to implement the generalized features introduced above.

[0040] The node-centric data structure 200 stores each node of the component graph structure at a node record storage location defined by a consistent node record identifier associated with the node. The node record identifier can be a numeric and / or textual identifier, a reference to a computer storage location (e.g., an address in computer memory or a file name on a computer disk), or any other suitable identifier (e.g., a uniform resource locator (URL)). For example, a node record NR A

[42] may be identified by a node record identifier 'A

[42] ', which includes a node field identifier 'A' identifying the node record as part of a component graph structure A, and includes an additional numeric identifier 42. Thus, the graph data structure may identify the node record NR A

[42] is stored in the node record storage location defined by the identifier 'A

[42] '. For example, the graph data structure may store the node record NR A

[42] Stored in row #42 of the database table associated with the component graph structure A. It should be noted that the bracket nomenclature (e.g., A

[42] ) is illustrative and is used to emphasize that while the node-centric data structure 200 ultimately defines the same node / graph as the broader graph data structure 100, the specific implementation of the node data structure 200 is different. However, the graph data structures described in this application may be implemented using any suitable nomenclature.

[0041] exist Fig. 10A In the example, the node records NR A

[42] The subject graph node S representing the more generalized graph data structure 100 A.3 and S A.4 , and the node records NR A

[43] The subject graph node S representing the more generalized graph data structure 100 A.7and S A.8 and object graph node O A.4 Therefore, in Fig. 10B In the node record NR A

[42] Depicted in the figure are Figure 3B and 4B The subject graph node S A.3 and S A.4 Same location. Return to Fig. 10A , node record NR A

[42] Extended to display subject graph nodes S in an application-independent data format A.3 and S A.4 In one example, the user-centric facts stored in the graph data structure are application-specific facts. In addition to the application-specific facts, the user-centric facts may include one or more enrichments, which are additional data including application-independent facts associated with the application-specific facts.

[0042] In addition to the application-independent representation of connections in the graph data structure, the application-independent data format for user-centric facts allows for efficient storage of application-specific facts. A.3 The subject of an application-specific fact may be represented, for example, the fact that a new meeting is scheduled. Therefore, the graph data structure stores, for an application-specific fact, an aspect pointer indicating auxiliary application-specific data associated with the application-specific fact. In this example, the aspect pointer P A

[42] Is the instruction with S A.3 An identifier (e.g., a numeric identifier) ​​of the storage location of the associated auxiliary application specific data (e.g., the storage location of a calendar entry representing the details of the meeting). A

[42] The aspect pointer may be associated with a particular type of data (e.g., calendar data) based on its inclusion in the application-specific graph structure A, because the application-specific graph structure A is associated with a calendaring software. In other examples, the aspect pointer may include additional identification information that specifies the type of the auxiliary application-specific data, so that the aspect pointer may be used to represent different kinds of auxiliary application-specific data (e.g., multiple file types that may be used by a word processing application). The graph data structure may be used to locate the auxiliary application-specific data via the aspect pointer while avoiding redundant storage of the auxiliary application-specific data in the graph data structure.

[0043] In addition to the aspect pointers stored in the subject graph nodes, application-specific facts are further represented by edges connecting the subject graph node to one or more object graph nodes. Although a single subject graph node may be included in more than one user-centric fact, the graph data structure 200 still efficiently stores only a single node record for the subject graph node. This node record includes a list of outgoing edges for all nodes, which reduces storage space requirements relative to storing a copy of the subject graph node for each user-centric fact in which the subject graph node appears. The list of outgoing edges may be empty for some nodes, for example, for nouns that are just object graph nodes. The list of outgoing edges for a subject graph node includes one or more edge records that define one or more edges. The one or more edges may be stored in an edge record, such as an ER graph node. A

[261] and ER A

[375] .although Fig. 10A A node record is shown with two outgoing edges, and a node record may include any number of edges to suitably represent relationships with respect to other nodes of the graph data structure, such as zero, one, or three or more edges.

[0044] An edge from a subject graph node to an object graph node specifies the object graph node record identifier associated with that object graph node, e.g., node record NR A

[42] The object graph node ID of 'A

[55] ' is a consistent node record identifier that indicates the storage location of the node record that defines the object graph node, such as NR A

[55] The edge may further specify the object domain identifier of the component graph structure storing the object graph node, for example, indicating NR A

[55] The node record NR stored in the component graph structure A A

[42] The object domain ID is 'A

[55] '. Therefore, Fig. 10B Shows the node record NR A

[42] Record ER via node A

[261] and ER A

[375] Indicates the edge connected to the node record NR A

[55] and NR A

[21] Return to Fig. 10A , an edge from a subject graph node to an object graph node (e.g., as represented in an edge record) can further specify the type of relationship between the subject graph node and the object graph node. A

[261] Define the relationship type R A

[261] , which may indicate a "scheduled meeting" relationship, while the edge records ER A

[375] Define the relation R A

[375] , which may indicate a "committed" relationship.

[0045] In addition to representing application-specific facts via aspect pointers and outgoing edge lists, subject graph nodes can represent one or more enrichments of the application-specific facts. In one example, the one or more enrichments include node confidence values, e.g., node record NR A

[42] The node confidence C A

[42] Node confidence C A

[42] Can instruct nodes to record NR A

[42] (and the subject graph nodes it represents) to the user. For example, the confidence value can be determined by a machine learning model trained to identify relevance to the user, by learning to distinguish labeled samples of relevant data from labeled samples of irrelevant data. For example, training the machine learning model can include supervised training using user-labeled samples (e.g., derived from direct user feedback during use by an application), and / or unsupervised training. Fig. 10A In the example, the node records NR A

[42] The node confidence C A

[42] Can be compared to the node record NR A

[55] The node confidence C A

[55] Higher values ​​indicate that the node is recording NR A

[42] is considered to be better than node record NR A

[55] More relevant to that user.

[0046] exist Fig. 10A In the example shown in FIG. 1 , the one or more enrichments also include edge confidence values ​​associated with each edge, for example, edge record ER A

[261] The edge confidence K A

[261] Like node confidence values, edge confidence values ​​can indicate whether a particular edge (e.g., ER A

[261] ) is relevant to the user. Different edges between a pair of nodes can have different confidence values. For example, an edge record indicating the “scheduled meeting” relationship A

[261] Can have a margin record ER than indicates "make a commitment" A

[375] The edge confidence K A

[375] Lower edge confidence K A

[261] , for example, if the scheduled meeting is deemed more relevant to the user than the commitment.

[0047] In addition to the node confidence value and the edge confidence value, one or more enrichments of the application-specific facts may include other application-independent and / or application-specific data. A

[42] Including indication and access node record NR A

[42] associated with application-specific data (eg, by aspect pointer P A

[42] Access metadata M of information associated with the indicated data) A

[42] . Access metadata M A

[42] A timestamp indicating the time and date of the most recent access, a delta value indicating changes resulting from the most recent access, or any other suitable metadata may be included.

[0048] The graph data structure may additionally store one or more tags defining auxiliary data associated with the user-centric fact for the user-centric fact. A

[42] Also includes label T A

[42] , which may include any other suitable assistance data associated with the node. A

[42] When representing a person (e.g., associated with a contact book entry), the tag T A

[42] You can include a nickname for the person and an alternate email address for the person.

[0049] The additional semantics stored in the label can be used to enrich the user-centric facts and the nodes / edges of the user-centric facts. For example, labels can be used to store one or more enrichments of user-centric facts. The labels stored in the node records can be associated with the user-centric facts in which the node record represents the subject graph node or in which the node record represents the object graph node. Alternatively or additionally, the labels stored in the node records can be associated with the node record itself (for example, associated with the subject graph node represented by the node record and / or associated with the object graph node represented by the node record), or associated with one or more edges connected to the node record. In other examples, labels can be used to store the metadata of user-centric facts (for example, replacing or in addition to the access metadata of user-centric facts). For example, when the user-centric facts are associated with timestamps, the timestamps can be optionally stored between the labels of the user-centric facts.

[0050] In some examples, the one or more tags are searchable tags, and the graph data structure is configured to enable searching for user-centric facts by searching within the searchable tags (e.g., searching for a tag that stores a search string, or searching for a tag that stores a specific type of data).

[0051] The graph data structure may be represented as a plurality of application-specific component graph structures, wherein nodes of the component graph structures are cross-referenced across the component graph structures (e.g., as described above with reference to FIG. 8A to FIG. 9B described, by cross-referencing edges and node cross-referencing). Fig.11A Focus on the subject graph node S A.7 , S A.8 Node NR A

[43] and object graph node O A.4 , shown in Fig. 10AAnother example view of the same node-centric data structure 200 is shown in FIG. A

[43] The list of outgoing edges includes an edge record ER indicating the object graph node ID 'A

[61] ' A

[213] , represents a node in the same component graph structure A. From NR A

[43] The list of outgoing edges also includes an edge record ER indicating the object domain ID 'B' and the object graph node ID 'B

[61] ' A

[435] , represents a node in the component graph structure B. Therefore, the edge record ER A

[435] is the cross-reference edge. Fig. 11B Depicted with Figures 1A to 9B The same graph data structure, including NR A

[43] and its NR A

[61] and NR B

[25] The exit edge.

[0052] Fig. 12A Focus on the object graph node O A.1 , O A.5 , O A.6 and O A.7 Node NR A

[67] Shows Fig. 10A Another exemplary view of a node-centric data structure 200 is shown in FIG. A

[67] The list of outgoing edges is empty because NR A

[67] represents only object graph nodes, which have only incoming edges. However, NR A

[67] Also includes a data structure pair representing the graph O A.1 , O A.5 , O A.6 and O A.7 Represent the same nodes as O of the component graph structure B B.3 Therefore, the cross-reference specifies the reference domain ID 'B' indicating the component graph structure B, and indicates the cross-reference node O B.3 A node record NR is stored in a data structure centered on another node representing the component graph structure B. B

[67] The reference node ID in is 'B

[76] '. Fig. 12B Graphically depicts the Figures 1A to 9B The same graph data structure as shown in . Note that the node records NR A

[67] Depicted with Figure 5B and 6B The object graph node O A.5 and O A.6 At the same location, Figure 8B The object graph node O A.5 Same location.

[0053] The node-centric data structure 200 is application-independent because it can be used to track facts from two or more potentially unrelated computer services, which may have different native data formats. The node-centric data structure 200 can store application-specific facts for any computer service by storing aspect pointers so that user-centric facts include application-specific facts even when application-specific facts can be stored in an application-specific format. In addition, the node-centric data structure 200 validates the representation of user-centric facts defined in the context of two or more computer services by storing cross-references in the form of domain identifiers (e.g., object domain identifiers in the outgoing edge list of each subject graph node, or reference domain identifiers in the representation of node cross-references) and node identifiers (e.g., object graph node identifiers and reference node identifiers). In addition, the node-centric data structure 200 is user-centric because a different node-centric data structure 200 can be defined for each user. The node-centric data structure 200 can be suitable for storing user-centric facts in the context of multiple different users interacting with a shared computer service (e.g., a web browser). Because the node-centric data structure 200 stores application-specific facts via aspect pointers and represents relationships to facts in other data structures via cross-references, it is able to store user-centric facts about a user in a user-centric graph data structure specific to the user without requiring write access to application-specific data of the shared computer service.

[0054] Fig.13 An exemplary computing environment 1300 for maintaining a user-centric AI knowledge base is shown. Computing environment 1300 includes a graph storage mechanism 1301, which is communicatively coupled to multiple application-specific data provider computers (e.g., application-specific data provider computers 1321, 1322, and 1329, etc.) via a network 1310. Graph storage mechanism 1301 and the multiple application-specific data provider computers are further communicatively coupled to one or more user computers of a user (e.g., user computer 1340 and user computer 1341) via network 1310. For example, user computer 1340 can be a user's desktop computer, and user computer 1341 can be the user's mobile computing device. Network 1310 can be any suitable computer network (e.g., the Internet).

[0055] In some examples, computing environment 1300 may additionally include a computerized personal assistant 1600 that is communicatively coupled to graph storage mechanism 1301 via network 1310. Computerized personal assistant 1600 may be implemented in any suitable manner, such as as an all-in-one computing device, or as a software application executable on any suitable computing device (such as a desktop computer or mobile phone). The computerized personal assistant may be a stand-alone computer service or an assistant component of another computer service (e.g., an email / calendar application, a search engine, an integrated development environment).

[0056] In some examples, computing environment 1300 may additionally include cloud services 1311. Cloud services 1311 may be communicatively coupled to other computing devices of computing environment 1300 via network 1310. Cloud services 1311 may include one or more computing devices configured to perform any suitable tasks. In some examples, cloud services 1311 may be used to offload functionality of another computing device to cloud services 1311. For example, functionality of graph storage mechanism 1301, application-specific data provider computer 1321, user computer 1340, and / or computerized personal assistant 1600 may be offloaded to cloud services 1311.

[0057] In one example, cloud service 1311 is configured to perform a natural language processing task. Graph storage mechanism 1301 may be configured to perform a natural language processing task by offloading the task to cloud service 1311. Thus, graph storage mechanism 1301 may offload input data of a natural language processing task to cloud service 1310, and receive output data indicating a result of the natural language processing task from cloud service 1310. Alternatively or additionally, computerized personal assistant 1600 may be configured to perform a natural language processing task with remote processing assistance from cloud service 1311. In a similar manner, a computing device of computing environment 1300 may offload any suitable task to cloud service 1311, where cloud service 1311 is configured to perform the offloaded task.

[0058] The graph storage mechanism 1301 may be implemented as a single machine (e.g., a computer server). Alternatively, the functionality of the graph storage mechanism 1301 may be distributed across multiple different physical devices (e.g., by implementing the graph storage mechanism 1301 as a virtual service provided by a computer cluster). Each application-specific data provider computer (e.g., application-specific data provider computer 1321) may be associated with one or more computer services of a user, such as a social networking application, an email application, a calendaring application, an office suite, an IoT appliance function, a web search application, etc.

[0059] As a user interacts with one or more computer services via user computer 1340 and / or user computer 1341, the application-specific data provider computer may aggregate information related to the user's interaction with the one or more computer services. In an example where a user interacts with a social networking application, application-specific data provider computer 1322 may be communicatively coupled to a server that provides functionality for the social networking application. Thus, as the user interacts with the social networking application, the server may provide an indication of the interaction to application-specific data provider computer 1322. In turn, application-specific data provider computer 1322 may provide one or more user-centric facts to graph storage mechanism 1301. In an example, user computer 1340 may execute one or more applications that may provide additional user-centric facts aggregated at user computer 1340 to graph storage mechanism 1301, making user computer 1340 an additional application-specific data provider. In an example, user computer 1340 may execute one or more applications that may request user-centric facts from graph storage mechanism 1301 (e.g., by sending a query). While the above examples include a user interacting with one or more computer services via user computer 1340, in other examples, the user may interact with the one or more computer services via user computer 1341 instead of or in addition to user computer 1340. For example, when user computer 1340 acts as an application-specific data provider by providing one or more facts to graph storage mechanism 1301, user computer 1341 may execute one or more application programs to request user-centric facts from graph storage mechanism 1301. In this manner, graph storage mechanism 1301 may aggregate user-centric facts from multiple different user computers of the user while also allowing each of the different user computers to request and use the user-centric facts.

[0060] In one example, the user computer 1340 may execute a computerized personal assistant configured to communicate with the graph storage mechanism 1301 via the network 1310. The computerized personal assistant may receive queries from the user via a NUI configured to receive audio and / or text natural language queries. The computerized personal assistant may send one or more queries to the graph storage mechanism 1301 to receive one or more user-centric facts output by the graph storage mechanism 1301 in response to the query. Alternatively or in addition, the computerized personal assistant may aggregate user-centric facts (e.g., user interests indicated in a conversation through the NUI) to enable the user computer 1340 to act as an application-specific data provider. In some cases, the graph storage mechanism 1301 and the application-specific data provider may be managed by a single entity or organization, in which case the application-specific data provider may be referred to as a "first-party" application-specific data provider. In other cases, the graph storage mechanism 1301 and the application-specific data provider may be managed by different entities or organizations, in which case the application-specific data provider may be referred to as a "third-party" application-specific data provider.

[0061] Fig.14 An exemplary method 1400 for maintaining a user-centric artificial intelligence knowledge base is shown. The user-centric AI knowledge base can be suitable for storing user-centric facts about a user interacting with multiple different, possibly unrelated computer services. In addition, the user-centric AI knowledge base can be used to answer queries about user-centric facts. Maintaining the user-centric AI knowledge base includes building a knowledge base by storing one or more user-centric facts, updating the user-centric AI knowledge base by adding additional facts, and updating the user-centric AI knowledge base as it is queried and used (e.g., storing cached answers to queries so that the queries can be quickly answered later).

[0062] At 1401, method 1400 includes maintaining a graph data structure including a plurality of user-centric facts associated with a user. Each user-centric fact may have an application-independent data format, for example, including a subject graph node, an object graph node, and an edge connecting the subject graph node to the object graph node. The graph data structure may be represented as a plurality of application-specific component graph structures, wherein nodes of the component graph structures are cross-referenced across the component graph structures. The graph data structure may be stored using any suitable application-independent data format, as described above with reference to FIG. 10A to FIG. 12B A node-centric data structure 200 is described.

[0063] At 1402, method 1400 optionally includes automatically sending a request to an application-specific data provider to provide user-centric facts. Sending a request to an application-specific data provider can be done in any suitable manner, such as over a computer network via an API of the application-specific data provider. The request can indicate that specific user-centric facts should be provided (e.g., user-centric facts from a specific range of time and / or date, user-centric facts that the application-specific data provider has not yet provided, and / or user-centric facts with specific associated tags). Alternatively, the request can indicate that all available user-centric facts should be provided. Sending a request to an application-specific data provider and receiving user-centric facts in response to the request can be referred to as "pulling" data from an application-specific data provider in this application. Additional user-centric facts can be requested according to any suitable schedule (e.g., periodically). In addition to providing additional user-centric facts in response to a request, one of the multiple application-specific data providers can send user-centric facts in the absence of a request to do so, which can be referred to as "pushing" user-centric facts to a user-centric AI knowledge base in this application.

[0064] At 1403, method 1400 includes receiving a first user-centric fact from a first application-specific data provider associated with a first computer service (e.g., due to pulling data from the application-specific data provider, or due to the application-specific data provider pushing the user-centric fact to a user-centric AI knowledge base). As described at 1404, the user-centric fact, such as the first user-centric fact, can be received via an update protocol (e.g., an update API) that constrains the storage format of the user-centric fact to an application-independent data format. For example, the update API can constrain the storage format to use a specific data storage format, such as an implementation of the node record format described above. In addition, the update API can constrain the maximum disk usage of the stored data, and / or require that the data be stored in an encrypted data format.

[0065] At 1405, method 1400 includes adding a first user-centric fact to the graph data structure in an application-independent data format. For example, adding the first user-centric fact to the graph data structure may include translating the first user-centric fact into the above reference FIG. 10A to FIG. 12B The first user-centric fact may be an application-specific fact. Thus, at 1406, the graph data structure may store an aspect pointer for indicating auxiliary application-specific data associated with the application-specific fact, as described above with respect to FIG. 10A to FIG. 12B as described.

[0066] At 1406, the graph data structure may optionally store application-independent enrichments, e.g., application-independent facts associated with application-specific facts. The enrichments may be included in user-centric facts received from application-specific data providers. Alternatively or in addition, user-centric facts provided by application-specific data providers may be pre-processed via an enrichment pipeline including one or more enrichment adapters to include one or more enrichments. When the graph data structure stores labels associated with user-centric facts (e.g., labels stored in node records), the one or more enrichments may be included in the labels.

[0067] In one example, the enrichment adapter includes a machine learning model configured to receive application-specific facts to identify the relevance of the application-specific facts to the user and output a confidence value numerically indicating the relevance. The machine learning model can be any applicable model, such as a statistical model or a neural network. The machine learning model can be trained, for example, based on user feedback. For example, when the machine learning model is a neural network, the output of the neural network can be evaluated via an objective function, which indicates the error level of the predicted relevance output by the neural network compared to the actual relevance indicated in the user feedback. The gradient of the objective function can be calculated in terms of the derivative of each function in the layer of the neural network using back propagation. Therefore, the weights of the neural network can be adjusted based on the gradient (e.g., via gradient descent) to minimize the error level indicated by the objective function. In some examples, the machine learning model can be trained for a specific user based on direct feedback provided by the user while interacting with the software application (e.g., indicating the relevance of search results in a search application). Therefore, the trained machine learning model may be able to estimate the relevance of the user. In some examples, the machine learning model can be trained based on indirect feedback from the user (e.g., by estimating the similarity of relevant content to other content that the user has indicated as relevant in the past).

[0068] In another example, the enrichment adapter includes a natural language program for identifying natural language features of application-specific facts. For example, the natural language program can determine the subject graph node and / or object graph node of the application-specific fact by identifying the natural language feature as associated with an existing subject and / or object graph node. In some examples, the natural language program can determine the relationship type of an edge of the application-specific fact. In some examples, the natural language program can determine one or more labels of the subject graph node and / or object graph node of the application-specific fact. The natural language program can be configured to recognize features including: 1) named entities (e.g., people, organizations, and / or objects), 2) intent (e.g., emotions or goals associated with natural language features), 3) events and tasks (e.g., tasks that a user intends to do at a later time), 4) topics (e.g., topics contained or represented by user-centric facts), 5) locations (e.g., geographic locations referenced by user-centric facts, or locations where user-centric facts are generated), and / or 6) dates and times (e.g., timestamps for indicating past events or future scheduled events associated with user-centric facts).

[0069] The enrichment associated with a user-centric fact can provide enriched semantics of the user-centric fact (e.g., additional meaningful information, information beyond that provided by the connectivity structure formed by the edges between the subject graph node and the object graph node of the user-centric fact). The graph data structure can identify and include additional user-centric facts that can be derived from the enriched semantics (e.g., based on one or more enrichments added to the enrichment pipeline). Therefore, adding a user-centric fact including one or more enrichments can also include identifying additional user-centric facts based on the one or more enrichments, and adding the additional user-centric facts to the graph data structure in an application-independent data format.

[0070] Identifying additional user-centric facts based on the one or more enrichments may include identifying that an enrichment of the one or more enrichments corresponds to another user-centric fact already included in the graph data structure (e.g., because the enrichment is associated with a subject noun or an object noun of the other user-centric fact). Alternatively or in addition, identifying the additional user-centric facts based on the one or more enrichments may include: identifying the additional user-centric facts based on the one or more enrichments may include identifying a first enrichment of the one or more enrichments associated with a subject noun not yet included in any user-centric fact, identifying that a second enrichment of the one or more enrichments is associated with an object noun, identifying a relationship between the subject noun and the object noun, and adding a new user-centric fact containing the object noun and the subject noun to the graph data structure. In some examples, identifying the additional user-centric facts based on the one or more enrichments includes identifying any suitable relationships between the one or more enrichments, and adding user-centric facts representing the identified relationships.

[0071] In one example, each of the first node and the second node includes an enrichment for specifying an identified named entity, wherein the two enrichments specify the same named entity. Thus, the graph data structure can store an edge connecting the first node to the second node, and the relationship type of the edge can indicate that the two nodes are inferred to be associated with the same entity. Alternatively or additionally, the graph data structure can store node cross-references in each node, indicating that other nodes are associated with the same named entity. Alternatively, the graph data structure can modify the first node to include data for the second node and delete the second node, thereby avoiding redundant storage of data for the second node by collapsing the representation to include a single node instead of two nodes.

[0072] In another example, a first node includes an enrichment for specifying a named entity, and an edge can be added connecting the first node to a second node representing the same named entity. In another example, an edge can be added between the first node and a second node having the same associated topic. In another example, an edge can be added between a first node and a second node having the same associated time and / or location. For example, an edge can be added between a first node that refers to a particular calendar date (e.g., in a tag) and a second node that was created on the particular calendar date (e.g., as indicated by accessing metadata). In another example, an edge can be added between two nodes that were created at the same location or that refer to the same location.

[0073] At 1407, adding user-centric facts to the graph data structure in an application-independent data format optionally includes storing the user-centric facts as encrypted user-centric facts, wherein access to the encrypted user-centric facts is constrained by a certificate. For example, the certificate may be a user account certificate associated with a user, and the encrypted user-centric facts may only be read by the holder of the user account certificate. In another example, the certificate may be an enterprise account certificate associated with a user in an enterprise computer network, and the encrypted user-centric facts may only be read by the holder of the enterprise account certificate and by one or more administrators of the enterprise computer network. The user-centric facts received from the application-specific data provider may be received as encrypted user-centric facts, encrypted using the certificate. If so, the encrypted user-centric facts may be stored in the same encrypted form using the same certificate without decrypting and re-encrypting the encrypted user-centric facts. In addition, the encrypted user-centric facts may be encrypted using a homogeneous encryption scheme, in which case the encrypted user-centric facts may be modified (e.g., to add enrichment) without decrypting and re-encrypting the encrypted user-centric facts, and then storing the modified encrypted user-centric facts. Alternatively, the user-centric facts received from the application-specific data provider may be encrypted using a different certificate, in which case the received user-centric facts may be decrypted and re-encrypted using the certificate before storing the resulting re-encrypted user-centric facts. Alternatively, the user-centric facts received from the application-specific data provider may not be encrypted, in which case the received user-centric facts may be encrypted using the certificate before storing the resulting encrypted user-centric facts.

[0074] In addition to storing user-centric facts as encrypted user-centric facts, the graph data structure can also provide additional privacy and security by refreshing the graph data structure in response to a flush trigger to redact one or more user-centric facts. For example, a refresh trigger can be a user command to redact one or more encrypted user-centric facts. The user command can indicate specific user-centric facts (e.g., user-centric facts in response to a specific query, or user-centric facts from a specific range of times / dates). Alternatively or in addition, the user command can indicate that the entire user-centric AI knowledge base should be revised. In another example, the refresh trigger can be an automatically scheduled trigger, for example, occurring periodically, or once at a specific schedule in the future.

[0075] At 1408, method 1400 includes receiving a second user-centric fact from a second application-specific data provider associated with a second computer service different from the first computer service. The second user-centric fact can be received in any suitable manner (e.g., as described above with respect to receiving the first user-centric fact).

[0076] At 1409, method 1400 includes adding a second user-centric fact to the graph data structure in an application-independent data format. The second user-centric fact can be added to the graph data structure in any suitable manner (e.g., as described above with respect to adding the first user-centric fact to the graph data structure). Compared to a knowledge base that includes only user-centric facts obtained from a single computer service, the application-independent data format can assist the user-centric AI knowledge base with improved utility (e.g., in answering user queries). Although the first user-centric fact and the second user-centric fact can be received from two different computer services, which may have different, incompatible native data formats, both the first user-centric fact and the second user-centric fact can be saved in the same application-independent data format. In addition, although the above example includes user-centric facts from two different computer services, there is no limit to the number of different computer services that can contribute to the knowledge base. In addition, there is no requirement that different computer services are related to each other or to the user-centric AI knowledge base in any particular way (e.g., different computer services and the user-centric AI knowledge base can be independent of each other and provided by different computer service providers). Therefore, a user-centric AI knowledge base can include user-centric facts obtained from multiple different computer services, thereby including more user-centric facts from more different contexts. In addition, cross-references between application-specific component graph structures enable user-centric facts to express relationships between aspects of different computer services, which can further improve practicality compared to maintaining multiple different, scattered knowledge bases without cross-references.

[0077] In some cases, a second user-centric fact may have the same subject noun as a first user-centric fact already stored in the graph data structure, while having an object noun and an edge that are different from the first user-centric fact. Thus, adding the second user-centric fact may include identifying that the second user-centric fact has the same subject noun as the first user-centric fact, and modifying a node record representing the first user-centric fact to include a new edge record representing a new outgoing edge from the subject noun to the object noun of the second user-centric fact. Thus, a node record may initially represent the first user-centric fact, and the node record may subsequently be updated to additionally represent the second user-centric fact.

[0078] At 1410, method 1400 optionally includes outputting a subset of user-centric facts included in the graph data structure in response to the query, wherein the subset of user-centric facts is selected to satisfy a set of constraints defined by the query. Responding to the query can be accomplished in any suitable manner, such as according to Fig.15 Method 1500.

[0079] Fig.15 An exemplary method 1500 for responding to a query is shown. The query defines a set of constraints that can be satisfied by a subset of user-centric facts in a user-centric AI knowledge base. The constraints may include any suitable features of the subject graph nodes, object graph nodes, and edges that define the user-centric facts, such as any of the following: 1) the type of subject and / or object graph nodes included in the user-centric facts (e.g., the subject graph node represents a person); 2) the identity of the subject and / or object graph nodes included in the user-centric facts (e.g., the object graph node represents a specific email message); 3) the type of edge connecting the subject graph node and the object graph node; 4) the confidence value of the subject graph node, object graph node, and / or edge; 5) the range of dates and / or times of day associated with the subject graph node, object graph node, and / or edge; and / or 6) any other features of the subject graph node, object graph node, and / or edge, such as access metadata and / or tags of the subject graph node. The answer to the query includes a subset of user-centric facts that satisfy the set of constraints, or an indication that the set of constraints cannot be satisfied.

[0080] The query may be received in any suitable manner. For example, the query may be received via a query protocol (e.g., a query API) that allows a client to programmatically define the set of constraints in a computer-readable query format. Alternatively, the query may be received as a natural language query and converted into a set of constraints by identifying the constraints using natural language processing techniques. For example, a user-centric AI knowledge base can train a semantic embedding model to represent a natural language query and a set of constraints as points in a latent space learned by the semantic embedding model, so as to translate a natural language query into a set of constraints by identifying a point in the latent space corresponding to a natural language query and outputting a set of constraints corresponding to the point in the latent space. Alternatively or additionally, the user-centric AI knowledge base can use a parsing model (e.g., dependency analysis) to match the grammatical structure of a natural language query with a template query, and specify a set of constraints by filling in the details of the template query with the details of the natural language query.

[0081] At 1501, method 1500 includes identifying a query that has been previously used as a cached query, and outputting a cached response to serve the cached query by answering with the same subset of user-centric facts that satisfied the cached query when previously received. Thus, at 1507, method 1500 includes outputting the subset of user-centric facts that were previously cached. Such a cache can efficiently (e.g., immediately) retrieve the cached response if the cached query is received again in the future.

[0082] If the query has not been served, then at 1502, method 1500 includes selecting a subset of user-centric facts that satisfy a set of constraints defined by the query. The set of constraints can be satisfied by traversing a graph data structure to find user-centric facts that at least partially satisfy the constraints. The graph data structure represents structured relationships between user-centric facts (e.g., two facts with the same subject noun can be represented by a single node in a component graph and cross-referenced between component graphs). In this way, traversing the graph data structure to find user-centric facts that satisfy the query may be more efficient than exhaustively searching a set of user-centric facts. For example, if a user frequently interacts with a particular other person, the frequent interaction may indicate that the other person may be related to the user. Therefore, there may be more user-centric facts that have that person as a subject or object, and at the same time, it is more likely to traverse the edges of the graph data structure and encounter nodes representing the other person because many edges lead to and point out of nodes representing that person.

[0083] Traversing the graph data structure may include a "random walk" along the edges of the graph data structure. The random walk may begin in a current user context, used in this application to refer to any suitable starting point for answering a query. In an example, the current user context may be defined by a query (e.g., by including a contextual keyword indicating a subject graph node to use as a starting point). In other examples, the current user context may be an application-specific context suitable for determining a subject graph node to use as a starting point (e.g., "answering an email").

[0084] When a node is encountered during a random walk (e.g., at the starting point), the node can be checked to determine whether it satisfies the constraints of the query. If so, it can be output to a subset of user-centric facts that respond to the query. Then, after encountering the node, the random walk can continue and encounter more nodes. In order to find more nodes, the random walk can continue along the outgoing edges of the encountered nodes. Determining whether to continue along the outgoing edge can be a weighted random determination, including evaluating the weight representing the possibility of following the edge, and sampling whether to follow the edge based on the weight and a random number (e.g., a "roulette wheel selection" algorithm implemented using a random number generator). The weight of the edge connecting the subject graph node to the object graph node can be determined based on the confidence value of the subject graph node, the edge node, and / or the object graph node. In one example, the confidence value can be interpreted as an indication of user relevance, so edges that are more relevant or connect more relevant nodes are more likely to be followed. The weight of the edge may be further determined based on other data of the subject graph node, object graph node, and edge, such as by evaluating the relevance of the edge to the query based on a natural language comparison of the edge type with one or more natural language features of the query.

[0085] By specifying constraints (e.g., characteristics of subject graph nodes, object graph nodes, and edges that define user-centric facts), users can formulate various questions that need to be answered using the user-centric AI knowledge base.

[0086] In addition to selecting a subset of user-centric facts that satisfy the constraints specified in the query, the user-centric AI knowledge base is able to respond to other specialized queries. Fig.15 Two exemplary specialized queries are shown: a slice query and a rank query.

[0087] In one example, at 1504, the query is a shard query indicating a start node and a distance parameter. The answer to the shard query is a subset of user-centric facts that includes user-centric facts reached by starting at the start node and traversing the edges of the graph data structure to form a path of at most the length of the distance parameter from the start node. For example, if the distance parameter is set to 1, the answer to the query will include the start node and all once-removed nodes directly connected to the start node; and if the distance parameter is set to 2, the answer to the query will include the start node, all once-removed nodes, and all twice-removed nodes directly connected to at least one of the once-removed nodes. The shard query can represent a set of user-centric facts that are potentially related to a specific user-centric fact of interest (e.g., user-centric facts involving the start node) by connecting to the start node via a path of at most the distance parameter. By setting a small distance parameter, the answer to the query can represent a relatively small set of facts that are closely related to the start node, and similarly, by setting a large distance parameter, the answer can represent a larger set of facts that are indirectly related to the start node. As an alternative to specifying a start node, a query may also use the current user context as a start node, thereby representing a set of user-centric facts about the user's current context.

[0088] In another example, at 1505, the query is a ranking query for ranking multiple user-centric facts based at least in part on a confidence value associated with each user-centric fact, and a subset of the user-centric facts is ranked according to the confidence value of each user-centric fact. The ranking query can be interpreted as collecting user-centric facts that may be relevant to the user without imposing additional specific constraints on the query. In addition to ranking the multiple user-centric facts based on confidence values, the multiple user-centric facts can also be ranked based on other features. For example, if the user-centric facts are more recent (e.g., based on a timestamp associated with each fact), they can be weighted as more relevant. In another example, the ranking query can include a keyword, and if the user-centric fact includes at least one node with the keyword in its label, it can be weighted as more relevant.

[0089] although Fig.15 Depicts two examples of specialized queries, but does not Fig.15A user-centric AI knowledge base capable of other categories of specialized queries is depicted in . For example, a subset of user-centric facts in response to a sharded query can be sorted by confidence value as in a ranked query, thereby combining the functionality of both queries. In another example, the query is a pivot query that indicates a starting node. The answer to the pivot query is a subset of user-centric facts that include user-centric facts reached by starting at the starting node and traversing the edges of the graph data structure to form a path of infinite (or arbitrary, large) length. A pivot query can be interpreted as a sharded query that does not limit the length of the path reached by the starting node (e.g., where the distance parameter is infinite). In some examples, the answer to the query may include a graphical diagram (e.g., visualizing a subset of answers to user-centric facts) Figure 2A ), which may be annotated or animated to include any suitable information for that user-centric fact (e.g., auxiliary application-specific data indicated by an aspect pointer included in one of the user-centric facts).

[0090] A query may define a time constraint such that a subset of user-centric facts output as a response to the query is limited to user-centric facts associated with timestamps indicating times within a range defined by the query. In some examples, time is an inherent property of nodes and edges in the graph data structure (e.g., each user-centric fact included in a plurality of user-centric facts in the graph data structure is associated with one or more timestamps). One or more timestamps associated with a node or edge may indicate a time when the user-centric fact was created, accessed, and / or modified (e.g., access metadata for a node included in the user-centric fact). Alternatively or in addition, the one or more timestamps may indicate a time referenced in the user-centric fact (e.g., the time a meeting was scheduled, or any other timestamp added by an enrichment adapter in an enrichment pipeline). The one or more timestamps may optionally be stored as a searchable tag.

[0091] In some examples, one or more constraints defined by the query include an answer type constraint, and thus, a subset of user-centric facts selected in response to the query may include only user-centric facts that satisfy the answer type constraint. For example, an answer type constraint may limit the characteristics of the subject graph nodes, object graph nodes, and / or edges of the user-centric fact. For example, an answer type constraint may indicate a specific type of subject and / or object graph node, such as: 1) either the subject or the object is a person; 2) both the subject and the object are colleagues; 3) the subject is a location; 4) the object is a topic; or 5) the subject is a person and the object is a scheduled event. Alternatively or additionally, the answer type constraint may indicate one or more specific subjects and / or objects, such as 1) the subject is a user; 2) the subject is the user's boss, Alice; or 3) the object is any one of Alice, Bob, or Charlie. Alternatively or additionally, the answer type constraint may indicate one or more edges of a particular type, for example, by indicating a relationship type such as a "sent email" relationship, a "went to lunch" relationship, or a "research topic" relationship.

[0092] In some examples, one or more constraints defined by the query may include a graph context constraint, and therefore, a subset of user-centric facts selected in response to the query may include only user-centric facts related to contextualized user-centric facts in the user-centric AI knowledge base that satisfy the graph context constraint. Based on any feature of the graph data structure that can indicate a possible relationship, two different user-centric facts may be described as related in the present application. For example, when a graph context constraint indicates Alice, the boss of the user, the contextualized user-centric fact may be any fact having a node representing Alice as a subject graph node or an object graph node. Therefore, the subset of user-centric facts selected in response to the query may include other user-centric facts having a subject and / or object graph node directly connected to the node representing Alice via an edge. Alternatively or in addition, the subset of user-centric facts may include user-centric facts indirectly related to Alice, for example, user-centric facts having a subject and / or object graph node indirectly connected to the node representing Alice via a path of two or more edges. In some cases, the query including the graph context constraint can be a sharded query, and the subset of user-centric facts can include only user-centric facts that are related to the contextualized user-centric fact and reachable at most within a certain distance of the node of the contextualized user-centric fact. In other examples, the query including the graph context constraint can be a sorted query, and the subset of user-centric facts can include some user-centric facts that are most likely related to the contextualized user-centric fact, for example, user-centric facts that are connected to the contextualized user-centric fact via many different paths, or via paths that include edges with high confidence values.

[0093] Answering a query may include traversing a graph data structure based on one or more timestamps associated with each user-centric fact, which may be referred to herein as a traversal of a time dimension of the graph data structure. For example, answering a query may include starting at a node associated with a time defined by the query and traversing the graph by following any edges having a timestamp indicating a later time, such that the timestamps increase in the same order as the traversal. In other examples, answering a query may include traversing the graph data structure by following any edges having a timestamp prior to a date defined by the query. In addition, the inherent time attributes of each node and edge in the graph data structure may form a timeline view of the graph data structure. In an example, an answer to a query may include a timeline view of the graph data structure, e.g., with the user-centric facts arranged in chronological order of occurrence according to the timestamps associated with each user-centric fact.

[0094] As another example of specialized queries, the graph data structure can be configured to allow searching for user-centric facts based on searchable tags stored by the graph data structure for the user-centric facts. Searching for user-centric facts based on searchable tags can include traversing the graph in any suitable manner (e.g., as described above with respect to shard queries or pivot queries), and while traversing the graph, outputting any user-centric facts encountered during the traversal for which the graph structure stores searchable tags.

[0095] As another example of specialized queries, a graph data structure may be configured to serve user context queries by searching for user-centric facts that may be relevant to the user's current context. Thus, a user context query may include one or more constraints related to the user's current context. For example, the one or more constraints may include a time constraint based on the current time when the user context query is served. Alternatively or in addition, the one or more constraints may include a graph context constraint related to the user's current context, for example, a graph context constraint that specifies tasks in which the user may participate.

[0096] In some examples, one or more constraints of a user context query may be based on state data of a computer service that issues the user context query. In some examples, the state data of the computer service includes natural language features (e.g., intent, entity, or subject), and the one or more constraints include an indication of the natural language features. For example, when the computer service is an email program, the one or more constraints of the user context query may include: 1) a time constraint based on the time when the user starts writing an email; 2) a graph context constraint indicating the subject of the subject of the email; and 3) a graph context constraint indicating the recipient of the email. Thus, the subset of user-centric facts selected in response to the user context query may include user-centric facts that are current (based on timestamps) and may be relevant to the user's task of writing an email (based on subject and recipient).

[0097] After selecting a subset of user-centric facts in response to the query, the subset of user-centric facts can be cached to enable immediate responses to the same query in the future. Thus, at 1506, method 1500 optionally includes caching the query as a cached query and caching the subset of user-centric facts in response to the query as a cached response. Then, at 1507, method 1500 includes outputting the subset of user-centric facts in response to the query, which may include directly outputting the selected subset of user-centric facts, or after caching the selected subset of user-centric facts, outputting the resulting subset of cached user-centric facts.

[0098] When the graph data structure includes encrypted user-centric facts, the graph data structure can be filtered to create a filtered graph data structure that does not include one or more encrypted user-centric facts, but includes other user-centric facts without revealing that one or more encrypted user-centric facts are excluded. For example, the encrypted user-centric facts to be excluded can be selected by a user (e.g., by selecting encrypted user-centric facts in response to a query, or by selecting encrypted user-centric facts from a specific time / date range). The filtered graph data structure can still be used to answer the query, but does not include any encrypted user-centric facts in the answer. The filtered graph data structure omits the excluded user-centric facts without in any way indicating the absence of the excluded user-centric facts. For example, the answer will not indicate that one or more encrypted user-centric facts appear but have been revised. Instead, the answer will simply omit the one or more encrypted user-centric facts, while potentially including other user-centric facts.

[0099] A user-centric AI knowledge base utilizing the above-described graph data structure can support improved interaction between a user and one or more computer services. A computer service can be used as an application-specific data provider by providing data (e.g., user-centric facts) to the user-centric AI knowledge base. Alternatively or additionally, the computer service can use the user-centric AI knowledge base to answer queries. For example, a computer service can use the user-centric AI knowledge base to provide information to a user (e.g., in response to a user query). In some examples, a computer service can be configured to automatically perform actions to assist a user based on user-centric facts in the user-centric AI knowledge base.

[0100] When each of a plurality of computer services contributes user-centric facts to a user-centric AI knowledge base, the user-centric AI knowledge base can assist information sharing between the plurality of computer services. Therefore, the user-centric AI knowledge base can enable one or more computer services to assist the user in a collaborative manner. For example, a first computer service can provide one or more user-centric facts to the user-centric AI knowledge base, and a second computer service can perform actions to assist the user based on the one or more user-centric facts. In this way, a second computer service can provide services related to the first computer service, even when the data required to provide such functionality is not directly available in the second computer service, and when such functionality is not included in the first computer service.

[0101] Examples of computer services that can use the user-centric AI knowledge base include: 1) computerized personal assistants; 2) email clients; 3) calendar / scheduling programs; 4) word processing programs; 5) presentation editing programs; 6) spreadsheet programs; 7) programming / publishing programs; 8) integrated development environments (IDEs) for computer programming; 9) social networking services; 10) workplace collaboration environments; and 11) cloud data storage and file synchronization programs. However, the use of the user-centric AI knowledge base is not limited to the above examples of computer services, and any computer service can use the user-centric AI knowledge base in any suitable manner (e.g., by providing data and / or by issuing queries).

[0102] The computer service utilizing the user-centric AI knowledge base may be a first-party computer service authorized and / or managed by an organization or entity that manages the user-centric AI knowledge base, or a third-party service authorized and / or managed by a different organization or entity. The computer service may utilize the giant user-centric AI knowledge base via one or more APIs (e.g., update APIs and query APIs) of the user-centric AI knowledge base, each of which may be used by multiple different computer services including first-party computer services and third-party services.

[0103] Fig.16 An exemplary computer service is shown in the form of a computerized personal assistant 1600. The computerized personal assistant 1600 can utilize the functionality of a user-centric AI knowledge base. Fig.13 In the computing environment 1300 of FIG. 1 , a computerized personal assistant 1600 is communicatively coupled to a graph storage mechanism 1301 that implements a user-centric AI knowledge base. Thus, the computerized personal assistant can be configured to interact with the graph storage mechanism 1301 to provide user-centric facts to the user-centric AI knowledge base and to issue queries to be served by the user-centric AI knowledge base. The computerized personal assistant 1600 can be a separate computer service or an auxiliary component of another computer service (e.g., an email / calendar application, a search engine, an integrated development environment).

[0104] Computerized personal assistant 1600 includes a natural language user interface 1610 configured to receive user input and / or user queries. Natural language user interface 1610 may include a keyboard or any other text input device configured to receive user input in text form. Natural language user interface 1610 may include a microphone 1611 configured to capture voice audio. Therefore, user input and / or user queries received by natural language user interface 1610 may include voice audio captured by microphone. In some examples, natural language user interface 1610 is configured to receive user voice audio and output text representing the user voice audio. Alternatively or additionally, natural language user interface 1610 may include ink input device 1612, and the user input received by natural language user receiving interface 1610 may include user handwriting and / or user gestures captured by the ink input device. "Ink input device" may be used in this application to refer to any device or device combination that can allow a user to provide ink input. For example, an ink input device may include any device that allows a user to indicate a series of two-dimensional or three-dimensional positions relative to a display or any other surface, such as 1) a capacitive touch screen controlled by a user's finger; 2) a capacitive touch screen controlled by a stylus; 3) a "hover" device of a touch screen including a stylus and configured to detect the position of the stylus when the stylus approaches the touch screen; 4) a mouse; or 5) a video game controller. In some examples, the ink input device may alternatively or additionally include a camera configured to detect user gestures. For example, a camera may be configured to detect gestures based on three-dimensional movement of a user's hand. Alternatively or additionally, a camera (e.g., a depth camera) may be configured to detect the movement of the user's hand as a two-dimensional position relative to a surface or plane, such as relative to a plane defined by the front side of the camera's cone of view.

[0105] The computerized personal assistant 1600 also includes a natural language processing (NLP) mechanism 1620 configured to output a computer-readable representation of a user input and / or a user query received at the natural language user interface 1610. Thus, when the natural language user interface 1610 is configured to receive a user input, the NLP mechanism 1620 is configured to output a computer-readable representation of the user input; when the natural language user interface 1610 is configured to receive a user query, the NLP mechanism 1620 is configured to output a computer-readable representation of a query based on the user query.

[0106] When the NLP mechanism 1620 is configured to output a computer-readable representation of the query based on the user query, the NLP mechanism 1620 may also be configured to output the identified user intent based on the user query. Thus, the one or more constraints defined by the computer-readable representation of the query may include constraints based on the identified user intent.

[0107] The NLP mechanism 1620 can be configured to parse a user's utterances to identify intents and / or entities defined by the utterances. An utterance is any user input, e.g., a sentence or sentence fragment, which may or may not be very grammatical (e.g., with respect to grammar, word usage, pronunciation, and / or spelling). An intent represents an action that a user may wish to perform, which may include questions or tasks, e.g., making a reservation at a restaurant, calling a taxi, displaying a reminder at a later time, and / or answering a question. Entities may include specific named entities (e.g., the user's boss Alice) or placeholders representing specific types of entities, e.g., people, animals, coworkers, places, or organizations.

[0108] The NLP mechanism 1620 may be configured to use any suitable natural language processing technology. For example, the NLP mechanism 1620 may include a dependency parser and / or a structural parser configured to identify the grammatical structure of an expression. The NLP mechanism 1620 may additionally be configured to identify a key-value pair representing the semantic content of an expression, such as a pairing of the type of entity of an expression with the name of a specific entity of the type. In some examples, the components of the NLP mechanism 1620 (e.g., a dependency parser) may utilize one or more machine learning techniques. Non-limiting examples of these machine learning techniques may include feedforward networks, recurrent neural networks (RNNs), long short-term memories (LSTMs), convolutional neural networks, support vector machines (SVMs), generative adversarial networks (GANs), variational autoencoding, Q learning, and decision trees. The various identifiers, engines, and other processing blocks described in this application may be trained via supervised and / or unsupervised learning utilizing these, or any other appropriate machine learning techniques, so as to perform described evaluations, decisions, identifications, and the like. Then, it should be understood that this specification is not intended to propose new techniques for performing these evaluations, decisions, identifications, and the like. Rather, the present specification is directed to managing computing resources and, therefore, is intended to be compatible with any type of processing module, including processing modules that have not yet been developed.

[0109] In some examples, the NLP mechanism 1620 may be configured to recognize a set of predefined intents and / or entities. Alternatively or additionally, the NLP mechanism 1620 may be trained based on example expressions. In some examples, the NLP mechanism 1620 may be configured to recognize new intents and / or entities by providing several labeled examples to the NLP mechanism 162, wherein the labeled examples include new intents to be recognized together with exemplary expressions annotated to represent related entities. The NLP mechanism 1620 may be trained for the specific purpose of parsing expressions in the context of the computerized personal assistant 1600, so that the accuracy and / or performance of the NLP mechanism 1620 is optimized for the computerized personal assistant 1600. For example, when the NLP mechanism 1620 is based on one or more neural networks, training the NLP mechanism 1620 may include training the one or more neural networks via stochastic gradient descent using a back-propagation algorithm.

[0110] In some cases, NLP mechanism 1620 may be configured to recognize a specific human language, such as English. Alternatively or additionally, NLP mechanism 1620 may be configured to recognize multiple different human languages. When NLP mechanism 1620 is configured to recognize multiple different human languages, NLP mechanism 1620 may be able to process expressions including text in multiple different languages.

[0111] In some examples, the computerized personal assistant 1600 can be implemented as an all-in-one computing device contained within a single housing. For example, the all-in-one computing device can include one or more logic devices and one or more storage devices that store instructions that can be executed by the logic device to provide the functionality of the natural language user interface 1610, the NLP mechanism 1620, the identity mechanism 1630, the enrichment adapter 1640, the knowledge base update mechanism 1650, the knowledge base query mechanism 1660, and the output system 1670. In some examples, the all-in-one computing device can additionally include one or more input devices (e.g., a microphone 1611 and / or an ink input device 1612). In some examples, the all-in-one computing device can include one or more output devices (e.g., a display and / or a speaker included in the output subsystem 1670). Alternatively or additionally, the all-in-one computing device can include a communication subsystem configured to be communicatively coupled to other computer services (e.g., as part of the output subsystem 1670).

[0112] In some examples, one or more components of computerized personal assistant 1600 (e.g., natural language user interface 1610, NLP mechanism 1620, identity mechanism 1630, enrichment adapter 1640, knowledge base update mechanism 1650, knowledge base query mechanism 1660, and / or output subsystem 1670) can be configured to collaborate with one or more other computer services (e.g., other computing devices) to perform computations, transfer data, and implement the functionality of one or more components described above.

[0113] In one example, computerized personal assistant 1600 can be implemented across two or more different computing devices. For example, NLP mechanism 1620 can offload one or more natural language processing tasks to one or more cloud services, such as Fig.13 1300 of the computing environment 1300 shown in . Thus, the cloud service 1311 can be configured to process natural language as described above and return a computer-readable representation of the user input to the computerized personal assistant 1600 as an output at the NLP mechanism 1620. In some examples, the NLP mechanism 1620 can partially pre-process the user input before sending the pre-processed input for further processing by the cloud service 1311, and / or post-process the computer-readable representation of the pre-processed input received from the cloud service 1311 before outputting the post-processed computer-readable representation of the pre-processed input at the NLP mechanism 1620. For example, the remote computer service can be a cloud service that provides processing of natural language input, such as MICROSOFT LUIS TM Language understanding services.

[0114] Alternatively or in addition to NLP mechanism 1620 offloading functionality to cloud service 1311 in the manner described above, any other component of computerized personal assistant 1600 can be similarly configured to offload tasks to cloud service 1311 or to any other computing device (e.g., graph storage mechanism 1310, application-specific data provider 1321, and / or user computer 1340). In this manner, the functionality of components of computerized personal assistant 1600 can utilize hardware and / or software included in the computerized personal assistant 1600 in addition to utilizing other computing devices and / or computer services.

[0115] For example, the knowledge base update mechanism 1650 can be configured to offload tasks to the graph storage mechanism 1310, such as adding new user-centric facts to the user-centric knowledge base. Thus, in order to add new user-centric facts to the user-centric knowledge base, the knowledge base update mechanism 1650 can be configured to provide the user-centric facts to the graph storage mechanism 1301 via the network 1310, and receive a subset of user-centric facts selected in response to the query, including a subset of user-centric facts in the user-centric AI knowledge base implemented by the graph storage mechanism 1301.

[0116] Alternatively or additionally, the knowledge base query mechanism 1660 may be configured to offload tasks, such as issuing queries to be served by the user-centric knowledge base, to the graph storage mechanism 1310. Thus, to issue queries to be served by the user-centric knowledge base, the knowledge base query mechanism 1660 may be configured to provide user-centric facts to the graph storage mechanism 1301 via the network 1301, and receive a response based on a subset of the user-centric facts (e.g., a textual answer to the query based on the subset of the user-centric facts).

[0117] The computerized personal assistant 1600 also includes an identity mechanism 1630 configured to associate user input with a particular user. The identity mechanism 1630 may use any appropriate information to determine the identity of the user. For example, when the natural language user interface 1610 includes a microphone, and when the user input includes voice audio, the identity mechanism 1630 may include a speaker recognition engine configured to distinguish between users based on the voice audio. Alternatively or additionally, when the computerized personal assistant 1600 includes a camera (e.g., as part of the natural language user interface 1610), the identity mechanism 1630 may include a facial recognition engine configured to distinguish between users based on a photo of the user's face. Alternatively or additionally, the identity mechanism 1630 may be configured to receive biometric information (e.g., a user fingerprint) and distinguish between users based on the biometric information. Alternatively or additionally, the identity mechanism 1630 may be configured to prompt the user to provide an identity identifier (e.g., login information such as a username and password).

[0118] In an example where the user-centric AI knowledge base includes one or more encrypted user-centric facts, access to the one or more encrypted user-centric facts may be constrained by a certificate associated with a particular user. Thus, a computerized personal assistant associated with a particular user may need to provide a certificate to access (e.g., read and / or modify) the user-centric AI knowledge base. Thus, identity mechanism 1630 may be configured to identify the particular user and provide a certificate associated with the particular user. For example, identity mechanism 1630 may store the certificate, such as a digital certificate, so that the certificate is provided whenever computerized personal assistant 1600 provides user-centric facts to the user-centric AI knowledge base or issues a query to be served by the user-centric AI knowledge base.

[0119] Optionally, in some examples, the computerized personal assistant 1600 further includes an enrichment adapter 1640 configured to output an enrichment based on the computer-readable representation of the user input, wherein the new or updated user-centric facts comprise the enrichment output by the enrichment adapter. For example, the enrichment adapter 1640 may be configured to add the above description of the user-centric facts. Fig.14 Any enrichment of the description, e.g., 1) named entities, 2) intents, 3) events and tasks, 4) topics, 5) locations, and / or 6) dates and times. In some examples, enrichment adapter 1640 is an enrichment pipeline comprising a plurality of enrichment adapters each configured to output an enrichment, wherein the new or updated user-centric fact comprises all enrichments output by each enrichment adapter of the enrichment pipeline.

[0120] Optionally, in some examples, the computerized personal assistant 1600 also includes a knowledge base update mechanism 1650 configured to update the user-centric AI knowledge base associated with the particular user to include new or updated user-centric facts based on the computer-readable representation of the user input. The knowledge base update mechanism 1650 can update the user-centric AI knowledge base via an update protocol (e.g., an update API). The update protocol can be used by a plurality of different computer services. As described above with reference to Fig.14 As described, the update protocol may constrain the storage format of new or updated user-centric facts to an application-independent data format.

[0121] Optionally, in some examples, the computerized personal assistant 1600 further includes a knowledge base query mechanism 1660 configured to query a user-centric artificial intelligence knowledge base associated with the particular user, and output a response based on a subset of user-centric facts in the user-centric artificial intelligence knowledge base that satisfy one or more constraints defined by the computer-readable representation of the user query. The knowledge base query mechanism can be configured to query the user-centric AI knowledge base via a query protocol (e.g., a query API) usable by a plurality of different computer services.

[0122] In some examples, one or more constraints defined by the computer-readable representation of the query may include an answer type constraint, and thus, the subset of user-centric facts may include only user-centric facts that satisfy the answer type constraint. Alternatively or additionally, one or more constraints defined by the computer-readable representation of the query may include a graph context constraint, and thus, the subset of user-centric facts may include only user-centric facts related to contextualized user-centric facts in the user-centric AI knowledge base that satisfy the graph context constraint. Thus, reference may be made to Fig.15 Describes the subset of facts that determine this user.

[0123] In some examples, the query is a user context query for determining the current context of the user. Fig.15 As described, the subset of user-centric facts may include one or more user-centric facts related to the user's current context. When the query is a user-context query, the computer-readable representation of the query may be independent of any user query received at the natural language user interface 1610 and / or interpreted at the NLP mechanism 1620. Instead, the user-context query may be automatically issued based on any suitable data available to the computerized personal assistant (e.g., sensor data such as GPS data, time / date data, state data of the computerized personal assistant, and / or state data of any other collaborating computer service (e.g., a computer service configured to share data with the computerized personal assistant by providing user-centric facts to a user-centric AI knowledge base, and / or a computer service that can be controlled by the computerized personal assistant)).

[0124] Optionally, in some examples, the computerized personal assistant 1600 includes an output subsystem 1670 configured to output data (e.g., a response to a query output by a knowledge base query mechanism). For example, the output subsystem 1670 may include a speech synthesis engine configured to generate speech audio based on a subset of user-centric facts selected in response to the query, and a speaker configured to output the speech audio. In some examples, the output subsystem 1670 may include a text answer configured to visually present a subset of user-centric facts selected in response to the query. In some examples, the display may be configured to visually present the subset of user-centric facts directly in graphical form (e.g., displayed as a graphical depiction of a graph data structure including the subset of user-centric facts, or implemented as a timeline including user-centric facts arranged in chronological order of occurrence).

[0125] In some examples, the response output by the knowledge base query mechanism includes computer-readable instructions configured to cause a collaborative computer service to perform actions to assist a user based on a subset of user-centric facts in the user-centric AI knowledge base that satisfy one or more constraints defined by a computer-readable representation of the query. For example, the collaborative computer service can be a software application running on one or more devices that implements the computerized personal assistant. In other examples, the collaborative computer service can be a networked computer service accessible via a computer network. Thus, the output subsystem 1670 can include a communication device (e.g., a radio) configured to be communicatively coupled to the computer network so as to convey computer-readable instructions to the collaborative computer service.

[0126] In some examples, actions performed by the collaborative computer service to assist the user include changing preference settings of the collaborative computer service based on a subset of user-centric facts in the user-centric AI knowledge base that satisfy one or more constraints defined by the computer-readable representation of the query (e.g., when the query includes a request to change a particular preference setting).

[0127] In some examples, actions performed by the collaborative computer service to assist the user include visually presenting a depiction of state data of the collaborative service computer, wherein the state data of the collaborative computer service is related to user-centric facts in the user-centric artificial intelligence knowledge base that satisfy one or more constraints defined by the computer-readable representation of the query.

[0128] although Fig.16 A stand-alone computerized personal assistant is depicted, but any other computer service may include Fig.16Any subset of the components depicted in (e.g., natural language user interface, NLP mechanism, identity mechanism, enrichment adapter, knowledge base update mechanism, knowledge base query mechanism, and / or output subsystem). These components can assist in using a user-centric AI knowledge base as described with respect to a computerized personal assistant (e.g., by providing data to the user-centric AI knowledge base, and by using the user-centric AI knowledge base to serve queries). Any computer service that utilizes the user-centric AI knowledge base is a computerized personal assistant, regardless of whether such a service is a stand-alone service or an auxiliary component of another computer service with a different primary function (e.g., an email / calendar application, a search engine, an integrated development environment).

[0129] In one example, a user composes an email in an email program, using the subject line "Weekly Report on Widget Development," and selects the user's boss Alice as the recipient. The user may click an "Analyze" button to receive suggestions for completing the email. Thus, the email program may issue a contextual query to be served by a user-centric AI knowledge base. The contextual query may indicate the state of the email program. In addition, the contextual query may include one or more natural language features based on the state of the application (e.g., content in the subject line of the email). For example, one or more natural language features may include an identified intent (e.g., "find information"), an identified subject ("widget"), and an identified entity (e.g., the user's boss Alice). Thus, in addition to user-centric facts about "widgets," a subset of user-centric facts selected in response to the query may include user-centric facts related to the user's exchanges with her boss Alice.

[0130] Based on the subset of the user-centric facts selected in response to the query, the email program may display one or more suggestions based on the subset of the user-centric facts. For example, the email program may display one or more related files that the user may want to consult when preparing the email and / or attaching to the email, such as a "Widget Report" spreadsheet file and a "Widget Development Notes" document file. Alternatively or additionally, the email program may display one or more links from the user's web search history that may be about "Widgets" and / or more generally about "Widgets." Alternatively or additionally, the email program may display one or more related emails, such as an email from Alice saying "Please include estimated development costs for next month in this week's weekly report." Alternatively or additionally, the email program may display one or more email addresses of other users that may be related to the email, such as Alice's colleague Bob and resident widget expert Charlie. Alternatively or additionally, the email program may suggest that the user schedule a meeting with Alice. The email program may be configured to suggest a specific meeting time, for example, based on user-centric facts about the user's availability in the user-centric AI knowledge base, and additional user-centric facts about Alice's availability in the enterprise knowledge base.

[0131] Computer services configured to utilize the user-centric AI knowledge base may be implemented and / or organized in any suitable manner, and are not limited to the components and organization shown in FIG. 6 . Fig.17 An exemplary method 1700 for providing a computer service with data to a user-centric AI knowledge base is shown, and Fig.18 An exemplary method 1800 for a computer service for serving queries using a user-centric AI knowledge base is shown. Methods 1700 and 1800 may be implemented by any suitable computing device and / or computer service, such as a computer service that cooperates with any computer service compatible with the AI ​​knowledge base. Fig.16 Computerized Personal Assistant 1600.

[0132] At 1701, Fig.17 The method 1700 includes identifying a computer-readable representation of a user input associated with a particular user of a computer service.

[0133] Optionally, in some examples, at 1702, via a natural language user interface (e.g., Fig.16 The user input is received by a natural language user interface 1610 of a computerized personal assistant 1600. In some examples, an identity mechanism (e.g., Fig.16The identity mechanism 1630) identifies the specific user of the computer service.

[0134] Optionally, in some examples, at 1703, a computer-readable representation of the user input is output by the NLP machine based on the user input. For example, Fig.16 The computerized personal assistant 1600 shown in FIG. 1 may be configured to output a computer-readable representation of the user input via an NLP mechanism 1620 .

[0135] At 1704, method 1700 includes updating a user-centric AI knowledge base associated with the particular user to include new or updated user-centric facts based on the computer-readable representation of the user input. "New user-centric facts" is used in this application to refer to any user-centric facts that are not yet included in the user-centric AI knowledge base, such as user-centric facts that include subjects and / or objects that are not yet included in any other user-centric facts in the user-centric AI knowledge base, or user-centric facts that include new edges between subjects and objects in the user-centric AI knowledge base. "Updated user-centric facts" is used in this application to refer to modifications of user-centric facts already defined in the user-centric AI knowledge base, such as modifications to user-centric facts to include one or more new tags and / or enrichments. In some examples, updates to user-centric facts are made by a knowledge base update mechanism (such as Fig.16 The knowledge base update mechanism 1650) updates the user-centered AI knowledge base.

[0136] Optionally, in some examples, at 1705, the new or updated user-centric fact includes an enrichment based on the computer-readable representation of the user input. For example, Fig.16 The computerized graph personal assistant 1600 includes an enrichment adapter configured to output enrichment based on the computer-readable representation of the user input. In other examples, the computer service may not include any enrichment adapter, but the new or updated user-centric fact may still include enrichment, for example, the enrichment may be added to the new or updated user-centric fact by an enrichment adapter of a graph storage mechanism included in an embodiment of the user-centric AI knowledge base.

[0137] At 1706, the update protocol may be used by a plurality of different computer services (e.g., as described above with reference to Fig.14 At 1707, the update protocol can constrain the storage format of the new or updated user-centric facts to an application-independent data format. For example, Fig.16The computerized diagram personal assistant 1600 includes a knowledge base update mechanism 1650 configured to update the user-centric AI knowledge base via an update API.

[0138] Fig.18 An exemplary method 1800 for a computer service to serve queries using a user-centric AI knowledge base is shown.

[0139] At 1801, method 1800 includes identifying a computer-readable representation of a query associated with a particular user of the computer service. In some examples, the query may be identified by an identity mechanism (e.g., Fig.16 In some examples, the specific user can be identified based on login information associated with the computer service and / or based on the user's ownership of a computer device (e.g., a mobile phone) that executes the computer service.

[0140] Optionally, at 1802, the query is sent via a natural language user interface (e.g., Fig.16 Thus, at 1803, the user query may be received by an NLP mechanism such as Fig.16 The NLP mechanism 1620 of the embodiment outputs a computer-readable representation of the query based on the user query. In some examples, the query includes constraints based on natural language features of the user query, such as based on intent, subject, and / or entity identified from the user query. For example, if a user asks "Who is an expert in widgets", the query may include a graph context constraint indicating that the answer should be about "widgets", and an answer type constraint based on identifying that the user's intent is to find a specific person, including a user-centric fact indicating that the answer should include a subject and / or object that is a person.

[0141] In some examples, the query may be a user context query for determining a current context of a user, and thus, the subset of user-centric facts may include one or more user-centric facts related to the current context. In some examples, the query may indicate a state of a computer service, and thus, the one or more user-centric facts related to the current context may include user-centric facts related to the state of the computer service. In some examples, method 1800 also includes identifying a computer-readable representation of a natural language feature defined by the state of the computer service, and thus, the one or more constraints include constraints based on the natural language feature. In some examples, identifying the computer-readable representation of the natural language feature may be performed by an NLP mechanism such as Fig.16 NLP mechanism 1620) is executed.

[0142] At 1804, method 1800 includes requesting a query via a query protocol usable by a plurality of different computer services (e.g., as described above with reference to Fig.15 In some examples, querying the user-centric AI knowledge base can be performed by a knowledge base query mechanism (such as a query API described in the description). Fig.16 The knowledge base query mechanism 1660) is used to execute.

[0143] At 1805, the user-centric AI knowledge base may be an update protocol that can be used via a plurality of different computer services (e.g., as described above with reference to Fig.14 The update protocol may be updated to include new or updated user-centric facts. At 1806, the update protocol may constrain the storage format of the new or updated user-centric facts to an application-independent data format.

[0144] At 1807 , method 1800 includes outputting a response based on a subset of user-centric facts in the user-centric AI knowledge base that satisfy one or more constraints defined by the computer-readable representation of the query.

[0145] At 1808, method 1800 optionally includes causing the computer service or a collaborating computer service to perform an action to assist the user based on a subset of user-centric facts in the user-centric AI knowledge base that satisfy one or more constraints defined by the computer-readable representation of the query. Causing the computer service to perform an action to assist the user may include outputting computer-readable instructions configured to cause the computer service or a collaborating computer service to perform the action to assist the user.

[0146] In some examples, a response based on a subset of the user-centric facts in the user-centric AI knowledge base is generated by an output subsystem (e.g., Fig.16 For example, the output subsystem 1670 may include a communication subsystem that is communicatively coupled to a collaborative computer device that implements the collaborative computer service via a computer network. Thus, outputting computer-readable instructions may include sending computer-readable instructions to the collaborative computer device via a computer network. In some examples, the output subsystem 1670 may include a display device, and the computer-readable instructions may be configured to cause the display device to visually present a depiction of state data of the computer service, the state data being related to a subset of user-centric facts in the user-centric AI knowledge base that satisfy one or more constraints defined by the query. Alternatively or in addition, the output subsystem 1670 may include a speaker, and the computer-readable instructions may be configured to generate speech audio that describes the state data of the computer service, and cause the speaker to output the speech audio.

[0147] In some examples, a computer service may automatically provide facts to a user-centric AI knowledge base (e.g., in accordance with method 1700 or in any other appropriate manner using an update protocol for the user-centric AI knowledge base). For example, a computer service may continuously monitor sensor and / or state data of the computer service to provide facts about the context of a user of the computer service, such as GPS data indicating the location of the user and clock data indicating the current time. Alternatively or in addition, in some examples, the computer service may automatically issue queries to be serviced by the user-centric AI knowledge base (e.g., in accordance with method 1800 or in any other appropriate manner using a query protocol for the user-centric AI knowledge base). For example, the computer service may repeatedly issue user context queries according to a schedule in order to monitor the context of the user (e.g., in order to automatically perform actions related to the context).

[0148] The computerized personal assistant 1600 can provide a wide range of assistance to the user (e.g., by providing information to the user, or by automatically performing tasks for the user). In one example, the computerized personal assistant 1600 can be configured to collaborate with multiple other computer services, including an email program and a sensor monitoring program configured to output global positioning system (GPS) data indicating the location of a user device (e.g., a mobile phone) belonging to the user 1690. Each of the computerized personal assistant 1600, the email program, and the sensor monitoring program provides one or more user-centric facts to the user-centric AI knowledge base, for example, according to the method 1700. For example, the email program can provide a new user-centric fact to the user-centric AI knowledge base each time the user sends an email. The new user-centric fact can include an object graph node indicating the identity of the user 1690, a subject graph node indicating the recipient of the email, and an edge indicating a relationship of "sending an email". The new user-centric fact can include one or more enrichments, for example, an enrichment for indicating the subject of the identified email and a timestamp for indicating the time when the email was sent. The sensor monitoring program may also provide one or more new user-centric facts to the user-centric AI knowledge base, for example by continuously monitoring the location of the user's mobile phone and providing a new user-centric fact indicating the location of the user's mobile phone and a corresponding timestamp each time the location of the user's mobile phone changes. Thus, the user-centric AI knowledge base may contain multiple user-centric facts indicating the location of user 1690's mobile phone and multiple user-centric facts indicating each email sent by user 1690.

[0149] In addition, the user-centric AI knowledge base can include additional user-centric facts (e.g., added by graph storage computer 1301) based on the enrichment of new user-centric facts added by the email program and new user-centric facts added by the sensor monitoring program. For example, the user-centric AI knowledge base can include additional facts indicating that an email was likely sent from a particular location based on comparing the timestamp associated with the email with the timestamp associated with the facts provided by the sensor monitoring program.

[0150] The computerized personal assistant 1600 may later issue a query to be serviced by the user-centric AI knowledge base (e.g., via method 1800) to determine the location of the workplace of user 1690. For example, the query may include a graph context constraint indicating "work-related emails" and an answer type constraint indicating "location." Thus, the subset of user-centric facts selected in response to the query may include locations from which user 1690 may have sent one or more work-related emails. Based on the subset of user-centric facts, the computerized personal assistant 1600 may identify that a significant portion of work-related emails were sent from a particular location within a particular time range. Thus, the computerized personal assistant 1600 can identify the location of user 1690's workplace and user 1690's work schedule.

[0151] Subsequently, in a similar manner, the user-centric AI knowledge base may contain a user-centric fact identifying that user 1690 frequently sets an alarm from a particular location at night (e.g., on her mobile phone) and each time mutes the alarm the next morning. Accordingly, computerized personal assistant 1600 may issue a query regarding the location and time of the alarm. Based on the response to the query, computerized personal assistant 1600 may identify user 1690's home address and user 1690's sleep schedule.

[0152] The computerized personal assistant 1600 may be able to assist the user with a variety of different tasks based on identifying one or more aspects of user-centric facts in the user-centric AI knowledge base (e.g., by issuing queries according to the method 1700). For example, when the computerized personal assistant 1600 identifies the work and sleep schedule of the user 1690, the computerized personal assistant 1600 may be able to set the user's alarm by default according to their typical scheduling preferences.

[0153] In some examples, such as Fig.16As depicted in , the computerized personal assistant 1600 may be able to provide an enhanced response to a user query based on one or more aspects of the user-centric facts. For example, the user 1690 may ask the computerized personal assistant 1600, as shown in the speech bubble 1691, "Are there any good restaurants near work?". Accordingly, the computerized personal assistant 1600 may recognize a computer-readable representation of the query (e.g., according to the method 1800 at 1801). The computerized personal assistant 1600 may issue a query to be served by a user-centric AI knowledge base (e.g., as described at 1804, via a query protocol that can be used by multiple different computer services). The query may include a graph context constraint indicating "near work" and an answer type constraint indicating "restaurant". Therefore, the computerized personal assistant 1600 may output a response to the query (e.g., as described at 1807) based on a subset of user-centric facts that satisfy the graph context constraint and the answer type constraint. For example, the computerized personal assistant 1600 may suggest one or more restaurants within a convenient distance of the user 1690's workplace.

[0154] In some examples, the computerized personal assistant 1600 may request more information from the user 1690 in order to make a selection based on the preferences of the user 1960. For example, when the user 1690 asks to find a good restaurant near work, the subset of user-centric facts selected in response to the query may indicate a number of different restaurants within a similar distance. Thus, the computerized personal assistant 1600 may ask a follow-up question indicating a specific restaurant, such as "How about a 'burrito' restaurant?" as shown in the speech bubble 1692.

[0155] In some examples, the computerized personal assistant 1600 can provide one or more additional user-centric facts to the user-centric AI knowledge base in answering the user query (e.g., according to methods 1700 and 1800). For example, the user 1690 can respond to the question "How is the 'Burrito' restaurant?" by saying "I don't like burritos" as shown in the speech bubble 1693. Thus, the computerized personal assistant 1600 can add a new fact to the user-centric AI knowledge base indicating that the user does not like burritos.

[0156] To determine restaurant options that meet the user's preferences, the computerized personal assistant 1600 may ask additional follow-up questions, such as, "How about 'Sushi Restaurant'?" as shown in speech bubble 1694. If the user 1690 responds, "OK, make a reservation after work," as shown in speech bubble 1695, the computerized personal assistant 1600 may infer when to schedule the reservation based on identifying the user's 1690 work schedule.

[0157] In addition to considering the work schedule of user 1690, the user-centric AI knowledge base can enable the computerized personal assistant 1600 to consider other potentially relevant factors, such as factors associated with one or more user-centric facts included in the user-centric AI knowledge base. For example, the user-centric AI knowledge base may include user-centric facts describing what time user 1690 typically prefers to eat. In some examples, the user-centric AI knowledge base may consider factors determined based on additional facts outside the user-centric knowledge base, such as a predicted duration of a journey from user 1690's workplace to a "sushi restaurant" as determined based on GPS data associated with the user's location and an identified work schedule. For example, the computerized personal assistant 1600 may be configured to determine the predicted journey duration via an API of a map service that provides geolocation and traffic planning capabilities.

[0158] Based on when user 1690 typically leaves work, when user 1690 typically prefers to eat, and the predicted duration of the journey from user 1690's work to the "sushi restaurant," the computerized personal assistant 1600 can determine a suitable time for the reservation, such as 6PM. Thus, in response to a series of user queries and user inputs, the computerized personal assistant can output a response as in speech bubble 1696 to confirm that a reservation has been made (e.g., according to method 1800 at 1807). In addition, the computerized personal assistant 1600 can output computer-readable instructions configured to cause the computerized personal assistant 1600 and / or other cooperating computer services to perform actions to assist the user (e.g., according to method 1800 at 1808). For example, the computerized personal assistant 1600 can output computer-readable instructions configured to arrange a reservation (e.g., via a restaurant reservation service that provides an API for making reservations). In addition, the computerized personal assistant 1600 can identify one or more new user-centric facts that can be added to the user-centric AI knowledge base, namely that user 1690 does like sushi.

[0159] Based on the interaction and assistance with the user 1690, the computerized personal assistant 1600 and other computer services can continuously add new user-centric facts to the user-centric AI knowledge base and issue queries to make intelligent decisions based on the user-centric facts in the user-centric AI knowledge base. Therefore, the user-centric AI knowledge base can enable the computerized personal assistant 1600 and other computer services to continuously improve and provide assistance to the user 1690 according to her preferences.

[0160] In some examples, the computerized personal assistant 1600 may automatically issue a series of queries (e.g., according to a schedule) to automatically perform actions to assist the user 1690 based on a subset of user-centric facts selected in response to each query. For example, when the user 1690 has a restaurant reservation at 'Sushi Restaurant' at 6:00 PM, the computerized personal assistant 1600 may be configured to repeatedly issue user context queries at 2 minute intervals between 5:30 PM and 6:00 PM. The user context query may include constraints related to the current time, the current activity of the user 1690, and / or the location of the user based on GPS data, and the subset of user-centric facts selected in response to the user context query may include one or more user-centric facts related to the restaurant reservation, such as based on similarity of timestamp and location information. Later, the user 1690 may leave work at 5:50 PM and travel to 'Sushi Restaurant'. Thus, computerized personal assistant 1600 can recognize that user 1690 is approaching the restaurant and automatically assist user 1690 by visually presenting relevant information (e.g., confirmation of the reservation, the restaurant menu, and a map showing directions to the destination).

[0161] In some embodiments, the methods and processes described in this application may rely on the computing system of one or more computing devices. Specifically, these methods and processes can be implemented as computer applications or services, application programming interfaces (APIs), libraries, and / or other computer program products.

[0162] Fig.19 A non-limiting embodiment of a computing system 1900 capable of specifying one or more of the above methods and processes is schematically illustrated. For example, the computing system 1900 can be used as a graph storage mechanism 1301, an application-specific data provider computer 1321, or a user computer 1340. In some examples, the computing system 1900 can provide the functionality of a computerized personal assistant 1600 or any other computer service configured to use a user-centric AI knowledge base (e.g., according to method 1700 or method 1800, or in any other suitable manner using an update protocol and / or query protocol of the user-centric AI knowledge base). The computing system 1900 is shown in simplified form. The computing system 1900 can take the form of one or more personal computers, server computers, tablet computers, home entertainment computers, network computing devices, gaming devices, mobile computing devices, mobile communication devices (e.g., smartphones), and / or other computing devices.

[0163] The computing system 1900 includes a logic mechanism 1901 and a storage mechanism 1902. The computing system 1900 may optionally include a display subsystem 1903, an input subsystem 1904, a communication subsystem 1905, and / or Fig.19 Other components not shown.

[0164] The logic mechanism 1901 includes one or more physical devices configured to execute instructions. For example, the logic mechanism can be configured to execute instructions that are part of one or more applications, services, programs, routines, libraries, objects, components, data structures, or other logical constructs. These instructions can be implemented to perform tasks, implement data types, transform the state of one or more components, achieve technical effects, or otherwise achieve a desired structure.

[0165] The logic mechanism may include one or more processors configured to execute software instructions. Additionally or alternatively, the logic mechanism may include one or more hardware or firmware logic mechanisms configured to execute hardware or firmware instructions. The processor of the logic mechanism may be single-core or multi-core, and the instructions executed thereon may be configured for serial, parallel and / or distributed processing. The individual components of the logic mechanism may optionally be distributed between two or more separate devices, which may be located in a remote location and / or configured for coordinated processing. Various aspects of the logic mechanism may be virtualized and executed by a networked computing device configured in a remotely accessible, cloud computing configuration.

[0166] Storage mechanism 1902 includes one or more physical devices configured to hold executable instructions that can be executed by the logic mechanism to implement the methods and processes described in this application. When implementing these methods and processes, the state of storage mechanism 1902 can be transformed—for example, to hold different data.

[0167] The storage mechanism 1902 may include removable and / or built-in devices. The storage mechanism 1902 may include optical storage (e.g., CD, DVD, HD-DVD, Blu-ray disc, etc.), semiconductor storage (e.g., RAM, EPROM, EEPROM, etc.), and / or magnetic storage (e.g., hard disk drive, floppy disk drive, tape drive, MRAM, etc.), etc. The storage mechanism 1902 may include volatile, non-volatile, dynamic, static, read / write, read-only, random access, sequential access, location addressable, file addressable, and / or content addressable devices.

[0168] Although storage mechanism 1902 includes one or more physical devices, aspects of the instructions described herein may alternatively be propagated via a communication medium (eg, electromagnetic signals, optical signals, etc.) that is not retained by a physical device for a finite duration.

[0169] Aspects of the logic mechanism 1901 and the storage mechanism 1902 may be integrated together into one or more hardware logic components. These hardware logic components may include, for example, field programmable gate arrays (FPGAs), program and application specific integrated circuits (PASIC / ASIC), program and application specific standard products (PSSP / ASSP), systems on chips (SOCs), and complex programmable logic devices (CPLDs).

[0170] The terms "module", "program" and "engine" can be used to describe several aspects of a computing system 1900 implemented to perform a specific function. In some cases, a module, program or engine may be instantiated via a logic mechanism 1901 that executes instructions held by a storage mechanism 1902. The term "machine" can be used to describe one or more logical machines that instantiate these modules, programs or engines. For example, the natural language processing mechanism described in this application can take the form of an ASIC or a general-purpose processor that runs software, firmware or hardware instructions that translate the original user input (e.g., voice audio detected by a microphone) into a computer-readable representation of the input that is more suitable for downstream processing. It should be understood that different modules, programs and / or engines can be instantiated on the same machine and / or from the same application, service, code block, object, library, routine, API, function, etc. Similarly, the same module, program and / or engine can span two or more different machines and / or be instantiated by different applications, services, code blocks, objects, routines, APIs, functions, etc. The terms "module," "program," and "engine" may cover individual or grouped executable files, data files, libraries, drivers, scripts, database records, and the like.

[0171] When included, the display subsystem 1903 can be used to present a visual representation of the data maintained by the storage mechanism 1902. This visual representation can take the form of a graphical user interface (GUI). As the methods and processes described in this application change the data maintained by the storage mechanism, and thus transform the state of the storage mechanism, the state of the display subsystem 1903 can also be transformed to visually represent the changes in the underlying data. The display subsystem 1903 can include one or more display devices using virtually any type of technology. These display devices can be combined with the logic mechanism 1901 and / or the storage mechanism 1902 in a shared package, or these display devices can be peripheral display devices.

[0172] When included, the input subsystem 1904 may include or interface with one or more user input devices, such as a keyboard, mouse, touch screen, or game controller. In some embodiments, the input subsystem may include or interface with selected natural user input (NUI) component portions. Such component portions may be integrated or peripheral devices, and the conversion and / or processing of input actions may be handled on-board or off-board. Example NUI component portions may include microphones for voice and / or sound recognition; infrared, color, stereo and / or depth cameras for machine vision and / or gesture recognition; head trackers, eye trackers, accelerometers and / or gyroscopes for motion detection and / or intent recognition; and electric field sensing component portions for assessing brain activity.

[0173] When included, the communication subsystem 1905 can be configured to communicatively couple the computing system 1900 to one or more other computing devices. The communication subsystem 1905 can include wired and / or wireless communication devices compatible with one or more different communication protocols. As non-limiting examples, the communication subsystem can be configured to communicate via a wireless telephone network, or a wired or wireless local area network or wide area network. In some embodiments, the communication subsystem can allow the computing system 1900 to send and / or receive messages to and / or from other devices via a network such as the Internet.

[0174] In one example, a computerized personal assistant includes: a natural language user interface configured to receive user input; a natural language processing mechanism configured to output a computer-readable representation of the user input; an identity mechanism configured to associate the user input with a specific user; a knowledge base update mechanism configured to update a user-centric artificial intelligence knowledge base associated with the specific user to include new or updated user-centric facts based on the computer-readable representation of the user input, wherein the knowledge base update mechanism updates the user-centric artificial intelligence knowledge base via an update protocol usable by multiple different computer services, the update protocol constraining the storage format of the new or updated user-centric facts to an application-independent data format. In this example or any other example, the computerized personal assistant further includes an enrichment adapter configured to output an enrichment based on the computer-readable representation of the user input, wherein the new or updated user-centric facts include the enrichment output by the enrichment adapter. In this example or any other example, the user-centric artificial intelligence knowledge base includes one or more encrypted user-centric facts, wherein access to the one or more encrypted user-centric facts is subject to a certificate associated with the specific user. In this example or any other example, the computerized personal assistant also includes a knowledge base query mechanism configured to query a user-centric artificial intelligence knowledge base associated with a specific user and output a response based on a subset of user-centric facts in the user-centric artificial intelligence knowledge base that satisfy one or more constraints defined by a computer-readable representation of the query. In this example or any other example, the knowledge base query mechanism is configured to query the user-centric artificial intelligence knowledge base via a query protocol that can be used by multiple different computer services. In this example or any other example, the natural language user interface is further configured to receive a user query; and the natural language processing mechanism is further configured to output a computer-readable representation of the query based on the user query. In this example or any other example, the natural language processing mechanism is further configured to output a user intent identified based on the user query, wherein one or more constraints defined by the computer-readable representation of the query include constraints based on the identified user intent. In this example or any other example, the query is a user context query for determining the current context of the user, and the subset of the user-centric facts includes one or more user-centric facts related to the current context. In this or any other example, the one or more constraints defined by the computer-readable representation of the query include an answer-type constraint; and the subset of user-centric facts includes only user-centric facts that satisfy the answer-type constraint.In this example or any other example, one or more constraints defined by the computer-readable representation of the query include graph context constraints; and the subset of user-centric facts includes only user-centric facts related to contextualized user-centric facts in the user-centric artificial intelligence knowledge base that satisfy the graph context constraints. In this example or any other example, the response output by the knowledge base query mechanism includes computer-readable instructions configured to cause the collaborative computer service to perform actions to assist the user based on a subset of user-centric facts in the user-centric artificial intelligence knowledge base that satisfy one or more constraints defined by the computer-readable representation of the query. In this example or any other example, the action for assisting the user includes changing the preference settings of the collaborative computer service based on a subset of user-centric facts in the user-centric artificial intelligence knowledge base that satisfy one or more constraints defined by the computer-readable representation of the query. In this example or any other example, the action for assisting the user includes visually presenting a depiction of state data of the collaborative computer service, which is related to a subset of user-centric facts in the user-centric artificial intelligence knowledge base that satisfy one or more constraints defined by the computer-readable representation of the query.

[0175] In one example, a computerized personal assistant includes: a natural language user interface configured to receive a user query; an identity mechanism configured to associate the user query with a specific user; a natural language processing mechanism configured to output a computer-readable representation of the user query; and a knowledge base query mechanism configured to query a user-centric artificial intelligence knowledge base associated with the specific user and output a response to the query based on a subset of user-centric facts in the user-centric artificial intelligence knowledge base that satisfy one or more constraints defined by the computer-readable representation of the query, wherein the user-centric artificial intelligence knowledge base can be updated to include new or updated user-centric facts via an update protocol that can be used by multiple different computer services, and the update protocol constrains the storage format of the new or updated user-centric facts to an application-independent data format.

[0176] In one example, a method for automatically responding to queries includes: identifying a computer-readable representation of a query associated with a particular user of a computer service; querying a user-centric artificial intelligence knowledge base associated with the particular user via a query protocol usable by a plurality of different computer services; and outputting a response based on a subset of user-centric facts in the user-centric artificial intelligence knowledge base that satisfy one or more constraints defined by the computer-readable representation of the query, wherein the user-centric artificial intelligence knowledge base can be updated to include new or updated user-centric facts via an update protocol usable by a plurality of different computer services, the update protocol constraining the storage format of the new or updated user-centric facts to an application-independent data format. In this example or any other example, the computer service is a computerized personal assistant. In this example or any other example, the query is a user context query for determining the current context of the user, wherein the subset of the user-centric facts includes one or more user-centric facts related to the current context. In this example or any other example, the query indicates the state of the computer service, and the one or more user-centric facts related to the current context include user-centric facts related to the state of the computer service. In this example or any other example, the method further includes identifying a computer-readable representation of a natural language feature defined by the state of the computer service, wherein the one or more constraints include constraints based on the natural language feature. In this example or any other example, the method further includes causing the computer service to perform an action to assist the user based on a subset of user-centric facts in the user-centric artificial intelligence knowledge base that satisfy the one or more constraints defined by the computer-readable representation of the query.

[0177] It should be understood that the configuration and / or method described in the present application are exemplary in nature, and these specific embodiments or examples should not be considered in a restrictive sense, because many variations are possible. Specific routines or methods described in the present application can represent one or more of any number of processing strategies. Equally, the various actions shown and / or described can be performed or omitted according to the sequence shown and / or described, with other sequences or in parallel. Equally, the order of the process described above can be changed.

[0178] The subject matter of the present disclosure includes all novel and nonobvious combinations and subcombinations of the various processes, systems and configurations, and other features, functions, acts, and / or properties disclosed in this application, and any and all equivalents thereof.

Claims

1. A computing device, comprising: a natural language user interface configured to receive user input; a natural language processing mechanism configured to output a computer-readable representation of said user input; an identity mechanism configured to associate the user input with a particular user; as well as a knowledge base update mechanism configured to update a user-centric artificial intelligence knowledge base associated with the particular user to include new or updated user-centric facts based on the computer-readable representation of the user input, wherein the knowledge base update mechanism updates the user-centric artificial intelligence knowledge base via an update protocol usable by a plurality of different computer services, the update protocol constraining a storage format of the new or updated user-centric facts to an application-independent data format, Wherein, the application-independent data format is associated with a node record format, and the node record format supports the following aspect pointers: auxiliary application-specific data associated with application-specific facts of the application-specific context of the user's interaction with the plurality of different computer services.

2. The computing device of claim 1 , further comprising an enrichment adapter configured to output an enrichment based on the computer-readable representation of the user input, and wherein, The new or updated user-centric facts include the enrichment output by the enrichment adapter.

3. The computing device of claim 1, wherein: The user-centric artificial intelligence knowledge base includes one or more encrypted user-centric facts, wherein access to the one or more encrypted user-centric facts is constrained by credentials associated with the particular user.

4. The computing device according to claim 1, further comprising a knowledge base query mechanism configured to: The user-centric artificial intelligence knowledge base associated with the particular user is queried and a response is output based on a subset of user-centric facts in the user-centric artificial intelligence knowledge base satisfying one or more constraints defined by a computer-readable representation of the query.

5. The computing device of claim 4, wherein: The knowledge base query mechanism is configured to query the user-centric artificial intelligence knowledge base via a query protocol usable by a plurality of different computer services.

6. The computing device of claim 4, wherein: The natural language user interface is further configured to receive a user query; and The natural language processing mechanism is further configured to output the computer-readable representation of the query based on the user query.

7. The computing device of claim 6, wherein: The natural language processing mechanism is further configured to output an identified user intent based on the user query, and wherein the one or more constraints defined by the computer-readable representation of the query include constraints based on the identified user intent.

8. The computing device of claim 4, wherein: The query is a user context query for determining a current context of the user, and wherein the subset of user-centric facts includes one or more user-centric facts related to the current context.

9. The computing device of claim 4, wherein: The one or more constraints defined by the computer-readable representation of the query include answer-type constraints; and The subset of user-centric facts includes only user-centric facts that satisfy the answer type constraint.

10. The computing device of claim 4, wherein: The one or more constraints defined by the computer-readable representation of the query include graph context constraints; and The subset of user-centric facts includes only user-centric facts related to contextualized user-centric facts in the user-centric artificial intelligence knowledge base that satisfy the graph context constraints.

11. The computing device of claim 4, wherein: The response output by the knowledge base query mechanism includes computer-readable instructions configured to: Causing a collaborative computer service to perform an action to assist the user based on a subset of the user-centric facts in the user-centric artificial intelligence knowledge base satisfying the one or more constraints defined by the computer-readable representation of the query.

12. The computing device of claim 11, wherein: The actions for assisting the user include: The preference setting of the collaborative computer service is changed based on a subset of the user-centric facts in the user-centric artificial intelligence knowledge base satisfying the one or more constraints defined by the computer-readable representation of the query.

13. The computing device of claim 11, wherein: The actions for assisting the user include: A depiction of state data of the collaborative computer service is visually presented, the state data relating to a subset of the user-centric facts in the user-centric artificial intelligence knowledge base that satisfy the one or more constraints defined by the computer-readable representation of the query.

Images