Evolving event-specific temporary knowledge graphs

Abstract

Techniques and frameworks for building and / or updating an "event-specific temporary knowledge graph", for example, on top of a general knowledge graph, are described herein. In various embodiments, a live data stream may be analyzed to identify entities associated with a developing event. The entity may form part of a general knowledge graph comprising entity nodes and edges between entity nodes. Based on the identified one or more entities, an event-specific temporary knowledge graph may be built or updated in association with the developing event. In some embodiments, an event-specific temporary knowledge graph may be queried for new information about the developing event. The computing device may be caused to present the new information as output.

Description

Background Art

[0001] With the proliferation of Internet-enabled devices such as smart phones and smart watches, it is increasingly common for information about new developing or "live" events to be disseminated by eyewitnesses using live data streams such as social media posts, search engine queries, digital image posts, etc. This often occurs even before such information is published by traditional news media. A data structure called a "knowledge graph" can be used, for example, by search engines to manage knowledge about entities and the relationships between entities. An entity can be a person, a place, or a thing (e.g., an event). Some knowledge graphs may include nodes representing entities and edges representing relationships between entities. However, for developing events such as sporting events, disasters, protests, etc., the knowledge graph may not be updated in real time to reflect current information. Because there may be a large number of live data streams conveying information about developing events, duplicate and / or contradictory data may be received from multiple sources. Summary of the invention

[0002] Described herein are techniques and frameworks for collecting information about developing events from multiple live data streams and pushing multiple new pieces of information about developing events to interested individuals or "subscribers". More particularly, described herein are techniques for building and / or updating an "event-specific temporary knowledge graph", for example, on top of a general knowledge graph. Similar to a general knowledge graph, the event-specific temporary knowledge graph can organize information posted by individuals into entity nodes and edges. However, the event-specific temporary knowledge graph, as its name implies, can be "temporary" or "transient" in nature, because the entities and / or relationships represented by at least some of its nodes may not yet be considered sufficiently confirmed or verified for inclusion in the general knowledge graph. Therefore, those unconfirmed and / or unverified entities and / or relationships can be represented by entity nodes and / or edges of the event-specific temporary knowledge graph, which do not exist in the general knowledge graph at least for a certain period of time after the start of the developing event. In other words, the general knowledge graph may lag behind the event-specific temporary knowledge graph in having information about the developing event.

[0003] The event-specific temporary knowledge graph provides functional data that can be utilized to collate, deduplicate, and propagate information related to existing data associated with the general knowledge graph. The event-specific temporary knowledge graph combined with the general knowledge graph enables a computing device to automatically identify and output event-specific data associated with an entity. Providing a different knowledge graph associated with an entity can, for example, allow a computing device to automatically present event-specific information associated with the entity in an appropriate format.

[0004] Information can be disseminated, for example, to users who have expressed interest, information about developing and / or live events such as disasters, sporting events, protests, rallies (spontaneous or otherwise) posted by individuals. For example, as newsworthy events unfold, live event streams such as social media posts from witnesses or other people claiming to know about the event can be analyzed to identify one or more entities associated with the event. Although the event itself may eventually be represented as an entity node in the general knowledge graph, there may be a certain delay before the general knowledge graph is updated. Before such an update, the developing event can be temporarily represented in the event-specific temporary knowledge graph, for example, at least by one or more entity nodes or edges between entity nodes. The data structure provided allows information related to the event to be provided before the information is updated. In addition or alternatively, it can be expected that event-specific information is temporary and the data structure provided can provide a temporary structure that can be used to output information without modifying existing stored data. In some embodiments, the event-specific temporary knowledge graph can be available to all individuals, or at least to multiple individuals. It is not necessarily associated with any particular individual.

[0005] In some embodiments, multiple live data streams may be crawled to identify a subset of the live data streams that are relevant to a developing event. Various data processing pipelines may then be used to process the content of each of the subsets of the live data streams to populate an event-specific temporary knowledge graph, and ultimately, to propagate new information about the developing event to interested persons (who may be referred to as "event subscribers"). For example, one or more of these data processing pipelines may be used to parse and / or natural language process (e.g., to identify topics and / or entities) the live data stream, and merge and / or de-duplicate data from multiple data streams for incorporation into an event-specific temporary knowledge graph. Once updated, the event-specific temporary knowledge graph may be queried, for example, periodically or continuously, to determine new information about the event that is propagated to event subscribers.

[0006] A developing event may be initially identified in a variety of ways. In some embodiments, one or more individuals may submit queries to a search engine seeking information about a developing event, such as "Is there a fire at the cathedral?" or "Why is a crowd gathering at Times Square?" If enough queries, e.g., a cluster of semantically related queries, demonstrate a new developing event, it may trigger analysis of multiple live data streams, such as social media posts, to obtain information to populate an event-specific knowledge graph. As another example, multiple live streams, such as social media posts, may be analyzed on an ongoing basis. Similar to the clustering of semantically related queries, if enough numbers or clusters of semantically related posts are identified, that may demonstrate a new developing event. In other embodiments, a specific live data stream, such as from a trusted individual / organization such as a first responder, may be monitored for new developing events.

[0007] Specific individuals may be selected to receive updates about a developing event from an event-specific temporary knowledge graph in a variety of ways—i.e., to “subscribe” to a developing event. In some embodiments, an individual may be one of the individuals who issued an early query related to the developing event, such as those previously described (“Is there a fire at the cathedral?” or “Why is a crowd gathering at Times Square?”). As more information about the developing event becomes known and the event-specific temporary knowledge graph is updated, individuals may receive updates with information that has not yet been presented to them. More generally, in some embodiments, an individual who issues a query about a developing event, regardless of whether the individual's query contributes to the initial detection of the developing event, effectively “subscribes” the individual to receive updates from the event-specific temporary knowledge graph as they become available. In other cases, individuals may subscribe to automatically receive updates about a developing event, for example based on alignment of those individuals' interests with the event “type” of the developing event.

[0008] In some embodiments, a method is provided, the method comprising: analyzing two or more live data streams; based on the analysis, identifying one or more entities associated with a developing event, wherein the one or more entities form part of a general knowledge graph including multiple entity nodes and multiple edges between the multiple entity nodes, wherein the multiple entity nodes represent entities and the multiple edges represent relationships between the entities; based on the identified one or more entities, building or updating an event-specific temporary knowledge graph associated with the developing event, wherein the event-specific temporary knowledge graph shares one or more entity nodes with the general knowledge graph, and wherein the event-specific temporary knowledge graph includes one or more additional nodes or edges that are not found in the general knowledge graph, which convey the relationship between the identified one or more entities and the developing event; and in response to the building or updating, querying the event-specific temporary knowledge graph for new information about the developing event; and causing one or more computing devices to render the new information as output.

[0009] These and other implementations of the technology disclosed herein can optionally include one or more of the following features.

[0010] In various embodiments, the method further includes: receiving a user query seeking information related to the developing event; and analyzing a superset of the live data streams based on the user query to identify the set of two or more live data streams as being related to the developing event. In various embodiments, the method further includes: receiving a user query seeking information related to the developing event; wherein the query includes querying an event-specific temporary knowledge graph based on the user query. In various embodiments, the method further includes determining that the general knowledge graph does not include information responsive to the user query. In various embodiments, the analyzing is responsive to the determination.

[0011] In various embodiments, the two or more live data streams include one or more posts from witnesses of the developing event. In various embodiments, the method further includes detecting the developing event based on the analysis. In various embodiments, the method further includes determining an event type of the developing event based on the analysis. In various embodiments, the causing includes: determining that a given user is interested in receiving information about an event having an event type; and based on determining that the given user is interested, pushing data indicating the new information to a computing device operated by the given user. In various described embodiments, the pushing causes the computing device operated by the given user to render the new information at one or more output components without the given user explicitly requesting the new information about the developing event. In various embodiments, determining the event type includes determining that one or more additional nodes or edges not found in the general knowledge graph match an event type template associated with the event type.

[0012] Other embodiments may include a non-transitory computer-readable storage medium storing instructions executable by one or more processors (e.g., a central processing unit (CPU), a graphics processing unit (GPU), and / or a tensor processing unit (TPU)) to perform methods such as one or more methods described above and / or elsewhere herein. Yet other embodiments may include a system of one or more computers including one or more processors operable to execute stored instructions for performing methods such as one or more methods described above and / or elsewhere herein.

[0013] It should be understood that all combinations of the foregoing concepts and additional concepts described in more detail herein are contemplated as part of the subject matter disclosed herein. For example, all combinations of the claimed subject matter appearing at the end of this disclosure are considered part of the subject matter disclosed herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a block diagram of an example environment in which implementations disclosed herein may be implemented.

[0015] Figure 2 Depicted are example scenarios in which the techniques described herein may be employed to detect developing events based on semantically related data detected at multiple live data streams.

[0016] Figure 3 Depicts the process after an event has been detected when live data streams and / or search engine queries are analyzed to determine additional information about the developing event. Figure 2 Example scene for .

[0017] Figure 4 Schematically illustrates how an event-specific temporary knowledge graph may be generated based in part on a general knowledge graph.

[0018] Figure 5 and Figure 6 Depicted are examples of how the techniques described herein may be employed to provide individuals with updates regarding developing events.

[0019] Figure 7 An example data processing pipeline configured with selected aspects of the present disclosure is depicted.

[0020] Figure 8 A flowchart illustrating an example method for practicing selected aspects of the present disclosure is depicted.

[0021] Fig. 9 An example architecture of a computing device is illustrated. DETAILED DESCRIPTION

[0022] Now turn to Figure 1 , illustrates an example environment in which the techniques disclosed herein may be implemented. The example environment includes one or more client computing devices 106. Each client device 106 may execute a respective instance of an automated assistant client 108, which may also be referred to herein as the "client portion" of the automated assistant. One or more cloud-based automated assistant components 119, which may also be referred to herein as the "server portion" of the automated assistant, may be implemented on one or more computing systems (collectively, "cloud" computing systems) communicatively coupled to the client devices 106 via one or more local area networks and / or wide area networks (e.g., the Internet), generally indicated at 114.

[0023] In various embodiments, an instance of automated assistant client 108, through its interaction with one or more cloud-based automated assistant components 119, may form what appears from the user's perspective to be a logical instance of automated assistant 120 that the user may engage with in a human-computer dialogue. Such an instance of automated assistant 120 is Figure 1 108 is depicted in dashed lines in FIG. 10. It should therefore be understood that each user engaging with an automated assistant client 108 executing on a client device 106 may actually engage with his or her own logical instance of automated assistant 120. For the sake of brevity and simplicity, the term "automated assistant" as used herein to "serve" a particular user will refer to the combination of an automated assistant client 108 executing on a client device 106 operated by the user and one or more cloud-based automated assistant components 119 (which may be shared among multiple automated assistant clients 108).

[0024] One or more client devices 106 may include, for example, one or more of the following: a desktop computing device, a laptop computing device, a tablet computing device, a mobile phone computing device, a computing device of a user's vehicle (e.g., an in-vehicle communication system, an in-vehicle entertainment system, an in-vehicle navigation system), a stand-alone interactive speaker (which in some cases may include a visual sensor), a smart appliance such as a smart TV (or a standard TV equipped with a networked dongle with automated assistant capabilities), and / or a user's wearable device including a computing device (e.g., a user's watch with a computing device, a user's glasses with a computing device, a virtual or augmented reality computing device). Additional and / or alternative client computing devices may be provided. Some client devices 106, such as stand-alone interactive speakers (or "smart speakers"), may take the form of an assistant device that is primarily designed to facilitate a conversation between a user and an automated assistant 120.

[0025] As described in greater detail herein, the automated assistant 120 participates in a human-computer conversation session with one or more users via user interface input and output devices of one or more client devices 106. In some implementations, the automated assistant 120 may participate in a human-computer conversation session with a user in response to a user interface input provided by the user via one or more user interface input devices of one of the client devices 106. In some of those implementations, the user interface input is explicitly directed to the automated assistant 120. For example, a user may verbally provide (e.g., type, speak) a predetermined invocation phrase, such as "OK, Assistant" or "Hey, Assistant", to cause the automated assistant 120 to begin actively listening to or monitoring typed text. Additionally or alternatively, in some implementations, the automated assistant 120 may be invoked based on one or more detected visual cues, either alone or in combination with a verbal invocation phrase.

[0026] In many embodiments, the automated assistant 120 may employ speech recognition processing to convert speech from the user into text, and accordingly respond to the text, for example, by providing search results, general information, and / or taking one or more responsive actions (e.g., playing media, launching a game, ordering food, etc.). In some embodiments, the automated assistant 120 is capable of responding to speech without converting the speech to text in addition or in lieu thereof. For example, the automated assistant 120 is capable of converting speech input into embeddings, into entity representations (which indicate one or more entities present in the speech input), and / or other "non-text" representations and operating on such non-text representations. Therefore, the embodiments described herein as operating based on text converted from speech input may additionally and / or alternatively operate directly on speech input and / or other non-text representations of speech input.

[0027] Each of the client computing device 106 and the computing device operating the cloud-based automated assistant component 119 may include one or more memories for storing data and software applications, one or more processors for accessing data and executing applications, and other components that facilitate communication over a network. The operations performed by the client computing device 106 and / or by the automated assistant 120 may be distributed across multiple computer systems. The automated assistant 120 may be implemented as, for example, a computer program running on one or more computers in one or more locations coupled to each other over a network.

[0028] The text-to-speech ("TTS") module 116 can be configured to convert text data (e.g., a natural language response formulated by the automated assistant 120) into computer-generated speech output. The speech-to-text ("STT") module 117 can be configured to convert the captured audio data into text. In some embodiments, this text (or alternatively, an embedding thereof) can then be provided to the natural language processor 122. In some embodiments, the STT module 117 can convert the audio recording of the speech into one or more phonemes, and then convert the one or more phonemes into text. Additionally or alternatively, in some embodiments, the STT module 117 can employ a state decoding diagram.

[0029] Automated assistant 120 (and in particular cloud-based automated assistant component 119) may include the aforementioned TTS module 116, the aforementioned STT module 117, and other components described in more detail below. In some embodiments, one or more of the modules of automated assistant 120 may be omitted, combined, and / or implemented in a component separate from automated assistant 120. In some embodiments, one or more of the components of automated assistant 120, such as natural language processor 122, TTS module 116, STT module 117, etc., may be implemented at least partially on client device 106 (e.g., excluding the cloud).

[0030] In some implementations, the automated assistant 120 generates responsive content in response to various inputs generated by a user of the client device 106 during a human-machine conversation session with the automated assistant 120. The automated assistant 120 may provide the responsive content for presentation to the user as part of the conversation session (e.g., over one or more networks while separate from the user's client device). For example, the automated assistant 120 may generate responsive content in response to free-form natural language input provided via the client device 106. As used herein, free-form input is input formulated by a user and is not limited to a set of options presented for selection by the user.

[0031] As used herein, a "conversation session" may include a logical, self-contained exchange of one or more messages between a user and automated assistant 120 (and, in some cases, other human participants). Automated assistant 120 may distinguish between multiple conversation sessions with a user based on various signals such as the passage of time between sessions, changes in user context (e.g., location, before / during / after a scheduled meeting, etc.) between sessions, detection of one or more intermediate interactions between the user and a client device other than the conversation between the user and the automated assistant (e.g., the user switches apps for a moment, the user walks away and then returns to a standalone voice-activated product), locking / hibernation of a client device between sessions, changes in the client device used to interface with one or more instances of automated assistant 120, etc.

[0032] The natural language processor 122 can be configured to process the natural language input generated by the user via the client device 106 and can generate annotated output (e.g., in text form) for use by one or more other components of the automated assistant 120. For example, the natural language processor 122 can process the natural language free-form input generated by the user via one or more user interface input devices of the client device 106. The generated annotated output includes one or more annotations of the natural language input and one or more (e.g., all) of the terms of the natural language input. In some implementations, the natural language processor 122 can be configured to ultimately predict the user's intent based on the user's natural language input.

[0033] In some implementations, the natural language processor 122 may additionally and / or alternatively include an entity tagger (not depicted) configured to annotate entity references in one or more segments, such as references to people (including, for example, literary characters, celebrities, public figures, etc.), organizations, locations (real and fictional), etc. In some implementations, the natural language processor 122 may additionally and / or alternatively include a coreference resolver (not depicted) configured to group or "cluster" references to the same entity based on one or more contextual clues.

[0034] In some embodiments, data about entities may be stored in one or more databases, such as in a general knowledge graph 138 (which is Figure 1In some embodiments, the general knowledge graph 138 may include nodes representing known entities (in some cases, attributes of entities), and edges connecting the nodes and representing relationships between entities. For example, a "banana" node may be connected (e.g., as a child node) to a "fruit" node, which in turn may be connected (e.g., as a child node) to a "produce" and / or "food" node. As another example, a restaurant called a "Hypothetical Café" may be represented by a node that also includes attributes such as its address, the types of food served, business hours, contact information, etc. The "Hypothetical Café" node may be connected to one or more other nodes, such as a "restaurant" node, a "business" node, a node representing the city and / or state in which the restaurant is located, etc., via edges (e.g., representing a child-to-parent relationship) in some embodiments.

[0035] The fulfillment module 124 (also referred to as a "query processor" or "parsing module") can be configured to receive the predicted / estimated intent output by the natural language processor 122 and fulfill (or "parse") the intent. In various embodiments, the fulfillment (or "parsing") of the user's intent can cause various fulfillment information (also referred to as "response" information or "parsed information") to be generated / obtained, for example, by the fulfillment module 124. In some embodiments, the fulfillment information can be provided to a natural language generator ("NLG" in some figures) 126, which can generate a natural language output based on the fulfillment information.

[0036] The fulfillment (or "resolution") information can take various forms, as the intent can be fulfilled (or "resolved") in various ways. Assuming a user requests pure information, such as "Where were the outdoor shots of 'The Shining' filmed?" the user's intent can be determined as a search query, for example, by intent matcher 135. The intent and content of the search query can be provided to fulfillment module 124, which can be used to perform a search. Figure 1As depicted, the fulfillment module 124 may communicate with one or more search modules 128 configured to search a corpus of documents of, for example, the knowledge system 130 and / or other data sources (e.g., the knowledge graph 138, etc.) for responsive information. The fulfillment module 124 may provide data indicative of the search query (e.g., the text of the query, a reduced dimensionality embedding, etc.) to the search module 128. The search module 128 may provide responsive information, such as GPS coordinates or other more explicit information, such as "Timberline Lodge, Mt. Hood, Oregon". The responsive information may form part of the fulfillment information generated by the fulfillment module 124.

[0037] Knowledge system 130 may include one or more computing devices, such as one or more server computers (e.g., blade servers) acting in concert to compile knowledge from various sources (e.g., a database of web crawled documents) and to provide various services and information to requesting entities. Knowledge system 130 may, among other things, function as a search engine that provides responsive information (e.g., search results, direct response information, deep links, etc.) to search queries received from individuals and / or from automated assistant 120. Knowledge system 130 may also perform a variety of other services, and thus is not limited to Figure 1 Those components depicted in .

[0038] The knowledge system 130 may include the previously mentioned knowledge graph 138 as well as a query monitor 132, a live stream monitor 134, and / or a temporary knowledge graph manager ( Figure 1 KG Manager") 136. In some embodiments, the knowledge system 130 may, among other things, receive search queries and provide response information. In various embodiments, the query monitor 132 may be configured to monitor queries submitted to, for example, a search engine such as the knowledge system 130, and cluster queries that are semantically related to each other and / or to the developing event. Clustering of semantically related queries may then be utilized to provide interested users with evolving information about the developing event, for example, from other queries and / or from a live data stream.

[0039] As previously mentioned, information about new developing events such as disasters, crimes, impromptu gatherings (e.g., protests), accidents, etc. may not be immediately available from traditional information sources such as the general knowledge graph 138 or from other data sources often utilized by search engines. Instead, these traditional data sources often lag behind other more agile sources such as live data streams. Therefore, in various embodiments, the live stream monitor 134 can be configured to monitor multiple live data streams to detect developing events.

[0040] As used herein, a "live data stream" may include an information stream generated by one or more individuals (e.g., by an entity such as a business or government organization) that includes multiple updates generated over time. These updates may include text, images, video, audio recordings, or any other form of user-created, user-curated, and / or user-compiled information that can be disseminated to other individuals who can access these live data streams. A common form of live data stream update is a social media post, in which the posting user can include text, audio data, video data, image data, hyperlinks, applets, etc.

[0041] exist Figure 1 For example, the first social network 140 may provide a first plurality of live data streams 142, for example, from a plurality of users of the first social network 140. 1-N Each live data stream 142 may be generated by one or more persons associated with a social media profile of the first social network 140. Similarly, the second social media network 144 may provide a second plurality of live data streams 142, for example, from a plurality of users of the second social network 144. N+1 -142 M . N and M are positive integers. Although Figure 1 Two social networks are depicted in FIG, but this is not intended to be limiting. Any number of social networks of various types may provide live data streams.

[0042] Some social networks may be designed to primarily allow users to create posts that are limited to a certain number of characters, words, images, etc., and to allow other users to "follow" the posting user, even if the other users do not know the posting user personally. For example, with some social networks, many users can follow (or subscribe to) a smaller number of "influencers," who are often celebrities or other well-known people (e.g., politicians, journalists). These influencers are unlikely to follow (or subscribe to) the live streams of their followers unless those followers are personal acquaintances of the influencer or are other influencers. By consistently disseminating information that followers find attractive or otherwise valuable, influencers can, in some cases, gain credibility and / or generate income based on their social media posts.

[0043] Other social networks may be designed primarily to allow users to connect to (or "become friends") with other users they know or meet in real life. Once connected, users are able to share content such as text, images, video, audio, hyperlinks, etc. with their "friends" or "contacts" on social media. These other social networks may be used by users to connect with friends on an informal basis, and / or to connect with business acquaintances on a more formal basis (e.g., similar to a virtual rolodex).

[0044] Despite Figure 1 Not depicted in , but other systems similar to social networks can also be sources of live data streams. For example, a user may have an account on a video sharing website and post a video to the video sharing website. In some cases, these video sharing websites can allow users to subscribe to other users' video updates, similar to some of the social networks described previously. Thus, for example, a particular user who posts a video regularly or periodically can actually create a live stream. Other similar websites are specifically designed to enable users to post images, particularly media that spread ideas, activities, concepts, jokes, buzzwords, or other information, commonly referred to as Internet "memes".

[0045] Based on the information obtained from the query monitor 132 and / or the live stream monitor 134, the temporary knowledge graph manager 136 can be configured to build, manage and / or maintain one or more event-specific temporary knowledge graphs 139. In some embodiments. For example, before more traditional data sources such as "rule" or "general" knowledge graphs 138 are updated to include information about developing events, each event-specific temporary knowledge graph 139 can be built for a specific developing event. For example, the information added to the event-specific temporary knowledge graph 139 may not be (as severely) subject to the press or other standards previously described. In some embodiments, the event-specific temporary knowledge graph 139 can be implemented, for example, by the temporary knowledge graph manager 136 as a layer on top of the general knowledge graph 138.

[0046] In some embodiments, a given event-specific temporary knowledge graph 139 may be temporary or “ephemeral” in nature (thus, the event-specific temporary knowledge graph 139 may be Figure 1 Rendered in dashed lines in FIG. 1 ). In some such embodiments, information added to the event-specific temporary knowledge graph 139 may eventually be added to the knowledge graph 138, for example, once the information is sufficiently corroborated. To the extent that any information (e.g., facts) in addition to or in place of the event-specific temporary knowledge graph 139 cannot be corroborated or verified, it may eventually be discarded or replaced with verifiable information.

[0047] Assume that an abandoned ice skating rink collapses, but initial live stream updates (e.g., social media posts) from one or more eyewitnesses incorrectly identify the collapsed structure as an abandoned bowling alley. In various embodiments, a location or building associated with a developing event will initially be identified as a bowling alley in the event-specific temporary knowledge graph 139. However, when later information contradicts this fact and correctly reports that the collapsed structure was an ice skating rink rather than a bowling alley, it can be added to the event-specific temporary knowledge graph 139 and / or updated within the event-specific temporary knowledge graph 139 to reflect its "collapsed" status.

[0048] Figure 2 An example scenario is depicted in which the techniques described herein may be employed, for example, by a live stream monitor 134 to detect developing events based on semantically related data detected at multiple live data streams (eg, 1421 -M). Figure 2 A schematic depicting a portion of the city from above, with black lines representing roads and a river winding from the upper left to the lower right center. A fire has broken out at a church located in the eastern part of the city, as indicated by the flames.

[0049] In the earliest stages of the fire, for example, when it is first detected by one or more witnesses, it is unlikely that traditional data sources such as the knowledge graph 138 or search engines will have available information about the fire. During this period, the fire can be considered a "developing event" whose information is being collected by multiple sources. These sources can propagate this information through live data streams. The live stream monitor 134 can employ the techniques described herein to determine the likelihood that a new event is developing. If the likelihood meets a certain criterion, such as a minimum threshold, the live stream monitor 134 can provide "temporary" information about the developing event to the temporary knowledge graph manager 136. The temporary knowledge graph manager 136 can organize the information into nodes and edges of an event-specific temporary knowledge graph 139, similar to those found in the general knowledge graph 138. Subsequently, the event-specific temporary knowledge graph 139 can be queried, for example, by various components to obtain information about the developing event.

[0050] exist Figure 2 In , a first user who is very close to the fire has posted to social media "OMG! There's a fire at the cathedral! ". Another nearby user posted "There must be ten fire trucks fighting the church fire! ". Yet another nearby user posted "I hope the parishioners were able to escape the fire... ". In various embodiments, the live stream monitor 134 may analyze multiple live streams, including these three live streams, to determine that the developing event is likely to be occurring. The live stream monitor 134 may syntactically and / or semantically process or cause another system to syntactically and / or semantically process these live data stream updates to determine that they are related. For example, the live stream monitor 134 may generate embeddings of these live stream updates and determine that they are related based on the fact that the embeddings are clustered together in the latent space.

[0051] In some implementations, the live stream monitor 134 may consider other information signals besides live stream updates when determining whether a developing event is occurring. Figure 2 In the embodiment of the present invention, the live stream monitor 134 can determine that the three users are close to each other according to the location coordinates (e.g., GPS) generated by the mobile device operated by three different users. The live stream monitor 134 can determine that the three live stream updates are close in time, such as within minutes or even seconds of each other in addition or alternatively. In conjunction with the syntax and / or semantic similarity of their live stream updates, the live stream monitor 134 can not only determine that the developing event is unfolding, but also determine that the developing event is unfolding somewhere close to the three users. In addition to those just mentioned or in place of those just mentioned, other information signals can also be considered, including but not limited to demographic information (e.g., age, gender) about the posting user, known preferences or interests (e.g., determined according to their online profiles), known affiliations (e.g., alma mater, organization, employer) of the posting user, etc.

[0052] The first three posting users mentioned above Figure 2 In some embodiments, their posts may be sufficient to determine with a reasonable amount of confidence that a developing event (church fire) is occurring. However, other users who are spatially and / or temporally far from the fire may also update their live streams with potentially relevant information. These more distant users may not be so directly aware of the fire, but may instead have situational knowledge of the fire. For example, another user a few blocks west of the fire station posts "There are so many fire trucks headed downtown!". The user may not know that a particular church is on fire, but can infer that there is a fire somewhere in the downtown area based on the trajectory of the fire trucks he or she observes. As another example, a user south of the river posts "Anyoneknow why there is smoke across the river? Again, the south user may not know which buildings are on fire, but can infer that a fire is occurring.

[0053] Whether or how useful live stream updates from more distant users are useful may depend on whether they include additional information that is not available in live stream updates from users that are closer (either spatially or temporally) to the developing event. Assume that a post from a west user, "There are so many fire trucks headed downtown!" appears and / or is detected by live stream monitor 134 before a post from a spatially closer user, "There must be ten fire trucks fighting the church fire!" The west user's post may signal that the fire is likely to be relatively large, whereas the other two posts from two other spatially close users ("I hope the parishioners... " and "OMG! There's a fire... ") only establish the presence of a fire, not its potential size.

[0054] In some embodiments, certain live data streams may be considered more reliable than others. In some cases, updates posted to a particularly reliable live data stream may trigger the live stream monitor 134 to search for additional information in other lesser-known live data streams. For example, in Figure 2 , the Fire Department ("FD") posts its own live stream update "CODE12345@STATE AND MAIN". Assuming that "CODE 12345" corresponds to a fire, such a post may be considered highly reliable due to its source being an official government agency rather than a little-known individual. Notably, the post does not indicate that it is a church that is on fire, but instead simply states the location as "STATE AND MAIN". Assuming there are other buildings in the area, it may not be possible to infer that the church is on fire based on the Fire Department's post alone. However, when combined with other corroborating posts indicating that the church is on fire, live monitor 134 can become very confident that the developing event of a church fire is occurring. Although the Fire Department may receive false alarms, even its posts may not necessarily be considered conclusive or definitive by live stream monitor 134.

[0055] In some embodiments, the live stream monitor 134 can monitor a variety of live streams as a matter of course. For example, the live stream monitor 134 can continuously or periodically monitor live streams that are known to be particularly timely and / or reliable, such as live streams generated by first responders, reporters, influencers, government entities, etc. Additionally or alternatively, the live stream monitor 134 can begin monitoring a certain set of live data streams 142 in response to various events. These live data streams can be selected, for example, randomly from live data streams that are known to be reliable, from live data streams that are known to have some spatial or other relationship to the developing event, etc.

[0056] Assume that the knowledge system 130 receives a user query seeking information related to a developing event, and the knowledge system 130 cannot find responsive information in its traditional data sources, such as the general knowledge graph 138 and / or documents that it has crawled and indexed. In some such embodiments, the query monitor 132 may determine that a new developing event may be occurring. The query monitor 132 may notify the live stream monitor 134. The live stream monitor 134 may respond by selecting and monitoring / analyzing a set of live data streams 142 based on the user query to identify a subset of two or more live data streams as being related to the developing event.

[0057] Assume that the knowledge system 130 receives a search query asking "Who is rioting in Times Square?" Assume further that the riots occurring in Times Square have recently begun so that traditional data sources such as the knowledge graph 138 and / or documents indexed / crawled by the knowledge system 130 do not yet have responsive information. For example, a live stream monitor 134 triggered by a query monitor 132 may select a set of live data streams that are most likely to provide new information about the potential developing event. For example, the live stream monitor 134 may select one or more live streams 142 that include recent updates to the developing event generated by a computing device that is geographically close to Times Square, possibly operated by a first-hand eyewitness. The live stream monitor 134 may additionally or alternatively select other live data streams 142 that are known to be related to Times Square, such as live data streams 142 generated by entities such as businesses, government agencies, etc. (e.g., departments of commerce, municipal live broadcasts, etc.) that are geographically close to Times Square or otherwise related to Times Square.

[0058] Figure 3 Depicted after the developing event has been established (e.g., the confidence measure that the developing event is real meets some threshold) Figure 2 Example scenario. At this point, the live data stream and / or search engine queries can be analyzed to determine additional information about the developing event. This data can then be used to provide information to other inquiring users. Figure 3 Follow the events in time Figure 2 events, but they may still be relatively "fresh" so that conventional information sources such as the knowledge graph 138 and / or search engines are still not fully informed.

[0059] exist Figure 3 , a user to the north submits the search engine query "Did the stain glass windows survive?". The query may be determined, for example, by the query monitor 132 to be semantically relevant to the developing church fire event, for example, because "stained glass windows" may generally have a relatively strong semantic relationship to churches, and the word "survive" implies the occurrence of some destructive event such as a fire. Another user relatively close to the fire asks "How long did repairs to the cathedral take after the 1948 fire?". Yet another user located at the southernmost end of the city asks "Where are there any injuries caused by Tuesday's fire?". A user to the southwest asks "What caused the cathedral fire?". A user to the west asks "Is the hypothetical crown still in the cathedral?". Any of these queries, whether or not they may still be answerable by conventional data sources such as a search engine or knowledge graph 138, can be exploited to quickly disseminate new information about developing events to interested users.

[0060] Figure 4 Schematically illustrates how an event-specific temporary knowledge graph 139 may be related to a conventional knowledge graph 138 in accordance with various embodiments. Figure 4 In the example, the general or conventional knowledge graph 138 includes a plurality of entity nodes and a plurality of edges between the entity nodes. The plurality of entity nodes may represent entities and the plurality of edges may represent relationships between entities. The event-specific knowledge graph 139 also includes entity nodes and edges, but is much smaller than the rule knowledge graph 138. In practice, the rule knowledge graph 138 will almost certainly be smaller than the event-specific knowledge graph 139. Figure 4 Much greater than what is depicted in the book. Figure 4 The limited number of nodes and edges of the knowledge graph 138 depicted in FIG. 1 is for illustrative purposes only.

[0061] As previously described, various components such as the live stream monitor 134 and / or the temporary knowledge graph manager 136 can analyze two or more live data streams. Based on the analysis, one or more entities associated with the developing event can be identified. In some embodiments, one or more identified entities can form part of and / or be identified from the general knowledge graph 138, such as Figure 4 As depicted by the shaded nodes of the general knowledge graph 138.

[0062] Based on the identified one or more entities, an event-specific temporary knowledge graph 139 associated with the developing event can be constructed (or updated if it already exists). Figure 4 As shown, the event-specific temporary knowledge graph 139 shares one or more entity nodes with the general knowledge graph 138. However, the event-specific temporary knowledge graph 139 includes one or more additional nodes or edges not found in the general knowledge graph 138, such as the developing event node 450. These new nodes and / or edges can convey the relationship between the identified one or more entities and the developing event.

[0063] Once the event-specific temporary knowledge graph 139 is created, it can be used for various purposes. In some embodiments, the event-specific temporary knowledge graph 139 can be queried for information about the developing event. For example, a user query seeking information related to the developing event can be received by the knowledge system 130, for example. The knowledge system 130 can determine that the general knowledge graph 138 does not include information responsive to the user query. Before the knowledge system 130 notifies the querying user that response information is not yet available, the knowledge system 130 can query the event-specific temporary knowledge graph 139 for information about the developing event. Assuming that the event-specific temporary knowledge graph 139 includes response information, the knowledge system 130 can obtain and return this response information from the event-specific temporary knowledge graph 139 (in some cases together with an annotation or notification that the information may not be confirmed).

[0064] In some embodiments, once one or more components of an event-specific temporary knowledge graph 139 are sufficiently validated and / or verified, those components may be merged or otherwise incorporated into the general knowledge graph 138. Additionally or alternatively, once the facts and / or relationships represented by one or more nodes or edges of the event-specific temporary knowledge graph 139 are no longer needed or there is no longer sufficient demand to warrant maintaining the event-specific temporary knowledge graph 139, they may be deleted and / or merged into existing nodes of the general knowledge graph 138.

[0065] Individuals may subscribe to information about developing events in a variety of ways. In some embodiments, an individual may be the one who issued an early query related to a developing event, such as Figure 3 One of the individuals who issues a query about a developing event (“Did the stainglass windows survive?” or “What caused the cathedral fire?”). As more information about the developing event is learned, for example from one or more live data streams, the event-specific temporary knowledge graph 139 associated with the developing event can be updated. As the event-specific temporary knowledge graph 139 is updated, individuals can receive updates with information that has not yet been presented to them. More generally, in some embodiments, an individual who issues a query about a developing event, regardless of whether the individual's query aids in the initial detection of the developing event, can effectively “subscribe” the individual to receive updates from the event-specific temporary knowledge graph as they become available.

[0066] Figure 5-6 An example of how a user 501 may subscribe to information about a developing event and later receive updates associated with the information about the developing event is depicted. Figure 5 In the embodiment, the client device 506 takes the form of a stand-alone interactive speaker. The client device 506 includes various components commonly found in such client devices, such as a microphone for capturing the speech of the user 501, which are not described in detail herein. Figure 5 In Figure 5, user 501 has invoked automated assistant 120 to ask the question "Tell me about the cathedral fire". Knowledge system 130 may not yet have information about recent cathedral fires, and therefore responds with "Many cathedrals and religious buildings have burned throughout history. What cathedral are you referring to?".

[0067] At this point, knowledge system 130 is unable to provide responsive information for automated assistant 120 to provide. However, in various embodiments, query monitor 132 may be set into motion and may, for example, in conjunction with live stream monitor 134, begin selecting and / or analyzing one or more live data streams and / or queries submitted to knowledge system 130 to determine whether there is a developing event potentially related to the church burning unfolding.

[0068] exist Figure 5, user 501 responds “I'm talking about the fire that's happening right now downtown.” The response contains multiple pieces of information that query monitor 132 and / or live stream monitor 134 can utilize to better identify and / or analyze live data streams and / or queries in providing information about potential developing events. In particular, it is now clear that the fire is close (“happening right now”) and that it is occurring “downtown,” which can be narrowed down to the nearest metropolitan area near user 501. Query monitor 132 and / or live stream monitor 134 can select queries and / or live data streams that were recently generated near the downtown area to determine information about the developing event.

[0069] Based on the updates obtained from these live data streams, the temporary knowledge graph manager 136 can generate and / or update an event-specific temporary knowledge graph 139 associated with the developing "church fire" event. Once this event-specific temporary knowledge graph 139 is established, it can be subsequently updated as new information is discovered from subsequent queries / live stream updates, and can be queried for additional information about the developing event. At the same time, the automated assistant says "OK, let me do some digging..."

[0070] For example, in Figure 5 After a certain amount of time (a few seconds, a few minutes, etc.), automated assistant 120 proactively follows up with additional information about the developing incident without requiring another call from user 501: "OK, Here's what I found: One source said that the fire has enveloped the nave. Another source said the fire was caused by lighting. Another source said one minute ago that the fire department is just arriving."

[0071] Notably, in some embodiments, if a particular live stream update or query identifies a point in time or relative time at which some development occurred in a developing event (e.g., "just" in the update, "the fire department just arrived"), the automated assistant 120 or another component, such as the temporary knowledge graph manager 136, the live stream monitor 134, etc., can add an additional time description (e.g., "one minute ago") so that the user 501 has a better understanding of when the alleged development occurred, or at least when the live stream update of the event occurred. Similarly, if the live stream update or query provides some relative geographic information (e.g., "across the river," "in my building," etc.), the automated assistant 120 and / or another component can identify the location of the source (e.g., using GPS coordinates) and use that to provide additional information about the context of the source post, such as "one source located south of the river said there's a lot of smoke across the river."

[0072] Figure 6 An example is depicted of how the same user 501 may receive updates about a developing event at a later time, e.g., outside of the conversation session with the automated assistant 120 in which the user 501 initially inquired about the developing event. Figure 6 , user 501 is watching television on a client device 606 in the form of a "smart" television, which is, for example, locally configured with an automated assistant client 108 and / or has an assistant-configured dongle installed. In either case, automated assistant 120 is pushed or requests updates from an event-specific temporary knowledge graph 139 associated with the church fire event. User 501 receives updates from automated assistant 120 as new information is determined. For example, in Figure 6 , automated assistant 120 proactively and without user 501 requesting it, provides a visual output overlaying content rendered on client device 606, “The cathedral fire has been 90 percent contained.”

[0073] In various embodiments, in addition to those individuals who submit search queries about events, individuals can subscribe to receive updates about developing events in response to other occurring events. In some embodiments, the event type of the developing event can be used to determine whether to subscribe to a specific individual. For example, some individuals may be interested in political protests. When a new developing event is detected, for example, by the query monitor 132 and / or the live stream monitor 134, and is estimated (for example, by the temporary knowledge graph manager 136) to be a political protest, at least some of those individuals who are interested in the political protest can subscribe to automatically receive updates. Other signals may affect such decisions, such as proximity to the developing event and / or the scope of the developing event. For example, individuals in Oregon who are interested in political protests may not necessarily be interested in political protests in distant states such as Kentucky, especially if the scope of those protests is limited to Kentucky politics. In contrast, the same person may be interested in nearby political protests, even if the scope of those protests is limited to Oregon politics. And the same person may be interested in distant protests across the country, such as protests in Washington, D.C. or other places related to national politics rather than local or regional politics.

[0074] The event type of a developing event can be predicted / estimated in various ways. In some embodiments, one or more live data streams can be analyzed, for example, by a temporary knowledge graph manager 136, as previously described to parse and analyze data about the developing event. Such analysis can include comparing the data with one or more "event type templates". Event type templates can be associated with specific event types, such as protests, riots, disasters, sporting events, concerts, political rallies, criminal acts, terrorist attacks, etc. Event type templates can include "slots" associated with expected event type data points that are typically associated with specific event types. For example, a sports event type template can include slots for "scores", "players", "coaches", "appearances", "teams", and other data points commonly found in social media posts related to sports events. A "building fire" event type template can include slots for "fire", "flames", "smoke", "fire trucks", "EMS", "hose", and other data points commonly found in social media posts related to building fires.

[0075] In some embodiments, in addition to or in lieu of event type templates, an event-specific temporary knowledge graph 139 and / or data indicating its constituent entities / edges may be applied as input across one or more event type machine learning models such as support vector machines, neural networks, graph neural networks, graph convolutional networks, graph attention networks, and the like. Such machine learning models may be trained to generate outputs that predict the event type of an event. For example, a neural network may be trained using training examples that each include multiple features extracted from events of known event types. Each training example may be labeled with a corresponding event type. The loss function of a machine learning model may be the difference between a model output generated by applying a training example across a model and a label assigned to the training example. The loss function may then be used to train the model, for example, using techniques such as gradient descent, back propagation, and the like.

[0076] In some embodiments, a single machine learning model can generate multiple outputs representing multiple probabilities that an event has multiple different event types. In some such embodiments, the event type with the highest probability can be selected as the inferred event type for the developing event. Additionally or alternatively, in some embodiments, multiple machine learning models or classifiers can be trained separately to generate outputs indicating specific event types.

[0077] In other embodiments, features of a developing event can be applied as input across a machine learning model to generate a semantically rich latent space embedding. The distance of this embedding to other "reference" embeddings in the latent space can indicate the similarity of the embedding to those reference embeddings. If those nearby reference embeddings were generated from a particular type of developing event, then the new embedding—and therefore, the developing event from which it was generated—can be estimated to be the same kind of event.

[0078] In some embodiments, a developing event and its event-specific temporary knowledge graph 139 can be assigned multiple candidate event types, each candidate event type having a corresponding probability of being the correct event type. These probabilities can change as more information is learned about the developing event and the event-specific temporary knowledge graph 139 is updated. For example, a developing event may initially be classified as most likely a political protest, but also potentially classified as a political rally or riot. However, as the participants in the event become excited and / or violent, the developing event may be reclassified as most likely a riot (or classified as both a political rally and a riot if it is determined that the event started as a political rally but evolved into a riot). In some such embodiments, an individual may initially receive updates about the developing event because the developing event is classified as an event type that the individual is interested in. However, once the developing event is reclassified as a different event type that the individual is not interested in, the individual may be unsubscribed from receiving updates about the developing event.

[0079] Figure 7 1 depicts a data processing pipeline 700 that can be implemented by the knowledge system 130 in various embodiments. The components of the data processing pipeline 700 can span query monitors 132, live stream monitors 134, temporary knowledge graph managers 136, and / or knowledge systems 130. Figure 1 One or more components different from those depicted are distributed.

[0080] In various embodiments, the crawler / parser 760 can be configured to crawl various sources of live data streams 142, such as social networks 140 / 144, to identify and / or select live data streams that may include information related to the developing event under consideration. For example, assume that the knowledge system 130 receives an initial search query about an event that is too fresh to have been recorded in the general knowledge graph 138. The live stream monitor 134 and / or the temporary knowledge graph manager 136 can identify live data streams 142 and / or other search queries submitted using computing devices that are known to be geographically close to, for example, the location identified in the initial search query and / or close to the coordinates generated by the computing device used to submit the initial search query. The crawler / parser 760 can then parse one or more text updates (e.g., social media posts) received through each identified live data stream for information of potential relevance to the developing event.

[0081] In some embodiments, the image / video sub-pipeline 761 can be configured to crawl and / or process non-text updates from one or more live data streams. These non-text updates can include, for example, images, videos, audio files, etc. In the case of images (and / or videos), various image processing techniques such as optical character recognition, object detection, facial recognition, etc. can be used to identify objects and / or text depicted in images published on the live data stream. These identified objects and / or text can then be analyzed by the components of the pipeline 700 described below, for example, to identify one or more entities, to determine information about the developing events depicted in the image, etc. In the case of audio data published on the live data stream, speech recognition processing can be applied to extract text related to the developing event information.

[0082] The parsed (and in some cases annotated) output generated by the crawler / parser 760 and / or the output of the image / video sub-pipeline 761 may be analyzed using one or more natural language processing (“NLP”) models for topic extraction 762 and / or entity recognition / linking 764. The NLP model for topic extraction 762 may take the form of a statistical model such as a probabilistic latent semantic analysis (“PLSA”) model, a latent Dirichlet allocation (“LDA”), a pachinko allocation, etc. The NLP model for entity recognition / linking 764 may take the form of an “entity tagger” similar to that previously described as part of the natural language processor 122 of the automated assistant 120. In fact, in some embodiments, a cloud-based automated assistant component 119 such as the natural language processor 122 and / or the natural language generator 126 may be implemented as part of the knowledge system 130. In some such embodiments, the natural language processor 122 may be used to implement both the automated assistant 120 and / or as part of the data processing pipeline 700.

[0083] The outputs of components 762 and 764 can be processed by a deduplicator 766 to ensure that entities and / or relationships are not repeated within an event-specific temporary knowledge graph 139. The deduplicator 766 can determine, for example, which information is new information and which information is duplicate information. For example, it is not uncommon for a first user to post a live stream update (e.g., text, image, video, audio, etc.) and then for followers of the first user to repost the same update so that their own downstream followers can see it. However, for the purpose of populating an event-specific temporary knowledge graph 139, the reposted update may not be useful for obtaining new information, but it may be used for other purposes, such as to confirm or verify the information conveyed in the reposted update. In some such cases, the reposted data may be discarded or otherwise ignored.

[0084] When receiving updates about developing events, individual subscribers may not wish to simply receive regurgitated live stream updates. They may prefer to extract information from those live stream updates and present them in a realistic manner. Therefore, a natural language model 768 for sentence summarization can be provided, which, for example, shares one or more characteristics with the natural language generator 126 of the automated assistant 120. In particular, the natural language model 768 for sentence summarization can facilitate the generation of sentences (or more generally, natural language) that summarize information received from one or more live data streams.

[0085] The notification streamer 770 and the TTS streamer 772 may generally be configured to provide new information about the developing event to individuals who have subscribed to the developing event, e.g., for presentation as a rendered output on one or more client devices 106. In some embodiments, an individual may query the event-specific temporary knowledge graph 139 for information about the developing event at any point in time. In contrast, components 770 and 772 may be configured to "push" new information generated by upstream components 760-766 of pipeline 700 to subscribed individuals, e.g., so that the new information is "surfaced" to those individuals without them necessarily specifically requesting it. As used herein, "surfacing" content to an individual means, for example, outputting the content audibly and / or visually on one or more computing devices operated by the individual. In some cases, surfacing may refer to providing a notification (e.g., a card, a pop-up notification, an audio alert, etc.) presented to an individual, regardless of whether the individual is currently engaged in a task and / or even operating a client device. For example, this can be accomplished by presenting a "card" on the home or lock screen of a smartphone or tablet computing device or by Figure 6 The notification is overlaid on the television screen as described to surface the information to the user.

[0086] The notification streamer 770 can be configured to process the parsed and / or annotated content generated by the upstream components of the pipeline 700 and generate content to be rendered primarily visually on a display of a computing device. The content can be presented as a card, a pop-up notification, a banner, a ribbon, a list, etc.

[0087] TTS streamer 772 can be configured to process parsed and / or annotated text generated from upstream components of pipeline 700 and convert the text into speech output that can be provided to the individual, for example, audibly or visually by automated assistant 120. In some such embodiments, automated assistant 120 can proactively provide these speech output updates as new information about the developing event comes in and / or is determined to be sufficiently reliable, for example, without the individual requesting them.

[0088] In some embodiments, the TTS streamer 772 (or notification streamer 770) can use various signals to determine which information and / or how much information to surface to a particular individual about a developing event. For example, the available output modalities of a computing device operated by an individual can be considered. A user driving a car configured with an assistant-enabled computing system and / or a user near an independent interactive speaker can be used to receive notifications in an auditory manner, such as via a speaker. However, audio notifications can be distracting or irritating if they are provided too frequently or if they do not contain new information. Therefore, components 770-772 can provide less frequent updates to individuals reachable via audio output than other individuals (e.g., sitting in front of a computer screen), for example, that can be used to receive visual updates.

[0089] Other signals that may be used by components 770-772 to determine how many and / or whether to surface live stream updates to an individual regarding a developing event include, but are not limited to, confidence in the event type currently associated with the developing event, user preferences (generally and related to specific developing event types), past behavior of the individual (did they previously reject surfaced information, or request more information than was proactively surfaced?), etc. For some event types, such as sporting events, a subscribing individual may desire more frequent updates, e.g., every time someone scores, gets injured, a foul is called, etc. For other event types, such as political rallies, an individual may wish to receive less frequent updates. In some embodiments, information about a developing event may be proactively surfaced to a subscribing individual only when the information has not already been presented to the individual and / or only when the information is from a source that is deemed sufficiently reliable.

[0090] In some embodiments, components 770-772 may be configured to notify subscribers when an event type has changed, e.g., so that individuals can decide whether they wish to continue receiving updates. For example, a politically minded subscriber may receive updates on a developing event that was initially classified as a spontaneous political rally. However, as more information from more live streams and / or search queries is analyzed, it may become clear that the rally was not political in nature, but instead was a "pump-up" rally intended to excite the fan base of a particular sports team. Those politically minded subscribers may be notified of new event types, e.g., so that they can unsubscribe if they choose to do so.

[0091] In some embodiments, an individual who receives an update about a developing event may be able to, for example, request information about the source of a particular update using voice input or by interacting with a graphical user interface. This may enable an individual to make his or her own determination about how reliable or unreliable a particular update is. In some cases, an individual may be able to provide feedback about a source when analyzing subsequent live stream updates from the source, which feedback can be used in association with the individual or more broadly across a population. For example, if an individual determines that a developing event was mischaracterized as a political rally when it was actually a rally for encouragement, the individual can mark one or more sources that caused the developing event to be classified as a political rally, for example, so that those sources are "downgraded" (e.g., trust in those sources is reduced) moving forward.

[0092] Figure 8 800 is a flowchart illustrating an example method 800 according to implementations disclosed herein. For convenience, the operations of the flowchart are described with reference to a system that performs the operations. The system may include various components of various computer systems, such as one or more components of a computing system that implements automated assistant 120. Furthermore, while the operations of method 800 are shown in a particular order, this is not intended to be limiting. One or more operations may be reordered, omitted, or added.

[0093] At block 802, the system may analyze two or more live data streams, such as by way of a live stream monitor 134 and / or a crawler / parser 760. As previously described, in some embodiments, the system may select a set of live data streams 142 that are considered particularly reliable and / or report new information quickly, such as journalists' live streams, first responders' live streams, etc. Additionally or alternatively, in some embodiments, the system may select and begin monitoring two or more live data streams based on the sources of those streams presumably being near a developing event.

[0094] Based on the analysis, at block 804, the system may identify one or more entities associated with the developing event. In various embodiments, the one or more entities may form part of a general knowledge graph 138, as previously described, which includes a plurality of entity nodes and a plurality of edges between the plurality of entity nodes, wherein the plurality of entity nodes represent entities and the plurality of edges represent relationships between the entities. In other embodiments, inferred entities may be temporarily identified based on the analysis, even if those inferred entities cannot be immediately matched to established entities in the general knowledge graph 138.

[0095] Based on the identified one or more entities, an event-specific temporary knowledge graph 139 may be constructed in association with the developing event (or updated if it has already been constructed) at block 806. As previously described, the event-specific temporary knowledge graph may share one or more entity nodes with the general knowledge graph, but may also include one or more additional nodes or edges not found in the general knowledge graph that convey the relationship between the identified one or more entities and the developing event.

[0096] In some embodiments, in response to the building or updating of block 806, or periodically / continuously, the system may query the event-specific temporary knowledge graph 139 for new information about the developing event at block 808. At block 810, the system may cause one or more computing devices to render the new information as output, for example, by notifying the streamer 770 and / or the TTS streamer 772.

[0097] Fig. 9 9 is a block diagram of an example computing device 910 that may optionally be used to perform one or more aspects of the techniques described herein. The computing device 910 typically includes at least one processor 914 that communicates with a number of peripheral devices via a bus subsystem 912. These peripheral devices may include a storage subsystem 924, including, for example, a memory subsystem 925 and a file storage subsystem 926; an interface output device 920; a user interface input device 922, and a network interface subsystem 916. The input and output devices allow a user to interact with the computing device 910. The network interface subsystem 916 provides an interface to an external network and is coupled to corresponding interface devices in other computing devices.

[0098] User interface input devices 922 may include a keyboard; a pointing device such as a mouse, trackball, touchpad, or graphics tablet; a scanner; a touch screen incorporated into a display; an audio input device such as a voice recognition system; a microphone; and / or other types of input devices. In general, the use of the term "input device" is intended to include all possible types of devices and ways of inputting information into computing device 910 or a communication network.

[0099] User interface output device 920 may include a display subsystem, a printer, a fax machine, or a non-visual display such as an audio output device. The display subsystem may include a cathode ray tube (CRT), a flat panel device such as a liquid crystal display (LCD), a projection device, or other mechanisms for creating visual images. The display subsystem may also provide a non-visual display such as via an audio output device. Generally, the use of the term "output device" is intended to include all possible types of devices and ways to output information from computing device 910 to a user or another machine or computing device.

[0100] The storage subsystem 924 stores programming and data structures that provide the functionality of some or all of the modules described herein. For example, the storage subsystem 924 may include a processor that executes Figure 7 Selected aspects of pipeline 700 and implementations Figure 1 The logic of the various components described in .

[0101] These software modules are typically executed by the processor 914 alone or in combination with other processors. The memory 925 used in the storage subsystem 924 can include multiple memories, including a main random access memory (RAM) 930 for storing instructions and data during program execution and a read-only memory (ROM) 932 for storing fixed instructions. The file storage subsystem 926 can provide persistent storage for program and data files, and can include a hard drive, a floppy disk drive and associated removable media, a CD-ROM drive, an optical drive, or a removable media box. Modules that implement the functionality of certain embodiments can be stored by the file storage subsystem 926 in the storage subsystem 924, or stored in other machines accessible to the processor 914.

[0102] The bus subsystem 912 provides a mechanism for the various components and subsystems of the computing device 910 to communicate with each other as intended. Although the bus subsystem 912 is shown schematically as a single bus, alternative implementations of the bus subsystem may use multiple busses.

[0103] The computing device 910 can be of various types, including a workstation, a server, a computing cluster, a blade server, a server farm, or any other data processing system or computing device. Due to the ever-changing nature of computers and networks, Fig. 9 The description of computing device 910 depicted in FIG. 1 is intended only as a specific example for purposes of illustrating some embodiments. Many other configurations of computing device 910 may have more Fig. 9 The computing devices depicted may have greater or fewer components.

[0104] Although several embodiments have been described and illustrated herein, various other means and / or structures for performing functions and / or obtaining results and / or one or more of the advantages described herein may be utilized, and each of such changes and / or modifications is considered to be within the scope of the embodiments described herein. More generally, all parameters, dimensions, materials, and configurations described herein are intended to be exemplary, and actual parameters, dimensions, materials, and / or configurations will depend on one or more specific applications to which the teachings are applied. Those skilled in the art will recognize or be able to use no more than routine experiments to ascertain many equivalents of the specific embodiments described herein. It should therefore be understood that the above embodiments are presented only as examples, and within the scope of the appended claims and their equivalents, embodiments may be practiced in a manner different from that specifically described and claimed. The embodiments of the present disclosure are directed to each individual feature, system, article, material, kit, and / or method described herein. In addition, if such features, systems, articles, materials, kits, and / or methods are not mutually inconsistent, any combination of two or more such features, systems, articles, materials, kits, and / or methods is included within the scope of the present disclosure.

Claims

1. A method implemented using one or more processors, characterized in that: The method comprises: analyzing two or more live data streams; Based on the analysis, a developing event is newly detected and one or more entities associated with the newly detected developing event are identified, wherein the one or more entities form a portion of a general knowledge graph including a plurality of entity nodes and a plurality of edges between the plurality of entity nodes, wherein the plurality of entity nodes represent entities and the plurality of edges represent relationships between the entities; In response to newly detecting the developing event, constructing an event-specific temporary knowledge graph associated with the newly detected developing event based on the identified one or more entities, wherein the event-specific temporary knowledge graph shares one or more entity nodes with the general knowledge graph, and wherein the event-specific temporary knowledge graph includes one or more additional nodes and edges not found in the general knowledge graph, the additional nodes and edges conveying the relationship between the identified one or more entities and the developing event, wherein one of the additional nodes is created to represent the newly detected developing event; and After said building, querying said event-specific temporary knowledge graph for new information of the newly detected developing event; and One or more computing devices are caused to render the new information as output.

2. The method according to claim 1, further comprising: prior to said new detection, receiving a user query seeking information related to said developing event; as well as A superset of the live data streams is analyzed based on the user query to identify a set of the two or more live data streams as being relevant to the developing event.

3. The method according to claim 1, further comprising: receiving a user query seeking information related to the developing event; Wherein, the query includes querying the event-specific temporary knowledge graph based on the user query.

4. The method of claim 1, further comprising, prior to the analyzing, determining that the general knowledge graph does not include information responsive to a user query.

5. The method according to claim 4, wherein: The analyzing is responsive to determining that the general knowledge graph does not include information responsive to the user query.

6. The method according to claim 1, wherein: The two or more live data streams include one or more postings from eyewitnesses to the developing event.

7. The method according to any one of claims 1-6, further comprising determining an event type of the developing event based on the analysis.

8. The method of claim 7, wherein said causing comprises: determining that a given user is interested in receiving information about an event having the event type; as well as based on determining that the given user is interested, pushing data indicative of the new information to a computing device operated by the given user; Wherein the pushing causes the computing device operated by the given user to render the new information at one or more output components without the given user explicitly requesting the new information about the developing event.

9. The method according to claim 8, wherein: Determining that the given user is interested in receiving information about events having the event type is in response to receiving a query from the given user that is not answerable via the general knowledge graph but is answerable via the event-specific temporary knowledge graph.

10. The method according to claim 7, wherein: Determining the event type includes determining that the one or more additional nodes or edges not found in the general knowledge graph match an event type template associated with the event type.

11. A system comprising one or more processors and a memory storing instructions, wherein the instructions, when executed by the one or more processors, cause the one or more processors to perform operations, characterized in that: The operations include: analyzing two or more live data streams; Based on the analysis, a developing event is newly detected and one or more entities associated with the newly detected developing event are identified, wherein the one or more entities form a portion of a general knowledge graph including a plurality of entity nodes and a plurality of edges between the plurality of entity nodes, wherein the plurality of entity nodes represent entities and the plurality of edges represent relationships between the entities; In response to newly detecting the developing event, constructing an event-specific temporary knowledge graph associated with the newly detected developing event based on the identified one or more entities, wherein the event-specific temporary knowledge graph shares one or more entity nodes with the general knowledge graph, and wherein the event-specific temporary knowledge graph includes one or more additional nodes and edges not found in the general knowledge graph, the additional nodes and edges conveying the relationship between the identified one or more entities and the developing event, wherein one of the additional nodes is created to represent the newly detected developing event; and After said building, querying said event-specific temporary knowledge graph for new information of the newly detected developing event; and One or more computing devices are caused to render the new information as output.

12. The system of claim 11, wherein the operations further comprise: prior to said detection of said newly detected developing event, receiving a user query seeking information related to said developing event; as well as A superset of the live data streams is analyzed based on the user query to identify a set of the two or more live data streams as being relevant to the developing event.

13. The system of claim 11, wherein the operations further comprise: receiving a user query seeking information related to the developing event; Wherein, the query includes querying the event-specific temporary knowledge graph based on the user query.

14. The system of claim 12, wherein the operations further comprise, prior to the analyzing, determining that the general knowledge graph does not include information responsive to the user query.

15. The system of claim 14, wherein: The analyzing is responsive to determining that the general knowledge graph does not include information responsive to the user query.

16. The system according to claim 11, wherein: The two or more live data streams include one or more postings from eyewitnesses to the developing event.

17. The system of any of claims 11-16, wherein the operations further comprise determining an event type of the developing event based on the analysis.

18. The system of claim 17, wherein said causing comprises: determining that a given user is interested in receiving information about an event having the event type; as well as based on determining that the given user is interested, pushing data indicative of the new information to a computing device operated by the given user; Wherein the pushing causes the computing device operated by the given user to render the new information at one or more output components without the given user explicitly requesting the new information about the developing event.

19. The system of claim 18, wherein: Determining that the given user is interested in receiving information about events having the event type is in response to receiving a query from the given user that is not answerable via the general knowledge graph but is answerable via the event-specific temporary knowledge graph.

20. At least one non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations characterized in that: The operations include: analyzing two or more live data streams; Based on the analysis, a developing event is newly detected and one or more entities associated with the newly detected developing event are identified, wherein the one or more entities form a portion of a general knowledge graph including a plurality of entity nodes and a plurality of edges between the plurality of entity nodes, wherein the plurality of entity nodes represent entities and the plurality of edges represent relationships between the entities; In response to newly detecting the developing event, constructing an event-specific temporary knowledge graph associated with the developing event based on the identified one or more entities, wherein the event-specific temporary knowledge graph shares one or more entity nodes with the general knowledge graph, and wherein the event-specific temporary knowledge graph includes one or more additional nodes and edges not found in the general knowledge graph, the additional nodes and edges conveying the relationship between the identified one or more entities and the developing event, wherein one of the additional nodes is created to represent the newly detected developing event; and After said building, querying said event-specific temporary knowledge graph for new information of the newly detected developing event; and One or more computing devices are caused to render the new information as output.

Images