Information processing method and device
The incremental update method for event graphs addresses the inefficiency of constructing event graphs from scratch by identifying additional events and using a key-value database for efficient correlation calculations, resulting in real-time and efficient updates.
Patent Information
- Application Number
- JP2024043380
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2023-04-26
- Filing Date
- 2024-03-19
- Publication Date
- 2025-11-18
- Estimated Expiration
- 2044-03-19
AI Technical Summary
Constructing an event graph from scratch with new news data is inefficient and complex due to the increasing amount of data, leading to low construction efficiency and longer update times.
An incremental update method for event graphs that identifies additional events from existing events, updates the graph based on these additional events, and uses a key-value database for efficient correlation calculations.
This method improves the efficiency of event graph construction by updating only the changed data, allowing real-time and incremental updates, transforming the static graph into a dynamically updated one.
Smart Images

Figure 0007772120000036 
Figure 0007772120000037 
Figure 0007772120000038
Abstract
Description
[Technical Field]
[0001] FIELD OF THE DISCLOSURE Exemplary embodiments of the present disclosure relate generally to the field of computers, and more particularly to information processing methods and apparatus. [Background technology]
[0002] With the development of network and multimedia technologies, the amount of news is increasing exponentially every day. Events can be extracted from news and the relationship between different events can be expressed using an event graph. As the number of news increases every day, the number of extracted events also increases. Therefore, the event graph needs to be updated accordingly. If an event graph is constructed from scratch every time new news is acquired, using the newly acquired news along with past news, the efficiency of constructing the event graph is very low. Constructing an event graph is a complex process. As the amount of data involved in the construction increases, the difficulty of constructing the event graph also increases significantly. Therefore, a solution for efficiently constructing an event graph is desired. Summary of the Invention
[0003] In a first aspect of the present disclosure, an information processing method is provided, the method including: determining a plurality of events to be processed from one or more media contents, where at least two of the plurality of events to be processed have an event relationship; determining an additional event set from the plurality of events to be processed based on an existing event set to construct an event graph, where the additional events in the additional event set are different from existing events in the existing event set; and updating the event graph based on the additional event set.
[0004] In a second aspect of the present disclosure, an electronic device is provided, the electronic device including at least one processing circuit configured to determine a plurality of events to process from one or more media contents, at least two of the plurality of events to process having an event relationship, determine a set of additional events from the plurality of events to process based on an existing set of events to construct an event graph, the additional events in the set of additional events being different from existing events in the existing set of events, and update the event graph based on the set of additional events.
[0005] In some embodiments of the second aspect, the at least one processing circuit is further configured to determine, for a given event among the plurality of events to be processed, a correspondence between text describing the given event and text describing each existing event in the set of existing events, and identify the given event as the additional event in response to all of the correspondence correspondences being less than a threshold correspondence. In some embodiments of the second aspect, the at least one processing circuit is further configured to add an item corresponding to the given event to a document-based database for event graphs to store information related to the given event, and set an added event field in the item to a predetermined value indicating the added event.
[0006] In some implementations of the second aspect, the at least one processing circuit is further configured to cancel identifying the given event as the additional event in response to updating the event graph.
[0007] In some embodiments of the second aspect, the at least one processing circuit is further configured to store information related to the target media content among the one or more media content in the document-based database and in an item corresponding to the first existing event in response to a match between the text describing the given event and the text describing the first existing event exceeding a threshold match, and the given event is determined from the target media content.
[0008] In some embodiments of the second aspect, the at least one processing circuit further determines, for a first additional event in the set of additional events, a first text element in a text describing the first additional event, updates element occurrence frequency information stored in the key-value database and corresponding to the first text element, determines a second existing event in the set of existing events, the text describing the second existing event includes the first text element, and updates the correlation degree of the event relationship between the second existing event and a third existing event in the event graph based on the updated element occurrence frequency information.
[0009] In some embodiments of the second aspect, the at least one processing circuit is further configured to determine a first event and a second text element in the text describing the first event that have an event relationship with a first additional event in the set of existing events or the set of additional events, update element-pair occurrence frequency information stored in the key-value database that indicates that the first text element appears in relation to the second text element, and in response to the text describing a third existing event including the second text element, further update the correlation degree based on the updated element-pair occurrence frequency information.
[0010] In some embodiments of the second aspect, the at least one processing circuit further determines, for a second additional event in the set of additional events, a similarity between the second additional event and a second event in the set of additional events or the set of existing events, and, in response to the similarity exceeding a threshold similarity, stores an indication of the similarity relationship between the second additional event and the second event in the document-based database for the event graph.
[0011] In some embodiments of the second aspect, the at least one processing circuit is further configured to summarize the second additional event and an abstract event for at least the second event having a similarity relationship with the second additional event, map the second additional event and the second event to the abstract event in a matter graph, and add a node representing the abstract event to a visual representation of the matter graph.
[0012] In a third aspect of the present disclosure, there is provided an electronic device including at least one processing unit and at least one memory, the at least one memory being coupled to the at least one processing unit and configured to store instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the device to perform the method of the first aspect.
[0013] In a fourth aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program that, when executed by a processor, implements the method of the first aspect.
[0014] It should be noted that the contents described in the Summary of the Invention section do not limit the key or important features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. It should be understood that other features of the present disclosure can be more easily understood by reading the following description. [Brief explanation of the drawings]
[0015] These and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which like or similar reference numerals indicate like or similar elements.
[0016] [Figure 1] FIG. 1 is a schematic diagram illustrating an exemplary environment in which embodiments of the present disclosure may be implemented. [Figure 2] FIG. 1 is a schematic diagram illustrating an example architecture of an information processing system according to some embodiments of the present disclosure. [Figure 3] FIG. 1 is a schematic diagram illustrating an event processing flow according to some embodiments of the present disclosure. [Figure 4] FIG. 1 is a schematic diagram illustrating the runtime of incremental construction of an event graph according to some embodiments of the present disclosure. [Figure 5] 1 is a flowchart illustrating a process for information processing according to some embodiments of the present disclosure. [Figure 6]1 is a block diagram illustrating several example electronic devices embodying the present disclosure. DETAILED DESCRIPTION OF THE INVENTION
[0017] Hereinafter, embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. Although the accompanying drawings show several embodiments of the present disclosure, it should be understood that the present disclosure can be implemented in various forms and is not limited to the embodiments herein, and these embodiments are used to more thoroughly and completely understand the present disclosure. It should be understood that the accompanying drawings and embodiments of the present disclosure are used for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.
[0018] It should be noted that any section / subsection titles provided herein are not limiting. Various embodiments are described throughout this specification, and any embodiment may be included in any section / subsection. Furthermore, any embodiment described in any section / subsection may be combined in any manner with any other embodiment described in the same section / subsection and / or different sections / subsections.
[0019] In describing embodiments of the present disclosure, the term "comprises" and similar terms should be understood as broadly inclusive, i.e., "including, but not limited to." The term "based on" means "based at least in part on." "An embodiment" or "the embodiment" means "at least one embodiment." "Some embodiments" means "at least some embodiments." The following may include explicit and implicit definitions.
[0020] As used herein, the term "circuitry" refers to a hardware circuit and / or a combination of a hardware circuit and software. For example, a circuit may be a combination of analog and / or digital hardware circuitry and software / firmware. As another example, a circuit may be any portion of a hardware processor, including software, where the hardware processor includes digital signal processor(s), software, and memory(s), which work together to operate a device and perform various functions. As another example, a circuit may be a hardware circuit and / or processor, e.g., a microprocessor or portion of a microprocessor, that requires software / firmware for operation, but may not require software if it is not required for operation. As used herein, the term "circuitry" may include implementations of only a hardware circuit or processor, or a portion of a hardware circuit or processor and its (or their) accompanying software and / or firmware.
[0021] As used herein, the term "event" refers to the occurrence of some action or situation in which participants participate, or a change in an objective state. Text describing an event may include multiple words and is used to describe the occurrence of the event or some component of the event. Formally, factors contributing to an event may include trigger words or types of the event, key participants in the event, time and place of the event, etc.
[0022] The term "incremental update" as used herein means that during an update, only changed data is updated, and unchanged or updated data is not repeatedly updated, thereby saving update operation time and improving efficiency.
[0023] As used herein, the term "text" refers to any length of language. By way of example, text may refer to one or more words, phrases, parts of sentences, sentences, etc.
[0024] The term "similar event pair" refers to events that have different descriptive text but express the same meaning. For example, the events "price increase" and "price rise" are a similar event pair.
[0025] The term "word" as used herein may have any suitable granularity. For example, in one language, a "word" may include one or more words. In another language, a "word" may be a single word made up of one or more characters.
[0026] Example Environment 1 is a schematic diagram of an example environment 100 in which embodiments of the present disclosure may be implemented. In environment 100, electronic device 120 obtains one or more media content items 110-1, 110-2, ..., 110-N, collectively or singly referred to as media content items 110, where N is an integer greater than or equal to 1. Electronic device 120 may construct an event graph 130 based on media content items 110.
[0027] In environment 100, electronic device 120 may be any device with computing capabilities, including a terminal device. A terminal device may be any mobile, fixed, or portable terminal, including a mobile phone, desktop computer, laptop computer, notebook computer, netbook computer, tablet computer, media computer, multimedia computer, personal communication system (PCS) device, personal navigation device, personal digital assistant (PDA), audio / video player, digital camera / camera, locator device, television receiver, radio receiver, e-book device, gaming device, or any combination thereof, accessories and peripherals including such devices, or any combination thereof.
[0028] The media content 110 may be any suitable content capable of providing information. For example, the media content 110 may be a news report in the form of text, images, audio, video, or a combination thereof. The media content 110 may be a news report in any sector, industry, or field, such as news from the financial sector. The media content 110 may be media content obtained from various platforms (e.g., news platforms) or may be stored media content. In the case of the media content 120 in text format, the electronic device 120 may directly extract information for constructing the event graph 130 from the text. In the case of the media content 110 in the form of images, videos, audio, or the like, the electronic device 120 may use any existing or future-developed technology to extract information for constructing the event graph 130 from the images, audio, or video. For example, the electronic device 120 may directly extract relevant information from image, video, or audio formats based on image or audio recognition technology.
[0029] A matter graph is a matter-logic knowledge base that describes evolutionary rules and patterns between events. Matter graphs are used to represent events and event relationships between different events. For example, matter graphs can represent events and event relationships using logical directed graphs. Such logical directed graphs have events as nodes and event relationships as directed edges. Constructing a matter graph is a complex process. For example, the construction process may include steps such as event extraction, relationship extraction, event generalization, relationship strength calculation, and knowledge storage.
[0030] In this specification, an event relationship is also referred to as an association relationship or simply a relationship. Such event relationships may include causal relationships, conditional relationships, inversion relationships, sequential relationships, superior-subordinate relationships, compositional relationships, simultaneous relationships, and similarity relationships. For example, a causal relationship is a relationship in which the occurrence of a previous event (cause) leads to the occurrence of a later event (result). A conditional relationship indicates that a previous event is a condition for the occurrence of a later event. An inversion relationship indicates a relationship in which one event is opposed to another, for example, one event occurs later but another event occurs earlier. A sequential relationship indicates that a previous event and a later event occur consecutively in time. A superordinate-subordinate relationship indicates that one event is a superior or subordinate event of another event, and includes noun superordinate and verb superordinate relationships. For example, the events "food price rise" and "vegetable price rise" form a noun superordinate relationship, and the events "murder" and "assassination" form a verb superordinate-subordinate relationship. A compositional relationship indicates that one event is a component of another event. A simultaneous relationship indicates that one event occurs simultaneously with another event. A similarity relationship refers to an event being similar to another event to a certain extent, and establishing the similarity relationship through, for example, a similarity calculation. The above event relationships are merely examples and are not intended to limit the scope of the present disclosure.
[0031] It should be understood that the structure and functionality of environment 100 is described for illustrative purposes only and is not meant to limit the scope of the present disclosure.
[0032] As briefly mentioned above, constructing an event graph is a complex process with many intermediate results. In some scenarios, event graphs have high timeframe requirements. For example, a causal financial event graph shows the causal transmission chain between financial events and can be used to predict financial risk events in advance. Time sensitivity is extremely important in predicting financial risk events. When using an event graph for financial risk prediction, only by incorporating time-sensitive financial events into the event graph in a timely manner can risk events be predicted more rationally and accurately. Therefore, constructing a time-sensitive event graph is of great significance.
[0033] On the other hand, the amount of media content, such as news, is growing rapidly. If the process of constructing the graph cannot be done incrementally, each time the graph is updated, we must start from scratch to reconstruct the event graph using the new information together with the previous information. As the graph scale increases, the graph update time also becomes longer and longer. This may lead to very low construction efficiency of the event graph.
[0034] In view of this, it is necessary to update the event graph incrementally. How to update the event graph efficiently and incrementally is an important issue.
[0035] Therefore, an embodiment of the present disclosure provides a solution for information processing, which includes determining a plurality of events to be processed from one or more media contents, at least two of which have an event relationship; determining a set of additional events from the plurality of events to be processed based on a set of existing events for constructing an event graph; determining an additional event set from the plurality of events to be processed, where the additional events in the set of additional events are different from the existing events in the set of existing events; filtering the existing events from the events to be processed to screen out the additional events not considered in the current event graph; and updating the event graph based on the set of additional events.
[0036] Therefore, the event graph is updated based on the additional events that are different from the existing events. In this way, the event graph can be updated only based on the additional events, and the event graph can be updated incrementally. This improves the efficiency of constructing the event graph.
[0037] Matter graph update architecture 2 is a schematic diagram of an exemplary architecture 200 for information processing according to some embodiments of the present disclosure. The architecture 200 includes a media content acquisition module 211, a preprocessing module 212, an event extraction module 213, an event identification module 214, a gradual correlation calculation module 215, a gradual similarity calculation module 216, an event generalization module 217, a correlation calculation module 218, and an event graph storage module 219. The architecture 200 may further include a data storage structure 220, a preprocessing data storage base 221, a related event storage base 222, a document-based database 223, a key-value-based database 224, an abstract event pair storage base 225, and an event graph storage base 226. These modules in the architecture 200 may be implemented in the electronic device 120.
[0038] In some embodiments, multiple events (also referred to as multiple events to be processed) may be determined from one or more media contents 110. At least two of these events to be processed have an event relationship, i.e., an associative relationship. An associative event pair may be a combination of two events that have a logical relationship, such as sequential, causal, conditional, superior-subordinate, etc.
[0039] The following describes examples of events to be determined and processed in relation to the media content acquisition module 211, the pre-processing module 212, and the event extraction module 213 in the architecture 200.
[0040] The media content acquisition module 211 is configured to acquire one or more pieces of media content 110. For example, the media content acquisition module 211 may acquire media content 110 from various platforms (e.g., news platforms) or may acquire stored media content 110 from any storage medium.
[0041] In some embodiments, the media content acquisition module 211 may use crawler technology to acquire media content 110 from the platform. The media content acquisition module 211 may acquire media content from the platform periodically or at any time to maintain the time sensitivity of the media content (e.g., news data), or may acquire media content from the platform in real time.
[0042] The preprocessing module 212 is configured to preprocess the media content 110 acquired by the media content acquisition module 211. For example, preprocessing for news reading may include, but is not limited to, segmentation, clause division, special punctuation processing, word segmentation, lexical labeling, or other preprocessing methods. After preprocessing is completed, the preprocessing module 212 may store the results obtained by preprocessing the media content 110 in the preprocessing data storage base 221.
[0043] The event extraction module 213 is configured to extract events from the pre-processed media content. The extracted events are events to be processed. For example, event pairs having an associative relationship may be extracted from the pre-processed media content. The event extraction module 213 may extract event pairs having a logical relationship, such as order, causality, condition, and / or superiority / subordinateity, from the pre-processed media content.
[0044] In some embodiments, the event extraction module 213 may use a deep learning model to extract related event pairs from the preprocessed media content. The deep learning model may be trained using a corpus labeled with related event pairs. Any suitable algorithmic deep learning model may be employed. For example, a BERT model, a Bidirectional Long Short-Term Memory Network (BiLSTM), or a Conditional Random Field (CRF) model may be employed. As another example, a BERT_BiLSTM_CRF model may be employed.
[0045] In some embodiments, the event extraction module 213 may store the extracted event pairs with the association relationship in the associated event storage base 222 .
[0046] An example of an event to be processed for decision-making will be described with reference to FIG. 3 . FIG. 3 is a schematic diagram of an event processing flow 300 according to some embodiments of the present disclosure. As shown in FIG. 3 , media content 301 is “The rise in crude oil prices will result in an increase in the company's road transport business costs, and the company will promptly formulate a corresponding pricing strategy taking into account market conditions, etc.” By pre-processing and event extraction of media content 301, a pair of events having an association relationship, i.e., a “crude oil price rise” event 311 and a “rise in the company's road transport business costs” event 312, can be extracted from media content 301. Since “rise in crude oil prices” is a cause of the result “rise in the company's road transport business costs,” the two events “rise in crude oil prices” and “rise in the company's road transport business costs” are an event pair having an association relationship, specifically, a causal event pair.
[0047] Continuing with reference to Figure 2, the event identification module 214 determines a set of additional events from the plurality of events to be processed based on the set of existing events that constitute the event graph 130. The additional events in the set of additional events are different from the existing events in the set of existing events and are also referred to as additional events.
[0048] The existing event set refers to a set of existing events that have already been used to construct the matter graph. That is, the existing events have already been embodied in the matter graph. For example, the existing events may be events that were previously extracted from media content to construct the matter graph. In contrast, the additional events refer to events that have not yet been used to construct the matter graph, for example, events extracted from newly acquired media content.
[0049] The event identification module 214 may use any suitable method to compare the event to be processed with the existing events to identify additional events among the events to be processed. In some embodiments, for any event to be processed (also referred to as a given event) among the plurality of events to be processed, the event identification module 214 may determine a correspondence match between the text describing the given event and the text describing each existing event in the set of existing events. If the correspondence matches are all below a threshold match, the given event may be identified as an additional event.
[0050] In some embodiments, the degree of match is determined by comparing the character strings contained in the text describing the given event and the text describing each existing event. For example, if the number of identical characters contained in the text describing the given event and the text describing an existing event in the set of existing events is equal to or greater than a threshold (or if both texts contain the exact same character string), the degree of match between the text of the given event and the text of the existing event is determined to be high. This means that the given event is not an additional event in the event graph. If the number of identical characters contained in the text describing the given event and the text describing each existing event in the set of existing events is less than a threshold (or if the given event and any existing event do not contain the exact same character), the degree of match between the text of the given event and the text of all existing events in the set of existing events is determined to be low. This means that the given event is an additional event in the event graph.
[0051] In some embodiments, the event identification module 214 may assign a unique identifier (ID) to each extracted event (i.e., each event to be processed). The ID may be a unique identifier for each event (including the event to be processed and the existing event). The event identification module 214 may store information about the additional event in a document-based database 223 for the event graph. The document-based database 223 may include, but is not limited to, a MongoDB database or other document-based database.
[0052] The document database 223 includes items for storing information about events. Each item is used to store information about a corresponding event, and the assigned event ID may be used as an index for the item. The item may also include a field for identifying whether the corresponding event is an additional event, also referred to as an additional event field. During the construction process of the event graph, because there are many event attributes and the associated data format is complex, the present disclosure adopts the document database 223 as a data storage method, which is convenient for adding, deleting, modifying, and checking data and contributes to the gradual construction of the event graph.
[0053] In some embodiments, if a given event is identified as an add event, the event identification module 214 may add an item corresponding to the given event to the document database 223. The event identification module 214 may set an add event field in the item to a predetermined value indicating that the given event is an add event. As an example, the field "new" may be added to the item to identify the corresponding event as an add event.
[0054] After each round or batch of event graph updates, the event identification module 214 may cancel the identification of additional events. For example, the event identification module 214 may modify the additional event field of the item for the additional event in that round of updates to a value indicating it is an existing event, or may remove the additional event field from the item.
[0055] In some embodiments, if the degree of match between the text describing the given event and the text describing an existing event (also referred to as a first existing event) in the set of existing events exceeds a threshold degree of match, it means that the given event is not an additional event but the existing event. In this case, information related to the target media content may be stored in an item corresponding to the existing event in the document-based database 223. The given event is determined from the target media content. In other words, the media content used in extracting the given event may be stored in the item corresponding to the existing event as another source of the existing event.
[0056] After the additional events are identified, the event graph is then updated based on the set of additional events, thereby realizing incremental update of the event graph. The incremental update of the event graph transforms the original static event graph into a dynamically updated event graph, and media content can be constructed in the event graph in real time.
[0057] The event graph includes various information such as events, association relationships between different events, correlation degrees of the association relationships, etc. Incremental updating of the event graph may include incremental updating of one or more pieces of information.
[0058] The correlation degree of an association relationship indicates to what extent or how likely a pair of related events have that association relationship. For example, the causal strength of a causal relationship indicates how likely a cause event is to cause an effect event. In some embodiments, the correlation degree may be updated incrementally. Thus, as shown in FIG. 2 , architecture 200 may further include an incremental correlation degree calculation module 215 configured to update the correlation degree of the association relationship between the events using additional events in the additional event set.
[0059] The correlation degree can be calculated using any suitable algorithm. In some embodiments, the correlation degree may be calculated based on the frequency of occurrence of words (also called word frequency) and the frequency of occurrence of word pairs (also called word pair co-frequency) in the text describing the event. The calculation of the correlation degree will be described below using a causal relationship as an example. In the case of a causal relationship, the correlation degree is also called causal strength.
[0060] As an example, event A may cause another event B. Therefore, event A is called the cause event, and event B is called the effect event. In a causal chain, causal strength reflects the probability that a cause event leads to the occurrence of an effect event. For example, in a causal event pair (A, B), necessary causation means that cause event A must exist for effect event B to occur, and sufficient causation means that all cause events A are necessary to bring about effect event B. In calculating causal strength, we first consider the words in cause event A:
number
number
[0061] An example of calculation of the causal strength between a word pair and an event pair will be explained below. First, a word pair is calculated from the viewpoint of necessity and sufficiency.
number
number
[0062] where:
number
number
number
number
number
number
[0063] Here, the word
number
number
number
number
number
number
number
number
[0064] where:
number
number
number
number
[0065] As a result, the word pair
number
number
number
[0066] where:
number
number
[0067] TIFF0007772120000028.tif11153Here, indicates the causal strength of events A and B.
[0068] The above describes the determination of correlation using causal strength as an example. In the calculation of correlation, a relatively large number of intermediate results are generated, such as frequency information of cause words, frequency information of result words, frequency information of related word pairs, necessary causal fractions of related word pairs, sufficient causal fractions of related word pairs, and causal fractions of result word pairs. These intermediate results need to be read and written frequently.
[0069] Therefore, in some embodiments, a key-value database 224 can be used to store intermediate results of the correlation calculation. The key-value database 224 can be, for example, a Redis cache database or another key-value database. Storing data that needs to be frequently read and written in a Redis cache database or the like can improve the efficiency of data reads and writes, thereby improving the efficiency of event graph construction. Furthermore, this can easily solve the problem of memory fullness that can occur during the event graph construction process.
[0070] In this embodiment, the incremental correlation calculation module 215 incrementally updates the correlation in the event graph. For any additional event (also referred to as a first additional event) in the additional event set, the incremental correlation calculation module 215 can determine each text element (also referred to as a first text element, e.g., word) in the text describing the additional event and correspondingly update the element occurrence frequency information corresponding to the text element stored in the key-value database 224. The incremental correlation calculation module 215 can further determine an existing event (also referred to as a second existing event) in the existing event set that includes the text element, i.e., the text describing the existing event includes the text element. The incremental correlation calculation module 215 can further update the correlation of the event relationship between the existing event and one or more other existing events (also referred to as a third existing event) in the event graph based on the updated element occurrence frequency information of the text element.
[0071] As an example, an additional event "Heavy Rain at Location A" is identified in the current round of updates, which means that the frequency of occurrence of the word "heavy rain" increases. Therefore, the frequency information of the word "heavy rain" is updated in the key-value database. If there is an existing event "Heavy Rain at Location B," the word "heavy rain" is also included. Since the frequency of occurrence of the word "heavy rain" has changed, the correlation between the existing event "Heavy Rain at Location B" and other existing events that have a related relationship with it is also updated.
[0072] In some embodiments, the appearance of an additional event may also trigger an update of element pair occurrence frequency information (e.g., word pair occurrence frequency information). Therefore, the correlation degree may be further updated based on the updated element pair occurrence frequency information. Specifically, the correlation degree incremental calculation module 215 may determine an event (also referred to as a first event or a related event) that has an association relationship with an additional event (e.g., the first additional event) in the set of existing events or the set of additional events, and a text element (also referred to as a second text element) in the text describing the related event. This means that the occurrence frequency of an element pair consisting of the first text element and the second text element increases. Therefore, the correlation degree incremental calculation module 215h updates the element pair occurrence frequency information, e.g., word pair occurrence frequency information, stored in the key-value database 224, in which the first text element appears in relation to the second text element. If the text describing the third existing event also includes the second text element, the correlation degree between the second existing event and the third existing event may be further updated based on the updated element pair occurrence frequency information.
[0073] As an example, assume that the first added event in the current round of updates is "Heavy Rain at Location A" and the first text element is "Heavy Rain." The related event related to the first added event is "Flood at Location A," and the second text element in the text of the first event is "Flood." Next, the occurrence frequency information for the "Heavy Rain"-"Flood" element pair needs to be updated. If the third existing event that includes "Flood" is "Flood at Location B," the correlation between the second existing event "Heavy Rain at Location B" and the third existing event "Flood at Location B" is further updated based on the occurrence frequency information for the "Heavy Rain"-"Flood" element pair.
[0074] In this embodiment, in each update round of the event graph, additional events are searched for based only on the additional event indicators, and these additional events are used to update the word frequency and causal word pair frequency. At the same time, a key-value database is selected to store frequency information, and the frequency information of text elements and the frequency information of text element pairs are stored in the key-value database. This allows for efficient reading and writing when updating frequency information, and greatly improves the efficiency of correlation calculation.
[0075] The above describes the incremental calculation of correlation. In some embodiments, the similarity between events may be calculated incrementally. As shown in FIG. 2 , architecture 200 may further include a similarity incremental calculation module 216 configured to determine whether an additional event in the additional event set has a similar event. For any additional event in the additional event set (also referred to as a second additional event), the similarity incremental calculation module 216 may determine a similarity between the second additional event and any event (also referred to as a second event) in the additional event set or the existing event set. If the similarity exceeds a threshold similarity, an indication of the similarity relationship between the second additional event and the second event is stored in document database 223.
[0076] If the similarity between the second additional event and the second event exceeds a threshold similarity, it means that the similarity between the second additional event and the second event is high, and therefore the second additional event and the second event may be referred to as a similar event pair. Information about similar event pairs may be stored in the document-based database 223. For example, if event 1 includes three similar events, event 2, event 3, and event 4, an indication of the similarity relationship between these four events, i.e., the similarity relationship between the events, may be stored. An exemplary storage format may be {event_id:1,sim_events:[2,3,4]}, which is used to indicate that similar events of event 1 include event 2, event 3, and event 4.
[0077] In the embodiments of the present disclosure, any suitable similarity calculation method can be used. For example, Jaccard similarity, Pearson correlation coefficient, Euclidean distance similarity, cosine similarity, etc. Of course, other methods that can be used for similarity calculation may also be used.
[0078] In each round of updating the event graph, the additional events can be determined by identifying the additional events. Assume that the number of existing events in the event graph is a, and the number of additional events in this round is b. Before this round, the similarity between any two existing events has already been calculated. In the update of this round, only the similarity between two events among the b additional events and the similarity between two combinations of a existing events and b additional events need to be calculated. Therefore, the number of similarity calculations required for the event graph in the update of this round is
number
number
number
number
[0079] In some embodiments, the similarity calculations required for this round of event graph updates may use multi-process or multi-thread parallelism to further increase the efficiency of event graph updates.
[0080] The event generalization module 217 is configured to perform matter logical abstraction of events to obtain abstract events. For example, if an event has two or more similar events, its occurrence is considered to have a general nature and can be generalized. Note that, compared with abstract events, events extracted from media content are also called concrete events.
[0081] The event generalization module 217 may perform generalization based on the event similarity calculation results. In each round of incremental update, when a similar event is added to an event, the abstract event generalized by the event and its similar event may be updated. For example, for the second additional event and its one or more similar events (also referred to as second events), the event generalization module 217 may determine an abstract event to be used to summarize the second additional event and its similar events. Furthermore, the event generalization module 217 may associate (e.g., map) the second additional event and its similar events to the abstract event in the event graph. The event generalization module 217 may also add a node representing the abstract event to a visual representation of the event graph.
[0082] By way of example and without intending to be limiting, an exemplary process of event generalization will be described. For example, text describing these similar events can first be subjected to word segmentation and word property marking, and then meaningless words can be removed according to word properties. For example, words whose word property is an exclamation can be designated as meaningless words. A generalized event can then be obtained by extracting and combining common words in the text. If there is only one common word, it is determined whether the core word in the text describing the event is an object. If the core word is the common word and is not an object, the common word can be used as a generalized event, i.e., the common word can be used as text describing an abstract event. Here, the core word may be a word that captures part of the meaning of the event.
[0083] The correlation calculation module 218 is configured to calculate the correlation between abstract events. The calculation of the correlation between abstract events is similar to the calculation of the correlation between two events A and B described above, and therefore will not be repeated here.
[0084] In some embodiments, the generalization results obtained by the event generalization module 217 and the correlation results of the abstract events obtained by the correlation calculation module 218 may be stored in the abstract event pair storage base 225 .
[0085] The event graph storage module 219 is configured to store the results of the aforementioned calculations in the form of nodes, edges, and attributes in the event graph storage base 226 to form a visualization graph. For example, events may be stored in the event graph storage base 226 in the form of nodes, association relationships may be stored in the event graph storage base 226 in the form of edges, and correlation degrees may be stored in the event graph storage base 226 in the form of edge attributes. In some embodiments, the event graph storage base 226 may be a graph database such as the Neo4j graph database or other graph databases. Embodiments of the present disclosure are not limited in this respect.
[0086] An exemplary architecture for incremental event graph updates has been described above with reference to Figure 2. An event processing flow 300 will now be described with reference to the specific example of Figure 3.
[0087] 3, the event extraction module 213 can extract causally related events 311 and 312 from the media content 301. In this example, the event 311 is a cause event "rising crude oil prices," and the event 312 is an effect event "rising costs of the company's road transport business." The correlation degree incremental calculation module 215 calculates the correlation degree between the events 311 and 312, and finds that the correlation degree between the events 311 and 312 is 0.25.
[0088] The similarity progressive calculation module 216 performs event similarity calculation for each of the events 311 and 312. It can determine similar events 321 for the event 311 and similar events 322 for the event 312. In this example, the similar events 321 include names such as "natural gas price rise" and "wheat price rise," and the similar events 322 include names such as "cost of living rise" and "flour production cost rise."
[0089] Event generalization module 217 performs event generalization on event 311 and its similar events 321 to determine abstract event 331. Event generalization module 217 performs event generalization on event 312 and its similar events 322 to determine abstract event 332. In this example, abstract event 331 is "price increase" and abstract event 332 is "cost increase."
[0090] The correlation calculation module 218 calculates the abstract event correlation between the abstract event 331 and the abstract event 332. Therefore, the correlation between the abstract event 331 and the abstract event 332 is calculated to be 0.55. Thus, a gradual update of the event graph is realized. It should be understood that the specific number of events and correlations shown in FIG. 3 are merely examples and do not limit the scope of the present disclosure.
[0091] The solution of the present disclosure can not only achieve efficient incremental construction of the event graph by incrementally updating the event graph to update additional events into the original event graph, but also achieve real-time updates of the event graph.
[0092] The incremental update solution described above may also be used to construct other types of graphs. In some embodiments, a knowledge graph may be incrementally updated, where each node in the knowledge graph represents an entity. In this case, the above embodiments describing "events" may be applied to "entities." For example, one or more additional entities or additional entities that are different from the existing entities in the knowledge graph may be determined from newly obtained or newly extracted entities, and the knowledge graph may then be updated using these additional entities. In some embodiments, a causal graph may be incrementally updated, where each node in the causal graph represents an element (e.g., a cause element or an effect element). In this case, the above embodiments describing "events" may be applied to "elements." For example, one or more additional elements or additional elements that are different from the existing elements in the causal graph may be determined from newly obtained or newly extracted elements, and the causal graph may then be updated using these additional elements.
[0093] Table 1 shows a comparison of the results obtained using the incremental construction method of the present solution and the results obtained without using the incremental construction method. It should be understood that Table 1 does not take a causal event graph having causal relationships as an example. [Table 1]
[0094] As can be seen from Table 1, from the first day (1d) to the twelfth day (12d), a series of indicators indicating the construction results of the event graph all show the construction results of matching, such as the concrete event match rate, the causal event pair match rate, the similar event pair match rate, the abstract event match rate, the abstract causal event pair match rate, and the concrete-abstract event projection relationship match rate. As can be seen from this, the event graph constructed using the incremental construction method of the present disclosure is completely the same as the result generated without using the incremental construction method, which proves that the incremental construction method proposed by the present disclosure is correct.
[0095] Table 2 shows the time it takes to build an event graph using the incremental construction method of this disclosure. [Table 2]
[0096] Table 3 shows the time it takes to build an event graph without using augmented construction. [Table 3]
[0097] Tables 2 and 3 take the example of a causal event graph with causal relationships. As can be seen from the comparison of Tables 2 and 3, the solution of the present disclosure significantly reduces the consumption of event similarity calculation and total time, and improves the efficiency of event graph construction over time. As can be seen from this, with the gradual improvement, the efficiency advantage of incremental construction becomes more and more obvious as the graph scale increases.
[0098] 4 is a schematic diagram of the runtime of incremental event graph construction according to some embodiments of the present disclosure. As shown in FIG. 4, the cumulative time for event graph construction varies linearly with the number of accumulated media contents. Since the event graph needs to be reconstructed for each media content acquired in each round, the cumulative time for event graph construction progresses exponentially as the number of media contents increases. As can be seen, when updating an event graph using the same amount of samples, the incremental construction method does not increase the time consumed as the number of existing event graphs increases, significantly improving the efficiency of event graph updates.
[0099] Example process 5 is a flowchart of a process 500 for information processing according to some embodiments of the present disclosure. Process 500 may be performed by electronic device 120. For ease of discussion, process 500 will be described with reference to FIG.
[0100] In block 510, the electronic device 120 determines a plurality of events to process from the one or more media contents, wherein at least two of the plurality of events to process have an event relationship.
[0101] In block 520, the electronic device 120 determines an additional event set from the plurality of events to be processed based on the existing event set for constructing the event graph, where the additional events in the additional event set are different from the existing events in the existing event set.
[0102] In some embodiments, to determine the set of additional events, the electronic device 120 determines, for a given event in the plurality of events to be processed, a correspondence between text describing the given event and text describing each existing event in the set of existing events, and identifies the given event as an additional event in response to all correspondence correspondences being less than a threshold correspondence.
[0103] In some embodiments, to identify a given event as an additional event, the electronic device 120 adds an item corresponding to the given event to a document-based database for the event graph, stores information related to the given event, and sets an additional event field in the item to a predetermined value indicating the additional event.
[0104] In some embodiments, process 500 may further include electronic device 120 storing information related to the target media content in the one or more media content items in the document-based database and in an item corresponding to the first pre-existing event in response to a match between the text describing the given event and the text describing the first pre-existing event exceeding a threshold match, wherein the given event is determined from the target media content.
[0105] In block 530, the electronic device 120 updates the event graph based on the additional event set.
[0106] In some embodiments, the process 500 may further include the electronic device 120 canceling the identification of the given event as an additional event in response to updating the event graph.
[0107] In some embodiments, to update the event graph, the electronic device 120 may further include: for a first additional event in the set of additional events, determining a first text element in a text describing the first additional event; updating element occurrence frequency information stored in the key-value database and corresponding to the first text element; determining a second existing event in the set of existing events, wherein the text describing the second existing event includes the first text element; and updating a correlation degree of an event relationship between the second existing event and a third existing event in the event graph based on the updated element occurrence frequency information.
[0108] In some embodiments, to update the correlation degree, the electronic device 120 determines a first event and a second text element in the text describing the first event that have an event relationship with a first additional event in the set of existing events or the set of additional events, updates element-pair occurrence frequency information stored in the key-value database indicating that the first text element occurs in relation to the second text element, and, in response to the text describing a third existing event including the second text element, further updates the correlation degree based on the updated element-pair occurrence frequency information.
[0109] In some embodiments, process 500 may further include electronic device 120 determining, for a second additional event in the set of additional events, a similarity between the second additional event and a second event in the set of additional events or the set of existing events, and, in response to the similarity exceeding a threshold similarity, storing an indication of the similarity relationship between the second additional event and the second event in the document-based database for the event graph.
[0110] In some embodiments, to update the event graph, the electronic device 120 determines an abstract event to summarize the second additional event and at least a second event that has a similarity relationship with the second additional event, maps the second additional event and the second event to the abstract event in the event graph, and adds a node representing the abstract event to the visual representation of the event graph.
[0111] Example Device 6 is a block diagram of one or more example electronic devices 600 in which the present disclosure may be implemented. It should be understood that the electronic device 600 shown in FIG. 6 is merely an example and is not intended to limit in any way the functionality or scope of the examples described herein. The electronic device 600 shown in FIG. 6 may be used to implement the electronic device 120 of FIG. 1.
[0112] 6, electronic device 600 is a form of general-purpose electronic device. Components of electronic device 600 include, but are not limited to, one or more processors or processing units 610, memory 620, storage device 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660. Processing unit 610 may be a real or virtual processor and performs various processes according to programs stored in memory 620. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to increase the parallel processing capabilities of electronic device 600.
[0113] Electronic device 600 typically includes multiple computer storage media. Such media may be any available media accessible by electronic device 600, including, but not limited to, volatile and nonvolatile media, removable and non-removable media. Memory 620 may be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or a combination thereof. Storage device 630 may be removable or non-removable media, and may be machine-readable media such as a flash memory drive, a magnetic disk, or any other medium for storing information and / or data (e.g., training data for training) and accessible within electronic device 600.
[0114] The electronic device 600 may further include other removable / non-removable, volatile / non-volatile storage media. Although not shown in FIG. 6, a magnetic disk drive capable of reading from or writing to a removable, non-volatile magnetic disk (e.g., a "floppy disk") and an optical disk drive capable of reading from or writing to a removable, non-volatile optical disk may also be provided. In these configurations, each drive is connected to a bus (not shown) via one or more data medium interfaces. The memory 620 may include a computer program product 625 having one or more program modules configured to perform various methods or operations of various embodiments of the present disclosure.
[0115] The communication unit 640 can communicate with other electronic devices via a communication medium. Furthermore, the functionality of the components of the electronic device 600 can be implemented as a single computational cluster or multiple computing machines, which can communicate via a communication connection. Thus, the electronic device 600 can operate in a networked environment via logical connections to one or more other servers, network personal computers (PCs), or other network nodes.
[0116] The input device(s) 650 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device(s) 660 may be one or more output devices, such as a display, a speaker, a printer, etc. If necessary, the electronic device 600 may communicate via the communication unit 640 with one or more external devices (not shown), such as a storage device, a display device, one or more devices for user interaction with the electronic device 600, or any device (e.g., a network card, a modem, etc.) for communication between the electronic device 600 and one or more other electronic devices. Such communication may be performed via an input / output (I / O) interface (not shown).
[0117] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium is provided having computer-executable instructions stored thereon, the computer-executable instructions being executed by a processor to perform the above-described method. According to an exemplary embodiment of the present disclosure, a computer program product is further provided, the computer program product being tangibly stored on a non-transitory computer-readable medium and including computer-executable instructions, the computer-executable instructions being executed by a processor to perform the above-described method.
[0118] Aspects of the present disclosure will now be described with reference to flowchart and / or block diagrams of methods, apparatus, devices and computer program products implemented in accordance with the present disclosure, where each block of the flowchart and / or block diagrams, and combinations of blocks in the flowchart and / or block diagrams, can all be implemented by computer-readable program instructions.
[0119] These computer-readable program instructions may be provided to a processing unit of a general-purpose computer, special-purpose computer, or other programmable data processing device to produce an apparatus, and when these instructions are executed by the processing unit of the computer or other programmable data processing device, the apparatus implements the functions / operations specified in one or more blocks in the flowcharts and / or block diagrams. These computer-readable program instructions may be stored on a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other apparatus to operate in a particular manner, and a computer-readable medium having instructions stored thereon includes an article of manufacture having various instructions for implementing the functions / operations specified in one or more blocks in the flowcharts and / or block diagrams.
[0120] The computer-readable program instructions are loaded into a computer, other programmable data processing apparatus, or other device and cause the computer, other programmable data processing apparatus, or other device to execute a series of operational steps to produce a computer-implemented process, and the instructions executing on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in one or more blocks in the flowcharts and / or block diagrams.
[0121] The flowcharts and block diagrams in the accompanying drawings illustrate possible architectures, functions, and operations of systems, methods, and computer program products that may be implemented according to aspects of the present disclosure. In this regard, each block in a flowchart or block diagram represents a module, program segment, or part of an instruction, which includes one or more executable instructions for implementing a specified logical function. In some alternative embodiments, the functions marked in the blocks may occur in a different order than that shown in the accompanying drawings. For example, two consecutive blocks may be executed substantially in parallel, and depending on the function, may be executed in reverse order. Furthermore, each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or by a combination of dedicated hardware and computer instructions.
[0122] Although each aspect of the present disclosure has been described above, the above description is illustrative and not exhaustive, and is not limited to each of the disclosed aspects. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of each of the described aspects. The terms used in this specification are selected to best explain the principles, practical applications, or technical improvements in the marketplace of each aspect, or to enable those skilled in the art to understand each embodiment disclosed herein.
Claims
1. An information processing method carried out by a computer, comprising at least one processor and at least one memory, the memory being coupled to the processor, and the processor executing a program stored in the memory, comprising: determining a plurality of events to process from one or more media contents, at least two of the plurality of events to process having an event relationship; determining an additional event set from the plurality of events to be processed based on an existing event set for constructing an event graph, and the additional events in the additional event set are different from the existing events in the existing event set; updating the event graph based on the additional event set; Updating the event graph comprises: for a first additional event in the set of additional events, determining a first text element in a text describing the first additional event; updating element occurrence frequency information stored in a key-value database and corresponding to the first text element; determining a second existing event in the set of existing events, wherein text describing the second existing event includes the first text element; and updating a correlation degree of an event relationship between the second existing event and a third existing event in the event graph based on the updated element appearance frequency information; Information processing methods.
2. Determining the set of additional events includes: For a given event in the plurality of events to be processed, determining a correspondence match between text describing the given event and text describing each existing event in the set of existing events; and 2. The method of claim 1, further comprising: identifying the given event as an additional event in response to the correspondence matches all being less than a threshold match.
3. Identifying the given event as an additional event includes: adding an item corresponding to the given event to a document-based database for the event graph to store information related to the given event; and 3. The method of claim 2, further comprising setting an additional event field in the item to a predetermined value indicating an additional event.
4. 3. The method of claim 2, further comprising canceling the identification of the given event as an additional event in response to the update of the event graph.
5. 4. The method of claim 3, further comprising: in response to the degree of match between the text describing the given event and the text describing a first existing event exceeding the threshold degree of match, storing information related to a target media content among the one or more media contents in an item corresponding to the first existing event in the document-based database, wherein the given event is determined from the target media content.
6. Updating the correlation degree determining a first event having an event relationship with the first additional event in the set of existing events or the set of additional events, and a second text element in a text describing the first event; updating element-pair frequency information stored in the key-value database that indicates the first text element occurs in relation to the second text element; and 2. The method of claim 1, further comprising: in response to the text describing the third existing event including the second text element, updating the correlation degree based on updated element pair occurrence frequency information.
7. For a second additional event in the set of additional events, determining a similarity between the second additional event and a second event in the set of additional events or the set of existing events; and 2. The method of claim 1, further comprising: in response to the similarity exceeding a threshold similarity, storing an indication of the similarity relationship between the second additional event and the second event in a document-based database for the matter graph.
8. Updating the event graph comprises: determining an abstract event for summarizing the second additional event and at least the second event having a similarity relationship with the second additional event; mapping the second additional event and the second event to the abstract event in the event graph; and The method of claim 7 , further comprising adding nodes representing the abstract events to the visual representation of the event graph.
9. A method for implementing a computer-implemented system, comprising: at least one processor and at least one memory, the memory being coupled to the processor; and the processor executing a program stored in the memory, An electronic device configured to perform the method according to any one of claims 1 to 8.
Citation Information
Patent Citations
Log analysis system, log analysis method and log analysis device
JP2018022248A
Graph generation method, information processing apparatus and graph generation program
JP2019117442A
Ft diagram update device, flow diagram update device for trouble shooting, and program
JP2020160868A
Relevance analyzing device and method
JP2021103406A
Techniques for building knowledge graphs within limited knowledge domains
JP2021534493A