News event sorting method and system
By constructing corpus and calculating correlation, we automatically sort out the relationship between news events, solving the problems of event extraction and sorting in massive news, and improving efficiency and accuracy.
Patent Information
- Application Number
- CN202410109053.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-25
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-01-25
AI Technical Summary
Extracting and sorting out the problems of events and relationships in massive news, the existing technology is inefficient and requires manual judgment, resulting in slower efficiency.
By constructing a corpus, using trigger words and association relationships to identify candidate events, calculate similarity and association degrees, and automatically sort out news event association relationships to improve accuracy.
It realizes accurate identification and automated sorting of the relationship between news events, improves efficiency, reduces manual intervention, and improves the accuracy of news events.
Smart Images

Figure CN117931997B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of information management, and in particular to a method and system for sorting out news events. Background Art
[0002] With the rapid development of the Internet, online media has gradually become the main platform for the dissemination of news information, and the amount of news has also exploded. In view of the excessive abundance of news information on hot events, how to extract events and event relationships from massive news and sort out the development process of events has become an urgent problem to be solved. Summary of the invention
[0003] In view of this, the purpose of the present invention is to propose a news event sorting method and system, which solves the problem of extracting and sorting events and event relationships in massive news. By identifying and extracting event elements, identifying the correlation between events, and calculating the correlation degree, the accuracy of identifying event correlation is improved.
[0004] In order to achieve one of the above-mentioned disclosed purposes, the present disclosure provides a method for combing news events, the method comprising: obtaining news data and processing to obtain multiple news events; constructing a corpus, the corpus including the association relationship between trigger words and the news events; obtaining an event to be tested, and in response to the event to be tested including the trigger word, the event to be tested is a candidate event; calculating the similarity between the candidate events, and forming a co-occurrence event group with two candidate events whose similarity is greater than or equal to a first threshold; associating the co-occurrence event group and the corpus, and determining the pending association relationship between the candidate events in the same co-occurrence event group; based on the pending association relationship, calculating the association degree between the candidate events in the same co-occurrence event group, and when the association degree is greater than or equal to a second threshold, confirming that the pending association relationship between the candidate events is an event association relationship.
[0005] As a further improvement of an embodiment of the present disclosure, the corpus is constructed, and the corpus includes the association relationship between trigger words and the news events; including: obtaining and marking the trigger words with the association relationship, and marking the association relationship; obtaining event elements in the news events, and in response to the news events having the same event elements, the news events have an association relationship; using the pattern matching method and the association relationship of the trigger words, confirming the type of the association relationship between the news events with the association relationship.
[0006] As a further improvement of an implementation mode of the present disclosure, the method of obtaining multiple news events includes: obtaining multiple news data from multiple open source websites; checking and deleting duplicate news data to obtain pre-processed news data; and clustering the pre-processed news data based on a K-MEANS algorithm to form the news events.
[0007] As a further improvement of an embodiment of the present disclosure, the type of association between the news events with association is confirmed by utilizing the pattern matching method and the association relationship of the trigger words, including: specifying a language pattern of the association relationship of the news events; matching the language pattern and the news events to obtain a matching statement, and confirming the association relationship of the corresponding associated news events based on the association relationship of the trigger words contained in the matching statement.
[0008] As a further improvement of an embodiment of the present disclosure, after constructing the corpus, the corpus includes the association relationship between the trigger word and the news event, and before obtaining the event to be tested, it includes obtaining the trigger word and the category of the trigger word in the news event; querying the synonyms of the trigger word, or the information that the trigger word has multiple meanings; classifying the synonyms into the category of the trigger word; or classifying the first trigger word into multiple categories based on the information of multiple meanings; and constructing the trigger word list based on the trigger word and its category, and the synonyms and its category.
[0009] As a further improvement of an embodiment of the present disclosure, the acquiring of the event to be tested, and in response to the event to be tested including the trigger word, the event to be tested is a candidate event, including: comparing the trigger word with the event to be tested, if the event to be tested includes the trigger word, the event to be tested is the candidate event, and the event category of the event to be tested is the same as the category of the trigger word that matches it; if the event to be tested does not include the trigger word, the event to be tested is excluded.
[0010] As a further improvement of an implementation mode of the present disclosure, the calculating of the similarity of the plurality of candidate events, when the similarity is greater than or equal to a first threshold, the plurality of candidate events form a co-occurrence event group, includes: calculating the similarity between the first candidate event and the second candidate event, the calculation method is as follows:
[0011] S(sen1, sen2)=λSim(sen1, sen2)+(1-λ)Sim′(sen1, sen2);
[0012] Wherein, Sim(sen1, sen2) is the similarity of the candidate events calculated according to the event elements, Sim′(sen1, sen2) is the similarity of the candidate events calculated according to semantics, and λ is the weight coefficient;
[0013] The calculation method of Sim(sen1, sen2) and Sim′(sen1, sen2) is as follows:
[0014]
[0015] Among them, W i and W i ′ represents the matching weights calculated using the event elements and semantics, dis() represents the edit distance, and len() represents the character length.
[0016] As a further improvement of an implementation mode of the present disclosure, the co-occurrence event group and the corpus are associated to determine the pending association relationship between the candidate events in the same co-occurrence event group, specifically: matching the first candidate event in the co-occurrence event group and the first trigger word in the corpus; expanding the second trigger word list based on the association relationship of the first trigger word; searching the second trigger word list for the second trigger word that matches the second candidate event; and confirming that the association relationship between the first trigger word and the second trigger word is the pending association relationship.
[0017] As a further improvement of an implementation mode of the present disclosure, based on the undetermined association relationship, the association degree of the co-occurrence event group is calculated, specifically: the method for calculating the association degree is:
[0018]
[0019] in, represents the weighted parameter of event keywords, T i (R(K 1x ,K 2y )) represents the relevance of event keywords, N represents the number of entity pairs of related events in the co-occurring event group, and m and n represent the number of event entities contained in the first candidate event and the second candidate event, respectively.
[0020] Based on the same inventive concept, the present disclosure provides a news event sorting system, including:
[0021] The first acquisition module is used to acquire news data and process it to obtain multiple news events; the construction module is used to construct a corpus, and the corpus includes the association relationship between the trigger word and the news event; the second acquisition module is used to acquire the event to be tested, and in response to the trigger word being included in the event to be tested, the event to be tested is a candidate event; the first calculation module is used to calculate the similarity between the candidate events, and form two candidate events with a similarity greater than or equal to a first threshold into a co-occurrence event group; the association module is used to associate the co-occurrence event group with the corpus, and determine the pending association relationship between the candidate events in the same co-occurrence event group; the second calculation module is used to calculate the association degree between the candidate events in the same co-occurrence event group based on the pending association relationship, and when the association degree is greater than or equal to a second threshold, it is confirmed that the pending association relationship between the candidate events is an event association relationship.
[0022] Compared with the prior art, the technical effect of the present invention is that: the corpus constructed according to event elements includes trigger words and association relationships. First, candidate events are selected using trigger words, and then co-occurring event pairs are selected by calculating similarity. Finally, the correlation of the co-occurring event pairs is calculated by the type of association relationship between the co-occurring event pairs. The final correlation is obtained by constructing a corpus and calculating twice, thereby improving the accuracy of identifying event association relationships. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the drawings required for use in the implementation methods or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only implementation methods of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0024] Figure 1 A flow chart of a method for combing news events provided by an embodiment of the present disclosure;
[0025] Figure 2 A flowchart of obtaining a news event provided by another embodiment of the present disclosure;
[0026] Figure 3 A flowchart of constructing a corpus provided in another embodiment of the present disclosure;
[0027] Figure 4 A flowchart for confirming news event association relationships provided by another embodiment of the present disclosure;
[0028] Figure 5 A flowchart of constructing a trigger word list provided for another embodiment of the present disclosure;
[0029] Figure 6 A flowchart for eliminating an event to be tested provided in another embodiment of the present disclosure;
[0030] Figure 7 A flowchart for confirming a pending association relationship provided in another embodiment of the present disclosure;
[0031] Figure 8 A schematic diagram of a news event sorting system provided by an embodiment of the present disclosure;
[0032] Fig. 9 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0033] The present invention will be described in detail below in conjunction with the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional changes made by a person skilled in the art based on these embodiments are all within the scope of protection of the present invention.
[0034] Embodiments It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprising" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects.
[0035] In the prior art, the general method of extracting event association relationships is to use pattern matching and rule reasoning. The pattern matching method compares the event corpus with the manually annotated corpus and proposes the event relationships with a certain matching degree. The rule reasoning method mainly summarizes certain rules based on the collected news data, and then obtains a new relationship set through the rule reasoning engine to classify the event relationship. The existing methods all require manual judgment of the event type and then analysis of the association relationship between events, which requires a certain amount of manpower and is inefficient.
[0036] In view of this, the embodiments of the present disclosure provide a method for combing news events, which can automatically comb the event relationships of news and improve the accuracy of the combing results. Figure 1 As shown in the figure, the methods for sorting out news events include:
[0037] Step S100: Acquire news data and process to obtain multiple news events.
[0038] Specifically, news data has multiple sources and types, and there are multiple ways of expressing the same news event, or there are multiple evaluations of the same event. Through data processing, news data of various types and sources are processed to obtain news events.
[0039] In one possible implementation of the present disclosure, for step S100, Figure 2 As shown, it also includes:
[0040] Step S110, obtaining multiple news data from multiple open source websites.
[0041] Step S120, checking and deleting the duplicated news data to obtain pre-processed news data.
[0042] Specifically, news data is collected from multiple open source websites through the news collection system, and then deduplication and other processing are performed to exclude duplicate news data, reduce the amount of calculation, and improve the processing speed.
[0043] Step S130, clustering the pre-processed news data based on the K-means algorithm to form the news event.
[0044] Specifically, based on the keywords in the preprocessed news data and the K-means algorithm, news events can be clustered. The K-means algorithm is simple, fast, easy to implement, and suitable for large-scale data sets. The specific calculation process is as follows:
[0045] Step S131, confirming news text cluster d;
[0046] Step S132, calculating the local density ρ and the distance δ to determine the cluster center point p;
[0047] Step S133, calculate and compare the data object w in the news data i The Euclidean distance value to the k initial center points, and w i Divide into the event class with the smallest distance; the preprocessed news data is w = w1, w2, ..., w n .
[0048] Step S134, repeat the calculation and continuously update the event center according to the calculation results.
[0049] Step S135, calculate the objective function value until the objective function converges and no longer changes, otherwise repeat S134.
[0050] Step S200, constructing a corpus, the corpus including the association between the trigger words and the news events.
[0051] Specifically, events have related or common attributes, and through such attributes, substantial connections are established between events, thereby achieving relationship binding. The trigger words in the corpus can represent related or common attributes between events. The association between trigger words and the association between news events establish substantial connections between trigger words and news events, improving the efficiency and accuracy of event relationship extraction.
[0052] It should be noted that building a corpus is a dynamic process. News events are acquired in real time. New trigger words are acquired from news events with confirmed associations to increase the number of trigger words and associations and improve the corpus.
[0053] In one possible implementation of the present disclosure, for step S200, if Figure 3 As shown, it also includes:
[0054] Step S210, obtaining and marking trigger words with associated relationships, and marking the associated relationships.
[0055] Specifically, a large amount of Chinese text data is obtained on the Internet, and the text data includes text data with known association relationships. The association features and trigger words are extracted from the above text data. The trigger words with association features are annotated, and the trigger words are annotated according to the specific content of their association features.
[0056] In an implementation of the present disclosure, the association relationship includes a hierarchical relationship, a causal relationship, and a pre- and post-positional relationship.
[0057] The difference between the three events used in this method and the prior art is that the relationships between events in the prior art are generally four types: composition relationship, causal relationship, follow-up relationship and concurrent relationship. The concurrent relationship is excluded in this technical solution. A typical event association relationship tag library is the CEC2.0 corpus. The main difference between the event association relationship tag library in this technical solution and the CEC2.0 corpus is that there are three types of event relationships instead of four.
[0058] In a specific embodiment, the corpus in the above embodiment can be obtained by directly modifying the CEC2.0 corpus, or by constructing the corpus in a manner similar to the CEC2.0 corpus construction method.
[0059] Specifically explained as follows: The hierarchical relationship is that there are two events s1 and s2. That is, when s2 is a type of s1, s2 and s1 have a hierarchical relationship, s1 is the superordinate event of s2, and s2 is the subordinate event of s1. The hierarchical relationship indicates that there is inheritance between events, and a substantial connection between the current event nodes can be directly established. For example:<s1,s2> = <natural disaster, earthquake>, s2 and s1 are inherited, that is, s2 inherits s1. The causal relationship is that there are two events s1 and s2, and the occurrence of s2 is caused by s1, so there is a causal relationship between s1 and s2. The causal relationship can show the cause and effect of the target event. The complexity and diversity of the causal relationship determine the complexity of the causal connection. The pre- and post-position relationship is that there are two events s1 and s2. Within a period of time, s1 and s2 occur successively, and the probability of occurrence is greater than a certain threshold, then s1 and s2 are in a pre- and post-position relationship. The pre- and post-position relationship can continuously update the trend of the target event over time, and the passage of the post-position event forms an event context display.
[0060] Step S220, obtaining event elements in the news event, and in response to the news events having the same event elements, the news events have an associated relationship.
[0061] Specifically, the concatenation of event elements can characterize the occurrence process of a news event. If two events have the same or certain related entities, it can be determined that there is a certain connection between the two news events.
[0062] In an implementation of the present disclosure, event elements include keywords and entities.
[0063] Specifically, entities include time, place, and object in event elements, and keywords are words that appear in the text of news events. In a specific embodiment, five keywords are extracted from each news event. When one of the keywords in two news events is the same, it is determined that there is a certain connection between the two news events.
[0064] Step S230, using the pattern matching method and the association relationship of the trigger words, confirm the type of association relationship between the news events with association relationship.
[0065] Specifically, pattern matching is a technique for finding specific patterns or pattern sequences in text data, and is often used in natural language processing, data mining, information retrieval and other fields; pattern matching compares the input text with a predefined pattern to determine whether there is a match. The pattern can be a single character, word, phrase, sentence or more complex structure. Pattern matching algorithms usually return a match result.
[0066] The association relationship of the trigger words is matched with the news events to obtain the association relationship of the news facts. The trigger words and the news events are interconnected to obtain the association relationship of the news events.
[0067] In one possible implementation of the present disclosure, for step S230, if Figure 4 As shown, it also includes:
[0068] Step S231, specifying the language mode of the association relationship of the news event.
[0069] Step S232, matching the language pattern and the news event to obtain a matching sentence, and confirming the association relationship of the corresponding associated news event based on the association relationship of the trigger words contained in the matching sentence.
[0070] Specifically, a specific language pattern is used as a formal representation method for event relations, and the three relations of event superiority and inferiority, cause and effect, and precedence and following are abstracted into specific language patterns respectively. By matching them, other matching sentences that meet the same pattern are found, thereby obtaining the correlation relationship of news events. It is simple and easy to use, highly flexible, and has high matching efficiency and accuracy.
[0071] In an implementation of the present disclosure, between step S200 and step S300, as Figure 5 As shown, it also includes:
[0072] Step S201, obtaining a trigger word and a category of the trigger word in the news event.
[0073] Step S202, searching for synonyms of the trigger word, or information that the trigger word has multiple meanings.
[0074] Step S203: classify the synonyms into the category of the trigger words.
[0075] Step S204: classify the first trigger word into multiple categories according to the information of polysemy.
[0076] Step S205: construct the trigger word list according to the trigger words and their categories, and the synonyms and their categories.
[0077] Specifically, a trigger word list is established based on the trigger words in the news events. Because Chinese words contain many synonyms and polysemy, and usually there is more than one word representing a certain type of event, it is necessary to expand the trigger words into synonyms or near-synonyms. The trigger word list contains the categories of trigger words, and after expanding the synonyms, it also includes the categories of synonyms.
[0078] Step S300: Acquire an event to be tested, and in response to the event to be tested including a trigger word, the event to be tested is a candidate event.
[0079] Specifically, the event to be tested is one of the news events that need to be sorted out. If the event to be tested contains a trigger word, it is calculated as a candidate event. This step performs the first filtering of news events, preliminarily reducing the number of news events entering the calculation. By filtering the trigger words, some irrelevant instances are filtered out, balancing the ratio of positive and negative examples. At the same time, the expansion of the trigger words improves the recall rate and effectively avoids the loss of news events.
[0080] It should be noted that trigger words are words in the trigger word list, including original trigger words and synonyms expanded from the original trigger words.
[0081] In one possible implementation of the present disclosure, for step S300, if Figure 6 As shown, it also includes:
[0082] Step S310, comparing the trigger word with the event to be tested. If the event to be tested contains the trigger word, the event to be tested is the candidate event, and the event category of the event to be tested is the same as the category of the trigger word that matches it.
[0083] Step S320: If the event to be tested does not contain the trigger word, the event to be tested is excluded.
[0084] Specifically, the trigger word list contains the category of trigger words, the candidate events contain trigger words, and the category of the trigger words contained in the candidate events is marked. In addition, news events that do not contain trigger words are excluded and do not need to enter the next step, and the number of news events is initially filtered.
[0085] Step S400: Calculate the similarity between candidate events, and group two candidate events whose similarity is greater than or equal to a first threshold into a co-occurrence event group.
[0086] Specifically, through similarity calculation, the individual candidate times are connected and a certain threshold is set. When the similarity is greater than or equal to the first threshold, it indicates that the two candidate events may have a correlation relationship, and the two candidate events form a co-occurrence event group, and enter the next step of calculation to confirm the correlation relationship.
[0087] In an implementation of the present disclosure, step S400 further includes:
[0088] Calculate the similarity between the first candidate event and the second candidate event as follows:
[0089] S(sen1, sen2)=λSim(sen1, sen2)+(1-λ)Sim′(sen1, sen2);
[0090] Wherein, Sim(sen1, sen2) is the similarity of the candidate events calculated according to the event elements, Sim′(sen1, sen2) is the similarity of the candidate events calculated according to semantics, and λ is the weight coefficient;
[0091] The calculation method of Sim(sen1, sen2) and Sim′(sen1, sen2) is as follows:
[0092]
[0093] Among them, W i and W′ i They respectively represent the matching weights calculated using the event elements and semantics, dis() represents the edit distance, and len() represents the character length.
[0094] Step S500 , associating the co-occurrence event group with the corpus, and determining the pending association relationship between the candidate events in the same co-occurrence event group.
[0095] Specifically, the corpus includes trigger words and their associations, and news events in the same co-occurrence event group include trigger words. Based on the associations of the trigger words, pending associations between candidate events are set.
[0096] In one possible implementation of the present disclosure, for step S500, if Figure 7 As shown, it also includes:
[0097] Step S510: matching a first candidate event in the co-occurrence event group with a first trigger word in the event association library.
[0098] Step S520: Expand the second trigger word list according to the association relationship of the first trigger word.
[0099] Step S530: Search the second trigger word list for a second trigger word that matches the second candidate event.
[0100] Step S540, confirming that the association relationship between the first trigger word and the second trigger word is the pending association relationship.
[0101] Specifically, each event sentence of the candidate event in the co-occurrence event group is numbered as (d1, d2, d3...dn) to form an event list, and the two candidate events in each co-occurrence event group constitute an event pair.<di,dj> Now find the first trigger word in the tag library, that is, the position of di, and then expand the second trigger word list based on the association relationship of di, that is, the list of second trigger words that have a hyponym, causal, and pre-postordinal relationship with di, find dj in the second trigger word list, and then get the type of association relationship corresponding to di and dj to form a news event relationship<di,dj,relation> .
[0102] Step S600, based on the pending association relationship, calculating the association degree between the candidate events in the same co-occurrence event group, when the association degree is greater than or equal to a second threshold, confirming that the pending association relationship between the candidate events is an event association relationship.
[0103] Specifically, the content and pending association relationship of two candidate events in the co-occurrence event group are used as parameters to calculate the degree of association. The pending association relationship is the result of speculation or analysis based on the relevance of keywords. For example, based on the relevance between keywords, other events that users may be interested in, related content can be recommended, or whether an event is related to other events can be determined. Further confirmation is made that the association relationship between the two candidate events meets the calculation requirements. The accuracy of the news event relationship combing is guaranteed.
[0104] In an implementation of the present disclosure, step S600 further includes:
[0105] The method for calculating the correlation degree is:
[0106]
[0107] in, represents the weighted parameter of event keywords, T i (R(K1x ,K 2y )) represents the correlation of event keywords, and the correlation is used to preliminarily guess the relationship between variables; N represents the number of entity pairs of related events in the co-occurring event group, and m and n represent the number of event entities contained in the first candidate event and the second candidate event, respectively.
[0108] In summary, the news event combing method provided by the present disclosure constructs trigger words and news events with certain correlation relationships through news events with known relationships, and improves the data in the tag library in real time. Uncompiled news events are first preliminarily screened to obtain candidate times related to the tag library, and then the co-occurrence event groups are preliminarily obtained by calculating the similarity; the correlation relationship of the co-linear event groups is preliminarily determined by associating with the correlation relationship in the tag library, and then the correlation degree of the correlation relationship is calculated to determine whether the correlation degree meets the requirements, confirm the correlation relationship between several co-occurrence event groups, and achieve the purpose of combing the news event occurrence process.
[0109] Based on the same inventive concept, corresponding to any of the above-mentioned implementation methods, the present disclosure also provides a news event sorting system.
[0110] like Figure 8 As shown, a news event sorting system includes:
[0111] The first acquisition module 100 is used to acquire news data and process it to obtain multiple news events;
[0112] A construction module 200 is used to construct a corpus, wherein the corpus includes the association relationship between the trigger words and the news events;
[0113] A second acquisition module 300 is used to acquire an event to be tested, and in response to the event to be tested including the trigger word, the event to be tested is a candidate event;
[0114] A first calculation module 400 is used to calculate the similarity between the candidate events, and form a co-occurrence event group with two candidate events whose similarity is greater than or equal to a first threshold;
[0115] An association module 500 is used to associate the co-occurrence event group with the corpus and determine the pending association relationship between the candidate events in the same co-occurrence event group;
[0116] The second calculation module 600 is used to calculate the correlation between the candidate events in the same co-occurrence event group based on the pending correlation relationship, and when the correlation is greater than or equal to a second threshold, confirm that the pending correlation relationship between the candidate events is an event correlation relationship.
[0117] For the convenience of description, the above device is described by dividing it into various modules according to its functions. Of course, when implementing the present disclosure, the functions of each module can be implemented in the same or multiple software and / or hardware.
[0118] The device of the above-mentioned embodiment is used to implement the corresponding news event combing method in any of the above-mentioned embodiments, and has the beneficial effects of the corresponding method implementation method, which will not be repeated here.
[0119] Based on the same inventive concept, corresponding to any of the above-mentioned implementation methods, the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for combing news events described in any of the above implementation methods is implemented.
[0120] Fig. 9 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.
[0121] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the implementation methods of this specification.
[0122] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.
[0123] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.
[0124] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).
[0125] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).
[0126] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the implementation scheme of this specification, and does not necessarily include all the components shown in the figure.
[0127] The electronic device of the above-mentioned embodiment is used to implement the corresponding news event combing method in any of the above-mentioned embodiments, and has the beneficial effects of the corresponding method implementation method, which will not be repeated here.
[0128] Based on the same inventive concept, corresponding to any of the above-mentioned implementation methods, the present disclosure also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the news event combing method described in any of the above implementation methods.
[0129] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.
[0130] The computer instructions stored in the storage medium of the above-mentioned implementation manner are used to enable the computer to execute the news event combing method described in any of the above-mentioned implementation manners, and have the beneficial effects of the corresponding method implementation manner, which will not be repeated here.
[0131] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. This narrative style of the specification is only for the sake of clarity, and those skilled in the art should take the specification as a whole. Under the concept of the present disclosure, the above embodiments or the technical features in different embodiments may be appropriately combined, the steps may be implemented in any order, and there are many other variations of different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of simplicity.
[0132] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present disclosure difficult to understand, the known power / ground connections to the integrated circuit (IC) chips and other components may or may not be shown in the provided figures. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present disclosure difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present disclosure will be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present disclosure, it is obvious to those skilled in the art that the embodiments of the present disclosure can be implemented without these specific details or with changes in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.
[0133] Although the present disclosure has been described in conjunction with specific embodiments of the present disclosure, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.
[0134] The embodiments of the present disclosure are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made without departing from the spirit and principles of the embodiments of the present disclosure shall be included in the scope of protection of the present disclosure.
Claims
1. A method for sorting out news events, characterized in that: The method comprises: Obtain news data and process it to obtain multiple news events; Constructing a corpus, wherein the corpus includes the association relationship between the trigger words and the news events; Acquire an event to be tested, and in response to the event to be tested including the trigger word, the event to be tested is a candidate event; Calculating the similarity between the candidate events, and forming a co-occurrence event group with two candidate events whose similarity is greater than or equal to a first threshold; Associating the co-occurrence event group with the corpus, and determining the pending association relationship between the candidate events in the same co-occurrence event group; Based on the pending association relationship, calculating the association degree between the candidate events in the same co-occurrence event group, and when the association degree is greater than or equal to a second threshold, confirming that the pending association relationship between the candidate events is an event association relationship; The calculating the similarity between the candidate events and forming a co-occurrence event group from two candidate events whose similarity is greater than or equal to a first threshold comprises: Calculate the similarity between the first candidate event and the second candidate event as follows: S(sen1, sen2)=λSim(sen1, sen2)+(1-λ)Sim′(sen1, sen2); Wherein, Sim(sen1, sen2) is the similarity of the candidate events calculated based on event elements, Sim′(sen1, sen2) is the similarity of the candidate events calculated based on semantics, and λ is the weight coefficient; The calculation method of Sim(sen1, sen2) and Sim′(sen1, sen2) is as follows: Among them, W i and W i ′ respectively represent the matching weights calculated using the event elements and semantics, dis() represents the edit distance, len() represents the character length, sen1 represents the first candidate event, and sen2 represents the second candidate event.
2. The method for sorting news events according to claim 1, characterized in that: The constructing of the corpus, wherein the corpus includes the association between the trigger words and the news events; includes: Acquire and annotate trigger words with associated relationships, and annotate the associated relationships; Acquire event elements in news events, and in response to the news events having the same event elements, the news events have an associated relationship; The type of association between the news events having association is confirmed by using the pattern matching method and the association between the trigger words.
3. The method for sorting news events according to claim 1, characterized in that: The step of obtaining news data and processing to obtain multiple news events includes: Get multiple news data from multiple open source websites; Checking and deleting duplicate news data to obtain pre-processed news data; The pre-processed news data is clustered based on the K-MEANS algorithm to form the news event.
4. The method for sorting news events according to claim 2, characterized in that: The method of using the pattern matching method and the association relationship of the trigger words to determine the types of association relationships between the news events having association relationships includes: A language pattern specifying the association relationship of the news event; The language pattern and the news event are matched to obtain a matching sentence, and the association relationship of the corresponding associated news event is confirmed based on the association relationship of the trigger words contained in the matching sentence.
5. The method for sorting news events according to claim 1, characterized in that: After constructing the corpus, wherein the corpus includes the association between the trigger words and the news events, and before obtaining the event to be tested, the following steps are included: Acquire trigger words and categories of the trigger words in the news event; Searching for synonyms of the trigger word, or information that the trigger word has multiple meanings; assigning the synonym to the category of the trigger word; or According to the information of polysemy of a word, the trigger word is classified into multiple categories; The trigger word list is constructed according to the trigger words and their categories, and the synonyms and their categories.
6. The method for sorting news events according to claim 1, characterized in that: The acquiring of the event to be tested, and in response to the event to be tested including the trigger word, the event to be tested being a candidate event, includes: Comparing the trigger word with the event to be tested, if the event to be tested contains the trigger word, the event to be tested is the candidate event, and the event category of the event to be tested is the same as the category of the trigger word matched with it; If the event to be tested does not contain the trigger word, the event to be tested is excluded.
7. The method for sorting news events according to claim 1, characterized in that: Associating the co-occurrence event group with the corpus and determining the pending association relationship between candidate events in the same co-occurrence event group is specifically as follows: Matching a first candidate event in the co-occurrence event group with a first trigger word in the corpus; Expanding a second trigger word list according to the association relationship of the first trigger word; Searching the second trigger word list for a second trigger word that matches the second candidate event; Confirm that the association relationship between the first trigger word and the second trigger word is the pending association relationship.
8. The method for sorting out news events according to claim 1, wherein the features include: Based on the pending association relationship, the association degree of the co-occurrence event group is calculated, specifically: The method for calculating the correlation degree is: in, represents the weighted parameter of event keywords, T i (R(K 1x ,K 2y )) represents the relevance of event keywords, N represents the number of entity pairs of related events in the co-occurring event group, and m and n represent the number of event entities contained in the first candidate event and the second candidate event, respectively.
9. A news event sorting system, characterized in that: include: The first acquisition module is used to acquire news data and process to obtain multiple news events; A construction module, used to construct a corpus, wherein the corpus includes the association relationship between the trigger words and the news events; A second acquisition module is used to acquire an event to be tested, and in response to the event to be tested including the trigger word, the event to be tested is a candidate event; A first calculation module, configured to calculate the similarity between the candidate events, and group two candidate events whose similarity is greater than or equal to a first threshold into a co-occurrence event group; An association module, used to associate the co-occurrence event group with the corpus, and determine the pending association relationship between the candidate events in the same co-occurrence event group; A second calculation module is used to calculate the correlation between the candidate events in the same co-occurrence event group based on the pending correlation relationship, and when the correlation is greater than or equal to a second threshold, confirm that the pending correlation between the candidate events is an event correlation relationship; The calculating the similarity between the candidate events and forming a co-occurrence event group from two candidate events whose similarity is greater than or equal to a first threshold comprises: Calculate the similarity between the first candidate event and the second candidate event as follows: S(sen1, sen2)=λSim(sen1, sen2)+(1-λ)Sim′(sen1, sen2); Wherein, Sim(sen1, sen2) is the similarity of the candidate events calculated based on event elements, Sim′(sen1, sen2) is the similarity of the candidate events calculated based on semantics, and λ is the weight coefficient; The calculation method of Sim(sen1, sen2) and Sim′(sen1, sen2) is as follows: Among them, W i and W′ i They respectively represent the matching weights calculated using the event elements and semantics, dis() represents the edit distance, len() represents the character length, sen1 represents the first candidate event, and sen2 represents the second candidate event.
Citation Information
Patent Citations
Event graph construction method based on social media
CN108763333A
Method for extracting political outbound news events of Internet
CN110941692A
Event-oriented dynamic knowledge graph construction method and device
CN111143576A
Event association graph construction method and device for news data
CN113282703A
Military news long text hierarchical event extraction method
CN113657090A
