News event processing method and device, electronic equipment and storage medium
By preprocessing and physical classification of news texts, extracting and comparing news events, the problem of screening key points and hot topics in massive news is solved, real-time events are quickly extracted, and the accuracy and relevance of news matching is improved.
Patent Information
- Application Number
- CN202410110556.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-25
- Publication Date
- 2024-12-03
AI Technical Summary
In the era of big data, it is difficult to quickly select key points, hot spots and real-time events in massive news, resulting in wasted time for news material collection, and the existing technology cannot effectively improve news similarity and the accuracy of news in related fields.
By preprocessing the news text, news entities are extracted and classified, classification is performed based on location, time and person, news events are extracted, and events with matching rates greater than the target value are processed and stored as related news events.
Extract key points, hot spots and real-time events from massive news, save time for news material collection, improve the accuracy of news similarity and the accuracy of news in related fields.
Smart Images

Figure CN119066245A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of big data analysis, and in particular to a news event processing method, device, electronic device and storage medium. Background Art
[0002] With the development of economy and the continuous progress of science, the era of big data has quietly arrived. The era of big data provides opportunities for the development of news communication. In the era of big data, people can obtain diversified information through multiple channels anytime and anywhere. The characteristics of news are new, fast and accurate, but now there are endless news websites, various gossips and machine news are flooding the Internet. When people want to get the information they want, they need to spend a lot of time to screen. Summary of the invention
[0003] In view of this, the purpose of the present invention is to propose a news event processing method, device, electronic device and storage medium, which can extract key points, hot spots and real-time events from massive amounts of news, saving a lot of time in collecting news materials.
[0004] In order to achieve one of the above disclosed purposes, the present disclosure provides a news event processing method, including: obtaining a news text, and preprocessing the news text;
[0005] Extracting news entities from the preprocessed news text and classifying the news entities;
[0006] Based on the news entities, the news text is divided into a plurality of categories;
[0007] Extracting a plurality of news events from the news text of the target category;
[0008] By comparing the multiple news events, when the matching rate between the multiple news events is greater than a target value, the news events are regarded as related news events and stored in a database.
[0009] As a further improvement of an embodiment of the present disclosure, the preprocessing of the news text includes:
[0010] Translating non-Chinese news data in the news text into Chinese news data;
[0011] The news data in Traditional Chinese in the news text is converted into news data in Simplified Chinese.
[0012] As a further improvement of an embodiment of the present disclosure, extracting news entities from the preprocessed news text and classifying the news entities includes:
[0013] Use entity recognition algorithm to extract news entities from preprocessed news text;
[0014] The news entities are classified based on time, person and place.
[0015] As a further improvement of an embodiment of the present disclosure, after classifying the news entities and before classifying the news texts into multiple categories, the method further includes:
[0016] Based on the category of the place, obtaining the longitude and latitude corresponding to at least one place in the news text;
[0017] The place that appears most frequently in the news text is taken as the occurrence place of the news text, and the longitude and latitude corresponding to the occurrence place are bound to the news text.
[0018] As a further improvement of an embodiment of the present disclosure, extracting a plurality of news events from the news text of the target category includes:
[0019] comparing said news text to historical events;
[0020] In response to the news text being able to be associated with the historical event, saving the news text as a related event of the historical event;
[0021] In response to no historical event being associated with the news text, treating the news text as a new news event;
[0022] The new news event is compared with the historical news. If the historical news can be associated with the new news event, the historical news is saved as related news.
[0023] As a further improvement of an embodiment of the present disclosure, after extracting a plurality of news events from the news text of the target category and before comparing the plurality of news events, the method further includes:
[0024] Matching the news entities in the extracted multiple news events, when the overlap rate between the news entities reaches a target value, the news texts are similar news;
[0025] Conduct credibility selection for the news texts. Whenever a new news text for a news event comes in, the news text from the website with high credibility is placed first.
[0026] If the credibility of the websites is the same, the news text published earlier will be placed first.
[0027] As a further improvement of an embodiment of the present disclosure, the comparing the news event, the comparing of multiple news events, when the matching rate between the multiple news events is greater than a target value, the news event is regarded as a related news event and stored in a database, including:
[0028] Extracting a set of related entities from the related news of the multiple news events, and using the set of related entities as an entity list of the multiple news events; when extracting the entity list of the multiple news events, the entity is matched at least once;
[0029] The entity lists between the multiple news events are compared, and if the entity matching rate is greater than the target value, the multiple news events are processed as related news events and the related news events are stored in the database.
[0030] In order to achieve one of the above disclosed purposes, an embodiment of the present disclosure provides a news event processing device, including:
[0031] A preprocessing module, used for acquiring news text and preprocessing the news text;
[0032] A first extraction module, used for extracting news entities from the preprocessed news text and classifying the news entities;
[0033] A classification module, used for classifying the news text into a plurality of categories based on the news entity;
[0034] A second extraction module, used to extract multiple news events from the news text of the target category;
[0035] The comparison module is used to compare the multiple news events. When the matching rate between the multiple news events is greater than the target value, the news events are regarded as related news events and stored in the database.
[0036] In order to achieve one of the above-mentioned disclosed purposes, an embodiment of the present disclosure provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a news event processing method as described in any one of the above items is implemented.
[0037] In order to achieve one of the above-mentioned disclosed purposes, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute any of the above-mentioned news event processing methods.
[0038] Compared with the prior art, the news event processing method provided by the present disclosure extracts news entities from the news text by preprocessing the acquired news text, thereby dividing the news text into multiple categories; extracts news events from the news text of the target category, and compares the extracted multiple news events. When the matching rate between the multiple news events is greater than the target value, the news events are processed as related news events and stored in the database. The present disclosure can extract key points, hot spots and real-time events from massive news data, saving a lot of time for collecting news materials, improving the accuracy of matching news similarities and improving the accuracy of matching related news in related fields. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the drawings required for use in the implementation methods or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only implementation methods of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0040] Figure 1 A flowchart of a news event processing method provided by an embodiment of the present disclosure;
[0041] Figure 2 A flowchart of a news event processing method provided by another embodiment of the present disclosure;
[0042] Figure 3 A flowchart of a news event processing method provided by another embodiment of the present disclosure;
[0043] Figure 4 A flowchart of a news event processing method provided by another embodiment of the present disclosure;
[0044] Figure 5 A flowchart of a news event processing method provided by another embodiment of the present disclosure;
[0045] Figure 6 A schematic diagram of comparing news events in a news event processing method provided in an embodiment of the present disclosure;
[0046] Figure 7 A schematic diagram of a module of a news event processing device provided in one embodiment of the present disclosure;
[0047] Figure 8 A schematic diagram of a module of a news event processing device provided by another embodiment of the present disclosure;
[0048] Fig. 9 A schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present disclosure. DETAILED DESCRIPTION
[0049] The present disclosure will be described in detail below in conjunction with the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present disclosure, and any structural, methodological, or functional changes made by a person skilled in the art based on these embodiments are all within the scope of protection of the present disclosure.
[0050] Embodiments It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not represent any order, quantity or importance, but are only used to distinguish similar objects, and do not have to be used to describe a specific order or precedence. "Include" and "comprising" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0051] Definition of news events: For an event, there can be multiple news articles matching this event, and the matching news are called related news; events can also be extracted between the matching news, and these events that can be associated with each other are called related events.
[0052] like Figure 1 As shown, an embodiment of the present disclosure provides a news event processing method, comprising the following steps:
[0053] Step S10: obtaining news text and preprocessing the news text;
[0054] Step S20, extracting news entities from the preprocessed news text and classifying the news entities;
[0055] Step S30: dividing the news text into multiple categories based on the news entity;
[0056] Step S40, extracting multiple news events from the news text of the target category;
[0057] Step S50: compare multiple news events. When the matching rate between the multiple news events is greater than the target value, the news events are regarded as related news events and stored in the database.
[0058] The news event processing method provided by the present disclosure extracts news entities from the news text by preprocessing the acquired news text, thereby dividing the news text into multiple categories; extracts news events from the news text of the target category, and compares the extracted multiple news events. When the matching rate between the multiple news events is greater than the target value, the news event is processed as a related news event and stored in a database. The present disclosure can extract key points, hot spots and real-time events from massive news data, save a lot of time for collecting news materials, improve the accuracy of matching news similarity and improve the accuracy of matching related news in related fields.
[0059] Due to the limitation of the existing text translation level, it is impossible to accurately translate news in different languages, which may also lead to the loss of accuracy of event extraction. In one embodiment, step S10 includes:
[0060] Translate non-Chinese news data in news texts into Chinese news data;
[0061] The news data in Traditional Chinese in the news text is converted into news data in Simplified Chinese.
[0062] By obtaining various news texts from various news websites, the scheduled task scheduling platform starts the task, queries the news text to be processed, and then hands over the news data in the news text to the governance platform for data cleaning; because the news data in the news text contains languages of various countries, such as Russian, Japanese, English, French and other languages; it is necessary to translate the non-Chinese news data into Chinese news data for unified processing; in addition, if there is traditional Chinese in the news texts of Taiwan, China, Hong Kong, China, etc., it is also necessary to convert the traditional Chinese news data into simplified Chinese news data for processing, so as to facilitate the extraction of news events.
[0063] Existing text algorithms are unable to understand the recognition of "quotations from classics", and semantic recognition is not yet able to think about the semantic meaning like humans do. Articles with the same meaning may seem completely unrelated, and there may be special sentences and code words that cannot be recognized.
[0064] In one embodiment, step S20 includes:
[0065] Use entity recognition algorithm to extract news entities from preprocessed news text;
[0066] Classify news entities based on time, people, and place.
[0067] Specifically, the rules are parsed and extracted to form a rule base; the sentences in the news text are processed by word segmentation, part-of-speech tagging, named entity recognition, and dependency analysis to form a sentence dependency syntax tree; the rules in the rule base are matched on the sentence dependency syntax tree structure, and a triple is generated each time a rule is matched; the extracted triples are expanded according to the expansion rules; the triple entities and trigger words are further processed to extract the relationship, so as to extract the news entities from the preprocessed news text. When extracting news entities from news texts, you can not only use entity recognition algorithms, but also use CAN-NER algorithms and Lattice LSTM algorithms, etc., which can be selected according to your needs.
[0068] When classifying the extracted news entities, it is not limited to time, person, place, etc. The specific classification of the news entities depends on the specific content of the news text.
[0069] By extracting news entities from news texts through entity recognition algorithms, we can think about the semantic meaning, making the matching of news more accurate.
[0070] like Figure 2 As shown, in one embodiment, after step S20 and before step S30, it also includes:
[0071] Step S210: based on the category of the place, obtaining the longitude and latitude corresponding to at least one place in the news text;
[0072] Step S220: The place that appears most frequently in the news text is used as the occurrence location of the news text, and the longitude and latitude corresponding to the occurrence location are bound to the news text.
[0073] Specifically, there may be multiple locations in a news text, such as St. Petersburg, Russia, the Pacific Ocean, etc., but there is always one main location in a news text. First, the locations that appear in the news text need to be processed to obtain the longitude and latitude corresponding to these locations; during the processing, aliases may appear, and the aliases need to be converted into formal names; then, by identifying the frequency of these locations appearing in the news, the location where the news occurred is selected, and the longitude and latitude of the location where the news occurred is bound to the news to improve the accuracy of matching news.
[0074] In step S30, the categories of news classification may include: diplomacy, science and technology, culture, economy, justice, natural disasters, education and health, etc. The specific classification of a piece of news depends on the specific content of the news and is classified according to the actual situation. By classifying the news, news events can be better extracted from the news of the same category, effectively improving the relevance of news event extraction.
[0075] like Figure 3 As shown, in one embodiment, step S400 includes:
[0076] Step S410, comparing the news text with the historical events;
[0077] Step S420: In response to the news text being able to be associated with the historical event, the news text is saved as a related event of the historical event;
[0078] Step S430: In response to no historical event being associated with the news text, treating the news text as a new news event;
[0079] Step S440: Compare the new news event with the historical news. If the historical news can be associated with the new news event, the historical news is saved as related news.
[0080] It should be noted that news can correspond to multiple events, and one event can also be associated with multiple news. The key lies in the matching rate threshold between news. If the matching rate threshold is too large, there may be many events that match; if the matching rate threshold is too small, it may cause many unrelated news to be associated. In the entire process of extracting news events, it is always ensured that the news can be matched to the greatest extent. As time changes, the credibility of the extracted events is guaranteed to be the highest. Such events are of reference value. The principle of extracting events is accurate, complete, and fast to save time in news screening.
[0081] like Figure 4 As shown, in one embodiment, after step S400 and before step S500, the method further includes:
[0082] Step S450, matching the news entities in the extracted multiple news events, when the overlap rate between the news entities reaches a target value, the news texts are similar news;
[0083] Step S460: Conduct credibility selection for news texts. Whenever a new news text for a news event comes in, the news text from the website with high credibility is placed first.
[0084] Step S470: If the credibility of the websites is the same, the news text with the earliest release time is placed first.
[0085] In step S450, the target value of the overlap rate between news entities can be 75%; for example, if two news stories are about the same event, there will be multiple news entities in the two news stories. Through news entity matching, if the target value of the overlap rate of news entities in the two news stories is higher than 75%, then it can be determined that the two news stories are similar news stories. The target value of the overlap rate is not limited to 75%, but can also be 70% and 80%, etc. The specific target value of the overlap rate can be selected according to actual needs.
[0086] In steps S460-S470, since the news itself has a source, such as the Chinese government website, People's Daily and other media news; these media are official and authoritative sources of information and have the highest credibility; and some small websites' news can only be used as a reference and have very low credibility. Therefore, it is necessary to sort the credibility of the news itself. Whenever a new news is associated with an event, it is always necessary to ensure that the news from the most credible website is placed first. If there are news with the same credibility, the news with the earliest release time is placed first.
[0087] If two news stories are similar, you need to select one of them as the "direct news" of the event through credibility election. Once there is new news to match with the event, the direct news of the event and the new news are used to match the news entities; if the news entity overlap rate is higher than 75%, the new news is used as the related news of the event, and the credibility of the new news and the "direct news" is determined, and a new "direct news" is re-elected through credibility election; if new news is associated, repeat the above steps. The location, longitude and latitude, time, title, pictures and other related resources of the new event all use the attributes of the "direct news".
[0088] like Figure 5-6 As shown, in one embodiment, step S500 includes:
[0089] Step S510: extracting a set of related entities from the related news of the multiple news events, and using the set of related entities as an entity list of the multiple news events; when extracting the entity list of the multiple news events, the entity is matched at least once;
[0090] Step S520: compare the entity lists of multiple news events. If the entity matching rate is greater than the target value, the multiple news events are treated as related news events and the related news events are stored in the database. The entity matching rate may be 40%. The specific value of the entity matching rate may be determined according to the use requirements and is not specifically limited.
[0091] Specifically, the related news of news event 1 includes news A, news B and news C, and related entities 1-6 are extracted from related news AC, and entities 1-6 are formed into a related entity set to form a related entity list of news event 1; similarly, the related news of news event 2 includes news B, news C, news D, news E and news F, and related entities 1, 3, 6, 8, 10, 12, 13 are extracted from related news BF, and related entities are formed into a related entity set to form a related entity list of news event 2; the related entity lists between news event 1 and news event 2 are compared, and when the entity matching rate of the entity list is greater than 40%, news event 1 and news event 2 are related news events to each other, and then the data that news event 1 and news event 2 are related news events to each other can be stored in the database. Among them, the database can include PostgreSQL, Mysql, SQL server, Sybase and Oracle, etc.; the specific database to be used can be selected according to the needs.
[0092] In summary, the news event processing method provided by the present disclosure preprocesses the acquired news text, extracts the news entities in the news text, and divides the news text into multiple categories; extracts news events from the news text of the target category, and compares the extracted multiple news events. When the matching rate between the multiple news events is greater than the target value, the news events are processed as related news events and stored in the database. The present disclosure can extract key points, hot spots and real-time events from massive news data, save a lot of time for collecting news materials, improve the accuracy of matching news similarity and improve the accuracy of matching related news in related fields.
[0093] It should be noted that the method of the embodiment of the present disclosure can be performed by a single device, such as a computer or a server. The method of the present embodiment can also be applied in a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present disclosure, and the multiple devices will interact with each other to complete the described method.
[0094] It should be noted that the above describes some embodiments of the present disclosure. Other embodiments are within the scope of the appended claims.
[0095] Based on the same inventive concept, corresponding to any of the above implementation methods, Figure 7 As shown, the present disclosure also provides a news event processing device, including:
[0096] The preprocessing module 100 is used to obtain news text and preprocess the news text;
[0097] The first extraction module 200 is used to extract news entities from the preprocessed news text and classify the news entities;
[0098] A classification module 300, for classifying news text into multiple categories based on news entities;
[0099] The second extraction module 400 is used to extract multiple news events from the news text of the target category;
[0100] The comparison module 500 is used to compare multiple news events. When the matching rate between the multiple news events is greater than the target value, the news events are regarded as related news events and stored in the database.
[0101] In one embodiment, the pre-processing module 100 is further used to translate non-Chinese news data in the news text into Chinese news data;
[0102] The news data in Traditional Chinese in the news text is converted into news data in Simplified Chinese.
[0103] In one embodiment, the first extraction module 200 is further used to extract news entities from the preprocessed news text using an entity recognition algorithm;
[0104] Classify news entities based on time, people, and place.
[0105] In one embodiment, the news event processing device further includes:
[0106] A binding module, used for obtaining the latitude and longitude corresponding to at least one location in the news text based on the category of the location;
[0107] The place that appears most frequently in the news text is taken as the occurrence location of the news text, and the longitude and latitude corresponding to the occurrence location are bound to the news text.
[0108] In one embodiment, the second extraction module 400 is further used to compare the news text with the historical events;
[0109] In response to the news text being able to be associated with the historical event, the news text is saved as a related event of the historical event;
[0110] In response to no historical event being able to be associated with the news text, the news text is treated as a new news event; the new news event is compared with the historical news, and if the historical news can be associated with the new news event, the historical news is saved as related news.
[0111] like Figure 8 As shown, in one embodiment, the news event processing device further includes:
[0112] The matching module 410 is used to match the news entities in the extracted multiple news events. When the overlap rate between the news entities reaches a target value, the news texts are similar news.
[0113] The election module 420 is used to elect the credibility of news texts. Whenever a new news text of a news event comes in, the news text of the website with high credibility is placed first;
[0114] If the credibility of the websites is the same, the news text published earlier will be placed first.
[0115] In one embodiment, the comparison module 500 is further used to extract a set of related entities from the related news of the multiple news events, and use the set of related entities as the entity list of the multiple news events; when extracting the entity list of the multiple news events, the entity is matched at least once;
[0116] The entity lists of multiple news events are compared. If the entity matching rate is greater than the target value, the multiple news events are treated as related news events and the related news events are stored in the database.
[0117] For the convenience of description, the above device is described in various modules according to their functions. Of course, when implementing the present disclosure, the functions of each module can be implemented in the same or multiple software and / or hardware. The device of the above embodiment is used to implement the corresponding news event processing method in any of the above embodiments, and has the beneficial effects of the corresponding method implementation method, which will not be repeated here.
[0118] Based on the same inventive concept, corresponding to any of the above-mentioned implementation methods, the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the news event processing method described in any of the above implementation methods is implemented.
[0119] Fig. 9 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.
[0120] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the implementation methods of this specification.
[0121] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.
[0122] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.
[0123] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).
[0124] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).
[0125] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the implementation scheme of this specification, and does not necessarily include all the components shown in the figure.
[0126] The electronic device of the above-mentioned embodiment is used to implement the corresponding news event processing method in any of the above-mentioned embodiments, and has the beneficial effects of the corresponding method implementation method, which will not be repeated here.
[0127] Based on the same inventive concept, corresponding to any of the above-mentioned implementation methods, the present disclosure also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the news event processing method described in any of the above implementation methods.
[0128] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.
[0129] The computer instructions stored in the storage medium of the above-mentioned implementation manner are used to enable the computer to execute the news event processing method described in any of the above-mentioned implementation manners, and have the beneficial effects of the corresponding method implementation manners, which will not be repeated here.
[0130] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. This narrative style of the specification is only for the sake of clarity, and those skilled in the art should take the specification as a whole. Under the concept of the present disclosure, the above embodiments or the technical features in different embodiments may be appropriately combined, the steps may be implemented in any order, and there are many other variations of different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of simplicity.
[0131] The embodiments of the present disclosure are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made without departing from the spirit and principles of the embodiments of the present disclosure shall be included in the scope of protection of the present disclosure.
Claims
1. A news event processing method, characterized in that: include: Acquire news text, and preprocess the news text; Extracting news entities from the preprocessed news text and classifying the news entities; Based on the news entities, the news text is divided into a plurality of categories; Extracting a plurality of news events from the news text of the target category; By comparing the multiple news events, when the matching rate between the multiple news events is greater than a target value, the news events are regarded as related news events and stored in a database.
2. The news event processing method according to claim 1, characterized in that: The preprocessing of the news text includes: Translating non-Chinese news data in the news text into Chinese news data; The news data in Traditional Chinese in the news text is converted into news data in Simplified Chinese.
3. The news event processing method according to claim 1, characterized in that: The extracting of news entities from the preprocessed news text and classifying the news entities includes: Use entity recognition algorithm to extract news entities from preprocessed news text; The news entities are classified based on time, person and place.
4. The news event processing method according to claim 3 is characterized in that: After classifying the news entities and before classifying the news texts into a plurality of categories, the method further includes: Based on the category of the place, obtaining the longitude and latitude corresponding to at least one place in the news text; The place that appears most frequently in the news text is taken as the occurrence place of the news text, and the longitude and latitude corresponding to the occurrence place are bound to the news text.
5. The news event processing method according to claim 1, characterized in that: The step of extracting a plurality of news events from the news text of the target category comprises: comparing said news text with historical events; In response to the news text being able to be associated with the historical event, saving the news text as a related event of the historical event; In response to no historical event being associated with the news text, treating the news text as a new news event; The new news event is compared with the historical news. If the historical news can be associated with the new news event, the historical news is saved as related news.
6. The news event processing method according to claim 1, characterized in that: After extracting a plurality of news events from the news text of the target category and before comparing the plurality of news events, the method further includes: Matching the news entities in the extracted multiple news events, when the overlap rate between the news entities reaches a target value, the news texts are similar news; Conduct credibility selection for the news texts. Whenever a new news text for a news event comes in, the news text from the website with high credibility is placed first. If the credibility of the websites is the same, the news text published earlier will be placed first.
7. The news event processing method according to claim 1, characterized in that: The comparing of the plurality of news events, when the matching rate between the plurality of news events is greater than a target value, treating the news events as related news events and storing them in a database, comprises: Extracting a set of related entities from the related news of the multiple news events, and using the set of related entities as an entity list of the multiple news events; when extracting the entity list of the multiple news events, the entity is matched at least once; The entity lists between the multiple news events are compared, and if the entity matching rate is greater than the target value, the multiple news events are processed as related news events and the related news events are stored in the database.
8. A news event processing device, characterized in that: include: A preprocessing module, used for acquiring news text and preprocessing the news text; A first extraction module, used to extract news entities from the preprocessed news text and classify the news entities; A classification module, used for classifying the news text into a plurality of categories based on the news entity; A second extraction module, used to extract multiple news events from the news text of the target category; The comparison module is used to compare the multiple news events. When the matching rate between the multiple news events is greater than the target value, the news events are regarded as related news events and stored in the database.
9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for processing news events as described in any one of claims 1 to 7 is implemented.
10. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the news event processing method described in any one of claims 1 to 7.
