System and method for computing dynamic relationships of entities

Abstract

The application discloses a system and method for calculating dynamic relations of entities, comprising an entity recognition module, a calculation module and a writing and pushing module, wherein the entity recognition module comprises a recognition processing unit, an extraction unit and a standardization and screening unit; the processing unit is used for processing personal names and institutional names; the extraction unit is used for splicing the title and the body of news and inputting into a model; the standardization and screening unit is used for standardizing the input personal names and institutional names by using a regular expression; according to the existing entity recognition technology, the news subject is recognized from the complicated news body by using a combination of a pre-training model and a traditional model, the change trend of the recognized entity relations with the news heat is analyzed, the real-time dynamic relations between the entities are established, and the system can be applied to future search word library expansion.

Description

Technical Field

[0001] This invention belongs to the field of dynamic algorithm processing, and in particular relates to a system and method for calculating dynamic relationships between entities. Background Technology

[0002] In this era of rapid development of computers and the internet, social media is rising in popularity, and users are becoming increasingly reliant on it. Different users have different needs for news information, resulting in varying levels of attention from different user groups. Macro-level statistics show that some news information is frequently accessed, indicating high user attention, while other news information is accessed very infrequently, indicating low user attention. Currently, the number of times news information is viewed can be used as a popularity metric to quantify the degree of user attention to news information. However, the changing trends in the entity relationships that determine the changes in news popularity are currently unpredictable. Therefore, we propose an improvement, outlining a system and method for calculating dynamic entity relationships. Summary of the Invention

[0003] The purpose of this invention is to overcome the above-mentioned problems in the existing technology and provide a system and method for calculating the dynamic relationship between entities. Based on existing entity recognition technology, including the use of a combination of pre-trained models and traditional models, the invention identifies the main body of news from complex news texts, analyzes the changing trend of the identified entity relationship with the news popularity, and establishes real-time dynamic relationship between entities, which can be applied to the future expansion of search term databases.

[0004] To achieve the above-mentioned technical objectives and effects, the present invention is implemented through the following technical solution:

[0005] A system and method for calculating dynamic relationships between entities, including an entity recognition module, a construction and calculation module, and a writing and pushing module;

[0006] The entity recognition module includes a recognition processing unit, an extraction unit, and a standardization filtering unit. The processing unit is used to process personal and organizational entities. The extraction unit is used to concatenate the headline and body text in the news and feed them into the model, maintaining a fixed truncation value for the input text. The standardization filtering unit is used to standardize the input personal and organizational entities using regular expressions.

[0007] The computational module includes a statistics unit, a processing and training unit, a processing and filtering unit, a popularity calculation unit, a data acquisition unit, and an extraction unit. The statistics unit inputs standardized personal and organizational names and entities, and counts the frequency of each entity in the text. The processing and training unit is used for more stringent extraction of geographical entities, ultimately aiming to output the main geographical location of the text. The processing and filtering unit applies different strategies to the headlines and body text of the input news articles. For the headlines, a keyword retrieval strategy is used, with keywords based on the national administrative planning dictionary. After extracting geographical terms from the headlines, corresponding administrative numbers are converted, and frequency statistics are performed. Entities that do not conform to the standards are removed. The degree calculation unit is used to remove regional information in the body text based on the most important regional information in the title, and to count the most district and county-level regional information in the subdivided regions, retaining the main regional mentions. The data collection unit is used to collect data from six major platforms including WeChat, Weibo, news clients, web pages, forums and newspapers. The extraction unit is used to call the Ner model to extract the names of people, organizations and regional entities in hot news, and to count the frequency of entities in all aggregated data for each event. The same entity appears once or multiple times in a news article, only counting it once. The heat thresholds for the three types of entities in the data are set, and only entities with heat thresholds are retained. The frequency of the entities is also retained for subsequent heat value calculation.

[0008] The write-and-push module is used to write qualified entity data into a file and then push it.

[0009] Furthermore, the processing and filtering unit employs the following strategy for the news text: using a regional model to extract regional entities, marking the locations of the extracted entities with asterisks, then using a keyword model to search the remaining text, filtering all identified regions, and searching for corresponding administrative numbers according to the administrative planning dictionary. The filtering methods mainly include constraining regional words through the administrative planning dictionary, querying the information database for entity words containing regional information, converting the encoding, and if the information is not found in the information database, then querying through Gaode Maps and Baidu Maps to obtain the regional information, converting the administrative number, and storing it in the information database, and removing entities that do not conform to the specifications.

[0010] A method for calculating dynamic relationships between entities:

[0011] A. The processing unit processes the entities of people and organizations. The extraction unit concatenates the headlines and body text in the news and feeds them into the model. The input text maintains a fixed truncation value. The standardization and filtering unit uses regular expressions to standardize the input entities of people and organizations.

[0012] B. The statistics unit inputs standardized personal and organizational names and entities, and counts the frequency of each entity in the text. The processing training unit is more rigorous in extracting regional entities, and its ultimate goal is to output the main geographical location of the text. The processing filtering unit adopts different strategies for the headlines and body text of the input news. The headline uses a keyword retrieval strategy, with keywords based on the national administrative planning dictionary. After extracting regional words from the headline, the corresponding administrative numbers are converted and frequency statistics are performed. Entities that do not conform to the standards are removed. The popularity calculation unit uses the most important regional information in the headline as a guide, removes regional information in the body text, counts the most district and county-level regional information in the subdivided regions, and retains the main regional mentions.

[0013] C. The data collection unit collects data from six major platforms, including WeChat, Weibo, news clients, web pages, forums, and newspapers. The extraction unit calls the Ner model to extract the names of people, organizations, and geographical entities in the hot news. It also performs entity frequency statistics on all aggregated data for each event. If the same entity appears once or multiple times in a news article, it is only counted as once. The heat thresholds for the three types of entities in the data are specified, and only entities with heat thresholds are retained. The frequency of the entities is also retained for subsequent heat value calculation.

[0014] D. The push module writes the qualified entity data into a file and pushes it. The beneficial effects of this invention are: This system and method for calculating the dynamic relationship of entities, based on existing entity recognition technology, including the use of a combination of pre-trained models and traditional models, identifies the main body of the news from the complex news text, analyzes the changing trend of the identified entity relationship with the change of news popularity, establishes the real-time dynamic relationship between entities, and can be applied to the future expansion of the search term library. Attached Figure Description

[0015] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0016] Figure 1 This is a flowchart of the present invention;

[0017] Figure 2 This is a schematic diagram of the identification and processing unit of the present invention;

[0018] Figure 3 This is a partial structural schematic diagram of the present invention;

[0019] Figure 4 This is a flowchart of the steps of the present invention;

[0020] Figure 5 This is a flowchart of the steps of the present invention;

[0021] Figure 6 This is a flowchart of the steps of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] like Figure 1 The system and method for calculating dynamic relationships between entities shown include an entity recognition module that extracts personal name entities, organization name entities, and geographical location entities from text; a construction and calculation module that constructs and calculates entity relationship heat values; and a writing and push module.

[0024] The entity recognition module includes a recognition processing unit, an extraction unit, and a standardization filtering unit. The processing unit is used to process the training set of news texts published and marked by the People's Daily in 1998, and to train and build a Ner entity recognition model with a BILSTM+CRF structure to process personal and organizational entities. The extraction unit is used to concatenate the title and body text in the news and feed them into the model, while maintaining a fixed truncation value for the input text. The standardization filtering unit is used to standardize the input personal and organizational entities using regular expressions.

[0025] The computational module comprises a statistics unit, a processing and training unit, a processing and filtering unit, a popularity calculation unit, a data acquisition unit, and an extraction unit. The statistics unit inputs standardized personal and organizational names and entities, and counts the frequency of each entity's appearance in the text. The processing and training unit is used for more stringent extraction of geographical entities, ultimately aiming to output the main geographical location of the text. The processing and filtering unit employs different strategies for the headlines and body text of the input news articles. For the headlines, a keyword retrieval strategy is used, with keywords derived from a nationally published administrative planning dictionary. After extracting geographical terms from the headlines, corresponding administrative numbers are converted, and frequency statistics are performed. Entities that do not conform to the standards are removed. The popularity calculation... The calculation unit is used to remove regional information from the body text based on the most important regional information in the title, and to count the most district and county-level regional information in the subdivided regions, retaining the main regional mentions. The data collection unit is used to collect data from six major platforms including WeChat, Weibo, news clients, web pages, forums and newspapers. The extraction unit is used to call the Ner model to extract the names of people, organizations and regional entities in hot news, and to count the frequency of entities in all aggregated data for each event. If the same entity appears once or multiple times in a news article, it is only counted as once. The heat thresholds for the three types of entities in the data are specified, and only entities with heat thresholds are retained. The frequency of the entities is also retained for subsequent heat value calculation.

[0026] The write-to-push module is used to write eligible entity data to a file and then push it.

[0027] Furthermore, the processing and filtering unit employs a strategy of using a regional model to extract regional entities from the news text, marking the locations of the extracted entities with asterisks, and then using a keyword model to search the remaining text. All identified regions are then filtered, and the corresponding administrative numbers are searched according to the administrative planning dictionary. The filtering methods mainly include constraining regional terms through the administrative planning dictionary, querying the information database for entity terms containing regional information, converting the codes, and if the information is not found in the information database, then querying through Gaode Maps and Baidu Maps to obtain the regional information, converting the administrative numbers, and storing them in the information database. Entities that do not conform to the specifications are removed.

[0028] A method for calculating dynamic relationships between entities:

[0029] A. The processing unit processes the entities of people and organizations. The extraction unit concatenates the headlines and body text in the news and feeds them into the model. The input text maintains a fixed truncation value. The standardization and filtering unit uses regular expressions to standardize the input entities of people and organizations.

[0030] B. The statistics unit inputs standardized personal and organizational names and entities, and counts the frequency of each entity in the text. The processing training unit is more rigorous in extracting regional entities, and its ultimate goal is to output the main geographical location of the text. The processing filtering unit adopts different strategies for the headlines and body text of the input news. The headline uses a keyword retrieval strategy, with keywords based on the national administrative planning dictionary. After extracting regional words from the headline, the corresponding administrative numbers are converted and frequency statistics are performed. Entities that do not conform to the standards are removed. The popularity calculation unit uses the most important regional information in the headline as a guide, removes regional information in the body text, counts the most district and county-level regional information in the subdivided regions, and retains the main regional mentions.

[0031] C. The data collection unit collects data from six major platforms, including WeChat, Weibo, news clients, web pages, forums, and newspapers. The extraction unit calls the Ner model to extract the names of people, organizations, and geographical entities in the hot news. It also performs entity frequency statistics on all aggregated data for each event. If the same entity appears once or multiple times in a news article, it is only counted as once. The heat thresholds for the three types of entities in the data are specified, and only entities with heat thresholds are retained. The frequency of the entities is also retained for subsequent heat value calculation.

[0032] D. The push module writes the qualified entity data into a file and pushes it. The beneficial effects of this invention are: This system and method for calculating the dynamic relationship of entities, based on existing entity recognition technology, including the use of a combination of pre-trained models and traditional models, identifies the main body of the news from the complex news text, analyzes the changing trend of the identified entity relationship with the change of news popularity, establishes the real-time dynamic relationship between entities, and can be applied to the future expansion of the search term library.

[0033] like Figure 2 As shown, the recognition and processing unit concatenates the headlines and body text in the news and feeds them into the model. It maintains a fixed truncation value for the input text. If the text length exceeds the truncation value, it searches for symbols before truncation, fills in the text after truncation, and finally outputs the entities of people's names and organizations in the text.

[0034] like Figure 3 , Figure 4 , Figure 5 and Figure 6 As shown, the entire entity relationship mining process includes three parts: 1. Entity identification, which involves extracting personal names, organization names, and geographical locations from the text; 2. Constructing and calculating entity relationship popularity values; and 3. Writing and pushing data.

[0035] It consists of twelve steps:

[0036] 1. Prepare to train a personal name and organization name entity recognition model, process the news text published and tagged by the People's Daily in 1998 as training set, and use it to train and build a Ner entity recognition model with a BILSTM+CRF structure to process personal name and organization name entities;

[0037] 2. Extract real names of persons and organizations using the model;

[0038] 3. Standardize the entities of personal names and organizations. Use regular expressions to standardize the input entities of personal names and organizations. For example, "Aunt Wang" etc. obviously does not meet the requirements for personal names. Irrational personal names should be removed.

[0039] 4. Calculate entity frequency: Input standardized personal and organizational names as entities, and count the number of times each entity appears in the text;

[0040] 5. Prepare to train a geographic entity recognition model. The extraction of geographic entities is relatively strict, and the ultimate goal is to output the main geographic location of the text. The geographic entity extraction and location classification process is similar to the entity relationship mining process. Figure 2 The right half shows the Ner model with a BILSTM+CRF structure, trained and constructed using news text published and tagged by the People's Daily in 1998, for processing geographic entities;

[0041] 6. For news headlines, different strategies are adopted for the headlines and body text of the input news. A keyword search strategy is used for the headlines. The keywords are based on the national administrative planning dictionary. After extracting the regional words in the headlines, they are converted into the corresponding administrative numbers and frequency statistics are performed.

[0042] 7. For the processing of the main text, the strategy adopted for the news text is to use the Ner regional model to extract regional entities. The location of the entities extracted by Ner is masked with *. Then, a keyword model is used to search the remaining text. All identified regions are filtered, and the corresponding administrative numbers are found according to the administrative planning dictionary. The filtering method mainly involves constraining regional words by the administrative planning dictionary. For entity words containing regional information (including some regional institutions), the information is queried in the Redis database, and the encoding is converted. If it is not found in the Redis database, the regional information is obtained through the APIs of Gaode Maps and Baidu Maps, the administrative number is converted, and it is stored in the Redis database. Entities that do not conform to the specifications are removed.

[0043] 8. For the processing of personal names, organization names, and geographical entities, the most important geographical information in the title is used as a guide to remove geographical information in the body text. The most district and county-level geographical information in the subdivided regions is counted, and the main geographical mentions are retained. Finally, the two parts of entities are merged to obtain entity data. Next, as shown in the second half of the flowchart, the popularity value is constructed.

[0044] 9. Source data collection for calculating entity relationships includes data from six major platforms: WeChat, Weibo, news apps, web pages, forums, and newspapers. A dynamic text matrix calculation method based on synonym conversion is adopted. After multiple online iterations and optimizations, it can quickly achieve similar text clustering while ensuring classification accuracy. Similar news is aggregated to form events, and a popularity value is generated based on the number of news articles. The top 100 news articles are taken as the original event text, and a similar article interface is called to aggregate as much news data as possible for each event within a certain time based on text similarity, and this data is used as the original text data.

[0045] 10. Extract three types of entity information from the text, call the Ner model to extract the names of people, organizations and regions in the hot news, and count the entity frequency of all aggregated data for each event. The same entity is counted only once if it appears once or multiple times in a news article.

[0046] 11. Extract key entity information and establish popularity thresholds for three types of entities in the data. Only entities with popularity thresholds above the thresholds are retained, and the frequency of each entity is also retained for subsequent popularity value calculations. The thresholds are determined as follows: 10% of the aggregated news volume of an event is used. If an entity appears more than this threshold in the event, it is retained; otherwise, it is discarded.

[0047] 12. Push: Write the qualified entity data into a file and push it. The construction of dynamic entity relationships mainly relies on the dynamic calculation of the frequency of entity association in the hot news of the day. Using the association frequency as the value and the day as the cycle, the dynamic display of the changing trend between entities as the news situation develops.

[0048] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0049] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A system for calculating dynamic relationships between entities, characterized in that, It includes an entity recognition module, a construction and calculation module, and a writing and pushing module; The entity recognition module includes a recognition processing unit, an extraction unit, and a standardization filtering unit. The processing unit is used to process personal and organizational entities. The extraction unit is used to concatenate the headline and body text in the news and feed them into the model, maintaining a fixed truncation value for the input text. The standardization filtering unit is used to standardize the input personal and organizational entities using regular expressions. The computational module includes a statistics unit, a processing and training unit, a processing and filtering unit, a popularity calculation unit, a data acquisition unit, and an extraction unit. The statistics unit inputs standardized personal and organizational names and entities, and counts the frequency of each entity's appearance in the text. The processing and training unit extracts geographical entities, ultimately outputting the main geographical location of the text. It trains and constructs a Ner model with a BILSTM+CRF structure using news text to process geographical entities. The processing and filtering unit applies different strategies to the headline and body of the input news article. For the headline, it uses a keyword retrieval strategy, extracts geographical terms, converts them into corresponding administrative numbers, and performs frequency statistics. Entities that do not conform to the standards are filtered out. The entity elimination process involves the following steps: The heat calculation unit uses the most important regional information in the title as a guide to eliminate regional information in the body text, counts the most district and county-level regional information in the subdivided regions, and retains the main regional mentions; The data collection unit collects data from news clients, web pages, forums, and newspaper platforms; The extraction unit calls the Ner model to extract the names of people, organizations, and regional entities in the hot news, and performs entity frequency statistics on all aggregated data for each event. The same entity is counted only once if it appears once or multiple times in a news article. Heat thresholds for the three types of entities in the data are set, and only entities with heat thresholds are retained. The frequency of the entities is also retained for subsequent heat value calculation. The write-and-push module is used to write qualified entity data into a file and push it. The standardized screening unit employs the following strategy for news text: using a regional model to extract regional entities, marking the locations of extracted entities with asterisks, then using a keyword model to search the remaining text, filtering all identified regions, and searching for corresponding administrative numbers according to the administrative planning dictionary. The screening methods include constraining regional terms through the administrative planning dictionary, querying the information database for entity terms containing regional information, converting the code, and if the information is not found in the information database, obtaining the regional information through a map query, converting the administrative number, and storing it in the information database. Entities that do not conform to the standard are removed.

Images