An active data collection method based on semantic extension

Abstract

This invention aims to propose a proactive data collection method based on semantic expansion. Initial keywords describing the topic are set. The keywords are then used to identify corresponding initial entities in the semantic web. These initial entities are then expanded using upward and downward expansion methods to obtain several new entities. A classification model is then constructed and used to predict the probability of each new entity being recommended as a relevant entity. The relevance weight of each new entity is calculated and ranked from high to low. The new entities with the highest scores are selected as relevant entities. This process is repeated to obtain more relevant entities describing the topic and their scores. This method can effectively semantically expand the initial keywords.

Description

Technical Field

[0001] The present invention relates to the fields of web crawlers and semantic analysis, and in particular to an active data collection method based on semantic extension. Background Art

[0002] The advent of the big data era has dramatically increased the value of data. According to statistical reports, the total amount of data in 2020 exceeded 44ZB, and based on current growth trends, 175ZB of data will be generated by 2025. Faced with such a massive amount of data, users are increasingly concerned about data availability—that is, how to obtain the data of interest.

[0003] Web crawlers play a vital role in collecting web data and are a crucial component of vertical search engines. Web crawlers can be categorized by their characteristics as general crawlers, prioritized crawlers, and incremental crawlers. General crawlers retrieve all web pages they encounter, regardless of domain or topic. Prioritized crawlers crawl web pages based on user-specified criteria. Incremental crawlers can crawl constantly changing web pages, keeping the index database up to date.

[0004] A topic crawler is a type of priority crawler. It requires users to provide keywords or page URLs related to the topic and only crawls content related to the topic. Traditional topic crawlers focus primarily on how to retrieve pages, including crawling strategies and relevance measurement. These studies limit the description of topics to fixed keywords, which often fail to accurately and comprehensively describe the topic. This can result in the retrieval of many pages that are irrelevant to the topic or the failure to retrieve many relevant pages.

[0005] Methods based on conceptual semantics can effectively circumvent this problem. Constructing a domain ontology is one of the most commonly used approaches based on conceptual semantics. An ontology is a clear, formalized specification of a shared conceptual model. Its goal is to capture relevant domain knowledge, provide a shared understanding of that knowledge, identify commonly recognized vocabulary within the domain, and clearly define these vocabularies and the relationships between them at different levels of formalization. While ontologies can effectively describe topics, constructing them often requires extensive domain knowledge and even the involvement of domain experts. Simply using existing domain ontologies cannot meet diverse research needs and is therefore limited in scope.

[0006] Query expansion is an important research area in the search engine field. This involves intelligently analyzing and semantically expanding user-supplied query terms or sentences to address diverse user queries. Query expansion methods can also be incorporated into topic descriptions in topic crawlers, effectively avoiding the many issues that arise when using only keywords to describe topics. Some researchers have employed search engine-based expansion. This involves initially entering user-supplied topic keywords into a search engine, searching the top N most relevant pages. The TF-IDF algorithm is then used to obtain the keywords and weights for each page. These N keyword weights are then combined, and the highest-weighted keywords are extracted as new keywords to supplement the initial keywords, further improving the topic description. This work is based on the assumption that words that frequently appear within the context of a given keyword in the search engine database are relevant to the topic. However, the TF-IDF method requires a large corpus, and the algorithm's performance is directly correlated with the quality of the corpus. If the corpus is not selected properly, irrelevant terms can be easily expanded, leading to a phenomenon known as "topic drift." Some researchers have improved upon this, replacing the TF-IDF algorithm with the corpus-free TextRank keyword extraction algorithm. To minimize the expansion of irrelevant keywords, this work also introduced BERT pre-trained word vectors, which are used to convert the expanded and initial keywords into a vector space. The topic relevance is measured by calculating the vector space distance, and this is used as a weight for selecting expanded keywords. However, training the BERT model still requires corpus support. If the corpus is not selected properly, it will be impossible to distinguish the topic similarity between different words. Furthermore, selecting the first few pages of search engine results for analysis is easily influenced by popular articles. Summary of the Invention

[0007] In view of the shortcomings of the existing technology, the present invention provides an active data collection method based on semantic extension. The specific technical solution is as follows:

[0008] An active data collection method based on semantic extension includes the following steps:

[0009] Step 1: Set the initial keywords to describe the topic;

[0010] Step 2: Use the initial keywords set in step 1 to identify the corresponding initial entities in the semantic web, and ensure that each keyword corresponds to a unique initial entity;

[0011] Step 3: Expand each initial entity identified in step 2 using the upward expansion and downward expansion methods, and obtain several corresponding new entities for each initial entity;

[0012] Step 4: Build and use a classification model to predict the probability of each new entity being recommended as a related entity. Then, based on the probability and the number of times the new entity appears, calculate the relevance weight of each new entity, and sort the relevance weights from high to low. Select the new entities with the highest scores as related entities.

[0013] Step 5: Repeat steps 3 to 4 to obtain more relevant entities and their scores that describe the topic, making the topic description more complete.

[0014] Furthermore, the step 2 is specifically implemented through the following sub-steps:

[0015] (1) calling the interface provided by the semantic web to obtain a list of candidate entities corresponding to the initial keywords sorted by relevance;

[0016] (2) Find the superordinate entity and subordinate entity of each candidate entity, and sort the superordinate entity and subordinate entity from high to low according to the number of links corresponding to each entity; the superordinate entity is the parent class or abstraction of the current entity; the subordinate entity is the subclass or instance of the current entity;

[0017] (3) Calculate the score of each candidate entity using the following formula:

[0018]

[0019] Among them, S i represents the score of the i-th candidate entity, φ j (E i ) represents entity E i The j-th superordinate entity of Representative entity E i The kth subordinate entity of j (E i ))and Represent the keywords corresponding to these superordinate entities and subordinate entities, K init Represents the initial keyword, K(E i ) represents the current entity E i The corresponding keyword, R ext () represents the number of times the keyword appears in the query expansion mode, and N represents the total number of superordinate entities of the current entity;

[0020] (4) Select the candidate entity with the highest score as the target entity.

[0021] Furthermore, the specific steps of expanding the entity upward in step 3 are as follows:

[0022] (1) For each initial entity E now, get its parent entity list;

[0023] For each entity E in the list i , calculate the initial entity E now To entity E i Confidence Among them, R all () function is the number of times the search engine appears in the exact match mode; if the confidence obtained is less than or equal to the threshold b0, the entity is deleted from the entity list;

[0024] (2) For each entity E remaining in the list i , respectively obtain its subordinate entities, for each obtained subordinate entity E j , calculate E now To entity E j Confidence V0, E now To entity E j The relative support V1, E now To entity E j Relative improvement V2, initial entity E init To entity E j Confidence V3, initial entity E init To entity E j The relative support V4 of the initial entity E init To entity E j The relative improvement V5 of each extended entity and the initial entity E are recorded. now depth;

[0025]

[0026]

[0027]

[0028]

[0029]

[0030]

[0031] (3) For the calculated V0~V5 indicators, if V0>b0 and When the entity E j Add extended entity set E ext , and record entity E j id, corresponding keywords, and number of links; calculate each entity E i If the number of expanded entities exceeds the threshold μn*depthα When entity E i is activated and continues to expand upward;

[0032] Among them, b0 to b5 are 6 constant thresholds with values ​​ranging from 0.5 to 0.7, m is the number of calculation indicators with a value of 6, which means that 6 indicators need to be calculated; the ceil() function is a floor rounding function; μ and α are two constant parameters, and n is the total number of subordinate entities of the entity being processed.

[0033] Furthermore, the specific steps of downward expansion of the entity in step 3 are as follows:

[0034] For each initial entity E now , get its subordinate entity list; for each entity E in the subordinate entity list j , calculate the same V0~V5 indicators as the upward expansion, and record the difference between each expanded entity and the initial entity E now depth;

[0035] If V0>b0 and And depth≤γ, then the entity E j Add extended entity set E ext , and record entity E j id, corresponding keywords, and number of links; γ is a constant parameter that can be any positive integer.

[0036] Furthermore, in step 4, the input of the classification model is depth, number of links, the corresponding label input during model training is the user's score for the relevance of each new entity to the initial entity, which takes a value of 0 or 1.

[0037] Furthermore, the relevance weight of each new entity is calculated using the following formula:

[0038] S=P pre *(1+b) cnt

[0039] Among them, S is the relevance weight, P pre is the correlation predicted by the classification model, b is a constant parameter ranging from 0 to 0.2, and cnt is the number of times the word is expanded.

[0040] The beneficial effects of the present invention are as follows:

[0041] (1) The method of the present invention can effectively expand the topic description of the topic crawler, so that the topic crawler can operate better.

[0042] (2) Compared with previous methods, the method of the present invention achieves better results in text classification experiments.

[0043] (3) By using the method of the present invention to obtain the position of a word in the semantic web, an interactive system can be designed to allow users to select the word to be expanded, such as upward / downward expansion, sibling entity expansion, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flow chart of the active data collection method based on semantic extension.

[0045] Figure 2 This is a line chart of the accuracy of the text classification experiment.

[0046] Figure 3 A line chart showing the recall rate of the text classification experiment.

[0047] Figure 4 This is a line chart of the F1 value of the recall rate of the text classification experiment. DETAILED DESCRIPTION

[0048] The present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments, and the purpose and effects of the present invention will become more apparent. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0049] In order to better expand the initial keywords to enrich the topic description, this embodiment borrows the open source knowledge base WikiData for semantic expansion and designs a series of screening methods to achieve a better expansion effect.

[0050] Before describing the specific implementation scheme, some terms or formulas need to be defined:

[0051] Superordinate entity: A new entity in Wikidata pointed to by a relation subclass of (P279) or instance of (P31), represented by the symbol φ.

[0052] Subordinate entity: An entity that is referred to in Wikidata by the relation subclass of (P279) or instance of (P31), using the symbol express.

[0053] Number of results in search engine query expansion mode: Enter keywords separated by spaces in the search engine. By default, query expansion search is used, that is, the documents displayed do not only contain the provided keywords, but also some are query expansion results. Use the symbol R ext (X) indicates.

[0054] The number of results in the search engine's exact match mode: Each keyword entered in the search engine is enclosed in quotation marks. In this case, an exact match search will be used, that is, the documents searched must explicitly contain all the keywords. Use the symbol R all (X) indicates.

[0055] Relative support: It comes from the concept of support in association rule theory and the formula is in Represents the relative support from item set X to item set Y, R all (X) and R all (Y) represents the number of results of item set X and item set Y in the search engine's exact match mode. This indicator reflects the ratio between the frequency of item set X and item set Y.

[0056] Confidence: It reflects the probability that item set Y will appear simultaneously when the document contains keyword item set X.

[0057] Relative improvement: Relative improvement is used to measure whether the occurrence probability of item set Y is improved when item set X appears.

[0058] Number of links: sitelinks(X), represents the number of times the candidate word is cited, which can be obtained in WikiData through sparql statements.

[0059] Figure 1 This is a flow chart of the method of this embodiment, and the specific steps are described as follows:

[0060] Step 1: Set the initial keywords to describe the topic;

[0061] Before using this method, users first need to provide 3-5 initial keywords related to a certain topic. The keywords must be in English. For example, if a user wants to obtain health-related information, he or she can provide the keywords "overweight", "exercise", and "disease".

[0062] Step 2: Use the initial keywords set in step 1 to identify the corresponding initial entities in the semantic web, and ensure that each keyword will correspond to a unique initial entity.

[0063] (1) calling the interface provided by the semantic web to obtain a list of candidate entities corresponding to the initial keywords sorted by relevance;

[0064] (2) Find the superordinate entity and subordinate entity of each candidate entity, and sort the superordinate entity and subordinate entity from high to low according to the number of links corresponding to each entity; the superordinate entity is the parent class or abstraction of the current entity; the subordinate entity is the subclass or instance of the current entity;

[0065] (3) Calculate the score of each candidate entity using the following formula:

[0066]

[0067] Among them, S i represents the score of the i-th candidate entity, φ j (E i ) represents entity E i The j-th superordinate entity of Representative entity E i The kth subordinate entity of j (E i ))and Represent the keywords corresponding to these superordinate entities and subordinate entities, K init Represents the initial keyword, K(E i ) represents the current entity E i The corresponding keyword, R ext () represents the number of times the keyword appears in the query expansion mode, and N represents the total number of superordinate entities of the current entity;

[0068] (4) Select the candidate entity with the highest score as the target entity.

[0069] In this example, the API https: / / www.wikidata.org / w / index.php?search={}&title=Special%3ASearch&fulltext=1&ns0=1&ns120=1 is called, where the search field contains the keywords to be identified. This call returns a webpage containing all candidate entities, sorted by relevance. The Python lxml library is used to parse the page and obtain the IDs of the top 10 entities. A score is calculated for each candidate entity, and the candidate with the highest score is selected as the target entity.

[0070] Step 3: Expand each initial entity identified in step 2 using the upward expansion and downward expansion methods, and obtain several corresponding new entities for each initial entity;

[0071] For an entity in the Semantic Web, its surrounding entities are semantically connected to it. A superordinate entity is an abstract concept or generalized representation of the entity. If the confidence level from the current entity to the superordinate entity exceeds a threshold, the superordinate entity is considered likely to be expanded, and its subordinate entities are also likely to have high semantic similarity with the current entity. When the superordinate entity meets certain conditions, it can be expanded further.

[0072] A subordinate entity is an instance or subclass concept of the entity. Subordinate entities with high semantic relevance can be expanded, and the expanded entity can continue to expand downward, but a maximum distance depth γ needs to be set.

[0073] (1) Upward expansion

[0074] (1) For each initial entity E now , get its parent entity list;

[0075] For each entity E in the list i , calculate the initial entity E now To entity E i Confidence Among them, R all () function is the number of times the search engine appears in the exact match mode; if the confidence obtained is less than or equal to the threshold b0, the entity is deleted from the entity list;

[0076] (2) For each entity E remaining in the list i , respectively obtain its subordinate entities, for each obtained subordinate entity E j , calculate E now To entity E j Confidence V0, E now To entity E j The relative support V1, E now To entity E j Relative improvement V2, initial entity E init To entity E j Confidence V3, initial entity E init To entity E j The relative support V4 of the initial entity E init To entity E j The relative improvement V5 of each extended entity and the initial entity E are recorded. now depth;

[0077]

[0078]

[0079]

[0080]

[0081]

[0082]

[0083] (3) For the calculated V0~V5 indicators, if V0>b0 and When the entity E j Add extended entity set E ext , and record entity E j id, corresponding keywords, and number of links; calculate each entity E i If the number of expanded entities exceeds the threshold μn*depth α When entity E i is activated and continues to expand upward;

[0084] Among them, b0 to b5 are 6 constant thresholds with values ​​ranging from 0.5 to 0.7, m is the number of calculation indicators with a value of 6, which means that 6 indicators need to be calculated; the ceil() function is a floor rounding function; μ and α are two constant parameters, and n is the total number of subordinate entities of the entity being processed.

[0085] (2) Downward expansion

[0086] For each initial entity E now , get its subordinate entity list; for each entity E in the subordinate entity list j , calculate the same V0~V5 indicators as the upward expansion, and record the difference between each expanded entity and the initial entity E now The distance dep h;

[0087] If V0>b0 and And depth≤γ, then the entity E j Add extended entity set E ext , and record entity E j id, corresponding keywords, and number of links; γ is a constant parameter that can be any positive integer.

[0088] Step 4: Build and use a classification model to predict the probability of each new entity being recommended as a related entity. Then, based on the probability and the number of times the new entity appears, calculate the relevance weight of each new entity, and sort the relevance weights from high to low. Select the new entities with the highest scores as related entities.

[0089] In step 3, many entities will be expanded, including some words that are not highly relevant to the theme. It is speculated that the relevance between the keyword and the theme is related to V 0-5 and the entity distance and the number of links. Therefore, a classification model can be used to predict whether an entity is relevant to the theme. Taking WikiData as an example, it will be explained from three aspects: data acquisition, model construction, and model use:

[0090] (1) Data acquisition. Manually define several themes, and provide x initial keywords for each theme. Using the random sampling method, each time randomly select y initial keywords from one of the themes as a group, where y < x. After step three, several candidate words will be expanded. Repeat multiple times and integrate all the candidate words. After removing duplicates, they will be used as the data for the classification model. An odd number of raters are required to mark the relevance of these candidate words. If it is considered that the expanded word is relevant to its preset theme, mark 1; if not, mark 0. For each candidate word, if the number of times it is marked as 1 is greater than half of the number of raters, then it is considered that the keyword is relevant to the theme. After all the markings are completed, an experimental data with labels is obtained, which can be used for supervised learning. The fields retained during the WikiData expansion process are shown in Table 1.

[0091] Table 1 Fields Retained during the WikiData Expansion Process

[0092] Attribute Name describe id The ID of the candidate word in Wikidata keyword Candidate words sitelinks Number of links, int type v0 The calculated value of v0, double type v1 The calculated value of v1, double type v2 The calculated value of v2, double type v3 The calculated value of v3, double type v4 The calculated value of v4, double type v5 The calculated value of v5, double type depth The distance between the candidate entity and the initial entity in Wikidata label The final result after the scorer scores is used as the label for supervised learning

[0093] (2) Model construction. The classification model uses the xgboost binary classification model. The inputs of the model are depth and the number of links. When the model is trained, the corresponding label input is the score given by the user for the relevance between each new entity and the initial entity, taking values of 0 or 1. Divide the obtained data into a training set and a validation set at a ratio of 4:1. Build an xgboost binary classification model on the training set. The evaluation index is the auc value, the learning rate is 0.1, and a total of 100 rounds of training are performed. If the auc value of the validation set has not improved in 10 rounds, stop the iteration.

[0094] (3) Using the model. For the keywords with corresponding metrics after WikiData expansion, this model can be used to judge the relevance. The prediction result is a value between 0 and 1, representing the probability that the model determines that the keyword is relevant to the theme. This probability can be used to judge the relevance of the keyword. Since wikidata may expand some duplicate words, the more times a word is repeated, the higher its relevance. Therefore, the final relevance weight of the keyword is S = P pre *(1 + b) cnt , where S is the relevance weight, P preis the correlation predicted by the classification model, b is a constant parameter ranging from 0 to 0.2, and cnt is the number of times the word is expanded.

[0095] Step 5: Repeat steps 3 to 4 to obtain more relevant entities and their scores that describe the topic, making the topic description more complete.

[0096] To further improve the model's performance, for entities expanded through steps 1-4, you can iterate again through steps 3 and 4. Since the expanded entities already have ID information, no further entity recognition is required. Each round of iteration selects the N highest-scoring keywords that have not yet been selected as the keywords for that round of expansion. A maximum number of iterations is set. If the number of iterations reaches a threshold or the average score of the keywords expanded in the current round and the average score of the previous round are less than a threshold α, the iteration stops and the expanded results of that round are returned as the final result.

[0097] Through the above steps, the model will generate M keywords, where the relevance scores calculated in step 3 are the keyword weights, and finally a topic model is generated. The topic crawler can use this topic model to choose whether to crawl the page.

[0098] The effect of the method of the present invention is verified by taking an example below.

[0099] The experimental data comes from the News Category Dataset on Kaggle, which includes historical news article headlines, links, and categorization labels from the HuffPost website. Six topics were selected and the original text of related articles was crawled based on their links. 1,200 articles were crawled for each topic, for a total of 7,200 articles. The six topics were: MONEY, CRIME, PARENTS, SCIENCE, BUSINESS, and WELLNESS.

[0100] For each topic, four initial keywords are first provided, and then the model Wiki-Crawler of the present invention and two other control models Tfidf-Crawler and Genetic-Crawler are used to expand them. The maximum number of iterations is 5 rounds, and 2 new words are added in each round to form a topic description word set of 6 specific topics. The keywords in each topic word set are expanded using NLTK's wordnet synonyms to eliminate the influence of synonyms, and finally 6 topic models are generated. The number of occurrences of the topic words of each topic is counted for each crawled article, and the topic of the current article is predicted to be the topic with the most topic words. The model that performs better in this six-category problem has better keyword expansion capabilities. Figures 2 to 4The results of the experiment are the precision, recall and F1 value. As can be seen from the figure, the precision, recall and F1 value of the method of the present invention are significantly better than those of the other two control models.

[0101] Those skilled in the art will understand that the foregoing descriptions are merely preferred embodiments of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art will still be able to modify the technical solutions described in the foregoing examples or substitute equivalents for some of the technical features therein. Any modifications, equivalent substitutions, etc. made within the spirit and principles of the invention shall be included within the scope of protection of the invention.

Claims

1. An active data collection method based on semantic extension, characterized in that: The following steps are involved: Step 1: Set the initial keywords to describe the topic; Step 2: Use the initial keywords set in step 1 to identify the corresponding initial entities in the semantic web, and ensure that each keyword corresponds to a unique initial entity; Step 3: Expand each initial entity identified in step 2 using the upward expansion and downward expansion methods, and obtain several corresponding new entities for each initial entity; The specific steps for expanding the entity in step 3 are as follows: (1) For each initial entity E now , get its parent entity list; For each entity E in the list i , calculate the initial entity E now To entity E i Confidence Among them, R all () function is the number of times the search engine appears in the exact match mode; if the confidence obtained is less than or equal to the threshold b0, the entity is deleted from the entity list; where K(E i ) represents the current entity E i The corresponding keywords; (2) For each entity E remaining in the list i , respectively obtain its subordinate entities, for each obtained subordinate entity E j , calculate E now To entity E j Confidence V0, E now To entity E j The relative support V1, E now To entity E j Relative improvement V2, initial entity E init To entity E j Confidence V3, initial entity E init To entity E j The relative support V4 of the initial entity E init To entity E j The relative improvement V5 of each extended entity and the initial entity E are recorded. now depth; (3) For the calculated V0~V5 indicators, if V0>b0 and When the entity E j Add extended entity set E ext , and record entity E j id, corresponding keywords, and number of links; calculate each entity E i If the number of expanded entities exceeds the threshold μn*depth α When entity E i is activated and continues to expand upward; Among them, b0 to b5 are 6 constant thresholds with values ​​ranging from 0.5 to 0.7, m is the number of calculation indicators with a value of 6, indicating that 6 indicators need to be calculated; the ceil() function is a floor rounding function; μ and α are two constant parameters, and n is the total number of subordinate entities of the processed entity; The specific steps for downward expansion of the entity in step 3 are as follows: For each initial entity E now , get its subordinate entity list; for each entity E in the subordinate entity list j , calculate the same V0~V5 indicators as the upward expansion, and record the difference between each expanded entity and the initial entity E now depth; If V0>b0 and And depth≤γ, then the entity E j Add extended entity set E ext , and record entity E j id, corresponding keywords, and number of links; γ is a constant parameter that can be any positive integer; Step 4: Build and use a classification model to predict the probability of each new entity being recommended as a related entity, i.e., the relevance. Then, based on the relevance and the number of times the new entity appears, calculate the relevance weight of each new entity, and sort the relevance weights from high to low. Select the new entities with the highest scores as related entities. Step 5: Repeat steps 3 to 4 to obtain more relevant entities and their scores that describe the topic, making the topic description more complete.

2. The active data collection method based on semantic extension according to claim 1, characterized in that: The second step is specifically implemented through the following sub-steps: (1) calling the interface provided by the semantic web to obtain a list of candidate entities corresponding to the initial keywords sorted by relevance; (2) Find the superordinate entity and subordinate entity of each candidate entity, and sort the superordinate entity and subordinate entity from high to low according to the number of links corresponding to each entity; the superordinate entity is the parent class or abstraction of the current entity; the subordinate entity is the subclass or instance of the current entity; (3) Calculate the score of each candidate entity using the following formula: Among them, S i represents the score of the i-th candidate entity, φ j (E i ) represents entity E i The j-th superordinate entity of Representative entity E i The kth subordinate entity of j (E i ))and Represent the keywords corresponding to these superordinate entities and subordinate entities, K init Represents the initial keyword, K(E i ) represents the current entity E i The corresponding keyword, R ext () represents the number of times the keyword appears in the query expansion mode, and N represents the total number of superordinate entities of the current entity; (4) Select the candidate entity with the highest score as the target entity.

3. The active data collection method based on semantic extension according to claim 1, characterized in that: In step 4, the input of the classification model is depth, number of links, the corresponding label input during model training is the user's score for the relevance of each new entity to the initial entity, which takes a value of 0 or 1.

4. The active data collection method based on semantic extension according to claim 1, characterized in that: The relevance weight of each new entity is calculated using the following formula: S=P pre *(1+b) cnt Among them, S is the relevance weight, P pre is the correlation predicted by the classification model, b is a constant parameter ranging from 0 to 0.2, and cnt is the number of times the new entity appears.

Images