Article topic extraction method, device, equipment, storage medium and processor

Abstract

Embodiments of the present application disclose an article topic extraction method, device, equipment, storage medium and processor. The article topic extraction method comprises two stages: 1) full amount calculation: extracting keywords from multiple articles, each article comprising multiple keywords; for each article, determining at least one associated combination according to the multiple keywords corresponding to the article, the associated combination comprising at least two keywords; determining the occurrence probability of each associated combination according to the number of each associated combination; 2) target article mapping: determining the topic of a target article according to the occurrence probability of each associated combination and the associated combination corresponding to the target article, wherein the target article is one of the multiple articles. Through the present application, the technical problem of poor accuracy of article topic determination in the process of article topic extraction in the related art is solved.

Description

Technical Field

[0001] This invention relates to the field of text tagging, and more particularly to a method, apparatus, device, storage medium, and processor for extracting article topics. Background Technology

[0002] Large-scale text tagging and topic aggregation have important applications in many scenarios, such as SEO (Search Engine Optimization, a method of improving a website's organic ranking in search engines by utilizing search engine rules), intelligent search, and text clustering.

[0003] Currently, in the process of large-scale text tagging, multiple keywords are identified from the article to extract the topic, and then the content formed by random combinations of these keywords is used as the article's topic. However, topics determined in this way often suffer from semantic inconsistencies and inaccurate wording, failing to clearly and accurately summarize the article's content.

[0004] Therefore, related technologies suffer from poor accuracy in identifying article topics during the topic extraction process. Currently, no effective solution has been proposed to address this problem.

[0005] The information disclosed in the background section is only intended to enhance the understanding of the background art described herein. Therefore, the background art may contain information that would not be considered part of the prior art by those skilled in the art. Summary of the Invention

[0006] This invention provides a method, apparatus, device, storage medium, and processor for extracting article topics, thereby at least addressing the technical problem of poor accuracy in determining article topics during the extraction process in related technologies.

[0007] According to a first aspect of the present invention, an article topic extraction method is provided, comprising: extracting keywords from multiple articles, each article including multiple keywords; for each article, determining at least one association combination based on its corresponding multiple keywords, the association combination including at least two keywords arranged in order within the article; determining the occurrence probability of each association combination based on the number of each association combination; and determining the topic of a target article based on the occurrence probability of each association combination and the association combination corresponding to a target article, wherein the target article is one of multiple articles.

[0008] Furthermore, for each article, based on its corresponding multiple keywords, at least one association combination is determined, including: for each keyword, generating an association combination between it and each of the subsequent keywords.

[0009] Furthermore, for each article, based on its corresponding multiple keywords, at least one association combination is determined, including: for any keyword, determining a span threshold; generating an association combination between the keyword and each subsequent target keyword, wherein the number of keywords between the keyword and the target keyword is less than or equal to the span threshold.

[0010] Furthermore, for any keyword, determining the span threshold includes: if the number of subsequent keywords is greater than a preset threshold, determining the preset threshold as the span threshold; if the number of subsequent keywords is less than the preset threshold, determining the number of subsequent keywords as the span threshold, wherein subsequent keywords are keywords within the article that follow the keywords of the association combination to be determined.

[0011] Furthermore, based on the number of each association combination, the probability of occurrence of each association combination is determined by using multiple keywords extracted from multiple articles as vertices and the association relationships between the keywords represented by multiple association combinations as edges, and a graph calculation method is used to determine the probability of occurrence of each association combination.

[0012] Furthermore, the probability of an associated combination is the ratio of the number of associated combinations to the number of all related associated combinations, wherein the first keyword in an associated combination is the same as the first keyword in a related associated combination, and the second keyword in an associated combination is the same as or different from the second keyword in a related associated combination.

[0013] Furthermore, based on the occurrence probability of each association combination and the association combination corresponding to the target article, determining the topic of the target article includes: determining one or more target combinations corresponding to the target article, where the target combination is a combination whose occurrence probability is greater than or equal to a preset probability among the association combinations corresponding to the target article; and determining the topic of the target article, where the topic includes keywords corresponding to one or more target combinations.

[0014] Furthermore, after determining the topic of the target article based on the occurrence probability of each association combination and the association combination corresponding to the target article, the article topic extraction method also includes: performing semantic analysis on the topics of multiple target articles to identify semantically unreasonable topics; and removing semantically unreasonable topics from the topics of multiple target articles.

[0015] Furthermore, after determining the topic of the target article based on the occurrence probability of each association combination and the association combination corresponding to the target article, the article topic extraction method also includes: classifying multiple target articles according to the topic of each target article.

[0016] Furthermore, the classification of multiple target articles based on their themes includes: classifying multiple target articles according to different classification levels based on their themes, where multiple target articles in the same category share some keywords or related combinations, and the number of keywords or related combinations corresponding to the target articles increases as the classification level decreases.

[0017] Furthermore, keywords are extracted from multiple articles, with each article containing multiple keywords. This process involves: for any given article, obtaining at least a portion of the article's text content; dividing the text content according to a predefined word segmentation logic to obtain multiple words; and removing some words from the multiple words according to a predefined filtering rule to obtain multiple keywords.

[0018] According to a second aspect of the present invention, an article topic extraction apparatus is also provided, comprising: an extraction unit for extracting keywords from multiple articles, each article including multiple keywords; a first determining unit for determining at least one association combination for each article based on its corresponding multiple keywords, the association combination including at least two keywords arranged in order within the article; a second determining unit for determining the occurrence probability of each association combination based on the number of each association combination; and a third determining unit for determining the topic of a target article based on the occurrence probability of each association combination and the association combination corresponding to a target article, wherein the target article is one of multiple articles.

[0019] Further, the first determining unit is used to: for each keyword, generate an association combination between it and each subsequent keyword; or, the first determining unit includes a first determining module and a generating module: the first determining module is used to determine a span threshold for any keyword; the generating module is used to generate an association combination between the keyword and each subsequent target keyword, wherein the number of keywords between the keyword and the target keyword is less than or equal to the span threshold; the first determining module is used to: determine the preset threshold as the span threshold when the number of subsequent keywords is greater than the preset threshold; determine the number of subsequent keywords as the span threshold when the number of subsequent keywords is less than the preset threshold, wherein the subsequent keywords are keywords in the article that follow the keyword in the association combination to be determined;

[0020] The second determining unit is used to: use multiple keywords extracted from multiple articles as vertices and the association relationships between keywords represented by multiple association combinations as edges, and use graph calculation to determine the probability of occurrence of each association combination;

[0021] The probability of an associated combination is the ratio of the number of associated combinations to the total number of related associated combinations. In this case, the first keyword in an associated combination is the same as the first keyword in a related associated combination, and the second keyword in an associated combination is either the same as or different from the second keyword in a related associated combination.

[0022] The third determining unit includes: a second determining module, used to determine one or more target combinations corresponding to the target article, wherein the target combination is a combination whose probability of occurrence is greater than or equal to a preset probability among the associated combinations corresponding to the target article; and a third determining module, used to determine the theme of the target article, wherein the theme includes keywords corresponding to one or more target combinations.

[0023] The article topic extraction device also includes: a fourth determining unit, used to perform semantic analysis on the topics of multiple target articles after determining the topic of the target article based on the occurrence probability of each association combination and the association combination corresponding to the target article, and to determine the semantically unreasonable topics; and a removal unit, used to remove the semantically unreasonable topics from the topics of multiple target articles.

[0024] The article topic extraction device also includes a classification unit, which, after determining the topic of the target article based on the occurrence probability of each association combination and the association combination corresponding to the target article, classifies multiple target articles according to the topic of each target article;

[0025] The classification unit is used to classify multiple target articles according to different classification levels based on the topic of each target article. Among them, multiple target articles in the same category have some keywords or related combinations. As the classification level decreases, the number of keywords or related combinations corresponding to the target articles increases.

[0026] The extraction unit includes: an acquisition module, used to acquire at least part of the text content of any article; a segmentation module, used to segment the text content according to a preset word segmentation logic to obtain multiple words; and a removal module, used to remove some words from the multiple words according to preset filtering rules to obtain multiple keywords.

[0027] According to a third aspect of the present invention, a non-volatile storage medium is also provided, wherein the non-volatile storage medium includes a stored program, wherein the program controls the device where the non-volatile storage medium is located to execute the above-described article topic extraction method during runtime.

[0028] According to a fourth aspect of the present invention, a processor is also provided, wherein the processor is configured to run a program, wherein the program executes the above-described article topic extraction method during runtime.

[0029] According to a fifth aspect of the present invention, an article topic extraction device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described article topic extraction method.

[0030] The article topic extraction method of this invention includes: extracting keywords from multiple articles, each article including multiple keywords; for each article, determining at least one association combination based on its corresponding multiple keywords, each association combination including at least two keywords arranged in the order within the article; determining the probability of occurrence of each association combination based on the number of each association combination; and determining the topic of the target article based on the probability of occurrence of each association combination and the association combination corresponding to the target article, wherein the target article is one of the multiple articles. The article topic extraction method described above involves two stages in determining the topic of the target article: a full calculation stage and a target article mapping stage. In the full calculation stage, keywords are first extracted from multiple articles, with at least two keywords extracted from each article. For each article, at least one association combination is determined based on its corresponding keywords, each association combination including at least two keywords arranged in the order within the article. Then, the probability of occurrence of each association combination is determined based on the number of occurrences of each association combination. Finally, in the target article mapping stage, the topic of the target article is determined based on the probability of occurrence of each association combination and the association combination corresponding to the target article. In the above implementation process, for each article, at least one association combination is established based on its different keywords, ensuring that at least two keywords are associated. The probability of occurrence of each association combination is calculated. The higher the probability of occurrence of an association combination, the stronger the correlation between the two keywords included in that combination, and the more universal the keyword expression form used in that association combination. Ultimately, the topic determined by combining the occurrence probabilities of each association combination with the association combination corresponding to the target article better takes into account the correlation between keywords and the universality of the expression form, thus better representing the content of the target article. Compared with the method of determining the article topic by randomly combining keywords in related technologies, this method can effectively improve the accuracy of topic determination and solve the problem of poor accuracy in article topic determination in related technologies. Attached Figure Description

[0031] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0032] Figure 1 A flowchart illustrating an article topic extraction method provided in an embodiment of the present invention;

[0033] Figure 2 A schematic diagram of an article topic extraction device provided in an embodiment of the present invention;

[0034] Figure 3 This is a schematic diagram illustrating the calculation of the occurrence probability of various association combinations in an article topic extraction method provided in an embodiment of the present invention.

[0035] Figure 4 This is a schematic diagram illustrating how an article topic extraction method, provided in an embodiment of the present invention, categorizes multiple target articles according to different classification levels. Detailed Implementation

[0036] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0037] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this invention are used to distinguish different objects, rather than to limit a specific order.

[0038] Figure 1 This is an article topic extraction method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0039] Step S102: Extract keywords from multiple articles, with each article containing multiple keywords;

[0040] Step S104: For each article, determine at least one association combination based on its corresponding multiple keywords. The association combination includes at least two keywords, and the at least two keywords are arranged in the order they appear in the article.

[0041] Step S106: Determine the probability of occurrence of each association combination based on the number of each association combination;

[0042] Step S108: Determine the topic of the target article based on the occurrence probability of each association combination and the association combination corresponding to the target article, wherein the target article is one of multiple articles.

[0043] The article topic extraction method using the above scheme includes: extracting keywords from multiple articles, with each article containing multiple keywords; for each article, determining at least one association combination based on its corresponding keywords, each association combination including at least two keywords arranged in the order they appear within the article; determining the probability of occurrence of each association combination based on its quantity; and determining the topic of the target article based on the probability of occurrence of each association combination and the association combination corresponding to the target article, where the target article is one of multiple articles. The process of determining the topic of the target article using the above article topic extraction method is divided into two stages: a full-scale calculation stage and a target article mapping stage. In the full-scale calculation stage, keywords are first extracted from multiple articles, with at least two keywords extracted from each article. For each article, at least one association combination is determined based on its corresponding keywords, with each association combination including at least two keywords arranged in the order they appear within the article. Then, the probability of occurrence of each association combination is determined based on its quantity. Finally, in the target article mapping stage, the topic of the target article is determined based on the probability of occurrence of each association combination and the association combination corresponding to the target article. In the above implementation process, for each article, at least one association combination is established based on its different keywords, ensuring that at least two keywords are associated. The probability of occurrence of each association combination is calculated. The higher the probability of occurrence of an association combination, the stronger the correlation between the two keywords included in that combination, and the more universal the keyword expression form used in that association combination. Ultimately, the topic determined by combining the occurrence probabilities of each association combination with the association combination corresponding to the target article better takes into account the correlation between keywords and the universality of the expression form, thus better representing the content of the target article. Compared with the method of determining the article topic by randomly combining keywords in related technologies, this method can effectively improve the accuracy of topic determination and solve the problem of poor accuracy in article topic determination in related technologies.

[0044] The above-mentioned themes are based on the probability of occurrence of various related combinations in multiple articles. This approach better considers the relevance between keywords and the universality of expression. Keyword relevance refers to the connections between different keywords, ensuring a strong correlation between keywords appearing in the theme and avoiding the misinterpretation of unrelated keywords. Universality of expression refers to whether the arrangement of multiple keywords using the corresponding related combinations is a universal expression. This involves whether the combination of keywords is universal and whether the order of keyword expression is universal. For example, the expression "Python function" is more universal than "function Python". The target article can be any of multiple articles, and the target article can correspond to one or more related combinations. The above-mentioned themes are the article themes generated according to the article theme extraction method described above, or aggregated titles. Specifically, the number of generated themes can be one or more. The keywords included in the above-mentioned related combinations are arranged sequentially according to their order within the article. This allows the related combinations to better consider word order and improve the universality of expression. The above-mentioned full calculation stage and target article mapping stage are descriptive terms and do not have any additional limiting effect. The operations performed in each stage are subject to the specific steps that follow.

[0045] As mentioned above, after identifying at least one association combination for each article, the probability of each association combination's occurrence is determined based on its quantity. This determines the relevance between keywords within the association combination and the universality of its expression. Therefore, different methods of determining association combinations directly affect the probability determination and the final topic generation effect. In practice, there are different ways to determine association combinations:

[0046] In one implementation, for each article, based on its multiple keywords, at least one association combination is determined, including: for each keyword, generating an association combination between it and each subsequent keyword. That is, for each keyword, an association combination is established with any subsequent keyword. This allows for the traversal of all keywords, linking them in pairs, thus more comprehensively determining the relationships between keywords and improving the flexibility and accuracy of the final topic determination. For example, if four keywords—keyword one, keyword two, keyword three, and keyword four—are identified from an article, the following association combinations will be determined: keyword one and keyword two, keyword one and keyword three, keyword one and keyword four, keyword two and keyword three, keyword two and keyword four, and keyword three and keyword four.

[0047] In another implementation, for each article, based on its corresponding keywords, at least one association combination is determined, including: for any given keyword, determining a span threshold; generating an association combination between this keyword and each subsequent target keyword, wherein the number of keywords separating the keyword from the target keyword is less than or equal to the span threshold. In other words, in this embodiment, not just any two keywords are combined; instead, the span threshold is considered to ensure that the number of keywords separating the two keywords forming an association combination is within a certain range. In other words, only two closely spaced keywords (with the number of keywords separating them less than or equal to the span threshold) will form an association combination, while two widely spaced keywords will not. This method of setting a span threshold imposes constraints on the keywords forming association combinations, reducing the number of final association combinations. This reduces computational effort and increases the speed of topic generation.

[0048] In practical implementation, different situations may arise when establishing associations between a keyword and subsequent keywords: For any given keyword, determining the span threshold includes: if the number of subsequent keywords exceeds a preset threshold, the preset threshold is used as the span threshold; if the number of subsequent keywords is less than the preset threshold, the number of subsequent keywords is used as the span threshold. Here, subsequent keywords are keywords within the article that follow the keyword in the association to be determined. That is, `vote_len = min((len(title) - index(word)), l)`, where `l` is a custom span, i.e., the preset threshold; `vote_len` is the span threshold; `len(title)` is the number of keywords extracted from the article in which the association is to be determined; and `index(word)` is the ranking of the keyword in the article in which the association is to be determined.

[0049] To more clearly illustrate the above-mentioned association combinations, the process of determining them is termed the voting mechanism. To introduce the most initial association relationships into unlabeled content, this embodiment designs a voting mechanism between keywords based on the principle of semantic rationality: given a keyword phrase and parameter span (preset threshold), each keyword can "vote" for the keywords following it, but cannot "vote" for the keywords preceding it. Each "vote" is essentially a directed association relationship. The maximum "voting" span for each word (keyword) is designed as an adjustable parameter to balance the quantity and quality of the final topics. A larger span results in a richer hierarchy on the aggregation page; more topics and a smaller span result in better semantic rationality of the aggregation page and lower costs for manual verification.

[0050] To facilitate the calculation of the probability of occurrence of each association combination while considering the relationships between keywords, this embodiment determines the probability of occurrence of each association combination based on its quantity. This involves using multiple keywords extracted from multiple articles as vertices and the relationships between keywords represented by multiple association combinations as edges, employing graph computation to determine the probability of occurrence of each association combination. In other words, keywords extracted from multiple articles are grouped together, with different keywords as different vertices, and the relationships represented by the association combinations between these keywords as edges connecting the vertices, thus constructing a graph for graph computation. This allows for a more convenient calculation of the probability of occurrence of each association combination based on graph computation, while considering the relationships between keywords.

[0051] In one optional implementation, the probability of an associated combination occurring is the ratio of the number of associated combinations to the number of all related associated combinations, wherein the first keyword in an associated combination is the same as the first keyword in a related associated combination, and the second keyword in an associated combination is either the same as or different from the second keyword in a related associated combination. That is, the probability of an associated combination occurring is defined as the ratio of the number of that associated combination to the number of related associated combinations, where a related associated combination is one whose first keyword is the same as the associated combination whose probability is to be determined. For example, Figure 3 In the diagram, S0, S1, S2, and S3 each represent a keyword. Figure 3 The arrows in the diagram represent the associations between keywords. Since the keywords in the association combinations have a specific order, the associations are directional, and therefore the arrows are also directional. For example, P0,1 represents the probability of an association combination formed by S0 and S1 in a specific order; P0,2 represents the probability of an association combination formed by S0 and S2 in a specific order; P0,3 represents the probability of an association combination formed by S0 and S3 in a specific order; and P1,0 represents the probability of an association combination formed by S1 and S0 in a specific order, which is different from P0,1. In this embodiment, the probability of an association combination is defined as the ratio of the number of such association combinations to the number of related association combinations. Taking P0,1 as an example, it is equal to the ratio of the number of association combinations formed by S0 and S1 in a specific order to the number of related association combinations. Here, related association combinations include: association combinations formed by S0 and S0, association combinations formed by S0 and S1 in a specific order, association combinations formed by S0 and S2 in a specific order, and association combinations formed by S0 and S3 in a specific order. To understand this probability more simply, it can be understood as four arrows originating from S0, pointing to S0, S1, S2, and S3 respectively, and P0,1 represents the probability of the arrow pointing from S0 to S1.

[0052] In determining the topic of a target article based on the occurrence probability of each association combination and the association combination corresponding to the target article, since the occurrence probability can characterize the correlation between corresponding keywords and the universality of corresponding expression forms, the higher the occurrence probability, the more suitable the corresponding association combination is as a topic or part of a topic to summarize the content of the article. Therefore, in this embodiment, determining the topic of the target article based on the occurrence probability of each association combination and the association combination corresponding to the target article includes: determining one or more target combinations corresponding to the target article, where the target combination is a combination among the association combinations corresponding to the target article with an occurrence probability greater than or equal to a preset probability; and determining the topic of the target article, where the topic includes the keywords corresponding to one or more target combinations. In the process of determining the topic of the target article, association combinations with a probability greater than or equal to the preset probability are first determined as target combinations, and then the topic is determined, such that the topic includes the keywords corresponding to one or more target combinations. For example, if the keywords extracted from an article include "python", "design patterns", and "introduction", and the determined association combinations include "python design patterns", "introduction to python", and "introduction to design patterns", and if the probability of occurrence of these three association combinations is greater than or equal to a preset probability (e.g., 80%) in the probability calculation based on multiple articles, then the keywords corresponding to one or more association combinations of the above three association combinations in the topic can be determined. For example, the topic could be "introduction to python design patterns", "introduction to python", etc., which can all summarize the content of the target article well.

[0053] In a preferred embodiment, after determining the topic of the target article based on the occurrence probability of each association combination and the association combination corresponding to the target article, the article topic extraction method further includes: performing semantic analysis on the topics of multiple target articles to identify semantically unreasonable topics; and removing semantically unreasonable topics from the topics of multiple target articles. In this embodiment, after determining the topic, semantic analysis is performed on the topics of multiple target articles, and semantically unreasonable topics are removed, thereby further ensuring the rationality of the determined article topic. The semantic analysis method and the definition of semantically unreasonable topics can be flexibly selected according to the actual situation. For example, a pre-trained recognition model can be used to identify semantic rationality, such as determining the semantic rationality based on the probability of the keyword order in the topic in the reference sample data. If the probability is greater than a certain value, the semantics are determined to be reasonable; otherwise, the semantics are determined to be unreasonable.

[0054] In this embodiment, after determining the topic of the target article based on the occurrence probability of each association combination and the association combination corresponding to the target article, the article topic extraction method further includes: classifying multiple target articles according to their topics. Preferably, after determining the topics of the target articles and removing semantically unreasonable topics, multiple target articles are classified according to their topics. Classifying the target articles facilitates the aggregated display of multiple target articles (e.g., placing articles of the same type on the same aggregated page), thereby improving user reading convenience, enhancing the accessibility of website content, increasing user clicks on related content under the same topic, and providing higher-quality backlinks.

[0055] In one optional implementation, classifying multiple target articles based on their themes includes: classifying the target articles according to different classification levels based on their themes, where multiple target articles within the same category share some keywords or related combinations; and the number of keywords or related combinations increases as the classification level decreases. In the process of classifying multiple target articles, different classification levels are used based on their themes. Multiple target articles within the same category share some keywords or related combinations, and the number of keywords or related combinations increases as the classification level decreases. In other words, hierarchical classification is achieved based on the themes of each target article, allowing multiple target articles to be categorized into different categories according to a hierarchical structure; different classification levels correspond to different classification granularities. Figure 4 As shown, the articles are divided into three levels: the top level is "Python," the second level is "Python variables," and the third level includes several subcategories, such as "Python variable data types," "Python variable values," etc. Figure 4 In this structure, the number following each category indicates the number of articles within that category. This hierarchical structure not only makes article indexing more organized and easier for users to read, but also makes the content results more in line with search engine requirements for hierarchical content structure, which is beneficial for search engine optimization.

[0056] The process of extracting keywords from multiple articles can be implemented in various ways. For example, TF-IDF (Term Frequency–Inverse Document Frequency) technology can be used to determine keywords. TF-IDF is a commonly used weighting technique in information retrieval and data mining to evaluate the importance of a word to a document in a document set or corpus. In this embodiment, extracting keywords from multiple articles, each article including multiple keywords, involves: for any article, obtaining at least a portion of the article's text content; dividing the text content according to a preset word segmentation logic to obtain multiple words; and removing some words from the multiple words according to preset filtering rules to obtain multiple keywords. In the process of extracting keywords from multiple articles, at least a portion of the article's content is first obtained, such as the article's title, abstract, etc., then divided according to a preset word segmentation logic to obtain multiple words, and then some words are removed according to a preset filtering logic to obtain multiple keywords. The word segmentation logic and filtering logic can be flexibly defined according to the actual situation. For example, the open-source library jieba can be used as a natural language processing package, and the open-source library re can be used as a regular expression processing module. The two can be combined as the word segmentation logic.

[0057] The article topic extraction method of the present invention will be further described below with reference to a specific embodiment:

[0058] Currently, content communities are typically massive collections of articles without hierarchical structure or thematic division. This not only fails to meet search engine requirements for hierarchical content structure but also hinders users' ability to expand their reading on the same topic. To improve website content accessibility, increase user clicks on related content within the same theme, and thus provide higher-quality backlinks, it is crucial to reorganize and sort existing website content according to specific themes or keywords to generate multi-level topic pages. Therefore, text content tagging and topic aggregation have become critical aspects of SEO content optimization. However, content tagging solutions primarily rely on manual tagging. For massive amounts of text content, differences in content understanding often lead to inconsistent manual tagging, requiring significant time and effort, resulting in a very long timeframe from publication to external search engine indexing. Therefore, this model struggles to meet SEO's demands for large-scale content aggregation and rapid ranking improvement of new content.

[0059] To address the aforementioned issues, this embodiment provides a method for extracting article themes, including: extracting keywords from multiple articles, each article containing multiple keywords; for each article, determining at least one association combination based on its corresponding multiple keywords, each association combination including at least two keywords arranged in the order they appear within the article; determining the probability of occurrence of each association combination based on its quantity; and determining the theme of the target article based on the probability of occurrence of each association combination and the association combination corresponding to the target article, wherein the target article is one of multiple articles. This embodiment, based on the analysis of word association relationships, establishes a fully intelligent process from content tagging, global calculation, semantic reasoning to hierarchical content aggregation, providing automated text tagging, content aggregation, and hierarchical indexing capabilities for SEO content optimization. This embodiment can automatically generate multi-level aggregation page titles from massive amounts of untagged content (i.e., classifying multiple target articles according to different classification levels based on their themes), expanding the original non-hierarchical aggregation pages to multi-level ones. While ensuring content theme consistency, it provides multi-level internal linking and standardized indexing capabilities for SEO content optimization. It greatly shortens the production cycle of aggregation pages, reduces manpower and time costs, and realizes the intelligent implementation of content from publishing, layering, tagging to inclusion.

[0060] Specifically, one of the first problems that content aggregation needs to solve is how to tag massive amounts of untagged content. Compared with solutions in related technologies, this embodiment considers the importance of tags to content from a global perspective. Based on the analysis of word associations, this embodiment designs a voting mechanism between words, introducing the most basic associations to the original untagged content. Based on these associations, this embodiment constructs a large content graph of the content community with words as nodes and relationships as edges, iteratively calculating the importance of words to content from a global perspective. Specifically, content aggregation / topic extraction schemes in related technologies usually use random combinations of popular keywords as topics, but this often leads to semantic inconsistencies and inaccurate wording. To ensure the semantic rationality of generated topics, this embodiment starts from the content of the target community and constructs a semantic reasoning mechanism based on large-scale association analysis and probabilistic graphical models. Specifically, when generating topics, this embodiment not only considers whether the topic is reasonable for the current content, but also measures whether the current topic is a general expression from a global perspective, thereby ensuring the readability and semantic rationality of the topics. Furthermore, to ensure the semantic rationality of the theme and the theme consistency of the aggregation page, this embodiment integrates semantic reasoning into the theme generation process, ultimately forming a fully intelligent end-to-end process that provides automated text tagging, content aggregation, and hierarchical indexing capabilities for SEO content optimization.

[0061] Based on the above steps, this embodiment completes the process from a content collection without hierarchical structure and topic division to tagging and topic aggregation, and then to topic generation. Through hierarchical inclusion, this embodiment constructs a multi-level inclusion structure, including the original unhierarchical content according to different topics into corresponding channel pages / aggregation pages. While ensuring the consistency of topics and semantics on the same page, it facilitates users to expand their reading on content related to the same topic, and also brings richer and higher-quality hyperlinks / backlinks for SEO traffic growth.

[0062] This embodiment incorporates a final manual verification mechanism. The classification results of multiple target articles are manually verified and adjusted to ensure the accuracy of the article topic classification. Besides SEO scenarios, this embodiment can also be used in intelligent search, text topic extraction, text clustering, and other fields.

[0063] Secondly, such as Figure 2As shown, embodiments of the present invention also provide an article topic extraction device, comprising: an extraction unit for extracting keywords from multiple articles, each article including multiple keywords; a first determining unit for determining at least one association combination for each article based on its corresponding multiple keywords, each association combination including at least two keywords arranged in order within the article; a second determining unit for determining the probability of occurrence of each association combination based on the number of each association combination; and a third determining unit for determining the topic of a target article based on the probability of occurrence of each association combination and the association combination corresponding to the target article, wherein the target article is one of multiple articles. In determining the topic of a target article using the above-described article topic extraction device, the extraction unit first extracts keywords from multiple articles, extracting at least two keywords from each article. For each article, based on its corresponding keywords, the first determining unit determines at least one association combination, each association combination including at least two keywords arranged in order within the article. Then, the second determining unit determines the probability of occurrence of each association combination based on the number of occurrences of each association combination. Finally, the third determining unit determines the topic of the target article based on the probability of occurrence of each association combination and the association combination corresponding to the target article. In the above implementation process, for each article, at least one association combination is established based on its different keywords, ensuring that at least two keywords are associated. The probability of occurrence of each association combination is calculated. The higher the probability of occurrence of an association combination, the stronger the correlation between the two keywords included in that combination, and the more universal the keyword expression form used in that association combination. Ultimately, the topic determined by combining the occurrence probabilities of each association combination with the association combination corresponding to the target article better takes into account the correlation between keywords and the universality of the expression form, thus better representing the content of the target article. Compared with the method of determining the article topic by randomly combining keywords in related technologies, this method can effectively improve the accuracy of topic determination and solve the problem of poor accuracy in article topic determination in related technologies.

[0064] Specifically, the first determining unit is used to: for each keyword, generate an association combination between it and each subsequent keyword; or, the first determining unit includes a first determining module and a generating module: the first determining module is used to determine a span threshold for any keyword; the generating module is used to generate an association combination between the keyword and each subsequent target keyword, wherein the number of keywords between the keyword and the target keyword is less than or equal to the span threshold; the first determining module is used to: determine the preset threshold as the span threshold when the number of subsequent keywords is greater than the preset threshold; determine the number of subsequent keywords as the span threshold when the number of subsequent keywords is less than the preset threshold, wherein the subsequent keywords are keywords in the article that follow the keyword in the association combination to be determined;

[0065] The second determining unit is used to: use multiple keywords extracted from multiple articles as vertices and the association relationships between keywords represented by multiple association combinations as edges, and use graph calculation to determine the probability of occurrence of each association combination;

[0066] The probability of an associated combination is the ratio of the number of associated combinations to the total number of related associated combinations. In this case, the first keyword in an associated combination is the same as the first keyword in a related associated combination, and the second keyword in an associated combination is either the same as or different from the second keyword in a related associated combination.

[0067] The third determining unit includes: a second determining module, used to determine one or more target combinations corresponding to the target article, wherein the target combination is a combination whose probability of occurrence is greater than or equal to a preset probability among the associated combinations corresponding to the target article; and a third determining module, used to determine the theme of the target article, wherein the theme includes keywords corresponding to one or more target combinations.

[0068] The article topic extraction device also includes: a fourth determining unit, used to perform semantic analysis on the topics of multiple target articles after determining the topic of the target article based on the occurrence probability of each association combination and the association combination corresponding to the target article, and to determine the semantically unreasonable topics; and a removal unit, used to remove the semantically unreasonable topics from the topics of multiple target articles.

[0069] The article topic extraction device also includes a classification unit, which, after determining the topic of the target article based on the occurrence probability of each association combination and the association combination corresponding to the target article, classifies multiple target articles according to the topic of each target article;

[0070] The classification unit is used to classify multiple target articles according to different classification levels based on the topic of each target article. Among them, multiple target articles in the same category have some keywords or related combinations. As the classification level decreases, the number of keywords or related combinations corresponding to the target articles increases.

[0071] The extraction unit includes: an acquisition module, used to acquire at least part of the text content of any article; a segmentation module, used to segment the text content according to a preset word segmentation logic to obtain multiple words; and a removal module, used to remove some words from the multiple words according to preset filtering rules to obtain multiple keywords.

[0072] In addition, embodiments of the present invention also provide a non-volatile storage medium, wherein the non-volatile storage medium includes a stored program, wherein the program controls the device where the non-volatile storage medium is located to execute the above-described article topic extraction method during program execution.

[0073] Furthermore, embodiments of the present invention also provide a processor, wherein the processor is used to run a program, wherein the program executes the above-described article topic extraction method during runtime.

[0074] Finally, embodiments of the present invention also provide an article topic extraction device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described article topic extraction method.

[0075] The sequence numbers of the above embodiments of the present invention are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. Moreover, the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in a different order than that shown here.

[0076] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0077] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0078] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0079] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0080] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0081] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for extracting the theme of an article, comprising: Keywords are extracted from multiple articles, with each article containing multiple of the keywords. For each article, at least one association combination is determined based on its corresponding multiple keywords, the association combination including at least two keywords, the at least two keywords being arranged in order within the article; The probability of occurrence of each of the aforementioned association combinations is determined based on the number of such combinations. The topic of the target article is determined based on the occurrence probability of each of the aforementioned association combinations and the association combination corresponding to the target article, wherein the target article is one of the plurality of articles; For each article, based on its corresponding keywords, at least one associated combination is determined, including: For any one of the keywords, determine the span threshold; Each keyword is paired with each subsequent target keyword to generate an association combination, wherein the number of keywords between the keyword and the target keywords is less than or equal to the span threshold.

2. The article topic extraction method according to claim 1, wherein, For each article, based on its corresponding keywords, at least one associated combination is determined, including: For each keyword, generate an association combination between it and each of the subsequent keywords.

3. The article topic extraction method according to claim 1, wherein, For any one of the aforementioned keywords, determining the span threshold includes: If the number of subsequent keywords exceeds a preset threshold, the preset threshold is determined to be the span threshold; If the number of subsequent keywords is less than the preset threshold, the number of subsequent keywords is determined as the span threshold, wherein the subsequent keywords are keywords in the article that follow the keywords of the association combination to be determined.

4. The article topic extraction method according to claim 1, wherein, Determining the probability of occurrence of each of the aforementioned association combinations based on their quantity includes: Using the keywords extracted from the multiple articles as vertices and the association relationships between the keywords represented by the multiple association combinations as edges, the probability of occurrence of each association combination is determined by graph computation.

5. The article topic extraction method according to claim 1, wherein, The probability of occurrence of the associated combination is the ratio of the number of the associated combination to the number of all related associated combinations, wherein the first keyword in the associated combination is the same as the first keyword in the related associated combination, and the second keyword in the associated combination is the same as or different from the second keyword in the related associated combination.

6. The article topic extraction method according to claim 1, wherein, Based on the occurrence probability of each of the aforementioned association combinations and the association combination corresponding to the target article, the topic of the target article is determined to include: Determine one or more target combinations corresponding to the target article, wherein the target combination is a combination whose probability of occurrence is greater than or equal to a preset probability among the associated combinations corresponding to the target article; The topic of the target article is determined, and the topic includes one or more keywords corresponding to the target combination.

7. The article topic extraction method according to claim 1, wherein, After determining the topic of the target article based on the occurrence probability of each association combination and the association combination corresponding to the target article, the article topic extraction method further includes: Semantic analysis is performed on the topics of multiple target articles to identify topics with semantically unreasonable content. Remove semantically unreasonable topics from the topics of the multiple target articles.

8. The article topic extraction method according to any one of claims 1 to 7, wherein, After determining the topic of the target article based on the occurrence probability of each association combination and the association combination corresponding to the target article, the article topic extraction method further includes: Based on the themes described in each of the target articles, the target articles are categorized.

9. The article topic extraction method according to claim 8, wherein, Based on the themes of each of the target articles, the multiple target articles are categorized as follows: Based on the themes of each target article, multiple target articles are classified according to different classification levels. Among them, multiple target articles in the same category have some of the same keywords or related combinations. As the classification level decreases, the number of keywords or related combinations corresponding to the target articles increases.

10. The article topic extraction method according to any one of claims 1 to 7, wherein, Keywords were extracted from multiple articles, with each article containing multiple keywords, including: For any of the articles mentioned, obtain at least a portion of the text content of the article; The text content is divided into multiple words according to a preset word segmentation logic; According to preset filtering rules, some of the words are removed from the multiple words to obtain multiple keywords.

11. An article topic extraction device, comprising: An extraction unit is used to extract keywords from multiple articles, each of which includes multiple keywords; The first determining unit is configured to, for each article, determine at least one association combination based on its corresponding plurality of keywords, wherein the association combination includes at least two keywords, and the at least two keywords are arranged in order within the article; The second determining unit is used to determine the probability of occurrence of each of the association combinations based on the number of each association combination; The third determining unit is used to determine the topic of the target article based on the occurrence probability of each of the association combinations and the association combination corresponding to the target article, wherein the target article is one of the plurality of articles; The first determining unit includes a first determining module and a generating module: the first determining module is used to determine a span threshold for any one of the keywords; the generating module is used to generate an association combination between the keyword and each subsequent target keyword, wherein the number of keywords between the keyword and the target keyword is less than or equal to the span threshold.

12. The article topic extraction device according to claim 11, wherein, The first determining unit is configured to: for each keyword, generate a pairwise association combination between it and each of the subsequent keywords; or, The first determining module is configured to: determine the preset threshold as the span threshold when the number of subsequent keywords is greater than the preset threshold; and determine the number of subsequent keywords as the span threshold when the number of subsequent keywords is less than the preset threshold, wherein the subsequent keywords are keywords in the article that follow the keywords of the association combination to be determined; The second determining unit is used to: use the multiple keywords extracted from the multiple articles as vertices and the association relationships between the keywords represented by the multiple association combinations as edges, and use graph computing to determine the probability of occurrence of each association combination; The probability of occurrence of the associated combination is the ratio of the number of the associated combination to the number of all related associated combinations, wherein the first keyword in the associated combination is the same as the first keyword in the related associated combination, and the second keyword in the associated combination is the same as or different from the second keyword in the related associated combination; The third determining unit includes: a second determining module, configured to determine one or more target combinations corresponding to the target article, wherein the target combination is a combination whose probability of occurrence is greater than or equal to a preset probability among the associated combinations corresponding to the target article; and a third determining module, configured to determine the theme of the target article, wherein the theme includes one or more keywords corresponding to the target combination. The article topic extraction device further includes: a fourth determining unit, used to perform semantic analysis on the topics of multiple target articles after determining the topic of the target article based on the occurrence probability of each association combination and the association combination corresponding to the target article, and to determine the topics that are semantically unreasonable; and a removal unit, used to remove the topics that are semantically unreasonable from the topics of multiple target articles. The article topic extraction device further includes: a classification unit, used to classify multiple target articles according to the topic of each target article after determining the topic of the target article based on the occurrence probability of each association combination and the association combination corresponding to the target article; The classification unit is used to classify multiple target articles according to different classification levels based on the topic of each target article, wherein some of the keywords or association combinations of multiple target articles in the same category are the same, and as the classification level decreases, the number of keywords or association combinations corresponding to the target article increases. The extraction unit includes: an acquisition module, used to acquire at least a portion of the text content of any article; a segmentation module, used to segment the text content according to a preset word segmentation logic to obtain multiple words; and a removal module, used to remove some of the words from the multiple words according to a preset filtering rule to obtain multiple keywords.

13. A non-volatile storage medium, wherein, The non-volatile storage medium includes a stored program, wherein, when the program is executed, it controls the device where the non-volatile storage medium is located to perform the article topic extraction method according to any one of claims 1 to 10.

14. A processor, wherein, The processor is used to run a program, wherein the program executes the article topic extraction method according to any one of claims 1 to 10.

15. An article topic extraction device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, When the processor executes the computer program, it implements the article topic extraction method according to any one of claims 1 to 11.

Images