A method, apparatus, device, and medium for event aggregation based on feature association.
By constructing and updating the event set through single-pass clustering of the title and summary of the event text using feature association, the problem of unsatisfactory accuracy and recall in the existing technology of event aggregation is solved, and more efficient and accurate event aggregation is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- E FUND MANAGEMENT CO LTD
- Filing Date
- 2026-01-30
- Publication Date
- 2026-05-26
AI Technical Summary
Existing event aggregation methods cannot effectively deal with clickbait headlines or diverse descriptions of the same event in event texts, resulting in unsatisfactory accuracy and recall rates, and failing to meet users' needs for continuous and stable tracking.
By using a feature-based event aggregation method, the title and summary of the event text are segmented and stop words are removed. A single-pass clustering algorithm is used to calculate similarity, construct and update the event set, merge historical and aggregated event clusters, and perform clustering by combining title and summary features, thereby reducing the number of clustering operations and improving accuracy.
It improves the accuracy and efficiency of event aggregation, effectively handles clickbait headlines or diverse descriptions of the same event, and ensures the continuity and stability of event aggregation.
Smart Images
Figure CN122087110A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of text classification and analysis, and in particular to an event aggregation method, apparatus, device, and medium based on feature association. Background Technology
[0002] With the rapid development of internet information technology, news, announcements, social media, and other information sources disclose massive amounts of public information. Each piece of public information is an independent event, with different themes and content. Users find it inconvenient to browse current hot topics, related events of interest, or similar events in a concentrated manner. Therefore, it is necessary to discover, classify, and aggregate these events to help users track events continuously and consistently.
[0003] Currently, with the practical application of artificial intelligence represented by Large Language Models (LLM), event discovery and aggregation typically involves using LLM to summarize a tagging system for various event types. This is then achieved through simple keyword matching, basic text similarity algorithms, or by directly applying LLM, utilizing only certain features of event information, such as titles, summaries, or keywords, to categorize and aggregate events into corresponding tagging systems, aiming to automate event information processing. However, these event classification and aggregation methods are ineffective in dealing with clickbait titles or diverse descriptions of the same event in current event texts. This results in unsatisfactory accuracy and recall rates for event discovery and aggregation, failing to meet actual user needs and hindering continuous and stable event tracking. Therefore, improving the accuracy of event aggregation to meet actual user needs remains a pressing technical problem that needs to be solved. Summary of the Invention
[0004] This application provides a method, apparatus, device, and medium for event aggregation based on feature association, in order to solve the technical problem that the accuracy of existing event aggregation needs to be improved.
[0005] According to a first aspect of the embodiments of this application, an event aggregation method based on feature association is provided, comprising: Obtain multiple event texts and multiple historical event clusters to be aggregated; each event text includes a corresponding title and summary; each historical event cluster includes a corresponding event identifier; Based on single-pass clustering, clustering operations are performed on the title and summary of each event text, using the multiple historical event clusters and their corresponding event identifiers as a basis, to determine the title identifier and summary identifier of each event text; Iterate through all summary identifiers, process each event text to obtain multiple aggregated event clusters; in each round of traversal, select the corresponding event text based on the summary identifier of the current traversal, construct the current event set, and update the current event set according to the title identifiers of all event texts in the current event set to obtain the aggregated event cluster corresponding to the summary identifier of the current traversal. Based on the title identifier and the summary identifier, the multiple historical event clusters and the multiple aggregated event clusters are merged and updated to obtain multiple updated event clusters, thereby completing event aggregation.
[0006] This application first obtains multiple event texts, including titles and summaries, and multiple historical event clusters, including event identifiers, to be aggregated. Then, based on single-pass clustering, using multiple historical event clusters and their corresponding event identifiers, it performs clustering on the titles and summaries of each event text to determine the corresponding title and summary identifiers. Next, it iterates through all summary identifiers to process each event text, resulting in multiple aggregated event clusters. In each iteration, it first selects event texts by summary identifiers to construct the current event set, then updates the current event set by associating the title identifiers of all event texts within the current event set, thus obtaining aggregated event clusters. Finally, based on the title and summary identifiers, it merges and updates multiple historical event clusters and multiple aggregated event clusters to obtain multiple more... New event clusters are used to complete event aggregation. By simultaneously processing both title and summary event text features, the insufficient processing dimensions caused by processing only one aspect, as in existing methods, are avoided, thereby improving the accuracy of subsequent event aggregation. By performing single-pass clustering with multiple historical event clusters, the number of clustering operations can be reduced, and clustering efficiency can be improved, thus ensuring the efficiency of event aggregation. By first constructing the current event set through the summary identifiers during the processing of summary identifiers, and then updating the current event set by associating the title identifiers of all event texts in the current event set to obtain aggregated event clusters, the two event text features of title and summary can be associated, effectively handling the situation of "clickbait" titles or diverse descriptions of the same event, thereby improving the accuracy of event aggregation.
[0007] In some embodiments of this application, the step of performing clustering operations on the title and summary of each event text based on single-pass clustering, using the multiple historical event clusters and their corresponding event identifiers, to determine the title identifier and summary identifier of each event text, specifically includes: Each event text is segmented and stop words are removed from its title and summary to obtain the title keyword set and summary keyword set for each event text. Based on the multiple historical event clusters and their corresponding event identifiers, single-pass clustering is performed on the title and abstract of each event text according to the title keyword set and abstract keyword set of each event text to determine the title identifier and abstract identifier of each event text.
[0008] This application first segments and removes stop words from the title and abstract of each event text to obtain a set of title keywords and a set of abstract keywords. Then, based on multiple historical event clusters, it performs single-pass clustering on the title and abstract to determine the title identifier and abstract identifier of each event text. By processing the two types of event text features, title and abstract, simultaneously, it avoids the insufficient processing dimensions caused by processing only one aspect, as in existing methods, thereby improving the accuracy of subsequent event aggregation. At the same time, performing single-pass clustering based on multiple historical event clusters can reduce the number of clustering operations and improve clustering efficiency, thus ensuring the efficiency of event aggregation.
[0009] In some embodiments of this application, the step of determining the title identifier and summary identifier of each event text by performing single-pass clustering on the title and summary of each event text based on the plurality of historical event clusters and their corresponding event identifiers, according to the title keyword set and summary keyword set of each event text, specifically includes: Traverse all event texts, and based on the multiple historical event clusters and their corresponding event identifiers, perform a single-pass clustering of the title of each event text to determine the title identifier of each event text. Traverse all event texts, and based on the multiple historical event clusters and their corresponding event identifiers, perform a single-pass clustering on the summary of each event text to determine the summary identifier of each event text. In each round of traversal during the single-pass clustering of the title or abstract, the similarity between the current traversed event text and each historical event cluster is calculated based on the title keyword set or abstract keyword set of the current traversed event text, and the title identifier or abstract identifier of the current traversed event text is determined based on the similarity.
[0010] This application first traverses all event texts, and based on multiple historical event clusters, performs single-pass clustering on the title and summary respectively. In each round of single-pass clustering, the similarity between the currently traversed event text and each historical event cluster is calculated, and the title identifier or summary identifier of the currently traversed event text is determined based on the similarity. By performing single-pass clustering based on multiple historical event clusters and determining the corresponding title identifier or summary identifier based on similarity in the single-pass clustering process, the number of clustering times can be reduced, and the clustering efficiency and accuracy can be improved, thereby ensuring the efficiency and accuracy of event aggregation.
[0011] In some embodiments of this application, determining the title identifier or summary identifier of the currently traversed event text based on the similarity specifically includes: If the similarity between the currently traversed event text and any historical event cluster is greater than a preset similarity threshold, then the event identifier corresponding to the one with the highest similarity among all historical event clusters that are greater than the similarity threshold will be used as the title identifier or summary identifier of the currently traversed event text. Otherwise, construct a first event cluster based on the currently traversed event text, generate a corresponding event identifier for the first event cluster, use the event identifier as the title identifier or summary identifier of the currently traversed event text, and add the first event cluster to the historical event cluster at the start of the next traversal.
[0012] This application, when the similarity between the currently traversed event text and any historical event cluster is greater than a similarity threshold, uses the event identifier of the historical event cluster with the highest similarity among all historical event clusters with similarity greater than the similarity threshold as the title identifier or summary identifier of the currently traversed event text. Otherwise, it constructs a first event cluster based on the currently traversed event text and generates a corresponding event identifier, uses the event identifier as the title identifier or summary identifier of the currently traversed event text, and adds the first event cluster to the historical event cluster at the start of the next traversal. By comparing the similarity with multiple historical event clusters and using the corresponding event identifier as the title identifier or summary identifier, the clustering accuracy can be improved, thereby ensuring the accuracy of subsequent event aggregation.
[0013] In some embodiments of this application, the multiple event texts belong to a set of events to be aggregated; the step of selecting the corresponding event text based on the summary identifier of the current traversal, constructing the current event set, and updating the current event set according to the title identifiers of all event texts in the current event set to obtain the aggregated event cluster corresponding to the summary identifier of the current traversal specifically includes: Based on the summary identifier of the current traversal, the set of events to be aggregated is filtered to construct the current event set; wherein, the summary identifier of all event texts in the current event set is the summary identifier of the current traversal; Based on the title identifier of each event text in the current event set, multiple first event texts are selected from the event set to be aggregated; wherein the title identifier of each first event text is the same as the title identifier of any event text in the current event set; The multiple first event texts are incorporated into the current event set to update the current event set, thereby obtaining the aggregated event cluster corresponding to the summary identifier of the current traversal.
[0014] This application first filters the set of events to be aggregated based on the summary identifier of the current traversal, constructs the current event set, and then selects multiple first event texts from the set of events to be aggregated based on the title identifier of each event text in the current event set. These multiple first event texts are then incorporated into the current event set to obtain an aggregated event cluster. This approach can associate the two event text features of title and summary, effectively handling situations of "clickbait" titles or diverse descriptions of the same event, thereby improving the accuracy of event aggregation.
[0015] In some embodiments of this application, it further includes: Remove all event texts and multiple first event texts from the current event set from the event set to be aggregated.
[0016] This application removes all event texts and multiple first event texts from the event set to be aggregated in the current event set to avoid incorrect duplicate selections during subsequent traversal, thereby avoiding incorrect crossover of event clusters and subsequent incorrect aggregation of event clusters.
[0017] In some embodiments of this application, the step of merging and updating the plurality of historical event clusters and the plurality of aggregated event clusters according to the title identifier and the summary identifier to obtain a plurality of updated event clusters, thereby completing event aggregation, specifically includes: Based on the summary identifier corresponding to each aggregated event cluster and the title identifier of all event texts in each aggregated event cluster, construct an event identifier set corresponding to each aggregated event cluster, and take the set element with the smallest value in the event identifier set of each aggregated event cluster as the event identifier of the corresponding aggregated event cluster; Based on the event identifier, the multiple historical event clusters and the multiple aggregated event clusters are merged and updated to obtain multiple updated event clusters, thereby completing the event aggregation of the multiple event texts.
[0018] This application first constructs a corresponding set of event identifiers based on the summary identifier of each aggregated event cluster and the title identifier of all event texts in each aggregated event cluster, and then determines the event identifier of the aggregated event cluster. Then, it merges and updates multiple historical event clusters and multiple aggregated event clusters based on the event identifiers to complete event aggregation. This method can simply and accurately merge and update aggregated event clusters that have been clustered, thereby ensuring the efficiency and accuracy of event aggregation.
[0019] According to a second aspect of the embodiments of this application, an event aggregation device based on feature association is provided, including an event data acquisition module, an event identifier determination module, an event traversal aggregation module, and an event aggregation update module; The event data acquisition module is used to acquire multiple event texts and multiple historical event clusters to be aggregated; wherein, each event text includes a corresponding title and summary; and each historical event cluster includes a corresponding event identifier; The event identifier determination module is used to perform clustering operations on the title and summary of each event text based on single-pass clustering, using the multiple historical event clusters and their corresponding event identifiers as a basis, to determine the title identifier and summary identifier of each event text. The event traversal aggregation module is used to traverse all summary identifiers, process each event text, and obtain multiple aggregated event clusters. In each round of traversal, the corresponding event text is selected based on the summary identifier of the current traversal, the current event set is constructed, and the current event set is updated according to the title identifiers of all event texts in the current event set to obtain the aggregated event cluster corresponding to the summary identifier of the current traversal. The event aggregation and update module is used to merge and update the multiple historical event clusters and the multiple aggregated event clusters according to the title identifier and the summary identifier, so as to obtain multiple updated event clusters and complete the event aggregation.
[0020] In some embodiments of this application, the event identifier determination module includes a keyword filtering unit and an identifier clustering determination unit; The keyword filtering unit is used to segment and remove stop words from the title and summary of each event text, respectively, to obtain the title keyword set and summary keyword set corresponding to each event text; The identifier clustering determination unit is used to determine the title identifier and summary identifier of each event text by performing single-pass clustering on the title and summary of each event text based on the multiple historical event clusters and their corresponding event identifiers, according to the title keyword set and summary keyword set of each event text.
[0021] In some embodiments of this application, the identifier clustering determination unit includes a title clustering subunit and a summary clustering subunit; The title clustering subunit is used to traverse all event texts, and based on the multiple historical event clusters and their corresponding event identifiers, to perform a single-pass clustering of the title of each event text to determine the title identifier of each event text. The summary clustering subunit is used to traverse all event texts, and based on the multiple historical event clusters and their corresponding event identifiers, perform single-pass clustering on the summary of each event text to determine the summary identifier of each event text. In each round of traversal during the single-pass clustering of the title or abstract, the similarity between the current traversed event text and each historical event cluster is calculated based on the title keyword set or abstract keyword set of the current traversed event text, and the title identifier or abstract identifier of the current traversed event text is determined based on the similarity.
[0022] In some embodiments of this application, determining the title identifier or summary identifier of the currently traversed event text based on the similarity specifically includes: If the similarity between the currently traversed event text and any historical event cluster is greater than a preset similarity threshold, then the event identifier corresponding to the one with the highest similarity among all historical event clusters that are greater than the similarity threshold will be used as the title identifier or summary identifier of the currently traversed event text. Otherwise, construct a first event cluster based on the currently traversed event text, generate a corresponding event identifier for the first event cluster, use the event identifier as the title identifier or summary identifier of the currently traversed event text, and add the first event cluster to the historical event cluster at the start of the next traversal.
[0023] In some embodiments of this application, the multiple event texts belong to a set of events to be aggregated; the step of selecting the corresponding event text based on the summary identifier of the current traversal, constructing the current event set, and updating the current event set according to the title identifiers of all event texts in the current event set to obtain the aggregated event cluster corresponding to the summary identifier of the current traversal specifically includes: Based on the summary identifier of the current traversal, construct the current event set; wherein, the summary identifier of all event texts in the current event set is the summary identifier of the current traversal; Based on the title identifier of each event text in the current event set, multiple first event texts are selected from the event set to be aggregated; wherein the title identifier of each first event text is the same as the title identifier of any event text in the current event set; The multiple first event texts are incorporated into the current event set to update the current event set, thereby obtaining the aggregated event cluster corresponding to the summary identifier of the current traversal.
[0024] In some embodiments of this application, the method further includes: removing all event texts and the multiple first event texts from the event set to be aggregated.
[0025] In some embodiments of this application, the event aggregation and update module includes an event cluster identifier determination unit and an event cluster update and aggregation unit; The event cluster identifier determination unit is used to construct an event identifier set corresponding to each aggregated event cluster based on the summary identifier corresponding to each aggregated event cluster and the title identifier of all event texts of each aggregated event cluster, and to take the set element with the smallest value in the event identifier set of each aggregated event cluster as the event identifier of the corresponding aggregated event cluster. The event cluster update aggregation unit is used to merge and update the multiple historical event clusters and the multiple aggregated event clusters according to the event identifier to obtain multiple updated event clusters, so as to complete the event aggregation of the multiple event texts.
[0026] This application first obtains multiple event texts, including titles and summaries, and multiple historical event clusters, including event identifiers, to be aggregated. Then, based on single-pass clustering, using multiple historical event clusters and their corresponding event identifiers, it performs clustering on the titles and summaries of each event text to determine the corresponding title and summary identifiers. Next, it iterates through all summary identifiers to process each event text, resulting in multiple aggregated event clusters. In each iteration, it first selects event texts by summary identifiers to construct the current event set, then updates the current event set by associating the title identifiers of all event texts within the current event set, thus obtaining aggregated event clusters. Finally, based on the title and summary identifiers, it merges and updates multiple historical event clusters and multiple aggregated event clusters to obtain multiple more... New event clusters are used to complete event aggregation. By simultaneously processing both title and summary event text features, the insufficient processing dimensions caused by processing only one aspect, as in existing methods, are avoided, thereby improving the accuracy of subsequent event aggregation. By performing single-pass clustering with multiple historical event clusters, the number of clustering operations can be reduced, and clustering efficiency can be improved, thus ensuring the efficiency of event aggregation. By first constructing the current event set through the summary identifiers during the processing of summary identifiers, and then updating the current event set by associating the title identifiers of all event texts in the current event set to obtain aggregated event clusters, the two event text features of title and summary can be associated, effectively handling the situation of "clickbait" titles or diverse descriptions of the same event, thereby improving the accuracy of event aggregation.
[0027] According to a third aspect of the embodiments of this application, a computer device is provided, comprising: a processor; a memory; and a computer program stored in the memory and configured to be executed by the processor; wherein the processor executes the computer program to implement a feature-based event aggregation method as described in this application.
[0028] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, the computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute a feature-association-based event aggregation method as described in this application. Attached Figure Description
[0029] Figure 1 This is a flowchart illustrating an event aggregation method based on feature association, as shown in some embodiments of this application. Figure 2 This is a module structure diagram of an event aggregation device based on feature association, as shown in some embodiments of this application. Detailed Implementation
[0030] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below in conjunction with the accompanying drawings are exemplary and are only used to explain some embodiments of this application, and should not be construed as limiting the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments shown in this application without inventive effort are within the protection scope of this application.
[0031] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, unless otherwise explicitly specified, "a plurality of" or "several" means two or more.
[0032] Currently, simple keyword matching, basic text similarity algorithms, or event classification and aggregation methods that directly apply large language models are insufficient to effectively address clickbait headlines or diverse descriptions of the same event in current event texts. This results in unsatisfactory accuracy and recall rates for event discovery and aggregation, failing to meet actual user needs and hindering continuous and stable event tracking. Therefore, improving the accuracy of event aggregation to meet actual user needs remains a pressing technical problem that needs to be solved.
[0033] Based on the above technical background, please refer to Figure 1 This application provides an event aggregation method based on feature association, including steps S101 to S104, each step as follows: Step S101: Obtain multiple event texts and multiple historical event clusters to be aggregated; wherein, each event text includes a corresponding title and summary; each historical event cluster includes a corresponding event identifier.
[0034] Specifically, an event cluster is represented as a set of events that the algorithm considers sufficiently similar. All event texts within each historical event cluster share the event identifier of the historical event cluster. The earliest event text in each historical event cluster is considered the cluster center. The title of an event text is the title content of the corresponding event text, while the summary of an event text is represented as a summary of the corresponding event text with a maximum of 200 characters. It can be a summary of the corresponding event text through a preset model, or it can be the first 200 characters of the main text of the corresponding event text.
[0035] Specifically, the event identifier, title identifier, and summary identifier are all numerical constants, usually integer constants; the event identifiers are unique to each other; the event identifier is assigned when the corresponding event cluster is created and updated when the event cluster is merged; the preferred implementation of assigning the event identifier is to use the timestamp when the corresponding event cluster is created as the event identifier, or to generate a unique event identifier based on the timestamp; the title identifier and summary identifier are obtained from historical event clusters through a single-pass clustering operation or assigned by creating a new event cluster.
[0036] Step S102: Based on single-pass clustering, clustering operations are performed on the title and summary of each event text based on the multiple historical event clusters and their corresponding event identifiers to determine the title identifier and summary identifier of each event text.
[0037] Specifically, the Single Pass clustering algorithm processes one data point at a time in the order of input for a sequentially arriving data stream. Based on the degree of matching between the current data and existing clusters (usually determined by similarity), the current data is assigned to an existing category or a new data class is created. It is suitable for incremental and dynamic clustering analysis of streaming data, and its characteristic of performing clustering only once makes it highly efficient in terms of time.
[0038] In some embodiments of this application, the step of performing clustering operations on the title and summary of each event text based on single-pass clustering, using the multiple historical event clusters and their corresponding event identifiers, to determine the title identifier and summary identifier of each event text, specifically includes: Each event text is segmented and stop words are removed from its title and summary to obtain the title keyword set and summary keyword set for each event text. Based on the multiple historical event clusters and their corresponding event identifiers, single-pass clustering is performed on the title and abstract of each event text according to the title keyword set and abstract keyword set of each event text to determine the title identifier and abstract identifier of each event text.
[0039] This application first segments and removes stop words from the title and abstract of each event text to obtain a set of title keywords and a set of abstract keywords. Then, based on multiple historical event clusters, it performs single-pass clustering on the title and abstract to determine the title identifier and abstract identifier of each event text. By processing the two types of event text features, title and abstract, simultaneously, it avoids the insufficient processing dimensions caused by processing only one aspect, as in existing methods, thereby improving the accuracy of subsequent event aggregation. At the same time, performing single-pass clustering based on multiple historical event clusters can reduce the number of clustering operations and improve clustering efficiency, thus ensuring the efficiency of event aggregation.
[0040] In some embodiments of this application, the step of determining the title identifier and summary identifier of each event text by performing single-pass clustering on the title and summary of each event text based on the plurality of historical event clusters and their corresponding event identifiers, according to the title keyword set and summary keyword set of each event text, specifically includes: Traverse all event texts, and based on the multiple historical event clusters and their corresponding event identifiers, perform a single-pass clustering of the title of each event text to determine the title identifier of each event text. Traverse all event texts, and based on the multiple historical event clusters and their corresponding event identifiers, perform a single-pass clustering on the summary of each event text to determine the summary identifier of each event text. In each round of traversal during the single-pass clustering of the title or abstract, the similarity between the current traversed event text and each historical event cluster is calculated based on the title keyword set or abstract keyword set of the current traversed event text, and the title identifier or abstract identifier of the current traversed event text is determined based on the similarity.
[0041] Specifically, when calculating the similarity between the currently traversed event text and each historical event cluster, the implementation methods of similarity include cosine similarity, Jaccard similarity, Pearson correlation coefficient, term frequency-inverse document frequency, etc., with Jaccard similarity being the preferred implementation method.
[0042] This application first traverses all event texts, and based on multiple historical event clusters, performs single-pass clustering on the title and summary respectively. In each round of single-pass clustering, the similarity between the currently traversed event text and each historical event cluster is calculated, and the title identifier or summary identifier of the currently traversed event text is determined based on the similarity. By performing single-pass clustering based on multiple historical event clusters and determining the corresponding title identifier or summary identifier based on similarity in the single-pass clustering process, the number of clustering times can be reduced, and the clustering efficiency and accuracy can be improved, thereby ensuring the efficiency and accuracy of event aggregation.
[0043] In some embodiments of this application, determining the title identifier or summary identifier of the currently traversed event text based on the similarity specifically includes: If the similarity between the currently traversed event text and any historical event cluster is greater than a preset similarity threshold, then the event identifier corresponding to the one with the highest similarity among all historical event clusters that are greater than the similarity threshold will be used as the title identifier or summary identifier of the currently traversed event text. Otherwise, construct a first event cluster based on the currently traversed event text, generate a corresponding event identifier for the first event cluster, use the event identifier as the title identifier or summary identifier of the currently traversed event text, and add the first event cluster to the historical event cluster at the start of the next traversal.
[0044] This application, when the similarity between the currently traversed event text and any historical event cluster is greater than a similarity threshold, uses the event identifier of the historical event cluster with the highest similarity among all historical event clusters with similarity greater than the similarity threshold as the title identifier or summary identifier of the currently traversed event text. Otherwise, it constructs a first event cluster based on the currently traversed event text and generates a corresponding event identifier, uses the event identifier as the title identifier or summary identifier of the currently traversed event text, and adds the first event cluster to the historical event cluster at the start of the next traversal. By comparing the similarity with multiple historical event clusters and using the corresponding event identifier as the title identifier or summary identifier, the clustering accuracy can be improved, thereby ensuring the accuracy of subsequent event aggregation.
[0045] Step S103: Traverse all summary identifiers, process each event text, and obtain multiple aggregated event clusters; wherein, in each round of traversal, select the corresponding event text based on the summary identifier of the current traversal, construct the current event set, and update the current event set according to the title identifiers of all event texts in the current event set, so as to obtain the aggregated event cluster corresponding to the summary identifier of the current traversal.
[0046] In some embodiments of this application, the multiple event texts belong to a set of events to be aggregated; the step of selecting the corresponding event text based on the summary identifier of the current traversal, constructing the current event set, and updating the current event set according to the title identifiers of all event texts in the current event set to obtain the aggregated event cluster corresponding to the summary identifier of the current traversal specifically includes: Based on the summary identifier of the current traversal, the set of events to be aggregated is filtered to construct the current event set; wherein, the summary identifier of all event texts in the current event set is the summary identifier of the current traversal; Based on the title identifier of each event text in the current event set, multiple first event texts are selected from the event set to be aggregated; wherein the title identifier of each first event text is the same as the title identifier of any event text in the current event set; The multiple first event texts are incorporated into the current event set to update the current event set, thereby obtaining the aggregated event cluster corresponding to the summary identifier of the current traversal.
[0047] Specifically, firstly, based on the summary identifier of the current traversal, select all event texts in the event set to be aggregated whose summary identifier is the same as the summary identifier of the current traversal, and construct the current event set. The current event set is characterized as a set of events that are sufficiently similar in terms of summary. Then, in the event set to be aggregated, retrieve all event texts that have the same title identifier as any event text in the current event set. These are the first event texts. Each retrieved first event text is characterized as being sufficiently similar in title to a certain event text in the current event set. Finally, all the obtained first event texts are added to the current event set to obtain the aggregated event cluster corresponding to the summary identifier of the current traversal. This cluster is characterized as a set of all event texts in the event set to be aggregated that are sufficiently similar in title or summary to the summary identifier of the current traversal.
[0048] This application first filters the set of events to be aggregated based on the summary identifier of the current traversal, constructs the current event set, and then selects multiple first event texts from the set of events to be aggregated based on the title identifier of each event text in the current event set. These multiple first event texts are then incorporated into the current event set to obtain an aggregated event cluster. This approach can associate the two event text features of title and summary, effectively handling situations of "clickbait" titles or diverse descriptions of the same event, thereby improving the accuracy of event aggregation.
[0049] In some embodiments of this application, it further includes: Remove all event texts and multiple first event texts from the current event set from the event set to be aggregated.
[0050] This application removes all event texts and multiple first event texts from the event set to be aggregated in the current event set to avoid incorrect duplicate selections during subsequent traversal, thereby avoiding incorrect crossover of event clusters and subsequent incorrect aggregation of event clusters.
[0051] Step S104: Based on the title identifier and the summary identifier, merge and update the multiple historical event clusters and the multiple aggregated event clusters to obtain multiple updated event clusters, thereby completing event aggregation.
[0052] In some embodiments of this application, the step of merging and updating the plurality of historical event clusters and the plurality of aggregated event clusters according to the title identifier and the summary identifier to obtain a plurality of updated event clusters, thereby completing event aggregation, specifically includes: Based on the summary identifier corresponding to each aggregated event cluster and the title identifier of all event texts in each aggregated event cluster, construct an event identifier set corresponding to each aggregated event cluster, and take the set element with the smallest value in the event identifier set of each aggregated event cluster as the event identifier of the corresponding aggregated event cluster; Based on the event identifier, the multiple historical event clusters and the multiple aggregated event clusters are merged and updated to obtain multiple updated event clusters, thereby completing the event aggregation of the multiple event texts.
[0053] Considering that event identifiers, summary identifiers, and title identifiers are actually integer constants, and each aggregated event cluster has multiple identifiers that can be selected as the representative of the corresponding aggregated event cluster, including the summary identifier of the aggregated event cluster and the title identifiers of all event texts within the aggregated event cluster, when determining the identifier representative of each aggregated event cluster, in order to ensure the order of the identifiers, it can be uniformly stipulated that the smallest identifier element in the event identifier set is taken as the event identifier of the current aggregated event cluster during merging, and event clusters (historical event clusters or aggregated event clusters) with the same event identifier are merged to obtain multiple updated event clusters.
[0054] This application first constructs a corresponding set of event identifiers based on the summary identifier of each aggregated event cluster and the title identifier of all event texts in each aggregated event cluster, and then determines the event identifier of the aggregated event cluster. Then, it merges and updates multiple historical event clusters and multiple aggregated event clusters based on the event identifiers to complete event aggregation. This method can simply and accurately merge and update aggregated event clusters that have been clustered, thereby ensuring the efficiency and accuracy of event aggregation.
[0055] Compared to existing technologies, this application first obtains multiple event texts including titles and summaries, and multiple historical event clusters including event identifiers. Then, based on single-pass clustering, using multiple historical event clusters and their corresponding event identifiers as a basis, it performs clustering on the titles and summaries of each event text to determine the corresponding title identifiers and summary identifiers. Next, it iterates through all summary identifiers to process each event text, obtaining multiple aggregated event clusters. In each iteration, it first selects event texts by summary identifiers to construct the current event set, then updates the current event set by associating the title identifiers of all event texts within the current event set, thus obtaining aggregated event clusters. Finally, based on the title identifiers and summary identifiers, it merges and updates multiple historical event clusters and multiple aggregated event clusters, resulting in... Multiple event clusters are updated to complete event aggregation; by processing both title and summary event text features simultaneously, the insufficient processing dimensions caused by processing only one aspect, as in existing methods, are avoided, thereby improving the accuracy of subsequent event aggregation; by performing single-pass clustering with multiple historical event clusters, the number of clustering times can be reduced, and clustering efficiency can be improved, thus ensuring event aggregation efficiency; by first constructing the current event set through summary identifiers in the process of traversing summary identifiers, and then updating the current event set by associating the title identifiers of all event texts in the current event set to obtain aggregated event clusters, the two event text features of title and summary can be associated, effectively handling the situation of "clickbait" titles or diverse descriptions of the same event, thereby improving the accuracy of event aggregation.
[0056] For a method corresponding to the one described above, please refer to [link to relevant documentation]. Figure 2 This application provides an event aggregation device based on feature association, including an event data acquisition module 210, an event identifier determination module 220, an event traversal aggregation module 230, and an event aggregation update module 240. The event data acquisition module 210 is used to acquire multiple event texts to be aggregated and multiple historical event clusters; wherein, each event text includes a corresponding title and summary; and each historical event cluster includes a corresponding event identifier; The event identifier determination module 220 is used to perform clustering operations on the title and summary of each event text based on single-pass clustering, using the multiple historical event clusters and their corresponding event identifiers as a basis, to determine the title identifier and summary identifier of each event text. The event traversal aggregation module 230 is used to traverse all summary identifiers, process each event text, and obtain multiple aggregated event clusters. In each round of traversal, the corresponding event text is selected based on the summary identifier of the current traversal, the current event set is constructed, and the current event set is updated according to the title identifiers of all event texts in the current event set to obtain the aggregated event cluster corresponding to the summary identifier of the current traversal. The event aggregation update module 240 is used to merge and update the multiple historical event clusters and the multiple aggregated event clusters according to the title identifier and the summary identifier to obtain multiple updated event clusters, so as to complete the event aggregation.
[0057] In some embodiments of this application, the event identifier determination module 220 includes a keyword filtering unit and an identifier clustering determination unit; The keyword filtering unit is used to segment and remove stop words from the title and summary of each event text, respectively, to obtain the title keyword set and summary keyword set corresponding to each event text; The identifier clustering determination unit is used to determine the title identifier and summary identifier of each event text by performing single-pass clustering on the title and summary of each event text based on the multiple historical event clusters and their corresponding event identifiers, according to the title keyword set and summary keyword set of each event text.
[0058] In some embodiments of this application, the identifier clustering determination unit includes a title clustering subunit and a summary clustering subunit; The title clustering subunit is used to traverse all event texts, and based on the multiple historical event clusters and their corresponding event identifiers, to perform a single-pass clustering of the title of each event text to determine the title identifier of each event text. The summary clustering subunit is used to traverse all event texts, and based on the multiple historical event clusters and their corresponding event identifiers, perform single-pass clustering on the summary of each event text to determine the summary identifier of each event text. In each round of traversal during the single-pass clustering of the title or abstract, the similarity between the current traversed event text and each historical event cluster is calculated based on the title keyword set or abstract keyword set of the current traversed event text, and the title identifier or abstract identifier of the current traversed event text is determined based on the similarity.
[0059] In some embodiments of this application, determining the title identifier or summary identifier of the currently traversed event text based on the similarity specifically includes: If the similarity between the currently traversed event text and any historical event cluster is greater than a preset similarity threshold, then the event identifier corresponding to the one with the highest similarity among all historical event clusters that are greater than the similarity threshold will be used as the title identifier or summary identifier of the currently traversed event text. Otherwise, construct a first event cluster based on the currently traversed event text, generate a corresponding event identifier for the first event cluster, use the event identifier as the title identifier or summary identifier of the currently traversed event text, and add the first event cluster to the historical event cluster at the start of the next traversal.
[0060] In some embodiments of this application, the multiple event texts belong to a set of events to be aggregated; the step of selecting the corresponding event text based on the summary identifier of the current traversal, constructing the current event set, and updating the current event set according to the title identifiers of all event texts in the current event set to obtain the aggregated event cluster corresponding to the summary identifier of the current traversal specifically includes: Based on the summary identifier of the current traversal, construct the current event set; wherein, the summary identifier of all event texts in the current event set is the summary identifier of the current traversal; Based on the title identifier of each event text in the current event set, multiple first event texts are selected from the event set to be aggregated; wherein the title identifier of each first event text is the same as the title identifier of any event text in the current event set; The multiple first event texts are incorporated into the current event set to update the current event set, thereby obtaining the aggregated event cluster corresponding to the summary identifier of the current traversal.
[0061] In some embodiments of this application, the method further includes: removing all event texts and the multiple first event texts from the event set to be aggregated.
[0062] In some embodiments of this application, the event aggregation and update module 240 includes an event cluster identifier determination unit and an event cluster update and aggregation unit; The event cluster identifier determination unit is used to construct an event identifier set corresponding to each aggregated event cluster based on the summary identifier corresponding to each aggregated event cluster and the title identifier of all event texts of each aggregated event cluster, and to take the set element with the smallest value in the event identifier set of each aggregated event cluster as the event identifier of the corresponding aggregated event cluster. The event cluster update aggregation unit is used to merge and update the multiple historical event clusters and the multiple aggregated event clusters according to the event identifier to obtain multiple updated event clusters, so as to complete the event aggregation of the multiple event texts.
[0063] This application first obtains multiple event texts, including titles and summaries, and multiple historical event clusters, including event identifiers, to be aggregated. Then, based on single-pass clustering, using multiple historical event clusters and their corresponding event identifiers, it performs clustering on the titles and summaries of each event text to determine the corresponding title and summary identifiers. Next, it iterates through all summary identifiers to process each event text, resulting in multiple aggregated event clusters. In each iteration, it first selects event texts by summary identifiers to construct the current event set, then updates the current event set by associating the title identifiers of all event texts within the current event set, thus obtaining aggregated event clusters. Finally, based on the title and summary identifiers, it merges and updates multiple historical event clusters and multiple aggregated event clusters to obtain multiple more... New event clusters are used to complete event aggregation. By simultaneously processing both title and summary event text features, the insufficient processing dimensions caused by processing only one aspect, as in existing methods, are avoided, thereby improving the accuracy of subsequent event aggregation. By performing single-pass clustering with multiple historical event clusters, the number of clustering operations can be reduced, and clustering efficiency can be improved, thus ensuring the efficiency of event aggregation. By first constructing the current event set through the summary identifiers during the processing of summary identifiers, and then updating the current event set by associating the title identifiers of all event texts in the current event set to obtain aggregated event clusters, the two event text features of title and summary can be associated, effectively handling the situation of "clickbait" titles or diverse descriptions of the same event, thereby improving the accuracy of event aggregation.
[0064] It should be understood that the apparatus provided in the embodiments of this application corresponds to the aforementioned method. The event aggregation apparatus based on feature association provided in the embodiments of this application can implement the event aggregation method based on feature association provided in any embodiment of this application.
[0065] Adaptively, embodiments of this application also provide a computer device and a computer-readable storage medium.
[0066] The computer device includes: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor; The processor, when executing the computer program, implements an event aggregation method based on feature association as described in this application.
[0067] The computer-readable storage medium stores multiple instructions adapted for loading by a processor to execute an event aggregation method based on feature association according to this application.
[0068] The above description represents some embodiments of this application, providing a further detailed explanation of the purpose, technical solution, and beneficial effects of this application. It should be understood that the above-described embodiments of this application should not be construed as limiting this application. In particular, any changes, modifications, equivalent substitutions, and variations made by those skilled in the art within the spirit and principles of this application should be included within the scope of protection of this application.
Claims
1. An event aggregation method based on feature association, characterized in that, include: Obtain multiple event texts and multiple historical event clusters to be aggregated; each event text includes a corresponding title and summary; each historical event cluster includes a corresponding event identifier; Based on single-pass clustering, clustering operations are performed on the title and summary of each event text, using the multiple historical event clusters and their corresponding event identifiers as a basis, to determine the title identifier and summary identifier of each event text; Iterate through all summary identifiers, process each event text to obtain multiple aggregated event clusters; in each round of traversal, select the corresponding event text based on the summary identifier of the current traversal, construct the current event set, and update the current event set according to the title identifiers of all event texts in the current event set to obtain the aggregated event cluster corresponding to the summary identifier of the current traversal. Based on the title identifier and the summary identifier, the multiple historical event clusters and the multiple aggregated event clusters are merged and updated to obtain multiple updated event clusters, thereby completing event aggregation.
2. The event aggregation method based on feature association according to claim 1, characterized in that, The single-pass clustering method, based on the multiple historical event clusters and their corresponding event identifiers, performs clustering operations on the title and summary of each event text to determine the title identifier and summary identifier of each event text. Specifically, this includes: Each event text is segmented and stop words are removed from its title and summary to obtain the title keyword set and summary keyword set for each event text. Based on the multiple historical event clusters and their corresponding event identifiers, single-pass clustering is performed on the title and abstract of each event text according to the title keyword set and abstract keyword set of each event text to determine the title identifier and abstract identifier of each event text.
3. The event aggregation method based on feature association according to claim 2, characterized in that, Based on the multiple historical event clusters and their corresponding event identifiers, and according to the title keyword set and summary keyword set of each event text, single-pass clustering is performed on the title and summary of each event text to determine the title identifier and summary identifier of each event text. Specifically, this includes: Traverse all event texts, and based on the multiple historical event clusters and their corresponding event identifiers, perform a single-pass clustering of the title of each event text to determine the title identifier of each event text. Traverse all event texts, and based on the multiple historical event clusters and their corresponding event identifiers, perform a single-pass clustering on the summary of each event text to determine the summary identifier of each event text. In each round of traversal during the single-pass clustering of the title or abstract, the similarity between the current traversed event text and each historical event cluster is calculated based on the title keyword set or abstract keyword set of the current traversed event text, and the title identifier or abstract identifier of the current traversed event text is determined based on the similarity.
4. The event aggregation method based on feature association according to claim 3, characterized in that, The step of determining the title identifier or summary identifier of the currently traversed event text based on the similarity specifically includes: If the similarity between the currently traversed event text and any historical event cluster is greater than a preset similarity threshold, then the event identifier corresponding to the one with the highest similarity among all historical event clusters that are greater than the similarity threshold will be used as the title identifier or summary identifier of the currently traversed event text. Otherwise, construct a first event cluster based on the currently traversed event text, generate a corresponding event identifier for the first event cluster, use the event identifier as the title identifier or summary identifier of the currently traversed event text, and add the first event cluster to the historical event cluster at the start of the next traversal.
5. The event aggregation method based on feature association according to claim 1, characterized in that, The multiple event texts belong to the event set to be aggregated; the selection of corresponding event texts based on the summary identifier of the current traversal, the construction of the current event set, and the updating of the current event set according to the title identifiers of all event texts in the current event set, to obtain the aggregated event cluster corresponding to the summary identifier of the current traversal, specifically includes: Based on the summary identifier of the current traversal, the set of events to be aggregated is filtered to construct the current event set; wherein, the summary identifier of all event texts in the current event set is the summary identifier of the current traversal; Based on the title identifier of each event text in the current event set, multiple first event texts are selected from the event set to be aggregated; wherein the title identifier of each first event text is the same as the title identifier of any event text in the current event set; The multiple first event texts are incorporated into the current event set to update the current event set, thereby obtaining the aggregated event cluster corresponding to the summary identifier of the current traversal.
6. The event aggregation method based on feature association according to claim 5, characterized in that, Also includes: Remove all event texts and multiple first event texts from the current event set from the event set to be aggregated.
7. The event aggregation method based on feature association according to claim 1, characterized in that, The step of merging and updating the multiple historical event clusters and the multiple aggregated event clusters based on the title identifier and the summary identifier to obtain multiple updated event clusters, thereby completing event aggregation, specifically includes: Based on the summary identifier corresponding to each aggregated event cluster and the title identifier of all event texts in each aggregated event cluster, construct an event identifier set corresponding to each aggregated event cluster, and take the set element with the smallest value in the event identifier set of each aggregated event cluster as the event identifier of the corresponding aggregated event cluster; Based on the event identifier, the multiple historical event clusters and the multiple aggregated event clusters are merged and updated to obtain multiple updated event clusters, thereby completing the event aggregation of the multiple event texts.
8. An event aggregation device based on feature association, characterized in that, It includes an event data acquisition module, an event identifier determination module, an event traversal and aggregation module, and an event aggregation and update module; The event data acquisition module is used to acquire multiple event texts and multiple historical event clusters to be aggregated; wherein, each event text includes a corresponding title and summary; and each historical event cluster includes a corresponding event identifier; The event identifier determination module is used to perform clustering operations on the title and summary of each event text based on single-pass clustering, using the multiple historical event clusters and their corresponding event identifiers as a basis, to determine the title identifier and summary identifier of each event text. The event traversal aggregation module is used to traverse all summary identifiers, process each event text, and obtain multiple aggregated event clusters. In each round of traversal, the corresponding event text is selected based on the summary identifier of the current traversal, the current event set is constructed, and the current event set is updated according to the title identifiers of all event texts in the current event set to obtain the aggregated event cluster corresponding to the summary identifier of the current traversal. The event aggregation and update module is used to merge and update the multiple historical event clusters and the multiple aggregated event clusters according to the title identifier and the summary identifier, so as to obtain multiple updated event clusters and complete the event aggregation.
9. A computer device, characterized in that, include: processor; Memory; A computer program stored in the memory and configured to be executed by the processor; When the processor executes the computer program, it implements an event aggregation method based on feature association as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to execute the event aggregation method based on feature association as described in any one of claims 1 to 7.