Method and device for generating event evolution relationship tree

Abstract

This disclosure provides a method, apparatus, device, and storage medium for generating an event evolution relationship tree, which can be applied to the field of natural language processing technology. The method includes: processing a text dataset to be processed based on a word frequency-inverse document frequency algorithm to obtain a first text feature matrix; obtaining a density boundary threshold based on the Euclidean distance between each element in the first text feature matrix and other elements; clustering the elements in the first text feature matrix based on the density boundary threshold to obtain multiple target text datasets corresponding to multiple topic events; processing the target text dataset corresponding to each topic event based on a hidden Dirichlet distribution algorithm to obtain multiple related sub-events; and generating an event evolution relationship tree corresponding to each topic event based on the occurrence times of the multiple related sub-events.

Description

Technical Field

[0001] This disclosure relates to the field of natural language processing, and more specifically to a method and apparatus for generating an event evolution relationship tree. Background Technology

[0002] The rapidly growing amount of information is characterized by redundancy and fragmentation, making it difficult for users to quickly and intuitively extract the dynamic development of an event from a large amount of text.

[0003] In the process of realizing the present invention, the inventors found that the related technologies have at least the following problems: the existing event story tree generation methods are difficult to reflect the hierarchical characteristics of the dynamic development process of events and the influence of external events on the theme events, lack analysis of the co-occurrence, evolution, correlation and other aspects of the nodes of the preceding and following events, and cannot fully explore the relationship between the nodes. Summary of the Invention

[0004] In view of the above problems, this disclosure provides a method and apparatus for generating an event evolution relationship tree.

[0005] According to the first aspect of this disclosure, a method for generating an event evolution relationship tree is provided, comprising: processing a text dataset to be processed based on a word frequency-inverse document frequency algorithm to obtain a first text feature matrix, wherein each element in the first text feature matrix represents the word frequency of a keyword used to describe an event;

[0006] Based on the Euclidean distance between each element and other elements in the first text feature matrix, the density boundary threshold is obtained.

[0007] Based on the density boundary threshold mentioned above, the elements in the first text feature matrix are clustered to obtain multiple target text datasets corresponding to multiple topic events.

[0008] For each target text dataset corresponding to a specific topic event, the Latent Dirichlet Algorithm is used to process the dataset, resulting in multiple related sub-events.

[0009] Based on the occurrence times of the aforementioned multiple related sub-events, an event evolution relationship tree corresponding to each main event is generated.

[0010] According to embodiments of this disclosure, the first text feature matrix includes I elements, where I is an integer greater than 1. The method for obtaining the density boundary threshold based on the Euclidean distance between each element and other elements in the first text feature matrix includes:

[0011] For the i-th element, based on the Euclidean distance between the i-th element and the other I-1 elements, obtain the j-th element corresponding to the i-th element from the I-1 elements, where i is an integer greater than or equal to 1 and less than or equal to 1;

[0012] If i is determined to be less than I, return to the operation of obtaining the j-th element corresponding to the i-th element from the (i-1)-th element above, and increment i.

[0013] Given that i equals I, we obtain J elements, where J is an integer equal to I, and j is an integer greater than or equal to 1 and less than J; and

[0014] The density boundary threshold is obtained based on the multiple Euclidean distances between the I elements and the corresponding J elements.

[0015] According to embodiments of this disclosure, obtaining the density boundary threshold based on the Euclidean distance between the aforementioned I elements and the corresponding J elements includes:

[0016] Based on the multiple Euclidean distances between the above I elements and the corresponding above J elements, the average Euclidean distance is obtained;

[0017] Arrange the above Euclidean distances in descending order to obtain the Euclidean distance sequence;

[0018] The residual processing of the above Euclidean distance sequence yields the rate of change of the difference between adjacent Euclidean distance values;

[0019] Based on the aforementioned rate of change of difference, the target Euclidean distance value is obtained from the aforementioned Euclidean distance sequence; and

[0020] Based on the average Euclidean distance and the target Euclidean distance value, the density boundary threshold is obtained.

[0021] According to embodiments of this disclosure, the elements in the first text feature matrix are clustered based on the density boundary threshold to obtain multiple target text datasets corresponding to multiple topic events, including:

[0022] Based on a density clustering algorithm, density analysis is performed on the elements in the first text feature matrix to obtain multiple candidate text datasets and the number of such candidate text datasets. The candidate text datasets represent text datasets whose element density distribution is less than or equal to the aforementioned density boundary threshold.

[0023] The number of candidate text datasets is taken as the number of target text datasets. The candidate text datasets are then clustered to obtain multiple target text datasets corresponding to the multiple topic events.

[0024] According to embodiments of this disclosure, the above-mentioned method of using the number of candidate text datasets as the number of target text datasets, and clustering the candidate text datasets to obtain multiple target text datasets corresponding to the multiple topic events, includes:

[0025] Using the number of candidate text datasets as the number of target text datasets, and based on the K-means clustering algorithm, cluster the candidate text datasets to obtain multiple clustered text datasets corresponding to the multiple topic events and unclustered text data; and

[0026] Based on the relevance of the unclustered text data to the multiple candidate topic events, the unclustered text data is merged with the clustered text datasets corresponding to the topic events to obtain the target text dataset.

[0027] According to embodiments of this disclosure, the target text dataset corresponding to each topic event is processed based on the Latent Dirichlet distribution algorithm to obtain multiple related sub-events, including:

[0028] For each topic event, keywords are extracted from the target text dataset to obtain a keyword set.

[0029] Based on the predetermined weights of the keywords, the keyword set is processed using the Latent Dirichlet Algorithm to obtain the number of relevant associated sub-events in the target text dataset.

[0030] Based on the term frequency-inverse document frequency algorithm, the target text dataset is processed to obtain a second text feature matrix. The elements of this second text feature matrix represent the term frequencies of keywords describing the topic events corresponding to the target text dataset.

[0031] Based on the K-centroid clustering algorithm, the second text feature matrix is ​​clustered according to the number of the aforementioned associated sub-events to obtain multiple associated sub-events with correlation.

[0032] According to embodiments of this disclosure, the above-mentioned keyword set is processed based on a latent Dirichlet distribution algorithm according to predetermined weights of keywords to obtain the number of relevant associated sub-events in the target text dataset, including:

[0033] Based on the predetermined weights of the keywords, the keyword set is processed using the Latent Dirichlet Algorithm to obtain the text consistency coefficients in the target text dataset corresponding to the predetermined number of associated events; and

[0034] Based on the aforementioned text consistency coefficient, the number of related sub-events with correlation in the target text dataset is obtained.

[0035] According to embodiments of this disclosure, the aforementioned plurality of associated sub-events include a plurality of sub-events in a first cluster and a plurality of sub-events in a second cluster. The generation of an event evolution relationship tree corresponding to each topic event based on the occurrence times of the aforementioned plurality of associated sub-events includes:

[0036] If it is determined that the earliest occurrence time of the sub-event in the first cluster is earlier than the earliest occurrence time of the sub-event in the second cluster, then the multiple sub-events in the first cluster and the earliest occurrence time of the sub-event in the second cluster are taken as the backbone nodes.

[0037] Based on the aforementioned main nodes and other sub-events in the second cluster, an event evolution relationship tree corresponding to each of the aforementioned thematic events is generated.

[0038] According to embodiments of this disclosure, generating an event evolution relationship tree corresponding to each of the aforementioned topic events based on the aforementioned trunk nodes and other sub-events in the aforementioned second cluster includes:

[0039] Following the forward order of the occurrence times of the sub-events corresponding to the main nodes, the main nodes are sequentially associated to generate the main branch of the event evolution relationship tree.

[0040] Following the forward order of the occurrence times of the other sub-events in the second cluster, these other sub-events are sequentially associated as branch nodes to generate branches of the event evolution relationship tree.

[0041] Based on the correlation between the main nodes and the branch nodes, the main nodes and branches are associated to generate the event evolution relationship tree.

[0042] The second aspect of this disclosure provides an apparatus for generating an event evolution relationship tree, comprising: a first processing module for processing a text dataset to be processed based on a word frequency-inverse document frequency algorithm to obtain a first text feature matrix, wherein each element in the first text feature matrix represents the word frequency of a keyword used to describe an event;

[0043] The module is used to obtain the density boundary threshold based on the Euclidean distance between each element in the first text feature matrix and other elements.

[0044] The clustering module is used to cluster the elements in the first text feature matrix based on the density boundary threshold mentioned above, so as to obtain multiple target text datasets corresponding to multiple topic events.

[0045] The second processing module is used to process the target text dataset corresponding to each topic event based on the Latent Dirichlet distribution algorithm, obtaining multiple related sub-events; and

[0046] The generation module is used to generate an event evolution relationship tree corresponding to each main event based on the occurrence time of the above-mentioned multiple related sub-events.

[0047] According to the event evolution relationship tree generation method and apparatus provided in this disclosure, a text dataset is processed to obtain a first text feature matrix; a density boundary threshold is obtained based on the Euclidean distance between each element in the first text feature matrix and other elements; the elements in the first text feature matrix are clustered based on the density boundary threshold to obtain multiple target text datasets corresponding to multiple topic events; the target text datasets are processed to obtain multiple related sub-events; and an event evolution relationship tree corresponding to each topic event is generated based on the occurrence time of the multiple related sub-events. The density boundary threshold is obtained by calculating the Euclidean distance between each element in each text data, unlike existing technologies that require manual annotation, thus exhibiting a certain degree of adaptability. Therefore, this method at least partially solves the problems of existing event context generation methods based on historical rules and time elements failing to perceive event evolution from the perspective of sub-events and to mine the development context of events from multiple perspectives, and the inability of existing algorithms to fully achieve adaptive clustering, resulting in low generalization ability and robustness. Attached Figure Description

[0048] The foregoing contents, as well as other objects, features, and advantages of this disclosure, will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0049] Figure 1 The illustration schematically depicts an application scenario of a method, apparatus, device, medium, and program product for generating an event evolution relationship tree according to embodiments of the present disclosure;

[0050] Figure 2 A flowchart illustrating a method for generating an event evolution relationship tree according to an embodiment of the present disclosure is shown schematically.

[0051] Figure 3 A flowchart illustrating topic event clustering analysis according to embodiments of the present disclosure is shown schematically;

[0052] Figure 4 A flowchart illustrating sub-event clustering analysis according to embodiments of the present disclosure is shown schematically;

[0053] Figure 5 The illustration shows a schematic diagram of the event evolution relationship tree construction process according to an embodiment of the present disclosure;

[0054] Figure 6 The illustration shows a schematic diagram of a method for generating an event evolution relationship tree according to an embodiment of the present disclosure;

[0055] Figure 7 A schematic diagram illustrating the structure of an apparatus for generating an event evolution relationship tree according to an embodiment of the present disclosure; and

[0056] Figure 8 A block diagram schematically illustrates an electronic device suitable for implementing a method for generating an event evolution relationship tree according to an embodiment of the present disclosure. Detailed Implementation

[0057] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0058] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0059] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0060] When using expressions such as "at least one of A, B, and C", they should generally be interpreted in accordance with the meaning that is commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc.).

[0061] In the technical solution disclosed herein, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding operation entry points are provided for users to choose to authorize or refuse.

[0062] Events are an important part of human society, and the rapid development of social networks has provided people with convenient ways to efficiently access trending news. However, the rapidly growing amount of information is characterized by redundancy and fragmentation, especially since news reports often have a hierarchical structure. Information filtered solely based on spatiotemporal relevance cannot accurately grasp the clues and development of events, while excessively constructing story trees around news elements will lead to a reversal of priorities.

[0063] Therefore, how to highlight the importance of news elements and construct story trees reasonably from multiple perspectives during the feature mining process using technical means is one of the important topics in the field of event graph research. In recent years, there has been a large amount of research on the related problems and core technologies of story tree generation, combining machine learning algorithms to determine the number of topics and the scope of events by performing community detection and cluster analysis on different data.

[0064] However, most topic clustering algorithms rely on manual annotation results and cannot fully realize an unsupervised adaptive clustering process. Furthermore, existing event story tree generation methods mainly extract structurally simple news events from a large amount of text and mechanically string them together according to chronological order to construct a single-chain story tree, which makes it difficult to reflect the hierarchical characteristics of the dynamic development of events and the influence of external events on topic events.

[0065] In addition, most historical rule-based methods use timelines to conduct coarse-grained analysis of events, lacking analysis of the co-occurrence, evolution, and correlation of previous and subsequent event nodes, and thus failing to fully explore the relationships between nodes.

[0066] To at least partially address the technical problems existing in related technologies, embodiments of this disclosure provide a method for generating an event evolution relationship tree, which can be applied to the field of natural language processing technology. The method includes: processing a text dataset to be processed based on a term frequency-inverse document frequency algorithm to obtain a first text feature matrix; obtaining a density boundary threshold based on the Euclidean distance between each element in the first text feature matrix and other elements; clustering the elements in the first text feature matrix based on the density boundary threshold to obtain multiple target text datasets corresponding to multiple topic events; processing the target text dataset corresponding to each topic event based on a hidden Dirichlet distribution algorithm to obtain multiple related sub-events; and generating an event evolution relationship tree corresponding to each topic event based on the occurrence times of the multiple related sub-events.

[0067] Figure 1 The diagram illustrates an application scenario of an event evolution relationship tree according to an embodiment of the present disclosure.

[0068] like Figure 1 As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0069] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0070] Terminal devices 101, 102, and 103 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0071] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using terminal devices 101, 102, and 103 (for example only). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0072] It should be noted that the event evolution tree generation method provided in this embodiment can generally be executed by server 105. Correspondingly, the event evolution tree generation apparatus provided in this embodiment can generally be located in server 105. The event evolution tree generation method provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105. Correspondingly, the event evolution tree generation apparatus provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105.

[0073] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0074] The following will be based on Figure 1 The described scene, through Figures 2-6 The event evolution relationship tree generation method of the disclosed embodiments is described in detail.

[0075] Figure 2 A flowchart illustrating an event evolution relationship tree generation method according to an embodiment of the present disclosure is shown schematically.

[0076] like Figure 2 As shown, the event evolution relationship tree generation method in this embodiment includes operations S210 to S250.

[0077] In operation S210, the text dataset to be processed is processed based on the word frequency-inverse document frequency algorithm to obtain the first text feature matrix.

[0078] For example, the text dataset W is first processed to obtain the word frequency matrix TF through word frequency statistics. Then, the inverse document frequency is calculated according to formula (1):

[0079]

[0080] Where Y represents the total number of texts, Y ω The number of texts containing term ω is calculated, and the result is converted into the inverse document frequency matrix (IDF). Finally, the Hadamard product is used to calculate the number of texts containing term ω. Obtain the TF-IDF matrix of keywords in the text dataset, which is the first text feature matrix.

[0081] In operation S220, the density boundary threshold is obtained based on the Euclidean distance between each element in the first text feature matrix and other elements.

[0082] For example, the Euclidean distance between each element a,b,c,d,e in the first text feature matrix [a,b,c,d,e] and other elements includes the Euclidean distance between element a and elements b,c,d,e.

[0083] In operation S230, based on the density boundary threshold, the elements in the first text feature matrix are clustered to obtain multiple target text datasets corresponding to multiple topic events.

[0084] According to embodiments of this disclosure, clustering yields multiple target text datasets, each corresponding to its own event theme. Each text data in each target text dataset is related to a certain extent and belongs to the same event theme.

[0085] For example, if the target text dataset A = [earthquake, tsunami, flood] and the target text dataset B = [fire, explosion], then the data in dataset A belongs to the theme of natural disasters, while the data in dataset B belongs to the theme of man-made disasters.

[0086] In operation S240, for the target text dataset corresponding to each topic event, the target text dataset is processed based on the Latent Dirichlet Distribution algorithm to obtain multiple related sub-events.

[0087] According to embodiments of this disclosure, based on the Latent Dirichlet Distribution algorithm, the target text dataset corresponding to each topic event is processed to obtain multiple related sub-events, and the related sub-events are correlated.

[0088] For example, sub-event A: a major earthquake in location A; sub-event B: the successful launch of experimental module B; sub-event C: the successful launch of manned spacecraft XX. Sub-events B and C both belong to Chinese space events, so they have a certain correlation and are considered related sub-events. However, sub-event A is a natural disaster event and has no correlation with sub-events B and C.

[0089] In operation S250, based on the occurrence time of multiple related sub-events, an event evolution relationship tree corresponding to each main event is generated according to the multiple related sub-events.

[0090] According to embodiments of this disclosure, the event evolution relationship tree is generated based on the occurrence times of associated sub-events. First, the occurrence time of each associated sub-event is determined. Then, the story tree is divided into trunk and branches, establishing the event start node, trunk node, branch node, and key node. Each node is then connected to generate the event evolution relationship tree.

[0091] According to embodiments of this disclosure, a feature matrix is ​​obtained based on a term frequency-inverse document frequency algorithm. The Euclidean distance between elements in the feature matrix is ​​calculated to obtain a density boundary threshold. Then, the elements in the feature matrix are clustered to obtain multiple target text datasets corresponding to multiple topic events. Based on the Latent Dirichlet algorithm, correlated sub-events are obtained. Based on the occurrence time of each correlated sub-event, a corresponding event evolution relationship tree is generated. Adaptive clustering is achieved by calculating the Euclidean distance between elements in each text dataset to obtain the density boundary threshold. The density boundary threshold is calculated based on the text data itself, not pre-set, thus the text data to be processed is not limited, overcoming the limitation of traditional algorithms relying on prior knowledge when adjusting parameters. Adaptively selecting optimal parameters through text content solves the problem of easily getting trapped in local optima during algorithm optimization. Simultaneously, the tree structure intuitively depicts the evolution and development process of events at each stage, and each event is mapped to a corresponding node, enabling accurate understanding of the clues and development trajectory of events.

[0092] According to an embodiment of this disclosure, the first text feature matrix includes I elements, where I is an integer greater than 1. A density boundary threshold is obtained based on the Euclidean distance between each element in the first text feature matrix and other elements, including: for the i-th element, obtaining the j-th element corresponding to the i-th element from the (I-1)-th elements based on the Euclidean distance between the i-th element and the other I-1 elements, where i is an integer greater than or equal to 1 and less than or equal to 1; if i is determined to be less than 1, returning to the operation of obtaining the j-th element corresponding to the i-th element from the (I-1)-th elements, and incrementing i; if i is determined to be equal to 1, obtaining J elements, where J is an integer equal to 1 and j is an integer greater than or equal to 1 and less than J; and obtaining the density boundary threshold based on multiple Euclidean distances between the I elements and the corresponding J elements.

[0093] According to embodiments of this disclosure, a density boundary threshold is obtained based on the Euclidean distances between I elements and their corresponding J elements, comprising the following steps: obtaining an average Euclidean distance based on multiple Euclidean distances between the I elements and their corresponding J elements; sorting the multiple Euclidean distances in descending order to obtain an Euclidean distance sequence; performing residual processing on the Euclidean distance sequence to obtain the rate of change of difference between adjacent Euclidean distance values; obtaining a target Euclidean distance value from the Euclidean distance sequence based on the rate of change of difference; and obtaining a density boundary threshold based on the average Euclidean distance and the target Euclidean distance value.

[0094] For example, the first text feature matrix [m,n,o,p,q] includes 5 elements. The Euclidean distance between each element m,n,o,p,q and the other 4 elements is calculated and sorted. The j-th element whose Euclidean distance to the current element is the nth element corresponds to the j-th element of each element, where j is one of the other elements besides the current element. Multiple Euclidean distances between the 5 elements and their corresponding 5 elements can be calculated using formula (2):

[0095]

[0096] The set of Euclidean distances between all elements m, n, o, p, q and their corresponding j-th elements is D = {d1, d2, ..., dq}. n The average Euclidean distance can be calculated using formula (3):

[0097]

[0098] Next, the Euclidean distances in the distance set D are sorted in descending order, and the K-Distance distance curve is plotted. Residuals are calculated for adjacent values, and the residual set is represented as E = {e1, e2, ..., e...}. n-1 The residual results are then subjected to secondary residual calculation to obtain the residual value point e corresponding to the maximum rate of change of the difference. max The Euclidean distance value corresponding to the residual point is the target Euclidean distance value, denoted as d. max The value is finally based on the average Euclidean distance d. aver Euclidean distance d from the target max The density boundary threshold is calculated according to formula (4):

[0099]

[0100] According to embodiments of this disclosure, a density boundary threshold is obtained based on the Euclidean distance between each element in the first text feature matrix and other elements. A novel adaptive eps parameter selection algorithm is adopted, which not only makes up for the limitations of traditional algorithms that rely on prior knowledge when adjusting parameters, but also adaptively selects the optimal parameters through text content, and solves the problem of easily getting trapped in local optima during the algorithm optimization process. This improves the robustness of the entire model while making it more interpretable.

[0101] According to embodiments of this disclosure, clustering elements in a first text feature matrix based on a density boundary threshold yields multiple target text datasets corresponding to multiple topic events. This includes: performing density analysis on elements in the first text feature matrix based on a density clustering algorithm to obtain multiple candidate text datasets and the number of candidate text datasets, wherein a candidate text dataset represents a text dataset whose element density distribution is less than or equal to a density boundary threshold; and using the number of multiple candidate text datasets as the number of target text datasets, clustering the multiple candidate text datasets to obtain multiple target text datasets corresponding to multiple topic events.

[0102] According to embodiments of this disclosure, density analysis of elements in the first text feature matrix can be performed using the density-based DBSCAN algorithm, without needing to pre-specify the number of clusters.

[0103] For example, if the density boundary threshold is 0.75, and the density distributions of elements a, b, c, and d in the first text feature matrix W are 3, 0.6, 0.75, and 0.92 respectively, then the density distributions of elements b and c are less than or equal to the density boundary threshold of 0.75. Therefore, [b, c] constitutes a candidate text dataset.

[0104] For example, if a density clustering algorithm yields 300 candidate text datasets, then the number of target text datasets will also be 300.

[0105] According to embodiments of this disclosure, the number of multiple candidate text datasets is used as the number of target text datasets. Multiple candidate text datasets are clustered to obtain multiple target text datasets corresponding to multiple topic events. This includes: using the number of multiple candidate text datasets as the number of target text datasets, clustering the multiple candidate text datasets based on the K-means clustering algorithm to obtain multiple clustered text datasets corresponding to multiple topic events and unclustered text data; and merging the unclustered text data with the clustered text datasets corresponding to the corresponding topic events based on the correlation between the unclustered text data and the multiple candidate topic events to obtain the target text dataset.

[0106] According to embodiments of this disclosure, the number of multiple candidate text datasets is taken as the number of target text datasets, and this number is defined as n. n is used as the K value and input into the K-means clustering algorithm for secondary clustering.

[0107] According to embodiments of this disclosure, merging unclustered text data with clustered text datasets corresponding to the topic events refers to merging isolated data points that failed to form clusters during the clustering process with existing topic clusters.

[0108] According to an embodiment of the present disclosure, the first text feature matrix is subjected to step-by-step secondary clustering through a density clustering algorithm and a K-means algorithm to obtain multiple target text data sets corresponding to multiple theme events. In the process of multiple clusterings, the data correlation within each obtained cluster is as high as possible. At the same time, both the density clustering algorithm and the K-means algorithm are unsupervised learning algorithms, which do not require training and do not need to preset categories, and can perform cluster analysis adaptively, solving the problem that the prior art does not utilize adaptive clustering to generate an event evolution relationship tree. At the same time, the unclustered text data is merged with the clustered text data set corresponding to the corresponding theme event to ensure the integrity and comprehensiveness of the text data, making the text extraction accuracy as high as possible.

[0109] Figure 3 Schematically shows a flowchart of theme event cluster analysis according to an embodiment of the present disclosure.

[0110] As Figure 3 shown, the method 300 includes operations S310 to S380.

[0111] In operation S310, input: The text data set is preprocessed to obtain a word segmentation set and a keyword set.

[0112] According to an embodiment of the present disclosure, the operations of preprocessing include: inputting the text data set Φ = {T1, T2,..., T n}(n is the number of input news texts), first using a Chinese word segmentation tool to perform word segmentation on each text content to obtain a word segmentation set denoted as Ω = {S1, S2,..., S n}, and loading a stop word list to filter out stop words in the text. Then, using the TextRank algorithm to extract keywords, and representing the keyword results of all the extracted texts as Ψ = {K1, K2,..., K n}, which is the keyword set. The keywords included in the i-th text can be denoted as K i = {k1, k2,..., k topK}, and the number of keywords in each text is topK.

[0113] According to an embodiment of the present disclosure, stop words refer to certain words or terms that are automatically filtered out before processing the text in information retrieval to save storage space and improve search efficiency.

[0114] For example, if the stop word list is [of, a, this, that], then "of, a, this, that" contained in each text is filtered out.

[0115] In operation S320, calculate and standardize the first text feature matrix.

[0116] According to embodiments of this disclosure, word frequency statistics are first performed using the keyword set Ψ obtained in S3010, and the results are converted into a word frequency matrix TF. Word frequency statistics involves counting the frequency of any keyword in the keyword set Ψ appearing among the keywords. The process of obtaining the first text feature matrix can be referred to in S210.

[0117] In operation S330, determine whether the PCA dimensionality reduction threshold is a positive integer.

[0118] In operation S331, when the dimension threshold is a positive integer, the first text feature matrix is ​​reduced to the feature matrix of that dimension.

[0119] When operating S332, if the dimension threshold is not a positive integer, the first text feature matrix is ​​adaptively reduced in dimension by a percentage.

[0120] According to the embodiments of this disclosure, the first text feature matrix is ​​dimensionality reduced using the PCA (Principal Component Analysis) algorithm. A dimensionality threshold is set to determine the dimensionality reduction range. When the value of is not a positive integer, the dimensionality is adaptively reduced to the optimal dimension according to the proportion of data that needs to be retained. When the value of is a positive integer, the first text feature matrix is ​​directly reduced to that dimension. The setting of the dimensionality threshold can improve the algorithm performance while taking into account data integrity.

[0121] In operation S340, the Euclidean distance of each element in the first text feature matrix is ​​calculated.

[0122] In operation S350, density boundary threshold calculation.

[0123] In operation S360, the number of target texts based on the density clustering algorithm is confirmed.

[0124] In operation S370, topic event clustering is performed based on the K-means clustering algorithm.

[0125] When operating S380, the output is: topic event clustering results.

[0126] According to embodiments of this disclosure, for each target text dataset corresponding to a topic event, the target text dataset is processed based on the Latent Dirichlet Distribution algorithm to obtain multiple related sub-events, including: extracting keywords from the target text dataset corresponding to each topic event to obtain a keyword set; processing the keyword set based on the Latent Dirichlet Distribution algorithm according to predetermined weights of the keywords to obtain the number of related sub-events in the target text dataset; processing the target text dataset based on the term frequency-inverse document frequency algorithm to obtain a second text feature matrix, wherein the elements in the second text feature matrix represent the term frequencies of the keywords used to describe the topic events corresponding to the target text dataset; and clustering the second text feature matrix based on the number of related sub-events using the K-centroid clustering algorithm to obtain multiple related sub-events.

[0127] According to embodiments of this disclosure, the keyword set is processed based on the Latent Dirichlet Algorithm (LDA) algorithm according to the predetermined weights of the keywords to obtain the number of relevant associated sub-events in the target text dataset. This includes: processing the keyword set based on the LDA algorithm according to the predetermined weights of the keywords to obtain a text consistency coefficient in the target text dataset corresponding to the predetermined number of associated events; and obtaining the number of relevant associated sub-events in the target text dataset based on the text consistency coefficient.

[0128] According to embodiments of this disclosure, a keyword set Ψ is extracted from the target text dataset corresponding to each topic event. C and word segmentation set Ω C Then according to Ψ C The predefined weights of each keyword in the document generate the corresponding Bow model and keyword corpus. Based on the latent Dirichlet distribution, the document consistency coefficient is calculated iteratively under different numbers of sub-event clusters. The result is denoted as H = {H1, H2, ..., H}. m}, where m is a positive integer representing the number of sub-event clusters, with a maximum value not exceeding the corresponding keyword set Ψ. C The number of documents included, and the number of related sub-events λ corresponding to the maximum value in set H. Then Ω C The corresponding TF-IDF is calculated after matrix standardization. C The matrix is ​​then used for dimensionality reduction via PCA. Finally, the magnitude λ and the dimensionality-reduced TF-IDF are compared. C The matrix is ​​used as input to the K-centroid clustering algorithm to perform sub-event clustering analysis on the events in the topic cluster, obtaining multiple related sub-events. The clustering result is denoted as V = {V1, V2, ..., V...} jEach cluster contains multiple sub-events with the highest degree of cross-correlation.

[0129] According to embodiments of this disclosure, the target text dataset corresponding to each topic event is processed using the implicit Dirichlet distribution and K-centroid clustering algorithm to obtain related sub-events that are relevant to multiple topic events. In conjunction with embodiment 300, this disclosure completes topic event clustering and sub-event clustering respectively based on a multi-stage hierarchical clustering algorithm, which makes up for the shortcomings of the prior art that does not perceive and analyze the event evolution from the perspective of the sub-events of the topic event. At the same time, the extraction accuracy is high and the integrity of the text is guaranteed.

[0130] Figure 4 A flowchart illustrating sub-event clustering analysis according to an embodiment of this disclosure is shown schematically.

[0131] In operation S410, a keyword set is obtained for the target text dataset corresponding to each topic event.

[0132] In operation S420, based on the predetermined weights of the keywords, the keyword set is processed using the Latent Dirichlet Distribution algorithm to obtain the text consistency coefficient in the target text dataset corresponding to the predetermined number of associated events.

[0133] According to embodiments of this disclosure, the set of keywords processed is extracted from the target text set corresponding to the topic event.

[0134] In operation S430, based on the text consistency coefficient, the number of related sub-events with correlation in the target text dataset is obtained.

[0135] For example, document consistency coefficients can be determined from both semantic and logical perspectives.

[0136] In operation S440, the target text dataset is processed to obtain the second text feature matrix.

[0137] According to an embodiment of this disclosure, the second text feature matrix is ​​calculated by standardizing the word segmentation set extracted from the target text set corresponding to the topic event.

[0138] In operation S450, based on the K-centroid clustering algorithm, the second text feature matrix is ​​clustered according to the number of associated sub-events to obtain multiple associated sub-events with correlation.

[0139] According to embodiments of this disclosure, each sub-event cluster contains multiple sub-events with the highest degree of cross-correlation.

[0140] According to embodiments of this disclosure, multiple associated sub-events include multiple sub-events in a first cluster and multiple sub-events in a second cluster. Based on the occurrence times of the multiple associated sub-events, an event evolution relationship tree corresponding to each topic event is generated, including: if it is determined that the earliest occurrence time of the sub-event in the first cluster is earlier than the earliest occurrence time of the sub-event in the second cluster, the multiple sub-events in the first cluster and the earliest occurrence time of the sub-event in the second cluster are taken as the backbone node; based on the backbone node and other sub-events in the second cluster, an event evolution relationship tree corresponding to each topic event is generated.

[0141] According to embodiments of this disclosure, an event evolution relationship tree corresponding to each topic event is generated based on the trunk node and other sub-events in the second cluster. This includes: sequentially associating the trunk node with the sub-events in the forward order of their occurrence times to generate the trunk of the event evolution relationship tree; sequentially associating the other sub-events as branch nodes in the forward order of their occurrence times to generate branches of the event evolution relationship tree; and associating the trunk and branches based on the correlation between the trunk node and the branch nodes to generate the event evolution relationship tree.

[0142] According to embodiments of this disclosure, the correlation between nodes is calculated using spatiotemporal correlation and the degree of importance of events. Isolated nodes that fail to form clusters are assigned to the sub-event clusters with the highest correlation. Based on the time mapping model of event nodes, corresponding starting nodes, trunk nodes, branch nodes, and key nodes are divided.

[0143] For example, in the first cluster: sub-event 1 occurs on Monday, and sub-event 2 occurs on Wednesday; in the second cluster: sub-event 3 occurs on Tuesday, sub-event 4 occurs on Thursday, and sub-event 5 occurs on Friday. The sub-events are sorted according to their occurrence time. Sub-events 1, 2, and 3, which occur on Monday, Tuesday, and Wednesday, are the main nodes, and the remaining sub-events are branch nodes. In the first cluster, the earliest occurring sub-event 1 occurs earlier than the earliest occurring sub-event 3 in the second cluster; therefore, all sub-events in the first cluster are main sub-events. Thus, sub-events 1, 2, and 3 are the main sub-events, arranged in the order of sub-event 1, sub-event 3, and sub-event 2; sub-events 4 and 5 are the branch sub-events, arranged in the order of sub-event 4, and sub-event 5. Since sub-event 3 and the branch sub-event both originate from the second cluster and have the highest correlation, branch sub-event 4 is associated with main sub-event 3, generating an event evolution relationship tree.

[0144] According to the embodiments of this disclosure, each sub-event is mapped to a corresponding node. The nodes are classified into categories based on time sequence information, event cross-correlation, and event importance, including initial nodes, trunk nodes, key nodes, and branch nodes. The nodes are then connected in a directed manner according to the classification results to generate a complete event story tree. The tree structure of the event evolution relationship tree can intuitively depict the event evolution and development process at each stage.

[0145] Figure 5 The illustration shows a schematic diagram of the event evolution relationship tree construction process according to an embodiment of the present disclosure.

[0146] Topic cluster 510 is a target text dataset, which is divided into 3 sub-event clusters through the operations in Example 400. The sub-events in each sub-event cluster have the highest correlation.

[0147] Extract the occurrence time information of each sub-event from topic cluster 510 to obtain the occurrence time of each sub-event in the topic cluster, and make a judgment on the main node and branch node. The judgment result is 520.

[0148] According to embodiments of this disclosure, based on the node classification results, the clustered event nodes are connected in a directed manner according to the temporal direction using a mapping model. Based on the principle of similarity and the temporal information of the events, the main branches and sub-branches are merged and embedded to generate a complete event evolution relationship tree structure 530.

[0149] According to embodiments of this disclosure, each topic event can generate a corresponding evolutionary relationship tree.

[0150] Figure 6 The diagram illustrates a method for generating an event evolution relationship tree according to an embodiment of the present disclosure.

[0151] When operating the S610, input the news report text and preprocess the text dataset.

[0152] In operation S620, the text dataset to be processed is processed based on the term frequency-inverse document frequency algorithm to obtain a first text feature matrix; and a density boundary threshold is obtained based on the Euclidean distance between each element in the first text feature matrix and other elements; based on the density boundary threshold, the elements in the first text feature matrix are clustered to obtain multiple target text datasets corresponding to multiple topic events. Specific operations can be found in Example 300, and will not be elaborated here.

[0153] In operation S630, the keyword set is processed based on the hidden Dirichlet distribution algorithm. According to the predetermined weight of the keywords, the text consistency coefficient in the target text dataset corresponding to the predetermined number of associated events is obtained.

[0154] In the S640 operation, the text consistency coefficient and the dimensionality-reduced second text matrix are used as input. The K-centroid clustering algorithm is used to perform sub-event partitioning and clustering analysis on each topic cluster. Each sub-event cluster contains multiple sub-events with the highest degree of cross-correlation.

[0155] During operation of S650, the occurrence time information of each sub-event is extracted to obtain the occurrence time of each sub-event in the topic cluster, and the nodes are judged to generate an event evolution relationship tree.

[0156] Based on the above-described method for generating an event evolution tree, this disclosure also provides an apparatus for generating an event evolution tree. The following will be combined with... Figure 7 The device is described in detail.

[0157] Figure 7 A schematic block diagram of an apparatus for generating an event evolution relationship tree according to an embodiment of the present disclosure is shown.

[0158] like Figure 7 As shown, the event evolution relationship tree generation device 700 of this embodiment includes a first processing module 710, an acquisition module 720, a clustering module 730, a second processing module 740, and a generation module 750.

[0159] The first processing module 710 is used to process the text dataset to be processed based on the term frequency-inverse document frequency algorithm to obtain a first text feature matrix, wherein each element in the first text feature matrix represents the term frequency of the keyword used to describe the event. In one embodiment, the first processing module 710 can be used to perform the operation S210 described above, which will not be repeated here.

[0160] The obtaining module 720 is used to obtain the density boundary threshold based on the Euclidean distance between each element in the first text feature matrix and other elements. In one embodiment, the obtaining module 720 can be used to perform the operation S220 described above, which will not be repeated here.

[0161] According to embodiments of this disclosure, the obtaining module 720 includes a first obtaining submodule, a second obtaining submodule, a third obtaining submodule, and a fourth obtaining submodule.

[0162] The first submodule is used to obtain the j-th element corresponding to the i-th element from the (I-1)-th elements based on the Euclidean distance between the i-th element and the other I-1 elements.

[0163] The second submodule is used to return to the operation of obtaining the j-th element corresponding to the i-th element from the (i-1)-th element if it is determined that i is less than I, and increment i.

[0164] The third submodule is used to obtain J elements if i equals I.

[0165] The fourth submodule is used to obtain the density boundary threshold based on multiple Euclidean distances between the I elements and the corresponding J elements.

[0166] According to embodiments of this disclosure, the fourth obtaining submodule includes a first obtaining unit, a second obtaining unit, a third obtaining unit, a fourth obtaining unit, and a fifth obtaining unit.

[0167] The first obtaining unit is used to obtain the average Euclidean distance based on the multiple Euclidean distances between the I elements and the corresponding J elements.

[0168] The second obtaining unit is used to sort multiple Euclidean distances in descending order to obtain an Euclidean distance sequence.

[0169] The third acquisition unit is used to perform residual processing on the Euclidean distance sequence to obtain the rate of change of the difference between adjacent Euclidean distance values.

[0170] The fourth acquisition unit is used to obtain the target Euclidean distance value from the Euclidean distance sequence based on the rate of change of difference.

[0171] The fifth obtaining unit is used to obtain the density boundary threshold based on the average Euclidean distance and the target Euclidean distance value.

[0172] The clustering module 730 is used to cluster the elements in the first text feature matrix based on a density boundary threshold to obtain multiple target text datasets corresponding to multiple topic events. In one embodiment, the clustering module 730 can be used to perform the operation S230 described above, which will not be repeated here.

[0173] According to embodiments of this disclosure, the clustering module 730 includes a first obtaining submodule and a second obtaining submodule.

[0174] The first acquisition submodule is used to perform density analysis on the elements in the first text feature matrix based on the density clustering algorithm to obtain multiple candidate text datasets and the number of multiple candidate text datasets.

[0175] The second submodule is used to cluster the multiple candidate text datasets as the number of target text datasets, and to obtain multiple target text datasets corresponding to multiple topic events.

[0176] According to embodiments of this disclosure, the second obtaining submodule includes a first obtaining unit and a second obtaining unit.

[0177] The first obtaining unit is used to take the number of multiple candidate text datasets as the number of target text datasets, and to cluster the multiple candidate text datasets based on the K-means clustering algorithm to obtain multiple clustered text datasets and unclustered text data corresponding to multiple topic events.

[0178] The second obtaining unit is used to merge the unclustered text data with the clustered text datasets corresponding to the corresponding topic events based on the relevance between the unclustered text data and multiple candidate topic events, so as to obtain the target text dataset.

[0179] The second processing module 740 is used to process the target text dataset corresponding to each topic event based on the Latent Dirichlet Distribution algorithm to obtain multiple related sub-events. In one embodiment, the second processing module 740 can be used to perform the operation S240 described above, which will not be repeated here.

[0180] According to embodiments of this disclosure, the second processing module 740 includes a first obtaining submodule, a second obtaining submodule, a third obtaining submodule, and a fourth obtaining submodule.

[0181] The first submodule is used to extract keywords from the target text dataset corresponding to each topic event, thus obtaining a keyword set.

[0182] The second submodule is used to process the keyword set based on the implicit Dirichlet distribution algorithm according to the predetermined weight of the keywords, and obtain the number of relevant associated sub-events in the target text dataset.

[0183] The third submodule is used to process the target text dataset based on the word frequency-inverse document frequency algorithm to obtain the second text feature matrix.

[0184] The fourth submodule is used to cluster the second text feature matrix based on the number of associated sub-events using the K-centroid clustering algorithm, thereby obtaining multiple associated sub-events with correlation.

[0185] The second obtaining submodule includes a first obtaining unit and a second obtaining unit.

[0186] The first obtaining unit is used to process the keyword set based on the implicit Dirichlet distribution algorithm according to the predetermined weight of the keywords, and obtain the text consistency coefficient in the target text dataset corresponding to the predetermined number of associated events.

[0187] The second unit is used to obtain the number of related sub-events with correlation in the target text dataset based on the text consistency coefficient.

[0188] The generation module 750 is used to generate an event evolution relationship tree corresponding to each main event based on the occurrence times of multiple related sub-events. In one embodiment, the generation module 750 can be used to perform the operation S250 described above, which will not be repeated here.

[0189] According to embodiments of this disclosure, the generation module 750 includes a determining submodule and a generation submodule.

[0190] The determination submodule is used to determine multiple sub-events in the first cluster and the earliest sub-event in the second cluster as backbone nodes when the earliest occurrence time of the sub-event in the first cluster is earlier than the earliest occurrence time of the sub-event in the second cluster.

[0191] The generation submodule is used to generate an event evolution relationship tree corresponding to each topic event based on the trunk node and other sub-events in the second cluster.

[0192] According to embodiments of this disclosure, the generation submodule includes a first generation unit, a second generation unit, and a third generation unit.

[0193] The first generation unit is used to sequentially associate the main nodes according to the forward order of the occurrence times of the sub-events corresponding to the main nodes, and generate the main branch of the event evolution relationship tree.

[0194] The second generation unit is used to generate branches of the event evolution relationship tree by sequentially associating other sub-events as branch nodes according to the forward order of their occurrence times in the second cluster.

[0195] The third generation unit is used to associate the trunk and branches based on the correlation between the trunk nodes and branch nodes, and generate an event evolution relationship tree.

[0196] According to embodiments of this disclosure, any plurality of modules among the first processing module 710, the obtaining module 720, the clustering module 730, the second processing module 740, and the generating module 750 may be combined into one module, or any one of these modules may be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the first processing module 710, the obtaining module 720, the clustering module 730, the second processing module 740, and the generating module 750 may be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the first processing module 710, the obtaining module 720, the clustering module 730, the second processing module 740, and the generating module 750 may be implemented at least partially as a computer program module, which can perform corresponding functions when the computer program module is run.

[0197] Figure 8 A block diagram schematically illustrates an electronic device suitable for implementing a method for generating an event evolution relationship tree according to an embodiment of the present disclosure.

[0198] like Figure 8 As shown, an electronic device 800 according to an embodiment of this disclosure includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage portion 808 into a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this disclosure.

[0199] RAM 803 stores various programs and data required for the operation of electronic device 800. Processor 801, ROM 802, and RAM 803 are interconnected via bus 804. Processor 801 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 802 and / or RAM 803. It should be noted that the programs may also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0200] According to embodiments of this disclosure, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to a bus 804. The electronic device 800 may also include one or more of the following components connected to the I / O interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 810 as needed so that computer programs read from it can be installed into the storage section 808 as needed.

[0201] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0202] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 802 and / or RAM 803 and / or one or more memories other than ROM 802 and RAM 803 described above.

[0203] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the item recommendation method provided in the embodiments of this disclosure.

[0204] When the computer program is executed by the processor 801, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0205] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 809, and / or installed from a removable medium 811. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0206] In such an embodiment, the computer program can be downloaded and installed from a network via communication section 809, and / or installed from removable medium 811. When the computer program is executed by processor 801, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0207] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0208] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0209] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0210] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A method for generating an event evolution relationship tree, comprising: Based on the term frequency-inverse document frequency algorithm, the text dataset to be processed is processed to obtain a first text feature matrix, wherein each element in the first text feature matrix represents the term frequency of the keywords used to describe the event; The density boundary threshold is obtained based on the Euclidean distance between each element and other elements in the first text feature matrix. Based on the density boundary threshold, the elements in the first text feature matrix are clustered to obtain multiple target text datasets corresponding to multiple topic events; For each target text dataset corresponding to a topic event, the target text dataset is processed based on the Latent Dirichlet Distribution algorithm to obtain multiple related sub-events; and Based on the occurrence times of the multiple related sub-events, an event evolution relationship tree corresponding to each main event is generated.

2. The method according to claim 1, wherein, The first text feature matrix includes I elements, where I is an integer greater than 1. The step of obtaining the density boundary threshold based on the Euclidean distance between each element and other elements in the first text feature matrix includes: For the i-th element, based on the Euclidean distance between the i-th element and the other I-1 elements, obtain the j-th element corresponding to the i-th element from the I-1 elements, where i is an integer greater than or equal to 1 and less than or equal to 1; If i is determined to be less than I, return to the operation of obtaining the j-th element corresponding to the i-th element from the (i-1)-th element, and increment i. Given that i equals I, we obtain J elements, where J is an integer equal to I, and j is an integer greater than or equal to 1 and less than J; and The density boundary threshold is obtained based on the multiple Euclidean distances between the I elements and the corresponding J elements.

3. The method according to claim 2, wherein, The step of obtaining the density boundary threshold based on the Euclidean distance between the I elements and the corresponding J elements includes: The average Euclidean distance is obtained based on the multiple Euclidean distances between the I elements and the corresponding J elements; The multiple Euclidean distances are sorted in descending order to obtain a sequence of Euclidean distances; The Euclidean distance sequence is subjected to residual processing to obtain the rate of change of the difference between adjacent Euclidean distance values; Based on the rate of change of difference, the target Euclidean distance value is obtained from the Euclidean distance sequence; and The density boundary threshold is obtained based on the average Euclidean distance and the target Euclidean distance.

4. The method according to claim 1, wherein, The elements in the first text feature matrix are clustered based on the density boundary threshold to obtain multiple target text datasets corresponding to multiple topic events, including: Based on a density clustering algorithm, density analysis is performed on the elements in the first text feature matrix to obtain multiple candidate text datasets and the number of such candidate text datasets. The candidate text datasets represent text datasets whose element density distribution is less than or equal to the density boundary threshold. The number of candidate text datasets is taken as the number of target text datasets. The candidate text datasets are then clustered to obtain multiple target text datasets corresponding to the multiple topic events.

5. The method according to claim 4, wherein, The step of using the number of the multiple candidate text datasets as the number of target text datasets, clustering the multiple candidate text datasets to obtain multiple target text datasets corresponding to the multiple topic events includes: Using the number of candidate text datasets as the number of target text datasets, and based on the K-means clustering algorithm, the candidate text datasets are clustered to obtain multiple clustered text datasets corresponding to the multiple topic events and unclustered text data; and Based on the relevance between the unclustered text data and the multiple candidate topic events, the unclustered text data is merged with the clustered text datasets corresponding to the topic events to obtain the target text dataset.

6. The method according to claim 1, wherein, For each topic event, the target text dataset is processed using the Latent Dirichlet Distribution algorithm to obtain multiple related sub-events, including: For each topic event, keywords are extracted from the target text dataset to obtain a keyword set; Based on the predetermined weights of the keywords, the keyword set is processed using the Latent Dirichlet Algorithm to obtain the number of relevant associated sub-events in the target text dataset. Based on the term frequency-inverse document frequency algorithm, the target text dataset is processed to obtain a second text feature matrix, wherein the elements in the second text feature matrix represent the term frequencies of keywords used to describe the topic events corresponding to the target text dataset; and Based on the K-centroid clustering algorithm, the second text feature matrix is ​​clustered according to the number of associated sub-events to obtain multiple associated sub-events with correlation.

7. The method according to claim 6, wherein, The step of processing the keyword set based on the predetermined weights of the keywords using the Latent Dirichlet Algorithm to obtain the number of relevant associated sub-events in the target text dataset includes: Based on the predetermined weights of the keywords, the keyword set is processed using the Latent Dirichlet Algorithm to obtain the text consistency coefficient in the target text dataset corresponding to the predetermined number of associated events; and Based on the text consistency coefficient, the number of related sub-events with correlation in the target text dataset is obtained.

8. The method according to claim 1, wherein, The multiple related sub-events include multiple sub-events in the first cluster and multiple sub-events in the second cluster. The step of generating an event evolution relationship tree corresponding to each topic event based on the occurrence times of the multiple related sub-events includes: If it is determined that the earliest occurrence time of the sub-event in the first cluster is earlier than the earliest occurrence time of the sub-event in the second cluster, then multiple sub-events in the first cluster and the earliest occurrence time of the sub-event in the second cluster are taken as the backbone nodes. Based on the main node and other sub-events in the second cluster, an event evolution relationship tree corresponding to each topic event is generated.

9. The method according to claim 8, wherein, The step of generating an event evolution relationship tree corresponding to each topic event based on the trunk node and other sub-events in the second cluster includes: The main nodes are sequentially associated according to the forward order of the occurrence times of the sub-events corresponding to the main nodes to generate the main branch of the event evolution relationship tree; According to the forward order of the occurrence times of other sub-events in the second cluster, the other sub-events are sequentially associated as branch nodes to generate branches of the event evolution relationship tree; Based on the correlation between the trunk node and the branch node, the trunk and the branch are associated to generate the event evolution relationship tree.

10. A device for generating an event evolution relationship tree, comprising: The first processing module is used to process the text dataset to be processed based on the word frequency-inverse document frequency algorithm to obtain a first text feature matrix, wherein each element in the first text feature matrix represents the word frequency of the keyword used to describe the event; The module is used to obtain the density boundary threshold based on the Euclidean distance between each element in the first text feature matrix and other elements. The clustering module is used to cluster the elements in the first text feature matrix based on the density boundary threshold to obtain multiple target text datasets corresponding to multiple topic events; The second processing module is used to process the target text dataset corresponding to each topic event based on the Latent Dirichlet Algorithm to obtain multiple related sub-events; and The generation module is used to generate an event evolution relationship tree corresponding to each main event based on the occurrence time of the multiple associated sub-events.

Images