Event extraction method and device, computer device, and storage medium

Abstract

The application discloses an event extraction method, which comprises the following steps: obtaining original corpus, performing recall processing on the original corpus to obtain a text set to be extracted corresponding to a preset date span; performing extraction processing on all the text to be extracted to obtain alternative peak phrases; obtaining time word frequency and inverse time word frequency corresponding to each alternative peak phrase, and determining time word frequency values of the alternative peak phrases according to the time word frequency and the inverse time word frequency; screening event peak phrases from all the alternative peak phrases according to the time word frequency values; and performing clustering and merging on all the event peak phrases to obtain at least one text event. The application screens the event peak phrases from all the alternative peak phrases through the time word frequency values, and performs clustering and merging on all the event peak phrases to obtain at least one text event, thereby improving the extraction accuracy of the text event and enhancing the extraction effect of the text event.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of text processing, and in particular to an event extraction method and device, computer equipment and a storage medium. BACKGROUND

[0002] With the continuous development of science and technology, the Internet has gradually become the main carrier for people to publish, obtain and transmit information. For hot events on the Internet, a large number of websites often report and reprint, so how to extract useful information for users from unstructured text of different websites has gradually become a problem that people are concerned about, and information extraction technology has emerged as the times require.

[0003] In the prior art, a coarse-grained or fine-grained extraction method is often used to extract text events. The coarse-grained extraction method determines the text event by detecting and tracking the event content of the text, but this method is not easy to distinguish events with similar semantics. The fine-grained extraction method determines the text event by detecting the event action of the text, but this method needs to define the event extraction mode in advance. Both of the above methods have the problems of low accuracy of event extraction and poor effect of event extraction. SUMMARY

[0004] The embodiments of the present application provide an event extraction method, device, computer equipment and storage medium to solve the problems of low accuracy of event extraction and poor effect of event extraction in the prior art.

[0005] An event extraction method comprises:

[0006] Obtaining original corpus, performing recall processing on the original corpus to obtain a set of to-be-extracted texts corresponding to a preset date span; the set of to-be-extracted texts comprises at least one to-be-extracted text; each to-be-extracted text is associated with a publication date within the preset date span;

[0007] Performing extraction processing on all the to-be-extracted texts to obtain candidate peak phrases;

[0008] Obtaining time word frequencies and inverse time word frequencies corresponding to each candidate peak phrase, and determining a time word frequency value of each candidate peak phrase according to the time word frequencies and the inverse time word frequencies; the time word frequency refers to the total number of occurrences of each candidate peak phrase in all to-be-extracted texts associated with the same publication date; the inverse time word frequency refers to the total number of dates with a time word frequency greater than zero corresponding to each candidate peak phrase within all publication dates within the preset date span;

[0009] According to the time word frequency value, filtering out an event peak phrase from all the candidate peak phrases;

[0010] clustering and merging all the event peak phrases to obtain at least one text event.

[0011] An event extraction apparatus comprises:

[0012] a text recall module configured to obtain an original corpus, and perform recall processing on the original corpus to obtain a set of to-be-extracted texts corresponding to a preset date span; the set of to-be-extracted texts comprises at least one to-be-extracted text; each to-be-extracted text is associated with a publication date within the preset date span;

[0013] a phrase extraction module configured to perform extraction processing on all the to-be-extracted texts to obtain candidate peak phrases;

[0014] a time word frequency value determination module configured to obtain time word frequencies and inverse time word frequencies corresponding to the candidate peak phrases, and determine time word frequency values of the candidate peak phrases according to the time word frequencies and the inverse time word frequencies; the time word frequency refers to a total occurrence number of each candidate peak phrase in all the to-be-extracted texts associated with the same publication date; the inverse time word frequency refers to a total number of dates corresponding to each candidate peak phrase with a time word frequency greater than zero within all the publication dates within the preset date span;

[0015] a phrase screening module configured to screen event peak phrases from all the candidate peak phrases according to the time word frequency values;

[0016] a text event determination module configured to cluster and merge all the event peak phrases to obtain at least one text event.

[0017] A computer device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the event extraction method described above when executing the computer program.

[0018] A computer readable storage medium stores a computer program, and the computer program is executable on a processor to implement the event extraction method described above.

[0019] This invention provides an event extraction method, apparatus, computer device, and storage medium. The method retrieves all text related to the original corpus through recall, thereby obtaining all text to be extracted. Phrase segmentation is performed on all text to be extracted to obtain phrase extraction results, and the word frequency value of each phrase extraction result is determined, thus enabling the selection of candidate peak phrases from all phrase extraction results. Event peak phrases are selected from all candidate peak phrases using time word frequency values, and all event peak phrases are clustered and merged to obtain at least one text event, thereby improving the accuracy and effectiveness of text event extraction. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of the application environment of the event extraction method in one embodiment of the present invention;

[0022] Figure 2 This is a flowchart of an event extraction method according to an embodiment of the present invention;

[0023] Figure 3 This is a flowchart of step S50 in the event extraction method of one embodiment of the present invention;

[0024] Figure 4 This is a flowchart of step S501 in the event extraction method of one embodiment of the present invention;

[0025] Figure 5 This is a schematic block diagram of an event extraction device according to an embodiment of the present invention;

[0026] Figure 6 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] The event extraction method provided in this embodiment of the invention can be applied to, for example... Figure 1The application environment shown. Specifically, the event extraction method is applied in an event extraction device, which includes a client and a server as shown Figure 1 The server, the client and the server communicate through the network, to solve the problems of low accuracy of event extraction and poor effect of event extraction in the prior art. The server can be an independent server, or a cloud server providing cloud services, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, content delivery network (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platform. The client, also known as the user end, is a program that provides classification services for clients corresponding to the server. The client can be installed on various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices, but is not limited thereto.

[0029] In an embodiment, as shown in Figure 2 An event extraction method is provided, which is applied in the client in Figure 1 The steps are as follows:

[0030] S10: Obtain the original corpus, perform recall processing on the original corpus, and obtain a set of to-be-extracted texts corresponding to a preset date span; the set of to-be-extracted texts includes at least one to-be-extracted text; each to-be-extracted text is associated with a publication date within the preset date span.

[0031] Understandably, the original corpus is the name or theme content of an event, such as the reduction of the automobile purchase tax. The set of to-be-extracted texts is a collection of all texts containing the original corpus. The to-be-extracted text is a text obtained by recall from the original corpus, and the content of the text needs to be event extracted. The preset date span is from the start date of the text to the end date, such as from August 1, 2022 to August 15, 2022, that is, the preset date span is 15 days. The publication date is the date of publication or the date of reprinting of the text containing the original corpus, such as the publication date of the Huawei phone conference on September 7, and the publication date of the reprinting of other websites on September 8.

[0032] Specifically, the original corpus is called from the server, or the original corpus is sent to the server by the user from the client. The original corpus is semantically expanded, that is, the original corpus is first corrected, and it is detected whether the text in the original corpus is wrong. When the text is wrong, the original corpus is corrected to obtain the correct corpus. Then the correct corpus is segmented, the segmentation result is obtained, the weight of the segmentation result is calculated, and the user query intention is determined according to the word weight. Finally, the semantic similarity of the user query intention is matched, or the similar corpus data is obtained through the defined synonym library, so that the subsequent recall content is more comprehensive. The user query intention and the similar corpus data are recalled, that is, all texts corresponding to the user query intention and the similar corpus data are obtained through text matching, and all the obtained texts are determined as the extracted text, and then the extracted text set is constructed through all the extracted texts.

[0033] S20: performing extraction processing on all the extracted texts to obtain candidate peak phrases.

[0034] Understandably, the candidate peak phrase is used as a candidate phrase for event extraction. The candidate peak phrase is a phrase obtained by performing phrase extraction processing on the extracted text through the HanLP tool. HanLP is a natural language processing toolkit, which covers many aspects of natural language processing, such as Chinese word segmentation, keyword extraction, and pinyin conversion.

[0035] Specifically, after obtaining the extracted text set, the extracted text set is input into the HanLP tool, and all the extracted texts are analyzed based on the Chinese natural language processing HanLP algorithm. That is, the extracted text is first segmented according to the HanLP algorithm, and then the phrase extraction is performed on the extracted text according to the dependency analysis, so as to generate the phrase extraction result corresponding to each extracted text. Then, the word frequency value is calculated, and the candidate peak phrase is selected from all the phrase extraction results. The dependency parsing (DP) analyzes the dependency relationship between the text phrases, reveals the syntactic structure, that is, analyzes the grammatical components such as “subject-predicate-object” or “determinative-adverbial-complement” in the sentence, and analyzes the relationship between the phrases, so as to extract the extracted text and obtain the phrase extraction result.

[0036] S30: Obtain the time term frequency and the inverse time term frequency corresponding to each of the candidate peak phrases, and determine the time term frequency value of each of the candidate peak phrases according to the time term frequency and the inverse time term frequency; the time term frequency refers to the total number of occurrences of each of the candidate peak phrases in all the texts to be extracted associated with the same publication date; the inverse time term frequency refers to the total number of dates corresponding to each of the candidate peak phrases with the time term frequency greater than zero in all the publication dates within the preset date span.

[0037] Understandably, the time term frequency refers to the total number of occurrences of each candidate peak phrase in all the texts to be extracted associated with the same publication date. The inverse time term frequency refers to the total number of dates corresponding to each candidate peak phrase with a time term frequency greater than zero in all publication dates within the preset date span. The time term frequency value is the value of the time term frequency-inverse time frequency of the phrase extraction result at the same time. The formula of the time term frequency is , where T is the time period, i.e. the number of days during which the text is continuously published, i is the number of days since the publication of the text, is the number of occurrences of the candidate peak phrase. The formula of the inverse time frequency is , maxT is the last publication date in the preset date span, e.g. August 15, 2022, and minT is the first publication date in the preset date span, e.g. August 1, 2022.

[0038] Specifically, after obtaining all the candidate peak phrases, the occurrence of each candidate peak phrase in all the texts to be extracted associated with the same publication date is determined, and the number of times the candidate peak phrase appears in all the texts to be extracted is determined. Then, each candidate peak phrase is calculated according to the preset time term frequency formula to obtain the time term frequency corresponding to the candidate peak phrase. The total number of days each candidate peak phrase appears in the preset time span is determined, and the inverse time term frequency is calculated according to the total number of days and the total number of occurrences of the candidate peak phrase in the total number of days, i.e. two data are input into the preset inverse time term frequency formula to calculate each candidate peak phrase to obtain the inverse time term frequency corresponding to the candidate peak phrase. The time term frequency and the inverse time term frequency corresponding to the candidate peak phrase are multiplied to determine the time term frequency value corresponding to the candidate peak phrase. In this way, the time term frequency values corresponding to all the candidate peak phrases are determined by the above method.

[0039] S40: According to the time term frequency value, screen the event peak phrase from all the candidate peak phrases.

[0040] Understandably, the event peak phrase is a candidate phrase for time extraction, which is screened from all the candidate peak phrases.

[0041] Specifically, after obtaining the time word frequency values, all the time word frequency values are sequentially sorted from large to small, the sorted all time word frequency values are screened by a box plot analysis method, time word frequency values greater than or equal to the interval maximum value are retained, that is, time word frequency values less than the interval maximum value are deleted. And the reserved time word frequency value corresponding to the candidate peak phrase is determined as the event peak phrase, and then the event peak phrase set is constructed according to all the event peak phrases.

[0042] S50, all the event peak phrases are clustered and merged to obtain at least one text event.

[0043] It can be understood that the text event is the theme content extracted from the text to be extracted, and the text event can be used to represent the text to be extracted.

[0044] Specifically, after obtaining the event peak phrase, the event peak phrase is clustered and merged by a community discovery algorithm, T cluster centers are selected as nodes, that is, the event peak phrase with the maximum time word frequency value in the same publication date is selected as the cluster center. The weight value between the two event peak phrases in the same publication date is determined by calculating the mutual information entropy and the cosine similarity between the two event peak phrases in the same publication date. And by calculating the weight value between the same event peak phrases in adjacent two publication dates, that is, the weight value of the same event peak phrase in adjacent two publication dates is set to a constant. And according to the phrase node graph composed of all the weight values, and inputting the phrase node graph into the community discovery algorithm, the community discovery algorithm is used to cluster and merge the time peak vector according to the phrase node graph, and a preset number (such as 4, 6 or 8) of text events are obtained.

[0045] In an event extraction method in an embodiment of the application, all texts related to the original corpus are obtained by recall, so as to obtain all the text to be extracted. The phrase extraction result is obtained by performing phrase segmentation on all the text to be extracted, and the word frequency value of each phrase extraction result is determined, so as to realize the screening of the candidate peak phrase from all the phrase extraction results. The event peak phrase is screened from all the candidate peak phrases by the time word frequency value, and all the event peak phrases are clustered and merged to obtain at least one text event, so as to improve the extraction accuracy of the text event and improve the extraction effect of the text event.

[0046] In an embodiment, in step S20, that is, the extraction processing of all the text to be extracted is performed to obtain the candidate peak phrase, which includes:

[0047] S201, all the text to be extracted is segmented to obtain a phrase segmentation result.

[0048] S202, extract all phrase segmentation results by dependency syntax analysis to obtain phrase extraction results corresponding to each of the to-be-extracted texts.

[0049] Understandably, the phrase segmentation result is obtained by segmenting the to-be-extracted text by the segmentation function in the hanlp tool. The phrase extraction result is a phrase with a subject-predicate-object relationship or a verb-object relationship in the to-be-extracted text.

[0050] Specifically, after obtaining the set of to-be-extracted texts, filter out noise words such as stop words and mood words, low-frequency words, and sentences that are not smooth in all to-be-extracted texts. Segment all to-be-extracted texts by the hanlp tool, that is, split all to-be-extracted texts into one by one phrase segmentation results according to the Chinese segmentation method in the hanlp tool. Further, extract phrases from the phrase segmentation results according to the dependency syntax analysis, that is, extract phrase extraction results from the phrase segmentation results according to the subject-predicate-object relationship, the determinative relationship, and the verb-object relationship. In this way, all phrase extraction results are extracted from all phrase segmentation results by this method.

[0051] S203, determine the word frequency and inverse document frequency of each phrase extraction result according to all phrase extraction results.

[0052] Understandably, the word frequency of the phrase extraction result is the frequency of the phrase extraction result appearing in the to-be-extracted text, that is, the proportion of the number of times the phrase extraction result appears in the to-be-extracted text to the total number of words in the to-be-extracted text (all phrase extraction results of the to-be-extracted text). The inverse document frequency is a way to measure the importance of a word item in a document. The principle of the inverse document frequency is that for a certain phrase extraction result, the fewer the to-be-extracted texts containing the phrase extraction result, the stronger the text features of the phrase extraction result.

[0053] Specifically, calculate the word frequency and inverse document frequency of the phrase extraction result in the same to-be-extracted text, that is, calculate the word frequency of the phrase extraction result according to the preset word frequency formula TF=F / N (F is the number of times the phrase extraction result appears in the to-be-extracted text, and N is the number of all phrase extraction results of the to-be-extracted text) to obtain the word frequency corresponding to the phrase extraction result. The word frequency corresponding to all phrase extraction results is determined by the above method. Calculate the inverse document frequency of the phrase extraction result according to the preset inverse document frequency formula IDF=log(M / (DF+1)) (M is the number of all to-be-extracted texts in the set of to-be-extracted texts, and DF is the number of all to-be-extracted texts containing the phrase extraction result) to obtain the inverse document frequency corresponding to the phrase extraction result. In this way, the inverse document frequency corresponding to all phrase extraction results is determined by the above method.

[0054] S204, determine a term frequency value of each phrase extraction result according to the term frequency and the inverse document frequency of all the phrase extraction results.

[0055] S205, screen all the phrase extraction results according to the term frequency value, and obtain all the candidate peak phrases.

[0056] Specifically, after determining the term frequency and the inverse document frequency corresponding to the same phrase extraction result, the term frequency and the inverse document frequency corresponding to the same phrase extraction result are multiplied, that is, TF-IDF=F / N*log(M / (DF+1)), so as to obtain the term frequency value corresponding to the phrase extraction result. In this way, the term frequency value corresponding to each phrase extraction result is obtained by the above-mentioned manner. Further, the term frequency values corresponding to all phrase extraction results are sequentially sorted from large to small, and the term frequency values are screened by a preset percentage (such as 70%), that is, the term frequency values ranked in the front of the preset percentage (such as 70%) are retained, and the phrase extraction results corresponding to the retained term frequency values are determined as the candidate peak phrases, and then the candidate peak phrase set is constructed according to all the candidate peak phrases.

[0057] The embodiment of the present application realizes the determination of the term frequency value of the phrase extraction result by calculating the term frequency and the inverse document frequency of all the phrase extraction results. The screening of all the phrase extraction results is realized by sorting all the term frequency values and screening the sorted term frequency values, the determination of the candidate peak phrase is realized, and the accuracy of the subsequent text event extraction is further improved.

[0058] In an embodiment, in step S40, that is, according to the time term frequency value, the event peak phrase is screened from all the candidate peak phrases, which includes:

[0059] S401, sort all the time term frequency values, and determine the lower quartile, the upper quartile and the interquartile range in the time term frequency values, and obtain a box plot.

[0060] Understandably, the lower quartile is the predicted sample value ranked in 25%. The upper quartile is the predicted sample value ranked in 75%. The interquartile range is the distance between the upper quartile and the lower quartile. The interval maximum value is Q1-1.5IQR in the interval of the box plot (Q1-1.5IQR, Q3+1.5IQR). Q1 is the lower quartile, that is, the time term frequency value ranked in 25%. Q3 is the upper quartile, that is, the time term frequency value ranked in 75%. IQR is the interquartile range, that is, the distance between Q3 and Q1.

[0061] Specifically, after obtaining the time word frequency values, the time word frequency values are sorted from large to small or from small to large, and in this embodiment, the time word frequency values are sorted from large to small. The time word frequency values ranked in the lower 25% are determined, and the time word frequency values are determined as the lower quartile. The position of the lower quartile on the number axis is determined. The time word frequency values ranked in the upper 75% are determined, and the time word frequency values are determined as the upper quartile. The position of the upper quartile on the number axis is determined. The lower quartile is subtracted from the upper quartile to obtain the interquartile range. Thus, the graph composed of the lower quartile, the upper quartile, and the interquartile range is determined as the box plot.

[0062] In S402, the interval maximum value is determined according to the lower quartile and the interquartile range in the box plot.

[0063] In S403, all the time word frequency values and the interval maximum value are compared, and the candidate peak phrase corresponding to the time word frequency value greater than the interval maximum value is recorded as the event peak phrase.

[0064] Specifically, after obtaining the box plot, the interval minimum value and the interval maximum value are determined according to the lower quartile, the upper quartile, and the interquartile range in the box plot, so as to obtain the screening interval. That is, the screening interval is (Q1-1.5IQR, Q3+1.5IQR), the interval maximum value is Q1-1.5IQR, and the interval minimum value is Q3+1.5IQR. All the time word frequency values and the interval maximum value are compared, and the time word frequency value greater than the interval maximum value is retained, that is, the time word frequency value less than the interval maximum value is removed, and the candidate peak phrase corresponding to the time word frequency value greater than the interval maximum value is determined as the event peak phrase. In this way, all the event peak phrases are determined by the above method.

[0065] In this embodiment, the lower quartile, the upper quartile, and the interquartile range in the sorted time word frequency values are determined, so as to construct the box plot. All the time word frequency values are screened by the box plot analysis method, so as to determine the event peak phrase, and the accuracy of subsequent text event extraction is improved.

[0066] In one embodiment, as shown in FIG. 5, Figure 3 In S50, all the event peak phrases are clustered and merged to obtain at least one text event, including:

[0067] In S501, all the event peak phrases appearing in the to-be-extracted text published on the same publication date are obtained, and nodes in which each event peak phrase is located in the publication date are constructed.

[0068] S502, connecting all the nodes in the same publishing date to each other to construct a first type of edge and determine a first edge weight value of the first type of edge.

[0069] It can be understood that the node is a connection point, which is used to represent an event peak phrase. The first type of edge is a connection between nodes in the same publishing date. The first edge weight value is the weight value of the first type of edge.

[0070] Specifically, after obtaining the event peak phrase, all event peak phrases appearing in all to-be-extracted texts published in the same publishing date are determined, and all event peak phrases in the same publishing date are determined as nodes of the publishing date. In this way, all nodes in all publishing dates within a preset date span are determined, that is, all event peak phrases in all publishing dates within the preset date span are determined as nodes of the publishing date. Further, all nodes in the same publishing date are connected to each other, thereby constructing the first type of edge. In this way, all nodes in all publishing dates within the preset date span are connected to each other, thereby constructing the first type of edge of all different publishing dates. The mutual information entropy between two nodes is calculated, such as ), is the node i, is the node j, and is the mutual information entropy between the node j and the node i. Then, the cosine similarity of the event vectors corresponding to the two nodes is calculated, such as calculating the cosine value between the event vector corresponding to the node i and the event vector corresponding to the node j, and determining the cosine value as the cosine similarity between the two nodes. Then, according to the mutual information entropy and the cosine similarity of the two nodes, the first edge weight value of the first type of edge is determined.

[0071] S503, connecting the nodes corresponding to the same event peak phrase and simultaneously in adjacent two publishing dates to each other to construct a second type of edge and determine a second edge weight value.

[0072] S504, constructing a phrase node graph according to the first edge weight value, the second edge weight value, and the edge, and clustering and merging all the event peak phrases according to the phrase node graph to obtain at least one text event.

[0073] It can be understood that the second type of edge is a connection between nodes corresponding to the same event peak phrase in adjacent two publishing dates. The second edge weight value is the weight value of the second type of edge, which is a constant greater than 1. The phrase node graph is a node graph composed of event peak phrases, which is commonly used in community discovery algorithms, such as Louvain algorithm.

[0074] Specifically, the phrase node graph is constructed according to all the first edge weight values, all the first type edges, all the second edge weight values and all the second type edges, that is, the graph formed by all the first edge weight values, all the first type edges, all the second edge weight values and all the second type edges is determined as the phrase node graph. Wherein, from the first publication date to the last publication date within the preset date span, the nodes of all the publication dates are sequentially encoded, such as node encoding. The phrase node graph is sent to a preset community discovery algorithm, and the community discovery algorithm clusters and merges the nodes in the phrase node graph, thereby clustering and merging the event peak phrases, and obtaining event clustering clusters. The center time peak phrase in each event clustering cluster is determined as a text event, thereby obtaining multiple text events.

[0075] The embodiment of the present application realizes the construction of the first type edges and the second type edges by connecting all the nodes within the same publication date with each other and connecting the nodes corresponding to the same event peak phrase in the adjacent two publication dates. The extraction of the text event is realized by constructing the phrase node graph and clustering and merging the event peak phrases according to the phrase node graph, thereby improving the subsequent extraction accuracy of the text event.

[0076] In an embodiment, as shown in FIG. 5, the step S502, that is, determining the first edge weight value of the first type edge, includes: Figure 6

[0077] S5021, determining the mutual information entropy and the cosine similarity of all the first type edges within the same publication date.

[0078] S5022, determining the first edge weight value according to the mutual information entropy and the cosine similarity corresponding to each first type edge.

[0079] Understandably, the mutual information entropy is a measure of the correlation degree between two event peak phrases. The cosine similarity is the similarity between two event vectors. The mutual information entropy is MI(X, Y)=H(X)+H(Y)-H(X, Y), wherein MI(X, Y) is the mutual information entropy between the event peak phrase X and the event peak phrase Y, H(X) is the information amount contained in the event peak phrase X about the other event peak phrase Y, H(Y) is the information amount contained in the event peak phrase Y about the other event peak phrase X, and H(X, Y) is the mutual information repeated by two event peak phrases.

[0080] ​Specifically, mutual information entropy between two nodes in the same publication date is calculated, that is, the amount of information about another event peak phrase Y contained in an event peak phrase X between two nodes and the amount of information about an event peak phrase X contained in another event peak phrase Y. The mutual information entropy between two event peak phrases is determined by the mutual information entropy relationship. And the cosine value between the event vector corresponding to the event peak phrase X and the event vector corresponding to the event peak phrase Y is calculated by the cosine similarity method, so as to obtain the cosine similarity between the two event vectors. Further, the square root of the two nodes is obtained by multiplying and square root processing of the mutual information entropy and the cosine similarity corresponding to the two nodes through a preset relationship, so as to determine the first edge weight value between the two nodes. In this way, the first edge weight value between all nodes is obtained through the above-mentioned manner.

[0081] The embodiments of the present application calculate the mutual information entropy and the cosine similarity between the event vectors in the same publication date, so as to determine the edge weight value between two nodes, and further determine the edge weight value between all two nodes. Further, the accuracy of the text event is improved, and the extraction effect of the text event is improved.

[0082] In an embodiment, in step S5021, that is, determining the first edge weight value of the first type of edge, includes:

[0083] S50211, obtaining all the to-be-extracted texts in which each event peak phrase appears.

[0084] S50212, converting the determined all to-be-extracted texts into vectors through a preset conversion model to obtain text vectors corresponding to each to-be-extracted text.

[0085] Understandably, the preset conversion model is a model used for converting text into vector in advance, which can be a neural network model trained by a large amount of training data, and the Bert model is used in the embodiment. The text vector is the vector corresponding to the to-be-extracted text, that is, the embedding vector.

[0086] Specifically, after obtaining the event peak phrase, the to-be-extracted text in which the event peak phrase appears is determined, and the to-be-extracted text in which the event peak phrase appears is obtained, such as the event peak phrase being the car purchase tax reduction, and the to-be-extracted text in which the car purchase tax reduction appears in all to-be-extracted texts is obtained. In this way, all to-be-extracted texts in which the event peak phrase appears are obtained by the above-mentioned manner. Further, a preset conversion model is obtained from the server, and all to-be-extracted event texts corresponding to the same event peak phrase are sequentially input into the preset conversion model, and all to-be-extracted event texts are sequentially converted into text vectors by the preset conversion model. The conversion process is to encode the to-be-extracted event text by the encoding module to obtain the text word vector corresponding to the to-be-extracted event text. Then, the multiple sets of three weight matrices W Q K V are used to calculate all text word vectors to obtain multiple sets of Query, Keys, and Values vectors.

[0087] Further, the correlation score between the text word vectors is calculated using the dot product method, that is, the dot product of each text word vector in Q and each text word vector in K is calculated. The correlation score between the text word vectors is normalized, that is, the score between the text word vectors is converted into a probability distribution between 0 and 1. According to the probability distribution between the text word vectors, the corresponding Values value is multiplied to obtain multiple matrices. The obtained multiple matrices are spliced, and the spliced matrix is processed by the first residual connection layer to obtain a residual matrix, which avoids the degradation problem in model training. Then, the residual matrix is normalized by the first normalization layer. Then, the normalized residual matrix is activated by the ReLU function, and the activated residual matrix is normalized by the second residual connection layer and the second normalization layer, that is, the text vector can be obtained after at least one of the above processes. In this way, the text vector corresponding to all to-be-extracted event texts is obtained by the above-mentioned manner.

[0088] S50213, weighted average processing is performed on all the text vectors to obtain an event vector corresponding to the event peak phrase.

[0089] S50214, according to the event vector, the cosine similarity of all the first type edges in the same publication date is determined.

[0090] Understandably, the event vector is the average vector of all text vectors corresponding to the same event peak phrase. The preset weighted relationship is , wherein, is the event vector, is the number of to-be-extracted event texts, ​​The text vector corresponding to the event peak phrase.

[0091] Specifically, after obtaining the text vectors corresponding to all the texts to be extracted, all the text vectors corresponding to the same event peak phrase are weighted and averaged according to a preset weighting relationship, so as to obtain the event vector corresponding to the event peak phrase. In this way, the event vectors of all the event peak phrases are sequentially calculated in the above manner. The cosine similarity of all the first-type edges in the same publication date is determined by calculating the cosine value between the two event vectors.

[0092] The embodiment of the present application determines all the texts to be extracted in which the event peak phrase appears, and converts all the texts to be extracted corresponding to the same event peak phrase into text vectors through a preset conversion model, so as to realize the conversion of the text vectors. The text vectors corresponding to the same event peak phrase are averaged according to a preset weighting relationship, so as to realize the acquisition of the event vector, and then the cosine value of the event vector is calculated, so as to realize the determination of the cosine similarity of the first-type edges.

[0093] In an embodiment, in step S504, that is, clustering and merging all the event peak phrases according to the phrase node graph to obtain at least one text event, comprises:

[0094] S5041, input the phrase node graph into the community discovery algorithm, so that the community discovery algorithm clusters and merges all the event peak phrases according to the phrase node graph to obtain at least one event clustering cluster.

[0095] S5042, determine the event peak phrase corresponding to each event clustering cluster center as the text event.

[0096] Understandably, the community discovery algorithm is an algorithm based on multi-level optimization modulation. In this embodiment, the Louvain algorithm is used, which is a community discovery algorithm based on modularity.

[0097] Specifically, after obtaining the phrase node graph, the phrase node graph is input to the Louvain algorithm. The Louvain algorithm divides all nodes according to the modularity to obtain different communities. That is, each node takes itself as a community label, and each node traverses all neighbor nodes of itself, tries to update the community label of itself to the community label of the neighbor node with the largest modularity increment, until all nodes cannot increase the modularity by changing the community label. Each community is merged into a new supernode, and the edge weight value of the supernode is the sum of the edge weight values of all nodes in the original community, forming a new network. Each supernode is determined as an event clustering cluster, thereby obtaining a plurality of event clustering clusters. The event peak phrase corresponding to the center of the plurality of event clustering clusters is determined as a text event, thereby obtaining a plurality of text events.

[0098] The embodiment of the present application clusters and merges the event peak phrases according to the event phrase graph by the community discovery algorithm, thereby achieving the acquisition of the preset number of event clustering clusters. And the event peak phrase of the center of each event clustering cluster is determined as a text event, thereby achieving the event extraction of the text to be extracted. Further, the text event extracted based on the method can supplement the research of event topic detection and tracking, and can provide additional information for the subsequent action extraction model.

[0099] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0100] In an embodiment, an event extraction device is provided, which corresponds to the event extraction method in the above embodiment. As shown in the figure, the event extraction device includes a text recall module 11, a phrase extraction module 12, a time word frequency value module 13, a phrase screening module 14 and a text event determination module 15. The functions of each module are described in detail as follows: Figure 5

[0101] The text recall module 11 is used to acquire an original corpus, perform recall processing on the original corpus, and obtain a set of text to be extracted corresponding to a preset date span; the set of text to be extracted includes at least one text to be extracted; each text to be extracted is associated with a publication date within the preset date span;

[0102] The phrase extraction module 12 is used to perform extraction processing on all the text to be extracted to obtain candidate peak phrases;

[0103] ​The time word frequency value determination module 13 is configured to acquire time word frequencies and inverse time word frequencies corresponding to the candidate peak phrases, and determine time word frequency values of the candidate peak phrases according to the time word frequencies and the inverse time word frequencies; the time word frequency refers to a total occurrence number of each candidate peak phrase in all the texts to be extracted associated with the same publication date; the inverse time word frequency refers to a total number of dates corresponding to each candidate peak phrase with the time word frequency greater than zero in all the publication dates within the preset date span;

[0104] The phrase screening module 14 is configured to screen event peak phrases from all the candidate peak phrases according to the time word frequency values.

[0105] The text event determination module 15 is configured to cluster and merge all the event peak phrases to obtain at least one text event.

[0106] In an embodiment, the phrase extraction module 12 comprises:

[0107] The word segmentation processing unit is configured to perform word segmentation processing on all the texts to be extracted to obtain phrase segmentation results;

[0108] The phrase extraction unit is configured to perform extraction processing on all the phrase segmentation results through dependency syntax analysis to obtain phrase extraction results corresponding to each text to be extracted;

[0109] The word frequency determination unit is configured to determine word frequencies and inverse text frequencies of the phrase extraction results according to all the phrase extraction results;

[0110] The word frequency value determination unit is configured to determine word frequency values of the phrase extraction results according to the word frequencies and the inverse text frequencies of all the phrase extraction results;

[0111] The candidate phrase screening unit is configured to screen all the phrase extraction results according to the word frequency values to obtain all the candidate peak phrases.

[0112] In an embodiment, the phrase screening module 14 comprises:

[0113] The box plot determination subunit is configured to sort all the time word frequency values, and determine lower quartiles, upper quartiles and interquartile ranges in the time word frequency values to obtain a box plot;

[0114] The interval maximum value determination subunit is configured to determine an interval maximum value according to the lower quartiles and the interquartile ranges in the box plot;

[0115] The event peak phrase determination subunit is configured to compare all the time word frequency values and the interval maximum values, and record the candidate peak phrase corresponding to the time word frequency value greater than the interval maximum value as the event peak phrase.

[0116] In an embodiment, the text event determination module 15 comprises:

[0117] The node construction sub-module is configured to obtain the event peak phrases appearing in all the to-be-extracted texts published on the same publication date, and construct nodes of each event peak phrase in the publication date;

[0118] The first-type edge sub-module is configured to connect all the nodes in the same publication date two by two to construct first-type edges, and determine first edge weight values of the first-type edges;

[0119] The second-type edge sub-module is configured to connect the nodes corresponding to the same event peak phrase and simultaneously in adjacent two publication dates to construct second-type edges, and determine second edge weight values;

[0120] The phrase clustering sub-module is configured to construct a phrase node graph according to the first edge weight values, the second edge weight values, and the edges, cluster and merge all the event peak phrases according to the phrase node graph, and obtain at least one text event.

[0121] In an embodiment, the first-type edge sub-module comprises:

[0122] The similarity determination unit is configured to determine mutual information entropy and cosine similarity of all the first-type edges in the same publication date;

[0123] The first edge weight value unit is configured to determine the first edge weight values according to the mutual information entropy and the cosine similarity corresponding to each first-type edge.

[0124] In an embodiment, the similarity determination unit comprises:

[0125] The text acquisition sub-unit is configured to acquire all the to-be-extracted texts in which each event peak phrase appears;

[0126] The vector conversion sub-unit is configured to perform vector conversion on all the to-be-extracted texts determined by a preset conversion model to obtain text vectors corresponding to each to-be-extracted text;

[0127] The event vector sub-unit is configured to perform weighted average processing on all the text vectors to obtain an event vector corresponding to the event peak phrase;

[0128] a cosine similarity subunit configured to determine cosine similarity of all the first-type edges within the same publishing date according to the event vector.

[0129] In an embodiment, the phrase clustering sub-module comprises:

[0130] an event cluster determination subunit configured to input the phrase node graph into a community discovery algorithm, so as to make the community discovery algorithm cluster and merge all the event peak phrases according to the phrase node graph, to obtain at least one event cluster;

[0131] a text event determination subunit configured to determine the event peak phrase corresponding to the event cluster center as the text event.

[0132] The specific limitations of the event extraction device can refer to the limitations of the event extraction method described above, which will not be repeated here. Each module in the above event extraction device can be realized by software, hardware and their combinations. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so that the processor calls and executes the operations corresponding to each module.

[0133] In an embodiment, a computer device is provided, which can be a server, and the internal structure diagram thereof can be as shown in Figure 6 The computer device comprises a processor, a memory, a network interface and a database connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store data used by the event extraction method in the above embodiments. The network interface of the computer device is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement an event extraction method.

[0134] In an embodiment, a computer device is provided, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the event extraction method in the above embodiments when executing the computer program.

[0135] In an embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the event extraction method in the above embodiments.

[0136] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct RAM bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0137] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of functional units and modules is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the above-described functions.

[0138] The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. An event extraction method, characterized by, The method comprises the following steps: obtaining original corpus, performing recall processing on the original corpus to obtain a set of to-be-extracted texts corresponding to a preset date span; the set of to-be-extracted texts comprises at least one to-be-extracted text; each to-be-extracted text is associated with a publication date within the preset date span; performing extraction processing on all the to-be-extracted texts to obtain candidate peak phrases; obtaining time word frequencies and inverse time word frequencies corresponding to each candidate peak phrase, and determining time word frequency values of each candidate peak phrase according to the time word frequencies and the inverse time word frequencies; the time word frequency refers to the total number of occurrences of each candidate peak phrase in all the to-be-extracted texts associated with the same publication date; the inverse time word frequency refers to the total number of dates corresponding to each candidate peak phrase with a time word frequency greater than zero within all the publication dates within the preset date span; screening event peak phrases from all the candidate peak phrases according to the time word frequency values; performing clustering and merging on all the event peak phrases to obtain at least one text event; the step of performing clustering and merging on all the event peak phrases to obtain at least one text event comprises the following steps: obtaining the event peak phrases appearing in all the to-be-extracted texts published on the same publication date, and constructing nodes of each event peak phrase within the publication date; connecting all the nodes within the same publication date two by two to construct a first type of edge, and determining a first edge weight value of the first type of edge; connecting the nodes corresponding to the same event peak phrase and simultaneously within two adjacent publication dates to construct a second type of edge, and determining a second edge weight value; constructing a phrase node graph according to the first edge weight value, the second edge weight value and the edge, and performing clustering and merging on all the event peak phrases according to the phrase node graph to obtain at least one text event.

2. The event extraction method of claim 1, wherein, the step of performing extraction processing on all the to-be-extracted texts to obtain candidate peak phrases comprises the following steps: performing word segmentation processing on all the to-be-extracted texts to obtain phrase segmentation results; performing extraction processing on all the phrase segmentation results through dependency syntax analysis to obtain phrase extraction results corresponding to each to-be-extracted text; determining word frequencies and inverse text frequencies of each phrase extraction result according to all the phrase extraction results; determining word frequency values of each phrase extraction result according to the word frequencies and the inverse text frequencies of all the phrase extraction results; screening all the phrase extraction results according to the word frequency values to obtain all the candidate peak phrases.

3. The event extraction method of claim 1, wherein, the step of screening event peak phrases from all the candidate peak phrases according to the time word frequency values comprises the following steps: sorting all the time word frequency values, and determining lower quartiles, upper quartiles and interquartile ranges in the time word frequency values to obtain a box plot; determining an interval maximum value according to the lower quartiles and the interquartile ranges in the box plot; Compare the time word frequency value and the interval maximum value, and record the candidate peak phrase corresponding to the time word frequency value greater than the interval maximum value as the event peak phrase.

4. The event extraction method of claim 1, wherein, The determining of the first edge weight value of the first type of edge comprises: determining mutual information entropy and cosine similarity of all the first type of edges in the same publishing date; determining the first edge weight value according to the mutual information entropy and the cosine similarity corresponding to each first type of edge.

5. The event extraction method of claim 4, wherein, The determining of the cosine similarity of all the first type of edges in the same publishing date comprises: obtaining all the to-be-extracted texts in which each event peak phrase appears; performing vector conversion on all the to-be-extracted texts determined by a preset conversion model to obtain a text vector corresponding to each to-be-extracted text; performing weighted average processing on all the text vectors to obtain an event vector corresponding to the event peak phrase; determining the cosine similarity of all the first type of edges in the same publishing date according to the event vector.

6. The event extraction method of claim 1, wherein, The clustering and merging of all the event peak phrases according to the phrase node graph to obtain at least one text event comprises: inputting the phrase node graph into a community discovery algorithm to enable the community discovery algorithm to cluster and merge all the event peak phrases according to the phrase node graph to obtain at least one event clustering cluster; determining the event peak phrase corresponding to each event clustering cluster center as the text event.

7. An event extraction apparatus characterized by comprising: comprises: a text recall module configured to obtain original corpus, perform recall processing on the original corpus, and obtain a to-be-extracted text set corresponding to a preset date span; the to-be-extracted text set comprises at least one to-be-extracted text; each to-be-extracted text is associated with a publishing date within the preset date span; a phrase extraction module configured to perform extraction processing on all the to-be-extracted texts to obtain candidate peak phrases; a time word frequency value determination module configured to obtain time word frequency and inverse time word frequency corresponding to each candidate peak phrase, and determine a time word frequency value of each candidate peak phrase according to the time word frequency and the inverse time word frequency; the time word frequency refers to the total number of occurrences of each candidate peak phrase in all to-be-extracted texts associated with the same publishing date; the inverse time word frequency refers to the total number of dates in which each candidate peak phrase has a time word frequency greater than zero within all publishing dates in the preset date span; a phrase screening module configured to screen event peak phrases from all the candidate peak phrases according to the time word frequency value; a text event determination module configured to cluster and merge all the event peak phrases to obtain at least one text event; the text event determination module comprises: a node construction submodule configured to obtain the event peak phrases appearing in all to-be-extracted texts published in the same publishing date, and construct nodes of each event peak phrase in the publishing date; The first edge submodule is configured to connect all the nodes in the same publication date to each other to build a first edge and determine a first edge weight value of the first edge; The second edge submodule is configured to connect the nodes corresponding to the same event peak phrase in adjacent two publication dates to build a second edge and determine a second edge weight value; The phrase clustering submodule is configured to build a phrase node graph according to the first edge weight value, the second edge weight value and the edge, and cluster and merge all the event peak phrases according to the phrase node graph to obtain at least one text event.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the event extraction method in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. The computer program is executed by the processor to implement the event extraction method in any one of claims 1 to 6.

Images