Financial document level event extraction method and system based on multi-semantic enhancement

Abstract

The invention provides a financial document level event extraction method and system based on multiple semantic enhancement, and belongs to the field of natural language processing, and the method comprises the steps: constructing a financial document event mode which comprises an event type set and an argument role set of event types; obtaining financial document data to be extracted, and preprocessing and labeling the financial document data to be extracted based on the event type and the argument role to obtain labeled financial document data; and inputting the labeled financial document data into the trained financial document level event extraction model, and obtaining an extraction result containing event types and event arguments for the labeled financial document data by using a semantic matching optimization module, a multi-granularity semantic enhancement module and an event center heterogeneous graph structure module of the financial document level event extraction model. When the financial document is processed, the event information is extracted from the multi-level and cross-paragraph text, and the event extraction accuracy and efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a financial document-level event extraction method and system based on multiple semantic enhancements. Background Art

[0002] Against the backdrop of the rapid development of information technology, event extraction technology in natural language processing has become an important research direction. Especially driven by financial technology, the financial industry has generated a large amount of unstructured text data, such as news reports, financial announcements, and analytical reports. Traditional event extraction methods are mostly based on rules and pattern matching. These methods rely on predefined event patterns. Although they can achieve good results in specific fields, they often perform poorly when dealing with new fields and complex texts. With the rise of deep learning, event extraction methods based on deep learning have gradually replaced traditional methods, especially the introduction of word embedding models (such as Word2Vec, BERT) and graph neural networks, which have greatly improved the model's ability to understand events and their semantic relationships.

[0003] Existing event extraction methods are mainly based on sentence-level extraction technology, which is mostly used to obtain event information from news headlines. However, financial events usually span multiple sentences or even multiple paragraphs, and a single sentence-level event extraction technology is difficult to capture the full picture of the event. For example, a company's financial report may contain multiple levels of information, including performance, financial indicators, business prospects, etc. Only by analyzing the entire report at the document level can relevant event information be fully extracted.

[0004] So far, researchers at home and abroad have proposed a series of methods for financial document-level event extraction. Although some progress has been made, the existing financial document-level event extraction methods still have many limitations, mainly concentrated in the following aspects: 1. Dataset limitations: The existing financial document-level event extraction datasets are not perfect in terms of scale, fine-grained annotation, and event type coverage, making it difficult to fully support diverse event extraction needs.

[0005] 2. Insufficient ability to process noisy data: Financial documents usually contain a large amount of irrelevant information or noisy text, such as background descriptions or other non-event-related content. Existing models find it difficult to effectively filter out these noises, resulting in a decrease in the accuracy of event extraction.

[0006] 3. Limited understanding of complex financial terms: Financial documents often contain a large number of complex financial terms, such as financial indicators, professional terms, and industry-specific expressions. Existing models often have difficulty accurately interpreting these terms, resulting in poor performance of the model when processing texts involving professional knowledge.

[0007] 4. Weak ability to capture cross-sentence and cross-paragraph information: Financial events usually span multiple sentences or even paragraphs, and existing models cannot effectively capture the complete event semantics across sentences and paragraphs in long text analysis, resulting in one-sided extraction results and difficulty in fully reflecting the full picture of the event.

[0008] Therefore, finding a method that can both filter out noise in financial documents to improve the ability to understand financial documents and improve the accuracy of financial event extraction is a technical problem that needs to be urgently solved by technical personnel in this field. Summary of the invention

[0009] The present invention provides a financial document-level event extraction method and system based on multiple semantic enhancement, which is used to solve the defect of the prior art that the deep semantic information in the financial text cannot be captured, and to achieve the extraction of event information from multi-level and cross-paragraph texts when processing financial documents, thereby improving the accuracy and efficiency of event extraction.

[0010] The present invention provides a financial document-level event extraction method based on multiple semantic enhancement, comprising the following steps: Constructing a financial document event model, wherein the financial document event model includes an event type set and an argument role set of the event type; Acquire financial document data to be extracted, and preprocess and annotate the financial document data to be extracted based on event types and argument roles to obtain annotated financial document data; The annotated financial document data is input into the trained financial document-level event extraction model, and the semantic matching optimization module, multi-granularity semantic enhancement module and event-centered heterogeneous graph structure module of the financial document-level event extraction model are used to extract financial document-level events from the annotated financial document data to obtain extraction results including event types and event arguments; wherein the semantic matching optimization module is used to filter and optimize the annotated financial document data, the multi-granularity semantic enhancement module is used to perform context encoding based on the output of the semantic matching optimization module to fuse sentence-level semantic features and paragraph-level semantic features and perform entity recognition, and the event-centered heterogeneous graph structure module is used to capture the interactive relationship between nodes in the annotated financial document data based on the output of the multi-granularity semantic enhancement module.

[0011] According to a financial document-level event extraction method based on multiple semantic enhancement provided by the present invention, the processing steps of the semantic matching optimization module are: Use the financial encoder to contextually encode the annotated financial document data and event templates to obtain high-dimensional vector representations of sentences and event templates; The semantic similarity between the sentence high-dimensional vector representation and the event template high-dimensional vector representation is calculated, and matched with a preset event template similarity threshold to obtain a matching result, and the annotated financial documents are filtered based on the matching result to obtain an optimized document.

[0012] According to a financial document-level event extraction method based on multiple semantic enhancement provided by the present invention, the processing steps of the multi-granularity semantic enhancement module are: The optimized document is used as input text and passed through the financial encoder to obtain sentence-level semantic features and paragraph-level semantic features; The sentence-level semantic features and paragraph-level semantic features are serialized and modeled using a bidirectional gated recurrent unit to obtain serialized semantic features; the formula is: ; ; in, Represents the sentence-level semantic features after Bi-GRU modeling, represents the paragraph-level semantic features after Bi-GRU modeling, represents the operation function of a bidirectional gated recurrent unit; The global information of the serialized semantic features is modeled through the self-attention mechanism to obtain sentence-level global features and paragraph-level global features; the formula is: ; ; in, represents the sentence-level global feature, represents the paragraph-level global features, represents the self-attention calculation function applied to sentence-level global features, represents the self-attention calculation function applied to paragraph-level global features; The sentence-level global features and paragraph-level global features are fused using a bidirectional interactive attention mechanism to obtain multi-granularity semantic fusion features, and entity recognition is performed on the multi-granularity semantic fusion features to obtain an entity set.

[0013] According to a financial document-level event extraction method based on multiple semantic enhancement provided by the present invention, the sentence-level global features and paragraph-level global features are fused using a bidirectional interactive attention mechanism, specifically including: Normalizing the sentence-level global features and the paragraph-level global features respectively to obtain sentence-level normalized features and paragraph-level normalized features; Based on multi-head attention, information interaction is performed on the sentence-level normalized features and the paragraph-level normalized features to obtain interactive features from the sentence level to the paragraph level and interactive features from the paragraph level to the sentence level; The interactive features from sentence level to paragraph level and the interactive features from paragraph level to sentence level are respectively summed with the sentence level normalized features and the paragraph level normalized features, and the results of the residual summation are normalized to generate sentence level secondary normalized features and paragraph level secondary normalized features; The sentence-level quadratic normalized features and the paragraph-level quadratic normalized features are respectively input into the feedforward neural network and subjected to residual connection and layer normalization operations to obtain sentence-level interaction features and paragraph-level interaction features; The sentence-level interaction features and the paragraph-level interaction features are concatenated to obtain multi-granularity semantic fusion features; Named entity recognition is performed based on the multi-granularity semantic fusion features, and an entity set is obtained through conditional random field decoding.

[0014] According to a financial document-level event extraction method based on multiple semantic enhancement provided by the present invention, the processing steps of the event center heterogeneous graph structure module are: Based on multi-granularity semantic fusion features and entity sets, a heterogeneous graph structure including event nodes, entity nodes and sentence nodes is constructed, wherein the edges between nodes in the heterogeneous graph structure include edges between events, edges between events and entities, edges between events and sentences, edges between entities and entities, edges between sentences and sentences, and self-loop edges of events; A graph neural network is used to propagate and update the features of the constructed heterogeneous graph structure to obtain a node representation that integrates global relationship information.

[0015] According to a financial document-level event extraction method based on multiple semantic enhancement provided by the present invention, the training process of the financial document-level event extraction model includes: Obtain financial document-level event data, preprocess and annotate the financial document-level event data, and form the training set data for the financial document-level event extraction model. The event type and argument annotated in each document of the training set data are used as the real event set. Use the financial document-level event extraction model in training to extract events from the training set data to obtain the predicted event set; The Hausdorff distance is used to calculate the matching degree between the predicted event set and the real event set, and the Hausdorff distance loss is obtained; Based on the Hausdorff distance and entity representation loss, an overall loss function is constructed. The entity representation loss is calculated based on the entity set extracted by the multi-granularity semantic enhancement module. The calculation formula of the overall loss function is: ; ; Among them, L represents the overall loss function, Represents the predicted event set With true events The Hausdorff distance loss between represents the set of predicted events, represents a set of real events, Represents an entity that represents a loss, represents the matching pair of predicted events and real events, Represents the set of all matching event pairs, represents the cross entropy loss of event types, represents the event type of the i-th predicted event, represents the event type of the jth real event, Represents the set of arguments in the event, e represents the argument, k represents the index of the argument, represents the type of the kth argument in the i-th predicted event, represents the type of the kth argument in the jth real event; The overall loss function is optimized through the back-propagation algorithm, the model parameters are updated, and the accuracy of event extraction is improved.

[0016] According to a financial document-level event extraction method based on multiple semantic enhancement provided by the present invention, the entity representation loss includes a sequence labeling loss function and a binary classification loss function; wherein, The sequence labeling loss function is determined based on named entity recognition of multi-granularity semantic features; The binary classification loss function is determined based on the probability that any two entities in the entity set belong to the same event using binary cross entropy.

[0017] The present invention also provides a financial document-level event extraction system based on multiple semantic enhancement, which implements the financial document-level event extraction method as described above, including: An event pattern building module, used to build a financial document event pattern, wherein the financial document event pattern includes an event type set and an argument role set of the event type; A document processing module, used for preprocessing and labeling the financial document data to be extracted according to event type and argument role, to obtain labeled financial document data; An event extraction module is used to input the annotated financial document data into a trained financial document-level event extraction model, and use the semantic matching optimization module, multi-granularity semantic enhancement module and event-centered heterogeneous graph structure module of the financial document-level event extraction model to perform financial document-level event extraction on the annotated financial document data to obtain an extraction result including event types and event arguments, wherein the semantic matching optimization module is used to filter and optimize the annotated financial document data, the multi-granularity semantic enhancement module is used to perform context encoding based on the output of the semantic matching optimization module to fuse sentence-level semantic features and paragraph-level semantic features and perform entity recognition, and the event-centered heterogeneous graph structure module is used to capture the interactive relationship between nodes in the annotated financial document data based on the output of the multi-granularity semantic enhancement module.

[0018] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned financial document-level event extraction methods is implemented.

[0019] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned financial document-level event extraction methods.

[0020] The financial document-level event extraction method and system based on multiple semantic enhancements provided by the present invention effectively handles the long-distance dependency and cross-paragraph information integration problems in document-level event extraction through a semantic matching optimization module, a multi-granularity semantic enhancement module and an event-centered heterogeneous graph structure module, thereby improving the extraction accuracy of complex financial events and realizing efficient extraction of event information in financial documents. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 is a flow chart of the financial document level event extraction method provided by the present invention; Figure 2 It is a flowchart of the financial document level event extraction method provided by the present invention; Figure 3 It is a flowchart of the financial document-level event extraction method provided by the present invention for generating multi-granularity semantic fusion features by using a bidirectional interactive attention mechanism; Figure 4It is a schematic diagram of the training set data and annotation implementation example of the financial document-level event extraction method provided by the present invention; Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] like Figure 1 and Figure 2 As shown, the present invention provides a financial document-level event extraction method based on multiple semantic enhancement, comprising the following steps: S1. Constructing a financial document event model, wherein the financial document event model includes an event type set and an argument role set of the event type; S2. Obtain financial document data to be extracted, and preprocess and annotate the financial document data to be extracted based on event types and argument roles to obtain annotated financial document data; S3. Input the annotated financial document data into the trained financial document-level event extraction model, and use the semantic matching optimization module, multi-granularity semantic enhancement module and event-centered heterogeneous graph structure module of the financial document-level event extraction model to extract financial document-level events from the annotated financial document data to obtain extraction results including event types and event arguments; wherein the semantic matching optimization module is used to filter and optimize the annotated financial document data, the multi-granularity semantic enhancement module is used to perform context encoding based on the output of the semantic matching optimization module to fuse sentence-level semantic features and paragraph-level semantic features and perform entity recognition, and the event-centered heterogeneous graph structure module is used to capture the interactive relationship between nodes in the annotated financial document data based on the output of the multi-granularity semantic enhancement module.

[0025] The present invention effectively handles the long-distance dependency and cross-paragraph information integration problems in document-level event extraction through a semantic matching optimization module, a multi-granularity semantic enhancement module, and an event-centered heterogeneous graph structure module, thereby improving the extraction accuracy of complex financial events and achieving efficient extraction of event information in financial documents.

[0026] In one embodiment of the present invention, the financial document event model includes 22 financial event types and 116 argument roles, which can effectively summarize the actual business needs of financial enterprises, making the financial document event model comprehensive and detailed.

[0027] In one embodiment of the present invention, the processing steps of the semantic matching optimization module are: Use the financial encoder to contextually encode the annotated financial document data and event templates to obtain high-dimensional vector representations of sentences and event templates; The semantic similarity between the high-dimensional vector representation of the sentence and the high-dimensional vector representation of the event template is calculated, and matched with the preset event template similarity threshold to obtain the matching result. Based on the matching result, the annotated financial documents are filtered to obtain the optimized documents.

[0028] In one embodiment of the present invention, the financial encoder may use Mengzi-FinBERT (Mengzi-BERT-base-fin), which is a financial-specific pre-training model based on the RoBERTa (A Robustly OptimizedBERT Pretraining Approach) architecture. It is fine-tuned on large-scale data in the financial field and aims to enhance the model's ability to process text in the financial field.

[0029] Specifically, the processing steps of the semantic matching optimization module are described with a specific embodiment: Financial Documents ,in represents the i-th sentence in the financial document, n represents the total number of sentences in the financial document, and the event template is ,in represents the jth event type, m represents the total number of event types in the event template, and each event type Contains a specific set of argument roles ; Use the financial encoder Mengzi-FinBERT to train each sentence in the financial document and event templates Encode word by word to get the sentence High-dimensional vectors and event templates The high-dimensional vector representation of:

[0030]

[0031] in and Respectively express sentences High-dimensional vector representation and event templates The high-dimensional vector representation of Expressing sentences The high-dimensional vector representation of Representation event template High-dimensional vector representation of ; We measure each sentence by cosine similarity. and event templates The semantic similarity between them is calculated as follows:

[0032] in, Expressing sentences and event templates The semantic similarity between represents the L2 norm; Set the similarity threshold based on the calculated similarity value , compare the semantic similarity with the similarity threshold: when When keeping the sentence ; Otherwise, filter out the sentence ; Based on the similarity threshold, we filter and extract the sentence set D′ that highly matches the event template T from the financial document D, namely:

[0033] The sentences in the filtered document D′ are all highly relevant to financial events, ensuring the effectiveness and pertinence of subsequent processing.

[0034] Understandably, similarity threshold It is set according to actual use requirements, and the present invention does not impose any specific limitations on this.

[0035] The semantic matching optimization module of the present invention adopts Mengzi-FinBERT for context encoding, performs content screening by calculating the semantic similarity between sentences and event templates in financial documents, and filters out irrelevant sentences, thereby effectively improving the ability of the financial document-level event extraction model to understand financial professional terms and reducing the interference of noise data on financial document-level event extraction.

[0036] In one embodiment of the present invention, the processing steps of the multi-granularity semantic enhancement module are: The optimized document is used as input text and passed through the financial encoder to obtain sentence-level semantic features and paragraph-level semantic features; Using a bidirectional gated recurrent unit to perform serialization modeling on the sentence-level semantic features and the paragraph-level semantic features to obtain serialized semantic features; The global information of serialized semantic features is modeled through the self-attention mechanism to obtain sentence-level global features and paragraph-level global features; The sentence-level global features and paragraph-level global features are fused using a bidirectional interactive attention mechanism to obtain multi-granularity semantic fusion features, and entity recognition is performed on the multi-granularity semantic fusion features to obtain an entity set.

[0037] It can be understood that using bidirectional gated recurrent units for sequential modeling can capture contextual dependencies in features and integrate contextual information.

[0038] Specifically, the processing steps of the multi-granularity semantic enhancement module are described with a specific embodiment: The optimized document is used as input text and passed through the financial encoder to obtain sentence-level semantic features. and paragraph-level semantic features ; Bidirectional Gated Recurrent Unit (Bi-GRU) is used for serial modeling, and the formula is: ; ; in, Represents the sentence-level semantic features after Bi-GRU modeling, represents the paragraph-level semantic features after Bi-GRU modeling, represents the operation function of a bidirectional gated recurrent unit; The self-attention mechanism is used to capture the correlation between each feature vector and other feature vectors. The formula is: ; ; in, represents the sentence-level global feature, represents the paragraph-level global features, represents the self-attention calculation function applied to sentence-level global features, Represents the self-attention calculation function applied to paragraph-level global features.

[0039] Furthermore, the sentence-level global features and paragraph-level global features are fused using a bidirectional interactive attention mechanism, specifically including: Normalizing the sentence-level global features and the paragraph-level global features respectively to obtain sentence-level normalized features and paragraph-level normalized features; Based on multi-head attention, information interaction is performed on the sentence-level normalized features and the paragraph-level normalized features to obtain interactive features from the sentence level to the paragraph level and interactive features from the paragraph level to the sentence level; The interactive features from sentence level to paragraph level and the interactive features from paragraph level to sentence level are respectively summed with the sentence level normalized features and the paragraph level normalized features, and the results of the residual summation are normalized to generate sentence level secondary normalized features and paragraph level secondary normalized features; The sentence-level quadratic normalized features and the paragraph-level quadratic normalized features are respectively input into the feedforward neural network and subjected to residual connection and layer normalization operations to obtain sentence-level interaction features and paragraph-level interaction features; The sentence-level interaction features and the paragraph-level interaction features are concatenated to obtain multi-granularity semantic fusion features; Named entity recognition is performed based on the multi-granularity semantic fusion features, and an entity set is obtained through conditional random field decoding.

[0040] Specifically, a specific embodiment is used to illustrate: like Figure 3 As shown in the figure, the sentence-level features and paragraph-level features are processed by layer normalization (LN) to obtain sentence-level normalized features and paragraph-level normalized features. The calculation formula is:

[0041]

[0042] in, represents the sentence-level normalized features, represents the paragraph-level normalized features, represents sentence-level features, represents paragraph-level features, Representation layer normalization function; Normalize sentence-level features and paragraph-level normalized features Input to Multi-Head Attention (MHA), using sentence-level features As a query matrix , the paragraph-level features As a key matrix Sum Matrix , the interactive features from paragraph level to sentence level are:

[0043] in, represents the interactive features from paragraph level to sentence level, represents the multi-head attention mechanism, Represents a splicing operation, represents the output feature of the i-th attention head, represents the output feature of the hth attention head, h represents the total number of attention heads, Represents the weight matrix that projects the connected multi-head attention output into the target dimension space; Paragraph-level features As a query matrix , the sentence-level features As a key matrix Sum Matrix , the interactive features from sentence level to paragraph level are:

[0044] in, Represents interactive features from sentence level to paragraph level; Among them, the specific calculation of each attention head is:

[0045] in, Represents the activation function softmax function operation, represents the query matrix of the ith attention head, represents the transpose of the key matrix of the i-th attention head, represents the dimension of the key vector, Represents the value matrix of the i-th attention head; The interactive features from sentence level to paragraph level and the interactive features from paragraph level to sentence level are respectively summed with the sentence level normalized features and paragraph level normalized features, and the results of the residual summation are normalized to generate sentence level secondary normalized features and paragraph level secondary normalized features. The calculation formula is:

[0046]

[0047] in, represents the sentence-level secondary normalized feature, Represents paragraph-level secondary normalized features; The sentence-level quadratic normalized features and paragraph-level quadratic normalized features are respectively input into the feedforward neural network (FNN) and subjected to residual connection and layer normalization operations to obtain sentence-level interaction features and paragraph-level interaction features. The calculation formula is:

[0048]

[0049] in, represents sentence-level interaction features, represents paragraph-level interaction features, Represents a feedforward neural network operation; Sentence-level interaction features and paragraph-level interaction features Splice to generate multi-granularity semantic fusion features , the calculation formula is:

[0050] In one embodiment of the present invention, the BIO annotation system is used to annotate multi-granularity semantic fusion features. Perform named entity recognition, decode through conditional random fields, and extract the entity set in the financial document. The calculation formula is:

[0051] in, Represents a collection of entities, Represents the decoding operation of a conditional random field.

[0052] The multi-granularity semantic enhancement module of the present invention captures sentence-level and paragraph-level semantic features respectively through BiGRU and self-attention mechanisms, and uses a bidirectional interactive attention mechanism for feature fusion, thereby achieving effective integration of fine-grained and coarse-grained semantic information, and enhancing the ability of the financial document-level event extraction model to capture cross-sentence and cross-paragraph event information.

[0053] In one embodiment of the present invention, the processing steps of the event center heterogeneous graph structure module are: Based on multi-granularity semantic fusion features and entity sets, a heterogeneous graph structure including event nodes, entity nodes and sentence nodes is constructed, wherein the edges between nodes in the heterogeneous graph structure include edges between events, edges between events and entities, edges between events and sentences, edges between entities and entities, edges between sentences and sentences, and self-loop edges of events; A graph neural network is used to propagate and update the features of the constructed heterogeneous graph structure to obtain a node representation that integrates global relationship information.

[0054] It can be understood that the present invention integrates various information types in financial documents into interrelated nodes and edges, and constructs an event-centric heterogeneous graph structure. The heterogeneous graph structure can fully utilize the relationship between different types of nodes (such as entities, sentences, and events) to more comprehensively model complex semantics.

[0055] Specifically, the processing steps of the event center heterogeneous graph structure module are described with a specific embodiment: Defining heterogeneous graph structures , where V is the set of nodes, including the event node , entity node and sentence nodes , where the event node Represents the semantic core of the event and randomly generates the initial embedding of the event node , learn reasonable event semantic representation through training to capture event context; entity node Through named entity recognition extraction in the multi-granularity semantic enhancement module, based on entity sets Determine the starting embedding of the entity node , used to pass entity information to event nodes; sentence nodes Represents document sentences based on sentence-level features Determine the initial embedding of sentence nodes , providing contextual semantic information for event nodes, is the set of edges, including the bidirectional edges of the event node , event self-loop edge , directed edges from entity to event node , directed edges from sentences to event nodes , bidirectional edges between entity nodes Bidirectional edges between sentence nodes ;in, Used for information exchange between event nodes; Connect the event node itself to capture the characteristics and information of the event node itself. Used to pass entity information to event nodes; Used to provide sentence context information to event nodes. Used to model semantic associations between entities and strengthen information sharing between entities; Used to model the contextual semantic association between sentences and enhance the contextual information transfer between sentences; The event node representation is decoded and the event node is divided into two parts: event type classification and event argument classification. The event type classification is achieved by converting the final representation vector of the i-th event node into Input the multi-layer perceptron (MLP) and use the softmax function to obtain the probability distribution of event types ;in is the initial embedding After the graph neural network message passing and updating, the probability distribution is calculated as follows:

[0056] in, represents the probability distribution of event types, represents the softmax function, represents a multi-layer perceptron, Represents the final representation vector of the i-th event node; Event argument classification aggregates multiple mentions of an entity through the multi-head attention mechanism (MHA), with event nodes as queries and entity mentions as keys and values ​​to obtain an aggregated representation of the entity:

[0057] in, Representing Entities At the event node The semantic vector representation is as follows: represents the multi-head attention mechanism, Represents an event node As a representation of a query, Represents entity As a representation of a key, Represents entity As a representation of a value; The type probability distribution of each argument is calculated by combining the aggregate representation of event nodes and entities through MLP :

[0058] in, Representing Entities At the event node The following aggregate representation; Determine the event type and argument , the formula is:

[0059]

[0060] in, Indicates the type of event being predicted, Indicates the type of argument to be predicted, Indicates taking the maximum value, Represents the type probability distribution of the kth argument of the ith event; Understandably, the event-centric heterogeneous graph structure module supports interactive modeling of complex events in financial documents and improves event extraction. The update of node features is achieved through the graph neural network with feature-wise linear modulation (GNN-FiLM), and the node features are modulated by the feature linear modulation mechanism (FiLM). The feature representation of each node is updated through message aggregation of neighboring nodes. The feature linear modulation mechanism further modulates the node features, fine-tunes the node features through scaling and offset parameters, integrates semantic information from entity nodes and context nodes, and generates event-related representations.

[0061] The event heterogeneous graph of the present invention contains three types of nodes, namely entities, sentences and events, and multiple types of edge relationships. The node features are updated through a graph neural network and a feature linear modulation mechanism, which effectively models the complex interactive relationship between various types of information in the document and improves the recognition accuracy of event elements.

[0062] In one embodiment of the present invention, the training process of the financial document-level event extraction model includes: Obtain financial document-level event data, preprocess and annotate the financial document-level event data, and form the training set data for the financial document-level event extraction model. The event type and argument annotated in each document of the training set data are used as the real event set. Use the financial document-level event extraction model in training to extract events from the training set data to obtain the predicted event set; The Hausdorff distance is used to calculate the matching degree between the predicted event set and the real event set, and the Hausdorff distance loss is obtained; Based on the Hausdorff distance and entity representation loss, an overall loss function is constructed. The entity representation loss is calculated based on the entity set extracted by the multi-granularity semantic enhancement module. The calculation formula of the overall loss function is: ; ; Among them, L represents the overall loss function, Represents the predicted event set With true events The Hausdorff distance loss between represents the set of predicted events, represents a set of real events, Represents an entity that represents a loss, represents the matching pair of predicted events and real events, Represents the set of all matching event pairs, represents the cross entropy loss of event types, represents the event type of the i-th predicted event, represents the event type of the jth real event, Represents the set of arguments in the event, e represents the argument, k represents the index of the argument, represents the type of the kth argument in the i-th predicted event, represents the type of the kth argument in the jth real event; The overall loss function is optimized through the back-propagation algorithm, the model parameters are updated, and the accuracy of event extraction is improved.

[0063] like Figure 4 As shown, specifically, a specific embodiment is used to illustrate: According to the business needs of financial enterprises, financial events are divided into six categories, including equity changes, company changes, performance changes, stock market fluctuations, executive changes, business changes and regulatory penalties. At the same time, the concept of opposing events is introduced, such as losses and profits, stock price increases and decreases, etc. By defining 22 event types and 116 argument roles, it is ensured that event types and argument roles do not overlap and are clear, thus constructing a comprehensive event model to meet the complex scenario requirements in the financial field; the 22 event types are divided into 6 categories; Collect data from seven financial news websites including Sina Finance, Tonghuashun Finance, and China Economic Net; First, duplicate news is removed through deduplication. Then, format verification is performed to delete documents that do not meet the standard format (such as PDF or table-based articles). Next, irrelevant content is filtered out to exclude non-financial advertisements and personal opinion articles. Based on exchange announcements, data from companies involved in financial fraud issues are screened out to prevent false information from interfering with model training. Finally, manual review is performed to ensure the relevance, accuracy, and authenticity of the data. The collected data is annotated using the doccano annotation tool. The annotation process is divided into two main stages: event classification and argument extraction. In the event classification stage, each document is assigned a corresponding event label according to the predefined event type, and the trigger words are annotated. In the argument extraction stage, the argument entities of each event are annotated according to the argument roles defined in the event model. The annotation process includes expert review and cross-validation mechanisms to ensure the accuracy and consistency of the annotated data and generate high-quality financial document event annotation data. The labeled data is used to train the financial document-level event extraction model to obtain a trained financial document-level event extraction model.

[0064] Furthermore, the entity representation loss includes the sequence labeling loss function and the binary classification loss function; among them, The sequence labeling loss function is determined based on named entity recognition of multi-granularity semantic features; The binary classification loss function is determined based on the probability that any two entities in the entity set belong to the same event using binary cross entropy.

[0065] In one embodiment of the present invention, the calculation formula based on the probability that any two entities in the entity set belong to the same event is as follows:

[0066] in, Representing Entities The probability that and entity j belong to the same event, is the Sigmoid function, Represents a splicing operation, represents the representation vector of entity i, The representation vector representing entity j; The binary classification loss function uses binary cross entropy (CE), and the specific calculation formula is:

[0067] in, represents the binary classification loss function, Represents the event association label of entity i and entity j, represents the binary cross entropy loss function; The total loss function of entity representation is:

[0068] in, represents the total loss function for entity representation, represents the sequence labeling loss function, represents the binary classification loss function.

[0069] It can be understood that the sequence annotation loss function is generated by using the BIO annotation system to perform named entity recognition on multi-granularity semantic features.

[0070] The present invention provides a financial document-level event extraction system based on multiple semantic enhancement, which implements the above-mentioned financial document-level event extraction method, including: An event pattern building module, used to build a financial document event pattern, wherein the financial document event pattern includes an event type set and an argument role set of the event type; A document processing module, used for preprocessing and labeling the financial document data to be extracted according to event type and argument role, to obtain labeled financial document data; An event extraction module is used to input the annotated financial document data into a trained financial document-level event extraction model, and use the semantic matching optimization module, multi-granularity semantic enhancement module and event-centered heterogeneous graph structure module of the financial document-level event extraction model to extract financial document-level events from the annotated financial document data to obtain extraction results including event types and event arguments; wherein the semantic matching optimization module is used to filter and optimize the annotated financial document data, the multi-granularity semantic enhancement module is used to perform context encoding based on the output of the semantic matching optimization module to fuse sentence-level semantic features and paragraph-level semantic features and perform entity recognition, and the event-centered heterogeneous graph structure module is used to capture the interactive relationship between nodes in the annotated financial document data based on the output of the multi-granularity semantic enhancement module.

[0071] The financial document level event extraction device provided by the present invention is described below. The financial document level event extraction device described below and the financial document level event extraction method described above can be referenced to each other.

[0072] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the financial document level event extraction method, which includes: constructing a financial document event model, obtaining financial document data to be extracted, pre-processing and annotating the financial document data to be extracted based on event types and argument roles to obtain annotated financial document data, inputting the annotated financial document data into a trained financial document level event extraction model, and extracting financial document level events from the annotated financial document data using the semantic matching optimization module, the multi-granularity semantic enhancement module and the event center heterogeneous graph structure module of the financial document level event extraction model to obtain an extraction result including event types and event arguments.

[0073] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0074] On the other hand, the present invention also provides a computer program product, which includes a computer program, and the computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the financial document-level event extraction method provided by the above-mentioned methods, and the method includes: constructing a financial document event model, obtaining financial document data to be extracted, preprocessing and labeling the financial document data to be extracted based on event types and argument roles to obtain labeled financial document data, inputting the labeled financial document data into a trained financial document-level event extraction model, and using the semantic matching optimization module, multi-granularity semantic enhancement module and event-centered heterogeneous graph structure module of the financial document-level event extraction model to perform financial document-level event extraction on the labeled financial document data to obtain extraction results including event types and event arguments.

[0075] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the financial document-level event extraction method provided by the above-mentioned methods, the method comprising: constructing a financial document event model, obtaining financial document data to be extracted, preprocessing and labeling the financial document data to be extracted based on event types and argument roles to obtain labeled financial document data, inputting the labeled financial document data into a trained financial document-level event extraction model, and using the semantic matching optimization module, multi-granularity semantic enhancement module and event-centered heterogeneous graph structure module of the financial document-level event extraction model to perform financial document-level event extraction on the labeled financial document data to obtain an extraction result including event types and event arguments.

[0076] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0077] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A financial document-level event extraction method based on multiple semantic enhancement, characterized in that: The following steps are involved: Constructing a financial document event model, wherein the financial document event model includes an event type set and an argument role set of the event type; Acquire financial document data to be extracted, and preprocess and annotate the financial document data to be extracted based on event types and argument roles to obtain annotated financial document data; The annotated financial document data is input into the trained financial document-level event extraction model, and the semantic matching optimization module, multi-granularity semantic enhancement module and event-centered heterogeneous graph structure module of the financial document-level event extraction model are used to extract financial document-level events from the annotated financial document data to obtain extraction results including event types and event arguments; wherein the semantic matching optimization module is used to filter and optimize the annotated financial document data, the multi-granularity semantic enhancement module is used to perform context encoding based on the output of the semantic matching optimization module to fuse sentence-level semantic features and paragraph-level semantic features and perform entity recognition, and the event-centered heterogeneous graph structure module is used to capture the interactive relationship between nodes in the annotated financial document data based on the output of the multi-granularity semantic enhancement module.

2. The financial document-level event extraction method based on multiple semantic enhancement according to claim 1 is characterized in that: The processing steps of the semantic matching optimization module are: Use the financial encoder to contextually encode the annotated financial document data and event templates to obtain high-dimensional vector representations of sentences and event templates; The semantic similarity between the sentence high-dimensional vector representation and the event template high-dimensional vector representation is calculated, and matched with a preset event template similarity threshold to obtain a matching result, and the annotated financial documents are filtered based on the matching result to obtain an optimized document.

3. The financial document-level event extraction method based on multiple semantic enhancement according to claim 1 is characterized in that: The processing steps of the multi-granularity semantic enhancement module are: The optimized document is used as input text and passed through the financial encoder to obtain sentence-level semantic features and paragraph-level semantic features; The sentence-level semantic features and paragraph-level semantic features are serialized and modeled using a bidirectional gated recurrent unit to obtain serialized semantic features; the formula is: ； ； in, Represents the sentence-level semantic features after Bi-GRU modeling, represents the paragraph-level semantic features after Bi-GRU modeling, represents the operation function of a bidirectional gated recurrent unit; The global information of the serialized semantic features is modeled through the self-attention mechanism to obtain sentence-level global features and paragraph-level global features; the formula is: ； ； in, represents the sentence-level global feature, represents the paragraph-level global features, represents the self-attention calculation function applied to sentence-level global features, represents the self-attention calculation function applied to paragraph-level global features; The sentence-level global features and paragraph-level global features are fused using a bidirectional interactive attention mechanism to obtain multi-granularity semantic fusion features, and entity recognition is performed on the multi-granularity semantic fusion features to obtain an entity set.

4. The financial document-level event extraction method based on multiple semantic enhancement according to claim 3 is characterized in that: The method of fusing the sentence-level global features and the paragraph-level global features by using a bidirectional interactive attention mechanism specifically includes: Normalizing the sentence-level global features and the paragraph-level global features respectively to obtain sentence-level normalized features and paragraph-level normalized features; Based on multi-head attention, information interaction is performed on the sentence-level normalized features and the paragraph-level normalized features to obtain interactive features from the sentence level to the paragraph level and interactive features from the paragraph level to the sentence level; The interactive features from sentence level to paragraph level and the interactive features from paragraph level to sentence level are respectively summed with the sentence level normalized features and the paragraph level normalized features, and the results of the residual summation are normalized to generate sentence level secondary normalized features and paragraph level secondary normalized features; The sentence-level quadratic normalized features and the paragraph-level quadratic normalized features are respectively input into the feedforward neural network and subjected to residual connection and layer normalization operations to obtain sentence-level interaction features and paragraph-level interaction features; The sentence-level interaction features and the paragraph-level interaction features are concatenated to obtain multi-granularity semantic fusion features; Named entity recognition is performed based on the multi-granularity semantic fusion features, and an entity set is obtained through conditional random field decoding.

5. The financial document-level event extraction method based on multiple semantic enhancement according to claim 4 is characterized in that: The processing steps of the event center heterogeneous graph structure module are: Based on multi-granularity semantic fusion features and entity sets, a heterogeneous graph structure including event nodes, entity nodes and sentence nodes is constructed, wherein the edges between nodes in the heterogeneous graph structure include edges between events, edges between events and entities, edges between events and sentences, edges between entities and entities, edges between sentences and sentences, and self-loop edges of events; A graph neural network is used to propagate and update the features of the constructed heterogeneous graph structure to obtain a node representation that integrates global relationship information.

6. The financial document-level event extraction method based on multiple semantic enhancement according to claim 5 is characterized in that: The training process of the financial document-level event extraction model includes: Obtain financial document-level event data, preprocess and annotate the financial document-level event data, and form the training set data for the financial document-level event extraction model. The event type and argument annotated in each document of the training set data are used as the real event set. Use the financial document-level event extraction model in training to extract events from the training set data to obtain the predicted event set; The Hausdorff distance is used to calculate the matching degree between the predicted event set and the real event set, and the Hausdorff distance loss is obtained; Based on the Hausdorff distance and entity representation loss, an overall loss function is constructed. The entity representation loss is calculated based on the entity set extracted by the multi-granularity semantic enhancement module. The calculation formula of the overall loss function is: ； ； Among them, L represents the overall loss function, Represents the predicted event set With true events The Hausdorff distance loss between represents the set of predicted events, represents a set of real events, Represents an entity that represents a loss, represents the matching pair of predicted events and real events, Represents the set of all matching event pairs, represents the cross entropy loss of event types, represents the event type of the i-th predicted event, represents the event type of the jth real event, Represents the set of arguments in the event, e represents the argument, k represents the index of the argument, represents the type of the kth argument in the i-th predicted event, represents the type of the kth argument in the jth real event; The overall loss function is optimized through the back-propagation algorithm, the model parameters are updated, and the accuracy of event extraction is improved.

7. The financial document-level event extraction method based on multiple semantic enhancement according to claim 6 is characterized in that: The entity representation loss includes a sequence labeling loss function and a binary classification loss function; wherein, The sequence labeling loss function is determined based on named entity recognition of multi-granularity semantic features; The binary classification loss function is determined based on the probability that any two entities in the entity set belong to the same event using binary cross entropy.

8. A financial document-level event extraction system based on multiple semantic enhancement, characterized in that: Implementing the financial document-level event extraction method as described in any one of claims 1 to 7, comprising: An event pattern building module, used to build a financial document event pattern, wherein the financial document event pattern includes an event type set and an argument role set of the event type; A document processing module, used for preprocessing and labeling the financial document data to be extracted according to event type and argument role, to obtain labeled financial document data; An event extraction module is used to input the annotated financial document data into a trained financial document-level event extraction model, and use the semantic matching optimization module, multi-granularity semantic enhancement module and event-centered heterogeneous graph structure module of the financial document-level event extraction model to extract financial document-level events from the annotated financial document data to obtain extraction results including event types and event arguments; wherein the semantic matching optimization module is used to filter and optimize the annotated financial document data, the multi-granularity semantic enhancement module is used to perform context encoding based on the output of the semantic matching optimization module to fuse sentence-level semantic features and paragraph-level semantic features and perform entity recognition, and the event-centered heterogeneous graph structure module is used to capture the interactive relationship between nodes in the annotated financial document data based on the output of the multi-granularity semantic enhancement module.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, it implements the financial document level event extraction method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the financial document-level event extraction method as described in any one of claims 1 to 7 is implemented.

Images