A method and system for event extraction of multi-modal financial documents

Abstract

The application discloses a kind of multi-modal financial document's event extraction method and system, belong to data processing technical field, the method includes: obtaining to be extracted area and its object, the object includes table and text;The table is parsed, obtains the content of table, the content includes event and argument;According to the event filter of the table, the text is filtered, obtains relevant sentence;Argument role and / or supplementary argument are extracted from the relevant sentence.It is relatively easy to extract information from semi-structured table, then use the information obtained from table to assist and improve the information extraction from text, improve extraction efficiency and reliability;Information extracted in table and text can be supplemented each other, obtain more complete information extraction result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to an event extraction method and system for multi-modal financial documents. BACKGROUND

[0002] Event extraction of financial documents is a key content of structured processing of financial documents, and provides basic data support for constructing a knowledge graph and an event / event graph. Event extraction includes identifying event types and event arguments.

[0003] In the financial professional field, relevant event types and arguments are usually defined according to task needs, taking stock equity as an example: for the capital increase event type, the extracted arguments include: shareholder name, capital increase amount, capital increase price, increased number of shares, and investment method; for the capital reduction event type, the extracted arguments include: shareholder name, capital reduction amount, and capital reduction price; for the transfer event type, the extracted arguments include: transferor, transferee, transfer amount, transfer price, transferred number of shares, and transferred investment amount. According to task needs and the detailed level of financial document disclosure, some arguments are optional.

[0004] In financial documents, most are multi-modal (Multi-Modal) documents: hybrid data forms of text and tables. This hybrid data form widely exists in various scenarios, such as technical reports, medical reports, and financial report documents. The identification of multi-modal (Multi-Modal) documents in the prior art usually includes a document-level event extraction method and a question and answer method (Q&A for Hybrid of Textual and Tabular data) based on hybrid data of text and tables. The document-level event extraction method is as follows: Doc2EDAG: An End-to-End Document-level Framework for Chinese Financial Event Extraction, EMNLP-2019, Tsinghua University, Microsoft Research, Document-level Event Extraction via Heterogeneous Graph-based Interaction Model with a Tracker, ACL-2021, Peking University, Peng Cheng Laboratory, Shenzhen, ByteDance AILab.Question answering methods based on mixed data of text and table, such as the following documents: “Representations for Question Answering from Documents with Tables and Text, EACL-2021, Google Research, University of Washington”, “Dual Reader-Parser on Hybrid Textual and Tabular Evidence for Open Domain Question Answering, ACL-2021, AWS AI Labs, Alexander Hanbo Li, Patrick Ng, Peng Xu, Henghui Zhu, Zhiguo Wang, Bing Xiang” query candidate answers from text and table respectively, and then select together by ranking; “TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance, ACL-2021, National University of Singapore, 6Estates Pte Ltd, Sichuan University, Bloomberg” proposes the evaluation dataset TAT-QA and the baseline method.

[0005] Although the document-level event extraction method has made progress compared to the traditional sentence-level event extraction method (which can only extract information from a single sentence), there are still some problems: 1. It only focuses on event extraction from text, but in many cases some event arguments of the event can only be obtained from tables, so ignoring tables may not be able to extract complete event information; 2. The current processing document length is relatively short, while in actual scenarios many financial documents are hundreds of pages long.

[0006] The question answering (Q&A) method based on mixed data of text and table has the following problems: Q&A task and event extraction task have some correlation, and in some cases specific prompt questions can be constructed to extract event arguments one by one through Q&A, but in general the focus of the two tasks is different: Q&A focuses on specific local information; while event extraction expects to obtain complete description information of the event. The current Q&A method has very low accuracy and cannot meet the current needs of financial applications.

[0007] In the document "Question Answering Representation of Multi-modal Documents", the representation of the table is enhanced by surrounding text, and the final answer is obtained from the table. However, in many practical scenarios, the answer needs to be obtained from the text.

[0008] In the document "Dual-Reading Decoder for Mixed Text and Table Evidence Based on Open Domain Question Answering", two independent parsers are used to find answers from text and table respectively, and then the candidate answers from the two sources are sorted together without considering the semantic correlation between the text content and the table content. However, in many cases, financial documents involve important matters, and in order to ensure the accuracy of the expression, both text and table are usually used to explain the same specific event, but the detailed level and angle of explanation of some events are different.

[0009] In the document "TAT-QA: Financial Question Answering Benchmark for Mixed Table and Text Content", the data set is manually matched with text-table, which considers the correlation and interaction between text and table to some extent, but simply flattens the table into a text sequence and inputs it into the RoBERTa model, and uses the conventional sequence labeling method to obtain candidate answers or intermediate answers, and then uses a series of classifiers for subsequent processing, without fully utilizing the structured information in the table.

[0010] It can be seen that the above-mentioned methods in the prior art do not utilize the correlation between text and table, resulting in low efficiency of event extraction. SUMMARY

[0011] To solve the above technical problems in the prior art, the present application provides a multi-modal financial document event extraction method and system, which first extracts information from semi-structured tables, then uses the information obtained from the table to assist and improve the extraction from the text information, and improves the extraction efficiency and reliability; the information extracted from the table and the text can be supplemented to each other to obtain more complete information extraction results.

[0012] The present application discloses a multi-modal financial document event extraction method, which comprises: obtaining a to-be-extracted region and its object, the object comprising a table and a text; analyzing the table to obtain the content of the table, the content comprising an event and an argument; screening the text according to the event filter of the table to obtain relevant sentences; and extracting argument roles and / or supplementary arguments from the relevant sentences.

[0013] The content of the table comprises an entity, and the method for extracting argument roles from the relevant sentences comprises:

[0014] Matching the entity of the table with the related sentence to obtain argument roles and argument mentions;

[0015] The method for obtaining the related sentence comprises:

[0016] According to the content of the table or the title of the table, the event of the table is obtained;

[0017] The classification of the sentence is obtained by a sentence text classifier;

[0018] According to the event and the classification, the sentence is screened.

[0019] The method for extracting the supplementary argument from the related sentence comprises a sentence classification filtering method, a sequence labeling method or a machine learning method,

[0020] The machine learning method is a sequence labeling method based on a pre-trained language model (BERT), a machine reading comprehension method based on BERT or a Prompt method.

[0021] The sequence labeling method based on the pre-trained language model comprises:

[0022] The entity is encoded by the pre-trained language model to obtain a first encoding;

[0023] In the first encoding, a vector of an entity type is added to obtain a second encoding;

[0024] The related sentence is encoded by the pre-trained language model to obtain a third encoding;

[0025] In the third encoding, an entity type encoding is added to the matched part of the entity;

[0026] After the unmatched part of the related sentence is labeled, the supplementary argument is obtained by using the sequence labeling method;

[0027] The machine reading comprehension method based on BERT comprises:

[0028] The extraction question of the supplementary argument is defined as a question and answer question;

[0029] Based on the question and answer question, the answer to the question is found in the sentence to obtain the supplementary argument;

[0030] The Prompt method is a method for defining the extraction question of the event argument as a method for directly generating an answer through seq2seq.

[0031] The method for obtaining the region to be extracted comprises:

[0032] The title, text, table and graph of the financial document are obtained by performing layout recognition on the financial document;

[0033] identify the level of the title according to the visual features and the text features of the title;

[0034] obtain a hierarchical structure of the financial document according to the level of the title;

[0035] locate and segment the region to be extracted according to the hierarchical structure.

[0036] the region to be extracted further comprises any of the following objects or combinations thereof: a picture, a caption of the picture, a header, a footer, a page number, a title of a table, a caption of the table;

[0037] the event type comprises a share transfer event, a capital reduction event and a capital increase event;

[0038] the object further comprises a title and a graph, and the parsing of the object further comprises title parsing, graph parsing and merging of cross-page content,

[0039] merging of cross-page content comprises deleting the header, the footer and the page number, and aligning the cross-page table.

[0040] the method of cross-validation:

[0041] cross-verify the event list extracted from the sketch map and the event list extracted from the title, and check whether there is a missing event;

[0042] after the supplementary argument calculation and the event completion information and the content of the event table extracted from the text are verified, whether they match is checked.

[0043] the method of training the text extraction model by using the table content:

[0044] label the text data according to the entity of the table;

[0045] train the text information extraction model by using the labeled text data.

[0046] The application also provides a system for implementing the event extraction method, comprising a positioning module, a table parsing module, a text screening module and an argument extraction module,

[0047] the positioning module is used to obtain the region to be extracted and its objects, and the objects comprise tables and texts;

[0048] the table parsing module is used to parse the table to obtain the content of the table, and the content comprises events and arguments;

[0049] the text screening module is used to screen the text according to the events of the table to obtain relevant sentences;

[0050] The argument extraction module is configured to extract argument roles and / or supplementary arguments from the relevant sentences.

[0051] The system further comprises a layout analysis module, a document hierarchical analysis module, a graph analysis module, a segmentation module, a title extraction module and a verification module,

[0052] The layout analysis module is configured to perform layout recognition on the financial document to obtain titles, texts, tables and graphs.

[0053] The document hierarchical analysis module is configured to identify the hierarchy of the titles according to visual features and text features of the titles, and obtain a hierarchical structure of the financial document according to the hierarchy of the titles.

[0054] The positioning module is configured to position the regions to be extracted according to the hierarchical structure and obtain objects of the regions to be extracted.

[0055] The graph analysis module is configured to parse the graphs to obtain graph structured information.

[0056] The segmentation module is configured to segment the regions to be extracted.

[0057] The title extraction module is configured to obtain events and event lists of the titles.

[0058] The verification module is configured to cross-verify the event lists extracted from the sketch map and the event lists extracted from the titles to check whether there are any missing events, or to verify the information after the calculation and deduction of the supplementary arguments extracted from the texts and the content of the event table of the current event to check whether they match.

[0059] Compared with the prior art, the present application has the advantages that: relatively easy information extraction is performed from semi-structured tables, and then the information obtained from the tables is used to assist and improve the information extraction from the texts, thereby improving the extraction efficiency and reliability; the information extracted from the tables and the texts can be supplemented to each other to obtain more complete information extraction results. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 is a flowchart of an event extraction method of an embodiment;

[0061] Figure 2 is a flowchart of a document structure analysis method;

[0062] Figure 3 is a flowchart of a method for obtaining target regions;

[0063] Figure 4 is a flowchart of a method for extracting structured information;

[0064] Figure 5 is a flowchart of structured analysis.

[0065] Figure 6 is a method flowchart of text extraction using structured information of tables;

[0066] Figure 7 is a schematic diagram of layout recognition;

[0067] Figure 8 is a schematic diagram of title hierarchy;

[0068] Figure 9 is a schematic diagram of event graph;

[0069] Figure 10 is a system logic block diagram of the present application;

[0070] Figure 11 is a method flowchart of event extraction of the multi-modal financial document of the present application. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0072] The present application will be described in further detail below with reference to the drawings:

[0073] A method for event extraction of a multi-modal financial document, as shown in Figure 11 the method comprises:

[0074] Step 11: obtaining a region to be extracted and objects thereof, the objects including tables and texts.

[0075] Step 12: analyzing the tables to obtain contents of the tables, the contents including events and arguments.

[0076] Step 13: screening the texts according to the event filter of the tables to obtain relevant sentences.

[0077] Step 14: extracting argument roles and / or supplementary arguments from the relevant sentences.

[0078] It is relatively easy to extract information from semi-structured tables, and then use the information obtained from the tables to assist and improve the extraction of information from texts, so as to improve the extraction efficiency and reliability; the information extracted from the tables and texts can be supplemented to each other to obtain a more complete information extraction result.

[0079] Embodiments

[0080] As shown in Figure 1 The object of the region to be extracted in the multi-modal financial document includes a table and text, and the event extraction method includes:

[0081] Step S1: document structure analysis;

[0082] Step S2: locate and obtain the target region or target content, i.e., the region to be extracted, in the document;

[0083] Step S3: extract structured information from the target region;

[0084] Step S4: construct an event graph according to the structured information. Using the extracted event structured data in the financial field, a corresponding event graph can be constructed. A specific event graph is shown in Figure 9

[0085] As shown in Figure 2 Step S1 includes:

[0086] Step 101: perform layout analysis on the financial document to obtain a title (Title), text (text), table (Table), and figure (Figure), etc. This can be achieved by training a machine learning model using labeled training data, for example, a method combining object detection (object detection) and instance segmentation (instance segmentation), such as a Mask-RCNN model using ResNet or Swin Transformer as the backbone network (backbone), and further enhanced with text features in the page, such as ViBERTgrid, a Pretrained Language Model enhanced with two-dimensional visual position information, such as the LayoutLM series, and StructuralLM, etc. Figure 7 The result of the layout analysis is shown.

[0087] Step 102: identify the level of the title according to the visual features and text features of the title.

[0088] ​The level of the title can be identified in combination with visual features and text features. The visual features of each title corresponding to the segmented image block are encoded (including the position of the block, the length and width of the block, the encoded features of the image region corresponding to the block, and the information of the context in the entire page obtained through the attention mechanism); then the text content of each title is obtained (which can be obtained through OCR, or obtained from the coordinates of the title in the PDF file page using a pdf information extraction tool); after the visual features and text features are fused, the overall features are obtained, for example, a method similar to ViBERTgrid is used; then the overall features are input into a classifier to automatically identify the level of the title. Only a small amount of labeled data is needed to obtain good results. Figure 8 The level of the title is shown.

[0089] Heuristic rules can also be used to identify the level of the title. Because many documents, especially financial announcements, have relatively fixed specifications for formats, for example, a multi-level hierarchy similar to the following is used:

[0090] Section 1,

[0091] 1,

[0092] (I),

[0093] 1, (1)

[0095] For such cases, after obtaining the text content corresponding to each title, the level of the title can be identified by simply matching the rules.

[0096] Step 103: obtaining the hierarchical structure of the financial document according to the level of the title.

[0097] After obtaining the level of each title, the hierarchical structure of the entire document can be obtained according to the position of each title in the document. The entire document can be divided into different levels according to the chapters of different levels, including the corresponding structure of different levels in the tree structure of the text, tables, etc.

[0098] As Figure 3 shown, in step S2, the method for locating and obtaining the target region or target content in the document includes:

[0099] Step 201: locating the extraction region according to the hierarchical structure. Based on the obtained document hierarchical structure and its document tree structure, the required node (chapter) is found.

[0100] Many financial announcement documents are hundreds of pages long, and for a specific task, we often only focus on extracting relevant event information from part of the chapter content. For example, if we want to build an event graph of investment history, we usually extract event information related to equity changes from the "Formation and Changes of Issuer's Capital" under "Section V. Basic Information of the Issuer" in the prospectus, and sometimes we also supplement information from "Section II. Issuer Restructuring Situation", and then according to the lower level titles, texts and tables contained in these parts, we can more accurately locate the target content required by the task.

[0101] The expression text of the upper directory is usually fixed, so the title can be encoded (for example, using a pre-trained language model such as BERT), and a small amount of labeled data can be used to train a classifier to recognize it well, or regular expression matching can be used. For example, "Section V. Basic Information of the Issuer" and "Section III. Formation and Changes of Issuer's Capital".

[0102] The lower directory expression varies greatly, and the hierarchy is not fixed, so after determining the scope of the upper directory, all subdirectories contained under it can be listed, and the text title can be encoded (for example, using a pre-trained model such as BERT) to train a 2-class classifier to filter and select each item. Because the subdirectory title has many different ways of expression, more training data needs to be labeled to cover different forms of explanatory text as much as possible.

[0103] Step 202: segmentation of the located content or area, and subsequent processing after segmentation.

[0104] The relevant content can be automatically segmented according to the located content directory title as the segmentation boundary to prepare for subsequent processing. The subsequent processing includes merging paragraphs, merging sentences, adding paragraphs, deleting paragraphs, adding sentences, deleting sentences, merging cross-page content, etc.

[0105] As shown in Figure 4 The method of extracting structured information from the target area in step S3 includes:

[0106] Step 301: Obtain various types of objects in the segmented part (the area to be extracted). The objects include: title, text, diagram, diagram explanatory text, table, table caption, table annotation, header, footer, and page number.

[0107] Step 302: Analysis of related objects, including preprocessing and structured analysis.

[0108] The object is preprocessed, i.e., the title, graphics, and table are preprocessed. It includes: graphic structure analysis, table structure analysis, and merging of cross-page content.

[0109] The graphic structure analysis can identify shape objects, lines, and text in the graphics according to visual features and text features of the graphics, establish the connection of each part according to the type of the line and the starting point and ending point of each part of the line, obtain complete lines, and establish the connection of the shape objects according to the starting point and ending point of the line, and establish the association of the text with the line or the shape object according to the distance between the text and the line or the shape object.

[0110] The table structure analysis identifies the structural layout of the table and the positional relationship between the cells. Machine learning-based methods or rule-based methods (based on the structural information in the PDF) can be used for identification (the input table can be in a picture format), such as the Seq2seq method (e.g., Master-Table), the through-line detection + cell merge method (e.g., SPLERGE), and the cell detection method (e.g., various extensions based on Mask-RCNN).

[0111] The merging of cross-page content: If the range of the current content spans different pages, it needs to be merged, and the header, footer, and page number are removed, and the cross-page table is aligned and merged.

[0112] Step 303: Structure analysis of the extraction area, including title analysis, graphic analysis, table analysis, and text analysis.

[0113] As shown in Figure 5 , the structure analysis includes:

[0114] Step 331: Title analysis. The title often provides an overview of an event: event type, date, number of times, and even amount. Structured analysis of the title and extraction of these overview information can provide a lot of useful guidance for further analysis of the current part of the content. A title may contain multiple events of different types, so a multi-label text classifier is needed to identify the corresponding event type (or not contain any type). The title is relatively short and the expression is relatively simple, and the event type features in the title are usually clear and obvious, so a small amount of title data can be used to train a good classifier. Then, according to the current event type, a sequence labeling tool is used to extract event-related information such as date and amount. Because the title is relatively short and the expression is relatively simple, this sequence labeling task is also relatively simple.

[0115] Step 332: Diagram and table parsing. The beginning of the partial document provides some outline diagrams, which are usually relatively simple in structure, and it is easy to extract the structured information of the respective outlines of individual events. For example, as can be seen from the given example, the horizontal connection relationship represents the components of the outline of the same event, and the vertical connection relationship represents the progress event line between different events, according to which the respective outlines of each event can be easily divided. The parsing of the short text part can be performed by using a method similar to the title parsing method using sequence labeling. The long text part needs to be parsed by using a special event extraction method.

[0116] Table parsing includes table content parsing, table classification, and semantic parsing. Tables are semi-structured information, and compared with text description information, tables are easier to be parsed into structured information, and the correctness and efficiency of the parsing are higher. Therefore, it is tried to extract event information from tables first, and if the information provided by the table is incomplete, it is tried to extract relevant information from the text. In the table preprocessing of step 302, the text content of each cell of the table and the positional relationship between the cells are parsed, that is, the table content is parsed.

[0117] Table classification can classify and filter tables according to the content of the table, and find the table related to the current event extraction. Table classification can be performed by a simple method based on regular expression matching rules, or a text classifier can be trained using the caption and table text content of the table to determine the table type. Through table classification, the table related to the event can be filtered out.

[0118] Semantic parsing: after the table related to the event of interest is filtered out, the content of the table can be further semantically parsed according to the header structure of the table, and the content in the table is mapped to the record and field in the database, or the entity and property defined in the knowledge graph. The mapping can be performed by semantic similarity, or by a statistically arranged term dictionary and matching rules. These fields or properties also correspond to the event arguments in event extraction. The table fields or entities can be directly mapped to the event arguments or the event arguments can be derived based on the table fields. The table types and formats involved in specific tasks and applications are relatively fixed and limited, so they are relatively easy to parse. In semantic parsing, the table fields can be directly mapped to the event arguments, or the event arguments can be extracted based on the table derivation.

[0119] In some cases, the financial document directly gives a table of detailed information of certain types of events. The fields needed in the table can be simply and directly mapped to the arguments of the event of interest, because the arguments of the event to be extracted can be found in the corresponding fields in the table. For example, the first field in the table is extracted (mapped) to the complete "establishment" event argument; the second field in the table is extracted (mapped) to the complete "capital increase" event argument.

[0120] For example, although the table contains information of the specific type of event of interest, the event argument to be extracted cannot be directly found in the corresponding field in the table, but can be indirectly derived from the content of the table: the event argument of the current event can be derived from the difference between the current event table and the table of the previous event; the derivation from the internal data of the table, such as the transfer of equity event.

[0121] Example 1: In the third increase of registered capital of a certain company, the equity structure after the capital increase is shown in Table 1:

[0122] Table 1

[0123] Serial number Name of shareholder Amount of shareholding (in RMB 10,000) Percentage of shareholding 1 Guo 24600000 100.00 Total 24600000 100.00

[0124] After the fourth increase of registered capital, the equity structure is shown in Table 2:

[0125] Table 2

[0126] Serial number Name of shareholder Amount of shareholding (in RMB 10,000) Percentage of shareholding 1 Guo 24600000 61.5 2 Liu 15400000 38.5 Total 40000000 100.00

[0127] In the fourth capital increase, the specific capital increase information of each shareholder is not listed, but can be derived from the comparison of the information of each shareholder in Table 2 and Table 1 after the end of this capital increase, and from the difference between the two tables, the specific capital increase information of each shareholder as shown in Table 3 can be derived:

[0128] Table 3

[0129]

[0130] Here, the investment method of Mr. Liu is still not derivable from the table, so it needs to be extracted from the text content later, i.e. additional information needs to be obtained from the text to complete the final accurate event argument extraction.

[0131] Example 2:

[0132] After the third transfer of equity of a certain company, the equity structure is shown in Table 4:

[0133] Table 4

[0134] Serial number Name of shareholder Amount of shareholding (in RMB 10,000) Percentage of shareholding 1 Guo 40,000,000 100.00 Total 40,000,000 100.00

[0135] The equity structure after the fifth capital increase is shown in Table 5:

[0136] Table 5

[0137] Serial number Name of shareholder Amount of shareholding (in RMB 10,000) Percentage of shareholding 1 Guo 60,000,000 100.00 Total 60,000,000 100.00

[0138] By comparing the changes and differences between Table 4 and Table 5, the detailed information of the equity transfer and capital increase events can be derived.

[0139] For example, in the equity transfer event, although the table related to the equity transfer information is given, usually only the number and proportion of each transferor / transfer recipient's transferred / accepted equity are listed, and the corresponding relationship between the transferor and the transfer recipient in each transfer is not specified, which is the information needed to extract the transfer event. Therefore, it is often necessary to match and deduce the transfer amount and proportion given in different rows in the table to try to extract the corresponding relationship between the "transferor" and the "transfer recipient" and the specific "amount" and "proportion" of each transfer.

[0140] Example 3:

[0141] The equity structure of a certain company after the third equity transfer is shown in Table 6:

[0142] Table 6

[0143]

[0144] From the transfer amount and proportion given in different rows in the table, it can be deduced that the "transfer recipient" is Du Er, the "transferor" is Liu, and the specific amount and proportion of the transfer.

[0145] In some more complex equity change scenarios, for example, involving a "transferor" transferring to multiple "transfer recipients" or a "transfer recipient" receiving from multiple "transferors", if the same amount and proportion appear, it is sometimes impossible to completely and unambiguously match the corresponding relationship between the "transferor" and the "transfer recipient" and the involved amount and proportion in each transfer.

[0146] Example 4: The equity structure of a certain company after the second equity transfer is shown in Table 7:

[0147] Table 7

[0148]

[0149] The same amount and proportion appear, and it is impossible to completely and unambiguously match the corresponding relationship between the "transferor" and the "transfer recipient" and the involved amount and proportion in each transfer. Additional information needs to be obtained from the text to complete the final accurate event argument extraction.

[0150] Step 333: Use the structured information in the table to enhance the text extraction. For event extraction from financial documents, such as various types of events related to equity change, there are usually a lot of arguments involved in an event. Using current methods and models to extract events directly from text is still very challenging and often has missing and errors. So we first consider extracting events from the table, and if the table cannot provide complete information, we then consider extracting events from the text to complete and supplement. In many cases, although complete event information cannot be extracted from the table, the information provided in the table can still greatly enhance the performance of extracting event arguments from the text.

[0151] As shown in Figure 6 , the method of using the structured information in the table to enhance text extraction includes:

[0152] Step 3331: Filter and screen event-related sentences.

[0153] The text content can be very long, but the number of sentences that specifically describe the arguments of the events that need to be extracted is very limited.

[0154] A sentence text classifier can be trained to classify each sentence to filter out relevant sentences. This can greatly reduce the workload and unnecessary noise of subsequent processing. Taking various event types of equity change as an example, the characteristics of relevant sentences are usually explicit, so a moderate amount of labeled sentences as training data should achieve good performance.

[0155] Some heuristic rules can also be used to further narrow the scope of processing. For example, in financial documents, the text description of equity change is usually in front of the relevant table, so if the relevant table can be located, the scope of processing the text can be reduced.

[0156] Example 5: There are 7 pages of content in a certain target area, but there are only two sentences that contain specific descriptions of capital increase and transfer events. Therefore, classifying and filtering out relevant sentences can greatly improve efficiency and performance. Step 3332: Match entities in the table to event-related sentences.

[0157] In the selected specific type of event-related sentences, matching the entities parsed in the table can not only find the corresponding mentions, but also directly give their argument roles in the event. When the type of the event-related sentence is determined, and the entities extracted from the table are matched in the sentence, the difficulty of extracting event arguments from the text is greatly reduced, because part of the arguments in the sentence have been determined, which means that the context available for extracting the remaining event arguments is greatly enriched, so it is easier and more accurate to extract the remaining event arguments.

[0158] In Example 5, when a company is established, the equity structure is shown in Table 8:

[0159] Table 8

[0160] Serial number Name of shareholder Amount of shareholding (in RMB 10,000) Percentage of shareholding 1 Liu Yi 1175.00 47.00 2 Qian 1175.00 47.00 3 Jin 125.00 5.00 4 Wang 12.50 0.50 5 Li 12.50 0.50

[0161] After the capital increase and the transfer of equity, it is shown in Table 9:

[0162] Table 9

[0163] Serial number Name of shareholder Amount of shareholding (in 10,000 shares) Percentage of shareholding 1 Liu Yi 1175.00 41.75 2 Qian 1175.00 41.74 3 Jin 100 3.55 4 Liu Er 37.50 1.33 5 Li 12.5 0.44 6 An Company 109.00 3.87 7 Hong Company 103.00 3.66 8 Guo Company 103.00 3.66

[0164] In Table 8 and Table 9, only the final capital structure table after the completion of the capital increase and the transfer is shown. There is no capital increase table and no transfer table in other announcements.

[0165] Through table semantic parsing and comparison of Table 8 and Table 9, the following entities and corresponding argument roles can be derived:

[0166] “Liu”, “An Company”, “Guo Company”, “Hong Company” are “shareholders” of the “capital increase” event or “transferees” of the “transfer” event, i.e. new shareholders; “Wang” is the “transferor” of the “transfer” event or “capital reduction”, corresponding to the “transfer amount” of “125,000 yuan”; “Jin” is the “transferor” of the “transfer” event. When the capital increase event and the transfer event are mixed together, the relevant arguments can be extracted from the text. The text content can be matched by the table entities. The full name and abbreviation of the institution can be matched by parsing the content of “Section 1 Interpretation” at the beginning of the announcement. The numerical entities can be unified into the same format before matching.

[0167] Through table semantic parsing and comparison of the before and after event tables, the following entities and corresponding argument roles are obtained: “Liu”, “An Company”, “Guo Company”, “Hong Company” are “shareholders” of the “capital increase” event or “transferees” of the “transfer” event, “Wang” is the “transferor” of the “transfer” event, corresponding to the “transfer amount” of “125,000 yuan”, “Jin Lei Li” is the “transferor” of the “transfer” event

[0168] But in Table 9, the content of "Wang" is missing; in Table 8, the content of "Liu Er" is missing, and in Table 8 and Table 9, the content of the transaction price is missing. Therefore, the following relevant sentences can be matched from the text by means of entity matching: Wang will transfer the 125,000 yuan of shares held by the company to Liu at a price of 125,000 yuan, and Wang will no longer be a shareholder of the company; Liu Er is a new shareholder of the company, holding 375,000 shares; and the company subscribes for 1.09 million shares at a price of 2.5 yuan per share. The corresponding supplementary argument and supplementary argument role can be extracted from the relevant sentences.

[0169] Step 3333: Extract the event argument that needs to be supplemented from the text.

[0170] As can be seen from Example 5, after determining the event type contained in the relevant sentence, when matching the entities extracted from the table to the sentence, the difficulty of extracting event arguments from the text is greatly reduced, because part of the arguments in the sentence have already been determined, which greatly enriches the context available for extracting the remaining event arguments, so it is easier and more accurate to extract the remaining event arguments.

[0171] For simple scenarios involving unambiguous entity argument roles, the results can be directly inferred; for entities matched according to table information, the corresponding roles cannot be directly determined; and for those event arguments not mentioned in the table, machine learning methods are required.

[0172] For example, in Example 5, there is the following sentence: "Wang will transfer the 125,000 yuan of shares held by the company to Liu at a price of 125,000 yuan, and Wang will no longer be a shareholder of the company." The event type of this sentence is transfer, and according to the previous deduction, the only event argument of "Wang" is "transferor", and although "Liu" has two possible event arguments, it can be uniquely determined as "transferee" in this sentence due to the limitation of the event type "transfer". The amount of 125,000 yuan is the "transfer amount" according to the table information. Therefore, the event arguments involved in this "transfer" event can be easily and directly deduced.

[0173] For example, there is the following sentence: "Jin will transfer the 250,000 yuan of shares held by the company to Liu at a price of 250,000 yuan, Jin's shareholding amount will be 1 million shares, Liu will be a new shareholder of the company, and the shareholding amount will be 375,000 shares." Among them, the event type contained in the sentence is determined to be "transfer"; "Jin" is the "transferor" of the "transfer" event, and "Liu" is the "transferee" of the "transfer" event; and the "transfer amount" of 25,000 yuan and the "transfer price" of 25,000 yuan are not mentioned in the table, and machine learning methods can be used to extract them.

[0174] For example, there is the following sentence: "An company subscribes 109 million shares at the price of 2.5 yuan per share", the event type of this sentence is "capital increase", "An company" is the "shareholder name" of the "capital increase" event, and "109 million shares" is the "increased share number" of the "capital increase" event. The "capital increase price" is "2.5 yuan per share" and is not mentioned in the table, which needs to be extracted by machine learning method.

[0175] According to the information of entity matching extracted from the table, the representation of the related sentence is enriched:

[0176] Initialize the encoding of the word (token) by using a large-scale pre-training model such as BERT;

[0177] Each entity type is represented by a learnable type encoding (embedding);

[0178] The representation of the part of the word (token) in the event-related sentence that matches the entity obtained from the table needs to be added with the encoding (embedding) of the entity type.

[0179] The machine learning extraction method of the supplementary event argument can be based on sequence labeling method, machine reading comprehension method (MRC, Machine Reading and Comprehension) or Prompt method.

[0180] In the sequence labeling method, since part of the argument in the sentence can be matched and obtained according to the entity in the table, only the argument type not mentioned in the table needs to be labeled.

[0181] Based on machine reading comprehension method (MRC): define the supplementary argument extraction problem as a question and answer problem; based on the question and answer problem, find the answer to the question in the sentence to obtain the supplementary argument.

[0182] Prompt method is to define the event argument extraction problem as a method that directly generates answers through seq2seq.

[0183] BERT-based sequence labeling method, BERT-based MRC or Prompt method to extract arguments are prior art, which will not be described in detail in the present application. The present application reduces the extraction range of arguments by using the entity and entity type vectors in the table.

[0184] Step 3334: extract a small amount of information that can only be obtained from text.

[0185] For example, part of the event argument only appears in the text, such as the date of business registration, which can be easily realized by sentence classification filtering and sequence labeling; part of the argument only appears in a specific text or sentence in financial documents.

[0186] Step 334: Cross verification of text and chart information.

[0187] On one hand, the event list extracted from the relevant title can be compared with the event list extracted from the sketch to check whether there is a missing event. On the other hand, according to the event argument extracted from the text, it is checked whether the stock ownership structure table between the result after the completion of the event and the current event matches. In the text extraction, manual annotation of the text is usually required, and the cost of annotation is high. However, by using the content of the table, automatic text annotation is facilitated to train a model for extracting information from the text, so that the system can be better applied to the scene lacking the table.

[0188] The application also provides a system for implementing the event extraction method, as shown in the accompanying drawings, comprising a positioning module 3, a table analysis module 11, a text screening module 13 and an argument extraction module 14, Figure 5

[0189] The positioning module 3 is used to obtain the extraction area and its objects, and the objects include tables and texts.

[0190] The table analysis module 11 is used to analyze the table to obtain the content of the table, and the content includes events and arguments.

[0191] The text screening module 13 is used to screen the text according to the events of the table to obtain relevant sentences.

[0192] The argument extraction module 14 is used to extract argument roles and / or supplementary arguments from the relevant sentences.

[0193] The system further comprises a layout analysis module 1, a document hierarchical analysis module 2, a graph analysis module 5, a segmentation module 4, a title extraction module 6 and a verification module 7,

[0194] The layout analysis module 1 is used to perform layout recognition on the financial document to obtain titles, texts, tables and graphs.

[0195] The document hierarchical analysis module 2 is used to identify the level of the title according to the visual features and text features of the title, and obtain the hierarchical structure of the financial document according to the level of the title.

[0196] The positioning module 3 is used to position the extraction area according to the hierarchical structure and obtain the objects of the extraction area.

[0197] The graph analysis module 5 is used to analyze the graph to obtain graph structured information, and obtain a second event list according to the graph structured information.

[0198] ​The segmentation module 4 is used for segmenting the region to be extracted and merging the cross-page content of the region to be extracted;

[0199] The title extraction module 6 is used for obtaining the event of the title and the event list thereof;

[0200] The verification module 7 is also used for cross-verifying the event list extracted in the sketch and the event list extracted in the title, checking whether there is a missed event or not, or verifying the content of the information after the supplementary argument is calculated and the event is deduced and the content of the event table of the current event, checking whether they are matched or not.

[0201] The application can also comprise an event graph construction module, which is used for constructing an event graph according to the first argument, the supplementary argument, the argument role and the event type.

[0202] The above only describes the preferred embodiments of the application and is not used to limit the application, and various modifications and changes can be made by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application shall be included in the protection scope of the application.

Claims

1. A method for event extraction of multi-modal financial documents, the method comprising: The method comprises: Obtaining the region to be extracted and its objects, the objects including tables and texts; Parsing the tables to obtain the content of the tables, the content including events and arguments; Screening the texts according to the event filter of the tables to obtain relevant sentences; Extracting argument roles and / or supplementary arguments from the relevant sentences; Wherein, the content of the tables includes entities, and the method for extracting argument roles from the relevant sentences comprises: matching the entities of the tables with the relevant sentences to obtain argument roles and argument mentions; Wherein, the method for obtaining relevant sentences comprises: obtaining the events of the tables according to the content of the tables or the titles of the tables; obtaining the classification of the sentences through a sentence text classifier; and screening the sentences according to the events and the classification; Wherein, the method for extracting supplementary arguments from relevant sentences comprises: a sentence classification filtering method, a sequence labeling method or a machine learning method, the machine learning method including a sequence labeling method based on a pre-trained language model, a machine reading comprehension method based on a pre-trained language model or a Prompt method; The sequence labeling method based on a pre-trained language model comprises: encoding the entities through a pre-trained language model to obtain a first encoding; adding a vector of an entity type in the first encoding to obtain a second encoding; encoding the relevant sentences through a pre-trained language model to obtain a third encoding; adding the entity type encoding in the part of the third encoding that matches the entities; and after labeling the part of the relevant sentences that does not match, obtaining the supplementary arguments by using a sequence labeling method; The machine reading comprehension method based on a pre-trained language model comprises: defining the extraction of supplementary arguments as a question and answer problem; and based on the question and answer problem, finding the answer to the question in the sentence to obtain the supplementary arguments; The Prompt method is a method for defining the extraction of event arguments as a method for directly generating answers through a seq2seq manner.

2. The event extraction method of claim 1, wherein, The method for obtaining the region to be extracted comprises: Performing layout recognition on the financial document to obtain titles, texts, tables and graphics; Identifying the level of the titles according to visual features and text features of the titles; Obtaining the hierarchical structure of the financial document according to the level of the titles; Positioning and segmenting the region to be extracted according to the hierarchical structure.

3. The event extraction method of claim 2, wherein, The region to be extracted further comprises any one of the following objects or a combination thereof: pictures, picture descriptions, headers, footers, page numbers, table titles and table descriptions; The event types include stock transfer events, capital reduction events and capital increase events; The objects further include titles and graphics, and the object parsing further includes title parsing, graphic parsing and merging cross-page content, Merging cross-page content comprises deleting headers, footers and page numbers, and aligning cross-page tables.

4. The event extraction method of claim 1, wherein, The method further comprises a cross-validation method: Cross-validating the event list extracted from the sketch map and the event list extracted from the titles to check whether there are any missing events; After calculating and deducing the information of the event after the supplementary arguments extracted from the texts and the content of the table of the event, verifying whether they match.

5. The event extraction method of claim 4, wherein, The method further comprises a method for training a text extraction model using table content: According to the entity of the table, the text data is labeled; The text information extraction model is trained by using the labeled text data.

6. A system for implementing the event extraction method according to any one of claims 1 to 5, characterized in that, The system comprises a positioning module, a table analysis module, a text screening module and an argument extraction module, The positioning module is used to obtain the region to be extracted and its objects, and the objects include tables and texts. The table analysis module is used to analyze the table to obtain the content of the table, and the content includes events and arguments. The text screening module is used to screen the text according to the events of the table to obtain relevant sentences. The argument extraction module is used to extract argument roles and / or supplementary arguments from the relevant sentences.

7. The system of claim 6, wherein, The system further comprises a layout analysis module, a document hierarchy analysis module, a graph analysis module, a segmentation module, a title extraction module and a verification module, The layout analysis module is used to perform layout recognition on the financial document to obtain titles, texts, tables and graphs. The document hierarchy analysis module is used to identify the hierarchy of the title according to the visual features and text features of the title, and obtain the hierarchical structure of the financial document according to the hierarchy of the title. The positioning module is used to position the region to be extracted according to the hierarchical structure and obtain the objects of the region to be extracted. The graph analysis module is used to analyze the graph to obtain graph structured information. The segmentation module is used to segment the region to be extracted; The title extraction module is used to obtain the events of the title and the event list thereof; The verification module is used to cross-verify the event list extracted from the sketch and the event list extracted from the title to check whether there is a missing event, or to verify the information after the supplementary argument extracted from the text is calculated and deduced and the content of the event table to check whether they match.

Images