Data extraction method and device, storage medium and electronic equipment

Abstract

The invention discloses a data extraction method and device, a storage medium and electronic equipment, and the method comprises the steps: receiving a security code and a financial report service classification, and obtaining a corresponding financial report document from at least one data source according to the security code; performing keyword positioning on each page of the financial report document, and taking the pages containing keywords as candidate pages to generate a candidate page set; for each candidate page in the candidate page set, performing page expansion based on adjacent pages of the candidate page according to a preset rule, and merging the expanded pages into a sub-document; converting the sub-document into an intermediate representation, and extracting candidate numerical evidence corresponding to the target financial index based on the intermediate representation; and verifying and fusing the candidate numerical evidence to output target financial data. The extraction efficiency and accuracy of the financial report data can be improved, and the cost is reduced.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a data extraction method, apparatus, storage medium, and electronic device. Background Technology

[0002] With the in-depth development of fields such as financial data analysis, corporate credit investigation and investment research, the efficient and accurate extraction of structured data from the financial reports of listed companies has become a core and demanding requirement.

[0003] Currently, PDF format financial reports have become the primary carrier of information on corporate operations and financial status. Efficiently and accurately extracting core financial indicators for specific business segments from massive amounts of financial reports is fundamental to financial analysis, risk management, and investment decisions.

[0004] However, financial reports are generally lengthy, have non-standard formats, complex tables, and are often spread across multiple pages. This makes the traditional manual review and data entry methods face serious challenges in extracting core financial indicators for specific business segments, as they are inefficient, costly, and prone to errors. Summary of the Invention

[0005] This application provides a data extraction method, apparatus, storage medium, and electronic device, which can improve the efficiency and accuracy of extracting financial report data and reduce costs.

[0006] In a first aspect, embodiments of this application provide a data extraction method, including: Receive securities codes and financial statement business categories, and retrieve the corresponding financial report documents from at least one data source based on the securities codes; Keyword location is performed on each page of the financial report document, and pages containing keywords are selected as candidate pages to generate a candidate page set; For each candidate page in the candidate page set, the page is expanded according to a predetermined rule based on the adjacent pages of the candidate page, and the expanded pages are merged into a sub-document; The sub-document is converted into an intermediate representation, and candidate numerical evidence corresponding to the target financial indicator is extracted based on the intermediate representation; The candidate numerical evidence is verified and fused to output the target financial data.

[0007] In the data extraction method provided in this application embodiment, the step of locating keywords for each page of the financial report document and using pages containing keywords as candidate pages to generate a candidate page set includes: Extract the text content of each page of the financial report document, and detect the category keywords corresponding to the financial report business category and the indicator keywords corresponding to the target financial indicators based on the text content; When a page contains the category keyword and at least one indicator keyword, the page is identified as a candidate page. All candidate pages are aggregated to form a candidate page set.

[0008] In the data extraction method provided in this application embodiment, the step of expanding the adjacent pages of the candidate page according to a predetermined rule and merging the expanded pages into a sub-document includes: For each candidate page, a continuity test is performed to determine whether the table structure and / or text theme of the candidate page have continuity in adjacent pages; When there is continuity, the adjacent page and the candidate page are merged to form an expanded page; Merge all the expanded pages into a subdocument.

[0009] In the data extraction method provided in this application embodiment, converting the sub-document into an intermediate representation includes: The page content of the sub-document is structured and parsed to identify and annotate table areas and text paragraph areas; Reconstruct the row and column structure based on the cell layout of the table area to generate a table element containing row and column coordinates and cell content; Based on the text paragraph region, paragraph elements containing semantic block divisions are generated; The intermediate representation is composed of the table elements and paragraph elements.

[0010] In the data extraction method provided in this application embodiment, the step of extracting candidate numerical evidence corresponding to the target financial indicator based on the intermediate representation includes: The intermediate representation is parsed using a rule-based parser to obtain the first candidate evidence; If the first candidate evidence does not meet the preset reliability requirement or preset quantity requirement, then a prompt message is constructed, and a large language model is called to perform semantic parsing on the prompt message to obtain the second candidate evidence; Candidate numerical evidence is constituted by the first candidate evidence and / or the second candidate evidence.

[0011] In the data extraction method provided in this application embodiment, the step of verifying and fusing the candidate numerical evidence to output target financial data includes: The candidate numerical evidence is cleaned and normalized to obtain standardized values. Based on preset confidence rules, multiple standardized values ​​for the same target financial indicator are fused to obtain target fused data; The target fused data is converted into target financial data conforming to a predefined structured data object format and then output.

[0012] In the data extraction method provided in this application embodiment, the step of fusing multiple standardized values ​​for the same target financial indicator based on a preset confidence rule to obtain target fused data includes: Based on the evidence source type corresponding to each standardized value, a corresponding confidence weight is assigned to the standardized value. When the evidence source type is a rule parser, the confidence weight assigned to the standardized value is a first weight. When the evidence source type is a large language model, the confidence weight assigned to the standardized value is a second weight. The first weight is greater than the second weight. Based on the confidence weights, the multiple standardized values ​​are weighted and summed, and the result is used as the target fused data.

[0013] Secondly, embodiments of this application provide a data extraction apparatus, comprising: The receiving unit is used to receive securities codes and financial statement business classifications, and to obtain the corresponding financial report documents from at least one data source based on the securities codes; The positioning unit is used to locate keywords in each page of the financial report document and select pages containing keywords as candidate pages to generate a set of candidate pages. An expansion unit is used to expand each candidate page in the candidate page set according to a predetermined rule based on the adjacent pages of the candidate page and merge the expanded pages into a sub-document; An extraction unit is used to convert the sub-document into an intermediate representation and extract candidate numerical evidence corresponding to the target financial indicator based on the intermediate representation. The output unit is used to verify and fuse the candidate numerical evidence to output the target financial data.

[0014] Thirdly, this application provides a storage medium storing a plurality of instructions adapted for loading by a processor to execute any of the data extraction methods described above.

[0015] Fourthly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the data extraction method described in any of the preceding claims.

[0016] In summary, the data extraction method provided in this application includes: receiving a securities code and a financial statement business classification, and obtaining the corresponding financial report document from at least one data source based on the securities code; performing keyword positioning on each page of the financial report document, and using pages containing keywords as candidate pages to generate a candidate page set; for each candidate page in the candidate page set, expanding the page based on the adjacent pages of the candidate page according to a predetermined rule, and merging the expanded pages into a sub-document; converting the sub-document into an intermediate representation, and extracting candidate numerical evidence corresponding to the target financial indicator based on the intermediate representation; verifying and fusing the candidate numerical evidence to output the target financial data. This application employs a fully automated processing flow. By locating keywords in each page and using pages containing those keywords as candidate pages to generate a candidate page set, the scope of documents to be processed is significantly reduced. Subsequently, candidate pages are expanded and merged based on adjacent pages to form complete sub-documents, avoiding repeated location or parsing attempts due to context breaks. This converts the sub-documents into an intermediate representation, and data extraction and processing are performed based on this intermediate representation. This optimizes the core parsing path, thereby improving the efficiency and accuracy of financial report data extraction and reducing costs. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram illustrating an application scenario of the data extraction method provided in the embodiments of this application.

[0019] Figure 2 This is a flowchart illustrating the data extraction method provided in the embodiments of this application.

[0020] Figure 3 This is a schematic diagram of the data extraction device provided in the embodiments of this application.

[0021] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0022] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0023] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of this application may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0024] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0025] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0026] In the description of this application, it should be noted that the terms "upper," "lower," "left," "right," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. In addition, terms such as "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0027] Currently, PDF format financial reports have become the primary carrier of information on corporate operations and financial status. Efficiently and accurately extracting core financial indicators for specific business segments from massive amounts of financial reports is fundamental to financial analysis, risk management, and investment decisions.

[0028] However, financial reports are generally lengthy, have non-standard formats, complex tables, and are often spread across multiple pages. This makes it difficult to extract core financial indicators for specific business segments using traditional manual review and data entry methods. Such methods are inefficient, costly, and prone to errors.

[0029] Based on this, embodiments of this application provide a data extraction method, apparatus, storage medium, and electronic device. Specifically, the data extraction apparatus can be integrated into an electronic device, which can be a server or a terminal, etc. The terminal can include mobile phones, wearable smart devices, tablets, laptops, and personal computers (PCs), etc. The server can be a single server or a server cluster composed of multiple servers, and can be a physical server or a virtual server.

[0030] For example, such as Figure 1 As shown, the electronic device can receive securities codes and financial statement business classifications, and obtain the corresponding financial report documents from at least one data source based on the securities codes; it performs keyword positioning on each page of the financial report documents, and uses pages containing keywords as candidate pages to generate a candidate page set; for each candidate page in the candidate page set, it expands the candidate page according to predetermined rules based on the adjacent pages of the candidate page, and merges the expanded pages into a sub-document; it converts the sub-document into an intermediate representation, and extracts candidate numerical evidence corresponding to the target financial indicators based on the intermediate representation (i.e., extracts candidate numerical evidence corresponding to the target financial indicators from the intermediate representation); it verifies and merges the candidate numerical evidence to output the target financial data.

[0031] The technical solutions shown in this application will be described in detail below through specific embodiments. It should be noted that the order of description of the following embodiments is not intended to limit the priority of the embodiments.

[0032] Please see Figure 2 , Figure 2 This is a flowchart illustrating the data extraction method provided in an embodiment of this application. The specific flow of the data extraction method is as follows: 101. Receive securities codes and financial statement business classifications, and retrieve the corresponding financial report documents from at least one data source based on the securities codes.

[0033] In some embodiments, the electronic device can receive input stock codes and financial statement business categories through a graphical user interface, application programming interface, or file import method. The stock code is used to uniquely identify a listed company, such as "600519.SH"; the financial statement business category is used to specify the specific business segment to which the financial data to be extracted belongs, such as "liquor business".

[0034] In this embodiment, after obtaining the securities code and financial statement business classification, they can be standardized. For example, the data format of the securities code can be unified into a format with market identifiers, and the financial statement business classification can be cleaned (e.g., removing spaces, special characters, etc.).

[0035] Once a standardized security code is obtained, one can use that code to query one or more data sources to retrieve the corresponding financial report documents.

[0036] Specifically, firstly, a pre-built enterprise information database (such as a relational database like MySQL) can be accessed based on the standardized security code. The database then queries the unique identifier of the enterprise associated with the standardized security code, along with the metadata of the enterprise's financial report documents for a specified reporting year (e.g., the latest fiscal year by default, or a year specified by the user). This metadata includes at least the storage address (e.g., a URL link) and document identifier of the financial report documents.

[0037] Subsequently, based on the retrieved storage address, the financial report document can be downloaded from the corresponding data source (such as the official website of a stock exchange, a corporate information disclosure platform, or the storage server of a third-party data service provider) via HTTP, FTP, or other file transfer protocols. This financial report document is typically in PDF format. During the download process, network anomaly handling mechanisms can be implemented, such as setting timeout limits and retrying on failure, to ensure the reliability of the download.

[0038] Finally, successfully downloaded financial report documents can be backed up and managed. For example, they can be uploaded to Object Storage Service (OSS) and a unique file access link can be generated, while the file's hash value (such as MD5 or SHA-256) can be recorded for subsequent integrity verification and deduplication.

[0039] 102. Locate keywords for each page of the financial report document and select pages containing keywords as candidate pages to generate a candidate page set.

[0040] In some embodiments, a PDF parsing library (such as pdfplumber or PyPDF2) can be used to parse the financial report document, allowing for page-by-page traversal of the document and enabling keyword location for each page.

[0041] Specifically, step 102 may include the following steps: 1021. Extract the text content of each page of the financial report document, and detect the category keywords corresponding to the financial report business categories and the indicator keywords corresponding to the target financial indicators based on the text content.

[0042] Specifically, you can first initialize the PDF parser, load the financial report document, and obtain its total number of pages. Then, start a loop and perform the following operations page by page (e.g., starting from page 0): ① Text Content Extraction: For the current page, the PDF parser's page text extraction program can be invoked to obtain the corresponding text content for that page. During this process, the PDF parser can be configured to preserve the original layout and order information of the text content as much as possible. For example, extraction can be set to follow the natural reading order of text chunks. The extracted text content is temporarily stored and associated with the page number of the current page.

[0043] ② Text content preprocessing: Perform basic text cleaning on the extracted text content, such as converting it to lowercase (or uppercase) to achieve case-insensitive matching, and removing redundant whitespace characters.

[0044] ③ Category Keyword Generation: The financial statement business categories can be standardized and expanded to generate category keywords for matching.

[0045] Specifically, the first step is to remove non-core words such as modifiers and connectors from the input strings corresponding to the financial report business categories. For example, the core word extracted from "high-end liquor business" is "liquor". Then, the core word can be expanded based on a pre-built industry thesaurus. For example, "liquor" can be expanded to "alcoholic beverages" or "spirits" to enhance the robustness of the matching. The expanded terms together constitute the category keywords.

[0046] ④ Keyword generation: This embodiment includes built-in keywords that strictly correspond to the target financial indicators. For example, if the target financial indicator can be "operating revenue", "operating cost", and "gross profit margin", then the corresponding keywords are "operating revenue", "operating cost", and "gross profit margin", respectively, achieving a strict correspondence between the keyword and the target financial indicator.

[0047] 1022. When a page contains category keywords and at least one indicator keyword, the page is identified as a candidate page.

[0048] In this embodiment, the condition for determining whether a page is a "candidate page" is that the page contains both category keywords and at least one indicator keyword.

[0049] This judgment mechanism can effectively filter descriptive pages that only mention business categories but do not involve specific financial indicators (such as business overviews) and pages that mention financial indicators but are unrelated to the business categories specified by the user (such as overall corporate financial summaries and notes on other business segments), thereby improving the accuracy of locating candidate pages and enabling precise identification of candidate pages from each subpage.

[0050] 1023. Aggregate all candidate pages to form a candidate page set.

[0051] Specifically, the page numbers of all pages marked as candidate pages can be collected, and then the candidate pages can be sorted in ascending order of page number to form an ordered set of candidate pages.

[0052] This candidate page set represents the most valuable portions of the financial reporting documents that require further processing. Typically, this set contains only a very small percentage of the pages in the financial reporting documents (e.g., 3 to 10 pages), thus providing an efficient foundation for subsequent computationally intensive steps such as page merging, structured parsing, and model invocation.

[0053] 103. For each candidate page in the candidate page set, expand the adjacent pages of the candidate page according to a predetermined rule and merge the expanded pages into a sub-document.

[0054] Specifically, a continuity check can be performed on each candidate page to determine whether the table structure and / or text theme of the candidate page are continuous in adjacent pages; if they are continuous, the adjacent pages are merged with the candidate pages to form an expanded page; all expanded pages are then merged into a sub-document.

[0055] First, the total number of pages in the candidate page set can be obtained to prepare for processing each candidate page. For each candidate page in the candidate page set (let its page number be p), a predetermined rule can be set to determine the range of adjacent pages that need to be merged with it. This predetermined rule can be implemented in different strategies.

[0056] For example, there's the fixed window expansion strategy. Under this strategy, a fixed expansion window can be calculated for a candidate page p. For instance, the window range is defined as the pages corresponding to page numbers p-1, p, and p+1. Boundary cases can be handled automatically: if p is the first page, then p-1 is invalid; if p is the last page, then p+1 is invalid. By traversing all candidate pages and collecting all valid page numbers calculated for each candidate page using this rule, a preliminary list containing duplicate page numbers is formed.

[0057] For example, a dynamic expansion strategy based on content continuity. Under this strategy, continuity detection can be performed on candidate page p. Specifically, the end of candidate page p can be analyzed to determine whether its content is highly likely to continue to subsequent pages. Detection criteria may include: determining whether there is a table at the end of page p that is not ended by a "Total" or "Summarize" row; analyzing whether the text at the end of page p is truncated in the middle of a complete sentence or paragraph; or checking whether there are explicit indicative words such as "Continue to next page" at the bottom of page p. If a high degree of continuity with page p+1 is detected, page p+1 is included in the expansion scope. Subsequently, the above continuity detection can be repeated iteratively on newly included pages (such as page p+1) to determine whether further expansion to page p+2 is needed, until no continuity is detected or a preset maximum expansion depth is reached (e.g., a maximum of 3 additional pages).

[0058] Understandably, regardless of the expansion strategy employed, after calculating the expansion range for all candidate pages, a set containing multiple page numbers (which may be duplicates) will be obtained. Next, a deduplication operation can be performed on this page number set to eliminate duplicate page numbers resulting from overlapping expansion ranges of different candidate pages. Afterward, the page numbers are sorted in ascending order to form a page number list.

[0059] Finally, based on this ordered and unique list of page numbers, the corresponding pages can be extracted from the financial report document, and these pages can be merged in order using a PDF processing library (such as PyPDF2's PdfMerger function) to generate a sub-document.

[0060] 104. Convert the sub-documents into intermediate representations and extract candidate numerical evidence corresponding to the target financial indicators based on the intermediate representations.

[0061] The step "converting the sub-document into an intermediate representation" can be as follows: performing structured parsing of the sub-document's page content, identifying and annotating table areas and text paragraph areas; reconstructing the row and column structure based on the cell layout of the table areas, generating table elements containing row and column coordinates and cell content; generating paragraph elements containing semantic block divisions based on the text paragraph areas; and constructing the intermediate representation together with the table elements and paragraph elements.

[0062] Specifically, a more advanced document understanding module can be invoked to process the sub-documents. This module performs structured parsing of the page content for each page, breaking it down into two core elements: table elements and paragraph elements.

[0063] For table areas, the system can detect table borders and cells, reconstruct its two-dimensional grid logical structure, handle the expansion of merged cells, and identify header rows and data rows. Each cell's content is associated with its precise row and column coordinates, forming a table data object with a well-defined structure.

[0064] For non-table text paragraphs, they can be divided into coherent paragraph elements according to reading order and paragraph format, and can be further segmented into finer-grained sentences or semantic blocks.

[0065] Paragraph and table elements, along with their original page numbers and position information on the page (such as bounding box coordinates), are organized together into a standardized intermediate representation rich in semantic and structural information.

[0066] In the embodiments of this application, a hierarchical collaborative parsing strategy can be used to execute the step of "extracting candidate numerical evidence corresponding to the target financial indicator based on the intermediate representation".

[0067] Specifically, a rule-based parser can be used to parse the intermediate representation to obtain the first candidate evidence.

[0068] This rule-based parser can scan intermediate representations according to a set of predefined pattern matching rules to obtain first candidate evidence. These pattern matching rules may include: in table elements, locating the intersection of the row and column header cells that perfectly match the indicator keyword (such as "operating revenue") and extracting the value from that cell; and in paragraph elements, using regular expressions to match specific numerical patterns immediately following the indicator keyword (such as "gross profit margin is 45.6%").

[0069] Next, the first candidate evidence can be evaluated. If the first candidate evidence does not meet the preset reliability or quantity requirements, a prompt message can be constructed, and a large language model can be called to perform semantic parsing on the prompt message to obtain the second candidate evidence; finally, the first candidate evidence and / or the second candidate evidence constitute the candidate numerical evidence.

[0070] Specifically, constructing the prompt information involves selecting the text fragment most relevant to the financial statement business category and the target financial indicator from the intermediate representation as the content to be analyzed. This text fragment can be a complete table element or a combination of several paragraph elements containing key data.

[0071] The content to be analyzed, the financial statement business categories, the list of target financial indicators, and the output instructions are then combined into a structured prompt text, i.e., the prompt message. This output instruction can require the large language model to strictly follow the predefined JSON format for outputting results. This format must at least contain the value, unit, and source_page fields for each target financial indicator, and stipulate that if an indicator cannot be determined from the given content, the corresponding field should be output as null.

[0072] By constructing this prompt, a clear task boundary, sufficient contextual information, and strict result constraints can be provided for the large language model, thereby guiding it to perform accurate semantic parsing and output second candidate evidence.

[0073] 105. Verify and integrate the candidate numerical evidence, and output the target financial data.

[0074] In some embodiments, step 105 may include the following steps: 1051. Perform numerical cleaning and unit normalization on the candidate numerical evidence to obtain standardized values.

[0075] Numerical cleaning refers to removing any thousands separators (such as the Chinese comma "、" or the English comma ",") from numbers. For example, "1,234.56" is cleaned to "1234.56". It identifies and processes negative numbers represented by parentheses, converting "(123.45)" to "-123.45". It converts the percentage sign "%" to a decimal, for example, "45.6%" is converted to "0.456". Simultaneously, it cleans up any non-numeric characters that may be attached before or after numbers.

[0076] Specifically, unit normalization involves identifying the currency unit, such as "yuan," "ten thousand yuan," or "one hundred million yuan," within the numerical string or determined by context (e.g., table headers). Then, based on a pre-defined unit conversion table (e.g., 10,000 yuan = 10,000 yuan, 100 million yuan = 100,000,000 yuan), all currency values ​​are uniformly converted to a base value in "yuan." For example, "1,234,500 yuan" becomes "1234500" after normalization.

[0077] After completing the numerical cleaning and unit normalization, a set of standardized values ​​for the same target financial indicator (such as "operating revenue") can be obtained.

[0078] 1052. Based on the preset confidence level rules, multiple standardized values ​​for the same target financial indicator are fused to obtain target fused data.

[0079] Specifically, a confidence weight can be assigned to each standardized value based on the type of evidence source corresponding to that standardized value. When the evidence source type is a rule-based parser, the standardized value is assigned the first confidence weight, and when the evidence source type is a large language model, the standardized value is assigned the second confidence weight. The first weight is greater than the second weight.

[0080] Understandably, the results obtained by the rule parser from directly extracting from well-structured tables are given a higher first weight (e.g., 0.7) due to their greater determinism and interpretability. The results extracted by the large language model through semantic understanding, as an important supplement, are given a relatively lower second weight (e.g., 0.3). If the rule parser itself outputs a confidence score, this confidence score can also be used to fine-tune its weights.

[0081] Then, based on the confidence weights, multiple standardized values ​​are weighted and summed, and the result is used as the target fused data.

[0082] 1053. Convert the target fusion data into target financial data that conforms to the preset structured data object format and output it.

[0083] In some embodiments, consistency verification can be performed on the target fused data, for example, by using the formula "gross profit margin = (operating revenue - operating cost) / operating revenue" for verification. If the calculation relationship of each target data exceeds the reasonable error range, the record can be marked as needing to be reviewed.

[0084] Finally, all target data can be merged and output according to a preset structured data object format (such as a JSON object containing company name, stock code, business category, year, and values ​​of various indicators). This structured data object can be directly written to a database, used to update an Excel file, or provided to downstream analysis systems, thus completing the fully automated extraction from unstructured PDF documents to structured financial data (target financial data).

[0085] In summary, the data extraction method provided in this application includes: receiving a securities code and a financial statement business classification, and obtaining the corresponding financial report document from at least one data source based on the securities code; performing keyword positioning on each page of the financial report document, and using pages containing keywords as candidate pages to generate a candidate page set; for each candidate page in the candidate page set, expanding the page based on the adjacent pages of the candidate page according to a predetermined rule, and merging the expanded pages into a sub-document; converting the sub-document into an intermediate representation, and extracting candidate numerical evidence corresponding to the target financial indicator from the intermediate representation; and verifying and fusing the candidate numerical evidence to output the target financial data. This application embodiment first performs pagination-level keyword positioning on the financial report document using the received securities code and financial statement business classification, quickly filtering out a set of candidate pages containing relevant keywords, thereby significantly reducing the scope of the document to be processed from hundreds of pages to at least a few pages. Next, by expanding each candidate page based on its adjacent pages and merging the expanded pages into sub-documents, the integrity of cross-page tables or related contexts is ensured, avoiding repeated positioning or parsing failures due to information fragmentation. Then, the sub-documents are converted into intermediate representations, and candidate numerical evidence is extracted based on these intermediate representations, enabling the data extraction process to be based on structured and standardized information. Finally, the candidate numerical evidence is verified and fused to output the target financial data, ensuring the reliability and accuracy of the output data. The entire process of this application embodiment is a fully automated processing flow. Through layer-by-layer focusing, structured processing, and strategic parsing, it achieves improved processing efficiency and a systematic reduction in computational resource consumption. In other words, this application embodiment can improve the efficiency and accuracy of financial report data extraction and reduce costs.

[0086] To facilitate better implementation of the data extraction method provided in the embodiments of this application, the embodiments of this application also provide a data extraction device. The meanings of the terms used are the same as in the data extraction method described above, and specific implementation details can be found in the descriptions in the method embodiments.

[0087] Please see Figure 4 , Figure 4 This is a schematic diagram of the data extraction device provided in an embodiment of this application. The data extraction device may include a receiving unit 201, a positioning unit 202, an expansion unit 203, an extraction unit 204, and an output unit 205. The receiving unit 201 is used to receive securities codes and financial statement business classifications, and to obtain the corresponding financial report documents from at least one data source based on the securities codes; The positioning unit 202 is used to locate keywords in each page of the financial report document and use the pages containing the keywords as candidate pages to generate a set of candidate pages; The expansion unit 203 is used to expand each candidate page in the candidate page set according to a predetermined rule based on the adjacent pages of the candidate page and merge the expanded pages into a sub-document; Extraction unit 204 is used to convert sub-documents into intermediate representations and extract candidate numerical evidence corresponding to the target financial indicators based on the intermediate representations; Output unit 205 is used to verify and fuse candidate numerical evidence to output target financial data.

[0088] For specific implementation methods of each of the above units, please refer to the embodiments of the data extraction method described above, which will not be repeated here.

[0089] In summary, the data extraction apparatus provided in this application embodiment can receive securities codes and financial statement business classifications through the receiving unit 201, and obtain corresponding financial report documents from at least one data source based on the securities codes; the positioning unit 202 performs keyword positioning on each page of the financial report document, and uses pages containing keywords as candidate pages to generate a candidate page set; the expansion unit 203 expands each candidate page in the candidate page set according to predetermined rules based on the adjacent pages of the candidate page, and merges the expanded pages into a sub-document; the extraction unit 204 converts the sub-document into an intermediate representation, and extracts candidate numerical evidence corresponding to the target financial indicators based on the intermediate representation; the output unit 205 verifies and fuses the candidate numerical evidence to output the target financial data. This application embodiment can improve the efficiency and accuracy of financial report data extraction, and reduce costs.

[0090] Figure 4 This application also provides an electronic device that may integrate the data extraction device described in this application, such as... Figure 4 As shown, it illustrates a structural schematic diagram of the electronic device involved in the embodiments of this application, specifically: The electronic device may include components such as a processor 301 with one or more processing cores and a memory 302 with one or more computer-readable storage media. Those skilled in the art will understand that... Figure 4 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein: The processor 301 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs stored in the memory 302 and / or this application, and by calling data stored in the memory 302, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. Optionally, the processor 301 may include one or more processing cores; preferably, the processor 301 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operation of the storage medium, user interface, and application programs, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 301.

[0091] The memory 302 can be used to store software programs and the data extraction method provided in this application. The processor 301 executes various functional applications and data processing by running the software programs stored in the memory 302 and the data extraction method of this application. The memory 302 may mainly include a program storage area and a data storage area. The program storage area may store applications required for operating the storage medium and at least one function; the data storage area may store data created according to the use of the electronic device. In addition, the memory 302 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 302 may also include a memory controller to provide the processor 301 with access to the memory 302.

[0092] Although not shown, the electronic device may also include a display unit, an input unit, and a power supply, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 301 in the electronic device loads the executable files corresponding to the processes of one or more application programs into the memory 302 according to the following instructions, and the processor 301 runs the application programs stored in the memory 302 to realize various functions, as follows: Receive securities codes and financial statement business categories, and retrieve the corresponding financial report documents from at least one data source based on the securities code; Keyword targeting is performed on each page of the financial report document, and pages containing keywords are selected as candidate pages to generate a candidate page set; For each candidate page in the candidate page set, expand the adjacent pages of the candidate page according to a predetermined rule and merge the expanded pages into a sub-document; The sub-documents are converted into intermediate representations, and candidate numerical evidence corresponding to the target financial indicators is extracted based on the intermediate representations. The candidate numerical evidence is verified and fused to output the target financial data.

[0093] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0094] Therefore, embodiments of this application provide a storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the methods provided in embodiments of this application. For example, the instructions can execute the following steps: Receive securities codes and financial statement business categories, and retrieve the corresponding financial report documents from at least one data source based on the securities code; Keyword targeting is performed on each page of the financial report document, and pages containing keywords are selected as candidate pages to generate a candidate page set; For each candidate page in the candidate page set, expand the adjacent pages of the candidate page according to a predetermined rule and merge the expanded pages into a sub-document; The sub-documents are converted into intermediate representations, and candidate numerical evidence corresponding to the target financial indicators is extracted based on the intermediate representations. The candidate numerical evidence is verified and fused to output the target financial data.

[0095] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0096] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0097] Since the instructions stored in the storage medium can execute the steps of any method provided in the embodiments of this application, the beneficial effects that any method provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0098] The data extraction method, apparatus, storage medium, and electronic device provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A data extraction method, characterized by, include: Receive securities codes and financial statement business categories, and retrieve the corresponding financial report documents from at least one data source based on the securities codes; Keyword location is performed on each page of the financial report document, and pages containing keywords are selected as candidate pages to generate a candidate page set; For each candidate page in the candidate page set, the page is expanded according to a predetermined rule based on the adjacent pages of the candidate page, and the expanded pages are merged into a sub-document; The sub-document is converted into an intermediate representation, and candidate numerical evidence corresponding to the target financial indicator is extracted based on the intermediate representation; The candidate numerical evidence is verified and fused to output the target financial data.

2. The data extraction method of claim 1, wherein, The step of locating keywords for each page of the financial report document and using pages containing those keywords as candidate pages to generate a candidate page set includes: Extract the text content of each page of the financial report document, and detect the category keywords corresponding to the financial report business category and the indicator keywords corresponding to the target financial indicators based on the text content; When a page contains the category keyword and at least one indicator keyword, the page is identified as a candidate page. All candidate pages are aggregated to form a candidate page set.

3. The data extraction method of claim 1, wherein, The step of expanding adjacent pages based on the candidate page according to predetermined rules and merging the expanded pages into a sub-document includes: For each candidate page, a continuity test is performed to determine whether the table structure and / or text theme of the candidate page have continuity in adjacent pages; When there is continuity, the adjacent page and the candidate page are merged to form an expanded page; Merge all the expanded pages into a subdocument.

4. The data extraction method as described in claim 1, characterized in that, The step of converting the sub-document into an intermediate representation includes: The page content of the sub-document is structured and parsed to identify and annotate table areas and text paragraph areas; Reconstruct the row and column structure based on the cell layout of the table area to generate a table element containing row and column coordinates and cell content; Based on the text paragraph region, paragraph elements containing semantic block divisions are generated; The intermediate representation is composed of the table elements and paragraph elements.

5. The data extraction method as described in claim 1, characterized in that, The extraction of candidate numerical evidence corresponding to the target financial indicator based on the intermediate representation includes: The intermediate representation is parsed using a rule-based parser to obtain the first candidate evidence; If the first candidate evidence does not meet the preset reliability requirement or preset quantity requirement, then a prompt message is constructed, and a large language model is called to perform semantic parsing on the prompt message to obtain the second candidate evidence; Candidate numerical evidence is constituted by the first candidate evidence and / or the second candidate evidence.

6. The data extraction method as described in claim 1, characterized in that, The process of verifying and fusing the candidate numerical evidence to output target financial data includes: The candidate numerical evidence is cleaned and normalized to obtain standardized values. Based on preset confidence rules, multiple standardized values ​​for the same target financial indicator are fused to obtain target fused data; The target fused data is converted into target financial data conforming to a predefined structured data object format and then output.

7. The data extraction method as described in claim 6, characterized in that, The method, based on a preset confidence rule, fuses multiple standardized values ​​for the same target financial indicator to obtain target fused data, including: Based on the evidence source type corresponding to each standardized value, a corresponding confidence weight is assigned to the standardized value. When the evidence source type is a rule parser, the confidence weight assigned to the standardized value is a first weight. When the evidence source type is a large language model, the confidence weight assigned to the standardized value is a second weight. The first weight is greater than the second weight. Based on the confidence weights, the multiple standardized values ​​are weighted and summed, and the result is used as the target fused data.

8. A data extraction device, characterized in that, include: The receiving unit is used to receive securities codes and financial statement business classifications, and to obtain the corresponding financial report documents from at least one data source based on the securities codes; The positioning unit is used to locate keywords in each page of the financial report document and select pages containing keywords as candidate pages to generate a set of candidate pages. An expansion unit is used to expand each candidate page in the candidate page set according to a predetermined rule based on the adjacent pages of the candidate page and merge the expanded pages into a sub-document; An extraction unit is used to convert the sub-document into an intermediate representation and extract candidate numerical evidence corresponding to the target financial indicator based on the intermediate representation. The output unit is used to verify and fuse the candidate numerical evidence to output the target financial data.

9. A storage medium, characterized in that, The storage medium stores multiple instructions, which are adapted for loading by a processor to execute the data extraction method according to any one of claims 1-7.

10. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the data extraction method as described in any one of claims 1-7.

Images