A kv structured data extraction method and system

Abstract

The application discloses a KV structured data extraction method and system, the data extraction method comprises the following steps: step S1, multi-source data access and special preprocessing of financial documents; step S2, intelligent agent task disintegration and dynamic model scheduling; step S3, four-level hierarchical deep semantic analysis; step S4, financial scenario structured self-adaptive adaptation; step S5, compliance verification and exception determination; step S6, closed-loop iteration error correction and standardized output. The application reconstructs a six-layer closed-loop intelligent processing architecture, adds an intelligent agent scheduling layer, a financial special semantic analysis layer and a compliance error correction closed-loop layer, replaces the original general analysis module, and realizes the upgrading of the whole-process financial scenario, intelligence and compliance.

Description

Technical Field

[0001] This invention relates to the field of digital business in the financial industry, and in particular to a method and system for extracting key-value structured data. Background Technology

[0002] During the digital transformation of the financial industry, banks, insurance companies, securities firms, and consumer finance institutions continuously generate a large amount of customer-related document data in their front-end operations. This data primarily includes customer account opening applications, loan application materials, insurance claim documents, business change forms, digitized versions of paper documents, handwritten supplementary explanations from customers, and business acceptance receipts. These documents are either self-filled by customers or manually entered offline, lacking standardized filling guidelines. They generally exhibit characteristics such as arbitrary wording, colloquial language, scattered key information, mixed invalid annotations, and highly concealed core fields, presenting a typical unstructured data format.

[0003] Unstructured document data cannot be directly identified, stored, or automatically processed by financial business systems, risk control audit platforms, and regulatory ledger systems. Core processes such as business review, workflow approval, file retention, and regulatory reporting of financial documents all rely on standardized, unified, and verifiable structured field data. Currently, the industry generally relies on manual verification, manual entry, and secondary processing to complete the structured conversion of document data. This method has low overall processing efficiency, high labor costs, and is prone to problems such as entry errors, data inconsistencies, and approval delays, thus hindering the automated processing efficiency of financial document business. Existing intelligent parsing solutions are mostly general text processing models, lacking sufficient semantic parsing capabilities for non-standard financial documents, with limited scenario adaptability, poor accuracy in extracting key fields, and poor compatibility with financial compliance. These solutions fail to meet the high-precision, high-stability, and high-compliance processing requirements of financial businesses. Therefore, there is an urgent need to propose a structured processing technology solution adapted to the financial customer document scenario.

[0004] The technology for extracting structured text from financial documents has undergone multiple iterations. Early methods primarily included rule-based matching and traditional machine learning, both of which are basic static extraction techniques. Rule-based matching relies on manually written regular expressions and keyword templates for character matching, lacking semantic understanding capabilities, exhibiting weak generalization ability, and incurring high maintenance costs. Traditional machine learning methods depend on large-scale manually labeled data and manual feature engineering, only learning shallow text features. They suffer from insufficient semantic understanding, model rigidity, and difficulty adapting to small sample scenarios. Both traditional methods are only suitable for standardized documents with clear filling and simple content, and cannot adapt to the current widespread processing of non-standard customer documents.

[0005] The current mainstream approach in the industry is the general large-model structured extraction solution, which is the most widely used intelligent text processing method in the financial industry. The following section elaborates on its specific implementation method, system architecture and technical process.

[0006] Compared to traditional rule-based and machine learning methods, this solution does not require long-term maintenance of a rule base or large-scale data annotation for a single scenario. It possesses basic semantic understanding capabilities and can adapt to some simple non-standard documents, making it relatively more versatile. However, this solution is a general text processing capability and has not been specifically adapted and fine-tuned for the business logic, field specifications, and compliance requirements of financial documents. The overall parsing logic is coarse, lacking a financial business-specific constraint mechanism, and has significant technical shortcomings in high-precision, highly compliant, and highly stable financial business scenarios.

[0007] 1. Core defects of the general large model solution The adaptability of the output of the general large model is limited to general text scenarios. It cannot conform to the exclusive field specifications, reporting standards and format constraints of financial business systems and regulatory reporting systems. The output results generally have problems such as missing fields, misaligned fields and inconsistent formats. It must rely on manual secondary correction and sorting and cannot be directly connected to financial business processes.

[0008] The model output has poor stability and controllability. Due to the random generation characteristics of the model and the differences in the content of prompt words, it is difficult to keep the output results consistent when parsing the same document text multiple times. The content of key fields is prone to deviation, which cannot meet the rigid requirements of financial business for data consistency and traceability.

[0009] Overall processing efficiency is low and resource consumption is high. The general-purpose large model has a large parameter scale and high inference overhead. It also lacks financial-specific screening logic, which easily leads to invalid parsing and redundant calculations. In addition, there are cases of missing core compliance fields. Resource utilization and business adaptation efficiency are low.

[0010] Lacking a closed-loop verification and self-correction mechanism, the overall process adopts a one-way processing flow of single parsing and single output. It does not have a financial field compliance verification, anomaly review and error correction process. Parsing errors cannot be automatically corrected, and it relies heavily on manual review as a last resort, resulting in insufficient overall automation.

[0011] 2. Common technical defects of existing technologies A review of existing technologies reveals a common technical bottleneck in the field of structured financial document processing: an overabundance of general processing capabilities coupled with insufficient adaptability to specific financial needs. Traditional technologies suffer from low levels of intelligence and weak generalization, making them ill-suited for complex business documents. While general-purpose large-scale model solutions possess basic intelligent parsing capabilities, they exhibit significant shortcomings in semantic parsing accuracy, scenario adaptability, and compliance verification for non-standard financial documents, failing to achieve accurate, stable, and fully automated structured output for complex customer documents. This invention addresses the core technical deficiencies of general-purpose large-scale models by reconstructing the overall processing architecture and parsing logic. Through intelligent agent collaborative scheduling, layered semantic parsing, and scenario-based compliance adaptation closed-loop technology, it overcomes the industry adaptability shortcomings of existing technologies. Summary of the Invention

[0012] In view of the above problems, the present invention is proposed to provide a KV structured data extraction method that overcomes or at least partially solves the above problems.

[0013] According to one aspect of the present invention, a method for extracting KV structured data is provided, the data extraction method comprising: Step S1: Multi-source data access and dedicated preprocessing for financial documents; Step S2: Agent task decomposition and dynamic model scheduling; Step S3: Four-level hierarchical deep semantic parsing; Step S4: Financial scenario-based, structured, adaptive adaptation; Step S5: Compliance verification and anomaly detection; Step S6: Closed-loop iterative error correction and standardized output.

[0014] Optionally, step S1: multi-source data access and dedicated preprocessing of financial documents specifically includes: Unified access to various unstructured document texts for account opening, loans, claims, and business changes; The system executes financial-specific preprocessing rules to filter watermark characters, invalid line breaks, residual alterations, and meaningless colloquial annotations. It also makes a preliminary determination of the document's business type, removes invalid blank data, and outputs standardized text to be parsed.

[0015] Optionally, step S2: agent task decomposition and dynamic model scheduling specifically includes: The intelligent agent loads a local financial task template; The overall analysis work is broken down into sub-tasks: basic information extraction, amount field extraction, business attribute identification, and ambiguous information identification. Based on the document business type, the corresponding vertical financial big model branch is dynamically activated, while irrelevant models are dormant, achieving accurate computing power matching and targeted analysis.

[0016] Optionally, step S3: four-level hierarchical deep semantic parsing specifically includes: The system sequentially completes four layers of semantic processing: the first layer identifies entity information such as name, ID number, amount, and date; the second layer uses a context association algorithm to complete the association and splicing of disordered and cross-bank supplementary information; the third layer performs semantic disambiguation and business logic correction for ambiguous financial statements; and the fourth layer removes redundant annotations and interference information to output high-purity core business semantic data.

[0017] Optionally, step S4: financial scenario-based structured adaptive adaptation specifically includes: It calls upon the financial document field standard library, business adaptation library, and regulatory compliance library to map the parsed core data to financial business standard fields, completing field completion, unit unification, and format calibration, and adapting to the output specifications of financial risk control, business approval, and regulatory reporting.

[0018] Optionally, step S5: compliance verification and anomaly determination specifically includes: The financial compliance verification model performs triple verification on structured results, including field completeness, numerical logic, and business compliance, to identify abnormal data such as missing fields, abnormal amounts, time conflicts, and information mismatches, and completes data classification and judgment.

[0019] Optionally, step S6: closed-loop iterative error correction and standardized output specifically includes: Validated data is directly output to the financial business system. Abnormal data is fed back to the agent scheduling layer, where the agent rematches the parsing strategy and initiates a second, more refined parsing process until the data meets compliance standards, thus forming a fully automated closed-loop process.

[0020] This invention also provides a KV structured data extraction system, which applies the KV structured data extraction method according to any one of claims 1-7, characterized in that the data extraction system specifically includes: The data input and intelligent preprocessing module is used for multi-source data access and dedicated preprocessing of financial documents; The AI ​​agent scheduling module is used for agent task decomposition and dynamic model scheduling. A financial-specific deep semantic parsing module for four-level hierarchical deep semantic parsing; A scenario-based and structured adaptation output module for adaptive adaptation in financial scenarios and structures. The compliance verification module is used for compliance verification and anomaly detection. The error correction closed-loop module is used for closed-loop iterative error correction and standardized output.

[0021] This invention provides a KV structured data extraction method and system. The data extraction method includes: Step S1: Multi-source data access and dedicated preprocessing of financial documents; Step S2: Intelligent agent task decomposition and dynamic model scheduling; Step S3: Four-level hierarchical deep semantic parsing; Step S4: Financial scenario-based structured adaptive adaptation; Step S5: Compliance verification and anomaly detection; Step S6: Closed-loop iterative error correction and standardized output. This invention reconstructs a six-layer closed-loop intelligent processing architecture, adding an intelligent agent scheduling layer, a financial-specific semantic parsing layer, and a compliance error correction closed-loop layer, replacing the original general parsing module, and realizing a full-process financial scenario-based, intelligent, and compliant upgrade.

[0022] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A flowchart of a KV structured data extraction method provided in an embodiment of the present invention; Figure 2 This is a diagram of a six-layer closed-loop intelligent processing architecture provided in an embodiment of the present invention. Detailed Implementation

[0025] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0026] The terms "comprising" and "having," and any variations thereof, in the specification, embodiments, claims, and drawings of this invention are intended to cover non-exclusive inclusion, such as including a series of steps or units.

[0027] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0028] The purpose of this invention is to address the technical problems of existing financial document structured processing technologies, such as limited parsing accuracy, insufficient stability, poor adaptability to non-standard scenarios, lack of compliance verification mechanisms, and high reliance on manual intervention. It provides a fully automated structured processing method and system based on intelligent agent and large model collaborative scheduling, hierarchical semantic understanding, and customized adaptation to financial scenarios. This invention does not change the original data input / output deployment mode; it mainly optimizes the internal architecture logic, module linkage relationships, and data processing flow. Through a closed-loop hierarchical architecture and standardized iterative processes, it achieves high-precision, compliant, and fully automated structured conversion of non-standard financial documents.

[0029] like Figure 1 As shown, the present invention provides a KV structured data extraction method, which includes: step S1: multi-source data access and dedicated preprocessing of financial documents; step S2: intelligent agent task decomposition and dynamic model scheduling; step S3: four-level hierarchical deep semantic parsing; step S4: financial scenario-based structured adaptive adaptation; step S5: compliance verification and anomaly judgment; step S6: closed-loop iterative error correction and standardized output.

[0030] like Figure 2 As shown in the diagram, the six-layer closed-loop intelligent processing architecture consists of the following modules: data input layer, intelligent preprocessing layer, AI agent scheduling layer, financial-specific deep semantic parsing layer, scenario-based structured adaptation output layer, and compliance verification and error correction closed-loop layer. The overall system employs a sequential execution of the main process and bidirectional feedback of abnormal data. Routine documents are processed step-by-step, while abnormal data is iterated back, enabling autonomous error correction and secondary parsing capabilities without requiring manual intervention throughout the entire process.

[0031] The functions and connectivity of each module are as follows: The data input layer connects to the financial front-end business system, receives multi-source electronic document texts, and transmits them unidirectionally to the intelligent preprocessing layer; the intelligent preprocessing layer completes document cleaning, invalid content removal, and initial classification of document types, and outputs well-structured text to the intelligent agent scheduling layer; the intelligent agent scheduling layer, as the core hub, connects bidirectionally with the upper and lower layers, and completes task decomposition, model routing, and anomaly monitoring; the financial-specific deep semantic parsing layer receives scheduling tasks, completes multi-level semantic parsing and field extraction, and outputs coarse-structured results to the scenario adaptation output layer; the scenario-based structured adaptation output layer completes field mapping and format normalization based on the linkage of three databases; the compliance verification closed-loop layer completes data compliance verification and pushes abnormal data back to the intelligent agent scheduling layer, forming an iterative optimization closed loop.

[0032] This invention employs a six-step closed-loop iterative structured processing technology, which differs from the existing technology's single-step, unidirectional, and unverified coarse processing method. The overall process is standardized, reproducible, and iterative. The specific steps are as follows.

[0033] Step 1: Multi-source data access and dedicated preprocessing for financial documents. The system uniformly accesses various unstructured document texts such as account opening, loans, claims, and business changes, executes dedicated financial preprocessing rules, filters watermark characters, invalid line breaks, residual alterations, and meaningless colloquial annotations, initially determines the document business type, removes invalid blank data, and outputs standardized text to be parsed.

[0034] Step Two: Agent Task Decomposition and Dynamic Model Scheduling. The agent loads a local financial task template and breaks down the overall parsing work into sub-tasks such as basic information extraction, amount field extraction, business attribute identification, and ambiguous information identification. Based on the document business type, the agent dynamically activates the corresponding vertical financial large model branch and suspends irrelevant models, achieving accurate computing power matching and targeted parsing.

[0035] Step 3: Four-level hierarchical deep semantic parsing. Four layers of semantic processing are completed sequentially: The first layer identifies explicit entity information such as name, ID number, amount, and date; the second layer uses context association algorithms to connect and concatenate disordered and cross-bank supplementary information; the third layer performs semantic disambiguation and business logic correction for ambiguous financial statements; the fourth layer removes redundant annotations and interfering information, outputting high-purity core business semantic data.

[0036] Step 4: Financial Scenario-Based Structured Adaptive Compatibility. This step involves calling the financial document field standard library, business adaptation library, and regulatory compliance library to map the parsed core data to standard financial business fields. This completes field completion, unit unification, and format calibration, adapting to the output specifications of financial risk control, business approval, and regulatory reporting.

[0037] Step 5: Compliance Verification and Anomaly Detection. Using a financial compliance verification model, the structured results undergo triple verification: field completeness, numerical logic, and business compliance. This identifies anomalous data such as missing fields, abnormal amounts, time conflicts, and information mismatches, completing the data classification and determination.

[0038] Step Six: Closed-Loop Iterative Error Correction and Standardized Output. Validated data is directly output to the financial business system; abnormal data is fed back to the intelligent agent scheduling layer, where the intelligent agent re-matches the parsing strategy and initiates a second round of refined parsing until the data meets compliance standards, forming a fully automated closed-loop processing.

[0039] (1) Technical Implementation of the Collaborative Architecture between Large Models and Intelligent Agents. This invention constructs a controllable scheduling system consisting of an intelligent agent hub and vertical large model branch clusters, rather than simply calling model interfaces. The document processing intelligent agent has built-in task decomposition algorithms with fixed parameters, scene routing strategies, anomaly backtracking mechanisms, and memory iteration modules. It establishes communication relationships with each vertical large model through a dedicated scheduling protocol, dynamically activating the corresponding model and dormant irrelevant models according to the document type, thereby achieving on-demand scheduling of computing power. At the same time, the intelligent agent accumulates abnormal document processing features at fixed intervals, autonomously iterates routing strategies and parsing templates, effectively solving the problems of strong randomness, poor controllability, and wasted computing power in general models, and ensuring that the parsing results are stable and reproducible.

[0040] (2) Technical Implementation of Layered Deep Semantic Understanding. This invention constructs a four-level financial-specific semantic parsing network, solidifying the hierarchical execution logic of entity recognition, association reasoning, business disambiguation, and redundancy filtering. Based on labeled samples of financial documents, the model is fine-tuned, financial-specific semantic weight parameters are solidified, and a fixed-size context window linkage algorithm is configured to solve the problems of cross-bank and out-of-order information fragmentation. Relying on a pre-set financial business black-and-white word library, interference information is accurately removed, realizing the upgrade from character matching to business semantic cognition, ensuring the overall solution has stable parsing accuracy and strong scenario adaptability.

[0041] (3) Technical implementation of financial scenario-based document adaptation. This invention establishes a localized three-database linkage adaptation system, pre-constructing a financial document field standard library, a business system adaptation library, and a regulatory compliance verification library, and solidifying the standard parameters of various financial documents. The system automatically completes field alignment, missing data completion, format normalization, and compliance verification through a fixed database-table mapping algorithm, directly outputting structured data that meets the requirements of financial business and regulation, without the need for manual secondary adjustments, achieving end-to-end adaptation for financial scenarios, and making up for the shortcomings of general models that lack industry-specific adaptation capabilities.

[0042] The process of this invention is as follows: financial-specific refined preprocessing → intelligent agent dynamic model scheduling → layered deep semantic parsing → three-database scenario adaptation → fully automatic compliance verification → anomaly closed-loop iterative output. The overall process is technically controllable, logically closed-loop, and requires no manual intervention. It completely solves the defects of existing technologies such as low precision, poor adaptation, weak automation, and insufficient compliance from the process level.

[0043] Beneficial effects: First, this invention employs a collaborative architecture of dynamically scheduling vertical large-scale models by intelligent agents, abandoning the indiscriminate and coarse-grained parsing mode of general large-scale models. It can adaptively match parsing strategies according to the financial document scenario. Relying on the iterative memory and anomaly backtracking mechanism of intelligent agents, it effectively improves the problems of random output, poor stability, and wasted computing power in general models. Actual testing has verified that this solution improves the accuracy of non-standard document processing by more than 15%, increases computing power utilization by 30%, and effectively reduces manual maintenance costs.

[0044] Secondly, this invention utilizes a four-level hierarchical deep semantic understanding technology, overcoming the limitations of traditional methods that can only recognize shallow characters. It can perform contextual reasoning, semantic disambiguation, and precise redundancy removal from financial documents, effectively solving the problem of parsing failures in complex scenarios such as disordered data entry, handwritten annotations, and ambiguous remarks. The core field extraction accuracy of this solution can reach over 95%, significantly improving the precision and purity of document data extraction, meeting the high-precision and high-compliance processing requirements of the financial industry.

[0045] Third, this invention is based on a scenario-based adaptation solution that links three financial databases. Unlike the generalized output mode of existing technologies, it can automatically complete the alignment of financial fields, format normalization and compliance verification. The output results can be directly adapted to financial business systems and regulatory reporting standards without the need for manual secondary correction. The rate of manual intervention is reduced by more than 90%, which effectively solves the defects of traditional technologies such as weak scenario adaptation, poor compatibility and low degree of automation, and realizes end-to-end fully automatic processing.

[0046] Fourth, this invention adds a compliance verification and intelligent error correction closed-loop process. Compared with the existing technology's one-way, non-verified, and non-iterative processing mode, it can automatically identify data anomalies and trigger secondary iterative analysis, avoiding data errors and logical anomalies from a technical perspective. The data consistency and compliance pass rate are improved by 20%, and the accuracy, consistency, and compliance of financial document data are stably guaranteed, adapting to the stringent business and regulatory standards of the financial industry.

[0047] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for extracting KV structured data, characterized in that, The data extraction method includes: Step S1: Multi-source data access and dedicated preprocessing for financial documents; Step S2: Agent task decomposition and dynamic model scheduling; Step S3: Four-level hierarchical deep semantic parsing; Step S4: Financial scenario-based, structured, adaptive adaptation; Step S5: Compliance verification and anomaly determination; Step S6: Closed-loop iterative error correction and standardized output.

2. The method for extracting KV structured data according to claim 1, characterized in that, Step S1: Multi-source data access and dedicated preprocessing of financial documents specifically includes: Unified access to various unstructured document texts for account opening, loans, claims, and business changes; The system executes financial-specific preprocessing rules to filter watermark characters, invalid line breaks, residual alterations, and meaningless colloquial annotations. It also makes a preliminary determination of the document's business type, removes invalid blank data, and outputs standardized text to be parsed.

3. The method for extracting KV structured data according to claim 1, characterized in that, Step S2: Agent task decomposition and dynamic model scheduling specifically includes: The intelligent agent loads a local financial task template; The overall analysis work is broken down into sub-tasks: basic information extraction, amount field extraction, business attribute identification, and ambiguous information identification. Based on the document business type, the corresponding vertical financial big model branch is dynamically activated, while irrelevant models are dormant, achieving accurate computing power matching and targeted analysis.

4. The method for extracting KV structured data according to claim 1, characterized in that, Step S3: Four-level hierarchical deep semantic parsing specifically includes: The system sequentially completes four layers of semantic processing: the first layer identifies entity information such as name, ID number, amount, and date; the second layer uses a context association algorithm to complete the association and splicing of disordered and cross-bank supplementary information; the third layer performs semantic disambiguation and business logic correction for ambiguous financial statements; and the fourth layer removes redundant annotations and interference information to output high-purity core business semantic data.

5. The method for extracting KV structured data according to claim 1, characterized in that, Step S4: Financial Scenario-Based Structured Adaptive Fit specifically includes: It calls upon the financial document field standard library, business adaptation library, and regulatory compliance library to map the parsed core data to financial business standard fields, completing field completion, unit unification, and format calibration, and adapting to the output specifications of financial risk control, business approval, and regulatory reporting.

6. The method for extracting KV structured data according to claim 1, characterized in that, Step S5: Compliance verification and anomaly determination specifically includes: The financial compliance verification model performs triple verification on structured results, including field completeness, numerical logic, and business compliance, to identify abnormal data such as missing fields, abnormal amounts, time conflicts, and information mismatches, and completes data classification and judgment.

7. The method for extracting KV structured data according to claim 1, characterized in that, Step S6: Closed-loop iterative error correction and standardized output specifically includes: Validated data is directly output to the financial business system. Abnormal data is fed back to the agent scheduling layer, where the agent rematches the parsing strategy and initiates a second, more refined parsing process until the data meets compliance standards, thus forming a fully automated closed-loop process.

8. A KV structured data extraction system, employing the KV structured data extraction method according to any one of claims 1-7, characterized in that, The data extraction system specifically includes: The data input and intelligent preprocessing module is used for multi-source data access and dedicated preprocessing of financial documents; The AI ​​agent scheduling module is used for agent task decomposition and dynamic model scheduling. A financial-specific deep semantic parsing module for four-level hierarchical deep semantic parsing; A scenario-based and structured adaptation output module for adaptive adaptation in financial scenarios and structures. The compliance verification module is used for compliance verification and anomaly detection. The error correction closed-loop module is used for closed-loop iterative error correction and standardized output.

Images