Financial data intelligent entry and verification method

Abstract

The invention discloses a financial data intelligent entry and verification method, which comprises the following steps: acquiring an original document image, and processing the original document image by adopting an image enhancement algorithm to obtain an optimized document image; extracting a structured field from the optimized document image, determining field coordinates, and generating a structured data table; performing semantic analysis on the non-standardized text in the structured data table to generate semantic vector representation; identifying missing fields according to the semantic vector representation, querying a historical database, and generating preliminary filling data; comparing the preliminary filling data with historical data through a cross validation mechanism to obtain a validated field value; processing the verified field value by adopting a classification model, and determining a document type; arranging the verified field value according to the document type, and generating a standardized data record; key indexes are extracted from the standardized data records, the index logic relation is verified, and corrected data are obtained.

Description

Technical Field

[0001] The present invention relates to the financial field, and in particular to a method for intelligent entry and verification of financial data. Background Art

[0002] Financial data processing is an indispensable core area of the modern financial industry, directly impacting the operational efficiency and decision-making quality of financial institutions. With the acceleration of digital transformation, automated data entry and verification technologies have become key to improving data processing capabilities. However, traditional data processing methods have significant limitations. Many existing solutions rely on manual data entry or semi-automated tools, which are prone to data inaccuracies due to human error or inconsistent formats. Furthermore, some automated systems lack recognition accuracy when processing unstructured or complex financial documents, making them difficult to cope with diverse document types and situations with missing information. These shortcomings reduce the reliability and efficiency of data processing and hinder the development of intelligent data processing in the financial industry.

[0003] In the field of intelligent financial data entry and verification, the core challenges lie in several key technical factors. First, optical character recognition technology is prone to errors in extracting key information when faced with documents with complex formats or poor scan quality. Second, natural language processing technology struggles to accurately understand semantics when parsing non-standardized text descriptions, resulting in incomplete or biased information extraction. Finally, pattern matching technology lacks sufficient intelligent inference capabilities when dealing with data-missing scenarios, making it impossible to effectively utilize historical data for supplemental processing. These technical challenges make it difficult for systems to achieve efficient automated processing while ensuring high accuracy, especially when dealing with diverse financial documents.

[0004] Therefore, how to use optical character recognition, natural language processing and pattern matching technologies in complex financial documents to accurately extract key information and intelligently fill in missing fields, while ensuring data accuracy through cross-validation of historical data, has become a key issue in the development of intelligent financial data entry and verification platforms. Summary of the Invention

[0005] The purpose of the present invention is to solve the above problems and provide a method for intelligent entry and verification of financial data, which significantly improves the automation and reliability of financial data processing, effectively solves the problems of diversified document formats, unstructured text parsing and data missing, and provides strong support for improving the efficiency of financial business.

[0006] The technical solution adopted by the present invention to solve the technical problem is:

[0007] A method for intelligent entry and verification of financial data, comprising the following steps:

[0008] S101 acquires an original document image, and processes the original document image using an image enhancement algorithm to obtain an optimized document image;

[0009] S102 extracts structured fields from the optimized document image, determines field coordinates, and generates a structured data table;

[0010] S103 performs semantic analysis on the non-standardized text in the structured data table to generate a semantic vector representation;

[0011] S104 identifies missing fields based on the semantic vector representation, queries the historical database, and generates preliminary filling data;

[0012] S105 compares the preliminary filled data with historical data through a cross-validation mechanism to obtain a verified field value;

[0013] S106 uses a classification model to process the verified field value to determine the document type;

[0014] S107 sorts the verified field values according to the document type to generate standardized data records;

[0015] S108 extracts key indicators from the standardized data records, verifies the logical relationship of the indicators, and obtains corrected data.

[0016] Furthermore, step S101 includes the following steps:

[0017] Parsing metadata of the original document image to obtain a resolution value;

[0018] If the resolution value is lower than a preset threshold, a bilateral filtering algorithm is used to perform denoising on the original document image to obtain a first image;

[0019] Processing the first image by histogram equalization to adjust pixel brightness values to obtain a second image;

[0020] performing high-frequency detail enhancement on the second image based on a Laplacian operator to obtain a third image;

[0021] The third image is processed using an adaptive binarization method to generate a black and white document image.

[0022] Furthermore, step S102 includes the following steps:

[0023] Processing the optimized document image through a model based on the U-Net architecture to generate a binary mask of the table area;

[0024] Using the Sobel operator to calculate the gradient distribution of the binary mask and determine the table field bounding box;

[0025] Processing the bounding boxes using a non-maximum suppression algorithm to obtain a corrected bounding box set;

[0026] spatially aligning the corrected bounding box set with the optical character recognition result using affine transformation to obtain an aligned text field set;

[0027] The aligned text field set is processed by a bidirectional long short-term memory network to generate an inter-field association probability matrix and a structured data table.

[0028] Furthermore, step S103 includes the following steps:

[0029] Using a word segmentation tool to perform word segmentation processing on the non-standardized text to obtain a word segmentation result set;

[0030] Annotating the words in the word segmentation result set using a pre-established term dictionary to obtain annotated text;

[0031] Using a syntax analysis tool to perform dependency syntax analysis on the annotated text to construct a semantic structure tree;

[0032] If the node missing rate of the semantic structure tree exceeds a preset threshold, nodes are supplemented to obtain a supplemented semantic structure tree;

[0033] If the cosine similarity between the supplemented semantic structure tree and the standard template is lower than a preset threshold, a semantic vector representation is generated through a pre-trained language model.

[0034] Furthermore, step S104 includes the following steps:

[0035] Parsing the structured data table, marking missing fields, and obtaining a missing field set;

[0036] Use the pre-trained language model to generate semantic vector representations for the identified fields and obtain a set of field semantic vectors;

[0037] Constructing a query condition based on the context information of the missing field and converting it into a Boolean query statement;

[0038] Perform nearest neighbor search in the history database to obtain similar historical records;

[0039] Calculating the cosine similarity between the field semantic vector set and the historical records; if the similarity exceeds a preset threshold, extracting the corresponding field value and generating a candidate value set;

[0040] The candidate value set is weightedly summed according to weight calculation to generate preliminary filling data.

[0041] Furthermore, step S105 includes the following steps:

[0042] Using a word segmentation tool to perform word segmentation and part-of-speech tagging on the preliminary filled data to obtain a word segmentation and tagging result;

[0043] Perform entity recognition on the word segmentation and annotation results based on a preset rule library to obtain a set of recognized entities;

[0044] Mapping the identified entity set to corresponding fields according to a field mapping table;

[0045] For unmapped entities, calculate their similarity with the standard vocabulary. If the similarity exceeds the preset threshold, map them to the standard field value to obtain the mapped field value.

[0046] For entities that do not exceed the preset threshold, a list of correction suggestions is generated to obtain the verified field values.

[0047] Furthermore, step S106 includes the following steps:

[0048] Performing contrast enhancement on the verified field value to generate an enhanced image;

[0049] Locate the text area through edge detection and generate a binary image;

[0050] Extracting character sequence and coordinate information from the binary image;

[0051] Encoding the character sequence through a bidirectional long short-term memory network to obtain a contextual code;

[0052] Decoding the context code using a conditional random field to obtain a set of field boundaries and initial field value pairs;

[0053] According to the field co-occurrence matrix of the historical document collection, a candidate list of missing fields is determined to obtain a set of completed fields;

[0054] A random forest classifier is used to process the completed field set to obtain a document type.

[0055] Furthermore, step S107 includes the following steps:

[0056] Obtaining a field mapping rule matching the document type from a predefined template library;

[0057] Analyze the semantic label candidate set through a bidirectional long short-term memory network to obtain feature representation;

[0058] Calculate the contextual dependencies between fields by combining domain dictionary weights;

[0059] Reconstruct the context dependency relationship through a table structure tree to obtain a data hierarchy;

[0060] If there are missing fields in the data hierarchy, the fields are supplemented by checking the matching degree of historical data;

[0061] A differential algorithm is used to align the field mapping rules with the column order of the output template to generate standardized data records.

[0062] The beneficial effects of the present invention are:

[0063] 1. The present invention discloses a method for intelligent entry and verification of financial data, and proposes a comprehensive solution to the challenges faced by key technologies such as optical character recognition, natural language processing and pattern matching when processing complex financial documents. The present invention first optimizes low-resolution documents through an image enhancement algorithm, and then combines deep learning and edge detection to accurately extract structured fields. For non-standardized texts, a pre-trained language model is used to generate semantic vectors, and historical databases are used for intelligent completion. Data accuracy is ensured through a cross-validation mechanism and an adaptive classification model, and standardized financial data records are finally generated. The present invention significantly improves the degree of automation and reliability of financial data processing, effectively solves problems such as diversified document formats, unstructured text parsing and data missing, and provides strong support for improving the efficiency of financial business. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0065] Figure 1 Flowchart of the present invention;

[0066] Figure 2 This is a flow chart of original document image processing according to the present invention;

[0067] Figure 3 Initially populate data to generate a flow chart for the present invention;

[0068] Figure 4 The data flow diagram is modified for the present invention. DETAILED DESCRIPTION

[0069] like Figure 1 As shown, a method for intelligent entry and verification of financial data includes the following steps:

[0070] S101 acquires an original document image, and processes the original document image using an image enhancement algorithm to obtain an optimized document image;

[0071] S102 extracts structured fields from the optimized document image, determines field coordinates, and generates a structured data table;

[0072] S103 performs semantic analysis on the non-standardized text in the structured data table to generate a semantic vector representation;

[0073] S104 identifies missing fields based on the semantic vector representation, queries the historical database, and generates preliminary filling data;

[0074] S105 compares the preliminary filled data with historical data through a cross-validation mechanism to obtain a verified field value;

[0075] S106 uses a classification model to process the verified field value to determine the document type;

[0076] S107 sorts the verified field values according to the document type to generate standardized data records;

[0077] S108 extracts key indicators from the standardized data records, verifies the logical relationship of the indicators, and obtains corrected data.

[0078] like Figure 2 As shown, in step S101, initial text data is obtained. If the pixel resolution of the initial text data is lower than a preset threshold, an image enhancement algorithm is used to adjust the contrast and clarity to obtain an optimized document image.

[0079] Specifically, the process includes obtaining an original document image, parsing metadata of the original document image, and obtaining a DPI value of the original document image. If the DPI value is lower than a preset resolution threshold, a bilateral filtering algorithm is used to denoise the original document image to obtain a first image, wherein the first image retains edge features. The first image is processed by histogram equalization to redistribute the pixel brightness values of the first image to obtain a second image. The second image is enhanced for high-frequency details based on the Laplace operator to obtain a third image. The third image is processed by an adaptive binarization method to generate a black and white document image.

[0080] Original document images are typically acquired using high-resolution scanners. Consider a scanned image containing text and a signature. After analyzing the metadata, the DPI is found to be 150. The DPI value reflects the number of pixels per inch, so 150 DPI means that the detail may not be sufficient for fine text recognition.

[0081] If the preset resolution threshold is 300 DPI, 150 DPI is below the standard and requires denoising. Bilateral filtering is selected because it can smooth noise while preserving edge features. This process ensures that subsequent enhancement steps will not distort the image due to noise amplification.

[0082] The first image is processed through histogram equalization to generate the second image. Histogram equalization redistributes pixel brightness values to enhance image contrast. This method is particularly effective in low-contrast areas, directly improving text readability.

[0083] The second image is further enhanced using the Laplacian operator to create a third image with high-frequency detail. The Laplacian operator detects areas of image brightness with dramatic changes, such as text edges, and enhances these high-frequency areas to make the text outlines stand out. This enhancement avoids oversharpening and ensures a natural-looking image.

[0084] Adaptive binarization is used to generate the final black-and-white document image for the third image. Adaptive binarization dynamically determines the threshold based on local pixel characteristics, making it suitable for contract images with uneven lighting. For example, if the signature area in an image is low-brightness while the text area is brighter, adaptive binarization can optimize the thresholds separately, ensuring that both the signature and text appear in clear black and white. This method generates black-and-white images that are particularly suitable for OCR recognition, minimizing background interference.

[0085] Each step in the above process is focused on improving the quality of the document image. Bilateral filtering ensures a clean base image, histogram equalization improves contrast, the Laplacian operator enhances details, and adaptive binarization optimizes the final output.

[0086] In OCR testing of black and white images, text recognition accuracy was significantly improved, and signatures were completely preserved. This processing chain is not only applicable to contracts, but can also be extended to similar documents such as invoices and receipts, demonstrating the high versatility of the technology.

[0087] Denoising provides a clear input for equalization, which in turn provides uniform brightness for detail enhancement, which in turn provides sharp edges for binarization. This logical progression ensures a smooth transition from the original low-quality image to a high-quality black-and-white document, significantly improving the efficiency and reliability of document digitization.

[0088] S102 extracts structured fields from the optimized document image, determines field coordinates, and generates a structured data table;

[0089] An input image is obtained, the input image containing a table area. The input image is processed using a model based on the U-Net architecture to obtain a binary mask of the table area. The horizontal and vertical gradients of the binary mask are calculated using the Sobel operator to obtain a gradient magnitude distribution. A histogram is generated based on the gradient magnitude distribution, and peaks are extracted from the histogram to determine the table field bounding boxes. The table field bounding boxes are processed using a non-maximum suppression algorithm to obtain a set of corrected bounding boxes. The corrected set of bounding boxes is spatially aligned with the optical character recognition results using an affine transformation to obtain a set of aligned text fields. The aligned set of text fields is processed using a bidirectional long short-term memory network to obtain an inter-field association probability matrix. If key fields are missing from the aligned set of text fields, historical records are retrieved using a preset retrieval module using the table title as a query condition. A matrix dot product is performed on the inter-field association probability matrix and the historical records to obtain a dot product result. The dot product result is normalized to obtain a set of confidence scores. A structured data table is generated according to the corrected bounding box set, the aligned text field set, and the confidence score set.

[0090] When processing images of financial statements containing tables, the input images are typically acquired via scanner or camera and may contain complex table structures. Models based on the U-Net architecture excel at segmentation tasks. Their encoder-decoder structure extracts global features through downsampling and restores spatial details through upsampling. For example, after inputting a balance sheet image, the model analyzes the pixel distribution and generates a binary mask that marks the table area as white and the background as black. This mask clearly distinguishes between table and non-table areas, providing a clear basis for subsequent processing.

[0091] The Sobel operator is used to calculate the gradient of a binary mask. Using horizontal and vertical convolution kernels, the Sobel operator detects edge changes, such as the boundaries of table lines. Assuming the width of table lines in the mask is 5 pixels, the Sobel operator captures these sudden changes in brightness and generates a gradient magnitude distribution. A histogram is generated based on this distribution, with peaks corresponding to areas with dense lines. For example, closely spaced cell lines in a report appear as significant peaks in the histogram, helping to locate field bounding boxes.

[0092] The non-maximum suppression algorithm optimizes bounding box sets. In report images, cell bounding boxes may overlap due to noise. Non-maximum suppression compares the gradient strengths of overlapping boxes, retaining the strongest boxes and eliminating redundancy. For example, for the "Total Assets" field in a balance sheet, the algorithm ensures that each field corresponds to only one bounding box, reducing interference during subsequent alignment.

[0093] Affine transformations are used for spatial alignment. Report images may be distorted due to the shooting angle. Affine transformations align the bounding box with the text recognized by the OCR through rotation and translation. The OCR might recognize the text "Accounts Receivable" but offset it. Affine transformations correct its coordinates based on the bounding box, ensuring that the text is accurately placed in the corresponding cell.

[0094] A bidirectional long short-term memory network processes aligned text fields to capture semantic associations between them. For example, in a balance sheet, "current assets" and "non-current assets" have a hierarchical relationship. The network analyzes the context and generates an association probability matrix to identify the logical connections between the fields.

[0095] This matrix provides a semantic basis for subsequent structuring. For example, if a key field such as "net profit" is missing, historical records can be queried using the table header "Profit Statement." Assuming the historical records contain fields similar to a report, the retrieval module extracts relevant field templates. A matrix dot product combines the historical records with the association probability matrix to generate preliminary results. Normalization converts the dot product result into a confidence score.

[0096] In step S103 , the natural language processing technology is applied to analyze the semantic structure of the extracted non-standardized text. If the semantic similarity is lower than a preset threshold, a semantic vector representation is generated through a pre-trained language model.

[0097] Obtain non-standardized text, use the Jieba word segmentation tool to perform word segmentation on the non-standardized text, and obtain a word segmentation result set. Load a pre-established financial terminology dictionary, and annotate the professional vocabulary in the word segmentation result set through dictionary matching to obtain annotated text. Use StanfordParser to perform dependency syntax analysis on the annotated text to construct a semantic structure tree. Calculate the node missing rate of the semantic structure tree. If the node missing rate exceeds a preset threshold, trigger the node supplement mechanism to obtain a supplemented semantic structure tree. Calculate the cosine similarity between the supplemented semantic structure tree and the standard template. If the cosine similarity is lower than the preset threshold, generate a semantic vector through the BERT model to obtain a 768-dimensional semantic vector set. Fill in the missing nodes in the supplemented semantic structure tree according to the matching rules in the historical data to obtain structured data. Calculate the confidence through the accuracy of node matching, and mark the structured data with confidence to obtain structured data with confidence marks.

[0098] Among them, the cosine similarity calculation formula is as follows:

[0099]

[0100] m represents the vector dimension, a_i and b_i represent the i-th component of vectors A and B respectively, and CosSim represents cosine similarity.

[0101] When processing non-standardized text, Jieba's word segmentation tool efficiently breaks down complex sentences. For example, consider a financial report describing "This quarter, the company achieved revenue growth, but cost control was poor." Jieba breaks it down into "This quarter," "Company," "Achieved," "Revenue," "Growth," "But," "Cost," "Control," and "Poor." This word segmentation lays the foundation for subsequent processing. Jieba uses a dynamic programming algorithm to identify word boundaries, making it particularly suitable for processing polysemous words or long phrases in Chinese text, ensuring accurate word segmentation results.

[0102] After loading the financial term dictionary, professional vocabulary will be accurately marked.

[0103] StanfordParser performs dependency parsing on annotated text to generate a semantic structure tree. For example, the sentence "Revenue growth but poor cost control" is parsed into the main clause "Revenue growth" and the subordinate clause "poor cost control." The subject "Revenue" and the predicate "Growth" form a core dependency relationship. The semantic structure tree uses nodes to represent the logical connections between words, intuitively displaying the sentence hierarchy.

[0104] The parser uses statistical models to predict dependency relationships, making it suitable for processing complex sentence structures found in financial text. When calculating the node missing rate, assume that the semantic structure tree is missing the modifier for the "cost" node, resulting in a missing rate of 30%, exceeding the preset threshold of 20%. Once the node supplementation mechanism is triggered, the system infers the missing modifier as "ascending" based on the context, thereby completing the tree structure. This mechanism ensures the integrity of the semantic structure and provides a reliable foundation for subsequent matching.

[0105] When calculating the cosine similarity between the supplemented semantic structure and the standard template, assuming the standard template is a complete financial statement description, the similarity between the supplemented tree and the template is 0.85, which exceeds the threshold of 0.8, indicating a reasonable structure. If the similarity falls below the threshold, the BERT model generates a 768-dimensional semantic vector. For example, "revenue growth" is encoded as a high-dimensional vector that captures its semantic features. This vector representation enhances the accuracy of semantic comparison.

[0106] The matching rules of historical data are generated through statistical frequency to ensure the rationality of the filled content.

[0107] Confidence tags are calculated based on node matching accuracy. For example, the matching accuracy for the "Revenue" node is 95%, while that for "Expense Overrun" is 80%. The system assigns confidence levels to each field, generating labeled structured data such as "Revenue: Growth, 95%" and "Expense Overrun: Poor, 80%." This labeling clearly reflects the confidence level of the data.

[0108] like Figure 3As shown, in step S104, the missing fields are identified according to the semantic vector representation, and the historical database is queried using pattern matching technology. If the matching degree is higher than a preset threshold, the relevant field values are extracted to generate preliminary filling data.

[0109] Parse field content from structured text data, mark unrecognized or blank fields, and obtain a set of missing fields. Use the BERT model to extract features from the text content of recognized fields, generate a 768-dimensional semantic vector representation, and obtain a set of field semantic vectors. Construct query conditions based on the contextual information of the missing fields to obtain a set of query conditions. Convert the query condition set into an Elasticsearch Boolean query statement to obtain a set of Boolean query statements. Perform a k-nearest neighbor search in Elasticsearch, limiting the return to the top 10 similar historical records, and obtain a set of historical records. Calculate the cosine similarity between the field semantic vector set and the historical record set to obtain a set of matching values. If the matching value is greater than the preset threshold of 0.85, extract the corresponding field value from the historical record set to obtain a set of candidate values. The weight of each candidate value in the candidate value set is calculated using the formula \(w_i=\frac{s_i}{\sums_j}\), where \(w_i\) represents the weight of the i-th candidate value, \(s_i\) represents the matching degree of the i-th candidate value, and \(\sums_j\) represents the sum of the matching degrees of all candidate values, thereby obtaining a weight set. The candidate value set is weighted and summed according to the weight set to obtain preliminary fill data. For the date field in the preliminary fill data, a regular expression is used to verify the YYYY-MM-DD format to obtain a standardized date field. For the numeric field in the preliminary fill data, two decimal places are retained to obtain a standardized numeric field. For the text field in the preliminary fill data, leading and trailing spaces are removed to obtain a standardized text field. The standardized date field, standardized numeric field, standardized text field, and identified field are merged according to the field name to obtain a complete structured data record.

[0110] Field parsing of structured text data requires accurate content identification. Unidentified or blank fields must be marked as missing field sets.

[0111] The BERT model extracts features from identified fields and generates a 768-dimensional semantic vector. For example, for a "Revenue" field containing "10% growth this quarter," BERT encodes it into a high-dimensional vector, capturing semantic features such as "growth" and "revenue." BERT pre-trains to capture contextual relationships, ensuring the vector reflects the financial meaning of the field. Once the vector is generated, a collection of field semantic vectors is formed, facilitating subsequent matching.

[0112] The contextual information associated with missing fields is used to construct query conditions. For example, if the "Cost" field is missing and the context shows "Revenue Growth" and "Date 2025-01-01," the query condition can be set to "records with revenue growth and a similar date." These conditions are converted into Elasticsearch Boolean queries, limiting the time range and keywords. Boolean queries ensure precise searches and narrow the scope of historical records.

[0113] For example, when performing a k-nearest neighbor search, Elasticsearch returns 10 similar records. Suppose a historical record describes "costs increased by 5%." The cosine similarity between its semantic vector and the current "revenue growth" vector is 0.9, exceeding the threshold of 0.85. Similarity is calculated based on the angle between the vectors, reflecting the strength of the semantic association. Candidate values such as "increased by 5%" and "increased by 3%" are extracted to form a candidate value set.

[0114] Candidate value weights are calculated by normalizing the matching degree. For example, if two records have matching degrees of 0.9 and 0.86, their weights are 0.51 and 0.49, respectively. After weighting, "a 5% increase" is used as the initial fill value. This weighting method balances the credibility of historical records and improves the rationality of the fill.

[0115] Date field standardization requires regular expression validation. For example, a populated value containing "2025-1-1" will be converted to "2025-01-01" after validation. Numeric fields such as "5%" are rounded to two decimal places, and text fields such as "rising" are stripped of spaces and cleaned to a non-redundant format. Standardization ensures data consistency and facilitates system processing.

[0116] Specifically, when merging fields, the identified "Revenue: Increased by 10%" and the populated "Cost: Increased by 5%" are combined by field name to form a complete record. This merging is based on field mapping rules, ensuring the integrity of the data structure. The resulting structured data clearly reflects the financial situation, facilitating analysis and decision-making.

[0117] Step S105 , comparing the preliminary filled data with the historical data through a cross-validation mechanism, and if the consistency score is higher than a preset threshold, confirming the validity of the field, and obtaining a verified field value.

[0118] Original description data is obtained from non-standardized text, and the original description data includes product-related information. The spaCy tool is used to perform word segmentation and part-of-speech tagging on the original description data to obtain word segmentation and tagging results. The word segmentation and tagging results are matched based on a preset entity recognition rule library to identify three types of entities: product name, specification parameter, and numerical unit, and obtain an identified entity set. If the entity type in the identified entity set is product name, the entity is mapped to the name field according to a pre-established field mapping table. If the entity type in the identified entity set is specification parameter, the entity is mapped to the parameter field according to the field mapping table. If the entity type in the identified entity set is numerical unit, the entity is mapped to the numerical field according to the field mapping table. For entities in the identified entity set that are not matched to the field mapping table, the Levenshtein distance is used to calculate the similarity between the entity and the entry in the standard vocabulary. If the similarity exceeds a preset threshold, the entity is mapped to the corresponding standard field value in the standard vocabulary to obtain a mapped field value. The mapped field values are output in JSON format, with missing fields marked as null in the JSON format. Entities that do not exceed the preset threshold are marked as data to be verified, and a correction suggestion list is generated, which includes the data to be verified and recommended standard field values.

[0119] The formula for calculating the similarity of terms is as follows:

[0120]

[0121] Sim represents the similarity score, A and B represent the two strings to be compared, |A| and |B| represent the lengths of strings A and B respectively, and the score ranges from 0 to 1. A larger value indicates greater similarity.

[0122] When extracting product-related information from non-standardized text, the original description data may come from user reviews or product descriptions on e-commerce platforms. This application identifies product names, specifications and numerical units based on a preset entity recognition rule base. For example, the rule base defines "watch" and "smart watch" as product names, "inches" and "battery life" as specification parameters, and "7 days" and "1.5" as numerical units. In the above description, "new smart watch" is matched as the product name, "1.5 inches" is identified as a specification parameter, and "7 days" is identified as a numerical unit. Unmatched words such as "new model" are temporarily stored as potential entities, awaiting further processing. This rule matching method ensures that key information is not missed.

[0123] The field mapping table assigns recognized entities to corresponding fields. Suppose the mapping table stipulates that the product name is mapped to the "name" field, the specification parameters are mapped to the "spec" field, and the numerical unit is mapped to the "value" field. In the above example, "New Smart Watch" is mapped to "name: Smart Watch", "1.5 inches" is mapped to "spec: Screen Size", and "7 days" is mapped to "value: Battery Life". The design of the mapping table is based on common terms in the product domain, ensuring the generality of field assignment.

[0124] For entities that are not matched to the mapping table, the Levenshtein distance is used to calculate their similarity to the standard thesaurus. For example, "New Model" is not directly matched to the mapping table and is compared with "New Type" and "Brand New" in the standard thesaurus. If the similarity between "New Model" and "Brand New" is 0.9, exceeding the threshold of 0.85, then "New Model" is mapped to "Brand New" and classified as a supplementary description of the "name" field. If the similarity is lower than the threshold, such as "Special" being only 0.6, it is marked as data to be verified. This method effectively reduces ambiguity and improves the degree of data standardization.

[0125] The JSON format output ensures that the structured results are clear. The above example may be output as: {"name": "Smart Watch", "spec": "Screen Size", "value": "1.5 inches", "Battery Life": "7 days"}. If a field such as "Weight" is not recognized, it is marked as "weight": null. The explicit marking of missing fields facilitates subsequent data completion.

[0126] The data to be verified that does not exceed the similarity threshold will generate a list of correction suggestions. This list of suggestions provides a reference basis for data cleaning. The generation of the list of correction suggestions depends on the completeness of the standard thesaurus.

[0127] The above process progresses step by step from word segmentation to mapping and then to verification, ensuring that non-standardized text is gradually transformed into structured data.

[0128] In step S106, an adaptive classification model is used to process the verified field values. If the classification confidence is lower than the preset threshold, reclassification is performed in combination with metadata features to determine the final type of the document.

[0129] Obtain a document image, and use the CLAHE algorithm to perform contrast enhancement on the document image to obtain an enhanced image. Use Canny edge detection to locate the text area from the enhanced image and generate a binary image. Use TesseractOCR to extract text from the binary image to obtain a character sequence and the coordinate information of each character. Input the character sequence into a bidirectional long short-term memory network, set the hidden layer dimension to a preset value, and obtain the context encoding of each character. Decode the context encoding through the CRF layer to obtain field boundaries and an initial field value pair set. Obtain a historical document collection from the MongoDB database, and use Pandas to calculate the field co-occurrence matrix of the historical document collection. Determine a candidate list of missing fields based on the co-occurrence matrix, calculate the conditional probability distribution of each candidate field, and obtain a completed field set. Initialize a random forest classifier, input the feature vector of the initial field value pair set, the feature vector includes field type, length and coordinate information, and obtain a classification result and confidence score. If the confidence score is lower than the preset threshold, sigmoid weighting is applied to the coordinate information, and the weight formula is w=1 / (1+exp(-x*scale)), where x is the normalized coordinate value and scale is the scaling factor, to obtain a weighted feature vector. The weighted feature vector is used for secondary classification to obtain a secondary classification result. The KL divergence of the classification result and the secondary classification result is calculated. If the KL divergence is greater than the preset threshold, the classification result is determined to be the final field set. The historical document pattern library in the MongoDB database is updated according to the sliding window mechanism, and the pattern statistics of the most recent preset number are retained to obtain an updated pattern library. Fields are obtained from the final field set, and the confidence score and source tag of each field are marked to obtain a marked field set.

[0130] When processing document images, the CLAHE algorithm is used to enhance contrast, which is particularly suitable for scanned documents or documents with uneven lighting.

[0131] Canny edge detection locates text areas. Assuming the invoice image contains multiple text blocks, Canny removes noise through Gaussian filtering and calculates gradients to detect edges.

[0132] The threshold selection is very important. The low threshold is set to 50 and the high threshold is set to 150 to ensure edge continuity.

[0133] TesseractOCR extracts text from binary images. Using an invoice as an example, OCR recognizes a character sequence like "Amount: 1,000 yuan," with coordinate information for each character. For example, "Amount" is located in the upper left corner of the image, within the coordinate range (50, 50, 100, 80). This coordinate record provides the basis for subsequent field division.

[0134] Preprocessing such as denoising can improve OCR accuracy.

[0135] A bidirectional long short-term memory network processes character sequences. For example, using "date: 2023-10-01," the network analyzes the context and identifies the association between "date" and "2023-10-01." The hidden layer dimension is set to 128 to balance performance and efficiency. Context encoding captures dependencies between characters, improving field segmentation accuracy.

[0136] The CRF layer decodes the context encoding and delineates field boundaries. Continuing with the invoice scenario, CRF recognizes "Amount: 1000 yuan" as a field pair with clear boundaries, avoiding the splitting of "1000" and "yuan." The initial set of field-value pairs includes fields such as "Date: 2023-10-01" and "Amount: 1000 yuan."

[0137] Retrieve a collection of historical documents from MongoDB and use Pandas to calculate a co-occurrence matrix. Assume that historical invoices often contain "tax number" and "amount," and the co-occurrence matrix shows a strong correlation between the two. If the current document lacks "tax number," the matrix recommends a replacement. Conditional probability calculations confirm that "tax number" replacement has a high priority.

[0138] The random forest classifier validates a field. For example, the feature vector for "Amount: 1,000 yuan" has a numeric field type, a length of 4, and centered coordinates. The classification result shows a confidence level of 0.9, which is higher than the threshold of 0.8 and is immediately considered valid. If the confidence level is only 0.7, sigmoid weighting is applied, with the normalized x coordinate set to 0.5 and the scale set to 5. After calculating the weights, the classification is re-evaluated to improve the reliability of the result.

[0139] KL divergence compares the differences in classification results. If the primary and secondary classification field sets differ significantly, such as "amount" being mistakenly classified as "remarks," and the KL divergence exceeds the threshold of 0.2, the primary result will be used first to ensure stability.

[0140] A sliding window updates the MongoDB schema library. It retains the most recent 1,000 schema statistics, e.g., "amount" is often numeric. If new invoice field schemas change, the library content is dynamically adjusted to ensure adaptability.

[0141] The final field set is annotated with confidence and source. For example, "Amount: 1,000 yuan" has a confidence of 0.9, derived from direct OCR extraction; "Tax ID" has a confidence of 0.85 after completion, derived from co-occurrence inference. This annotated field set clearly records the reliability of each field for easy subsequent use.

[0142] Step S107 , arranging the verified field values according to the final type of the document, adjusting the field order using a data alignment algorithm, and generating standardized financial data records.

[0143] Obtain the document type identifier, match the corresponding version of the field mapping rules from the predefined template library, and obtain the field mapping rules. Using the field mapping rules, analyze the semantic tag candidate set through a bidirectional LSTM model to obtain the feature representation of the semantic tag candidate set. Combined with the domain dictionary weights, calculate the context dependency between fields for the feature representation to obtain the context dependency. Reconstruct the context dependency through a table structure tree to obtain a reconstructed data hierarchy. If there is a missing field mark in the data hierarchy, activate the historical data matching threshold check to obtain a matching check result. Using the matching check result, align the standardized field name mapping table with the column order defined by the output template identifier through a differential algorithm to obtain an aligned field name mapping table.

[0144] Obtaining a document type identifier is the starting point of the document processing process, used to determine the document's category. The system uses a pre-trained text classification model to analyze keywords in the document header or fixed area to determine the type identifier, such as "VAT-001" for a VAT invoice.

[0145] The classification model compares the identifier with the definitions in the template library based on features such as font and position to ensure accuracy. Matching field mapping rules from the predefined template library is a key step. The template library stores the field structures of different document types.

[0146] Based on the type identifier "VAT-001," the system retrieves the corresponding template from the MongoDB database and generates field mapping rules. For example, "Invoice Code" should be mapped to a 10-digit format. This approach ensures field consistency in subsequent processing. Using these field mapping rules, a bidirectional LSTM model analyzes the semantic label candidate set to capture the semantic information of the field.

[0147] Bidirectional LSTM can consider the previous and next context of the field at the same time and generate more accurate feature representation.

[0148] The domain dictionary stores invoice-related terms, such as "tax amount" and "tax rate." The system assigns a weight of 0.8 to "tax amount" and a weight of 0.2 to "invoice date." The system calculates their dependencies with "total amount" and generates a dependency matrix. This matrix reflects the logical relationships between fields and facilitates subsequent structured processing. The table structure tree reconstructs contextual dependencies and organizes the fields into a hierarchical data structure.

[0149] Based on the dependency matrix, the system classifies "invoice code" and "invoice date" as root nodes, and "total amount" and "tax amount" as child nodes, forming a tree structure.

[0150] Missing fields can be caused by scanning errors or incomplete documents. For example, if an invoice is missing the "Tax Rate" field, the system retrieves similar documents from historical invoice data and calculates a match. If the match exceeds a threshold of 0.9, the "Tax Rate" field is completed. This checking mechanism improves the reliability of field completion. Using the match check results, a differential algorithm is used to align the column order defined in the standardized field name mapping table with the output template identifier, ensuring a consistent output format.

[0151] The system found that the standardized field name "Total Amount" was inconsistent with the "Total Amount" in the template. It adjusted the column order through the difference algorithm and generated an aligned mapping table.

[0152] like Figure 4 As shown, in step S108, key indicators are extracted from the standardized financial data records. If the logical relationship of the indicators deviates from the preset range, the historical data is re-compared to obtain revised financial data.

[0153] Obtain input text, perform syntactic analysis on the input text using a dependency syntax analysis tool, extract subject-verb-object triples, and obtain first structured data. Extract numerical fields from the first structured data, compare the numerical fields using a preset threshold interval, and if the numerical value exceeds the threshold interval, mark it as second abnormal data. For the text segment corresponding to the second abnormal data, use a regular expression engine to perform pattern matching, and extract a third historical record set with the same subject and predicate from the historical database. Based on the third historical record set, calculate the arithmetic mean of the numerical fields therein to obtain average value data. Use the average value data to replace the numerical fields in the second abnormal data to generate fourth corrected data. Verify the logical relationship between the indicators in the fourth corrected data through foreign key constraints to obtain verified corrected data. Write the verified corrected data into a relational database to generate a storage result.

[0154] Dependency parsing tools analyze the syntactic structure of input text, aiming to identify subject-verb-object triples. For example, in the financial sector, the input text might be "Customer A applies for a loan of 1 million yuan." Through dependency parsing, the tool identifies "Customer A" as the subject, "apply" as the predicate, and "loan of 1 million yuan" as the object, generating the first structured data. The core of this method is to convert unstructured text into a structured form for subsequent processing. Syntactic analysis relies on a pretrained language model optimized for financial-specific vocabulary such as "loan" and "repayment" to ensure accurate parsing.

[0155] A numeric field, such as "1 million yuan," is extracted from the first structured data. A preset threshold range might be based on historical loan data, such as 100,000 to 5 million yuan. If the value falls outside this range, such as "10 million yuan," the value is flagged as a second anomaly. This threshold comparison logic is based on financial risk control, and anomalies may indicate potential input errors or high-risk applications.

[0156] The system will record the context of abnormal data, such as applicant identity and loan purpose, for subsequent analysis.

[0157] A regular expression engine is used to perform pattern matching on the text segment corresponding to the second abnormal data.

[0158] For example, for the query "Customer A applied for a loan of 10 million yuan," a regular expression can extract keywords such as "customer A" and "apply," search the historical database for records with the same subject and predicate, and generate a third set of historical records. Suppose the historical records show that Customer A has applied for three loans in the past, for amounts of 2 million yuan, 3 million yuan, and 4 million yuan, respectively. This matching method leverages the regularity of historical data to provide a basis for correcting abnormal data.

[0159] The average calculation takes into account the distribution of the data to avoid excessive influence of a single outlier on the results. The generated average data of 3 million yuan is used to replace the outlier of 10 million yuan, forming the fourth revised data.

[0160] The logical relationship between indicators in the fourth revised data is verified through foreign key constraints.

[0161] The revised loan amount of 3 million yuan must match customer A's income level, credit score, and other indicators. Assuming customer A has an annual income of 500,000 yuan and a good credit score, the system verifies that the 3 million yuan loan amount complies with the loan policy. Foreign key constraints ensure data consistency and avoid logical conflicts.

[0162] It should be noted that this verification is based on the relationship between database tables to ensure the reliability of the revised data.

[0163] Specifically, the verified corrected data is written into a relational database to generate a storage result.

[0164] For example, the corrected record includes fields such as Customer A, Loan Amount 3 million yuan, and Application Date, and is stored in the loan application form. Database write operations typically use a transaction mechanism to ensure data integrity.

[0165] Preferably, the system will generate a log to record the data changes before and after the correction, which is convenient for audit and tracking. This storage method supports subsequent query and analysis, and supports financial decision-making.

Claims

1. A method for intelligent entry and verification of financial data, characterized in that: The following steps are involved: S101 acquires an original document image, and processes the original document image using an image enhancement algorithm to obtain an optimized document image; S102 extracts structured fields from the optimized document image, determines field coordinates, and generates a structured data table; S103 performs semantic analysis on the non-standardized text in the structured data table to generate a semantic vector representation; S104 identifies missing fields based on the semantic vector representation, queries the historical database, and generates preliminary filling data; S105 compares the preliminary filled data with historical data through a cross-validation mechanism to obtain a verified field value; S106 uses a classification model to process the verified field value to determine the document type; S107 sorts the verified field values according to the document type to generate standardized data records; S108 extracts key indicators from the standardized data records, verifies the logical relationship of the indicators, and obtains corrected data.

2. A financial data intelligent entry and verification method according to claim 1, characterized in that: Step S101 includes the following steps: Parsing metadata of the original document image to obtain a resolution value; If the resolution value is lower than a preset threshold, a bilateral filtering algorithm is used to perform denoising on the original document image to obtain a first image; Processing the first image by histogram equalization to adjust pixel brightness values to obtain a second image; performing high-frequency detail enhancement on the second image based on a Laplacian operator to obtain a third image; The third image is processed using an adaptive binarization method to generate a black and white document image.

3. The method for intelligent financial data entry and verification according to claim 1, wherein: Step S102 includes the following steps: Processing the optimized document image through a model based on the U-Net architecture to generate a binary mask of the table area; Using the Sobel operator to calculate the gradient distribution of the binary mask and determine the table field bounding box; Processing the bounding boxes using a non-maximum suppression algorithm to obtain a corrected bounding box set; spatially aligning the corrected bounding box set with the optical character recognition result using affine transformation to obtain an aligned text field set; The aligned text field set is processed by a bidirectional long short-term memory network to generate an inter-field association probability matrix and a structured data table.

4. The method for intelligent financial data entry and verification according to claim 1, wherein: Step S103 includes the following steps: Using a word segmentation tool to perform word segmentation processing on the non-standardized text to obtain a word segmentation result set; Annotating the words in the word segmentation result set using a pre-established term dictionary to obtain annotated text; Using a syntax analysis tool to perform dependency syntax analysis on the annotated text to construct a semantic structure tree; If the node missing rate of the semantic structure tree exceeds a preset threshold, nodes are supplemented to obtain a supplemented semantic structure tree; If the cosine similarity between the supplemented semantic structure tree and the standard template is lower than a preset threshold, a semantic vector representation is generated through a pre-trained language model.

5. The method for intelligent financial data entry and verification according to claim 1, wherein: Step S104 includes the following steps: Parsing the structured data table, marking missing fields, and obtaining a missing field set; Use the pre-trained language model to generate semantic vector representations for the identified fields and obtain a set of field semantic vectors; Constructing a query condition based on the context information of the missing field and converting it into a Boolean query statement; Perform nearest neighbor search in the history database to obtain similar historical records; Calculating the cosine similarity between the field semantic vector set and the historical records; if the similarity exceeds a preset threshold, extracting the corresponding field value and generating a candidate value set; The candidate value set is weightedly summed according to weight calculation to generate preliminary filling data.

6. A financial data intelligent entry and verification method according to claim 1, characterized in that: Step S105 includes the following steps: Using a word segmentation tool to perform word segmentation and part-of-speech tagging on the preliminary filled data to obtain a word segmentation and tagging result; Perform entity recognition on the word segmentation and annotation results based on a preset rule library to obtain a set of recognized entities; Mapping the identified entity set to corresponding fields according to a field mapping table; For unmapped entities, calculate their similarity with the standard vocabulary. If the similarity exceeds the preset threshold, map them to the standard field value to obtain the mapped field value. For entities that do not exceed the preset threshold, a list of correction suggestions is generated to obtain the verified field values.

7. A financial data intelligent entry and verification method according to claim 1, characterized in that: Step S106 includes the following steps: Performing contrast enhancement on the verified field value to generate an enhanced image; Locate the text area through edge detection and generate a binary image; Extracting character sequence and coordinate information from the binary image; Encoding the character sequence through a bidirectional long short-term memory network to obtain a contextual code; Decoding the context code using a conditional random field to obtain a set of field boundaries and initial field value pairs; According to the field co-occurrence matrix of the historical document collection, a candidate list of missing fields is determined to obtain a set of completed fields; A random forest classifier is used to process the completed field set to obtain a document type.

8. The method for intelligent entry and verification of financial data according to claim 1, wherein: Step S107 includes the following steps: Obtaining a field mapping rule matching the document type from a predefined template library; Analyze the semantic label candidate set through a bidirectional long short-term memory network to obtain feature representation; Calculate the contextual dependencies between fields by combining domain dictionary weights; Reconstruct the context dependency relationship through a table structure tree to obtain a data hierarchy; If there are missing fields in the data hierarchy, the fields are supplemented by checking the matching degree of historical data; A differential algorithm is used to align the field mapping rules with the column order of the output template to generate standardized data records.

Images