Bank fund data format self-adaptive parsing method based on large model technology, self-adaptive parser, device and medium

Abstract

The application discloses a bank fund data format self-adaptive analysis method and self-adaptive parser based on large model technology, equipment and medium, which comprises the following steps: obtaining bank fund transaction data as original data; using the templates and analysis rules in the bank fund transaction template library to analyze the original data, and outputting the original data if there is a difference between the original data and the templates in the bank fund transaction template library; judging the type of the original data and obtaining data of the bank fund transaction type; using a large language model to extract key field data from the data of the bank fund transaction type; converting the key field data into a standard format by using the large language model and outputting the standard format; and storing the templates and processing rules corresponding to the standard format into the bank fund transaction template library, so as to improve the analysis ability of subsequent bank fund transaction data. Therefore, the application can automatically identify and adapt to bank fund transaction data of different formats.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to an adaptive parsing method, adaptive parser, device, and medium for bank funds data format based on large model technology. Background Technology

[0002] The formats of bank transaction records vary significantly across different banks. Some banks use tables, others use a combination of text and tables, and still others use specific text formats, sometimes even including official seals and watermarks. For example, the publicly disclosed "Account Detail Data Processing Method, Device, Server, and Client" used by the Industrial and Commercial Bank of China (ICBC) employs a data caching service component to process account detail data, effectively reducing frequent database operations and improving processing efficiency. However, this technology primarily focuses on optimizing data storage and transmission, and does not address the issues of diverse bank transaction record formats and inconsistent field names.

[0003] Existing technology discloses an automatic parsing method for bank transaction data. This method obtains various bank transaction data files from the databases of various bank servers through a text acquisition module, configures parsing rules for various bank transaction data files using a configuration module, extracts header rows through a header extraction module to determine the text information in the header, and matches the field text information with standard and non-standard fields pre-stored in a dictionary using a matching module. However, this method still relies on pre-set parsing rules and field matching dictionaries, and has the following main problems: 1) New format data requires manual writing of new parsing rules, resulting in poor adaptability; 2) For data with non-standard formats or variations, the parsing accuracy is low; 3) Manually maintaining parsing rules requires a lot of time and cost; 4) When dealing with massive amounts of data, the processing efficiency is low, there is a lack of effective error handling and disaster recovery mechanisms, and the system stability is poor.

[0004] Therefore, there is an urgent need for a parser that can automatically adapt to different bank fund flow data formats and efficiently and accurately extract and standardize key information. Summary of the Invention

[0005] This invention aims to at least solve one of the technical problems existing in the prior art. Therefore, in response to the above-mentioned problems, the purpose of this invention is to provide a method, adaptive parser, device, and medium for adaptive parsing of bank fund transaction data formats based on large model technology, capable of automatically identifying data formats, extracting key fields, and standardizing fund transaction data from different banks.

[0006] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows: In a first aspect, the present invention provides an adaptive parsing method for bank fund data formats based on large model technology, comprising: Obtain bank transaction data as raw data; The original data is parsed using templates and parsing rules in the bank's fund transaction template library. If the original data differs from the templates in the bank's fund transaction template library, the original data is output. The type of the raw data is determined, and the data is identified as bank fund transaction data. The key field data of the data, which is of the type of bank fund transactions, is extracted using a large language model; The key field data is converted into a standard format and output using a large language model. The templates and processing rules corresponding to the standard format are stored in the bank's fund transaction template library to improve the ability to parse subsequent bank fund transaction data.

[0007] In some possible implementations, the raw data is parsed using templates and parsing rules in the bank's fund transaction template library. If the raw data differs from the templates in the bank's fund transaction template library, the raw data is output. The process is as follows: The raw data is matched with templates in the bank's fund transaction template library; When a match is successful, the template and corresponding rules are directly applied to process the original data, and standard format bank transaction data is output. If the original data fails to match the template, the original data is considered to be in a new data format and will not be processed; the original data will be output.

[0008] In some possible implementations, the original data is matched with templates in the bank's fund transaction template library. The process involves hashing the header of the original data and matching it with the hash value of the header of the template in the template library to determine whether they have the same structure.

[0009] In some possible implementations, the type of the original data is determined, and data of bank fund transaction type is obtained. The process is as follows: Obtain the original data; Design prompt word templates to guide the large language model in determining the type of the raw data; The output data is bank fund transaction data.

[0010] In some possible implementations, the key field data of the data, which is of the type of bank fund transactions, is extracted using a large language model. The process is as follows: Retrieve data whose data type is bank fund transactions; Using predefined prompts, a large language model is invoked to understand bank fund transaction data and extract key field data; Perform a completeness and accuracy check on all records containing the extracted key field data; If key fields are missing or extracted incorrectly, the large model will be prompted to reflect, check, and re-extract. Output all key field data.

[0011] In some possible implementations, a large language model is used to convert the key field data into a standard format for output. The process is as follows: Retrieve all key field data; Use standard or custom prompt words to invoke the large language model to perform a standardized data format conversion on all key field data; Perform integrity and continuity checks on the converted standardized data, and output the standardized data that passes the checks. For critical fields that cannot be standardized or data that fails inspection, mark them and provide processing suggestions.

[0012] In some possible implementations, the templates and processing rules corresponding to the standard format are stored in the bank's fund transaction template library. The process is as follows: The extracted key field data is compared with the fields of the original data to obtain the field mapping rules; The standardized data and the original data are compared using prompt words to obtain the data standardization code; Record the original data, field mapping rules, and standardized data processing code in the template library. When encountering data with this structure in subsequent data processing, directly use the template in the template library for processing, without needing to call the large model.

[0013] Secondly, the present invention also provides an adaptive parser for bank funds data format based on large model technology, comprising: The data parsing module based on the template library is configured to obtain bank fund transaction data as raw data, parse the raw data using templates and parsing rules in the bank fund transaction template library, and output the raw data if there are differences between the raw data and the templates in the bank fund transaction template library. The data type identification and judgment module is configured to judge the type of the original data and obtain data whose data type is bank fund transaction; The key field extraction module is configured to extract key field data from data whose data type is bank fund transaction using a large language model. The data standardization module uses a large language model to convert the key field data into a standard format for output. The knowledge accumulation module stores the templates and processing rules corresponding to the standard format into the bank fund transaction template library to improve the ability to parse subsequent bank fund transaction data.

[0014] Thirdly, the present invention also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to perform any of the methods described herein.

[0015] Fourthly, the present invention also provides a computer-readable storage medium for storing one or more programs, said one or more programs including computer instructions for causing a computer to perform any of the methods described herein.

[0016] Because the present invention adopts the above technical solution, it has the following characteristics: 1. Strong adaptability: This invention can automatically identify and adapt to bank fund transaction data in different formats without the need for manual writing of new parsing rules.

[0017] 2. High parsing accuracy: This invention utilizes the semantic understanding capabilities of a large language model to accurately extract key field data, and can effectively process even data with non-standard formats.

[0018] 3. High processing efficiency: This invention utilizes the existing rule-based method to parse existing template data, reducing GPU consumption, and also uses the capabilities of a large language model to parse new and unseen data.

[0019] 4. Low maintenance cost: This invention reduces the need for manual maintenance of parsing rules, thus lowering system maintenance costs.

[0020] 5. High scalability: This invention is easy to add support for new data formats to adapt to ever-changing business needs.

[0021] In summary, this invention can be widely applied to processing fund transaction data from different banks. Attached Figure Description

[0022] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. In the drawings: Figure 1This is a schematic diagram of the adaptive parsing method for bank fund data format based on large model technology according to an embodiment of the present invention.

[0023] Figure 2 This is a schematic diagram of the structure of the adaptive parser for bank funds data format based on large model technology in an embodiment of the present invention.

[0024] Figure 3 This is a structural diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0025] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a specific order as described or illustrated, unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0026] Although terms such as first, second, third, etc., may be used in this document to describe multiple elements, components, regions, layers, and / or segments, these elements, components, regions, layers, and / or segments should not be limited by these terms. These terms may be used only to distinguish one element, component, region, layer, or segment from another. Unless the context clearly indicates otherwise, terms such as "first," "second," and other numerical terms used herein do not imply order or sequence. Therefore, the first element, component, region, layer, or segment discussed below may be referred to as the second element, component, region, layer, or segment without departing from the teachings of the exemplary embodiments.

[0027] For ease of description, spatial relative terms may be used in the text to describe the relationship of one element or feature relative to another element or feature as shown in the figure. These relative terms include, for example, "inside," "outside," "middle," "outer," "below," "above," etc. Such spatial relative terms are intended to include different orientations of the device in use or operation, other than those depicted in the figure.

[0028] The following problems exist in processing fund flow data from different banks: 1) Difficulty in mapping field names: Different banks use different names for the same field, such as "transaction date" and "occurrence date," both of which represent transaction time. Accurately identifying these semantically identical fields is a technical challenge. 2) Difficulty in data standardization: The content formats of fund flow data fields vary greatly among different banks, such as transaction time formats and amount representation methods. Unifying and converting them to a standard format is a challenge. 3) Difficulty in adaptive processing: How to achieve adaptive processing without manual intervention when faced with new bank formats or field variations. 4) Difficulty in balancing processing efficiency and accuracy: When processing massive amounts of data, ensuring processing efficiency while maintaining parsing accuracy is a technical challenge that needs to be solved. This invention provides a method, adaptive parser, device, and medium for adaptive parsing of bank fund transaction data based on large model technology. The method includes: acquiring bank fund transaction data as raw data; parsing the raw data using templates and parsing rules in a bank fund transaction template library; if the raw data differs from the templates in the library, outputting the raw data; determining the type of the raw data and identifying it as bank fund transaction data; extracting key field data from the bank fund transaction data using a large language model; converting the key field data into a standard format using the large language model for output; and storing the template and processing rules corresponding to the standard format in a bank fund transaction template library to improve the parsing capability of subsequent bank fund transaction data. Therefore, this invention can automatically identify the data format, extract key fields, and standardize fund transaction data from different banks.

[0029] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the invention and to fully convey the scope of the invention to those skilled in the art.

[0030] Example 1: As Figure 1 As shown in the figure, the adaptive parsing method for bank fund data format based on large model technology provided in this embodiment, by combining large language model (LLM) technology, realizes intelligent identification, key field extraction and standardization processing of different bank fund flow data. The specific process is as follows: S1. Data parsing based on template library.

[0031] In this embodiment, template-based data parsing refers to using templates and parsing rules of processed bank fund transaction data to parse the data, reducing the number of calls to the large language model, reducing GPU usage, and improving processing efficiency. The specific process is as follows: S11. Receive bank fund flow data as raw data; S12. Match the structure of the input bank transaction data with the templates in the existing template library; S13. When a perfect match is found, the existing template and corresponding parsing rules are directly applied to process the original data. After successful processing, standardized bank transaction data is output.

[0032] S14. When the original data differs from the existing template, it indicates that the original data is in a new data format, and the original data is output.

[0033] S2, Data type identification and judgment.

[0034] In this embodiment, data type identification and judgment refers to using the contextual understanding capability of the large language model to determine the type of the raw data output in step S1, identifying whether it is fund-related data. Even if it is fund-related data, it may be different types of data such as transaction flow data or basic account information. The specific process is as follows: S21. Receive raw data; S22. Design specific prompt word templates to guide the large language model to accurately determine the data type; S23. Output the data type identification result. If it is confirmed to be bank fund transaction data, continue processing.

[0035] S3, Key Field Extraction.

[0036] In this embodiment, key field extraction refers to leveraging the semantic understanding and information extraction capabilities of a large language model to achieve high-precision key field extraction, thus resolving the issue of different banks using different names for the same field. Specific implementation methods include: S31. Accept fund transaction flow data; S32. Using prompt words, call the large language model to further understand the data, extract fields, and match them; S33. Perform a completeness and accuracy check on all extracted records; S34. If key fields are missing or extracted incorrectly, prompt the large model to reflect, check, and re-extract. S35. Output data containing key fields.

[0037] S4. Data standardization.

[0038] In this embodiment, data standardization refers to using the understanding and conversion capabilities of a large language model to convert the extracted non-standardized field data into a unified standard format, thus solving the problem of significant differences in the content format of fund flow data fields from different banks. Specific implementation methods include: S41. Receive the output results of the key field extraction in step S3; S42. As needed, use standard or customized prompt words to call the large language model to perform standardized data format conversion; S43. Perform integrity and continuity checks on the converted fund transaction records, and output the data that meets the required standardization criteria. S44. For non-standardized abnormal data, mark it and provide processing suggestions.

[0039] S5. Knowledge Accumulation.

[0040] In this embodiment, knowledge accumulation refers to storing templates and processing rules for new data formats processed by the large model into a template library. By continuously accumulating new templates and rules, the ability to analyze subsequent financial data is improved. Specific implementation methods include: S51. Receive the output result of data standardization in step S4; S52. Compare the results of extracting key fields with the fields of the original data to obtain the mapping rules between specific fields and standard fields of the data; S53. Using prompt words, compare the output of the data standardization step with the input data records to obtain the code for standardized data processing. S54. Record the original data, field mapping rules, and standardized data processing code into the template library. When encountering similar data structures in subsequent data processing, the mapping rules and code can be used directly for processing without calling the large model.

[0041] The following detailed embodiments illustrate the specific application of the adaptive parsing method for bank fund data format based on large model technology of the present invention. The prompt words provided by the present invention have been verified in DeepSeek R1. The specific process is as follows: S1. Data parsing based on template library.

[0042] In this embodiment, the specific process of data parsing based on the template library is as follows: S11. Calculate the hash value of the header of the input bank transaction data and match it with the hash value of the header of templates in the existing template library to determine if they have the same structure. A successful match is considered only when the hash value of the input data header is exactly the same (100% identical) as the hash value stored in the template library. The specific process is as follows: I. Existing templates in the template library and their corresponding parsing rules.

[0043] ① Hash value of the header of the bank fund flow template: deed8fcc0c531b4c532b34adebeda19e7374712764b10042d701b0f8416824a1 ② Template field mapping and parsing rules: { "Rule ID": "RULE_9527", "Field Mapping Table": { "Transaction Card Number": "standard_card_no", "Trading Account": "standard_account_no", "Trading Time": "standard_trade_time", "Transaction Amount": "standard_trade_amount", "Transaction Balance": "standard_balance", "Payment / Receipt Flag": "standard_dc_flag", "Transaction serial number": "standard_serial_no", "...(one-to-one correspondence of the remaining 24 plain text fields)": "..." } } II. Specific calculation and matching process of the header hash.

[0044] In the actual matching process, the present invention can be divided into the following four steps: Step 1: Extract the original table headers and clean the format.

[0045] The system reads the first line of the input raw bank statement file (such as bank fund flow.csv).

[0046] The system calls the filtering component to globally remove all quotation marks and backslashes, restoring the list of column names to plain text. ['Transaction Card Number', 'Transaction Account Number', 'Transaction Account Name', 'Transaction ID Number', 'Transaction Time', ..., 'Transaction Type'] Step 2: Sequential assembly.

[0047] Strictly following the physical order in which these 31 fields appear in the original file, use a fixed concatenation operator (such as |) to directly concatenate them into a unique, lossless long text string: "Transaction Card Number | Transaction Account Number | Transaction Account Name | Transaction ID Number | Transaction Time | ... | Transaction Type" Step 3: Calculate the hash value.

[0048] The above long text string is input into a standard hash algorithm (SHA-256 algorithm is used in this embodiment) to calculate the unique hash value corresponding to the current structure of the input data: "deed8fcc0c531b4c532b34adebeda19e7374712764b10042d701b0f8416824a1" Step 4: Perform a completely identical search and match.

[0049] Scenario A (Completely identical hashes → Direct rule parsing): The system searches the template library using the hash value. Since the hash value is exactly the same as the template hash value already stored in the library, the system determines that the match is successful. The corresponding rule RULE_9527 is directly retrieved within seconds and delivered to the parsing engine for high-speed output in a standard format, eliminating the need to call large models and thus significantly saving GPU computing power.

[0050] Scenario B (Hash values ​​are not completely identical → Data splitting triggers large model): When searching in the template library, if no identical hash value is found, the matching is immediately determined to be unsuccessful (considered as new format data). The system abandons applying the inherent rules and directly splits and intercepts the raw data, sending it to the large language model (LLM) for intelligent field extraction and standardization transformation.

[0051] In this embodiment, the header of the bank fund flow data table is as follows: "Transaction Card Number" "Trading Account" "Transaction Account Name" "Transaction document number" Trading Hours Transaction Amount "Transaction Balance" "Payment / Receipt Symbol" "Counterparty account number" "Cash symbol" "Opponent's Account Name" "Opponent's ID number" "Counterparty's bank" "Summary Description" "Trading Currencies" "Trading outlet name" "Location where the transaction took place" Was the transaction successful? "Slip Number" "IP address" MAC address "Counterparty transaction balance" "Transaction serial number" "Log number" "Types of vouchers" "Certificate Number" "Trading Teller Number" "Remark" Merchant Name Merchant Code "Transaction type".

[0052] In this embodiment, the header of the input bank transaction data is hashed. The specific steps are as follows: 1) Format cleaning (normalization): Different storage methods introduce different physical noise (e.g., column names in CSV files may contain double quotes " or tabs \t, such as "transaction card number\t"). Before calculating the hash, the system uses global preprocessing to uniformly filter out all spaces, newline characters (\n), tabs (\t), and double quotes ("), retaining only plain text column names.

[0053] 2) Lossless Concatenation of Table Header Strings: The cleaned column names are concatenated directly into a unique long string using a fixed concatenation operator (such as |) according to their order of appearance in the original file. For example, if the cleaned table header order is [Transaction Card Number, Transaction Account], it will be concatenated as "Transaction Card Number|Transaction Account". If the order or names are changed (such as "Transaction Account|Transaction Card Number"), the concatenated string will change accordingly.

[0054] 3) Calculate the hash: Use a standard hash algorithm (such as SHA-256 or MD5) to generate a hash value for this unique long string: Hash = SHA256("transaction card number|transaction account").

[0055] In this embodiment, the definition and composition of the template library are as follows: Definition of Template Library: A template library is a mapping database maintained by the system that stores the structural features of accumulated bank transaction records and standardized data parsing rules. Its main function is to perform millisecond-level precise structural matching of the input raw data using "header hash values." If a match is successful, the pre-stored parsing rules in the library are directly called, eliminating the need for the large model to call them; if a match fails (i.e., a new structure is determined), the data is handed over to the large model for processing, and after the large model successfully transforms the data, the features and rules of the new structure are fed back into the template library.

[0056] The template library consists of: ① Template header hash value (featured primary key): After filtering out physical noise, the existing bank transaction headers are concatenated into a unique string according to the original column order, and then calculated using a standard hash algorithm (such as SHA-256 or MD5) to obtain an absolutely unique hash value. This hash value serves as a globally unique index key in the template library, used for a fast and absolutely rigorous comparison with the hash of the input data header. ② Template field mapping and parsing rules (data rule body): A set of parsing rules strongly bound to the hash value above, recording the mapping relationship between the original column names and the absolute positions or names of the system's standard cleaned fields (e.g., "query card number" corresponds to "transaction card number").

[0057] S12. When a match is successful, the template and corresponding parsing rules in the template library are directly applied for processing. After successful processing, standardized bank transaction data is output and the program ends.

[0058] S13. When no matching hash value is found in the template, it means that there is no relevant template and parsing rule in the template library. Therefore, no processing is performed on the original data, and the unprocessed original data is output.

[0059] S2, Data type identification and judgment.

[0060] In this embodiment, the specific implementation method for data type identification and judgment is as follows: S21. Obtain the original data and select, for example, the first 1000 records (first 1000 rows) for judgment. A suitable subset of the data is sufficient to help with the judgment and can improve efficiency. Here, the suitable subset of the data refers to the local continuous sample rows (in this embodiment, the first 1000 rows are selected) that are extracted sequentially from the massive original data and are sufficient to reflect the text features and business semantics. Using the first 1000 rows as a subset for reasoning can ensure that the large model can accurately identify the data type 100% and control the reasoning time of the large model to the second level, which greatly saves token consumption and GPU computing power.

[0061] S22. Design specific prompt word templates to guide the large language model to accurately determine the data type.

[0062] In this embodiment, a specific prompt word template is designed as follows: "You are a financial data expert. Please analyze the textual characteristics of the input data: professional field naming (loan and lending indicators, transaction numbers), standardized data format (amount, timestamp); structural characteristics: complete coverage of transaction elements in rows and columns, chronological arrangement, unique transaction numbers; semantic characteristics: clear fund flow (in and out indicators), dynamic balance updates, and real transaction scenarios (consumption / transfer / payment). Based on this comprehensive judgment, determine whether the data type is fund transaction flow data." S23. The large language model makes judgments based on prompt words and outputs the data type recognition results.

[0063] S3, Key Field Extraction.

[0064] In this embodiment, the specific implementation method for extracting key fields is as follows: S31. Use appropriate prompts to further understand the data identified as bank fund transaction records, extract fields, and match them.

[0065] Specifically, for example: "For the input bank transaction data, extract the following key fields: account number, card number, transaction amount, transaction time, counterparty account number, counterparty card number, transaction type, loan / debit flag, currency, transaction balance, transaction serial number, and transaction summary. Note that different banks may use different names for the same fields, such as 'transaction date' and 'date of occurrence' both representing the transaction time. Please output the results in JSON format."

[0066] S32. Perform integrity and accuracy checks on all extracted records.

[0067] In this embodiment, data integrity refers to the consistency of the number of records and the non-empty rate of key fields; data accuracy refers to checking whether the data values ​​are within a reasonable range and verifying the rationality of the extracted results based on the logical relationship between fields (such as the relationship between amount and balance). Specifically, data integrity and data accuracy are judged automatically by the "automated verification script (such as Python verification program)" inside the parser combined with mathematical logic.

[0068] S33. If key fields are missing or extracted incorrectly, prompt the large model to reflect, check, and re-extract, and output a structured set of key fields and content data.

[0069] Specifically, the preconditions for triggering the large model reflection in this step (i.e., the discovery of missing information or extraction errors) are automatically determined by the internal verification program of the parser (such as background logic code), and the entire process requires no manual verification. The program acts as a supervisor, using strict rules to ensure the output of the large model, including: 1) Automated judgment logic for "missing key fields" (based on core anchor comparison).

[0070] Anchor field preset: The underlying system of this invention predefines the "minimum core anchor field set" (including: transaction card number, transaction direction, transaction time, transaction amount, transaction counterparty card number, etc.) for the establishment of bank fund flow data.

[0071] Automated comparison and verification: After the large model outputs the extracted JSON structure in step S31, the background program will automatically extract all the keys of the JSON and perform intersection or comparison with the preset "anchor field set".

[0072] Business logic interception: The essential attributes of cash flow data are determined by specific elements. If the program finds that the JSON output by the large model is missing absolutely core fields such as "transaction time" or "transaction amount," the system cannot logically recognize it as a valid "cash flow" (because without time and amount, the definition of cash flow is invalid). At this time, the program will automatically throw an exception signal of "missing key field."

[0073] 2) Automated judgment logic for "extraction error" (based on data type and logic verification).

[0074] Type and range error: For example, the amount field was extracted as garbled characters containing letters, or the date format was outside the calendar range, and the program's regular expression validation failed.

[0075] Calculation logic error: By automatically executing the identity "beginning balance + deposits - withdrawals = ending balance", if the equation does not hold true, it proves that the large model has made an error in extracting the amount or the direction of the loan (such as the number being misplaced, or the withdrawal being extracted as a deposit).

[0076] S4. Data standardization.

[0077] In this embodiment, the specific implementation methods of data standardization include: S41. Based on requirements, use standard or customized prompts to invoke the large language model to perform standardized data format conversion, namely: "Standardize bank fund transaction data according to the following standards: transaction time is uniformly formatted as "YYYY-MM-DD hh:mm:ss", amount and balance are uniformly numerical and retained to two decimal places, and the debit / credit indicator is uniformly "in / out". Note that different bank statements may express the same content differently, and uniform conversion is required." S42. Perform an integrity check on the converted fund transaction records.

[0078] In this embodiment, the integrity of transaction records is verified based on the bank account's initial balance, total receipts, total payments, and ending balance. The initial balance is the balance before the first transaction, and the ending balance is the balance after the last transaction. The verification script uses code aggregation to automatically extract the account's balance before the first transaction (initial balance) and the balance after the last transaction (ending balance) across the entire time series. Simultaneously, the total receipts and total payments in the input data pool are calculated. Then, the program automatically runs the formula: Initial Balance + Total Receipts - Total Payments = Ending Balance. If the equation is true, the script automatically tags "Integrity Verification Passed".

[0079] S43. Perform a continuity check on the converted fund transaction records.

[0080] In this embodiment, the continuity of transaction records is verified based on the balance and transaction amount. After sorting bank accounts by transaction time, the verification formula is: Previous balance + Receipt - Payment - Current balance = 0. Specifically, the verification script first uses a standardized "transaction time" field to automatically sort all transaction records in ascending order. Then, the program uses the row traversal mechanism of the data frame to substitute each row into the micro-verification formula: Previous balance + Receipt - Payment - Current balance = 0. The number of abnormal rows whose result is not 0 (exceeding reasonable floating-point error) is counted. The abnormality rate is calculated as: Abnormality rate = Number of abnormal rows / Total number of transaction rows. The calculated abnormality rate is compared with the system's preset acceptable threshold (e.g., 5%). If the abnormality rate is lower than the threshold, the script determines that the overall continuity of the batch of data is qualified; if it is greater than or equal to the threshold, the continuity verification is deemed unsuccessful.

[0081] S44. Output standardized data, mark abnormal data that cannot be standardized or data that fails inspection, and provide processing suggestions.

[0082] In this embodiment, non-standardized abnormal data refers to data rows that cannot be forcibly and securely converted into a unified standard format by the large language model and subsequent verification scripts due to incomplete original file export, inclusion of non-transactional text, or serious semantic ambiguity, in terms of mathematical logic and format requirements.

[0083] Specifically, for example, if a data row contains phrases such as "Subtotal for this page," "Summary of beginning / ending balances," "Print date: 2023-12-01," "Teller operator number: 001," or a bank disclaimer, these rows are not transaction records in essence and are severely lacking core business elements such as "transaction amount," "transaction time," and "loan / debit status." A large-scale model cannot and should not forcibly fabricate a transaction record from these elements, as this is unstandardized.

[0084] Specifically, for example, transaction amounts are displayed as #VALUE!, ***, or scientific notation with lost precision, and transaction times are displayed as invalid dates (such as 9999-99-99 or 2023-02-30). Such data violates objective physical meaning and date logic, and cannot be standardized.

[0085] S5. Knowledge Accumulation.

[0086] In this embodiment, the specific implementation of knowledge accumulation is as follows: S51. Compare the results of extracting key fields with the fields of the original data to obtain the mapping rules between specific fields and standard fields of the data. For example, the mapping rule can be in the form of "query card number" corresponding to "transaction card number". When encountering similar data in the future, field matching and conversion can be performed quickly.

[0087] S52. Using prompts, compare the standardized output data with the original input data records to obtain standardized processing code. For example: "Compare the standardized bank transaction data with the original data, please provide me with a piece of Python code to standardize the transaction time, amount, and loan / borrowing information."

[0088] S53. Record the original data, field mapping rules, and standardized data processing code in the template library. When encountering similar data structures later, the mapping rules and code can be used directly for processing, without needing to call the large model and output an updated template library.

[0089] In summary, this invention aims to address the following specific technical bottlenecks in existing bank fund transaction data parsing technologies: 1) Significant differences in the format of fund transaction data between different banks: There are significant differences in the content and field format of fund transaction data between different banks. Some have separate header content, while others use different names to express the same field. Existing technology is difficult to process these different data formats simultaneously. 2) Rigid parsing rules are difficult to adapt to new formats: Traditional methods rely on fixed rules written by humans. When faced with new format data from different banks, the rules need to be continuously updated and cannot adapt to new formats. 3) Frequent manual intervention leads to low efficiency: When encountering format variations or non-standard data, manual intervention is often required for correction, resulting in low processing efficiency and failing to meet the needs of processing massive amounts of data. 4) High parsing error rate: Traditional methods are prone to parsing errors when dealing with data that has format variations, missing fields, or non-standard descriptions, which affects data quality.

[0090] It should be noted that images or paper documents in bank transaction data need to be converted using OCR technology, but this is not within the scope of the data processing of this invention. This invention focuses on solving the problem of inconsistent content and field formats of different bank transaction data.

[0091] Example 2: Following the above example 1 which provides an adaptive parsing method for bank fund data formats based on large model technology, this example provides an adaptive parser for bank fund data formats based on large model technology. The adaptive parser for fund data formats provided in this example can implement the adaptive parsing method for bank fund data formats based on large model technology in Example 1. This adaptive parser can be implemented through software, hardware, or a combination of both. For ease of description, this example is described by dividing the functions into various units. Of course, in implementation, the functions of each unit can be implemented in one or more software and / or hardware components. For example, the adaptive parser for fund data formats may include integrated or separate functional modules or units to execute the corresponding steps in the methods of Example 1. Since the adaptive parser for fund data formats in this example is basically similar to the method example, the description process of this example is relatively simple. For relevant details, please refer to the description in Example 1. The example of the adaptive parser for bank fund data formats based on large model technology provided by this invention is merely illustrative.

[0092] Specifically, such as Figure 2 As shown, this invention provides an adaptive parser for bank fund transaction data formats based on large language model (LLM) technology. This parser, by combining LLM technology, achieves intelligent identification, key field extraction, and standardization processing of different bank fund transaction data. It includes: a template-based data parsing module, a data type identification and judgment module, a key field extraction module, a data standardization module, and a knowledge accumulation module, wherein: The template-based data parsing module is configured to obtain bank transaction data as raw data, parse the raw data using templates and parsing rules in the bank transaction template library, and output the raw data if there are differences between the raw data and the templates in the bank transaction template library. The data type identification and judgment module is configured to judge the type of the original data and obtain data whose data type is bank fund transaction; The key field extraction module is configured to extract key field data from data whose data type is bank fund transaction using a large language model. The data standardization module uses a large language model to convert the key field data into a standard format for output. The knowledge accumulation module stores the templates and processing rules corresponding to the standard format into the bank fund transaction template library to improve the ability to parse subsequent bank fund transaction data.

[0093] Example 3: This example provides an electronic device corresponding to the adaptive parsing method for bank fund data format based on large model technology provided in Example 1. The electronic device can be an electronic device for the client, such as a mobile phone, laptop, tablet computer, desktop computer, etc., to execute the method of Example 1.

[0094] like Figure 3 As shown, the electronic device includes a processor, a memory, a communication interface, and a bus. The processor, memory, and communication interface are connected via the bus to enable communication between them. The memory stores a computer program that can run on the processor. When the processor runs the computer program, it executes the method of Embodiment 1. The implementation principle and technical effects are similar to those of Embodiment 1, and will not be repeated here. Those skilled in the art will understand that... Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computing device on which the present application is applied. The specific computing device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0095] In a preferred embodiment, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), and optical discs.

[0096] In a preferred embodiment, the processor can be any type of general-purpose processor such as a central processing unit (CPU) or a digital signal processor (DSP), and is not limited thereto.

[0097] Example 4: This example provides a computer-readable storage medium for storing one or more programs, the one or more programs including computer instructions, which, when executed by a computer, cause the computer to perform the method provided in Example 1 above.

[0098] In a preferred embodiment, the computer-readable storage medium may be a tangible device for holding and storing instructions executable, such as, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof. The computer-readable storage medium stores computer program instructions that cause a computer to perform the method provided in Embodiment 1 above.

[0099] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0100] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0101] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0102] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In the description of this specification, the terms "a preferred embodiment," "furthermore," "specifically," "in this embodiment," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the embodiments in this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An adaptive parsing method for bank funds data format based on large model technology, characterized in that, include: Obtain bank transaction data as raw data; The original data is parsed using templates and parsing rules in the bank's fund transaction template library. If the original data differs from the templates in the bank's fund transaction template library, the original data is output. The type of the raw data is determined, and the data is identified as bank fund transaction data. The key field data of the data, which is of the type of bank fund transactions, is extracted using a large language model; The key field data is converted into a standard format and output using a large language model. The templates and processing rules corresponding to the standard format are stored in the bank's fund transaction template library to improve the ability to parse subsequent bank fund transaction data.

2. The adaptive parsing method for bank fund data format based on large model technology according to claim 1, characterized in that, The original data is parsed using templates and parsing rules from the bank's fund transaction template library. If the original data differs from the templates in the bank's fund transaction template library, the original data is output. The process is as follows: The raw data is matched with templates in the bank's fund transaction template library; When a match is successful, the template and corresponding rules are directly applied to process the original data, and standard format bank transaction data is output. If the original data fails to match the template, the original data is considered to be in a new data format and will not be processed; the original data will be output.

3. The adaptive parsing method for bank fund data format based on large model technology according to claim 2, characterized in that, The original data is matched with templates in the bank's fund transaction template library. The process is as follows: the header of the original data is hashed and matched with the hash value of the header of the template in the template library to determine whether they have the same structure.

4. The adaptive parsing method for bank fund data format based on large model technology according to claim 1, characterized in that, The data type of the original data is determined, and data of bank fund transaction type is obtained. The process is as follows: Obtain the original data; Design prompt word templates to guide the large language model in determining the type of the raw data; The output data is bank fund transaction data.

5. The adaptive parsing method for bank fund data format based on large model technology according to claim 1, characterized in that, The process of extracting key field data from bank fund transaction data using a large language model is as follows: Retrieve data whose data type is bank fund transactions; Using predefined prompts, a large language model is invoked to understand bank fund transaction data and extract key field data; Perform a completeness and accuracy check on all records containing the extracted key field data; If key fields are missing or extracted incorrectly, the large model will be prompted to reflect, check, and re-extract. Output all key field data.

6. The adaptive parsing method for bank fund data format based on large model technology according to claim 5, characterized in that, The key field data is converted into a standard format and output using a large language model. The process is as follows: Retrieve all key field data; Use standard or custom prompt words to invoke the large language model to perform a standardized data format conversion on all key field data; Perform integrity and continuity checks on the converted standardized data, and output the standardized data that passes the checks. For critical fields that cannot be standardized or data that fails inspection, mark them and provide processing suggestions.

7. The adaptive parsing method for bank funds data format based on large model technology according to claim 1, characterized in that, The process of storing the standard format template and processing rules into the bank's fund transaction template library is as follows: The extracted key field data is compared with the fields of the original data to obtain the field mapping rules; The standardized data and the original data are compared using prompt words to obtain the data standardization code; Record the original data, field mapping rules, and standardized data processing code in the template library. When encountering data with this structure in subsequent data processing, directly use the template in the template library for processing, without needing to call the large model.

8. An adaptive parser for bank funds data format based on large model technology, characterized in that, include: The data parsing module based on the template library is configured to obtain bank fund transaction data as raw data, parse the raw data using templates and parsing rules in the bank fund transaction template library, and output the raw data if there are differences between the raw data and the templates in the bank fund transaction template library. The data type identification and judgment module is configured to judge the type of the original data and obtain data whose data type is bank fund transaction; The key field extraction module is configured to extract key field data from data whose data type is bank fund transaction using a large language model. The data standardization module uses a large language model to convert the key field data into a standard format for output. The knowledge accumulation module stores the templates and processing rules corresponding to the standard format into the bank fund transaction template library to improve the ability to parse subsequent bank fund transaction data.

9. An electronic device, characterized in that, include: At least one processor; And a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to perform the method according to any one of claims 1-7.

10. A computer-readable storage medium for storing one or more programs, characterized in that, The one or more programs include computer instructions for causing a computer to perform the method according to any one of claims 1-7.

Images