Financial data synchronization method and system, computer equipment and storage medium

Abstract

The invention relates to the technical field of data processing, in particular to a financial data synchronization method and system, computer equipment and a storage medium, and the method comprises the following steps: based on input financial data, performing formatting and data cleaning, preprocessing the data, analyzing fund flow data, and performing consistency verification on the fund data; inconsistent data are identified and recorded, and a consistency verification result is generated. According to the method, by analyzing the data and extracting the incremental data needing to be synchronized, the synchronization efficiency is improved, real-time updating of the data is ensured, prediction of synchronization task time is carried out in combination with calculation resource requirements and real-time system loads, the synchronization priority is calculated, the reasonability of resource use is improved, the synchronized data is comprehensively verified, and the synchronization efficiency is improved. Data missing, repetition and inconsistency are found and corrected in time, the accuracy and consistency of financial data are ensured, the efficiency of financial data synchronization is improved, accurate and timely transmission and updating of the data are ensured, and the financial management process is optimized.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a financial data synchronization method, system, computer equipment and storage medium. Background Art

[0002] The field of data processing technology involves multiple processes of collecting, organizing, storing, analyzing and transmitting raw data, aiming to improve the effectiveness, accuracy and value of data, including data mining, data cleaning, data conversion, data integration and data synchronization, involving database management, distributed computing, machine learning, artificial intelligence, real-time data processing, big data technology, and is applied to enterprise information management, finance, medical care, the Internet and government industries. It aims to support decision-making, optimize operational processes and improve data-driven business results through efficient data flow and analysis.

[0003] Among them, the financial data synchronization method is used to ensure data consistency and real-time updates between multiple financial systems. Through a variety of technical means, including API interface, batch synchronization, and real-time synchronization, it ensures that the financial data stored in different systems are accurately and timely transmitted and updated, avoiding data inconsistencies or delays, improving the efficiency, accuracy and transparency of corporate financial management, reducing errors in manual data entry, enhancing the basis for financial decision-making, and improving the accuracy and timeliness of financial statements.

[0004] Traditional financial data synchronization methods rely on batch synchronization and traditional scheduled synchronization methods, which cannot capture data changes in real time, resulting in excessive system load and transmission of large amounts of redundant data, affecting the efficiency of data synchronization. The lack of precise synchronization control between multiple financial systems leads to data consistency issues, affecting the accuracy of financial statements. In terms of priority management of synchronization tasks, there is a lack of dynamic adjustment of real-time system loads, resulting in unreasonable task execution order and resource allocation, reducing synchronization efficiency and resource utilization. Summary of the invention

[0005] In order to solve the technical problems existing in the prior art, the embodiments of the present invention provide a financial data synchronization method, system, computer device and storage medium. The technical solution is as follows: In one aspect, a financial data synchronization method is provided, the method comprising: S1: Based on the input financial data, pre-process the data through formatting and data cleaning, analyze the capital flow data, perform consistency check on the capital data, identify and record inconsistent data, and generate consistency check results; S2: Based on the consistency check result, the financial data is classified by analyzing the amount, timestamp, and account type information of the financial data to generate a financial data classification result; S3: Based on the classification result of the financial data, by comparing the differences of multiple data fields in the financial data, identifying and extracting the incremental data that needs to be synchronized, packaging the target data, and generating an incremental data packet; S4: Based on the incremental data packet, according to the computing resource requirements of the data synchronization task and the real-time system load, the execution time of multiple tasks is predicted, and the synchronization priority of multiple data is calculated in combination with the urgency and sensitivity of multiple transaction types, and a synchronization task priority queue is generated; S5: Based on the synchronization task priority queue, the synchronized data is analyzed to verify the integrity of the data, identify data missing, duplicate, and inconsistent data, and correct abnormal data to generate a financial data synchronization record.

[0006] As a further solution of the present invention, the consistency verification results include cash flow data, data anomaly identification, and timestamp consistency verification results; the financial data classification results include income financial data, expenditure financial data, and accounts payable financial data; the incremental data packet includes incremental data fields, difference data records, and incremental synchronization marks; the synchronous task priority queue includes task execution time estimation information, task priority level, and system load forecast information; the financial data synchronization records include synchronous data sets, data correction records, and synchronous abnormal data detection results.

[0007] As a further solution of the present invention, based on the input financial data, the data is pre-processed by formatting and data cleaning, the fund flow data is analyzed, the fund data is consistency checked, inconsistent data is identified and recorded, and the steps of generating consistency check results are specifically as follows: S101: Based on the input financial data, format and clean the data, including standardizing the amount, account, and timestamp fields, and adjusting the currency unit and time format to generate data preprocessing results; S102: Analyze the capital flow data according to the data preprocessing result, identify non-compliant transaction data by comparing the capital flow direction with financial rules, and generate capital flow analysis results; S103: Based on the fund flow analysis result, verify the consistency of the data, including the amount balance, fund flow rules between accounts, and timestamp sorting, identify and mark abnormal data, and generate a consistency verification result.

[0008] As a further solution of the present invention, based on the consistency check result, the financial data is classified by analyzing the amount, timestamp, and account type information of the financial data, and the steps of generating the financial data classification result are specifically as follows: S201: Based on the consistency check result, extract the amount and time stamp of multiple financial data to generate a basic information extraction result; S202: Based on the basic information extraction result, identify the outflow account and inflow account of funds, analyze the flow direction and business type of funds between accounts, and generate fund type information; S203: Based on the fund type information, multiple financial data are classified, including income, expenditure, and accounts payable, and a financial data classification result is generated.

[0009] As a further solution of the present invention, based on the financial data classification result, by comparing the differences of multiple data fields in the financial data, identifying and extracting the incremental data that needs to be synchronized, packaging the target data, and generating the incremental data packet are specifically as follows: S301: Based on the financial data classification result, extract the amount, account, and transaction type field information of multiple data, identify and mark the changed data points through data comparison, and generate changed data identification information; S302: Based on the changed data identification information, identify the incremental data that needs to be synchronized, and generate a synchronized processing data set; S303: Based on the synchronously processed data set, the incremental data is packaged according to the data packet capacity requirement of data transmission to generate an incremental data packet.

[0010] As a further solution of the present invention, based on the incremental data packet, according to the computing resource requirements of the data synchronization task and the real-time system load, the execution time of multiple tasks is predicted, and the synchronization priority of multiple data is calculated in combination with the urgency and sensitivity of multiple transaction types. The steps of generating the synchronization task priority queue are specifically as follows: S401: Based on the incremental data packet, evaluate the computing resources required for multiple synchronization tasks and generate a resource requirement evaluation result; S402: Based on the resource demand evaluation result and in combination with real-time system load information, the actual execution time of multiple synchronization tasks is evaluated and predicted to generate an execution time prediction result; S403: Based on the execution time prediction result, combined with the urgency and sensitivity of multiple transaction types, the processing priorities of multiple synchronization tasks are calculated to generate a synchronization task priority queue.

[0011] As a further solution of the present invention, the specific formula for calculating the processing priority of multiple synchronization tasks is: ; Among them, P represents the priority score of the synchronization task, U represents the urgency feature vector value of the task, S represents the business sensitivity feature vector value, and T represents the standardized predicted execution time. represents the deviation between the current system load and the benchmark load, N represents the number of tasks of the same type waiting to be executed in the current queue, represents the comprehensive weight coefficient of the feature vector, represents the time impact adjustment coefficient, Represents the queue saturation balance factor.

[0012] As a further solution of the present invention, S501: based on the synchronization task priority queue, analyzing the synchronized data, verifying the integrity of the data, and generating data integrity verification information; S502: Based on the data integrity check information, duplicate data is marked by comparing multiple key fields, including timestamp, amount, and account, and duplicate data identification results are generated; S503: Based on the duplicate data identification result, the abnormal data is corrected according to the type of the abnormal data field to generate a financial data synchronization record.

[0013] As a further solution of the present invention, the specific formula for verifying the integrity of the data is: ; in, represents the data integrity assessment indicator, Indicates the total number of data fields that have completed data synchronization. Represents the total number of data fields to be synchronized. Represents the number of financial data records that were successfully synchronized. Represents the total number of data records synchronized by the target. represents the number of missing data fields detected, Represents the total number of key data fields to be verified, Represents the data timestamp continuity ratio, Represents the data field integrity weight coefficient, Represents the data record integrity weight coefficient, Represents the weight coefficient of missing data impact.

[0014] On the other hand, a financial data synchronization system is provided, which is applied to a financial data synchronization method, and the system includes: The data preprocessing module formats and cleans the input financial data. It identifies and records inconsistent data by analyzing the amounts, account balances, and capital flows of multiple financial data, and generates consistency verification results. The data type analysis module extracts the amount, timestamp, and account type information from the financial data based on the consistency check result, analyzes the data according to preset classification rules, classifies the data into multiple types, and generates financial data classification results; The incremental synchronization processing module identifies the incremental data that needs to be synchronized and packages it based on the financial data classification result, by comparing the differences of multiple data fields, to generate an incremental data packet; The task queue adjustment module evaluates and predicts the execution time of multiple tasks based on the incremental data packet, according to the computing resource requirements of the data synchronization task and the real-time system load, calculates the synchronization priority of multiple tasks in combination with the urgency and sensitivity of multiple transaction types, and generates a synchronization task priority queue; The synchronization data correction module analyzes the integrity of the synchronized data based on the synchronization task priority queue, identifies data missing, duplication and data inconsistency, corrects abnormal data, and generates financial data synchronization records.

[0015] On the other hand, a financial data synchronization device is provided, comprising: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned financial data synchronization methods is implemented.

[0016] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned financial data synchronization methods.

[0017] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least: By analyzing data and extracting incremental data that needs to be synchronized, the synchronization efficiency is improved, the real-time update of data is ensured, the synchronization task time is predicted based on the computing resource requirements and the real-time system load, the synchronization priority is calculated, and the rationality of resource use is improved. The synchronized data is fully verified to promptly discover and correct data missing, duplication and inconsistency, ensure the accuracy and consistency of financial data, improve the efficiency of financial data synchronization, ensure accurate and timely transmission and updating of data, and optimize the financial management process. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 It is a schematic diagram of the workflow of the present invention; Figure 2 This is a detailed flow chart of S1 of the present invention; Figure 3This is a detailed flow chart of S2 of the present invention; Figure 4 This is a detailed flow chart of S3 of the present invention; Figure 5 This is a detailed flow chart of S4 of the present invention; Figure 6 This is a detailed flow chart of S5 of the present invention; Figure 7 It is a system flow chart of the present invention. DETAILED DESCRIPTION

[0020] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0021] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0022] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0023] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0024] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0025] The embodiment of the present invention provides a financial data synchronization method, such as Figure 1 The flowchart of the financial data synchronization method shown in the figure may include the following steps: S1: Based on the input financial data, pre-process the data through formatting and data cleaning, analyze the capital flow data, perform consistency check on the capital data, identify and record inconsistent data, and generate consistency check results; S2: Based on the consistency check result, the financial data is classified by analyzing the amount, timestamp, and account type information of the financial data to generate a financial data classification result; S3: Based on the classification results of financial data, by comparing the differences of multiple data fields in the financial data, identify and extract the incremental data that needs to be synchronized, package the target data, and generate incremental data packets; S4: Based on the incremental data packets, according to the computing resource requirements of the data synchronization tasks and the real-time system load, the execution time of multiple tasks is predicted, and the synchronization priority of multiple data is calculated based on the urgency and sensitivity of multiple transaction types, and the synchronization task priority queue is generated; S5: Based on the synchronization task priority queue, analyze the synchronized data, verify the integrity of the data, identify data missing, duplication, and data inconsistency, correct abnormal data, and generate financial data synchronization records.

[0026] The consistency check results include fund flow data, data anomaly identification, and timestamp consistency check results. The financial data classification results include income financial data, expenditure financial data, and accounts payable financial data. The incremental data packet includes incremental data fields, difference data records, and incremental synchronization marks. The synchronization task priority queue includes task execution time estimate information, task priority level, and system load forecast information. The financial data synchronization records include synchronization data sets, data correction records, and synchronization anomaly data detection results.

[0027] See also Figure 2 Based on the input financial data, the data is pre-processed through formatting and data cleaning, the fund flow data is analyzed, the fund data is checked for consistency, and inconsistent data is identified and recorded. The specific steps for generating consistency check results are as follows: S101: Based on the input financial data, format and clean the data, including standardizing the amount, account, and timestamp fields, and adjusting the currency unit and time format to generate data preprocessing results; In sub-step S101, for the amount field, a currency unit conversion algorithm based on market exchange rates is used to convert all currency units into a unified base currency and ensure that they comply with the currency specifications of international financial accounting standards. When standardizing the account field, the account type mapping table pre-defined in the database is used to check whether all account formats comply with the account coding standards of the international financial system. The timestamp field is standardized by unifying the time information into the ISO 8601 format to ensure that it accurately represents the date and time, avoiding erroneous data caused by time zone or format differences. All fields are filled with null values, and a filling method based on domain knowledge is used to supplement null values. After processing through the target step, the generated data preprocessing result is the cleaned financial data set.

[0028] S102: Analyze the capital flow data according to the data preprocessing results, identify non-compliant transaction data by comparing the capital flow direction with financial rules, and generate capital flow analysis results; In sub-step S102, the financial rule model is used to compare and match the fund flow direction of each transaction. The model uses a decision tree algorithm based on a rule engine to set predefined fund flow rules to ensure that each transaction complies with the fund flow direction of the account to which it belongs. Data mining techniques such as association rule mining are used to further analyze the potential correlation of fund flows and identify potential non-compliant transactions, such as cross-border fund flows or fund transfers between unauthorized accounts. Through target analysis, fund flow analysis results are generated, transaction data that does not comply with predetermined financial rules is identified, and corresponding non-compliant identifiers are output. The analysis results provide a basis for subsequent abnormal data processing and fund flow optimization.

[0029] S103: Based on the fund flow analysis results, verify the consistency of the data, including the amount balance, fund flow rules between accounts, and timestamp sorting, identify and mark abnormal data, and generate consistency verification results; In sub-step S103, check whether the balance of the amount in the data meets the transaction rules, that is, ensure that the amount in each fund flow transaction matches the transfer and balance between each account, and use the amount balance verification algorithm to check. The algorithm is based on the comparison of the account inflow and outflow amounts in each transaction to ensure that there are no balance errors in all fund flow transactions. Check whether the fund flow rules between accounts meet the predetermined financial requirements. Use the rule verification model to compare the transfer rules between accounts to ensure that each fund flow complies with the agreed payment and receipt rules, such as whether the funds in the fund outflow account are sufficient to pay the fund inflow account. Check the timing of the data through the timestamp sorting algorithm to ensure that each transaction is carried out in the correct time sequence to avoid data timing errors. Mark and record transactions that do not meet the rules, generate consistency verification results, and output the identified abnormal data as a basis for subsequent data repair or troubleshooting.

[0030] See also Figure 3 Based on the consistency check results, the financial data is classified by analyzing the amount, timestamp, and account type information of the financial data. The specific steps for generating the financial data classification results are as follows: S201: Based on the consistency check result, extract the amount and timestamp of multiple financial data to generate a basic information extraction result; In sub-step S201, the amount field is extracted through the financial data standardization system. The system extracts the amount from the original financial record according to the preset rules and converts the amount into a unified currency unit. The process applies the currency unit conversion model and standardizes the amounts of different currencies into the target currency based on the real-time exchange rate data. The timestamp field is extracted through the time parsing model, which converts the original timestamp data into the ISO 8601 standard format to ensure that the time stamp of each data is consistent. All extracted amount and timestamp information will be summarized as the basic information extraction result to form a financial data set that meets the specifications. The result provides basic data support for further analysis of capital flow, account identification and classification.

[0031] S202: Based on the basic information extraction result, identify the outflow account and inflow account of funds, analyze the flow direction and business type of funds between accounts, and generate fund type information; In sub-step S202, the account fields in the extracted financial data are compared with the pre-defined account code database to determine the outflow account and inflow account of each transaction, analyze the flow direction of funds between accounts, and use a fund flow analysis model to identify the direction of fund outflow and inflow by comparing the amount, account and timestamp data in the transaction. The model makes judgments based on the fund flow direction set by the rule engine to ensure that the flow direction of each transaction complies with the predetermined rules. According to the fund flow direction of the transaction, the business type of each transaction is identified. The business type analysis is performed through a classification algorithm based on a decision tree. The model determines which business type the transaction belongs to based on the tags and features in the historical data. After the analysis is completed, fund type information is generated for subsequent fund flow and financial data classification processing.

[0032] S203: Classifying multiple pieces of financial data, including income, expenditure, and accounts payable, based on the fund type information, and generating a financial data classification result; In sub-step S203, a financial classification model is used. Based on known business type information, the classification algorithm classifies each financial data according to the direction of capital flow, transaction amount and business characteristics. During the classification process, the model clusters data through a support vector machine algorithm, automatically learns based on the characteristics of categories such as income, expenditure, accounts payable in historical financial data, adjusts classification rules, and adapts to the ever-changing financial data. In the determination process of each category, the classification results are verified according to the set financial rules to ensure that each data is accurately classified into the corresponding category, and a financial data classification result is generated, which includes transaction data of each category and its corresponding classification label, and the classification result is used for subsequent financial report generation and analysis.

[0033] See also Figure 4Based on the classification results of financial data, by comparing the differences of multiple data fields in the financial data, the incremental data that needs to be synchronized is identified and extracted, and the target data is packaged. The specific steps of generating incremental data packets are as follows: S301: Based on the financial data classification results, the amount, account, and transaction type field information of multiple data are extracted, and the changed data points are identified and marked through data comparison to generate changed data identification information; In sub-step S301, the amount field is extracted through a standardized conversion model. The model converts all amounts into a unified base currency unit based on different currency units to ensure that the amounts of all financial data are under a unified standard. The extraction of the account field depends on the account identification model. The model identifies the account information of each transaction according to the account coding specification and classifies the account type according to the predefined account classification rules. The extraction of the transaction type field adopts a rule matching model to classify transactions according to different types. By comparing field information such as transaction description and amount, it is determined which type each transaction belongs to. After the extraction is completed, the data is compared and the changed data points are identified through the difference detection model. The detection model adopts a threshold-based change analysis method to compare the differences between current and historical data, mark the changed data points, and generate changed data identification information to provide a basis for subsequent synchronization processing.

[0034] S302: Based on the changed data identification information, identify the incremental data that needs to be synchronized, and generate a synchronized processing data set; In sub-step S302, the changed data in the financial data is screened according to the mark of the changed data identification information. The identification process finds all the changed fields, such as changes in amount, new accounts or adjustments in transaction types, by comparing the differences between the current data and the historical data. The incremental recognition algorithm uses a timestamp sorting method to ensure that the screened incremental data is in chronological order and avoids data sequence errors. By comparing historical data and current data, the incremental recognition algorithm ensures the uniqueness of each incremental data, excludes duplicate data, and generates an incremental data set that needs to be synchronized. The generated synchronized processing data set provides a basis for subsequent data transmission and packaging processing, ensuring the efficient execution of subsequent steps.

[0035] S303: Based on the synchronously processed data set, the incremental data is packaged according to the data packet capacity requirement for data transmission to generate an incremental data packet; In sub-step S303, a data packet allocation algorithm is used to group incremental data according to a preset maximum data packet capacity. The algorithm checks the total amount of data in the synchronously processed data set and calculates the required number of packets based on the capacity limit of each data packet. The data packet allocation algorithm dynamically adjusts the data packet size to ensure that each data packet meets the transmission capacity requirements, minimizes the number of data packets, and improves transmission efficiency. Incremental data is packaged according to the data packet allocation strategy to ensure that the data in each data packet has high cohesion, that is, data of the same type of transactions or inter-segment data is placed in the same data packet as much as possible. After packaging, an incremental data packet is generated and preparations are made for the data transmission phase to ensure that the data can be transmitted in the best manner.

[0036] See also Figure 5 ,Based on the incremental data packets, according to the computing resource requirements of the data synchronization tasks and the real-time system load, the execution time of multiple tasks is predicted, and the synchronization priority of multiple data is calculated in combination with the urgency and sensitivity of various transaction types. The specific steps for generating the synchronization task priority queue are as follows: S401: Based on the incremental data packet, the computing resources required by multiple synchronization tasks are evaluated to generate a resource requirement evaluation result; In sub-step S401, the computing resource requirements are estimated based on the size and complexity of the incremental data packet through the resource requirement assessment model. For each synchronization task, the resource requirement assessment model will consider the processing time of the data packet, the size of the data packet, and the requirements of computing-intensive operations. The computing resource assessment model will analyze the number of fields, data types, complexity of encryption and decryption operations, and possible computing tasks of the incremental data packet. According to the characteristics of the data packet, the CPU, memory, storage and bandwidth resources required for each synchronization task are calculated. According to historical execution data and load analysis, adjustments are made in combination with preset computing resource standards to ensure that the assessment results meet the actual system load requirements and generate resource requirement assessment results as the basic data for subsequent resource scheduling and optimization.

[0037] S402: Based on the resource demand evaluation result and in combination with the real-time system load information, the actual execution time of multiple synchronization tasks is evaluated and predicted, and an execution time prediction result is generated; In sub-step S402, an execution time prediction model is used to predict the execution time of each synchronization task based on historical execution data, resource requirements and system load information. The execution time prediction model collects the current load information of the system, including CPU usage, memory usage and network bandwidth utilization, etc. The target information is obtained through a real-time system monitoring tool, which provides performance bottlenecks or resource limitations that may be encountered during task execution. The execution time prediction model combines the computing resource requirement evaluation results of each synchronization task to simulate the execution process of the task under the current system load. The model is based on machine learning algorithms, including regression analysis or decision tree algorithms. By training historical data, it learns the relationship between the execution time and resource requirements of the task and predicts the actual execution time. The process takes into account the differences in resource usage of different tasks, generates execution time prediction results for multiple synchronization tasks, and provides support for task scheduling and priority calculation.

[0038] S403: Based on the execution time prediction result, combined with the urgency and sensitivity of multiple transaction types, the processing priorities of multiple synchronization tasks are calculated, and a synchronization task priority queue is generated; The specific formula for calculating the processing priority of multiple synchronous tasks is: ; Among them, P represents the priority score of the synchronization task, U represents the urgency feature vector value of the task, S represents the business sensitivity feature vector value, and T represents the standardized predicted execution time. represents the deviation between the current system load and the benchmark load, N represents the number of tasks of the same type waiting to be executed in the current queue, represents the comprehensive weight coefficient of the feature vector, represents the time impact adjustment coefficient, Represents the queue saturation balance factor.

[0039] formula: ; Detailed explanation of the formula and the process of formula calculation and derivation: The formula is used to calculate the priority score of the financial data synchronization task, and the result is used to sort the synchronization task priority queue; Parameter meaning and setting value: U represents the urgency eigenvector value, which is assumed to be 0.85; S represents the service sensitivity characteristic vector value, which is assumed to be 0.72; T represents the normalized predicted execution time, which is assumed to be 1.5 hours; ΔR represents the system load deviation value, which is assumed to be 15%; N represents the number of queue tasks, assuming it is 3; β represents the comprehensive weight coefficient of the feature vector, which is assumed to be 0.85; α represents the time effect adjustment coefficient, which is assumed to be 1.2; δ represents the queue saturation balance factor, which is assumed to be 0.5; Substitute the parameters into the formula for calculation: ; The calculation result 4.22 indicates that the priority score of the target task is 4.22. The value is used to dynamically adjust the priority of the task queue to ensure that key business data synchronization tasks are executed first.

[0040] See also Figure 6 ,S501: Based on the synchronization task priority queue, analyze the synchronized data, verify the integrity of the data, and generate data integrity verification information; The specific formula for verifying the integrity of data is: ; in, represents the data integrity assessment indicator, Indicates the total number of data fields that have completed data synchronization. Represents the total number of data fields to be synchronized. Represents the number of financial data records that were successfully synchronized. Represents the total number of data records synchronized by the target. represents the number of missing data fields detected, Represents the total number of key data fields to be verified, Represents the data timestamp continuity ratio, Represents the data field integrity weight coefficient, Represents the data record integrity weight coefficient, Represents the weight coefficient of missing data impact.

[0041] formula: ; Detailed explanation of the formula and the process of formula calculation and derivation: The formula is used to calculate the data integrity assessment index, and the result is used to measure the integrity of the data after synchronization; Parameter meaning and setting value: is the data field integrity weight coefficient, assumed to be 0.4; is the data record integrity weight coefficient, assumed to be 0.35; is the weight coefficient for the impact of missing data, which is assumed to be 0.25; The total number of data fields completed for data synchronization is assumed to be 875; is the total number of data fields to be synchronized, assuming it is 1000; is the number of financial data records successfully synchronized, assuming it is 9200; The total number of data records synchronized for the target, assuming it is 10,000; is the number of missing data fields detected, assumed to be 50; The total number of key data fields to be verified is assumed to be 500; is the data timestamp continuity ratio, assumed to be 0.98; Substitute the parameters into the formula for calculation: ; The result of 0.893 shows that the data synchronization integrity has reached a high level. The data integrity verification information shows that the synchronization process maintains good data quality. The evaluation value will be used to generate data integrity verification information, providing an important reference for subsequent data processing and quality control.

[0042] S502: Based on the data integrity check information, multiple key fields including timestamp, amount, and account are compared to mark duplicate data and generate duplicate data identification results; In sub-step S502, a duplicate data identification algorithm is used to detect whether there are duplicate records in the data set by comparing multiple field values ​​such as timestamp, amount, and account. Specifically, the timestamp field is used to determine the uniqueness of the data. If two data have alternating stamps and the amount and account information are consistent, they are judged as duplicate data. The amount field and account field are also used as comparison basis. If the amount and account fields are repeated and have the same signs, the system will mark them as duplicate data. Through multiple comparisons of the target fields, the system can identify duplicate data records in the data set and generate duplicate data identification results. A threshold-based duplicate data screening strategy is used to avoid incorrectly marking duplicate data when some fields change, thereby ensuring the accuracy of the identification results.

[0043] S503: Based on the duplicate data identification result, the abnormal data is corrected according to the type of the abnormal data field, and a financial data synchronization record is generated; In sub-step S503, a data cleaning algorithm is used to perform specific repairs on different types of abnormal data. If an error is identified in the amount field, the budget calculation model is used to automatically calculate and adjust the amount field value based on historical transaction data or the flow of funds between accounts to ensure consistency with other fields. If an abnormality occurs in the account field, the system will match it with the predefined account library and correct it through the account verification rules. If the timestamp is inconsistent, the system will use the time sorting algorithm to correct the timestamp to ensure that the data is arranged in the correct chronological order. After the target correction operation, a financial data synchronization record is generated and the correction result is output to ensure the accuracy and consistency of all synchronized data.

[0044] See also Figure 7 , a financial data synchronization system, the financial data synchronization system is used to execute the above financial data synchronization method, the system includes: The data preprocessing module formats and cleans the input financial data. It identifies and records inconsistent data by analyzing the amounts, account balances, and capital flows of multiple financial data, and generates consistency verification results. The data type analysis module extracts the amount, timestamp, and account type information from the financial data based on the consistency check results, analyzes the data according to the preset classification rules, classifies the data into multiple types, and generates financial data classification results; The incremental synchronization processing module identifies and packages the incremental data that needs to be synchronized based on the financial data classification results by comparing the differences in multiple data fields to generate incremental data packets; The task queue adjustment module evaluates and predicts the execution time of multiple tasks based on the incremental data packets, the computing resource requirements of the data synchronization tasks and the real-time system load, and calculates the synchronization priorities of multiple tasks based on the urgency and sensitivity of various transaction types to generate a synchronization task priority queue; The synchronization data correction module analyzes the integrity of the synchronized data based on the synchronization task priority queue, identifies data missing, duplication and data inconsistency, corrects abnormal data, and generates financial data synchronization records.

[0045] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When a computer instruction or computer program is loaded or executed on a computer, a process or function according to an embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0046] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0047] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0048] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0049] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0050] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0051] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0052] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0053] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0054] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0055] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A financial data synchronization method, characterized in that: The method comprises: S1: Based on the input financial data, pre-process the data through formatting and data cleaning, analyze the capital flow data, perform consistency check on the capital data, identify and record inconsistent data, and generate consistency check results; S2: Based on the consistency check result, the financial data is classified by analyzing the amount, timestamp, and account type information of the financial data to generate a financial data classification result; S3: Based on the classification result of the financial data, by comparing the differences of multiple data fields in the financial data, identifying and extracting the incremental data that needs to be synchronized, packaging the target data, and generating an incremental data packet; S4: Based on the incremental data packet, according to the computing resource requirements of the data synchronization task and the real-time system load, the execution time of multiple tasks is predicted, and the synchronization priority of multiple data is calculated in combination with the urgency and sensitivity of multiple transaction types, and a synchronization task priority queue is generated; S5: Based on the synchronization task priority queue, the synchronized data is analyzed to verify the integrity of the data, identify data missing, duplicate, and inconsistent data, and correct abnormal data to generate a financial data synchronization record.

2. The financial data synchronization method according to claim 1, characterized in that: The consistency check result includes cash flow data, data anomaly identification, and timestamp consistency check result; the financial data classification result includes income financial data, expenditure financial data, and accounts payable financial data; the incremental data packet includes incremental data fields, difference data records, and incremental synchronization marks; the synchronous task priority queue includes task execution time estimation information, task priority level, and system load prediction information; the financial data synchronization record includes synchronous data set, data correction record, and synchronous abnormal data detection result.

3. The financial data synchronization method according to claim 1, characterized in that: Based on the input financial data, the data is pre-processed through formatting and data cleaning, the fund flow data is analyzed, the fund data is consistency checked, and inconsistent data is identified and recorded. The specific steps for generating consistency check results are as follows: S101: Based on the input financial data, format and clean the data, including standardizing the amount, account, and timestamp fields, and adjusting the currency unit and time format to generate data preprocessing results; S102: Analyze the capital flow data according to the data preprocessing result, identify non-compliant transaction data by comparing the capital flow direction with financial rules, and generate capital flow analysis results; S103: Based on the fund flow analysis result, verify the consistency of the data, including the amount balance, fund flow rules between accounts, and timestamp sorting, identify and mark abnormal data, and generate a consistency verification result.

4. The financial data synchronization method according to claim 1, characterized in that: Based on the consistency check result, the financial data is classified by analyzing the amount, timestamp, and account type information of the financial data. The steps of generating the financial data classification result are specifically as follows: S201: Based on the consistency check result, extract the amount and time stamp of multiple financial data to generate a basic information extraction result; S202: Based on the basic information extraction result, identify the outflow account and inflow account of funds, analyze the flow direction and business type of funds between accounts, and generate fund type information; S203: Based on the fund type information, multiple financial data are classified, including income, expenditure, and accounts payable, and a financial data classification result is generated.

5. The financial data synchronization method according to claim 1, characterized in that: Based on the financial data classification result, by comparing the differences of multiple data fields in the financial data, identifying and extracting the incremental data that needs to be synchronized, and packaging the target data, the steps of generating the incremental data packet are specifically as follows: S301: Based on the financial data classification result, extract the amount, account, and transaction type field information of multiple data, identify and mark the changed data points through data comparison, and generate changed data identification information; S302: Based on the changed data identification information, identify the incremental data that needs to be synchronized, and generate a synchronized processing data set; S303: Based on the synchronously processed data set, the incremental data is packaged according to the data packet capacity requirement of data transmission to generate an incremental data packet.

6. The financial data synchronization method according to claim 1, characterized in that: Based on the incremental data packet, according to the computing resource requirements of the data synchronization task and the real-time system load, the execution time of multiple tasks is predicted, and the synchronization priority of multiple data is calculated in combination with the urgency and sensitivity of multiple transaction types. The steps of generating the synchronization task priority queue are specifically as follows: S401: Based on the incremental data packet, evaluate the computing resources required for multiple synchronization tasks and generate a resource requirement evaluation result; S402: Based on the resource demand evaluation result and in combination with real-time system load information, the actual execution time of multiple synchronization tasks is evaluated and predicted to generate an execution time prediction result; S403: Based on the execution time prediction result, combined with the urgency and sensitivity of multiple transaction types, the processing priorities of multiple synchronization tasks are calculated to generate a synchronization task priority queue; The specific formula for calculating the processing priority of multiple synchronization tasks is: ； Among them, P represents the priority score of the synchronization task, U represents the urgency feature vector value of the task, S represents the business sensitivity feature vector value, and T represents the standardized predicted execution time. represents the deviation between the current system load and the benchmark load, N represents the number of tasks of the same type waiting to be executed in the current queue, represents the comprehensive weight coefficient of the feature vector, represents the time impact adjustment coefficient, Represents the queue saturation balance factor.

7. The financial data synchronization method according to claim 1, characterized in that: Based on the synchronization task priority queue, the synchronized data is analyzed, the integrity of the data is verified, data missing, duplicated, and inconsistent data are identified, and abnormal data is corrected. The specific steps of generating financial data synchronization records are: S501: Analyze the synchronized data based on the synchronization task priority queue, verify the integrity of the data, and generate data integrity verification information; The specific formula for verifying the integrity of the data is: ； in, represents the data integrity assessment indicator, Indicates the total number of data fields that have completed data synchronization. Represents the total number of data fields to be synchronized. Represents the number of financial data records that were successfully synchronized. Represents the total number of data records synchronized by the target. represents the number of missing data fields detected, Represents the total number of key data fields to be verified, Represents the data timestamp continuity ratio, Represents the data field integrity weight coefficient, Represents the data record integrity weight coefficient, Represents the weight coefficient of missing data impact; S502: Based on the data integrity check information, duplicate data is marked by comparing multiple key fields, including timestamp, amount, and account, and duplicate data identification results are generated; S503: Based on the duplicate data identification result, the abnormal data is corrected according to the type of the abnormal data field to generate a financial data synchronization record.

8. Financial data synchronization system, characterized in that: According to any one of claims 1 to 7, the financial data synchronization method comprises: The data preprocessing module formats and cleans the input financial data. It identifies and records inconsistent data by analyzing the amounts, account balances, and capital flows of multiple financial data, and generates consistency verification results. The data type analysis module extracts the amount, timestamp, and account type information from the financial data based on the consistency check result, analyzes the data according to preset classification rules, classifies the data into multiple types, and generates financial data classification results; The incremental synchronization processing module identifies the incremental data that needs to be synchronized and packages it based on the financial data classification result, by comparing the differences of multiple data fields, to generate an incremental data packet; The task queue adjustment module evaluates and predicts the execution time of multiple tasks based on the incremental data packet, according to the computing resource requirements of the data synchronization task and the real-time system load, calculates the synchronization priority of multiple tasks in combination with the urgency and sensitivity of multiple transaction types, and generates a synchronization task priority queue; The synchronization data correction module analyzes the integrity of the synchronized data based on the synchronization task priority queue, identifies data missing, duplication and data inconsistency, corrects abnormal data, and generates financial data synchronization records.

9. Financial data synchronization device, characterized in that, The financial data synchronization device comprises: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images