Cross-system processing method and device for transaction data, storage medium and electronic equipment

Abstract

The invention discloses a cross-system processing method and device for transaction data, a storage medium and electronic equipment, and relates to the field of financial science and technology. The method comprises the steps that a data processing strategy is determined according to the total data amount of target transaction data in a first system, the data processing strategy is used for determining that the target transaction data is divided into N batches of sub-transaction data, and the cross-system processing time of each batch of sub-transaction data is determined; according to a data processing strategy, copying the target transaction data into a second system in batches through a mirroring state; and when it is detected that fund data difference exists between the transaction data received by the second system and the target transaction data and the difference value of the fund data is smaller than a preset threshold value, fund compensation data is generated according to the difference value and the fund compensation data is stored in the block chain. According to the method and the device, the technical problem of low data migration efficiency during cross-system data migration in the prior art is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of financial technology, in particular, to a cross-system processing method and device for transaction data, a storage medium and an electronic device. BACKGROUND

[0002] In the wave of digital transformation of the financial industry, financial institutions frequently face the need for system upgrade and reconstruction to adapt to the continuous progress of technology. In particular, during the migration of core business systems from old architecture to modern platforms, data migration is considered a crucial link.

[0003] However, in the prior art, there is a significant problem of low data migration efficiency when performing cross-system data migration. For example, traditional data migration methods, including switchover and one-way data migration, are difficult to effectively handle complex queries and batch operations of data when performing large-scale data migration, resulting in a significant decrease in data migration efficiency. Moreover, the prior art does not provide a data compensation mechanism for data migration, and the data correction process and data compensation process rely on manual operation, further reducing data migration efficiency.

[0004] To address the above problems, no effective solutions have been proposed so far. SUMMARY

[0005] The embodiments of the present application provide a cross-system processing method and device for transaction data, a storage medium and an electronic device to at least solve the technical problem of low data migration efficiency in the prior art when performing cross-system data migration.

[0006] According to an aspect of an embodiment of the present application, a cross-system processing method for transaction data is provided, comprising: determining a data processing strategy according to a total amount of data of target transaction data in a first system, wherein the data processing strategy is used to determine the division of the target transaction data into N batches of sub-transaction data, and to determine the cross-system processing time of each batch of sub-transaction data, N being an integer greater than 1; according to the data processing strategy, the target transaction data is batched and copied to a second system through a mirror state, wherein the second system and the first system are independent of each other; in the case where it is detected that there is a fund data difference between the transaction data received by the second system and the target transaction data and the difference value of the fund data is less than a preset threshold, generating fund compensation data according to the difference value and storing the fund compensation data in a blockchain, wherein the blockchain opens access rights to the first system and the second system.

[0007] Optionally, after the target transaction data is batched and copied to the second system in the mirror state according to the data processing strategy, the transaction data received by the second system is taken as to-be-corrected data; it is detected whether there is a difference between the to-be-corrected data and the target transaction data based on the field values in the same field; in the case that it is detected that there is a difference between the to-be-corrected data and the target transaction data based on the field values in any field, an early warning information is generated according to the field values with the difference.

[0008] Optionally, after it is detected that the second system receives the i-th batch of sub-transaction data, the check code of the i-th batch of sub-transaction data read from the second system is taken as a first check code; the check code of the i-th batch of sub-transaction data read from the first system is taken as a second check code, wherein i is a positive integer less than or equal to N; the consistency detection result of the first check code and the second check code is used to determine the replication success rate of the i-th batch of sub-transaction data.

[0009] Optionally, after the target transaction data is batched and copied to the second system in the mirror state according to the data processing strategy, in the case that it is detected that there is a difference between the transaction data received by the second system and the target transaction data in fund data and the difference value of the fund data is greater than or equal to a preset threshold, the transaction data replication operation between the first system and the second system is stopped.

[0010] Optionally, in the process of batched copying the target transaction data to the second system in the mirror state according to the data processing strategy, the replication success rate of each batch of sub-transaction data is detected; in the case that it is detected that the replication success rate of the j-th batch of sub-transaction data is less than a preset success rate, the data processing strategy is adjusted according to the replication success rate of the j-th batch of sub-transaction data, wherein j is a positive integer less than N.

[0011] Optionally, in the case that it is detected that the replication success rate of the j-th batch of sub-transaction data is less than a preset success rate, the data processing strategy is adjusted according to the j-th batch of sub-transaction data, including: the division of batches and the transmission time of each batch of data are re-determined for the data in the target transaction data that has not been replicated to the second system according to the replication success rate of the j-th batch of sub-transaction data, the performance data of the first system and the performance data of the second system; and the data processing strategy is adjusted according to the re-determined transmission time and data amount of each batch of data.

[0012] Optionally, in the process of determining the data processing strategy according to the total data amount of the target transaction data in the first system, the target transaction data is divided into N batches of sub-transaction data according to the total data amount of the target transaction data and the user identifier, the transaction type and the business priority of each transaction detail data in the target transaction data, and the cross-system processing time of each batch of sub-transaction data is determined.

[0013] According to another aspect of the embodiments of the present application, a device for cross-system processing of transaction data is also provided, which comprises: a determination unit configured to determine a data processing strategy according to a total amount of data of target transaction data in a first system, wherein the data processing strategy is used to determine that the target transaction data is divided into N batches of sub-transaction data, and determine a cross-system processing time of each batch of sub-transaction data, N being an integer greater than 1; a replication unit configured to replicate the target transaction data to a second system in batches according to the mirror state in accordance with the data processing strategy, wherein the second system is independent of the first system; and a processing unit configured to generate compensation data for funds according to a difference value of the funds data when it is detected that there is a difference in the funds data between the transaction data received by the second system and the target transaction data and the difference value of the funds data is less than a preset threshold, and store the compensation data for funds in a blockchain, wherein the blockchain has open access to the first system and the second system.

[0014] According to another aspect of the embodiments of the present application, a computer readable storage medium is also provided, wherein the computer readable storage medium stores a computer program, and when the computer program is executed, the computer readable storage medium causes the device in which the computer readable storage medium is located to perform the cross-system processing method of transaction data.

[0015] According to another aspect of the embodiments of the present application, an electronic device is also provided, which comprises one or more processors and a memory, and the memory is configured to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors perform the cross-system processing method of transaction data.

[0016] According to another aspect of the embodiments of the present application, a computer program product is also provided, wherein the computer program product comprises a computer program or instructions, and when the computer program or instructions are executed by a processor, the cross-system processing method of transaction data is implemented.

[0017] In the present application, first, a data processing strategy is determined according to a total amount of data of target transaction data in a first system, wherein the data processing strategy is used to determine that the target transaction data is divided into N batches of sub-transaction data, and determine a cross-system processing time of each batch of sub-transaction data, N being an integer greater than 1. Then, the target transaction data is replicated to a second system in batches according to the mirror state in accordance with the data processing strategy, wherein the second system is independent of the first system. When it is detected that there is a difference in the funds data between the transaction data received by the second system and the target transaction data and the difference value of the funds data is less than a preset threshold, compensation data for funds is generated according to the difference value of the funds data, and the compensation data for funds is stored in a blockchain, wherein the blockchain has open access to the first system and the second system.

[0018] From the above, by performing total amount evaluation on the target transaction data of the first system, the data processing strategy of the application can intelligently divide the target transaction data into N small batches of sub-transaction data, thereby optimizing the cross-system processing time. This strategy avoids the system burden and efficiency bottleneck caused by one-time large volume data migration in traditional methods, and improves the throughput and speed of data migration. And using the mirror state replication technology to batch copy the target transaction data of the first system to the second system ensures the real-time synchronization of data between the two systems, effectively reduces the error rate in the data migration process, and improves the accuracy and reliability of data migration.

[0019] Secondly, when detecting that there is a fund difference between the data received by the second system and the target transaction data, the application sets a preset threshold, and when the difference value is lower than the threshold, the fund compensation data is generated immediately and stored through the blockchain, realizing the safe sharing and transparent auditing of the compensation data. This not only reduces the fund error caused by common problems such as decimal point processing difference, but also simplifies the compensation process through the decentralized blockchain technology, avoids the cumbersome steps in traditional data correction, and greatly improves the efficiency of data migration.

[0020] Therefore, the data migration technology based on mirror state replication and data processing strategy optimization proposed in the application can significantly improve the efficiency of cross-system data migration, reduce the risk of data inconsistency, realize the rapid identification and compensation of fund difference, and effectively guarantee the business continuity and user experience. BRIEF DESCRIPTION OF DRAWINGS

[0021] The drawings described herein are used to provide further understanding of the application, and form a part of the application. The illustrative embodiments of the application and their descriptions are used to explain the application, and do not constitute an improper limitation on the application. In the drawings:

[0022] Figure 1 is a flowchart of an optional cross-system processing method for transaction data according to an embodiment of the application;

[0023] Figure 2 is a schematic diagram of an optional data migration system according to an embodiment of the application;

[0024] Figure 3 is a schematic diagram of an optional cross-system processing device for transaction data according to an embodiment of the application. DETAILED DESCRIPTION

[0025] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application, so that those skilled in the art can better understand the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first", "second" and the like in the description and claims of the present application and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in other than the order illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product or device.

[0027] It should also be noted that the information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) collected by the present application are authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards in relevant regions, necessary security measures are taken, public order is not violated, and appropriate operation portals are provided for users to choose authorization or refusal. For example, interfaces are provided between the system and related users or agencies, and before obtaining the relevant information, the interface needs to send a request to the aforementioned user or agency, and after receiving the consent information feedback from the aforementioned user or agency, the relevant information is obtained.

[0028] According to the embodiments of the present application, an embodiment of a cross-system processing method of transaction data is provided. It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in a different order.

[0029] According to the embodiment of the present application, a data migration system can be used as the execution subject of the cross-system processing method of transaction data of the embodiment of the present application, wherein the system can be a software system or a combination of software and hardware embedded system. Of course, the execution subject of the method in the embodiment of the present application can also be other forms of execution subject, such as device, equipment, etc. Those skilled in the art should know that the embodiment of the present application does not particularly limit the specific form of the execution subject.

[0030] Figure 1 is a flowchart of an optional cross-system processing method of transaction data according to the embodiment of the present application, as shown in Figure 1 The method comprises the following steps:

[0031] In step S101, the data processing strategy is determined according to the total amount of the target transaction data in the first system.

[0032] In step S101, the data processing strategy is used to determine the division of the target transaction data into N batches of sub-transaction data, and the cross-system processing time of each batch of sub-transaction data, N is an integer greater than 1.

[0033] Firstly, the data migration system can evaluate the total amount of the target transaction data in the first system (usually refers to the original, running system). Among them, this process includes but is not limited to the calculation of various data such as historical transaction records, user account information, transaction state changes, aiming to fully understand the size and complexity of the data to be migrated. On the basis of determining the total amount of data, the data processing strategy can be determined, and the key of the strategy is to divide the target transaction data into N batches of sub-transaction data, N is an integer greater than 1. This division is not random, but can be intelligently decided according to the characteristics of the data (such as type, size, processing priority), the processing capacity of the second system (new system), the network transmission condition and the business continuity requirement and other factors.

[0034] By dividing the total amount of data into N sub-batches, parallel processing can be realized, greatly shortening the total migration time. The size of each batch needs to be carefully designed, which not only ensures the effectiveness and consistency of data migration, but also maximizes the use of system resources, avoids frequent start caused by too small data volume and system pressure caused by too large data volume. In addition, the cross-system processing time of each batch of sub-transaction data between the first system and the second system is determined, which can realize the time planning of the whole data migration process. For example, the determination of processing time can consider the data transmission rate, the data processing speed of the second system, and the possible data checking and repairing time. By estimating the completion time of the whole data migration, real-time monitoring and adjustment strategies can be made for each batch of migration, so as to ensure the efficiency and stability of the migration process.

[0035] In step S102, the target transaction data is copied in batches to the second system according to the data processing strategy, wherein the second system is independent of the first system.

[0036] Optionally, the core of the data processing strategy is to divide the large-scale data set into multiple smaller and more manageable subsets, i.e., N batches of sub-transaction data. Such an intelligent strategy not only improves the efficiency of data processing, but also enables more precise monitoring and more timely error feedback during data migration, ensuring smooth data migration.

[0037] Optionally, mirror state replication is a data migration technique that ensures data state consistency between the two systems during data migration by replicating data changes in the first system to the second system in real time. When a transaction is processed in the first system, the data changes of the transaction are captured and immediately replicated in the second system. This process can be regarded as a real-time "mirror" of the data.

[0038] Optionally, according to the data processing strategy, the target transaction data is subdivided into N batches for replication. Each batch of replication is independent and can be executed in parallel, which greatly improves the efficiency of data migration. At the same time, by replicating in batches, the pace of data migration can be better controlled, avoiding resource bottlenecks caused by a large amount of data flowing into the second system at once. In addition, the first system and the second system remain completely independent during data migration. This means that even during data migration, the first system can still handle new transaction requests, while the second system can simultaneously receive and process transaction data mirrors from the first system. This independence ensures that data migration does not affect the normal business of the first system, and also allows the second system to perform independent testing and optimization during migration, improving the stability and security of the entire system.

[0039] From the above, by batch replication, the present application can significantly improve the efficiency of data migration and shorten the overall migration time. The mirror state replication technology ensures real-time consistency of data between the first system and the second system, reducing the risk of data inconsistency. The dynamic adjustment of the data processing strategy makes the entire data migration process more flexible, allowing real-time optimization of batch size and timing of data migration according to actual conditions. The maintenance of system independence ensures the stable operation of the first system and the second system during data migration, avoiding business interruption and ensuring user experience.

[0040] In summary, according to the data processing strategy, the mirror state batch replication technology is used to migrate the target transaction data from the first system to the second system, which not only improves the efficiency and accuracy of data migration, but also maintains the independent operation and stability of the system.

[0041] Step S103, in the case that the fund data difference between the transaction data received by the second system and the target transaction data is detected and the difference value of the fund data is less than the preset threshold, generating fund compensation data according to the difference value and storing the fund compensation data in the blockchain.

[0042] In step S103, the blockchain opens access permission to the first system and the second system.

[0043] Optionally, during the data migration process, especially when financial transaction data is involved, the data processing methods between different systems may cause slight fund data differences. For example, the first system (old system) may use an A-type database, while the second system (new system) uses a B-type database. The mechanism differences between the two in numerical operation and decimal point processing may cause slight deviations in transaction amounts. In order to ensure the consistency of fund data after data migration, the technical solution of the present application provides a real-time fund data difference detection mechanism. Once it is detected that there is a fund data difference between the transaction data in the second system and the target transaction data, and the difference value is controlled within the preset threshold (for example, a very small amount of money, which is not enough to affect the substantive validity of the transaction), the system will automatically generate fund compensation data to correct the difference. The generation of such fund compensation data is based on the accurate calculation of the difference value and the in-depth understanding of the transaction context, ensuring that the compensation operation is accurate and reasonable.

[0044] The generated fund compensation data will be stored in the blockchain, which is a decentralized and distributed database technology with high security, transparency and non-tamperability. The fund compensation data stored in the blockchain can be accessed by the first system and the second system, ensuring the consistency and traceability of the data. The application of blockchain technology not only solves the security problem of data storage, but also provides a unified data access approach, facilitating the collaborative work between the two systems.

[0045] As can be seen, through real-time fund data difference detection and automatic compensation, the present application can effectively eliminate the fund data deviation that may occur during the data migration process, ensuring the consistency of fund data between the old and new systems. With the distributed storage and encryption technology of the blockchain, the storage and access of fund compensation data become more secure and reliable, reducing the risk of data tampering and loss. The fund compensation data stored in the blockchain provides complete records and support for subsequent auditing and tracking, which is conducive to compliance management and internal audit. The open access permission feature of the blockchain promotes data sharing and business collaboration between the first system and the second system, improving the operation efficiency and response speed of the entire system.

[0046] In an alternative embodiment, as Figure 2As shown in the figure, the data migration system consists of four modules: dual-track parallel engine module, data lossless migration module, intelligent circuit breaker module and grayscale release module.

[0047] Optionally, the dual-track parallel engine module is used to build a hybrid architecture in which the old and new systems run in parallel, dynamically allocating transaction traffic through an intelligent routing controller (for example, initially 1% of the traffic is directed to the new system). The dual-track parallel engine module can also employ transaction mirroring technology to copy each transaction from the old system to the new system in real time for execution, and record difference logs to enable field-level data comparison.

[0048] Optionally, the data lossless migration module can implement the following three levels of data verification mechanisms:

[0049] Static data migration mechanism: can realize batch data comparison based on CRC32 check code;

[0050] Dynamic data synchronization mechanism: can capture incremental data in real time;

[0051] Business logic verification mechanism: The accuracy of new system calculations can be verified through historical transaction playback.

[0052] Optionally, the intelligent circuit breaker module can be used to establish multi-dimensional monitoring indicators (including but not limited to response time, error code distribution, and fund verification differences). For example, when it is detected that the difference in the amount of five consecutive transactions is greater than 0.01 yuan or the success rate is less than 99.99%, it will automatically trigger the operation of switching traffic back to the old system.

[0053] Optionally, the grayscale release module is used to provide a visual configuration interface, which can gradually route transaction requests to the new system at a ratio of 1% to 100% based on dimensions such as user ID hash value, business priority, and transaction type, and monitor the new system performance indicators (such as TPS, response time, and error code) in real time to dynamically adjust the diversion ratio.

[0054] Optionally, the implementation steps of the intelligent routing controller can refer to the following code example:

[0055] class TrafficRouter:

[0056] def__init__(self,old_sys,new_sys):

[0057] self.gray_ratio = 1 # Initial grayscale ratio 1%;

[0058] self.error_count = 0;

[0059] def route_transaction(self,tx):

[0060] # Decide routing path according to gray scale rule

[0061] if tx.tx_type=="QUERY"and random.random()<self.gray_ratio:

[0062] result=new_sys.execute(tx);

[0063] if self.validate_result(tx,result):

[0064] return result;

[0065] else:

[0066] self.error_count+=1;

[0067] self.adjust_gray_ratio();

[0068] return old_sys.execute(tx);

[0069] else:

[0070] return old_sys.execute(tx);

[0071] def adjust_gray_ratio(self):

[0072] # Dynamically adjust gray scale ratio based on error rate

[0073] if self.error_count>5:

[0074] self.gray_ratio=max(0,self.gray_ratio-0.2);

[0075] else:

[0076] self.gray_ratio=min(1,self.gray_ratio+0.05);

[0077] In an alternative embodiment, after the target transaction data is batched and replicated to the second system according to the data processing strategy, the data migration system can treat the transaction data received by the second system as the data to be reconciled, and then detect whether there is a difference between the field values under the same field between the data to be reconciled and the target transaction data. In the case where it is detected that there is a difference between the field values under any field between the data to be reconciled and the target transaction data, the data migration system generates a warning message according to the difference in the field values.

[0078] Optionally, after the target transaction data in the first system is batched and migrated to the second system using the mirror state replication technology, the data migration system treats the transaction data received by the second system as the data to be reconciled. The data difference detection is a fine comparison based on the field level, which means that the data migration system will check each field in each transaction data one by one, such as the amount of money, the transaction time, the account ID, etc., to confirm whether the values of these fields in the first system and the second system are completely consistent. Compared with the traditional rough comparison, this field-level detection method can more accurately and comprehensively identify data differences, ensuring the quality of data migration.

[0079] Optionally, when the data migration system detects that there is a difference between the field values under any field between the data to be reconciled and the target transaction data, it will automatically generate a warning message according to these difference field values. The generation of the warning message is part of the system's automatic response mechanism, aiming to immediately notify the system administrator or the data migration team, and timely identify and locate the problem of data difference. The triggering of the warning mechanism is not limited to the difference in the amount of money, but also includes any field difference that may affect the integrity and accuracy of the transaction, such as timestamp, status code, etc., so as to ensure that problems in the data migration process can be quickly discovered and handled, avoiding the accumulation and spread of data inconsistency.

[0080] In an alternative embodiment, after detecting that the second system receives the ith batch of sub-transaction data, the data migration system reads the check code of the ith batch of sub-transaction data from the second system as the first check code, and reads the check code of the ith batch of sub-transaction data from the first system as the second check code, where i is a positive integer less than or equal to N. Then, the data migration system determines the replication success rate of the ith batch of sub-transaction data according to the consistency detection result of the first check code and the second check code.

[0081] Optionally, the data migration system performs a data consistency detection based on the check code after each batch of sub-transaction data is replicated from the first system to the second system. The check code is a value calculated by a specific algorithm, which is used to verify the integrity and consistency of the data. The specific implementation process is as follows:

[0082] The data migration system first reads the ith batch of sub-transaction data from the second system and generates a first check code based on the data. The check code algorithm used here (such as CRC32, MD5, or SHA, etc.) needs to ensure that it can cover all the characteristics of the data, so as to accurately reflect the state of the data. For comparison, the data migration system also obtains the check code of the ith batch of sub-transaction data from the first system as a second check code. The purpose of generating the second check code is to provide a benchmark for comparing the first check code to verify whether the data remains consistent during migration.

[0083] Optionally, after the generation of the first check code and the second check code is completed, the data migration system will compare the values of the two check codes. If the first check code and the second check code are completely consistent, it can be determined that the replication success rate of the ith batch of sub-transaction data is 100%, that is, no changes or damages have occurred during the data migration process, and the data remains consistent between the first system and the second system. On the contrary, if the first check code and the second check code are inconsistent, it means that errors have occurred during the data migration process, such as data damage, loss, or format conversion problems, etc. At this time, the replication success rate will be marked as less than 100%, and the system will further analyze the nature and scope of the differences and take appropriate data repair measures, such as re-replicating the batch of data, adjusting the data conversion logic, or executing the data repair script, etc., to ensure the final consistency of the data.

[0084] From the above, through the consistency detection of the check code, the present application can timely discover and correct any differences in the data migration process, thereby ensuring the integrity of the data in the new and old systems. The rapid generation and comparison of the check code avoids the large amount of time and computing resources consumed by traditional data comparison, improving the efficiency of data migration.

[0085] Secondly, the check code comparison can be performed immediately after each batch of data migration, realizing real-time monitoring of data migration, and any inconsistency can be timely fed back and processed, reducing the risk of data migration. Moreover, the batch-based check code comparison allows the data problems to be refined to specific batches, facilitating quick positioning and repair of problems, improving the accuracy and speed of problem handling.

[0086] In an optional embodiment, after the target transaction data is replicated to the second system in batches according to the data processing strategy, in the case that the fund data difference between the transaction data received by the second system and the target transaction data is detected and the fund data difference value is greater than or equal to a preset threshold, the data migration system can stop the transaction data replication operation between the first system and the second system.

[0087] Optionally, during the parallel running of the dual systems, the data migration system continuously monitors the consistency of transaction data between the second system and the first system, for example, the integrity and accuracy of the fund data. Due to the sensitivity and importance of fund data, even a small difference can have a significant impact on business. Therefore, the system sets a preset threshold to determine whether the fund data difference is within an acceptable range. The preset threshold is to draw a clear line during data migration, distinguishing between normal and negligible differences (such as minor fluctuations after decimal points) and abnormal differences that may lead to serious consequences. The setting of this threshold needs to consider business needs, risk management strategies, and system fault tolerance capabilities, etc., to ensure that when fund data differences beyond the tolerance range occur, a timely response can be made.

[0088] Optionally, once the data migration system detects a fund data difference between the transaction data received by the second system and the target transaction data in the first system, and the difference value reaches or exceeds the preset threshold, the system will automatically stop the ongoing transaction data replication operation between the first system and the second system. This is to avoid further spread of potential problems, protect the data security of the second system, and prevent business anomalies that may be caused by data differences.

[0089] Optionally, after stopping data replication, the data migration system needs to immediately conduct in-depth analysis of the difference causes, which may include errors in the data conversion process, system configuration differences, third-party interface problems, etc. Once the problem is located, appropriate repair measures should be taken, such as adjusting data processing logic, correcting data, or restarting the data migration process, etc. If the difference cannot be immediately repaired, or indicates a deeper system compatibility problem, the data migration system may need to perform partial or full data rollback operations to restore the second system to its pre-migration state to prevent business risks caused by data inconsistency.

[0090] In an optional embodiment, during the process of copying the target transaction data to the second system in batches according to the data processing strategy, the data migration system can detect the replication success rate of each batch of sub-transaction data. In the case where the replication success rate of the jth batch of sub-transaction data is less than the preset success rate, the data processing strategy is adjusted according to the replication success rate of the jth batch of sub-transaction data, where j is a positive integer less than N.

[0091] Optionally, while performing the data migration operation, the data migration system tracks the success rate of each sub-transaction data copy from the first system to the second system, i.e., whether the copy was completed completely and without errors. For the jth batch of sub-transaction data, the system calculates its copy success rate, which can be based on verifying the integrity and consistency of the data before and after the copy. The high or low copy success rate reflects the quality and efficiency of the data migration for that batch.

[0092] Optionally, the preset success rate is used to determine whether the data migration quality meets expectations. This metric can be set by the data migration team based on a combination of factors, including business requirements, data complexity, and system performance. Its purpose is to establish a quality control baseline during the data migration process, ensuring that each batch of data migration meets this standard. If the data migration system detects that the replication success rate of the jth batch of sub-transaction data is lower than the preset success rate, this indicates that the current data processing strategy may have encountered certain issues or limitations, such as data format mismatch, insufficient system resources, or network latency, resulting in reduced data migration efficiency and accuracy.

[0093] To address this situation, the data migration system dynamically adjusts its data processing strategy based on the success rate of the jth batch replication. This adjustment may include: if data inconsistency is due to data format conversion issues, the system can optimize the conversion logic to improve data compatibility and consistency; if the low replication success rate is due to excessive batch data volume, resulting in system resource constraints, the system can reduce the batch size and complete the data migration in multiple steps to avoid resource contention; if network latency or packet loss is the root cause, the system can implement network transmission optimization measures, such as using more efficient data compression algorithms and adjusting data transmission priorities, to reduce errors during data transmission.

[0094] From the above, it can be seen that by dynamically adjusting the data processing strategy, this application can promptly identify and resolve problems in the data migration process, thereby improving the overall quality of data migration and ensuring accurate and lossless data transmission. This mechanism increases the flexibility and adaptability of the data migration system, enabling it to cope with various complex migration scenarios, including large amounts of data, diverse data structures, and system performance fluctuations. Dynamic adjustment strategies help to more reasonably allocate and utilize system resources, avoid inefficiencies caused by excessive concentration of resources in a certain batch, and achieve efficient resource utilization.

[0095] In an optional embodiment, in the case where it is detected that the replication success rate of the jth batch of sub-transaction data is less than the preset success rate, the data processing strategy is adjusted according to the jth batch of sub-transaction data, including: the data migration system can re-determine the division of batches and the transmission time of each batch of data for the data in the target transaction data that has not been replicated to the second system according to the replication success rate of the jth batch of sub-transaction data, the performance data of the first system, and the performance data of the second system. Then, the data migration system can adjust the data processing strategy according to the re-determined transmission time and data volume of each batch of data.

[0096] Optionally, when the data migration system monitors that the replication success rate of the jth batch of sub-transaction data is lower than the predetermined threshold, the system can immediately conduct in-depth analysis on this situation. The content of the analysis not only covers the specific reasons for the replication failure (such as data format mismatch, network delay, insufficient system resources, etc.), but also involves the evaluation of the performance data of the first system and the second system, including but not limited to CPU utilization, memory occupation, disk I / O, network bandwidth, and other key indicators. Through comprehensive analysis of these data, the reasons affecting the replication success rate can be identified.

[0097] According to the above analysis results, the data migration system will adjust the subsequent data processing strategy, specifically including: if the analysis determines that the replication success rate is reduced due to unreasonable batch division, the data migration system can re-evaluate the size, type, and complexity of the remaining data, and adjust the batch division strategy. For example, large batches can be split into smaller sub-batches to reduce the pressure of single data migration; or in the case of complex data structure and difficult format conversion, the batch size can be appropriately increased to improve efficiency through batch processing. The data migration system can also reasonably plan the transmission time window of each batch of data according to the real-time performance data of the first system and the second system. For example, if it is detected that the replication success rate is low during the resource peak period of the first system or the second system, data migration can be performed during the off-peak period to reduce competition between systems and improve the success rate of data transmission.

[0098] Optionally, after the above analysis and planning are completed, the data migration system will adjust the data processing strategy according to the re-determined transmission time and data volume of each batch of data, including but not limited to: updating the data migration plan, optimizing the data conversion logic, adjusting the system resource allocation, or improving the network transmission mechanism, and a series of specific optimization measures.

[0099] It should be noted that through the strategy adjustment, the application can solve the key problems affecting the replication success rate, thereby improving the accuracy and reliability of subsequent data migration. Moreover, dynamic strategy adjustment helps to more reasonably utilize system resources, avoid resource waste, and reduce data migration failures caused by resource shortage.

[0100] In an optional embodiment, in the process of determining the data processing strategy according to the total amount of target transaction data in the first system, the data migration system can further divide the target transaction data into N batches of sub-transaction data according to the total amount of target transaction data and the user identifier, transaction type and business priority of each transaction detail data in the target transaction data, and determine the cross-system processing time of each batch of sub-transaction data.

[0101] Optionally, when planning the data processing strategy, the data migration system primarily considers the total amount of target transaction data. The total amount of data is a key indicator that directly affects the time, required resources and potential risks of data migration. The system will preliminarily judge whether batch processing is needed according to the size of the total amount of data, as well as the number and size of batches, to ensure that the data migration process is both efficient and stable. Based on the total amount of data, the data migration system will further analyze each transaction detail data in the target transaction data. These analyses are based on attributes such as user identifier, transaction type and business priority of transaction detail data, which can help identify which users' transaction data need to be processed first, especially for VIP users or high-risk users, who may require higher data migration priority to ensure the safety and business continuity of these key users' data. Secondly, different transaction types may involve different data processing complexity and resource requirements. For example, transfer transactions may require additional compliance checks, while query transactions are relatively simple. The data migration system can arrange the order of data migration and resource allocation according to the characteristics of transaction types. In addition, considering the diversity and urgency of financial business, the data migration system can adjust the data migration strategy according to the business priority. For example, high-priority business data needs to be migrated faster and more securely to avoid affecting the normal operation of key businesses.

[0102] Based on the above analysis, the data migration system will divide the target transaction data into N batches of sub-transaction data. The division of each batch not only balances the data volume, but also considers user identifier, transaction type and business priority to ensure fairness and reasonableness of data migration.

[0103] Next, the system will determine the cross-system processing time of each batch of sub-transaction data. This decision takes into account several factors, including:

[0104] Assess the processing capacity of the first and second systems to ensure that data migration does not cause excessive stress on either system.

[0105] Consider the stability of network transmission during data migration to reasonably plan the migration time of each batch and avoid large data migration during network congestion periods.

[0106] Ensure that financial services are not affected during data migration, especially during periods of active financial market trading, which requires avoiding large-scale data migration activities.

[0107] According to another aspect of the embodiments of the present application, a cross-system processing device for transaction data is also provided, wherein Figure 3 is a schematic diagram of a cross-system processing device for transaction data according to an embodiment of the present application, as Figure 3 shown, the device comprises a determination unit 301, a replication unit 302 and a processing unit 303.

[0108] The determination unit 301 is configured to determine a data processing strategy according to the total amount of target transaction data in the first system, wherein the data processing strategy is used to determine the division of the target transaction data into N batches of sub-transaction data and the cross-system processing time of each batch of sub-transaction data, and N is an integer greater than 1. The replication unit 302 is configured to replicate the target transaction data in batches through a mirror state to the second system according to the data processing strategy, wherein the second system and the first system are independent of each other. The processing unit 303 is configured to, in the case that a fund data difference exists between the transaction data received by the second system and the target transaction data and the difference value of the fund data is less than a preset threshold, generate fund compensation data according to the difference value and store the fund compensation data in a blockchain, wherein the blockchain has open access rights to the first system and the second system.

[0109] Optionally, the cross-system processing device for transaction data further comprises a first processing unit configured to take the transaction data received by the second system as to-be-proofread data; a detection unit configured to detect whether there is a difference between the to-be-proofread data and the target transaction data based on the field values in the same field; and a second processing unit configured to, in the case that a difference exists between the to-be-proofread data and the target transaction data based on the field values in any field, generate warning information according to the field values with the difference.

[0110] Optionally, the cross-system transaction data processing apparatus further comprises a third processing unit configured to read the check code of the i-th batch of sub-transaction data from the second system as a first check code after detecting that the second system receives the i-th batch of sub-transaction data, and read the check code of the i-th batch of sub-transaction data from the first system as a second check code, wherein i is a positive integer less than or equal to N; and a first determination unit configured to determine the replication success rate of the i-th batch of sub-transaction data according to the consistency detection result of the first check code and the second check code.

[0111] Optionally, the cross-system transaction data processing apparatus further comprises a fourth processing unit configured to stop the transaction data replication operation between the first system and the second system in a case where the fund data difference between the transaction data received by the second system and the target transaction data exists and the fund data difference value is greater than or equal to a preset threshold.

[0112] Optionally, the cross-system transaction data processing apparatus further comprises a first detection unit configured to detect the replication success rate of each batch of sub-transaction data; and an adjustment unit configured to adjust the data processing strategy according to the replication success rate of the j-th batch of sub-transaction data in a case where the replication success rate of the j-th batch of sub-transaction data is less than a preset success rate, wherein j is a positive integer less than N.

[0113] Optionally, the adjustment unit comprises a determination sub-unit configured to re-determine the batch division and the transmission time of each batch of data for the data in the target transaction data that is not replicated to the second system according to the replication success rate of the j-th batch of sub-transaction data, the performance data of the first system, and the performance data of the second system; and an adjustment sub-unit configured to adjust the data processing strategy according to the re-determined transmission time of each batch of data and the data amount.

[0114] Optionally, the cross-system transaction data processing apparatus further comprises a second determination unit configured to divide the target transaction data into N batches of sub-transaction data according to the total data amount of the target transaction data and the user identifier, the transaction type, and the business priority of each transaction detail data in the target transaction data, and determine the cross-system processing time of each batch of sub-transaction data.

[0115] According to another aspect of the embodiments of the present application, a computer readable storage medium is also provided, wherein the computer readable storage medium stores a computer program, and when the computer program runs, the computer readable storage medium makes the device on which the computer readable storage medium is located execute the cross-system transaction data processing method.

[0116] According to another aspect of the embodiments of the present application, an electronic device is provided, which includes one or more processors and a memory, the memory being configured to store one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the cross-system processing method of transaction data.

[0117] According to another aspect of the embodiments of the present application, a computer program product is provided, which includes computer programs or instructions, and the computer programs or instructions, when executed by a processor, implement the cross-system processing method of transaction data.

[0118] The serial numbers of the embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0119] In the above-described embodiments of the present application, the description of each embodiment focuses on different aspects, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0120] In the several embodiments provided by the present application, it should be understood that the disclosed technology can be implemented in other ways. Of course, the device embodiment described above is only illustrative, and for example, the division of the units can be a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, unit or module, and can be electrical or other forms.

[0121] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple units. Part or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0122] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0123] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or say the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0124] The above is only the preferred embodiment of the present application, and it should be pointed out that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should be considered as the protection scope of the present application.

Claims

1. A cross-system processing method for transaction data, characterized in that: include: Determining a data processing strategy based on the total amount of target transaction data in the first system, wherein the data processing strategy is used to determine how to divide the target transaction data into N batches of sub-transaction data and determine a cross-system processing time for each batch of sub-transaction data, where N is an integer greater than 1; According to the data processing strategy, the target transaction data is copied in batches to a second system in a mirrored state, wherein the second system is independent of the first system; When it is detected that there is a difference in financial data between the transaction data received by the second system and the target transaction data and the difference value of the financial data is less than a preset threshold, financial compensation data is generated according to the difference value and the financial compensation data is stored in the blockchain, wherein the blockchain opens access rights to the first system and the second system.

2. The method according to claim 1, characterized in that After copying the target transaction data in batches to the second system in a mirrored state according to the data processing strategy, the method further includes: using the transaction data received by the second system as data to be verified; Detecting whether there are differences between the data to be collated and the target transaction data based on field values ​​under the same field; When a difference is detected between the data to be collated and the target transaction data based on a field value under any field, an early warning message is generated based on the field value with the difference.

3. The method according to claim 1, characterized in that After copying the target transaction data in batches to the second system in a mirrored state according to the data processing strategy, the method further includes: After detecting that the second system has received the i-th batch of sub-transaction data, using the verification code of the i-th batch of sub-transaction data read from the second system as a first verification code; and using the verification code of the i-th batch of sub-transaction data read from the first system as a second verification code, where i is a positive integer less than or equal to N; The success rate of copying the i-th batch of sub-transaction data is determined according to a consistency test result of the first verification code and the second verification code.

4. The method according to claim 1, wherein After copying the target transaction data in batches to the second system in a mirrored state according to the data processing strategy, the method further includes: When it is detected that there is a fund data difference between the transaction data received by the second system and the target transaction data and the difference value of the fund data is greater than or equal to the preset threshold, the transaction data copying operation between the first system and the second system is stopped.

5. The method according to claim 1, wherein In the process of copying the target transaction data in batches to the second system in a mirrored state according to the data processing strategy, the method further includes: Check the replication success rate of each batch of sub-transaction data; When it is detected that the replication success rate of the j-th batch of sub-transaction data is less than the preset success rate, the data processing strategy is adjusted according to the replication success rate of the j-th batch of sub-transaction data, where j is a positive integer less than N.

6. The method according to claim 5, characterized in that When it is detected that the replication success rate of the j-th batch of sub-transaction data is less than a preset success rate, adjusting the data processing strategy according to the j-th batch of sub-transaction data includes: Re-determining the batches and the transmission time of each batch of data for the data in the target transaction data that has not been copied to the second system based on the replication success rate of the j-th batch of sub-transaction data, the performance data of the first system, and the performance data of the second system; The data processing strategy is adjusted according to the re-determined transmission time and data volume of each batch of data.

7. The method according to claim 1, characterized in that In the process of determining the data processing strategy according to the total amount of target transaction data in the first system, the method further includes: According to the total amount of the target transaction data and the user identification, transaction type and business priority corresponding to each transaction detail data in the target transaction data, the target transaction data is divided into N batches of sub-transaction data, and the cross-system processing time of each batch of sub-transaction data is determined.

8. A cross-system processing device for transaction data, characterized in that: include: a determining unit, configured to determine a data processing strategy based on the total amount of target transaction data in the first system, wherein the data processing strategy is configured to determine division of the target transaction data into N batches of sub-transaction data and to determine a cross-system processing time for each batch of sub-transaction data, where N is an integer greater than 1; a replication unit, configured to replicate the target transaction data in batches to a second system in a mirrored state according to the data processing strategy, wherein the second system is independent of the first system; A processing unit is configured to, upon detecting that there is a difference in fund data between the transaction data received by the second system and the target transaction data and that the difference in fund data is less than a preset threshold, generate fund compensation data based on the difference value and store the fund compensation data in a blockchain, wherein the blockchain opens access rights to the first system and the second system.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located executes the cross-system processing method for transaction data according to any one of claims 1 to 7.

10. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the cross-system processing method for transaction data as described in any one of claims 1 to 7.

11. A computer program product, characterized in that The method comprises a computer program or an instruction, which, when executed by a processor, implements the cross-system processing method for transaction data according to any one of claims 1 to 7.

Images