Verification method and verification device for data migration, storage medium and electronic equipment

Abstract

The invention provides a verification method and a verification device for data migration, a storage medium and electronic equipment. The method comprises the following steps: determining to adopt a substitute authorization system to process a task transaction; after the data migration from the original system to the target system is completed by adopting the first migration system, carrying out data migration from the substitute authorization system to the target system by adopting a second migration system; acquiring a transaction data table of the substitute authorization system and substitute authorization migration data in the target system under the condition that the data volume migrated from the substitute authorization system to the target system reaches a first preset data volume; the verification tool is adopted to compare the data in the transaction data table of the authorization system with the authorization migration data in the target system for multiple times, a comparison result is obtained, and the comparison result comprises the similarity between the data in the transaction data table of the authorization system and the authorization migration data in the target system. According to the method, the migration quality of real-time production data by the authorization system can be comprehensively verified, and the system switching confidence is stabilized.

Description

Technical Field

[0001] This application relates to the field of data storage technology, and more specifically, to a data migration verification method, a data migration verification device, a computer-readable storage medium, and an electronic device. Background Technology

[0002] During the upgrade and switchover of the bank card core system, to ensure transaction continuity, it is necessary to migrate existing data from the original system to the target system and perform transaction verification and other processes before the target system is put into operation. System downtime during this period would impact cardholder user experience; therefore, a proxy authorization system is needed to provide services to cardholders and achieve a seamless migration. After the data from the original system is migrated to the target system, it is also necessary to promptly retroactively update transactions generated during the proxy authorization period in the target system and verify the consistency of the retroactive data to ensure the subsequent availability of the target system.

[0003] When switching between the old and new core systems for bank cards, system downtime during the migration of existing data from the original system to the target system will impact cardholders' user experience. Therefore, a proxy authorization system is needed to serve cardholders. The traffic handled by the proxy authorization system needs to be promptly updated in the target system, and the data's correctness must be verified after being stored in the database. Switching to the target system only occurs after successful data verification. During the proxy authorization data migration process, data accuracy is maintained as much as possible, but worst-case scenarios are still considered; therefore, this data verification method and device are designed.

[0004] For existing bank card core system migration and switching solutions that use a proxy authorization system, the current verification of the quality of real-time transaction data migration mainly relies on manual traffic verification, which is difficult to fully verify and also makes it difficult to ensure that the data processed by the authorization system in the production era can be correctly traced back to the target system. Summary of the Invention

[0005] The main objective of this application is to provide a data migration verification method, a data migration verification device, a computer-readable storage medium, and an electronic device, so as to at least solve the problem that existing methods for verifying the quality of real-time transaction data migration are unable to guarantee the sufficiency of verification and the accuracy of data migration.

[0006] To achieve the above objectives, according to one aspect of this application, a data migration verification method is provided, comprising: determining that a proxy authorization system will be used to process task transactions; after data migration from the original system to the target system is completed using a first migration system, performing data migration from the proxy authorization system to the target system using a second migration system; when the amount of data migrated from the proxy authorization system to the target system reaches a first preset amount of data, obtaining the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system; and using a verification tool to repeatedly compare the data in the transaction data table of the proxy authorization system with the proxy authorization migration data in the target system to obtain a comparison result, wherein the comparison result includes the similarity between the data in the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system.

[0007] Optionally, a verification tool is used to repeatedly compare the data in the transaction data table of the proxy authorization system with the proxy authorization migration data in the target system to obtain comparison results. This includes: filtering out successfully traded transaction data from the transaction data table to obtain target data; using the verification tool to convert the initial structured language of the target data into a target structured language to obtain format-converted target data, where the target structured language is a data structured language that the target system can recognize; splitting the comparison task into multiple sub-tasks, where each comparison task is to compare the format-converted target data with the corresponding proxy authorization migration data, and each sub-task includes a portion of the format-converted target data and the corresponding proxy authorization migration data; distributing the multiple sub-tasks to different servers for parallel comparison to obtain multiple comparison sub-results, and integrating all the comparison sub-results to obtain the final comparison result.

[0008] Optionally, filtering successfully completed transaction data from the transaction data table to obtain target data includes: determining whether initial data exists in the transaction data table, wherein the initial data is transaction data before the task transaction was processed using the proxy authorization system; if the initial data exists in the transaction data table, filtering out data whose transaction time is later than the time when the task transaction was started using the proxy authorization system to obtain initial filtered data; and filtering successfully completed transaction data from the initial filtered data to obtain the target data.

[0009] Optionally, a verification tool is used to compare the data in the transaction data table of the proxy authorization system with the proxy authorization migration data in the target system multiple times to obtain a comparison result, including: obtaining a preset interval time; using the verification tool to compare the data in the transaction data table of the proxy authorization system with the proxy authorization migration data in the target system once every preset interval time, and overwriting the previous comparison result obtained in the previous comparison with the current comparison result obtained in the current comparison, to obtain the comparison result.

[0010] Optionally, data migration from the proxy authorization system to the target system is performed using a second migration system, including: using the second migration system to convert data in the transaction data table into a target message, wherein the target message is in a message format that the target system can recognize; and using the second migration system to send the target message to the target system so that the target system can process the target message, thereby realizing data migration from the proxy authorization system to the target system.

[0011] Optionally, after repeatedly comparing the data in the transaction data table of the proxy authorization system with the proxy authorization migration data in the target system using a verification tool to obtain a comparison result, the method further includes: if the similarity between the data in the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system is greater than or equal to a preset similarity, and the amount of data migrated from the proxy authorization system to the target system reaches a second preset data amount, then the target system is used to process the task transaction; or, if the similarity between the data in the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system is greater than or equal to a preset similarity, and the proxy authorization system stops at a first preset duration for the task transaction, then the target system is used to process the task transaction; if the target system is used to process the task transaction for a second preset duration, the verification tool is used to compare the data in the transaction data table of the proxy authorization system at the current time with the proxy authorization migration data in the target system to obtain a final comparison result.

[0012] Optionally, the verification tool is a MapReduce architecture, comprising a master node and child nodes. The master node includes a task factory, a task scheduler, a heartbeat service, and a result aggregation module. The child nodes include a data connection module, a data transformation module, a comparison module, and a file generation module. The verification tool repeatedly compares the data in the transaction data table of the proxy authorization system with the proxy authorization migration data in the target system to obtain comparison results. This includes: initiating a comparison request to the master node, causing the master node to generate multiple shard tasks according to the sharding logic based on the comparison request, and using the task scheduler to allocate the shard tasks to different child nodes for execution. This allows the child nodes to establish dual data source connections and read error record files generated in comparison rounds within historical time periods, read the comparison data in the shard tasks in parallel and page by page, perform format conversion and comparison according to field mapping rules, and generate incremental difference files. This allows the master node to merge the difference files using MapReduce mode, generate a difference report, and obtain the comparison results. The comparison request is a request to repeatedly compare the data in the transaction data table of the proxy authorization system with the proxy authorization migration data in the target system.

[0013] According to another aspect of this application, a data migration verification apparatus is provided, comprising: a determining unit, configured to determine that a proxy authorization system is used to process task transactions; a first processing unit, configured to perform data migration from the proxy authorization system to the target system using a second migration system after data migration from the original system to the target system is completed using a first migration system; a second processing unit, configured to acquire a transaction data table of the proxy authorization system and proxy authorization migration data in the target system when the amount of data migrated from the proxy authorization system to the target system reaches a first preset data amount; and a verification unit, configured to use a verification tool to repeatedly compare the data in the transaction data table of the proxy authorization system with the proxy authorization migration data in the target system to obtain a comparison result, the comparison result including the similarity between the data in the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system.

[0014] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform any of the data migration verification methods described above.

[0015] According to another aspect of this application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including a verification method for performing any of the data migration methods described above.

[0016] Applying the technical solution of this application, the above-mentioned data migration verification method first determines that a proxy authorization system will be used to process task transactions; after the first migration system completes the data migration from the original system to the target system, a second migration system is used to migrate data from the proxy authorization system to the target system; when the amount of data migrated from the proxy authorization system to the target system reaches a first preset data amount, the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system are obtained; a verification tool is used to compare the data in the transaction data table of the proxy authorization system with the proxy authorization migration data in the target system multiple times to obtain comparison results, including the similarity between the data in the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system. This method can comprehensively verify the migration quality of real-time production data by the proxy authorization system, stabilize system switchover confidence, and promptly identify production issues, facilitating post-switchover problem analysis and backtracking. It solves the problem that existing verification methods for real-time transaction data migration quality cannot guarantee the sufficiency of verification and the accuracy of data migration. Attached Figure Description

[0017] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 A hardware structure block diagram of a mobile terminal for performing a verification method for data migration according to an embodiment of this application is shown.

[0019] Figure 2 A flowchart illustrating a data migration verification method according to an embodiment of this application is shown.

[0020] Figure 3 A system overall processing flowchart of a data migration verification method provided according to an embodiment of this application is shown;

[0021] Figure 4 A nonstatic data verification strategy diagram of a data migration verification method provided according to an embodiment of this application is shown;

[0022] Figure 5 A non-verification tool design diagram of a data migration verification method provided according to an embodiment of this application is shown;

[0023] Figure 6 A structural block diagram of a data migration verification apparatus provided according to an embodiment of this application is shown.

[0024] The above figures include the following reference numerals:

[0025] 102. Processor; 104. Memory; 106. Transmission device; 108. Input / output device. Detailed Implementation

[0026] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0027] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] For ease of description, the following explains some of the nouns or terms used in the embodiments of this application:

[0030] Financial transactions: Financial transactions are those that cause changes in a customer's balance or credit limit, such as: consumption, repayment, return, installment application, etc.

[0031] Non-financial transactions: Non-financial transactions are inquiry and maintenance transactions that will cause changes to the customer's balance and credit limit.

[0032] Authorized transaction: An authorized transaction refers to the process by which a merchant, when accepting credit card payments, sends a request to the issuing bank through the payment processing network to obtain authorization for the transaction.

[0033] Data migration: Data migration refers to the smooth transfer of business and technical information carried by data in the original system to the target system, while ensuring that the business meaning of all customer service data remains unchanged before and after the migration.

[0034] Authorization on Behalf: Authorization on Behalf refers to a functional module that simplifies the authorization of financial transactions in place of the credit card authorization function. It can quickly return transaction responses in place of the original credit card authorization function during upgrades, business closures, or peak transaction periods.

[0035] As described in the background section, current migration and switching schemes for bank card core systems using a proxy authorization system primarily rely on manual traffic verification to check the quality of real-time transaction data migration. This method is insufficient for comprehensive verification and cannot guarantee that data processed by the authorization system during production can be correctly traced back to the target system. To address the shortcomings of existing methods for verifying the quality of real-time transaction data migration, which fail to guarantee sufficient verification and accurate data migration, embodiments of this application provide a data migration verification method, a data migration verification device, a computer-readable storage medium, and an electronic device.

[0036] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0037] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for a data migration verification method according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0038] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to method Z in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. The transmission device 106 is used to receive or send data via a network. Specific examples of the aforementioned networks may include wireless networks provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0039] This embodiment provides a data migration verification method running on a mobile terminal, computer terminal or similar computing device. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0040] Figure 2 This is a flowchart of a data migration verification method according to an embodiment of this application. Figure 2 As shown, the method includes the following steps:

[0041] Step S201: Determine whether to use the proxy authorization system to process the task transaction;

[0042] Specifically, when financial institutions face the need to upgrade or switch their core systems, in order to ensure transaction continuity and customer experience during the system upgrade, it is necessary to introduce a proxy authorization system to handle real-time transactions. That is, during the system migration process, the proxy authorization system will take on the role of real-time transaction processing to ensure a smooth transition of transaction data.

[0043] Step S202: After the data migration from the original system to the target system is completed using the first migration system, the data migration from the aforementioned authorized system to the aforementioned target system is carried out using the second migration system.

[0044] Specifically, after migrating existing data from the original system to the target system using the first migration system, the second migration system is then used to migrate the transaction data processed by the proxy system during this period to the target system. This two-tier migration strategy ensures that both existing data and real-time transaction data can be smoothly transitioned to the new target system, thereby achieving a seamless system upgrade.

[0045] Step S203: When the amount of data migrated from the above-mentioned proxy authorization system to the above-mentioned target system reaches the first preset data amount, obtain the transaction data table of the above-mentioned proxy authorization system and the proxy authorization migration data in the above-mentioned target system.

[0046] Specifically, when the data migration volume of the second migration system reaches a pre-set threshold (e.g., 80% of the data processed by the proxy system), corresponding data is extracted from the transaction data table of the proxy system and the new core system for subsequent data comparison and verification. This threshold is set to ensure that the data verification process is initiated at a reasonable time, neither prematurely interfering with the normal operation of the proxy system nor delaying preparations before the new system goes live.

[0047] Step S204: Using a verification tool, the data in the transaction data table of the above-mentioned proxy authorization system is compared with the proxy authorization migration data in the above-mentioned target system multiple times to obtain the comparison results. The comparison results include the similarity between the data in the transaction data table of the above-mentioned proxy authorization system and the proxy authorization migration data in the above-mentioned target system.

[0048] Specifically, a custom-developed automated verification tool repeatedly compares the transaction data recorded in the outsourced authorization system with the transaction data received in the new core system to detect any subtle differences during the data migration process. This process utilizes the MapReduce distributed computing framework to ensure the efficiency and accuracy of large-scale data comparison. The verification tool will generate detailed discrepancy reports, including but not limited to the completeness and consistency of transaction data, as well as the location of potential data quality issues, providing crucial information for system switchover decisions.

[0049] The above embodiments, through the design of a reasonable verification scheme and the use of automated tools to verify real-time production traffic in stages, can comprehensively verify the migration quality of real-time production data by the proxy licensing system, stabilize system switchover confidence, and promptly identify production issues, facilitating post-switchover problem analysis and backtracking. The above embodiments provide an automated data verification scheme that can quickly and effectively verify the correctness of migrated data, solve the problem of verifying the data quality of transactions processed during the retroactive proxy licensing period, stabilize system switchover confidence, and promptly identify production issues, facilitating post-switchover problem backtracking and accountability.

[0050] In addition, the above embodiments introduce a real-time data comparison and verification process based on the proxy authorization system, which can fully verify and ensure the quality of real-time transaction data migration under the data heterogeneous mode, promptly supplement problematic data, and enhance confidence in system upgrades or system switching. A real-time data comparison and verification scheme and strategy based on the proxy authorization system is designed, using automated data verification tools and a phased verification strategy to draw data entry difference curves, obtain data verification reports, and dynamically perceive the status of each stage of data migration in a timely manner, ensuring the quality of data migration and enhancing confidence in system upgrades or system switching.

[0051] Furthermore, current technical solutions that utilize proxy authorization systems to provide uninterrupted upgrade services during system upgrades or switches often face challenges in ensuring consistent data quality when transactions from the proxy authorization period are transferred to the new system, especially in cases of system or data heterogeneity. Research indicates that no relevant patents have yet proposed a verification scheme for the data processing of proxy authorization systems. This embodiment addresses this issue by proposing the necessity and solution for verifying the real-time traffic handled by the proxy authorization system, further ensuring a smooth switch and operation of the new system, and facilitating problem retrospective analysis and account tracking after system switchover.

[0052] Meanwhile, this embodiment proposes a verification scheme and strategy for supplementing transaction data in proxy authorization processing, including the entry point and verification rounds. It proposes a phased verification strategy, and during the system switchover process, the data entry status can be viewed in real time through dynamic difference curves. This allows for timely and accurate understanding of the real-time data supplementation status, timely detection and handling of problems, and further verification of the migration quality of existing data, providing strong evidence for the execution results of the entire system migration scheme.

[0053] The implementation method provided in this application overcomes the shortcomings of existing technologies that rely solely on manual data quality verification. Through automated tools and a phased verification strategy, it can more accurately and comprehensively assess the migration quality of data processed by the proxy licensing system. This method not only improves verification efficiency and reduces labor costs, but also identifies and resolves problems in advance during system upgrades, enhancing the security and stability of system switching. Simultaneously, the introduction of automated verification tools, especially the use of a MapReduce architecture in the data comparison stage, significantly improves the capacity for large-scale data processing, enabling data verification to be completed in a shorter time and reducing the risk of data stagnation. Furthermore, through multiple verifications and result aggregation, even with dynamically changing data, uncertainties caused by real-time transactions can be gradually eliminated, ultimately obtaining a stable data discrepancy report, providing a reliable reference for switching decisions. At the same time, the phased verification strategy, which compares data at different points in the proxy licensing data migration process, effectively monitors the progress of data recovery and adjusts the data migration strategy in a timely manner, avoiding system risks that may arise from a single verification failing to cover all complex situations. This method is particularly suitable for complex and ever-changing data environments during system switching or upgrade scenarios.

[0054] The data migration verification method described in this application first determines that a proxy authorization system will be used to process task transactions. After the first migration system completes the data migration from the original system to the target system, a second migration system is used to migrate data from the proxy authorization system to the target system. When the amount of data migrated from the proxy authorization system to the target system reaches a first preset data amount, the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system are obtained. A verification tool is used to repeatedly compare the data in the transaction data table of the proxy authorization system with the proxy authorization migration data in the target system to obtain comparison results. The comparison results include the similarity between the data in the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system. This method can comprehensively verify the migration quality of real-time production data by the proxy authorization system, stabilizing system switchover confidence. It solves the problem that existing verification methods for real-time transaction data migration quality cannot guarantee the sufficiency of verification and the accuracy of data migration.

[0055] This embodiment effectively verifies the quality of real-time data migration, especially during the upgrade and switchover of the bank card core system when a proxy authorization system is used to process real-time production data. This solution determines whether to use the proxy authorization system to handle transactions and, after the first data migration is completed, initiates a second data migration process to supplement the real-time transaction data processed by the proxy authorization system into the new system. When the amount of supplemented data reaches a preset threshold, the system automatically extracts the transaction data table from the proxy authorization system and collects the corresponding proxy authorization migration data from the new system. Subsequently, it uses automated verification tools to perform multiple data comparisons. This multi-round comparison strategy effectively addresses dynamic data changes, resulting in more accurate comparison results, namely, the similarity between the transaction data of the proxy authorization system and the proxy authorization migration data in the new system. This method not only greatly improves the efficiency and accuracy of the verification process but also enables real-time monitoring of data migration quality, timely detection and correction of data problems, ensuring data consistency and integrity when the new system goes live, and enhancing the stability and reliability of system upgrades or switchovers.

[0056] In some embodiments, a verification tool is used to repeatedly compare the data in the transaction data table of the aforementioned proxy authorization system with the proxy authorization migration data in the aforementioned target system to obtain comparison results, including the following steps:

[0057] Step S301: Filter out the successfully completed transaction data from the above transaction data table to obtain the target data;

[0058] Step S302: The initial structured language of the target data is converted into the target structured language using the above-mentioned verification tool to obtain the target data after format conversion. The target structured language is a data structured language that the target system can recognize.

[0059] Step S303: The comparison task is split into multiple sub-tasks. The comparison task is to compare the target data after the above format conversion with the corresponding above-mentioned proxy license migration data. The sub-tasks include part of the target data after the above format conversion and the corresponding above-mentioned proxy license migration data.

[0060] Step 304 distributes the above subtasks to different servers for parallel comparison, obtains multiple comparison sub-results, and integrates all the above comparison sub-results to obtain the above comparison result.

[0061] During core system switching, data transmitted by the proxy authorization system is stored in the database through the proxy authorization record accounting system. The accuracy of this data needs to be verified, and the verification results serve as one of the criteria for switching to the new system. This verification scheme verifies the quality of the supplementary data by comparing the data recorded in the proxy authorization system with the data appended to the new system and statistically analyzing the differences. To achieve the goal of a one-time migration without downtime, the verification process for the supplementary data in the proxy authorization system and the supplementary data process need to be parallel during actual production deployment. Therefore, the verification data is in a real-time changing state, requiring the design of a suitable verification strategy to verify the dynamic data in stages to achieve the desired effect.

[0062] Successful records are selected from the proxy authorization record table for comparison. This verifies the portion successfully processed during the proxy authorization period and compares it with the data after the proxy authorization was migrated to the new core. Due to the large amount of data to be compared, the task will be split using MapReduce and distributed to different servers outside the three systems for comparison, improving comparison efficiency. The goal of generating the report is to ensure consistency with the aforementioned full transaction data across all fields.

[0063] In this embodiment, to verify the quality of real-time production data migration from the proxy licensing system to the target system, an automated verification tool was used to perform multiple rounds of parallel data comparison. Specifically, firstly, records of successfully completed transactions were selected from the proxy licensing transaction data table to form the target dataset. Subsequently, the automated verification tool converted the data in the target dataset from its initial structured language to a structured language recognizable by the target system, ensuring data compatibility and comparability. Next, the large-scale comparison task was subdivided into several subtasks. Each subtask contained a portion of the converted target data and the corresponding proxy licensing migration data obtained from the target system. This task division improved the efficiency and accuracy of data comparison. Subsequently, multiple subtasks were distributed to different servers for parallel processing. This step fully utilized distributed computing resources, significantly accelerating the data comparison speed. Each server independently completed the subtask and generated its own comparison sub-results. Finally, all comparison sub-results were aggregated and compiled to form the final comparison result. This process revealed any discrepancies during the data migration process, providing crucial decision-making basis for system switching. The overall solution not only improved the efficiency of data verification but also ensured the accuracy of the verification results, thereby enhancing data consistency during system upgrades or switching, and improving service reliability and user experience.

[0064] In some embodiments, the successful transaction data is filtered from the above transaction data table to obtain the target data, including the following steps:

[0065] Step S401: Determine whether there is initial data in the above transaction data table. The initial data is the transaction data before the above task transaction was processed by the proxy authorization system.

[0066] Step S402: If the initial data exists in the transaction data table, filter out the data whose transaction time is later than the time when the proxy authorization system starts processing the task transaction to obtain the initial filtered data.

[0067] Step S403: Select successful transaction data from the initial screening data to obtain the target data.

[0068] Among them, such as Figure 3 As shown, during production deployment, after switching to the proxy licensing online mode to handle user traffic, the proxy licensing-related data tables begin to be recorded. After completing the migration of existing data, the proxy licensing transaction traffic is then supplemented to the new core system, and the data quality supplemented to the new core system is verified according to the data verification strategy. Once the proxy licensing data quality verification is complete, the new core system will handle the transaction traffic.

[0069] This solution inevitably requires concurrent queries on both the old and new databases. The resulting high CPU and disk I / O load needs to be addressed separately for each system. New Core System: Since the new system is not currently handling external transaction traffic, it can directly use the primary database for concurrent data queries. Old Core System: Because the old core database is still handling external transaction traffic, even though it's only querying, a direct connection to the primary database could still increase response time; therefore, a direct connection is not possible. Instead, a connection to the standby database is required, with the expectation that the master-slave data synchronization latency will be less than 5 minutes to obtain more accurate verification results.

[0070] In this embodiment, when filtering successfully completed transaction data from the transaction data table to obtain the target data, the process includes the following steps: First, identifying whether initial data exists in the transaction data table. This initial data refers to transaction records before the task transaction was processed by the proxy authorization system. Second, once it is confirmed that the transaction data table contains such initial data, all data whose transaction time is later than the time when the proxy authorization system started processing the task transaction will be filtered out from the table, thus obtaining the initial filtered data. Finally, further filtering is performed on this initial filtered data, retaining only successfully completed transaction data, thereby accurately obtaining the target data. This series of operations ensures that the real-time transaction data processed by the proxy authorization system can be accurately used in subsequent data comparison and verification processes, effectively improving the accuracy and reliability of data migration. In other embodiments, the data filtering conditions and processes can be adjusted according to actual conditions. For example, by setting different time thresholds or transaction status parameters, it can be more flexibly adapted to various transaction environments and business needs, ensuring the effectiveness and practicality of the data verification scheme.

[0071] In some embodiments, a verification tool is used to repeatedly compare the data in the transaction data table of the aforementioned proxy authorization system with the proxy authorization migration data in the aforementioned target system to obtain comparison results, including the following steps:

[0072] Step S501: Obtain the preset interval time;

[0073] Step S502: Using a verification tool, the data in the transaction data table of the above-mentioned authorization system is compared with the authorization migration data in the above-mentioned target system at the above-mentioned preset interval time, and the current comparison result obtained in the current comparison is overwritten with the previous comparison result obtained in the previous comparison to obtain the above comparison result.

[0074] Since there are no periods of static data during the transaction period when the authorization is delegated, multiple verifications are required to mitigate changes caused by the exchange. For example... Figure 4As shown, the operation steps are as follows: For example, on the day of migration, at 6:00 AM, the proxy authorization transmission and record-keeping system—the small system responsible for migrating proxy authorization data to the new core database—completed most of the data back-up. This time is used as the start time for data verification. The proxy authorization record table is filtered by the value of the record time column being greater than the start time of the proxy authorization online module handling transaction traffic. This filters out which credit cards had transactions during the proxy authorization transaction traffic period, using this as the base dataset to query the corresponding cardholder information. This dataset is then used to query the old core database by behavioral dimension. The proxy authorization data is matched with the new core database in the specified data format, and the new core database is queried for comparison, generating a report asynchronously. This completes the first data comparison. However, since the data is not static, ongoing or subsequent transactions may have affected the data. Therefore, data that fails to match is recorded. Data comparison is performed again at fixed time intervals, repeating this process to eliminate the impact of dynamic transactions on the data. The final result is a curve showing decreasing differences. When the curve stabilizes, it becomes the final difference report. This report is then used as one of the bases for determining whether a new system should be launched and as a source for subsequent manual tracking.

[0075] In this embodiment, a verification tool is used to compare the data in the transaction data table of the proxy authorization system with the proxy authorization migration data in the target system to ensure the quality of data migration. Specifically, a preset interval is set, and the verification tool periodically compares the data according to this interval. Each comparison result overwrites the previous result, thus forming a dynamically updated comparison result. This strategy can capture changes in data status in a timely manner, and can continuously track differences even in an environment where data is constantly updated, improving the accuracy and timeliness of verification. Through multiple rounds of comparison, dynamic data interference caused by real-time transactions can be gradually eliminated until the differences tend to stabilize. This not only helps to identify the real data quality problems, but also provides a reliable decision-making basis for the smooth switch to the new system. In addition, after the system switch, by comparing the difference between the proxy authorization records of the old system and the successful proxy authorization records of the new system, erroneous data can be accurately located, facilitating subsequent tracking operations and ensuring transaction integrity.

[0076] In some embodiments, a second migration system is used to migrate data from the aforementioned proxy licensing system to the aforementioned target system, including the following steps:

[0077] Step S601: The data in the transaction data table is converted into a target message using the second migration system described above. The target message is in a format that the target system can recognize.

[0078] Step S602: The target message is sent to the target system using the second migration system described above, so that the target system processes the target message and realizes the data migration from the authorized system to the target system.

[0079] Because the new system immediately begins handling transaction traffic after the outsourced authorization completes data supplementation, there isn't sufficient time for data verification. Therefore, the timing of outsourced authorization data verification is primarily distributed before and after the system switch. Before the system switch, data verification begins after the outsourced authorization record data transmission is complete. To eliminate the influence of transactions on the data comparison results, repeated verification is required, re-verifying data that failed the previous verification each time, until the number of data verifications exceeds three or the data accuracy rate reaches 100%. For example, after the outsourced authorization record data transmission is complete and data synchronization is achieved, the first data verification begins. The purpose is to verify the period from when the outsourced authorization starts handling customer traffic to when the accumulated transaction transmission is completed (transactions account for approximately 80%), and to initially calculate transaction accounting discrepancies. A second data verification is performed when real-time traffic is about to be supplemented. The purpose is to re-verify previously failed transactions to see if the discrepancies are due to transaction delays; and to verify new real-time transactions from the previous verification time to the current time, further calculating transaction accounting discrepancies. After data verification is completed, a verification report for authorized data will be issued, including the total number of verified customers, the number of transactions for each transaction type, the number of successful transactions, the number of failed transactions, the differences between the old and new data, and customer information. Data verification must meet certain standards before switching to the new core system to handle transaction traffic. After the system switch, the main purpose of post-switch data verification is to track errors and accurately locate erroneous data caused by network fluctuations or other issues during the period when authorized data handled transaction traffic. Successful data will be recorded in the authorized data record table in the new system, while the total data will be stored in the authorized data record accounting table in the old core system. Therefore, the difference between the total data in the old core system and the successful data set in the new core system represents the data that needs to be tracked. This data verification will also generate an authorized data recovery report, which includes: the number of authorized records in the old system, the number of authorized records in the new system, information on the related tables in the new system, information on the related tables in the old system, and the index values ​​corresponding to the discrepancies. This information is used for subsequent data recovery by relevant personnel.

[0080] In this embodiment, a second migration system is used to migrate data from the proxy authorization system to the target system. Specifically, the second migration system first processes the transaction data table in the proxy authorization system, converting it into a target message format that the target system can recognize. This format conversion is crucial, ensuring the compatibility and interpretability of data migration between different systems. Subsequently, the second migration system is responsible for sending these target messages to the target system. After receiving the messages, the target system processes them accordingly, thereby achieving seamless data migration. The advantage of this approach is that it not only simplifies the complexity of data migration but also improves the efficiency and accuracy of data migration, ensuring a smooth transition of transaction data between the proxy authorization system and the target system, thereby enhancing the overall stability and user experience of system upgrades or switching. Of course, in other embodiments not shown, the data conversion and transmission mechanisms can be adjusted according to specific needs to adapt to the data migration requirements in different scenarios, but the core concept remains the same: ensuring the quality of data migration through format conversion and secure transmission.

[0081] After repeatedly comparing the data in the transaction data table of the aforementioned proxy authorization system with the proxy authorization migration data in the aforementioned target system using verification tools, and obtaining the comparison results, the aforementioned method further includes the following steps:

[0082] Step S701: If the similarity between the data in the transaction data table of the above-mentioned authorization system and the authorization migration data in the above-mentioned target system is greater than or equal to a preset similarity, and the amount of data migrated from the above-mentioned authorization system to the above-mentioned target system reaches a second preset amount of data, then the above-mentioned target system is used to process the above-mentioned task transaction; or, if the similarity between the data in the transaction data table of the above-mentioned authorization system and the authorization migration data in the above-mentioned target system is greater than or equal to a preset similarity, and the above-mentioned authorization system stops being in the above-mentioned task transaction for a first preset duration, then the above-mentioned target system is used to process the above-mentioned task transaction.

[0083] Step S702: When the target system is used to process the task transaction for the second preset duration, the verification tool is used to compare the data in the transaction data table of the proxy authorization system at the current time with the proxy authorization migration data in the target system to obtain the final comparison result.

[0084] Because the new system immediately took over transaction traffic after the outsourced authorization completed data supplementation, there wasn't enough time for the outsourced authorization to verify the data. Therefore, the timing of the outsourced authorization data verification process was mainly distributed as follows: before the system switch and after the system switch. Figure 5As shown, before the system switch, a verification report of authorized data will be issued after data verification is completed. This report includes the total number of verified customers, the number of transactions for each transaction type, the number of successful transactions, the number of failed transactions, the differences between the old and new data, and customer information. Data verification must meet certain standards before the switch to the new core system to handle transaction traffic can proceed. After the system switch, an authorized data supplementation report will also be issued based on this data verification. This report includes: the number of authorized records in the old system, the number of authorized records in the new system, information on the related tables in the new system, information on the related tables in the old system, and the index values ​​corresponding to the differing data. This information is used for subsequent data supplementation by relevant personnel.

[0085] In this embodiment, after repeatedly comparing the data in the transaction data table of the proxy authorization system with the proxy authorization migration data in the target system using a verification tool, and obtaining the comparison results, further, if the similarity between the data in the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system is greater than or equal to a preset similarity, and the amount of data migrated from the proxy authorization system to the target system reaches a second preset data amount, the system will use the target system to process the task transaction. Alternatively, if the data similarity in the proxy authorization system meets a preset condition, and the proxy authorization system stops processing task transactions for a first preset time, the target system will also be used to process the task transaction. After the target system has processed task transactions for a second preset time, the verification tool is used again to compare the data in the transaction data table of the proxy authorization system at the current moment with the proxy authorization migration data in the target system to obtain the final comparison result. This design effectively verifies the data consistency before and after the system switch, ensuring that the real-time transaction data processed by the proxy authorization system can be accurately and error-free traced to the target system, improving the accuracy and efficiency of data migration, and also providing stability and reliability for the entire system upgrade or switch, facilitating timely detection and resolution of problems, and further ensuring the continuity and security of financial services.

[0086] In some embodiments, the verification tool is a MapReduce architecture. The verification tool includes a master node and child nodes. The master node includes a task factory, a task scheduler, a heartbeat service, and a result aggregation module. The child nodes include a data connection module, a data transformation module, a comparison module, and a file generation module. The verification tool repeatedly compares the data in the transaction data table of the above-mentioned proxy authorization system with the proxy authorization migration data in the above-mentioned target system to obtain comparison results, including the following steps:

[0087] A comparison request is initiated to the aforementioned master node, which then generates multiple sharding tasks according to the sharding logic based on the comparison request. The task scheduler then distributes these sharding tasks to different child nodes for execution. This allows the child nodes to establish dual data source connections and read error record files generated from comparison rounds within historical time periods. They also read the comparison data from the sharding tasks in parallel and in pages, perform format conversion and comparison according to field mapping rules, and generate incremental difference files. The master node then uses MapReduce mode to merge these difference files, generate a difference report, and obtain the comparison results. The comparison request involves multiple comparisons of the data in the transaction data table of the aforementioned proxy authorization system with the proxy authorization migration data in the aforementioned target system.

[0088] In the data verification process, each round is completed by automated verification tools. These tools read the differences between the old and new data sources and the comparison results from previous rounds, match and transform the old and new data according to certain rules, and then generate a difference report through a full-field comparison. For example... Figure 5 As shown, the clustered design of the verification tool is based on the MapReduce architecture, supporting horizontal scaling and fault tolerance. The core modules include master nodes and child nodes. The master node contains a task factory, task scheduler, heartbeat service, and result aggregation module. The child nodes contain a data connection module, data transformation module, comparison module, and file generation module. The main process is as follows: The user initiates a comparison request to the master node. The master node generates sharded tasks according to the sharding logic based on the request. The task scheduler distributes the tasks to different child nodes. The child nodes establish dual data source connections and read the error record files generated in previous comparison rounds. They read the comparison data in parallel, page by page, perform format conversion and comparison according to field mapping rules, generate incremental difference files, and update the progress file periodically. The master node merges the difference files using the MapReduce model, ultimately generating a unified difference report.

[0089] In this embodiment, to ensure the accuracy and consistency of real-time transaction data processed by the proxy authorization system after migration to the target system, an automated data verification tool based on the MapReduce architecture was designed. This tool can efficiently handle the comparison of large amounts of data. The verification tool consists of a master node and child nodes. The master node is responsible for task generation, scheduling, and result aggregation, while the child nodes are responsible for data reading, transformation, comparison, and difference file generation. In specific implementation, the user first sends a comparison request to the master node. The master node generates sharded tasks based on the request and distributes the tasks to each child node through a task scheduler. After receiving the task, the child node connects to the old and new data sources, reads the transaction data table recorded by the proxy authorization system and the error record file generated in the previous comparison, reads the data required for comparison in batches in parallel, performs format conversion according to preset field mapping rules, and compares it with the proxy authorization migration data in the target system. The incremental difference file generated during the comparison process is periodically updated. Finally, the difference file is aggregated through the MapReduce mode of the master node to generate the final difference report, thereby effectively verifying the quality of the real-time transaction data processed by the proxy authorization system. Through multiple iterations and comparisons, not only was the accuracy of data verification improved, but the data supplementation status was also promptly fed back, ensuring a smooth system switchover.

[0090] To enable those skilled in the art to better understand the technical solution of this application, the implementation process of the data migration verification method of this application will be described in detail below with reference to specific embodiments.

[0091] This embodiment relates to a specific data migration verification method, such as... Figure 3 As shown, it includes the following steps:

[0092] Step S1: Turn on the switching switch to enter Phase 1 and Phase 2. Phase 1 enters the old core system, and Phase 2 enters the authorized system. The old core system and the authorized system enter the main core data together.

[0093] Step S2: Enter Phase 3 through the proxy authorization system, the proxy authorization record transmission component, then enters the core system, and then enters the new core database;

[0094] Step S3: Phase 4, after multiple data verifications, the data enters the new core database and the backup core database;

[0095] Step S4: After the backup core database is synchronized with RAC data, it enters the main core database.

[0096] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0097] This application also provides a data migration verification device. It should be noted that the data migration verification device of this application can be used to execute the data migration verification method provided in this application. This device is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0098] The following describes the data migration verification apparatus provided in the embodiments of this application.

[0099] Figure 6 This is a schematic diagram of a data migration verification apparatus according to an embodiment of this application. Figure 6 As shown, the device includes: a determining unit 10, a first processing unit 20, a second processing unit 30, and a verification unit 40. The determining unit 10 is used to determine whether to use a proxy authorization system to process task transactions. The first processing unit 20 is used to use a second migration system to migrate data from the proxy authorization system to the target system after the data migration from the original system to the target system is completed using a first migration system. The second processing unit is used to obtain the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system when the amount of data migrated from the proxy authorization system to the target system reaches a first preset data amount. The verification unit is used to use a verification tool to compare the data in the transaction data table of the proxy authorization system with the proxy authorization migration data in the target system multiple times to obtain a comparison result, which includes the similarity between the data in the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system.

[0100] The data migration verification device described in this application includes a determining unit, a first processing unit, a second processing unit, and a verification unit. The determining unit is used to determine whether to use a proxy authorization system to process the transaction task. The first processing unit is used to use a second migration system to migrate data from the proxy authorization system to the target system after the data migration from the original system to the target system is completed using a first migration system. The second determining unit is used to obtain the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system when the amount of data migrated from the proxy authorization system to the target system reaches a first preset data amount. The verification unit is used to use a verification tool to repeatedly compare the data in the transaction data table of the proxy authorization system with the proxy authorization migration data in the target system to obtain a comparison result, which includes the similarity between the data in the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system. By repeatedly verifying and gradually reducing data differences, a verification result is finally obtained, including the similarity between the transaction data of the proxy authorization system and the data received by the target system. This ensures the integrity and consistency of the data before the new system goes online, improves the reliability of data migration, and solves the problem that existing methods for verifying the quality of real-time transaction data migration are difficult to guarantee the sufficiency of verification and the accuracy of data migration.

[0101] In some embodiments, the verification unit includes a first determining module, a first verifying module, a first processing module, and a second processing module. The first determining module is used to filter out successfully traded transaction data from the aforementioned transaction data table to obtain target data. The first verifying module is used to convert the initial structured language of the target data into a target structured language using the aforementioned verification tool to obtain the format-converted target data. The target structured language is a data structured language that the target system can recognize. The first processing module is used to split the comparison task into multiple sub-tasks. The comparison task is to compare the format-converted target data with the corresponding proxy migration data. The sub-tasks include a portion of the format-converted target data and the corresponding proxy migration data. The second processing module is used to distribute the multiple sub-tasks to different servers for parallel comparison to obtain multiple comparison sub-results, and integrate all the comparison sub-results to obtain the comparison result. This fully utilizes the resources of distributed computing, significantly accelerating the data comparison speed. Each server independently completes the sub-task and generates its own comparison sub-result.

[0102] In some embodiments, the first determining module includes a second determining module, a third determining module, and a third processing module. The second determining module is used to determine whether initial data exists in the transaction data table. The initial data is transaction data before the task transaction is processed by the proxy authorization system. The third processing module is used to filter out data whose transaction time is later than the time when the task transaction is started being processed by the proxy authorization system, if the initial data exists in the transaction data table, to obtain initial filtered data. The third determining module is used to filter out successfully processed transaction data from the initial filtered data to obtain the target data. This ensures that the real-time transaction data processed by the proxy authorization system can be accurately used in subsequent data comparison and verification processes, effectively improving the accuracy and reliability of data migration.

[0103] In some embodiments, the verification unit further includes a fourth determining module and a second verification module. The fourth determining module is used to obtain a preset interval time. The second verification module is used to compare the data in the transaction data table of the above-mentioned authorization system with the authorization migration data in the above-mentioned target system at the preset interval time using a verification tool, and to overwrite the previous comparison result obtained in the previous comparison with the current comparison result obtained in the current comparison, thereby obtaining the above comparison result. By comparing the difference between the authorization records of the old system and the authorization success records of the new system, erroneous data can be accurately located, facilitating subsequent tracking operations and ensuring transaction integrity.

[0104] In some embodiments, the first processing module further includes a fourth processing module and a fifth processing module. The fourth processing module is used to convert the data in the transaction data table into a target message using the second migration system described above. The target message is in a format recognizable by the target system. The fifth processing module is used to send the target message to the target system using the second migration system, so that the target system can process the target message, thereby realizing the data migration from the proxy authorization system to the target system. This ensures a smooth transition of transaction data between the proxy authorization system and the target system, thereby improving the overall stability and user experience of system upgrades or switching.

[0105] In some embodiments, the above-mentioned apparatus further includes a third verification module and a fourth verification module. The third verification module is used to, after repeatedly comparing the data in the transaction data table of the proxy authorization system with the proxy authorization migration data in the target system using a verification tool, and obtaining comparison results, use the target system to process the task transaction if the similarity between the data in the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system is greater than or equal to a preset similarity, and the amount of data migrated from the proxy authorization system to the target system reaches a second preset data amount; or, use the target system to process the task transaction if the similarity between the data in the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system is greater than or equal to a preset similarity, and the proxy authorization system stops at a first preset duration for the task transaction. The fourth verification module is used to, after using the target system to process the task transaction for a second preset duration, use the verification tool to compare the data in the transaction data table of the proxy authorization system with the proxy authorization migration data in the target system at the current moment, and obtain a final comparison result. This ensured that the data consistency before and after the system switch was effectively verified, and that the real-time transaction data processed by the authorized system could be accurately traced back to the target system, thus improving the accuracy and efficiency of data migration.

[0106] In some embodiments, the verification tool is a MapReduce architecture, comprising a master node and child nodes. The master node includes a task factory, a task scheduler, a heartbeat service, and a result aggregation module. The child nodes include a data connection module, a data transformation module, a comparison module, and a file generation module. The verification unit includes a fifth verification module, used to initiate a comparison request to the master node, so that the master node generates multiple sharded tasks according to the sharding logic based on the comparison request, and uses the task scheduler to allocate the sharded tasks to different child nodes for execution. This allows the child nodes to establish dual data source connections and read error record files generated in comparison rounds within historical time periods, read the comparison data in the sharded tasks in parallel and page by page, perform format conversion and comparison according to field mapping rules, and generate incremental difference files. The master node then uses MapReduce mode to merge the difference files, generate a difference report, and obtain the comparison result. The comparison request is a request to compare the data in the transaction data table of the proxy authorization system with the proxy authorization migration data in the target system multiple times. Through multiple iterations and comparisons, not only was the accuracy of data verification improved, but the data supplementation status was also promptly fed back, ensuring a smooth system switchover.

[0107] The aforementioned data migration verification device includes a processor and a memory. The aforementioned verification units are all stored as program units in the memory, and the processor executes these program units to achieve the corresponding functions. All of the aforementioned modules are located in the same processor; alternatively, the aforementioned modules may be located in different processors in any combination.

[0108] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can address the limitations of existing methods for verifying the quality of real-time transaction data migration, which often fail to guarantee sufficient verification and accurate data migration.

[0109] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0110] This invention provides a computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform the data migration verification method.

[0111] This invention provides a processor for running a program, wherein the program executes the data migration verification method.

[0112] This invention provides a device that includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs at least the verification step of data migration. The device described herein may be a server, PC, PAD, mobile phone, etc.

[0113] This application also provides a computer program product that, when executed on a data processing device, is adapted to perform a program that initializes a verification method step having at least data migration.

[0114] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0115] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0116] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0117] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0118] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0119] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0120] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0121] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0122] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0123] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0124] As can be seen from the above description, the embodiments of this application achieve the following technical effects:

[0125] 1) The data migration verification method described in this application first determines that a proxy authorization system will be used to process task transactions; after the first migration system completes the data migration from the original system to the target system, a second migration system is used to migrate data from the proxy authorization system to the target system; when the amount of data migrated from the proxy authorization system to the target system reaches a first preset amount, the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system are obtained; a verification tool is used to compare the data in the transaction data table of the proxy authorization system with the proxy authorization migration data in the target system multiple times to obtain comparison results, including the similarity between the data in the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system. This method can comprehensively verify the migration quality of real-time production data by the proxy authorization system, stabilize system switchover confidence, and promptly identify production issues, facilitating problem backtracking and accountability after system switchover. It solves the problem in existing technologies where verification methods for real-time transaction data migration quality cannot guarantee the sufficiency of verification and the accuracy of data migration.

[0126] 2) The data migration verification device of this application includes a determining unit, a first processing unit, a second processing unit, and a verification unit. The determining unit is used to determine whether to use a proxy authorization system to process the transaction task. The first processing unit is used to use a second migration system to migrate data from the proxy authorization system to the target system after the data migration from the original system to the target system is completed using the first migration system. The second processing unit is used to obtain the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system when the amount of data migrated from the proxy authorization system to the target system reaches a first preset data amount. The verification unit is used to use a verification tool to compare the data in the transaction data table of the proxy authorization system with the proxy authorization migration data in the target system multiple times to obtain a comparison result. The comparison result includes the similarity between the data in the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system. This device can quickly and effectively verify the correctness of the migration data, solve the problem of verifying the data quality of transactions processed during the proxy authorization period, stabilize system switching confidence, and promptly detect production problems. It facilitates the backtracking and accountability of problems after system switching and solves the problem that the existing verification methods for real-time transaction data migration quality cannot guarantee the sufficiency of verification and the accuracy of data migration.

[0127] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for verifying data migration, characterized in that, include: It has been decided to use a proxy authorization system to process task transactions; After the data migration from the original system to the target system is completed using the first migration system, the data migration from the authorized system to the target system is carried out using the second migration system. When the amount of data migrated from the proxy authorization system to the target system reaches a first preset amount of data, the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system are obtained. The data in the transaction data table of the proxy authorization system is compared with the proxy authorization migration data in the target system multiple times using a verification tool to obtain comparison results. The comparison results include the similarity between the data in the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system.

2. The method according to claim 1, characterized in that, The data in the transaction data table of the proxy authorization system is compared with the proxy authorization migration data in the target system multiple times using verification tools to obtain comparison results, including: The target data is obtained by filtering out successfully completed transactions from the transaction data table. The verification tool is used to convert the initial structured language of the target data into the target structured language to obtain the format-converted target data. The target structured language is a data structured language that the target system can recognize. The comparison task is split into multiple sub-tasks. The comparison task is to compare the target data after the format conversion and the corresponding proxy license migration data. The sub-tasks include part of the target data after the format conversion and the corresponding proxy license migration data. Multiple subtasks are distributed to different servers for parallel comparison to obtain multiple comparison sub-results, and all the comparison sub-results are integrated to obtain the comparison result.

3. The method according to claim 2, characterized in that, Successful transactions are filtered from the transaction data table to obtain the target data, including: Determine whether initial data exists in the transaction data table, where the initial data is the transaction data prior to the processing of the task transaction using the proxy authorization system; If the initial data exists in the transaction data table, the data whose transaction time is later than the time when the task transaction is started to be processed by the proxy authorization system will be filtered out to obtain the initial filtered data; The target data is obtained by filtering out successfully traded transactions from the initial filtered data.

4. The method according to claim 1, characterized in that, The data in the transaction data table of the proxy authorization system is compared with the proxy authorization migration data in the target system multiple times using verification tools to obtain comparison results, including: Get the preset interval time; The verification tool compares the data in the transaction data table of the proxy authorization system with the proxy authorization migration data in the target system at preset intervals, and overwrites the previous comparison result with the current comparison result to obtain the comparison result.

5. The method according to claim 1, characterized in that, The second migration system is used to migrate data from the proxy licensing system to the target system, including: The second migration system is used to convert the data in the transaction data table into a target message, and the format of the target message is a message format that the target system can recognize; The second migration system is used to send the target message to the target system, so that the target system can process the target message to realize the data migration from the proxy authorization system to the target system.

6. The method according to claim 1, characterized in that, After repeatedly comparing the data in the transaction data table of the proxy authorization system with the proxy authorization migration data in the target system using verification tools to obtain comparison results, the method further includes: If the similarity between the data in the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system is greater than or equal to a preset similarity, and the amount of data migrated from the proxy authorization system to the target system reaches a second preset amount, then the target system is used to process the task transaction; or, if the similarity between the data in the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system is greater than or equal to a preset similarity, and the proxy authorization system stops being in the task transaction for a first preset duration, then the target system is used to process the task transaction. When the target system processes the task transaction for a second preset duration, the verification tool is used to compare the data in the transaction data table of the proxy authorization system with the proxy authorization migration data in the target system at the current moment to obtain the final comparison result.

7. The method according to claim 1, characterized in that, The verification tool is based on a MapReduce architecture and includes a master node and child nodes. The master node includes a task factory, a task scheduler, a heartbeat service, and a result aggregation module. The child nodes include a data connection module, a data transformation module, a comparison module, and a file generation module. The verification tool repeatedly compares the data in the transaction data table of the proxy authorization system with the proxy authorization migration data in the target system to obtain comparison results, including: A comparison request is initiated to the master node, which generates multiple sharding tasks according to the sharding logic based on the comparison request. The task scheduler then distributes the sharding tasks to different child nodes for execution. This allows the child nodes to establish dual data source connections and read error record files generated from comparison rounds within historical time periods. They also read the comparison data from the sharding tasks in parallel and in pages, perform format conversion and comparison according to field mapping rules, and generate incremental difference files. The master node then merges these difference files using MapReduce mode, generates a difference report, and obtains the comparison result. The comparison request is a request to compare the data in the transaction data table of the proxy authorization system with the proxy authorization migration data in the target system multiple times.

8. A data migration verification device, characterized in that, include: The determining unit is used to determine whether to use the proxy authorization system to process task transactions. The first processing unit is configured to, after completing the data migration from the original system to the target system using the first migration system, migrate the data from the authorized system to the target system using the second migration system. The second processing unit is used to obtain the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system when the amount of data migrated from the proxy authorization system to the target system reaches a first preset data amount. The verification unit is used to compare the data in the transaction data table of the proxy authorization system with the proxy authorization migration data in the target system multiple times using verification tools to obtain a comparison result. The comparison result includes the similarity between the data in the transaction data table of the proxy authorization system and the proxy authorization migration data in the target system.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the data migration verification method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including a verification method for performing a data migration as described in any one of claims 1 to 7.

Images