Parallel clearing method, device and equipment based on mirror image production data and storage medium

Abstract

The embodiment of the invention relates to the technical field of computer software, and discloses a parallel clearing method, device and equipment based on mirror image production data and a storage medium, and the method comprises the steps: building a mirror image production environment corresponding to a real production environment, and running a monitoring sub-task and a mirror image data sub-task on the T-1 day; based on the monitoring sub-task and the mirror image data sub-task, transmitting each production data file in each transaction system participating in parallel liquidation from the real production environment to the mirror image production environment; on the T day, in the mirror image production environment, according to a preset database silent recovery algorithm, recovering a database instance corresponding to the production data file in the same database; and extracting the settlement file from the real production environment to the mirror image production environment, and executing parallel settlement based on the settlement file, the production data file and the database instance in the mirror image production environment. According to the embodiment of the invention, the method can achieve the timely parallel settlement, and improves the efficiency and accuracy.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer software technology, and in particular, to a parallel clearing method, apparatus, device, and storage medium based on mirror production data. Background Art

[0002] End-of-day clearing refers to a series of clearing and settlement activities that financial institutions perform on their trading accounts at the end of each trading day to ensure the accuracy and finality of all transactions. Parallel clearing can process multiple clearing requests simultaneously.

[0003] In traditional parallel clearing, generally after the end-of-day clearing is completed daily, the operation and maintenance personnel compress and archive the clearing data of the day, and then restore the required data files to a temporary space for desensitization processing the next day, and then transfer them to the development and test environment for parallel clearing. This process takes about 10 hours and requires multiple data export, compression, transmission, and decompression operations. Finally, the test personnel restore the data and start parallel clearing. This method has the following problems: 1. Data lag: The data for parallel clearing lags behind the production environment by at least two to three days, resulting in untimely comparison of clearing results; 2. Low data accuracy: The data is processed and transmitted multiple times, which may introduce errors and affect the accuracy of the parallel clearing results. 3. Low efficiency: The processes of data extraction, desensitization, transmission, and restoration are cumbersome and time-consuming. Generally, only one end-of-day clearing verification can be performed on the same day, and the efficiency is low; and the end-of-day clearing involves extracting data from multiple trading systems (such as ordinary order trading systems, ultra-fast trading systems, over-the-counter trading systems, etc.) and operating on multiple databases (such as MSSQL, ORACLE, GAUSSDB, TDSQL, etc.). It is difficult for traditional parallel clearing methods to efficiently implement this complex operation. Summary of the Invention

[0004] In view of the above problems, embodiments of the present invention provide a parallel clearing method, apparatus, device, and storage medium based on mirror production data to solve at least one of the above problems.

[0005] According to one aspect of the embodiments of the present invention, a parallel clearing method based on mirror production data is provided, and the method includes:

[0006] Establish a mirror production environment corresponding to the real production environment. On T-1 day, run a preset monitoring subtask and a mirror data subtask. The mirror production environment is established in the real production environment or the test environment, and the network and devices of the mirror production environment are isolated from those of the real production environment;

[0007] Based on the monitoring subtask and the mirror data subtask, transfer each production data file in each trading system participating in parallel clearing from the real production environment to the mirror production environment;

[0008] On day T, in the mirror production environment, according to a predetermined database silent recovery algorithm, recover the database instance corresponding to the production data file in the same database;

[0009] Extract the settlement file from the real production environment to the mirror production environment, and perform parallel clearing in the mirror production environment based on the settlement file, the production data file, and the database instance.

[0010] In an alternative manner, the transferring of each production data file in each trading system participating in parallel clearing from the real production environment to the mirror production environment based on the monitoring subtask and the mirror data subtask includes:

[0011] Run at least one of the monitoring subtasks for each trading system, and monitor whether the production data file of the trading system is in a generated state based on the monitoring subtask;

[0012] If so, trigger the mirror data subtask, and transfer the production data file to the mirror production environment based on the mirror data subtask;

[0013] The transferring of the production data file to the mirror production environment based on the mirror data subtask includes:

[0014] Clean the target storage space corresponding to the production data file;

[0015] Chunk, encrypt, and pack and compress the production data file to obtain a processed production data file;

[0016] Determine the dynamic transmission rate according to a predetermined dynamic rate limiting policy, and transfer the processed production data file to the mirror production environment according to the dynamic transmission rate.

[0017] In an alternative manner, the determining of the dynamic transmission rate according to a predetermined dynamic rate limiting policy includes:

[0018] Obtain the total network bandwidth and network bandwidth utilization rate of the real production environment, and determine whether the current time is trading time;

[0019] If the current time is the transaction time, determine whether the network bandwidth utilization rate is less than a preset first utilization rate threshold. If it is less than the preset first utilization rate threshold, determine a first transmission rate according to the total network bandwidth, the network bandwidth utilization rate, and the first utilization rate threshold. If the first transmission rate is less than or equal to a predetermined transmission rate threshold, use the first transmission rate as the dynamic transmission rate. If the first transmission rate is greater than the predetermined transmission rate threshold, or the network bandwidth utilization rate is greater than or equal to the first utilization rate threshold, set the dynamic transmission rate to zero and wait for the network bandwidth to be released.

[0020] If the current time is a non-transaction time, determine whether the network bandwidth utilization rate is less than a preset second utilization rate threshold. If it is less than the preset second utilization rate threshold, determine a second transmission rate according to the total network bandwidth, the network bandwidth utilization rate, and the second utilization rate threshold. If the second transmission rate is greater than the transmission rate threshold, use the second transmission rate as the dynamic transmission rate. If the second transmission rate is less than or equal to the transmission rate threshold, use the transmission rate threshold as the dynamic transmission rate. If the network bandwidth utilization rate is greater than or equal to the second utilization rate threshold, set the dynamic transmission rate to zero and wait for the network bandwidth to be released.

[0021] In an alternative way, after performing block division, encryption, and packaging compression on the production data file to obtain a processed production data file, it further includes:

[0022] Obtain a first hash value corresponding to the processed production data file;

[0023] After determining the dynamic transmission rate according to a predetermined dynamic speed limit policy and transmitting the processed production data file to the mirror production environment according to the dynamic transmission rate, it further includes:

[0024] Obtain a second hash value of the production data file in the mirror production environment;

[0025] Compare the first hash value with the second hash value to determine whether the production data file in the real production environment is consistent with the production data file in the mirror production environment.

[0026] In an alternative way, on day T, in the mirror production environment, according to a predetermined database silent recovery algorithm, recover the database instance corresponding to the production data file in the same database, including:

[0027] On day T, obtain the key information corresponding to the production data file in the mirror production environment, and generate a restored private file according to the key information. The key information at least includes the file name, location path, database name to which the file belongs, new database name, and new target path of the production data file;

[0028] Restore the database instance corresponding to the production data file in the same database of the mirror production environment according to the restored private file, rename the database instances with the same name to obtain the new database name, and redirect the production data file of the database instance to restore the production data file to the new target path.

[0029] In an optional manner, before restoring the database instance corresponding to the production data file in the same database of the mirror production environment according to the restored private file and renaming the database instances with the same name, it further includes:

[0030] Automatically release all links and processes of the database instance through the original database command;

[0031] After redirecting the production data file of the database instance to restore the production data file to a predetermined new path, it further includes:

[0032] Verify the key fields of the production data file of the database instance to make the production data file of the database instance consistent with the production data file in the real production environment. The key fields at least include the transaction date and transaction status.

[0033] In an optional manner, the performing parallel liquidation based on the settlement file, the production data file, and the database instance in the mirror production environment includes:

[0034] Perform multiple rounds of parallel liquidation based on the settlement file, the production data file, and the database instance in the mirror production environment.

[0035] According to another aspect of the embodiments of the present invention, there is provided a parallel liquidation device based on mirror production data. The device includes:

[0036] A establishment module, configured to establish a mirror production environment corresponding to the real production environment, and on day T - 1, run a preset monitoring sub-task and mirror data sub-task. The mirror production environment is established in the real production environment or the test environment, and the network and devices of the mirror production environment are isolated from those of the real production environment;

[0037] A transmission module, configured to transfer each production data file in each transaction system participating in parallel settlement from the real production environment to the mirrored production environment based on the monitoring subtask and the mirrored data subtask;

[0038] A recovery module, configured to, on day T, in the mirrored production environment, according to a predetermined database silent recovery algorithm, recover the database instance corresponding to the production data file in the same database;

[0039] A settlement module, configured to extract a settlement file from the real production environment to the mirrored production environment, and perform parallel settlement in the mirrored production environment based on the settlement file, the production data file, and the database instance.

[0040] According to another aspect of the embodiments of the present invention, there is provided a computer device, including: a processor, a memory, a communication interface, and a communication bus, where the processor, the memory, and the communication interface complete mutual communication through the communication bus; the memory is used to store at least one executable instruction, and the executable instruction causes the processor to execute the method as described above.

[0041] According to another aspect of the embodiments of the present invention, there is provided a computer-readable storage medium, where at least one executable instruction is stored in the storage medium, and when the executable instruction runs on a computer device, it causes the computer device to execute the method as described above.

[0042] The embodiments of the present invention split the parallel settlement process, splitting the parallel settlement process on day T into an automated process on day T-1 and a settlement process on day T. Since the production data can be extracted to the mirrored production environment using the automated process at night on day T-1, the parallel settlement on day T only needs to complete operations such as settlement file extraction, data instance recovery, and restart of relevant settlement systems to perform parallel settlement. Therefore, parallel settlement can be carried out in a timely manner, and the efficiency of parallel settlement can be greatly improved; in addition, using the production data of the real production environment for mirrored parallel settlement effectively solves the problem of data inconsistency between the data and the production environment in traditional parallel settlement, improving accuracy and reliability.

[0043] The above description is only an overview of the technical solutions of the embodiments of the present invention. In order to be able to understand the technical means of the embodiments of the present invention more clearly, it can be implemented according to the content of the description. And in order to make the above and other purposes, features, and advantages of the embodiments of the present invention more obvious and understandable, the following specifically illustrates the embodiments of the present invention. Description of the Drawings

[0044] The drawings are only used to illustrate the embodiments and are not considered to be a limitation of the present invention. Moreover, throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0045] Figure 1 shows a schematic flowchart of a parallel clearing method based on mirror production data provided by an embodiment of the present invention;

[0046] Figure 2 shows a schematic diagram of the environment of a parallel clearing method based on mirror production data provided by an embodiment of the present invention;

[0047] Figure 3 shows a schematic structural diagram of a parallel clearing device based on mirror production data provided by an embodiment of the present invention;

[0048] Figure 4 shows a schematic structural diagram of a computer device provided by an embodiment of the present invention. Detailed implementation manners

[0049] Hereinafter, exemplary embodiments of the present invention will be described in more detail with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein.

[0050] Referring to Figure 1 and Figure 2 , Figure 1 shows a flowchart of a parallel clearing method based on mirror production data according to an embodiment of the present invention, Figure 2 shows a schematic diagram of the environment of a parallel clearing method based on mirror production data according to an embodiment of the present invention. Among them, on the T-1 day, each trading system performs end-of-day clearing in the production environment, including end-of-day clearing of margin trading data, ordinary order data, fund data, etc. The data after end-of-day clearing is used as production data, which is sent to the corresponding server and transmitted to the pre-established mirror production environment through rate-limited encryption. Among them, the mirror production environment is a secure network space completely isolated from production in the real production environment. Of course, the mirror production environment can also be established in the test environment. The mirror production environment is completely isolated from the real production environment in terms of network policies, machine equipment, etc., and can effectively ensure that the opened network space is securely isolated from the clearing network space of the real production environment. Specifically, a group of operating machines are planned and constructed to form a highly available cluster, and at the same time, it is ensured that the machines in the cluster can access both the mirror production environment and the real production environment simultaneously. The machines in the cluster are used for data and file interaction between the two environments. On the T day, the database instance corresponding to the production data is restored in the same database of the mirror production environment, and multiple rounds of parallel clearing are performed according to the settlement file, production data, and database instance.

[0051] In the embodiments of the present invention, the parallel settlement process is split. The parallel settlement process on day T is split into an automated process on day T - 1 and a settlement process on day T. Since the automated process can be used on day T - 1 to extract production data to the mirrored production environment in the evening, on day T, only operations such as extracting settlement files, restoring data instances, and restarting relevant settlement systems need to be completed to execute parallel settlement.

[0052] As Figure 1 shown, the parallel settlement method based on mirrored production data in the embodiments of the present invention includes the following steps:

[0053] Step 101, establish a mirrored production environment corresponding to the real production environment. On day T - 1, run a preset monitoring sub - task and a mirrored data sub - task. The mirrored production environment is established in the production environment or the test environment, and the network and devices of the mirrored production environment are isolated from those of the real production environment.

[0054] In this embodiment, since the production data readiness times among the trading systems of ordinary transactions, credit transactions, and over - the - counter transactions are inconsistent, and ordinary transactions include multiple core systems, each core system conducts independent settlement, and the generated end - of - day settlement data times are also inconsistent. Therefore, by running the monitoring sub - task in a hidden manner in the background, the monitoring sub - task monitors whether the data of each trading system is ready. Among them, the number of monitoring sub - tasks is determined by the number of trading systems participating in parallel settlement, and at least one hidden monitoring sub - task corresponds to each trading system. The mirrored data sub - task is used to transfer the ready production data files from the real production environment to the mirrored production environment. Among them, the monitoring sub - tasks and the mirrored data sub - tasks within each trading system form a production data mirror task group on day T - 1, and each task group runs completely independently.

[0055] Step 102, based on the monitoring sub - task and the mirrored data sub - task, transfer each production data file in each trading system participating in parallel settlement from the real production environment to the mirrored production environment.

[0056] In this embodiment, when the monitoring sub - task monitors that the production data readiness status of each trading system is completed, the mirrored data sub - task is triggered. Through the mirrored data sub - task, each production data file in each trading system participating in parallel settlement is transferred from the real production environment to the mirrored production environment. Generally, one trading system may correspond to one production data file.

[0057] In one embodiment, the transferring each production data file in each trading system participating in parallel settlement from the real production environment to the mirrored production environment based on the monitoring sub - task and the mirrored data sub - task includes:

[0058] Run at least one of the monitoring subtasks for each trading system, and monitor whether the production data file of the trading system is in a generated state based on the monitoring subtask; if so, trigger the mirror data subtask, and transfer the production data file to the mirror production environment based on the mirror data subtask, otherwise continue to monitor the generation state of the production data file; wherein, transferring the production data file to the mirror production environment based on the mirror data subtask includes: clearing the target path corresponding to the production data file; splitting, encrypting, and packing and compressing the production data file to obtain a processed production data file; determining a dynamic transmission rate according to a predetermined dynamic speed limit policy, and transferring the processed production data file to the mirror production environment according to the dynamic transmission rate.

[0059] In this embodiment, the target paths corresponding to the production data files of each trading system are different, and the target storage space is cleared to ensure that the target storage space for storing the production data files is sufficient.

[0060] Generally, the production data files of each trading system are larger than 10GB. To reduce the impact of file transmission on the bandwidth of the real production environment, in this embodiment, the production data files are divided into smaller blocks to achieve fast transmission without affecting the bandwidth of the real production environment. Block transmission can effectively reduce the amount of data transmitted at one time, reduce the occupancy of network bandwidth, and improve the flexibility and reliability of transmission. Exemplarily, the production data file is divided into blocks of 100MB each. After splitting, each data block is independently encrypted using a predefined encryption method. Preferably, AES encryption can be used for each data block and a randomly generated initialization vector (IV6) can be used to enhance data security and prevent data leakage and tampering. After encryption, they are packaged together, and after packaging, the file is compressed to finally obtain a processed production data file.

[0061] In this embodiment, a dynamic speed limit policy is formulated based on the source of the production data (i.e., the server where the current library of each trading system is located) and according to the current network bandwidth situation of the real production environment, and the dynamic transmission rate is determined according to the predetermined dynamic speed limit policy. Among them, the dynamic speed limit policy can dynamically adjust the transmission rate according to the current network bandwidth situation of the real production environment. The initial default transmission rate is set to 100M / S. On the premise of ensuring the network stability of the real production environment, the transmission rate is dynamically adjusted based on the dynamic speed limit policy, which can efficiently complete the transfer of the production data files of each trading system to the mirror production environment.

[0062] Further, the determining the dynamic transmission rate according to the predetermined dynamic speed limit policy includes:

[0063] Obtain the total network bandwidth and network bandwidth utilization rate of the real production environment, and determine whether the current time is the trading time; if the current time is the trading time, determine whether the network bandwidth utilization rate is less than a preset first utilization rate threshold. If it is less than the preset first utilization rate threshold, determine the first transmission rate according to the total network bandwidth, the network bandwidth utilization rate, and the first utilization rate threshold; if the first transmission rate is less than or equal to the predetermined transmission rate threshold, use the first transmission rate as the dynamic transmission rate. If the first transmission rate is greater than the predetermined transmission rate threshold, or the network bandwidth utilization rate is greater than or equal to the first utilization rate threshold, set the dynamic transmission rate to zero and wait for the network bandwidth to be released; if the current time is non-trading time, determine whether the network bandwidth utilization rate is less than a preset second utilization rate threshold. If it is less than the preset second utilization rate threshold, determine the second transmission rate according to the total network bandwidth, the network bandwidth utilization rate, and the second utilization rate threshold; if the second transmission rate is greater than the transmission rate threshold, use the second transmission rate as the dynamic transmission rate. If the second transmission rate is less than or equal to the transmission rate threshold, use the transmission rate threshold as the dynamic transmission rate. If the network bandwidth utilization rate is greater than or equal to the second utilization rate threshold, set the dynamic transmission rate to zero and wait for the network bandwidth to be released.

[0064] Among them, during the process of transmitting the production data file, the automation tool is used to obtain the current total network bandwidth and network bandwidth utilization rate of the real production environment, and the total network bandwidth and network bandwidth utilization rate of the real production environment are monitored in real time. In order not to affect the network bandwidth of the real production environment, preferably, the first utilization rate threshold corresponding to the trading time is 40%, and the second utilization rate threshold corresponding to the non-trading time is 85%.

[0065] Determining the first transmission rate V1 according to the total network bandwidth, the network bandwidth utilization rate, and the first utilization rate threshold is specifically:

[0066] V1 = T * (U1 - U), where T is the total network bandwidth, U1 is the first utilization rate threshold, and U is the network bandwidth utilization rate.

[0067] Preferably, the transmission rate threshold is 100 mb / s. To avoid affecting the network bandwidth of the real production environment, during the transaction, the dynamic transmission rate is preferably set to a smaller value as much as possible. If the first transmission rate V1 is less than or equal to the transmission rate threshold, the first transmission rate is used as the dynamic transmission rate. If the first transmission rate is greater than the predetermined transmission rate threshold, or the network bandwidth utilization rate is greater than or equal to the first utilization rate threshold, the dynamic transmission rate is set to zero, and the network bandwidth is waited to be released until the corresponding first transmission rate V1 is less than or equal to the transmission rate threshold, or the network bandwidth utilization rate is less than the first utilization rate threshold. That is, during the transaction, the production data file is transmitted to the mirrored production environment only when the network bandwidth utilization rate is less than the first utilization rate threshold and the first transmission rate is less than the transmission rate threshold. For example, if the total network bandwidth is 500 mb / s, when the network bandwidth utilization rate is 30%, which is less than the first utilization rate threshold of 40%, the first transmission rate V1 = 500*(40% - 30%) = 50 mb / s, and the first transmission rate V1 is less than the transmission rate threshold of 100 mb / s, then the dynamic transmission rate is set to the first transmission rate V1 = 50 mb / s; when the network bandwidth utilization rate is 5%, the first transmission rate V1 = 500*(40% - 5%) = 175 mb / s, and the first transmission rate V1 is greater than the transmission rate threshold of 100 mb / s, then the dynamic transmission rate is set to zero, and the production data file is not transmitted to the mirrored production environment.

[0068] During the non-transaction period, determining the second transmission rate V2 according to the total network bandwidth, the network bandwidth utilization rate and the second utilization rate threshold is specifically:

[0069] V2 = T*(U2 - U), where T is the total network bandwidth, U2 is the second utilization rate threshold, and U is the network bandwidth utilization rate.

[0070] If the second transmission rate V2 is less than or equal to the transmission rate threshold, the transmission rate threshold is still used as the dynamic transmission rate. If the network bandwidth utilization rate is greater than or equal to the second utilization rate threshold, the dynamic transmission rate is set to zero, and the network bandwidth is waited to be released until the network bandwidth utilization rate is less than the second utilization rate threshold. In this embodiment, during the non-transaction period, the dynamic transmission rate is preferably set to a larger value. Even if the second transmission rate V2 is less than or equal to the transmission rate threshold, the transmission rate threshold is still used as the dynamic transmission rate, so that the network bandwidth can be fully utilized during the non-transaction period to accelerate the transmission speed of the production data file.

[0071] In traditional parallel settlement, the production data transmission usually adopts a fixed transmission rate, which cannot be dynamically adjusted according to the real-time network bandwidth of the actual production environment. This easily leads to problems such as bandwidth pressure, low transmission efficiency, and data insecurity. Among them, when the network bandwidth of the actual production environment is low, fixed-rate transmission may cause network congestion and affect the normal operation of the actual production environment; when the network bandwidth of the actual production environment is high, fixed-rate transmission cannot make full use of the available network bandwidth, resulting in low transmission efficiency. In this embodiment, the dynamic transmission rate is determined according to the dynamic speed limit policy, which can make full use of the available network bandwidth of the actual production environment during non-trading hours, significantly improving the data transmission efficiency. At the same time, during trading hours, the dynamic transmission rate is controlled by the network bandwidth usage rate and the transmission rate threshold to ensure that the data transmission will not affect the network stability of the actual production environment.

[0072] In one embodiment, after performing block division, encryption, and packaging and compression processing on the production data file to obtain the processed production data file, it further includes: obtaining the first hash value corresponding to the processed production data file;

[0073] After determining the dynamic transmission rate according to the predetermined dynamic speed limit policy and transmitting the processed production data file to the mirror production environment according to the dynamic transmission rate, it further includes: obtaining the second hash value of the production data file in the mirror production environment; comparing the first hash value with the second hash value to determine whether the production data file in the mirror production environment is consistent with the production data file in the actual production environment.

[0074] In this embodiment, after transmitting the production data file and before decrypting and decompressing the production data file, the second hash value corresponding to the production data file is calculated, and the first hash value is compared with the second hash value to determine whether the production data file in the mirror production environment is consistent with the production data file in the actual production environment, preventing the production data file from being tampered with, thereby improving the accuracy of parallel settlement.

[0075] Further, during the process of transmitting the production data file, the transmission status of each data block is monitored in real time. If a certain data block fails to be transmitted, it is automatically retried until the transmission is successful, preventing data loss and being beneficial to improving the accuracy of parallel settlement.

[0076] Step 103, on day T, in the mirror production environment, according to the predetermined database silent recovery algorithm, recover the database instance corresponding to the production data file in the same database.

[0077] In this embodiment, the production data file has been extracted to the mirrored production environment on T-1 day, and the consistency check of the production data file has been performed. Therefore, on T day, the database instance recovery needs to be completed. Since parallel clearing needs to verify aspects such as the new generation of high-order clearing capabilities and the coverage of clearing functions, parallel clearing needs to recover multiple database instances with the same name on the same database at the same time, such as: the database instance of public offering node data, the database instance of private placement node data, the database instance of pre-settlement node data, and the database instance of pre-initialization node data. The database in this embodiment is the database in the mirrored production environment. With the help of the database's own recovery ability, the database instance corresponding to the production data file can be recovered in the same database through a predetermined database silent recovery algorithm. Generally, one production data file of a trading system corresponds to one database instance.

[0078] In one embodiment, on T day, in the mirrored production environment, according to a predetermined database silent recovery algorithm, recovering the database instance corresponding to the production data file in the same database includes:

[0079] On T day, obtain the key information corresponding to the production data file in the mirrored production environment, generate a recovery private file according to the key information, and the key information at least includes the file name, the path where it is located, the database name to which the file belongs, the new database name, and the new target path of the production data file; according to the recovery private file, recover the database instance corresponding to the production data file in the same database in the mirrored production environment, rename the database instances with the same name to obtain the new database name, and redirect the production data file of the database instance to restore the production data file to the new target path.

[0080] In this embodiment, the key information corresponding to the production data file is dynamically generated into a recovery private file, and the corresponding database instance can be directly recovered through the key information. During the recovery process, rename the database instances with the same name to avoid database conflicts; through the redirection technology, restore the production data file to the specified new target path to ensure that the storage location of the production data file is real, available, and does not conflict with other files.

[0081] In traditional parallel clearing, the recovery of database instances usually requires manual operations, which has problems such as conflicts of database instances with the same name and low efficiency. In this embodiment, the database instances are automatically recovered through the database silent recovery algorithm, and the database instances with the same name can be renamed and the production data files can be redirected, improving the efficiency of database instance recovery, reducing manual intervention, and reducing the risk of operation errors, which is beneficial to improving the accuracy of parallel clearing.

[0082] In addition, this embodiment supports the compatibility with various databases (such as GAUSSDB, TDSQL, SQL Server, Oracle, etc.), ensuring the universality and flexibility of the recovery process; this embodiment can concurrently call to recover private files and perform recovery operations on multiple database instances simultaneously, significantly improving the recovery efficiency; this embodiment can use multi-threading technology to ensure that the recovery tasks of multiple database instances can be executed in parallel, reducing the recovery time.

[0083] Further, before renaming the database instances with the same name when recovering the database instance corresponding to the production data file in the same database of the mirror production environment according to the recovered private file, it further includes: automatically releasing all links and processes of the database instance through the original database command;

[0084] After redirecting the production data file of the database instance to restore the production data file to a predetermined new path, it further includes: verifying key fields of the production data file of the database instance to make the production data file of the database instance consistent with the production data file of the real production environment, and the key fields at least include the transaction date and the transaction status.

[0085] In this embodiment, before recovering the database instance, automatically releasing all corresponding links and processes through the original database command can ensure that the recovery process will not fail due to resource occupation. This embodiment can detect and release links and processes through an automated tool to ensure the cleanliness and stability of the recovery environment.

[0086] In this embodiment, verification is performed through key fields to ensure that the production data file of the database instance is consistent with the production data file of the real production environment, ensuring the success of the recovery operation, which is beneficial to improving the accuracy of parallel settlement. If the key field verification fails, the re-recovery operation is automatically triggered until the recovery is successful.

[0087] Step 104, extract the settlement file from the real production environment to the mirror production environment, and perform parallel settlement in the mirror production environment based on the settlement file, the production data file, and the database instance.

[0088] In this embodiment, the settlement files are all small files and will not affect the network bandwidth, so the settlement files can be directly extracted from the real production environment to the mirror production environment one by one.

[0089] Further, since the production data is extracted to the mirror production environment in advance, this embodiment can perform multiple rounds of parallel settlement on T day in view of the sufficient time, greatly shortening the parallel settlement verification time, improving the efficiency, and multiple rounds of settlement verification can be performed on T day.

[0090] In an embodiment of the present invention, the parallel settlement process is refined and split. The parallel settlement process on day T is split into an automated process on day T-1 and a settlement process on day T. Since the production data can be extracted to the mirrored production environment using the automated process in the evening on day T-1, the parallel settlement on day T only needs to perform operations such as extracting settlement files, restoring data instances, and restarting relevant settlement systems to execute the parallel settlement. Therefore, parallel settlement can be carried out in a timely manner, and the efficiency of parallel settlement can be greatly improved. In addition, by using the production data in the real production environment for mirrored parallel settlement, the problem of inconsistent data between the data in traditional parallel settlement and the production environment is effectively solved, improving accuracy and reliability.

[0091] Figure 3 The structure diagram of the parallel settlement device based on mirrored production data according to an embodiment of the present invention is shown. As Figure 3 shown, the parallel settlement device 300 based on mirrored production data includes:

[0092] A creation module 301, configured to create a mirrored production environment corresponding to the real production environment. On day T-1, a preset monitoring subtask and a mirrored data subtask are run. The mirrored production environment is created in the real production environment or the test environment, and the network and devices of the mirrored production environment are isolated from those of the real production environment.

[0093] A transmission module 302, configured to transfer each production data file in each trading system participating in the parallel settlement from the real production environment to the mirrored production environment based on the monitoring subtask and the mirrored data subtask.

[0094] A restoration module 303, configured to, on day T, restore the database instance corresponding to the production data file in the same database in the mirrored production environment according to a predetermined database silent restoration algorithm.

[0095] A settlement module 304, configured to extract the settlement file from the real production environment to the mirrored production environment and perform parallel settlement in the mirrored production environment based on the settlement file, the production data file, and the database instance.

[0096] Among them, the embodiment of the parallel settlement device 300 based on mirrored production data is basically the same as the embodiment of the parallel settlement method based on mirrored production data, and will not be elaborated here.

[0097] Figure 4 The structure diagram of the computer device according to an embodiment of the present invention is shown. The specific implementation of the computer device is not limited in the specific embodiment of the present invention.

[0098] As Figure 4As shown in the figure, the computer device may include: a processor 402, a communications interface 404, a memory 406, and a communication bus 408.

[0099] Among them: The processor 402, the communications interface 404, and the memory 406 communicate with each other through the communication bus 408. The communications interface 404 is used to communicate with network elements of other computer devices such as clients or other servers. The processor 402 is used to execute the program 410, and specifically can execute the relevant steps in the above embodiments for the computer device.

[0100] Specifically, the program 410 may include program code, and the program code includes computer-executable instructions.

[0101] The processor 402 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention. One or more processors included in the computer device may be of the same type of processor, such as one or more CPUs; or may be of different types of processors, such as one or more CPUs and one or more ASICs.

[0102] The memory 406 is used to store the program 410. The memory 406 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.

[0103] Specifically, the program 410 can be specifically called by the processor 402 to cause the computer device to perform the following operations:

[0104] Establish an image production environment corresponding to the real production environment, and on the day T-1, run a preset monitoring subtask and an image data subtask. The image production environment is established in the real production environment or the test environment, and the network and devices of the image production environment are isolated from those of the real production environment;

[0105] Based on the monitoring subtask and the image data subtask, transfer each production data file in each trading system participating in parallel settlement from the real production environment to the image production environment;

[0106] On the day T, in the image production environment, according to a predetermined database silent recovery algorithm, recover the database instance corresponding to the production data file in the same database;

[0107] Extract the settlement file from the real production environment to the mirrored production environment, and perform parallel liquidation in the mirrored production environment based on the settlement file, the production data file, and the database instance.

[0108] In an alternative manner, the transfer of each production data file in each trading system participating in parallel liquidation from the real production environment to the mirrored production environment based on the monitoring subtask and the mirrored data subtask includes:

[0109] Run at least one of the monitoring subtasks for each trading system, and monitor whether the production data file of the trading system is in a generated state based on the monitoring subtask;

[0110] If so, trigger the mirrored data subtask, and transfer the production data file to the mirrored production environment based on the mirrored data subtask;

[0111] The transfer of the production data file to the mirrored production environment based on the mirrored data subtask includes:

[0112] Clean the target storage space corresponding to the production data file;

[0113] Chunk, encrypt, and pack and compress the production data file to obtain a processed production data file;

[0114] Determine the dynamic transfer rate according to a predetermined dynamic rate limiting policy, and transfer the processed production data file to the mirrored production environment according to the dynamic transfer rate.

[0115] In an alternative manner, the determination of the dynamic transfer rate according to a predetermined dynamic rate limiting policy includes:

[0116] Obtain the total network bandwidth and network bandwidth utilization rate of the real production environment, and determine whether the current time is trading time;

[0117] If the current time is the trading time, determine whether the network bandwidth utilization rate is less than a preset first utilization rate threshold. If it is less than the preset first utilization rate threshold, determine the first transfer rate according to the total network bandwidth, the network bandwidth utilization rate, and the first utilization rate threshold. If the first transfer rate is less than or equal to a predetermined transfer rate threshold, use the first transfer rate as the dynamic transfer rate. If the first transfer rate is greater than the predetermined transfer rate threshold, or the network bandwidth utilization rate is greater than or equal to the first utilization rate threshold, set the dynamic transfer rate to zero and wait for the network bandwidth to be released;

[0118] If the current time is a non-trading time, determine whether the network bandwidth utilization rate is less than a preset second utilization rate threshold. If it is less than the preset second utilization rate threshold, determine a second transmission rate according to the total network bandwidth, the network bandwidth utilization rate, and the second utilization rate threshold. If the second transmission rate is greater than the transmission rate threshold, use the second transmission rate as the dynamic transmission rate. If the second transmission rate is less than or equal to the transmission rate threshold, use the transmission rate threshold as the dynamic transmission rate. If the network bandwidth utilization rate is greater than or equal to the second utilization rate threshold, set the dynamic transmission rate to zero and wait for the network bandwidth to be released.

[0119] In an alternative implementation, after performing the operations of chunking, encrypting, and packing and compressing the production data file to obtain the processed production data file, the method further includes:

[0120] Obtaining a first hash value corresponding to the processed production data file;

[0121] After determining the dynamic transmission rate according to a predetermined dynamic speed limit policy and transmitting the processed production data file to the mirror production environment according to the dynamic transmission rate, the method further includes:

[0122] Obtaining a second hash value of the production data file in the mirror production environment;

[0123] Comparing the first hash value with the second hash value to determine whether the production data file in the real production environment is consistent with the production data file in the mirror production environment.

[0124] In an alternative implementation, on day T, in the mirror production environment, according to a predetermined database silent recovery algorithm, recovering a database instance corresponding to the production data file in the same database includes:

[0125] On day T, in the mirror production environment, obtaining key information corresponding to the production data file, and generating a recovery private file according to the key information. The key information includes at least the file name, the path where the file is located, the name of the database to which the file belongs, the name of the new database, and the new target path.

[0126] Recovering the database instance corresponding to the production data file in the same database in the mirror production environment according to the recovery private file, renaming the database instances with the same name to obtain the new database name, and redirecting the production data file of the database instance to restore the production data file to the new target path.

[0127] In an alternative approach, before renaming the database instances with the same name when restoring the database instance corresponding to the production data file in the same database of the mirror production environment according to the restored private file, it further includes:

[0128] Automatically release all links and processes of the database instance through the native database commands;

[0129] After redirecting the production data file of the database instance to restore the production data file to a predetermined new path, it further includes:

[0130] Verify the key fields of the production data file of the database instance to make the production data file of the database instance consistent with the production data file of the real production environment. The key fields at least include the transaction date and the transaction status.

[0131] In an alternative approach, the performing parallel liquidation based on the settlement file, the production data file, and the database instance in the mirror production environment includes:

[0132] Perform multiple rounds of parallel liquidation based on the settlement file, the production data file, and the database instance in the mirror production environment.

[0133] An embodiment of the present invention provides a computer-readable storage medium. The storage medium stores at least one executable instruction. When the executable instruction runs on a computer device, it causes the computer device to execute any of the above method embodiments.

[0134] An embodiment of the present invention provides a computer program. The computer program can be called by a processor to cause a computer device to execute any of the above method embodiments.

[0135] An embodiment of the present invention provides a computer program product. The computer program product includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions. When the program instructions run on a computer, it causes the computer to execute any of the above method embodiments.

[0136] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The structure required to construct such systems will be apparent from the above description. In addition, the embodiments of the present invention are not directed to any particular programming language. It should be understood that the content of the present invention described herein can be implemented using various programming languages, and the description of the specific language above is for disclosing the best mode of the present invention.

[0137] In the specification provided herein, numerous specific details are set forth. However, it will be understood that embodiments of the present invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail so as not to obscure an understanding of this description.

[0138] Similarly, it should be understood that in order to streamline the present invention and assist in understanding one or more of the various inventive aspects, in the foregoing description of the exemplary embodiments of the present invention, the various features of the embodiments of the present invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, the disclosed method should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim.

[0139] Those skilled in the art will appreciate that the modules in the computer devices in the embodiments can be adaptively changed and disposed in one or more computer devices different from those of the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all the features disclosed in this specification (including the accompanying claims, abstract and drawings) and all the processes or units of any method or computer device so disclosed. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract and drawings) can be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0140] It should be noted that the above embodiments illustrate rather than limit the present invention, and those skilled in the art can design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in the claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In the unit claims listing several devices, several of these devices can be embodied by the same item of hardware. The use of the words first, second, and third, etc. does not denote any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

Claims

1. A parallel liquidation method based on mirror production data, characterized in that The method includes: Establishing a mirror production environment corresponding to the real production environment, and on the (T - 1)th day, running a preset monitoring sub - task and a mirror data sub - task. The mirror production environment is established in the real production environment or the test environment, and the network and devices of the mirror production environment are isolated from those of the real production environment; Based on the monitoring sub - task and the mirror data sub - task, transferring each production data file in each trading system participating in parallel settlement from the real production environment to the mirror production environment; On the Tth day, in the mirror production environment, according to a predetermined database silent recovery algorithm, recovering the database instance corresponding to the production data file in the same database; Extracting the settlement file from the real production environment to the mirror production environment, and performing parallel settlement in the mirror production environment based on the settlement file, the production data file, and the database instance.

2. The method according to claim 1, wherein The transferring each production data file in each trading system participating in parallel settlement from the real production environment to the mirror production environment based on the monitoring sub - task and the mirror data sub - task includes: Running at least one of the monitoring sub - tasks for each trading system, and based on the monitoring sub - task, monitoring whether the production data file of the trading system is in a generated state; If so, triggering the mirror data sub - task, and based on the mirror data sub - task, transferring the production data file to the mirror production environment; The transferring the production data file to the mirror production environment based on the mirror data sub - task includes: Cleaning the target storage space corresponding to the production data file; Chunking, encrypting, and packing and compressing the production data file to obtain a processed production data file; Determining a dynamic transmission rate according to a predetermined dynamic rate - limiting policy, and transferring the processed production data file to the mirror production environment according to the dynamic transmission rate.

3. The method according to claim 2, wherein The determining a dynamic transmission rate according to a predetermined dynamic rate - limiting policy includes: Obtaining the total network bandwidth and network bandwidth utilization rate of the real production environment, and determining whether the current time is trading time; If the current time is trading time, determining whether the network bandwidth utilization rate is less than a preset first utilization rate threshold. If it is less than the preset first utilization rate threshold, determining a first transmission rate according to the total network bandwidth, the network bandwidth utilization rate, and the first utilization rate threshold. If the first transmission rate is less than or equal to a predetermined transmission rate threshold, using the first transmission rate as the dynamic transmission rate. If the first transmission rate is greater than the predetermined transmission rate threshold, or the network bandwidth utilization rate is greater than or equal to the first utilization rate threshold, setting the dynamic transmission rate to zero and waiting for the network bandwidth to be released; If the current time is a non-trading time, determine whether the network bandwidth utilization rate is less than a preset second utilization rate threshold. If it is less than the preset second utilization rate threshold, determine a second transmission rate according to the total network bandwidth, the network bandwidth utilization rate, and the second utilization rate threshold. If the second transmission rate is greater than the transmission rate threshold, use the second transmission rate as the dynamic transmission rate. If the second transmission rate is less than or equal to the transmission rate threshold, use the transmission rate threshold as the dynamic transmission rate. If the network bandwidth utilization rate is greater than or equal to the second utilization rate threshold, set the dynamic transmission rate to zero and wait for the network bandwidth to be released.

4. The method according to claim 2, wherein After performing block division, encryption, and packaging and compression processing on the production data file to obtain the processed production data file, it further includes: Obtain the first hash value corresponding to the processed production data file; After determining the dynamic transmission rate according to the predetermined dynamic speed limit policy and transmitting the processed production data file to the mirror production environment according to the dynamic transmission rate, it further includes: Obtain the second hash value of the production data file in the mirror production environment; Compare the first hash value with the second hash value to determine whether the production data file in the real production environment is consistent with the production data file in the mirror production environment.

5. The method according to claim 1, characterized in that On day T, in the mirror production environment, according to the predetermined database silent recovery algorithm, recover the database instance corresponding to the production data file in the same database, including: On day T, in the mirror production environment, obtain the key information corresponding to the production data file, and generate a recovery private file according to the key information. The key information at least includes the file name, location path, database name to which the file belongs, new database name, and new target path of the production data file; According to the recovery private file, recover the database instance corresponding to the production data file in the same database in the mirror production environment. Rename the database instances with the same name to obtain the new database name, and redirect the production data file of the database instance to restore the production data file to the new target path.

6. The method according to claim 5, characterized in that, Before renaming the database instances with the same name according to the recovery private file to recover the database instance corresponding to the production data file in the same database in the mirror production environment, it further includes: Automatically release all links and processes of the database instance through the database native command; After redirecting the production data file of the database instance to restore the production data file to a predetermined new path, it further includes: Verify the key fields of the production data file of the database instance to make the production data file of the database instance consistent with the production data file in the real production environment. The key fields at least include the transaction date and transaction status.

7. The method according to any one of claims 1 to 6, characterized in that Performing parallel liquidation in the mirror production environment based on the settlement file, the production data file, and the database instance, including: Execute multiple rounds of parallel settlement in the mirror production environment based on the settlement document, the production data file, and the database instance.

8. A parallel clearing device based on mirror production data, characterized in that, The device includes: A building module, configured to build a mirror production environment corresponding to the real production environment. On day T-1, run a preset monitoring subtask and a mirror data subtask. The mirror production environment is built in the real production environment or the test environment, and the network and devices of the mirror production environment are isolated from those of the real production environment. A transmission module, configured to transfer each production data file in each trading system participating in the parallel settlement from the real production environment to the mirror production environment based on the monitoring subtask and the mirror data subtask. A recovery module, configured to, on day T, in the mirror production environment, according to a predetermined database silent recovery algorithm, recover the database instance corresponding to the production data file in the same database. A settlement module, configured to extract the settlement document from the real production environment to the mirror production environment, and perform parallel settlement in the mirror production environment based on the settlement document, the production data file, and the database instance.

9. A computer device, characterized in that, It includes: A processor, a memory, a communication interface, and a communication bus. The processor, the memory, and the communication interface complete communication with each other through the communication bus. The memory is used to store at least one executable instruction, and the executable instruction causes the processor to execute the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, At least one executable instruction is stored in the storage medium. When the executable instruction runs on a computer device, it causes the computer device to execute the method according to any one of claims 1-7.

Images