Data processing method and device, equipment, medium and program product

Abstract

The invention provides a data processing method and device, equipment, a medium and a program product, which can be applied to the technical field of data processing and financial science and technology. The method comprises the steps that a plurality of preparation commands used for daily switching preprocessing are sent to a plurality of distributed nodes respectively, so that the distributed nodes generate target date data based on a target date according to the preparation commands respectively, and data access routes of the distributed nodes point to current data corresponding to a current date before a target moment, the target date data indicates that any distributed node switches the current date to a preparation state before the target date; determining a processing result of daily switching preprocessing corresponding to each of the plurality of distributed nodes; and respectively sending a plurality of daily switching commands to the plurality of distributed nodes at the target moment, so that the plurality of distributed nodes respectively switch a data access route pointing to the current data to a data access route pointing to the target date data at the target moment.

Description

Technical Field

[0001] This application relates to the fields of data processing and financial technology, and specifically to a data processing method, apparatus, device, medium, and program product. Background Technology

[0002] "Daily switching" refers to the process by which a business system switches between two dates to complete the termination, settlement, and reconciliation of transactions for the old date, and to begin transactions for the new date. It is also a periodic operation performed by the business system to complete daily settlements and perform system maintenance. However, the efficiency of daily switching methods in related technologies is relatively low. Summary of the Invention

[0003] In view of the above problems, embodiments of this application provide a data processing method, apparatus, device, medium, and program product.

[0004] According to a first aspect of the embodiments of this application, a data processing method is provided, the method comprising: when the time between the current node time of a global primary node and a target time is less than a preset time period, sending a plurality of preparation commands for day-cutting preprocessing to a plurality of distributed nodes, such that the plurality of distributed nodes generate target date data based on the target date according to the plurality of preparation commands, wherein the data access routes of the distributed nodes point to the current data corresponding to the current date before the target time, and the target date data indicates that any of the distributed nodes will switch the current date to a preparation state before the target date; determining the processing result of the day-cutting preprocessing corresponding to the plurality of distributed nodes according to the reply information of the day-cutting preprocessing corresponding to the plurality of distributed nodes; and when it is determined that the plurality of processing results indicate that the processing results of the day-cutting preprocessing of the plurality of distributed nodes all indicate that the preparation state of the distributed nodes is complete, sending a plurality of day-cutting commands to the plurality of distributed nodes at the target time, such that the plurality of distributed nodes switch the data access routes pointing to the current data to the target date data at the target time.

[0005] According to an embodiment of this application, determining the processing result of the daily cut-off preprocessing corresponding to each of the multiple distributed nodes based on the response information of the daily cut-off preprocessing corresponding to each of the multiple distributed nodes includes: if all the response information indicates that the target date data was successfully generated, determining that the processing result of the daily cut-off preprocessing corresponding to the distributed node indicates that the preparation status of the distributed node is "preparation complete"; and / or if one of the response information indicates that the target date data was not generated, determining that the processing result of the daily cut-off preprocessing corresponding to the distributed node indicates that the preparation status of the distributed node is "preparation incomplete"; and / or if no response information corresponding to the distributed node is obtained before the target time, determining that the processing result of the daily cut-off preprocessing corresponding to each of the multiple distributed nodes indicates that the preparation status of the distributed node is "preparation incomplete".

[0006] According to an embodiment of this application, the distributed node includes a first sub-distributed node with a first priority and a second sub-distributed node with a second priority, wherein the first priority has a higher priority than the second priority; the first sub-distributed node generates target date data based on the target date in a first sub-time period, and the second sub-distributed node generates target date data based on the target date in a second sub-time period; the start time of the first sub-time period is after the current node time, the end time of the first sub-time period is before the start time of the second sub-time period, and the end time of the second sub-time period is before the target time.

[0007] According to an embodiment of this application, the method further includes: when it is determined that at least one of the multiple processing results representing the daily cutting preprocessing of the multiple distributed nodes indicates that the preparation status of the distributed node is incomplete, sending a rollback instruction to the multiple distributed nodes, so that the multiple distributed nodes roll back their respective target date data according to the rollback instruction; and generating alarm information based on the distributed nodes that are incomplete in preparation.

[0008] According to an embodiment of this application, the method further includes: in the event of a failure of the global primary node, determining a target global backup node from at least one global backup node; and restoring interaction with the multiple distributed nodes through the target global backup node based on the node log data of the global primary node.

[0009] According to an embodiment of this application, the operation of the distributed node switching the data access route from the current data to the target date data at the target time is an atomic operation.

[0010] According to embodiments of this application, each of the aforementioned distributed nodes includes a clock synchronization process, and performs time synchronization with the aforementioned global master node through the aforementioned clock synchronization process.

[0011] A second aspect of this application provides a data processing apparatus, comprising: a first sending module, configured to send multiple preparation commands for day-cutting preprocessing to multiple distributed nodes when the time between the current node time of a global primary node and a target time is less than a preset time interval, such that each of the multiple distributed nodes generates target date data based on the target date according to the multiple preparation commands, wherein the data access route of the distributed nodes points to the current data corresponding to the current date before the target time, and the target date data indicates that any of the distributed nodes switches the current date to a preparation state before the target date; a first determining module, configured to determine the processing result of the day-cutting preprocessing corresponding to each of the multiple distributed nodes according to the reply information of the day-cutting preprocessing corresponding to each of the multiple distributed nodes; and a second sending module, configured to send multiple day-cutting commands to each of the multiple distributed nodes at the target time when it is determined that the processing results of the multiple processing results indicate that the preparation state of the multiple distributed nodes is complete, such that each of the multiple distributed nodes switches the data access route pointing to the current data to the target date data at the target time.

[0012] A third aspect of this application provides an electronic device, including: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0013] A fourth aspect of this application provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0014] A fifth aspect of this application also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method.

[0015] According to embodiments of this application, the data processing method divides day-switching into two stages: a preparation stage and a switching stage. In the preparation stage, i.e., before the target time, the target date data to be generated for day-switching is generated in advance, and the data access route still points to the current data corresponding to the current date. Therefore, the generation of the target date data will not affect the business of the distributed nodes corresponding to the current date. In the switching stage, based on the response information from the distributed nodes' day-switching preprocessing, it is determined whether multiple distributed nodes are ready. If all distributed nodes are ready, a day-switching command can be sent to multiple distributed nodes at the target time, enabling them to perform day-switching at the target time. Since the target date data is generated in advance, the day-switching process at the target time only requires changing the access data route to point to the target date data, thus improving the efficiency of day-switching. Attached Figure Description

[0016] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0017] Figure 1 The illustrations depict application scenarios of data processing methods, apparatuses, devices, media, and program products according to embodiments of this application.

[0018] Figure 2 A flowchart illustrating a data processing method according to an embodiment of this application is shown schematically.

[0019] Figure 3 A flowchart illustrating a data processing method according to yet another embodiment of this application is shown.

[0020] Figure 4 This illustration schematically shows the relationship between the global primary node and multiple distributed nodes according to an embodiment of this application;

[0021] Figure 5 A schematic block diagram of a data processing apparatus according to an embodiment of this application is shown; and

[0022] Figure 6 A block diagram schematically illustrates an electronic device suitable for implementing a data processing method according to an embodiment of this application. Detailed Implementation

[0023] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0024] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0025] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0026] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0027] In the technical solution of this application, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0028] In scenarios involving automated decision-making using personal information, the methods, devices, and systems provided in this application all offer users corresponding entry points for choosing to agree to or reject the automated decision-making results. If the user chooses to reject, the process proceeds to the expert decision-making stage. Here, "automated decision-making" refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests, or economic, health, and credit status through computer programs, and then making a decision. Here, "expert decision-making" refers to the activity of making decisions by personnel who specialize in a particular field, possess specialized experience, knowledge, and skills, and have reached a certain level of professional expertise.

[0029] Daily cutoffs can be initiated manually by operations and maintenance personnel according to a pre-defined daily cutoff schedule or operational plan, issuing cutoff instructions to each application individually, or executing the daily cutoff program manually on a batch control platform. Each application performs the daily cutoff operation upon receiving the instruction, including business date switching, data transfer, and status updates. However, manual operation is a sequential process, resulting in a long overall system daily cutoff completion time and low efficiency. Furthermore, manual operation requires the participation of operations and maintenance personnel, necessitating 24-hour on-call duty, leading to high labor costs. Manual operation is also prone to errors, such as misoperation, omissions, and incorrect sequence, potentially causing inconsistencies in data across business systems. Moreover, due to the inconsistent daily cutoff times across different business systems, arbitrage opportunities may arise in cross-application transactions due to the time difference in daily cutoffs (for example, trading when application A has already completed its daily cutoff while application B has not, potentially profiting from the time difference).

[0030] Therefore, a unified daily switching method using a distributed platform is proposed. This method involves a centralized accounting application (usually the core system) issuing daily switching instructions to all other business systems after completing its own daily switching. Other business systems must simultaneously meet two conditions to perform a daily switching: reaching the predetermined switching time and receiving the unified instructions. However, since data generation is required during daily switching, it consumes time, reducing efficiency. Furthermore, network latency or delays in the central application issuing instructions can lead to inconsistent instruction receipt times across applications, resulting in inconsistent actual execution times for the daily switching. If the central application malfunctions and cannot issue instructions, the entire system's daily switching cannot proceed. The small time difference between instruction reception and execution can create arbitrage opportunities between different business systems (e.g., asynchronous daily switching times between two business systems can be exploited for arbitrage in interbank transactions). Moreover, time consistency cannot be guaranteed: true synchronous switching is impossible, and unavoidable time differences exist.

[0031] In view of this, embodiments of this application provide a data processing method, including: when the time between the current node time of the global primary node and the target time is less than a preset time period, sending multiple preparation commands for day-cutting preprocessing to multiple distributed nodes, so that each of the multiple distributed nodes generates target date data based on the target date according to the multiple preparation commands, the data access route of the distributed nodes points to the current data corresponding to the current date before the target time, and the target date data indicates that any distributed node switches the current date to the preparation state before the target date; determining the processing result of the day-cutting preprocessing corresponding to each of the multiple distributed nodes according to the reply information of the day-cutting preprocessing corresponding to each of the multiple distributed nodes; when it is determined that the multiple processing results all indicate that the preparation state of the distributed nodes is complete, sending multiple day-cutting commands to each of the multiple distributed nodes at the target time, so that each of the multiple distributed nodes switches the data access route pointing to the current data to the target date data at the target time.

[0032] Figure 1 The illustrations depict application scenarios of data processing methods, apparatuses, devices, media, and program products according to embodiments of this application.

[0033] like Figure 1 As shown, application scenario 100 according to this embodiment may include a first distributed server 101, a second distributed server 102, a third distributed server 103, a network 104, and a global primary server 105. Network 104 serves as a medium for providing communication links between the first distributed server 101, the second distributed server 102, the third distributed server 103, and the global primary server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0034] The first distributed server 101, the second distributed server 102, and the third distributed server 103 can interact with the global primary server 105 through the network 104 to receive or send messages, etc.

[0035] The global primary server 105 can send instructions to the first distributed server 101, the second distributed server 102, and the third distributed server 103, such as preparation instructions or day-cutting instructions.

[0036] It should be noted that the data processing method provided in this application embodiment can generally be executed by the global primary server 105. Correspondingly, the data processing device provided in this application embodiment can generally be located in the global primary server 105. The data processing method provided in this application embodiment can also be executed by a server or server cluster that is different from the global primary server 105 and capable of communicating with the first distributed server 101, the second distributed server 102, the third distributed server 103, and / or the global primary server 105. Correspondingly, the data processing device provided in this application embodiment can also be located in a server or server cluster that is different from the global primary server 105 and capable of communicating with the first distributed server 101, the second distributed server 102, the third distributed server 103, and / or the global primary server 105.

[0037] It should be understood that Figure 1 The number of distributed servers, networks, and global master servers shown is merely illustrative. Depending on implementation needs, any number of distributed servers, networks, and global master servers can be included.

[0038] Figure 2 A flowchart illustrating a data processing method according to an embodiment of this application is shown.

[0039] like Figure 2 As shown, the data processing method of this embodiment may include operations S210 to S230.

[0040] In operation S210, if the time between the current node time of the global primary node and the target time is less than a preset time period, multiple preparation commands for day cutting preprocessing are sent to multiple distributed nodes, so that each of the multiple distributed nodes generates target date data based on the target date according to the multiple preparation commands.

[0041] The data access route of the distributed nodes points to the current data corresponding to the current date before the target time. The target date data indicates that any distributed node will switch the current date to the preparation state before the target date.

[0042] In operation S220, the processing results of the daily cut-off preprocessing corresponding to each of the multiple distributed nodes are determined based on the response information of the daily cut-off preprocessing corresponding to each of the multiple distributed nodes.

[0043] In operation S230, if it is determined that multiple processing results indicate that the preparation status of the distributed nodes is complete, multiple date-switching commands are sent to each of the multiple distributed nodes at the target time, so that each of the multiple distributed nodes switches the data access route pointing to the current data to the target date data at the target time.

[0044] The global master node can interact with distributed nodes, enabling it to send commands and receive messages. The global master node can be used for coordination at various stages of the day-switching process, including preparation and execution, and ultimately sends the day-switching command to the distributed nodes.

[0045] By setting up a scheduled task, multiple preparation commands can be sent to each of the distributed nodes when the time difference between the current node time of the global primary node and the target time is less than a preset time period. For example, if the target time is 00:00 on day T+1, the current node time of the global primary node is 23:46 on day T, and the preset time period is 15 minutes, then multiple preparation commands can be sent to each of the distributed nodes based on the current node time of the global primary node.

[0046] Distributed nodes can generate target date data based on a target date according to preparation commands. Target date data can be the data originally intended to be generated when switching the current date to the target date at a target time. Distributed nodes can generate target date data through batch programs. Batch programs typically execute automatically during off-peak business periods (such as 11 PM to 2 AM) to complete non-real-time business operations such as data calculation, accounting processing, and report generation. Batch programs are characterized by non-real-time processing, execution within a specific time window, handling large amounts of data (including full or large batches), relatively fixed execution time, periodicity, and automated operation without manual intervention. Therefore, distributed nodes can generate target date data through batch programs, which are computer programs that centrally process large batches of data according to preset business logic and time schedules.

[0047] Since the target date data is generated on the current date based on the target date, and the data access routes of the distributed nodes point to the current data corresponding to the current date before the target time, the target date data will not affect the business processing of the distributed nodes on the current date.

[0048] Distributed nodes can send reply information to the global master node after daily cut-off preprocessing, such as "ready to go," thereby determining the processing results of the daily cut-off preprocessing corresponding to each of the multiple distributed nodes.

[0049] If the preprocessing results of the corresponding daily cut-off operations on multiple distributed nodes all indicate that the preparation readiness of the distributed nodes is complete, meaning the distributed nodes can perform the daily cut-off, a daily cut-off command can be sent to multiple distributed nodes at the target time. This causes the multiple distributed nodes to perform the daily cut-off at the target time, switching the data access route from the current data to the target date data. Thus, the target date data can be accessed at the target time. Furthermore, the distributed nodes can send reply information, indicating that the connection between the global primary node and the distributed nodes is normal.

[0050] According to embodiments of this application, the data processing method divides day-switching into two stages: a preparation stage and a switching stage. In the preparation stage, i.e., before the target time, the target date data to be generated for day-switching is generated in advance, and the data access route still points to the current data corresponding to the current date. Therefore, the generation of the target date data will not affect the business of the distributed nodes corresponding to the current date. In the switching stage, based on the response information from the distributed nodes' day-switching preprocessing, it is determined whether multiple distributed nodes are ready. If all distributed nodes are ready, a day-switching command can be sent to multiple distributed nodes at the target time, enabling them to perform day-switching at the target time. Since the target date data is generated in advance, the day-switching process at the target time only requires changing the access data route to point to the target date data, thus improving the efficiency of day-switching.

[0051] According to an embodiment of this application, the data processing method may further include: determining a target global standby node from at least one global standby node in the event of a failure of the global primary node; and restoring interaction with multiple distributed nodes through the target global standby node based on the node log data of the global primary node.

[0052] Multiple global standby nodes can be configured for the global primary node; for example, there can be one, two, or three global standby nodes. The region of the global standby nodes can be different from that of the global primary node.

[0053] In the event of a failure in the global primary node, such as a hardware failure on the server hosting the global primary node, a target global backup node can be determined based on the backup status of at least one global backup node. The time of failure is then determined based on the node log data of the global primary node, and interaction with multiple distributed nodes is restored accordingly.

[0054] According to the embodiments of this application, by setting up a global backup node, the security of daily switching processing is increased, the fault tolerance of the data processing method of this application embodiment is improved, and the global blocking caused by a single point of failure of the global primary node is reduced.

[0055] According to embodiments of this application, determining the processing result of the daily cut-off preprocessing corresponding to each of the multiple distributed nodes based on the response information of the daily cut-off preprocessing corresponding to each of the multiple distributed nodes may include: if multiple response information indicates that the target date data was successfully generated, determining that the processing result of the daily cut-off preprocessing corresponding to the distributed node indicates that the preparation status of the distributed node is "preparation complete"; and / or if one of the multiple response information indicates that the target date data was not generated, determining that the processing result of the daily cut-off preprocessing corresponding to the distributed node indicates that the preparation status of the distributed node is "preparation incomplete"; and / or if no response information corresponding to the distributed node is obtained before the target time, determining that the processing result of the daily cut-off preprocessing corresponding to each of the multiple distributed nodes indicates that the preparation status of the distributed node is "preparation incomplete".

[0056] The response information for the daily cutoff preprocessing of distributed nodes can include both successful and failed generation of target date data. For example, successful generation can be responded with "Ready," while failed generation can be responded with "Abort." Distributed nodes may also not send a response after daily cutoff preprocessing. Therefore, if multiple responses indicate successful generation of target date data, it can be determined that the processing results of the daily cutoff preprocessing corresponding to the distributed node all indicate that the distributed node's preparation status is complete, meaning daily cutoff can proceed. If one of the multiple responses indicates that the target date data generation failed, it can be determined that the processing results of the daily cutoff preprocessing corresponding to the distributed node indicate that the distributed node's preparation status is incomplete, meaning daily cutoff cannot proceed. If no response information corresponding to the distributed node is obtained before the target time, it can be determined that the processing results of the daily cutoff preprocessing corresponding to each of the distributed nodes indicate that the distributed node's preparation status is incomplete, meaning daily cutoff cannot proceed.

[0057] According to the embodiments of this application, by judging the response information of the distributed nodes or the situation where no response information is obtained, it is possible to judge in advance whether multiple distributed nodes can be switched on a daily basis, thus avoiding the inability to switch on a daily basis near the target time.

[0058] According to an embodiment of this application, the data processing method may further include: when it is determined that at least one of the multiple processing results representing the daily cutting preprocessing of multiple distributed nodes indicates that the preparation status of the distributed node is incomplete, sending a rollback instruction to the multiple distributed nodes so that the multiple distributed nodes roll back their respective target date data according to the rollback instruction; and generating alarm information based on the distributed nodes that are incomplete in preparation.

[0059] Multiple distributed nodes can be related to each other. If at least one of the distributed nodes has not completed preparation, that is, if day cutting cannot be performed, a rollback command can be sent to multiple distributed nodes to roll back the target date data and restore the state in which day cutting preprocessing is being performed.

[0060] Distributed nodes that are not yet ready may be faulty. Therefore, alarm information can be generated based on the node information of the distributed nodes that are not yet ready.

[0061] According to an embodiment of this application, by having multiple distributed nodes roll back the target date data and generate alarm information when day switching cannot be performed, multiple distributed nodes can recover quickly, and the distributed nodes that have not completed preparation can be checked based on the alarm information.

[0062] According to an embodiment of this application, a distributed node may include a first sub-distributed node with a first priority and a second sub-distributed node with a second priority, wherein the first priority has a higher priority than the second priority; the first sub-distributed node generates target date data based on a target date in a first sub-time period, and the second sub-distributed node generates target date data based on a target date in a second sub-time period; the start time of the first sub-time period is after the current node time, the end time of the first sub-time period is before the start time of the second sub-time period, and the end time of the second sub-time period is before the target time.

[0063] Distributed nodes can have different priorities. Priority characterizes the impact of a distributed node on the business system after a daily switchover. Higher-priority distributed nodes can be prioritized for daily switchover preprocessing to avoid congestion caused by multiple nodes performing preprocessing simultaneously. The highest-priority first distributed node can also be prioritized for daily switchover, while the lowest-priority second distributed node can be switched over after the first node. The switching time of the second distributed node will not affect the business processing of the first distributed node. Distributed nodes can be tagged to distinguish different priorities; for example, tags could be "accounting-related," "channel," or "corporate." The global primary node can define the priorities and dependencies of distributed nodes during the preparation and switchover phases based on these tags, ensuring that the highest-priority first distributed node is prepared and switched over first.

[0064] According to embodiments of this application, by distinguishing between first and second distributed nodes with different priorities, network congestion caused by all distributed nodes preparing and switching at the same time can be avoided.

[0065] According to an embodiment of this application, the operation of a distributed node switching the data access route from the current data to the target date data at a target time is an atomic operation.

[0066] According to embodiments of this application, atomic operations can ensure the continuity of data access route switching and improve the efficiency of data access route switching.

[0067] The date can also be changed by updating global configuration items in memory; for example, a key-value pair that identifies the current business date can be modified. This switching action is designed to be completed within 10 milliseconds.

[0068] According to embodiments of this application, each of the multiple distributed nodes includes a clock synchronization process, and performs time synchronization with the global master node through the clock synchronization process.

[0069] Distributed nodes can ensure high-precision synchronization between their local clocks and the global master node and other nodes under that distributed node through a clock synchronization process. High-precision time protocols (such as PTP) can be connected to all nodes, controlling clock deviations between nodes to the microsecond or even nanosecond level, laying the foundation for synchronized day-switching. Because each node is connected to a high-precision time protocol, day-switching commands can be sent to multiple distributed nodes at a precise target time, and multiple distributed nodes can also perform day-switching at the precise target time.

[0070] According to embodiments of this application, by using high-precision time synchronization and parallel switching, the daily switching latency of thousands of applications can be controlled within 10 milliseconds.

[0071] Figure 3 A flowchart illustrating a data processing method according to yet another embodiment of this application is shown.

[0072] like Figure 3 As shown, the method includes operations S310 to S350.

[0073] In operation S310, if the time between the current node time of the global primary node and the target time is less than a preset time period, multiple preparation commands for day-cutting preprocessing are sent to multiple distributed nodes, so that each of the multiple distributed nodes generates target date data based on the target date according to the multiple preparation commands.

[0074] In operation S320, based on the response information of the daily cut-off preprocessing corresponding to each of the multiple distributed nodes, it is determined whether the processing results of the daily cut-off preprocessing corresponding to each of the multiple distributed nodes indicate that the preparation status of the distributed nodes is complete. If yes, operation S330 is executed; otherwise, operation S340 is executed.

[0075] In operation S330, multiple date-switching commands are sent to multiple distributed nodes at the target time, so that each of the multiple distributed nodes switches the data access route that points to the current data to the target date data at the target time.

[0076] In operation S340, rollback instructions are sent to multiple distributed nodes so that the multiple distributed nodes can roll back their respective target date data according to the rollback instructions.

[0077] When operating the S350, alarm information is generated based on the distributed nodes that have not yet completed preparation.

[0078] According to the embodiments of this application, operations S310 to S350 can be described with reference to the descriptions of other embodiments of this application, and will not be repeated here.

[0079] Figure 4 The diagram illustrates the relationship between the global primary node and multiple distributed nodes according to an embodiment of this application.

[0080] like Figure 4 As shown, the global primary node 410 can connect to multiple distributed nodes 420, and the global backup node 430 can also connect to multiple distributed nodes 420. During the preparation phase, if the time difference between the current node time and the target time is less than a preset period, the global primary node 410 can send multiple preparation commands for day-cutting preprocessing to each of the distributed nodes. Each of the distributed nodes 420 generates target date data based on the target date according to the multiple preparation commands and sends a response message for day-cutting preprocessing to the global primary node 410. The global primary node 410 determines the processing result of the day-cutting preprocessing corresponding to each of the distributed nodes based on the response messages. During the day-cutting phase, if multiple processing results indicate that the preparation status of the distributed nodes is complete, multiple day-cutting commands are sent to each of the distributed nodes at the target time, causing each of the distributed nodes to switch its data access route from the current data to the target date data at the target time. The number of global standby nodes 430 can be multiple. In the event of a failure of the global primary node 410, a target global standby node can be determined from the multiple global standby nodes 430. Based on the node log data of the global primary node 410, the interaction between the target global standby node and the multiple distributed nodes 420 can be restored.

[0081] According to embodiments of this application, the interaction between the global primary node and multiple distributed nodes can be applied to end-of-day accounting, interest calculation, risk indicator calculation, regulatory report generation, and customer statement generation in financial scenarios. The daily preprocessing of each distributed node enables the decomposition and parallel processing of daily tasks, multi-node collaborative work, resource sharing and load balancing, system fault tolerance, and high availability. This improves the data throughput of the daily processing method in this application embodiment, enhances system reliability and fault tolerance, supports horizontal system scaling, and improves resource utilization efficiency.

[0082] Distributed batch jobs typically refer to batch job processing implemented using distributed computing technology. Traditional centralized batch jobs are decomposed into multiple parallel-executable subtasks, which are then collaboratively completed through a distributed computing cluster, or distributed nodes, to handle large-scale data processing. Compared to traditional serial batch jobs, the processing method changes from serial to parallel processing, resource utilization shifts from single-machine resources to cluster resources, significantly increasing processing capacity and providing failover and retry mechanisms. By using multiple distributed nodes to complete distributed batch jobs, distributed decomposition and scheduling of batch jobs, multi-node parallel computing, data partitioning and load balancing, task execution status monitoring, and fault tolerance can be achieved.

[0083] Daily batch switching operations are special batch programs executed at the end of each business day in financial systems. They are responsible for switching the system from the current business date to the next, completing date-related data carry-over, status updates, and initialization, ensuring the system can smoothly transition to the new business cycle. They handle business date switching and updates, daily business data carry-over, and system status initialization. This is a critical step in ensuring business continuity, guaranteeing the accuracy and completeness of financial data, maintaining system data consistency, and supporting the normal operation of the following day's business.

[0084] The global primary node and multiple distributed nodes can be managed through the Unified Batch Job Management Platform (UBJMP). The UBJMP enables batch job scheduling and management, providing a unified interface and standardized processes to manage, monitor, and schedule batch job execution across the entire financial institution. It offers unified scheduling and orchestration of batch jobs, management of job dependencies, real-time monitoring of execution status, resource allocation and load balancing, anomaly handling and fault recovery, performance analysis and optimization, and more. This achieves standardized management of batch jobs, improves batch processing efficiency and reliability, reduces operational complexity and labor costs, provides comprehensive monitoring and early warning capabilities, and supports performance optimization for batch jobs.

[0085] Based on the above data processing method, this application also provides a data processing apparatus. The following will be combined with... Figure 5 The device is described in detail.

[0086] Figure 5 A schematic block diagram of a data processing apparatus according to an embodiment of this application is shown.

[0087] like Figure 5 As shown, the data processing device 500 of this embodiment includes a first sending module 510, a first determining module 520, and a second sending module 530.

[0088] The first sending module 510 is configured to send multiple preparation commands for day-switching preprocessing to multiple distributed nodes when the time between the current node time and the target time of the global primary node is less than a preset time period. This allows each distributed node to generate target date data based on the target date according to the multiple preparation commands. The data access routes of the distributed nodes point to the current data corresponding to the current date before the target time. The target date data indicates that any distributed node will switch its current date to a preparation state prior to the target date. In one embodiment, the first sending module 510 can be used to execute the operation S210 described above, which will not be repeated here.

[0089] The first determining module 520 is used to determine the processing result of the daily cutting preprocessing corresponding to each of the multiple distributed nodes based on the response information of the daily cutting preprocessing corresponding to each of the multiple distributed nodes. In one embodiment, the first determining module 520 can be used to execute the operation S220 described above, which will not be repeated here.

[0090] The second sending module 530 is configured to send multiple date-switching commands to each of the distributed nodes at a target time when multiple processing results indicate that the prepared state of the distributed nodes is ready. This causes each of the distributed nodes to switch its data access route from the current data to the target date data at the target time. In one embodiment, the second sending module 530 can be used to perform the operation S230 described above, which will not be repeated here.

[0091] According to an embodiment of this application, a first determining module 520 for determining the processing result of the daily cutting preprocessing corresponding to each of the multiple distributed nodes based on the response information of the daily cutting preprocessing corresponding to each of the multiple distributed nodes includes a first determining unit, a second determining unit, and a third determining unit.

[0092] The first determining unit is used to determine, when multiple response messages all indicate that the target date data has been successfully generated, that the processing results of the day-cutting preprocessing corresponding to the distributed node all indicate that the preparation status of the distributed node is complete.

[0093] The second determining unit is used to determine, in the case where one of the multiple response messages indicates that the target date data generation has failed, the processing result of the day-cutting preprocessing corresponding to the distributed node indicates that the preparation status of the distributed node is incomplete.

[0094] The third determining unit is used to determine, in the case that no response information corresponding to the distributed node is obtained before the target time, the processing result of the daily cutting preprocessing corresponding to each of the multiple distributed nodes indicates that the preparation status of the distributed node is not completed.

[0095] According to an embodiment of this application, the distributed node includes a first sub-distributed node with a first priority and a second sub-distributed node with a second priority, wherein the first priority has a higher priority than the second priority; the first sub-distributed node generates target date data based on the target date in the first sub-time period, and the second sub-distributed node generates target date data based on the target date in the second sub-time period; the start time of the first sub-time period is after the current node time, the end time of the first sub-time period is before the start time of the second sub-time period, and the end time of the second sub-time period is before the target time.

[0096] According to embodiments of this application, the apparatus further includes a third sending module and a generating module.

[0097] The third sending module is used to send a rollback instruction to multiple distributed nodes when it is determined that at least one of the multiple processing results representing the daily cutting preprocessing of multiple distributed nodes indicates that the preparation status of the distributed node is incomplete, so that the multiple distributed nodes can roll back their respective target date data according to the rollback instruction.

[0098] The generation module is used to generate alarm information based on the distributed nodes that are not yet ready.

[0099] According to embodiments of this application, the device further includes a second determining module and a recovery module.

[0100] The second determination module is used to determine a target global standby node from at least one global standby node in the event that the global primary node is faulty.

[0101] The recovery module is used to restore the interaction between multiple distributed nodes through the target global standby node based on the node log data of the global primary node.

[0102] According to an embodiment of this application, the operation of a distributed node switching the data access route from the current data to the target date data at a target time is an atomic operation.

[0103] According to embodiments of this application, each of the multiple distributed nodes includes a clock synchronization process, and performs time synchronization with the global master node through the clock synchronization process.

[0104] According to embodiments of this application, any plurality of modules among the first transmitting module 510, the first determining module 520, and the second transmitting module 530 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least a portion of the functionality of one or more of these modules can be combined with at least a portion of the functionality of other modules and implemented in one module. According to embodiments of this application, at least one of the first transmitting module 510, the first determining module 520, and the second transmitting module 530 can be at least partially implemented as a hardware circuit, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented by any other reasonable means of integrating or packaging the circuit, or implemented in any one of the three methods of software, hardware, and firmware, or in a suitable combination of any of these. Alternatively, at least one of the first transmitting module 510, the first determining module 520, and the second transmitting module 530 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0105] Figure 6 A block diagram schematically illustrates an electronic device suitable for implementing a data processing method according to an embodiment of this application.

[0106] like Figure 6 As shown, an electronic device 600 according to an embodiment of this application includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage portion 608 into a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this application.

[0107] RAM 603 stores various programs and data required for the operation of electronic device 600. Processor 601, ROM 602, and RAM 603 are interconnected via bus 604. Processor 601 executes various operations of the method flow according to embodiments of this application by executing programs in ROM 602 and / or RAM 603. It should be noted that the programs may also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 may also execute various operations of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0108] According to embodiments of this application, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to a bus 604. The electronic device 600 may also include one or more of the following components connected to the input / output (I / O) interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 610 as needed so that computer programs read from it can be installed into the storage section 608 as needed.

[0109] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0110] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include ROM 602 and / or RAM 603 and / or one or more memories other than ROM 602 and RAM 603 described above.

[0111] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to enable the computer system to implement the data processing methods provided in the embodiments of this application.

[0112] When the computer program is executed by the processor 601, it performs the functions defined in the system / apparatus of this application embodiment. According to the embodiments of this application, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0113] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 609, and / or installed from the removable medium 611. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0114] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, it performs the functions defined in the system of this application embodiment. According to the embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0115] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0116] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0117] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

Claims

1. A data processing method, characterized in that, The method includes: If the time between the current node time and the target time of the global primary node is less than a preset time period, multiple preparation commands for daily cutting preprocessing are sent to multiple distributed nodes, so that each of the multiple distributed nodes generates target date data based on the target date according to the multiple preparation commands. The data access route of the distributed nodes points to the current data corresponding to the current date before the target time. The target date data indicates that any of the distributed nodes will switch the current date to the preparation state before the target date. Based on the response information of the daily cut-off preprocessing corresponding to each of the multiple distributed nodes, determine the processing result of the daily cut-off preprocessing corresponding to each of the multiple distributed nodes; If all the processing results indicate that the preparation status of the distributed node is complete, multiple date-switching commands are sent to each of the distributed nodes at the target time, so that each of the distributed nodes switches the data access route pointing to the current data to the target date data at the target time.

2. The method according to claim 1, characterized in that, The step of determining the processing result of the daily cut-off preprocessing corresponding to each of the multiple distributed nodes based on the response information of the daily cut-off preprocessing corresponding to each of the multiple distributed nodes includes: If multiple response messages indicate that the target date data was successfully generated, the processing results of the day-cutting preprocessing corresponding to the distributed node all indicate that the preparation status of the distributed node is complete; and / or If, among multiple response messages, one message indicates that the target date data generation failed, the processing result of the day-cutting preprocessing corresponding to the distributed node is determined to indicate that the preparation status of the distributed node is incomplete; and / or If no response information corresponding to the distributed node is obtained before the target time, the processing result of the daily preprocessing corresponding to each of the distributed nodes is determined to indicate that the preparation status of the distributed node is not complete.

3. The method according to claim 1, characterized in that, The distributed nodes include a first sub-distributed node with a first priority and a second sub-distributed node with a second priority, wherein the first priority has a higher priority than the second priority. The first sub-distributed node generates target date data based on the target date in the first sub-time period, and the second sub-distributed node generates target date data based on the target date in the second sub-time period. The start time of the first sub-time period is after the current node time, the end time of the first sub-time period is before the start time of the second sub-time period, and the end time of the second sub-time period is before the target time.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: If it is determined that at least one of the multiple processing results characterizes the day-cutting preprocessing of the multiple distributed nodes and indicates that the preparation status of the distributed node is incomplete, a rollback instruction is sent to the multiple distributed nodes so that the multiple distributed nodes roll back their respective target date data according to the rollback instruction. Based on the distributed nodes that have not yet completed preparation, generate alarm information.

5. The method according to claim 4, characterized in that, The method further includes: In the event of a failure of the global primary node, a target global backup node is determined from at least one global backup node; Based on the node log data of the global primary node, the interaction between the target global backup node and the multiple distributed nodes is restored.

6. The method according to any one of claims 1 to 4, characterized in that, The operation by which the distributed node switches the data access route from the current data to the target date data at the target time is an atomic operation.

7. The method according to any one of claims 1 to 4, characterized in that, Each of the distributed nodes includes a clock synchronization process, and performs time synchronization with the global master node through the clock synchronization process.

8. A data processing apparatus, characterized in that, The device includes: The first sending module is used to send multiple preparation commands for day-cutting preprocessing to multiple distributed nodes when the time between the current node time and the target time of the global primary node is less than a preset time period. This allows the multiple distributed nodes to generate target date data based on the target date according to the multiple preparation commands. The data access route of the distributed nodes points to the current data corresponding to the current date before the target time. The target date data indicates that any of the distributed nodes will switch the current date to the preparation state before the target date. The first determining module is used to determine the processing result of the daily cutting preprocessing corresponding to each of the multiple distributed nodes based on the response information of the daily cutting preprocessing corresponding to each of the multiple distributed nodes. The second sending module is used to send multiple date-switching commands to each of the distributed nodes at the target time when it is determined that multiple processing results indicate that the preparation status of the distributed nodes is complete, so that each of the distributed nodes switches the data access route pointing to the current data to the target date data at the target time.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

Images