Data synchronization method, apparatus, device, medium, and program product

Abstract

This specification discloses a data synchronization method, apparatus, device, medium, and program product. The method is applied to a distributed cluster system comprising multiple single-machine replicas, each running an independent virtual machine. The method includes: responding to a timed scheduling instruction to control each single-machine replica to start a data loading and parsing task at the beginning of a synchronization time window; the duration of the synchronization time window is less than the scheduling cycle of the timed scheduling instruction; during the execution of the data loading and parsing task, when it is detected that a target single-machine replica has completed the data loading and parsing task and generated data to be deployed, triggering a deployment blocking mechanism for the target single-machine replica; when the end of the synchronization time window is reached, releasing the blocking state of all single-machine replicas under the deployment blocking mechanism, controlling multiple single-machine replicas to perform data deployment operations simultaneously, synchronizing their respective data to be deployed to their respective virtual machines.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a data synchronization method, apparatus, device, medium, and program product. Background Technology

[0002] In consumer finance reserve fund operation scenarios, real-time consistency of strategy deployment is crucial. Related technologies typically employ a distributed cluster architecture to distribute and deploy operational strategies across the Java Virtual Machines (JVMs) of individual nodes within the cluster, supporting high-throughput marketing traffic. However, in actual strategy updates and deployments, differences in hardware load, network conditions, and parsing capabilities among the individual nodes within the cluster lead to varying processing times for sub-nodes such as "querying, parsing, and deployment."

[0003] This time difference directly leads to a serious "second-level inconsistency" problem: within the same second, different versions of policies may exist simultaneously in the JVM memory of different single-machine replicas in the cluster, or there may be missing policy replicas. Under high-concurrency marketing traffic, this can cause consecutive requests from the same customer group or the same user to be routed to different policy branches, resulting in policy jumps, inconsistent underwriting results, and even the risk of financial loss. Traditional asynchronous parallel deployment methods cannot solve this problem of node state asynchrony caused by differences in processing time. Summary of the Invention

[0004] This specification provides a data synchronization method, apparatus, device, medium, and program product. Without affecting data loading and parsing efficiency, it avoids data deployment time differences caused by variations in hardware load, network conditions, etc., by briefly blocking a single-machine copy of the task to complete it in advance. This achieves uniformity in global data deployment time in a distributed cluster system, reduces business logic confusion or data conflicts that may be caused by asynchronous data deployment, solves the problem of second-level inconsistency in data synchronization in distributed cluster systems, and further enhances the reliability and accuracy of system operation. The above technical solution is as follows: In a first aspect, embodiments of this specification provide a data synchronization method applied to a distributed cluster system comprising multiple single-machine replicas, each of which runs an independent virtual machine to perform data processing. The method includes: In response to the timed scheduling command, each of the aforementioned single-machine replicas is controlled to start the data loading and parsing task at the beginning of the synchronization time window; the duration of the aforementioned synchronization time window is less than the scheduling cycle duration of the aforementioned timed scheduling command. During the execution of the above data loading and parsing tasks, the task completion status of each of the above single-machine replicas is monitored. When it is detected that the target single-machine replica has completed the above data loading and parsing tasks and generated data to be deployed, the deployment blocking mechanism for the above target single-machine replica is triggered to block the data deployment operation of the above target single-machine replica until the end of the above synchronization time window. When the end time of the above synchronization time window is reached, the blocking state of all single-machine replicas under the deployment blocking mechanism is released, and the above multiple single-machine replicas are controlled to perform data deployment operations at the same time, synchronizing their respective data to be deployed to their respective virtual machines.

[0005] In one possible implementation, the aforementioned timed scheduling instruction is configured to define a synchronization time window at the end of each scheduling cycle, wherein the aforementioned start time corresponds to the opening time of the aforementioned synchronization time window, and the aforementioned end time corresponds to the end time of the aforementioned scheduling cycle or the start time of the next scheduling cycle.

[0006] In one possible implementation, the duration of the synchronization time window is determined based on the historical processing time of the historical data loading and parsing tasks corresponding to each of the multiple single-machine replicas; or, the duration of the synchronization time window is determined based on the estimated processing time of the current data loading and parsing tasks corresponding to each of the multiple single-machine replicas; or, the duration of the synchronization time window is greater than or equal to the maximum historical processing time of the historical data loading and parsing tasks corresponding to each of the multiple single-machine replicas or the maximum estimated processing time of the current data loading and parsing tasks corresponding to each of the multiple single-machine replicas.

[0007] In one possible implementation, the aforementioned deployment blocking mechanism for the target single-machine replica includes: Start a blocking timer. The duration of the blocking timer is equal to the time difference from the current moment to the end of the synchronization time window. Set the thread state of the aforementioned target single-machine replica to a waiting state until the aforementioned blocking timer times out.

[0008] In one possible implementation, after monitoring the task completion status of each of the aforementioned single-machine replicas during the execution of the aforementioned data loading and parsing tasks, the method further includes: When the end time of the above synchronization time window is reached, check whether there are any non-target single-machine replicas among the above multiple single-machine replicas that have not yet completed the data loading and parsing tasks; If so, it is determined that the data synchronization of the aforementioned non-target single-machine replica has failed, and an alarm signal is triggered to interrupt the data deployment operation of the aforementioned non-target single-machine replica.

[0009] In one possible implementation, after triggering the deployment blocking mechanism for the target single-machine replica and before releasing all single-machine replicas from the deployment blocking mechanism, the method further includes: The integrity of the data to be deployed is verified by utilizing the waiting time reserved by the deployment blocking mechanism. If the verification fails, the above deployment blocking mechanism will be terminated, and the target single-machine replica will be prevented from performing subsequent data deployment operations.

[0010] In one possible implementation, the start time of the aforementioned synchronization time window is dynamically adjustable; the method also includes: Within the historical preset scheduling period, the maximum historical time taken for each single-machine replica to complete the data loading and parsing tasks; The target window duration for this scheduling cycle is calculated based on the maximum historical latency and the preset safety margin. Based on the target window duration and scheduling cycle duration mentioned above, the start time of the corresponding synchronization time window for this scheduling cycle is calculated in reverse to dynamically adjust the trigger time of the aforementioned timed scheduling instructions.

[0011] Secondly, embodiments of this specification provide a data synchronization device applied to a distributed cluster system comprising multiple single-machine replicas, each of which runs an independent virtual machine to perform data processing. The device includes: The timed scheduling module is used to respond to the timed scheduling command and control each of the above-mentioned single-machine replicas to start the data loading and parsing task at the beginning of the synchronization time window; the duration of the above-mentioned synchronization time window is less than the scheduling cycle duration of the above-mentioned timed scheduling command. The deployment blocking module is used to monitor the task completion status of each of the above-mentioned single-machine replicas during the execution of the above-mentioned data loading and parsing tasks. When it is detected that the target single-machine replica has completed the above-mentioned data loading and parsing tasks and generated data to be deployed, the deployment blocking mechanism for the target single-machine replica is triggered to block the data deployment operation of the target single-machine replica until the end of the above-mentioned synchronization time window. The data deployment module is used to release the blocking state of all single-machine replicas under the deployment blocking mechanism when the end time of the above synchronization time window is reached, and control the above multiple single-machine replicas to perform data deployment operations at the same time, synchronizing their respective data to be deployed to their respective virtual machines.

[0012] Thirdly, embodiments of this specification provide an electronic device, including: a processor and a memory; The processor is connected to the memory. The aforementioned memory is used to store executable program code; The processor reads the executable program code stored in the memory to run the program corresponding to the executable program code, so as to execute the method provided in the first aspect of the embodiments of this specification or any possible implementation of the first aspect.

[0013] Fourthly, embodiments of this specification provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the method provided in the first aspect of the embodiments of this specification or any possible implementation thereof.

[0014] Fifthly, embodiments of this specification provide a computer program product containing instructions that, when run on a computer or processor, cause the computer or processor to execute the data synchronization method provided in the first aspect or any possible implementation of the first aspect of the embodiments of this specification.

[0015] In the embodiments of this specification, on the one hand, the timed scheduling instructions control all single-machine replicas in the distributed cluster system to start data loading and parsing at the same time (the start time of the synchronization time window), avoiding the problem of asynchronous progress of data loading and parsing tasks due to differences in start time. Furthermore, by ensuring that the duration of the synchronization time window is less than the scheduling cycle of the timed scheduling instructions, sufficient time is reserved for data processing, avoiding task backlog or timeout risks, and ensuring the reliability of the timed scheduling. On the other hand, during the execution of data loading and parsing tasks, the task completion status of each single-machine replica is monitored. When it is detected that the target single-machine replica has completed the data loading and parsing task and generated data to be deployed, a deployment blocking mechanism for the target single-machine replica is triggered. This blocks the data deployment operation of the target single-machine replica until the end time of the synchronization time window, thereby avoiding the problem of hardware load, etc., without affecting the efficiency of data loading and parsing, by briefly blocking single-machine replicas that have completed their tasks ahead of time. The time difference in data deployment caused by differences in network conditions and other factors has achieved uniformity in the global data deployment time in the distributed cluster system. This reduces business logic chaos or data conflicts that may be caused by asynchronous data deployment, and solves the problem of second-level inconsistency in data synchronization in the distributed cluster system, further enhancing the reliability and accuracy of system operation. On the other hand, when the end time of the synchronization time window is reached, the blocking state of all single-machine replicas under the deployment blocking mechanism is released, and multiple single-machine replicas are controlled to perform data deployment operations at the same time, synchronizing their respective data to be deployed to their respective virtual machines. This ensures that the data versions of all virtual machines in the distributed cluster are completely consistent, avoiding policy jumps or insurance result conflicts. Through millisecond or even nanosecond-level data deployment synchronization, high throughput and real-time performance are maintained while approaching strong consistency, bridging the gap between strong and weak consistency, and preventing continuous requests from the same customer group or user from being routed to different policy branches, reducing the risk of asset loss. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A schematic diagram of the architecture of a distributed cluster system provided for an exemplary embodiment of this specification; Figure 2 A flowchart illustrating a data synchronization method provided for an exemplary embodiment of this specification; Figure 3 A flowchart illustrating another data synchronization method provided as an exemplary embodiment of this specification; Figure 4 A schematic diagram illustrating a dynamic adjustment process for a synchronization time window, provided as an exemplary embodiment of this specification; Figure 5 A schematic diagram of a data synchronization device provided for an exemplary embodiment of this specification; Figure 6 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this specification. Detailed Implementation

[0018] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings.

[0019] The terms "first," "second," "third," etc., used in this specification, claims, and the foregoing drawings are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0020] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in the embodiments of this specification are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the data to be deployed involved in this specification were obtained under full authorization.

[0021] Please refer to the following. Figure 1 , Figure 1 This is a schematic diagram of the architecture of a distributed cluster system provided for an exemplary embodiment of this specification. Figure 1 As shown, the distributed cluster system may include: cloud storage 110 and a virtual machine cluster. Among them: Cloud Storage 110 serves as a centralized data source for each virtual machine in a virtual machine cluster (e.g., Figure 1 The virtual machines 120a, 120b, and 120c shown provide the raw data. The virtual machine cluster may, but is not limited to, consist of multiple independently running virtual machines, each reading data from cloud storage 110 in parallel and executing its own tasks.

[0022] The data processing flow for each virtual machine can be divided into, but is not limited to, three stages: data loading, data parsing, and data deployment. In the data loading stage, the virtual machine pulls the data to be processed from cloud storage 110. In the data parsing stage, the virtual machine formats, calculates, and processes the pulled data to generate the data processing results (data to be deployed). In the data deployment stage, the data to be deployed is written to the virtual machine's heap memory area to atomically replace the old version of the policy data already existing in the virtual machine, thus achieving seamless switching of policy versions.

[0023] The aforementioned distributed cluster system comprises multiple single-machine replicas, each running an independent virtual machine to perform data processing. A single-machine replica refers to the program instance and the data it manages within the virtual machine; it is the entity that performs actions such as data loading, data parsing, and data deployment, while the virtual machine is the carrier that runs the corresponding single-machine replica.

[0024] The aforementioned distributed cluster system may also include, but is not limited to, a controller ( Figure 1(Not shown in the diagram). Specifically, the controller can respond to timed scheduling instructions triggered at regular intervals to control each single-machine replica to start data loading and parsing tasks at the beginning of the synchronization time window. The duration of the synchronization time window is shorter than the scheduling cycle of the timed scheduling instruction. During the execution of the data loading and parsing tasks, the controller monitors the task completion status of each single-machine replica. When it detects that the target single-machine replica has completed the data loading and parsing tasks and generated data to be deployed, it triggers a deployment blocking mechanism for the target single-machine replica to block the data deployment operation of the target single-machine replica until the end of the synchronization time window. When the end of the synchronization time window is reached, the controller releases the blocking state of all single-machine replicas under the deployment blocking mechanism, controls multiple single-machine replicas to perform data deployment operations at the same time, and synchronizes their respective data to be deployed to their respective virtual machines.

[0025] The network can be a medium that provides communication links between virtual machines in cloud storage 110 and virtual machine clusters, or it can be the Internet, which includes network devices and transmission media, and is not limited to these. The transmission media can be a wired link, such as, but not limited to, coaxial cable, fiber optic cable, and digital subscriber line (DSL), or a wireless link, such as, but not limited to, wireless fidelity (WIFI), Bluetooth, and mobile device networks.

[0026] Understandably, Figure 1 The number of cloud storage units 110 and virtual machines in the distributed cluster system shown is merely an example. In a specific implementation, the distributed cluster system can contain any number of cloud storage units 110 and virtual machines. This specification does not impose specific limitations on this aspect in the embodiments.

[0027] Next, combine Figure 1 This document describes the data synchronization method provided in the embodiments of this specification. Please refer to [link / reference needed] for details. Figure 2 This is a flowchart illustrating a data synchronization method provided in an exemplary embodiment of this specification. Figure 2 As shown, this data synchronization method includes the following steps: S202, in response to the timed scheduling command, controls each single-machine replica to start the data loading and parsing task at the beginning of the synchronization time window; the duration of the synchronization time window is less than the scheduling cycle duration of the timed scheduling command.

[0028] Specifically, the aforementioned timed scheduling instruction refers to a trigger signal issued by the controller of the distributed cluster at a preset time. For example, the controller may be set to send a timed scheduling instruction at 00:00:00 every day or every minute, and the sending period of this timed scheduling instruction (the time interval between two timed scheduling instructions) is the scheduling period (e.g., 24 hours, 1 minute, etc.). The aforementioned synchronization time window refers to a specific time period reserved within each scheduling period to complete the data loading and parsing tasks, such as, but not limited to, setting a 1-second time window from the 59th to the 60th second of each minute. The aforementioned single-machine replica refers to each independently running application instance (running in a virtual machine or container) in the distributed cluster, responsible for processing business requests and maintaining local cached data.

[0029] In a distributed cluster, the controller broadcasts a scheduled command to all replica machines at the start of the synchronization time window (e.g., the 59th second of every minute). Upon receiving the scheduled command, each replica machine immediately initiates its data loading and parsing tasks at the start of the synchronization time window. Within the synchronization time window, the replica machine pulls the latest data from persistent storage (such as a database, object storage, or cloud storage), parses it into in-memory objects, and prepares it for subsequent deployment.

[0030] Optionally, the duration of the aforementioned synchronization time window is determined based on the historical processing time of the historical data loading and parsing tasks corresponding to each of the multiple single-machine replicas; or, the duration of the aforementioned synchronization time window is determined based on the estimated processing time of the current data loading and parsing tasks corresponding to each of the multiple single-machine replicas; or, the duration of the aforementioned synchronization time window is greater than or equal to the maximum historical processing time or the maximum estimated processing time of the current data loading and parsing tasks corresponding to each of the multiple single-machine replicas.

[0031] The historical processing time mentioned above refers to the actual time spent by each single-machine replica in executing data loading and parsing tasks during past scheduling cycles. Recording and analyzing historical processing times provides a reference for setting synchronization time windows based on actual operating conditions, avoiding the blind setting of window durations. In other words, statistical analysis based on historical data (the historical processing time of each of the multiple single-machine replicas for their respective historical data loading and parsing tasks) can reflect the processing capabilities of different single-machine replicas under different data volumes and system loads. This allows the determined synchronization time window to better fit the actual application scenario, improving the rationality and accuracy of window settings and reducing the risk of tasks failing to complete due to excessively short windows.

[0032] The aforementioned estimated processing time refers to the pre-estimation of the time required for each individual machine to complete the task before the execution of this data loading and parsing task, based on factors such as the estimated size of the data to be loaded, the complexity of the data structure, and the current resource status of each individual machine (e.g., CPU, memory, network bandwidth utilization). Compared to simply relying on historical data, estimated processing time can more dynamically adapt to real-time task conditions. For example, when the amount of data to be loaded is much larger than the historical average, the estimated processing time can detect this change in advance and adjust the duration of the synchronization time window accordingly, making the window setting more forward-looking and flexible, further optimizing resource utilization and the reliability of task completion.

[0033] The aforementioned maximum historical processing time refers to the longest processing time recorded in the historical processing time records of multiple individual replicas. Setting the synchronization time window length to be greater than or equal to this value ensures that even the slowest-processing replica in history has sufficient time to complete data loading and parsing tasks within the current scheduling cycle. This minimizes the risk of data synchronization failure or inconsistency across the entire cluster due to the long processing time of a single replica, thus improving system robustness.

[0034] The maximum estimated processing time mentioned above refers to the largest estimated processing time among the multiple single-machine replicas for this data loading and parsing task. Setting the synchronization time window duration to be greater than or equal to this value ensures that the single-machine replica with the longest estimated processing time has sufficient time to complete the task for this specific data loading and parsing task. This allows for a more accurate match with the actual needs of the task and avoids insufficient or redundant window settings due to the limitations of historical data (such as not having encountered data of similar scale or complexity before). Thus, while ensuring task completion, time resources can be utilized more effectively.

[0035] In the embodiments described in this specification, determining the duration of the synchronization time window using the above method enables intelligent and dynamic allocation of reasonable time resources for the data synchronization process in the distributed cluster. This not only ensures that all single-machine replicas can complete data loading and parsing within a unified time window, thereby guaranteeing the consistency and timeliness of data across replicas within the cluster and providing a reliable data foundation for subsequent business deployments; it also avoids resource waste caused by excessively long time windows or task failures caused by excessively short time windows, optimizing the overall performance and operational efficiency of the system. This mechanism makes the distributed system more adaptable and stable when facing nodes with different data volumes, different processing capabilities, and dynamically changing tasks.

[0036] Please continue to refer to the following. Figure 2 ,like Figure 2As shown, in S202 above, in response to the timed scheduling instruction, each single-machine replica is controlled to start the data loading and parsing task at the beginning of the synchronization time window; if the duration of the synchronization time window is less than the scheduling cycle of the timed scheduling instruction, the data synchronization method may also include, but is not limited to: S204 During the execution of the data loading and parsing task, the task completion status of each single-machine replica is monitored. When it is detected that the target single-machine replica has completed the data loading and parsing task and generated data to be deployed, the deployment blocking mechanism for the target single-machine replica is triggered to block the data deployment operation of the target single-machine replica until the end of the synchronization time window.

[0037] Specifically, the aforementioned target single-machine replica refers to the single-machine replica that completes the data loading and parsing task first and generates the data to be deployed during the data loading and parsing process. The aforementioned deployment blocking mechanism is a strategy used to control the timing of data deployment operations. Its core function is to prevent the single-machine replica (target single-machine replica) that has completed data loading and parsing from immediately deploying the data, but instead delays the operation until the end of the synchronization time window for unified execution.

[0038] First, the system sets up a dedicated task status monitoring module for each single-machine replica. This module tracks the progress of data loading and parsing tasks for the corresponding single-machine replica in real time, for example, by listening to the return value of the task process and checking the integrity and size of the target files in the data generation directory to determine whether the task is completed. When the task status monitoring module of a single-machine replica (i.e., the target single-machine replica) detects that it has successfully completed data loading and parsing and generated complete and accurate data to be deployed, it will immediately send a task completion signal to the system controller. This signal may include, but is not limited to, the identification information of the target single-machine replica and the relevant metadata of the data to be deployed (e.g., but not limited to, data version, generation timestamp, etc.). Upon receiving this signal, the controller will immediately start the deployment blocking mechanism for the target single-machine replica. Specifically, the controller may, but is not limited to, mark the data to be deployed for the target single-machine replica as "ready" and temporarily remove the data deployment task item for the target single-machine replica from its internal task scheduling queue, or set a blocking flag so that the thread or process responsible for performing the data deployment operation will not process the data to be deployed for the target single-machine replica when it detects the flag. Meanwhile, the system continuously monitors the remaining duration of the synchronization time window. When the synchronization time window ends, the controller sends a deployment release signal to all blocked target single-machine replicas. At this time, the deployment blocking mechanism of these single-machine replicas is released, and their respective data to be deployed is deployed in a unified manner, thereby ensuring that the data deployment of all single-machine replicas is completed at the same time, further guaranteeing the consistency of data within the cluster.

[0039] Optionally, the aforementioned timed scheduling instruction is configured to define a synchronization time window at the end of each scheduling cycle, wherein the aforementioned start time corresponds to the opening time of the synchronization time window, and the aforementioned end time corresponds to the end time of the scheduling cycle or the start time of the next scheduling cycle.

[0040] In the embodiments of this specification, on the one hand, by setting the synchronization time window at the end of the scheduling cycle, it is ensured that the data loading and parsing tasks are completed within the scheduling cycle, avoiding resource conflicts caused by the superposition of cross-cycle tasks; on the other hand, by aligning the end time of the synchronization time window with the boundary of the scheduling cycle (such as the end time of the scheduling cycle or the start time of the next scheduling cycle), it is ensured that the old version data runs stably within the cycle, and the new version data is atomically replaced at the end of the window, avoiding the problem of cross-cycle data mixing, making the data synchronization logic of each cycle independent and traceable, and facilitating problem troubleshooting.

[0041] Optionally, the above-mentioned process of triggering the deployment blocking mechanism for the target single-machine replica may include, but is not limited to: starting a blocking timer, the duration of which is equal to the time difference from the current moment to the end of the synchronization time window; setting the thread state of the target single-machine replica to a waiting state until the blocking timer times out.

[0042] The aforementioned blocking timer is a timing tool used to control the waiting time of threads. Its core function is to count down according to a set timeout period, and trigger a specific event to release the thread from its waiting state when the timeout ends (i.e., timeout). In the embodiments of this specification, the timeout period of the blocking timer is precisely set to the time difference from the current moment to the end of the synchronization time window, so that the blocking duration of the target single-machine replica can dynamically adapt to the remaining time of the synchronization time window, ensuring that the blocking process is closely bound to the lifecycle of the synchronization time window. The aforementioned target single-machine replica refers to a single service instance or node in the distributed cluster system that has completed data loading and parsing tasks and generated corresponding data to be deployed, requiring data synchronization or version update, and is the direct target of the deployment blocking mechanism. The aforementioned setting of the thread state to a waiting state means pausing the execution of threads related to data deployment in the target single-machine replica through the thread control mechanism provided by the operating system or programming language, allowing it to temporarily perform data deployment operations.

[0043] First, when the system determines that a deployment blocking mechanism needs to be triggered on the target single-machine replica, it immediately starts a blocking timer. At this time, the system calculates the time interval between the current moment and the end of the synchronization time window, and sets this interval as the duration of the blocking timer. For example, if the current moment is the 8th second of the scheduling cycle and the synchronization time window ends at the 10th second of the cycle, the blocking timer duration is set to 2 seconds. Next, the system calls the thread control interface to switch the state of the critical thread in the target single-machine replica responsible for handling business requests or data operations from the running state to the waiting state. Threads in the waiting state release their occupied CPU resources but retain their context information so that they can quickly resume execution after the blocking timer expires. During the blocking timer's operation, the target single-machine replica will not directly enter the data deployment phase, nor will it execute internal operations unrelated to data synchronization, thus ensuring that it remains in a "silent" state within the synchronization time window and avoiding interference with the data synchronization process. When the blocking timer expires, a wake-up signal is sent to the target single-machine replica. At this time, the thread state of the target single-machine replica is restored from the waiting state to the running state, and the data deployment operation is started to complete the data synchronization.

[0044] In the embodiments of this specification, firstly, by precisely setting the duration of the blocking timer, it can be ensured that the blocking time of the target single-machine replica exactly covers the remaining part of the synchronization time window. This avoids version chaos caused by incomplete data synchronization due to premature unblocking, and also prevents service resource waste and response delay caused by delayed unblocking, thus achieving precise control of the blocking time. Secondly, by setting the thread state to a waiting state instead of directly terminating the thread, the overhead of thread reconstruction can be minimized. When the blocking ends, the thread can quickly resume execution to its pre-blocking state, improving the system's operating efficiency and response speed. Thirdly, through... By blocking the target single-machine replica within the synchronization time window, the data synchronization process is effectively isolated from normal business services. This ensures that the old version data will not be interfered with or damaged before the atomic replacement of data is completed, and the new version data will not be accessed in advance. This fundamentally eliminates the risk of cross-cycle data mixing and guarantees the security and consistency of data synchronization. Fourthly, this timer-based blocking mechanism is simple and efficient to implement, easy to integrate into existing scheduling frameworks, and has a small impact on system resources. It will not significantly affect overall performance and is easy for developers to debug and maintain, improving the operability and stability of the system.

[0045] Please continue to refer to the following. Figure 2 ,like Figure 2As shown in S204 above, during the execution of the data loading and parsing task, the task completion status of each single-machine replica is monitored. When it is detected that the target single-machine replica has completed the data loading and parsing task and generated data to be deployed, a deployment blocking mechanism for the target single-machine replica is triggered to block the data deployment operation of the target single-machine replica until the end of the synchronization time window. This data synchronization method may also include, but is not limited to, the following: S206, when the end time of the synchronization time window is reached, release the blocking state of all single-machine replicas under the deployment blocking mechanism, control multiple single-machine replicas to perform data deployment operations at the same time, and synchronize their respective data to be deployed to their respective virtual machines.

[0046] Specifically, the aforementioned data to be deployed refers to the final data set that conforms to the virtual machine data format requirements, generated after the single-machine replica has completed loading (such as reading data from a database, file system, or cloud storage) and parsing (such as format conversion, data validation, field mapping, etc.) of the original data. This data is usually stored in a temporary cache area on the replica's local machine and is in a ready state, ready for deployment immediately once the blockage is cleared.

[0047] The system may, but is not limited to, have a global timer that tracks in real time whether the current time has reached the end of the synchronization time window. When the timer detects that the current time has reached the end time, it broadcasts an unblocking signal to all single-machine replicas that are in a deployment blocked state. Upon receiving the signal, each single-machine replica resets its blocking flag to "false," and the deployment thread is awakened from the waiting state. Subsequently, each single-machine replica starts the data deployment process, reads the data to be deployed from the local temporary cache, and transmits the data to the corresponding virtual machine through a preset communication protocol (such as TCP / IP). Since all single-machine replicas unblock and start deployment at the same time, it can be guaranteed that the data received by each virtual machine is the same version, and the deployment process is highly consistent in time, effectively avoiding data inconsistency problems caused by deployment time differences. For example, if the synchronization time window ends at 2:10, the system sends an unblocking signal to all blocked single-machine replicas at this time, and replicas A, B, and C simultaneously begin to synchronize their respective data to be deployed to virtual machines A, B, and C. The entire deployment process is completed synchronously under the unified control of the system clock.

[0048] In the embodiments of this specification, on the one hand, the timed scheduling instructions control all single-machine replicas in the distributed cluster system to start data loading and parsing at the same time (the start time of the synchronization time window), avoiding the problem of asynchronous progress of data loading and parsing tasks due to differences in start time. Furthermore, by ensuring that the duration of the synchronization time window is less than the scheduling cycle of the timed scheduling instructions, sufficient time is reserved for data processing, avoiding task backlog or timeout risks, and ensuring the reliability of the timed scheduling. On the other hand, during the execution of data loading and parsing tasks, the task completion status of each single-machine replica is monitored. When it is detected that the target single-machine replica has completed the data loading and parsing task and generated data to be deployed, a deployment blocking mechanism for the target single-machine replica is triggered. This blocks the data deployment operation of the target single-machine replica until the end time of the synchronization time window, thereby avoiding the problem of hardware load, etc., without affecting the efficiency of data loading and parsing, by briefly blocking single-machine replicas that have completed their tasks ahead of time. The time difference in data deployment caused by differences in network conditions and other factors has achieved uniformity in the global data deployment time in the distributed cluster system. This reduces business logic chaos or data conflicts that may be caused by asynchronous data deployment, and solves the problem of second-level inconsistency in data synchronization in the distributed cluster system, further enhancing the reliability and accuracy of system operation. On the other hand, when the end time of the synchronization time window is reached, the blocking state of all single-machine replicas under the deployment blocking mechanism is released, and multiple single-machine replicas are controlled to perform data deployment operations at the same time, synchronizing their respective data to be deployed to their respective virtual machines. This ensures that the data versions of all virtual machines in the distributed cluster are completely consistent, avoiding policy jumps or insurance result conflicts. Through millisecond or even nanosecond-level data deployment synchronization, high throughput and real-time performance are maintained while approaching strong consistency, bridging the gap between strong and weak consistency, and preventing continuous requests from the same customer group or user from being routed to different policy branches, reducing the risk of asset loss.

[0049] In some possible embodiments, after the deployment blocking mechanism for the target single-machine replica is triggered as described above, and before the blocking state of all single-machine replicas under the deployment blocking mechanism is lifted as described above, the data synchronization method may also include, but is not limited to, using the waiting time reserved by the deployment blocking mechanism to perform integrity verification on the data to be deployed; if the verification fails, the deployment blocking mechanism is terminated and the target single-machine replica is prevented from performing subsequent data deployment operations.

[0050] The waiting time reserved in the aforementioned deployment blocking mechanism refers to the time interval from the triggering of the deployment blocking mechanism to the end of the synchronization time window. In a distributed cluster system, the synchronization time window is a pre-set fixed duration used to unify the data deployment rhythm of each single-machine replica. When the target single-machine replica is triggered by the deployment blocking mechanism, it will not immediately execute data deployment, but will enter a waiting state. The duration of this waiting state is the "reserved waiting time". The aforementioned integrity verification refers to using a preset verification algorithm (such as, but not limited to, CRC check, MD5 hash check, etc.) to check the integrity of the data to be deployed, to ensure that the data has not been lost, damaged, or tampered with during transmission or storage. The implementation process of the aforementioned integrity verification includes, for example, but not limited to: calculating the checksum of the data to be deployed and comparing it with the metadata, or verifying whether the data version number strictly increments, etc.

[0051] When the system triggers the deployment blocking mechanism for the target single-machine replica, the target single-machine replica begins waiting for the end of the synchronization time window. During this waiting period, the system can automatically initiate a data integrity verification process. For example, but not limited to, first parsing the data to be deployed, extracting key feature information, and then using a selected verification algorithm to calculate a verification value. Next, the calculated verification value is compared with the original verification value provided by the data sender. If the two are completely consistent, the data integrity verification is deemed to have passed; if they are inconsistent, the verification is deemed to have failed. Once the verification fails, the system immediately terminates the deployment blocking mechanism for the current target single-machine replica, preventing it from waiting for the end of the synchronization time window, and explicitly preventing the target single-machine replica from performing subsequent data deployment operations, i.e., incomplete or erroneous data will not be synchronized to its corresponding virtual machine.

[0052] In the embodiments of this specification, firstly, by performing data integrity verification during the deployment blocking wait period, data anomalies can be detected in a timely manner before the data is officially deployed. If the data to be deployed is damaged or incomplete, terminating the deployment process in advance can prevent the erroneous data from being synchronized to the virtual machine, thereby effectively preventing business logic anomalies, calculation result errors, and other problems caused by erroneous data, ensuring the accuracy of data deployment from the source. Secondly, preventing the target single-machine replica that fails the verification from executing data deployment avoids the spread of erroneous data within the cluster, reduces the risk of data inconsistency in the distributed system, and maintains the reliability of cluster data. Thirdly, compared with the method of verifying and rolling back after the data deployment is completed, terminating the erroneous deployment in advance during the verification stage can save system resources, reduce unnecessary computing and storage overhead, and improve system operating efficiency. Fourthly, for scenarios with high business continuity requirements, such as financial transactions and real-time decision-making, this mechanism can minimize business interruptions caused by data problems and ensure the stable operation and service quality of the system.

[0053] Please refer to the following. Figure 3 This is a flowchart illustrating another data synchronization method provided in an exemplary embodiment of this specification. Figure 3 As shown, this data synchronization method includes the following steps: S302, in response to the timed scheduling command, controls each single-machine replica to start the data loading and parsing task at the beginning of the synchronization time window; the duration of the synchronization time window is less than the scheduling cycle duration of the timed scheduling command.

[0054] Specifically, S302 is the same as S202, and will not be repeated here.

[0055] S304 During the execution of the data loading and parsing task, the task completion status of each single-machine replica is monitored. When it is detected that the target single-machine replica has completed the data loading and parsing task and generated data to be deployed, the deployment blocking mechanism for the target single-machine replica is triggered to block the data deployment operation of the target single-machine replica until the end of the synchronization time window.

[0056] Specifically, S304 is the same as S204, and will not be repeated here.

[0057] S306, when the end time of the synchronization time window is reached, detect whether there is a non-target single-machine replica among the multiple single-machine replicas that has not yet completed the data loading and parsing task.

[0058] Specifically, the aforementioned non-target single-machine replicas refer to those that have not completed data loading and parsing tasks and have not generated data to be deployed before the end of the synchronization time window. When the end of the synchronization time window arrives, the system will conduct a centralized scan and detection of the task completion status of all single-machine replicas participating in this synchronization. This detection process can be implemented, but is not limited to, by traversing the task status flags of each single-machine replica. These flags are updated in real time by the data loading and parsing task module during task execution. For example, but not limited to, when the task has not started, it is marked as "pending startup," when the task is executing, it is marked as "in progress," and after the task is successfully completed and the data to be deployed is generated, it is marked as "completed." The system will check the current flag of each single-machine replica one by one. If it is marked as "pending startup" or "in progress," then the single-machine replica is determined to be a "non-target single-machine replica." In this way, the system can accurately identify single-machine replicas that have failed to complete data preparation within the specified synchronization time window, providing a basis for subsequent processing strategies.

[0059] S308, if so, then it is determined that the non-target single-machine replica has failed to synchronize data this time, and an alarm signal is triggered to interrupt the data deployment operation of the non-target single-machine replica.

[0060] Specifically, the aforementioned data synchronization failure refers to the non-target single-machine replica's inability to complete the entire data synchronization process from data loading and parsing to final deployment as planned within the current synchronization period, resulting in its data status failing to maintain consistency with the source data or the primary replica. The aforementioned alarm signal is an internal or external notification mechanism, which may take the form of, but is not limited to, system log recordings, management interface prompts, email notifications, SMS alerts, or audible and visual alarms, to promptly inform system administrators or relevant processing modules of any abnormal situations. The aforementioned interruption of data deployment operations for the non-target single-machine replica means that once a data synchronization failure is determined and an alarm signal is triggered, the system will immediately block or suspend subsequent data deployment processes for that non-target single-machine replica, preventing incomplete, erroneous, or outdated data from being deployed to that replica, thereby avoiding potential risks to the business system. The specific content of the aforementioned alarm signal may include, but is not limited to, detailed information such as the identifier of the failed replica, the synchronization period, and the reason for the failure (e.g., "the task is still in progress when the synchronization time window ends" or "the task has not started"), ensuring that relevant personnel can quickly locate the problem.

[0061] In the embodiments of this specification, firstly, by interrupting the data deployment operation of non-target single-machine replicas, it is possible to effectively prevent incomplete, erroneous, or outdated data that may be generated due to incomplete data loading and parsing from being deployed to the business system, thereby ensuring that all single-machine replicas with successfully deployed data can provide accurate and consistent services and maintain the overall data quality of the system; secondly, it avoids the risks of application exceptions, business logic errors, or even system crashes that may be caused by deploying problematic data to replicas, reducing system downtime due to data problems and enhancing the stability and risk resistance of the entire distributed system; thirdly, by triggering alarm signals, system administrators or maintenance personnel can be informed of problems that occur during data synchronization in the first instance, facilitating their rapid intervention, analysis of the cause of failure (such as network failure, insufficient resources, etc.), and the implementation of corresponding remedial measures, such as restarting tasks, adjusting resource allocation, or optimizing synchronization strategies, thereby shortening fault recovery time and ensuring business continuity.

[0062] Please continue to refer to the following. Figure 3 ,like Figure 3 As shown in S306 above, when the end time of the synchronization time window is reached, after detecting whether there are any non-target single-machine replicas among the multiple single-machine replicas that have not yet completed the data loading and parsing task, the data synchronization method may also include, but is not limited to: S310, if not, release the blocking state of all target single-machine replicas under the deployment blocking mechanism, control each target single-machine replica to perform data deployment operations at the same time, and synchronize their respective data to be deployed to their respective virtual machines.

[0063] Specifically, the implementation process of S310 is similar to that of S206, and will not be repeated here.

[0064] In some possible embodiments, the start time of the aforementioned synchronization time window is dynamically adjustable. For example... Figure 4 As shown, the above data synchronization method may also include, but is not limited to, the following steps: S402, the maximum historical time taken for each single-machine replica to complete the data loading and parsing task within the preset historical scheduling period.

[0065] Specifically, by iterating through all data synchronization records within a preset historical scheduling period, the time consumed by each single-machine replica in completing data loading and parsing tasks in each scheduling period can be extracted. For example, if the preset scheduling period is 5, meaning data from the past 5 scheduling periods is collected, then the time consumed by each single-machine replica for each task within these 5 scheduling periods needs to be collected. The maximum historical time is selected from all the time consumed by the single-machine replica within these 5 scheduling periods, taking the largest value as its maximum historical time. In this way, the maximum historical time consumed by each single-machine replica in completing data loading and parsing tasks within the preset historical scheduling period can be accurately calculated, providing data support for dynamically adjusting the start time of the synchronization time window.

[0066] S404 calculates the target window duration for this scheduling cycle based on the maximum historical latency and the preset safety margin.

[0067] Specifically, the target window duration can be calculated, but is not limited to, by adding the maximum historical latency of each individual replica to a preset safety margin. The preset safety margin can be, but is not limited to, a fixed or dynamically adjusted value set based on factors such as the system's reliability requirements for data synchronization tasks, network fluctuations, and potential sudden task delays. For example, if the maximum historical latency of a certain individual replica is 120 seconds and the preset safety margin is set to 30 seconds, then the initial target window duration for that individual replica is calculated to be 150 seconds. In practical applications, it is necessary to iterate through all individual replicas participating in data synchronization, calculate the sum of their respective maximum historical latency and preset safety margins, and then select the maximum value as the final target window duration for this scheduling cycle. This ensures that even in extreme cases, all individual replicas can complete the data loading and parsing tasks within the target window duration, thereby guaranteeing the stability and integrity of the entire data synchronization process and avoiding synchronization task failures or data inconsistencies caused by excessive latency of individual replicas.

[0068] S406: Based on the target window duration and scheduling cycle duration, the start time of the corresponding synchronization time window for this scheduling cycle is calculated in reverse to dynamically adjust the trigger time of the timed scheduling instruction.

[0069] Specifically, the start time of the synchronization time window can be determined by subtracting the target window length from the scheduling cycle length. For example, if the scheduling cycle length is set to 300 seconds, and the calculated target window length is 150 seconds, then the start time of the synchronization time window is the 150th second after the start of the scheduling cycle. In this way, the scheduled command will be triggered at this moment, ensuring that the data synchronization task can be completed within the remaining 150-second target window length. Through this reverse calculation method, the start time of the synchronization task can be flexibly adjusted according to the dynamically changing target window length of each scheduling cycle, avoiding the problems of insufficient task execution time or resource waste that may result from a fixed trigger time. This further optimizes the time management of data synchronization, making the entire scheduling process more adaptable and efficient.

[0070] Please refer to the following. Figure 5 , Figure 5 This is a schematic diagram of a data synchronization device provided for an exemplary embodiment of this specification. Figure 5 As shown, the data synchronization device 500 includes: The timed scheduling module 510 is used to respond to the timed scheduling command and control each of the above-mentioned single-machine replicas to start the data loading and parsing task at the beginning of the synchronization time window; the duration of the above-mentioned synchronization time window is less than the scheduling cycle duration of the above-mentioned timed scheduling command. The deployment blocking module 520 is used to monitor the task completion status of each of the above-mentioned single-machine replicas during the execution of the above-mentioned data loading and parsing tasks. When it is detected that the target single-machine replica has completed the above-mentioned data loading and parsing tasks and generated data to be deployed, the deployment blocking mechanism for the target single-machine replica is triggered to block the data deployment operation of the target single-machine replica until the termination time of the above-mentioned synchronization time window. The data deployment module 530 is used to release the blocking state of all single-machine replicas under the deployment blocking mechanism when the termination time of the above synchronization time window is reached, and control the above multiple single-machine replicas to perform data deployment operations at the same time, and synchronize their respective data to be deployed to their respective virtual machines.

[0071] In one possible implementation, the aforementioned timed scheduling instruction is configured to define a synchronization time window at the end of each scheduling cycle, wherein the aforementioned start time corresponds to the opening time of the aforementioned synchronization time window, and the aforementioned end time corresponds to the end time of the aforementioned scheduling cycle or the start time of the next scheduling cycle.

[0072] In one possible implementation, the duration of the synchronization time window is determined based on the historical processing time of the historical data loading and parsing tasks corresponding to each of the multiple single-machine replicas; or, the duration of the synchronization time window is determined based on the estimated processing time of the current data loading and parsing tasks corresponding to each of the multiple single-machine replicas; or, the duration of the synchronization time window is greater than or equal to the maximum historical processing time of the historical data loading and parsing tasks corresponding to each of the multiple single-machine replicas or the maximum estimated processing time of the current data loading and parsing tasks corresponding to each of the multiple single-machine replicas.

[0073] In one possible implementation, the aforementioned deployment blocking module 520 is specifically used for: Start a blocking timer, the duration of which is equal to the time difference from the current moment to the end of the synchronization time window; set the thread state of the target single-machine replica to a waiting state until the blocking timer times out.

[0074] In one possible implementation, the data synchronization device 500 further includes: The detection module is used to detect, when the end time of the above synchronization time window is reached, whether there are any non-target single-machine replicas among the above multiple single-machine replicas that have not yet completed the data loading and parsing tasks; The deployment interrupt module is used to determine if the above-mentioned non-target single-machine replica has failed to synchronize data this time, and to trigger an alarm signal to interrupt the data deployment operation of the above-mentioned non-target single-machine replica.

[0075] In one possible implementation, the data synchronization device 500 further includes: The verification module is used to perform integrity verification on the data to be deployed by taking advantage of the waiting time reserved by the deployment blocking mechanism. The blocking and termination module is used to terminate the above deployment blocking mechanism and prevent the target single-machine replica from performing subsequent data deployment operations if the verification fails.

[0076] In one possible implementation, the start time of the aforementioned synchronization time window is dynamically adjustable; the aforementioned data synchronization device 500 further includes: The time consumption statistics module is used to calculate the maximum historical time consumed by each single machine replica to complete the data loading and parsing task within the historical preset scheduling period. The calculation module is used to calculate the target window duration for the current scheduling cycle based on the maximum historical latency and the preset safety margin. The dynamic adjustment module is used to reverse-calculate the start time of the synchronization time window corresponding to the current scheduling cycle based on the target window duration and scheduling cycle duration, so as to dynamically adjust the trigger time of the aforementioned timed scheduling instructions.

[0077] The division of modules in the above-described data synchronization device is for illustrative purposes only. In other embodiments, the data synchronization device can be divided into different modules as needed to complete all or part of the functions of the above-described data synchronization device. The implementation of each module in the data synchronization device provided in the embodiments of this specification can be in the form of a computer program. This computer program can run on a terminal or server. The program modules constituted by this computer program can be stored in the memory of the terminal or server. When the computer program is executed by a processor, it implements all or part of the steps of the data synchronization method described in the embodiments of this specification.

[0078] Please refer to the following. Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this specification. For example... Figure 6 As shown, the electronic device 600 may include: at least one processor 610, at least one communication bus 620, a user interface 630, at least one network interface 640, and a memory 650. The communication bus 620 can be used to enable communication between the aforementioned components.

[0079] The user interface 630 may include a display screen and a camera. Optionally, the user interface 630 may also include a standard wired interface and a wireless interface.

[0080] The network interface 640 may optionally include a Bluetooth module, a Near Field Communication (NFC) module, a Wireless Fidelity (Wi-Fi) module, etc.

[0081] The processor 610 may include one or more processing cores. The processor 610 connects to various parts within the electronic device 600 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 650, and by calling data stored in the memory 650. Optionally, the processor 610 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 610 may integrate one or a combination of several of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor 610.

[0082] The memory 650 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 650 may include a non-transitory computer-readable medium. The memory 650 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 650 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as deployment blocking functions, data synchronization functions, etc.), instructions for implementing the various method embodiments described above, etc.; the data storage area may store data involved in the various method embodiments described above, etc. Optionally, the memory 650 may also be at least one storage device located remotely from the aforementioned processor 610. Figure 6 As shown, the memory 650, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and program instructions.

[0083] Specifically, the electronic device 600 can be applied to a distributed cluster system comprising multiple single-machine replicas, each of which runs an independent virtual machine to perform data processing. The processor 610 can be used to call program instructions stored in the memory 650 and specifically perform the following operations: In response to the timed scheduling command, each of the aforementioned single-machine replicas is controlled to start the data loading and parsing task at the beginning of the synchronization time window; the duration of the aforementioned synchronization time window is less than the scheduling cycle duration of the aforementioned timed scheduling command; during the execution of the aforementioned data loading and parsing task, the task completion status of each of the aforementioned single-machine replicas is monitored; when it is detected that the target single-machine replica has completed the aforementioned data loading and parsing task and generated data to be deployed, a deployment blocking mechanism for the aforementioned target single-machine replica is triggered to block the data deployment operation of the aforementioned target single-machine replica until the end of the aforementioned synchronization time window; when the end of the aforementioned synchronization time window is reached, the blocking state of all single-machine replicas under the deployment blocking mechanism is released, and the aforementioned multiple single-machine replicas are controlled to perform data deployment operations at the same time, synchronizing their respective data to be deployed to their respective virtual machines.

[0084] In some possible embodiments, the aforementioned timing scheduling instruction is configured to define a synchronization time window at the end of each scheduling cycle, wherein the aforementioned start time corresponds to the opening time of the aforementioned synchronization time window, and the aforementioned end time corresponds to the end time of the aforementioned scheduling cycle or the start time of the next scheduling cycle.

[0085] In some possible embodiments, the duration of the synchronization time window is determined based on the historical processing time of the historical data loading and parsing tasks corresponding to each of the plurality of single-machine replicas; or, the duration of the synchronization time window is determined based on the estimated processing time of the current data loading and parsing tasks corresponding to each of the plurality of single-machine replicas; or, the duration of the synchronization time window is greater than or equal to the maximum historical processing time of the historical data loading and parsing tasks corresponding to each of the plurality of single-machine replicas or the maximum estimated processing time of the current data loading and parsing tasks corresponding to each of the plurality of single-machine replicas.

[0086] In some possible embodiments, when the processor 610 executes the deployment blocking mechanism that triggers the target single-machine replica, it is specifically used to perform the following: Start a blocking timer, the duration of which is equal to the time difference from the current moment to the end of the synchronization time window; set the thread state of the target single-machine replica to a waiting state until the blocking timer times out.

[0087] In some possible embodiments, after the processor 610 performs the task completion status monitoring of each of the aforementioned single-machine replicas during the execution of the aforementioned data loading and parsing task, it is further configured to perform the following: when the termination time of the aforementioned synchronization time window is reached, detect whether there is a non-target single-machine replica among the aforementioned multiple single-machine replicas that has not yet completed the data loading and parsing task; if so, determine that the aforementioned non-target single-machine replica has failed in this data synchronization, and trigger an alarm signal to interrupt the data deployment operation of the aforementioned non-target single-machine replica.

[0088] In some possible embodiments, after the processor 610 executes the above-mentioned triggering of the deployment blocking mechanism for the target single-machine replica, and before executing the above-mentioned unblocking of all single-machine replicas under the deployment blocking mechanism, it is further configured to execute: The integrity of the data to be deployed is verified using the waiting time reserved by the deployment blocking mechanism. If the verification fails, the deployment blocking mechanism is terminated and subsequent data deployment operations are prevented from being performed on the target single-machine replica.

[0089] In some possible embodiments, the start time of the aforementioned synchronization time window is dynamically adjustable; the processor 610 is also configured to perform: Within the historical preset scheduling period, the maximum historical time taken for each single-machine replica to complete the data loading and parsing task is counted; based on the above maximum historical time taken and the preset safety margin, the target window duration of this scheduling period is calculated; based on the above target window duration and scheduling period duration, the start time of the corresponding synchronization time window of this scheduling period is calculated in reverse, so as to dynamically adjust the trigger time of the above timed scheduling instructions.

[0090] This specification also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform one or more steps in the above embodiments. If the constituent modules of the above-described data synchronization device are implemented as software functional units and sold or used as independent products, they can be stored in the above-described computer-readable storage medium.

[0091] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this specification are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired means, such as coaxial cable, fiber optic cable, Digital Subscriber Line (DSL), or wireless means (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs, DVDs), or semiconductor media (e.g., Solid State Disks, SSDs).

[0092] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks. Unless otherwise specified, the technical features of this embodiment and its implementation can be combined arbitrarily.

[0093] The embodiments described above are merely preferred embodiments of this specification and are not intended to limit the scope of this specification. Any modifications and improvements made to the technical solutions of this specification by those skilled in the art without departing from the spirit of this specification should fall within the protection scope defined by the claims.

[0094] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims and specification may be performed in a different order than in the embodiments described in the specification and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A data synchronization method applied to a distributed cluster system comprising multiple single-machine replicas, each of the single-machine replicas running an independent virtual machine to perform data processing, the method comprising: In response to the timed scheduling command, each of the aforementioned single-machine replicas is controlled to start the data loading and parsing task at the beginning of the synchronization time window; The duration of the synchronization time window is less than the scheduling cycle duration of the timed scheduling instruction; During the execution of the data loading and parsing task, the task completion status of each of the single-machine replicas is monitored. When it is detected that the target single-machine replica has completed the data loading and parsing task and generated data to be deployed, a deployment blocking mechanism for the target single-machine replica is triggered to block the data deployment operation of the target single-machine replica until the end time of the synchronization time window. When the end time of the synchronization time window is reached, the blocking state of all single-machine replicas under the deployment blocking mechanism is released, and the multiple single-machine replicas are controlled to perform data deployment operations at the same time, synchronizing their respective data to be deployed to their respective virtual machines.

2. The method as described in claim 1, wherein the timing scheduling instruction is configured to define a synchronization time window at the end of each scheduling cycle, the start time corresponding to the opening time of the synchronization time window, and the end time corresponding to the end time of the scheduling cycle or the start time of the next scheduling cycle.

3. The method as described in claim 1, wherein the duration of the synchronization time window is determined based on the historical processing time of the historical data loading and parsing tasks corresponding to each of the plurality of single-machine replicas; or, the duration of the synchronization time window is determined based on the estimated processing time of the current data loading and parsing tasks corresponding to each of the plurality of single-machine replicas; or, the duration of the synchronization time window is greater than or equal to the maximum historical processing time or the maximum estimated processing time of the current data loading and parsing tasks corresponding to each of the plurality of single-machine replicas.

4. The method of claim 1, wherein triggering the deployment blocking mechanism for the target single-machine replica includes: Start a blocking timer, the duration of which is equal to the time difference from the current moment to the end moment of the synchronization time window; Set the thread state of the target single-machine replica to a waiting state until the blocking timer times out.

5. The method according to any one of claims 1-4, wherein after monitoring the task completion status of each of the single-machine replicas during the execution of the data loading and parsing task, the method further comprises: When the end time of the synchronization time window is reached, it is detected whether there is a non-target single-machine replica among the multiple single-machine replicas that has not yet completed the data loading and parsing task; If so, it is determined that the non-target single-machine replica has failed to synchronize data this time, and an alarm signal is triggered to interrupt the data deployment operation of the non-target single-machine replica.

6. The method as described in any one of claims 1-4, wherein after triggering the deployment blocking mechanism for the target single-machine replica and before releasing the blocking state of all single-machine replicas under the deployment blocking mechanism, the method further comprises: The integrity of the data to be deployed is verified by utilizing the waiting time reserved by the deployment blocking mechanism. If the verification fails, the deployment blocking mechanism will be terminated, and the target single-machine replica will be prevented from performing subsequent data deployment operations.

7. The method according to any one of claims 1-4, wherein the start time of the synchronization time window is dynamically adjustable; the method further comprises: Within the historical preset scheduling period, the maximum historical time taken for each single-machine replica to complete the data loading and parsing tasks; The target window duration for this scheduling cycle is calculated based on the maximum historical time consumption and the preset safety margin. Based on the target window duration and scheduling cycle duration, the start time of the corresponding synchronization time window for this scheduling cycle is calculated in reverse to dynamically adjust the trigger time of the timed scheduling instruction.

8. A data synchronization device, applied to a distributed cluster system comprising multiple single-machine replicas, each of the single-machine replicas running an independent virtual machine to perform data processing, the device comprising: The timed scheduling module is used to respond to timed scheduling instructions and control each of the single-machine replicas to start the data loading and parsing tasks at the beginning of the synchronization time window; The duration of the synchronization time window is less than the scheduling cycle duration of the timed scheduling instruction; The deployment blocking module is used to monitor the task completion status of each of the single-machine replicas during the execution of the data loading and parsing task. When it is detected that the target single-machine replica has completed the data loading and parsing task and generated data to be deployed, the deployment blocking mechanism for the target single-machine replica is triggered to block the data deployment operation of the target single-machine replica until the end time of the synchronization time window. The data deployment module is used to release the blocking state of all single-machine replicas under the deployment blocking mechanism when the end time of the synchronization time window is reached, and control the multiple single-machine replicas to perform data deployment operations at the same time, synchronizing their respective data to be deployed to their respective virtual machines.

9. An electronic device, comprising: Processor and memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code stored in the memory to perform the method as described in any one of claims 1-7.

10. A computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the steps of the method as claimed in any one of claims 1-7.

11. A computer program product comprising instructions that, when run on a computer or processor, cause the computer or processor to perform the data synchronization method as described in any one of claims 1-7.

Images