Data synchronization method and apparatus, computer program product, and electronic device

Abstract

A data synchronization method and device, a computer program product and an electronic device, relate to the technical field of computers. The method comprises: sending a connection request to a first data source node to mark the client node as a to-be-authorized node by the first data source node, and authenticating the client node by an authorization management service and obtaining a distribution task configuration; wherein the distribution task configuration at least includes target branch information corresponding to the client node; a second client node receives target to-be-synchronized data corresponding to the target branch information from the first data source node or a first client node that has completed data synchronization, and takes the second client node as a distribution node to provide the target to-be-synchronized data to subsequent client nodes. The load of the data source node is reduced, and the data synchronization efficiency and client download speed are improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a data synchronization method and apparatus, computer program products, and electronic devices. Background Technology

[0002] In distributed software development environments, multiple terminal nodes typically need to frequently synchronize build artifacts, configuration files, and code resources to maintain data consistency across nodes. Related technologies mainly include centralized synchronization schemes based on a central server, shared schemes based on cloud storage, and decentralized synchronization schemes based on peer-to-peer protocols. Centralized synchronization schemes based on central servers employ a star topology, where each client node obtains data through the central server. When a large number of client nodes concurrently request data updates, the central server's network egress bandwidth, disk read / write performance, and computing resources can easily become system bottlenecks, leading to decreased overall distribution efficiency and delays in data acquisition for terminal nodes. Shared schemes based on cloud storage rely on centralized storage nodes for file hosting and version management. In scenarios with frequent file updates, maintaining historical versions often generates a large amount of redundant snapshot data, significantly increasing the storage system's space overhead; simultaneously, limited by the throughput capacity of the central node, transmission link congestion and synchronization lag issues still exist. While decentralized synchronization schemes based on peer-to-peer protocols can alleviate the transmission pressure on central nodes to some extent, in the initial stages of a new version's data release, typically only the initial data source node holds the complete data. This causes a surge of concurrent download requests from client nodes to that data source node, resulting in an unbalanced network load in the initial phase. Furthermore, some peer-to-peer synchronization schemes employ a bidirectional read-write model, where each node can write data to the other, lacking a mechanism to restrict the data source. This can easily lead to erroneous data transmissions, version conflicts, and data corruption, resulting in additional overhead for data consistency verification and repair. Summary of the Invention

[0003] This disclosure provides a data synchronization method to at least partially solve the problems of high distribution pressure on source nodes and slow download speeds on clients in related technologies.

[0004] According to a first aspect of this disclosure, a data synchronization method is provided, applied to a client node, the method comprising: A connection request is sent to the first data source node, so that the first data source node marks the client node as a node to be authorized, and the authorization management service authenticates the client node and obtains the distribution task configuration; wherein, the distribution task configuration includes at least the target branch information corresponding to the client node; The second client node receives the target data to be synchronized corresponding to the target branch information from the first data source node or the first client node that has completed data synchronization, and uses the second client node as a distribution node to provide the target data to be synchronized to subsequent client nodes.

[0005] According to a second aspect of this disclosure, a data synchronization method is provided, applied to a data source node, the method comprising: Generate data to be synchronized; The data to be synchronized is synchronized to the client nodes, so that the first client node that receives the data to be synchronized acts as a distribution node and synchronizes the data to be synchronized to the second client node among the client nodes.

[0006] According to a third aspect of this disclosure, a data synchronization apparatus is provided, applied to a client node, the apparatus comprising: A connection request sending module is used to send a connection request to a first data source node, so that the first data source node marks the client node as a node to be authorized, and the authorization management service authenticates the client node and obtains the distribution task configuration; wherein, the distribution task configuration includes at least the target branch information corresponding to the client node; The data receiving module is used for the second client node to receive target data to be synchronized corresponding to the target branch information from the first data source node or the first client node that has completed data synchronization, and to use the second client node as a distribution node to provide the target data to be synchronized to subsequent client nodes.

[0007] According to a fourth aspect of this disclosure, a data synchronization apparatus is provided, applied to a data source node, the apparatus comprising: The data generation module is used to generate the data to be synchronized. The data synchronization module is used to synchronize the data to be synchronized to the client nodes, so that the first client node that receives the data to be synchronized acts as a distribution node and synchronizes the data to be synchronized to the second client node among the client nodes.

[0008] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method of the first aspect described above and possible implementations thereof.

[0009] According to a sixth aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the method of the first aspect and possible implementations thereof by executing the executable instructions.

[0010] This disclosure provides a data synchronization method in which a connection request is sent to a first data source node, causing the first data source node to mark the client node as an authorized node. An authorization management service then authenticates the client node and obtains a distribution task configuration. The distribution task configuration includes at least target branch information corresponding to the client node. A second client node receives target data to be synchronized corresponding to the target branch information from the first data source node or a first client node that has completed data synchronization. The second client node is then designated as a distribution node to provide the target data to be synchronized to subsequent client nodes. On one hand, after receiving the target data to be synchronized, the client node is configured as a distribution node to provide the target data to be synchronized to subsequent client nodes. By having synchronized client nodes participate in data propagation, the target data to be synchronized can be synchronized among multiple client nodes, thereby reducing the concentrated load caused by multiple client nodes simultaneously requesting data from the data source node, reducing the bandwidth and computational pressure on the data source node, and improving overall file synchronization efficiency and client download speed. On the other hand, by having the first data source node uniformly generate and distribute the target data to be synchronized, the target data to be synchronized has a fixed and reliable source. Meanwhile, the authorization management service authenticates client nodes and determines the target branch information of client nodes based on the distribution task configuration, thereby reducing the risk of unauthorized nodes accessing the synchronization network and improving the security and reliability of the data synchronization process. Attached Figure Description

[0011] Figure 1 A flowchart illustrating a data synchronization method in this exemplary embodiment is shown; Figure 2 This exemplary embodiment illustrates a system architecture for data synchronization; Figure 3 This exemplary embodiment illustrates a method for authenticating client nodes using an authorization management service. Figure 4 This diagram illustrates a flowchart of deploying a target synchronization service on a client node in this exemplary embodiment. Figure 5 This illustration shows a flowchart of a data synchronization process between a first data source node and a client node in this exemplary embodiment. Figure 6 This illustrates a flowchart of synchronizing data to the running directory in this exemplary embodiment; Figure 7 A flowchart illustrating a data synchronization method in this exemplary embodiment is shown; Figure 8 A timing diagram of a data synchronization method in this example embodiment is shown; Figure 9 This diagram illustrates a data synchronization apparatus according to an exemplary embodiment of the present invention. Figure 10 A block diagram of a data synchronization device in this exemplary embodiment is shown. Figure 11 A schematic diagram of the structure of an electronic device in this exemplary embodiment is shown. Detailed Implementation

[0012] Exemplary embodiments of this disclosure will be described more fully below with reference to the accompanying drawings.

[0013] The accompanying drawings are schematic illustrations of this disclosure and are not necessarily drawn to scale. Some block diagrams shown in the drawings may be functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in hardware modules or integrated circuits, or in networks, processors, or microcontrollers. Implementations can be carried out in various forms and should not be construed as limited to the examples set forth herein. The features, structures, or characteristics described in this disclosure can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough description of embodiments of this disclosure. However, those skilled in the art will recognize that one or more specific details may be omitted when implementing the technical solutions of this disclosure, or other methods, components, apparatuses, steps, etc., may be used to replace one or more specific details.

[0014] In software development and project collaboration, intranet file distribution is a crucial step in ensuring efficient team collaboration. As projects grow in scale and iteration speeds accelerate, development teams need to frequently synchronize code, configuration files, build artifacts, and other data across multiple nodes. Traditional file synchronization methods, when faced with large-scale, high-frequency distribution demands, have gradually revealed efficiency bottlenecks and operational complexity, making it difficult to meet the real-time and reliability requirements of modern agile development.

[0015] Among the relevant technologies, there are centralized synchronization solutions based on rsync, file sharing solutions based on cloud disks, and native Syncthing peer-to-peer synchronization solutions.

[0016] A centralized synchronization scheme based on rsync uses the rsync tool to push files from a central server (packaging machine) to each client node. Clients then pull the latest data from the server via scheduled tasks or manual commands. While this scheme offers some incremental synchronization capabilities, the central server's bandwidth, disk read / write speeds, and computing resources can become performance bottlenecks when multiple client nodes update data simultaneously. This leads to slower download speeds for client nodes and reduced overall distribution efficiency. Furthermore, this type of scheme typically requires client nodes to actively perform pull operations, making it difficult to meet the real-time update needs of frequent iteration scenarios.

[0017] File sharing solutions based on cloud disks achieve file sharing and synchronization through centralized file storage services. While they can achieve unified version management, the reliance on centralized storage nodes for file synchronization still presents the problem of high load on the central nodes. Furthermore, in scenarios with frequent updates, numerous version snapshots are easily generated, leading to increased storage overhead. Additionally, cloud disk-based solutions typically suffer from synchronization latency, making it difficult to meet the needs of research and development scenarios for rapid data distribution and near real-time updates.

[0018] The native Syncthing peer-to-peer synchronization solution achieves file propagation through mutual synchronization among multiple nodes. Nodes that have completed synchronization can continue to provide data to other nodes, thereby reducing the distribution pressure on the central node to some extent. However, in the early stages of a new version release, typically only the source node holds complete data, and client download requests will still be concentrated on the source node. This causes the source node to bear a large distribution load in the initial stage, making it difficult to effectively solve the download speed problem in scenarios with multiple clients downloading concurrently. In addition, the native peer-to-peer synchronization solution usually adopts a bidirectional synchronization mode, allowing nodes to write data to each other, which can easily lead to problems such as erroneous file back transmission, version conflicts, and source data pollution. This makes it difficult to meet the needs of development scenarios where build artifacts are uniformly generated from a fixed data source.

[0019] In view of the above problems, exemplary embodiments of this disclosure provide a data synchronization method. (See reference...) Figure 1 As shown, the data synchronization method applied to client nodes may include the following steps: Step S110: Send a connection request to the first data source node so that the first data source node marks the client node as a node to be authorized, and the authorization management service authenticates the client node and obtains the distribution task configuration; wherein, the distribution task configuration includes at least the target branch information corresponding to the client node; Step S120: The second client node receives the target data to be synchronized corresponding to the target branch information from the first data source node or the first client node that has completed data synchronization, and uses the second client node as a distribution node to provide the target data to be synchronized to subsequent client nodes.

[0020] In the aforementioned data synchronization method, on the one hand, after receiving the target data to be synchronized, the client node is configured as a distribution node to provide the target data to subsequent client nodes. By having synchronized client nodes participate in data propagation, the target data can be synchronized across multiple client nodes, thereby reducing the concentrated load caused by multiple client nodes simultaneously requesting data from the data source node, lowering the bandwidth and computational pressure on the data source node, and improving overall file synchronization efficiency and client download speed. On the other hand, by having the first data source node uniformly generate and distribute the target data to be synchronized, the target data has a fixed and reliable source. Simultaneously, the authorization management service authenticates client nodes and determines the target branch information of client nodes based on the distribution task configuration, thereby reducing the risk of unauthorized nodes accessing the synchronization network and improving the security and reliability of the data synchronization process.

[0021] Figure 2 The system architecture of this disclosure is illustrated. Specifically, it includes a data source layer, a client layer, and an authorization management service layer. The data source layer is responsible for content production and storage, including a first data source node and a second data source node. The first data source node is the core production node, used to execute Jenkins build tasks and generate data to be synchronized. The second data source node acts as an auxiliary node during the day, pre-synchronizing all data from the first data source node via Jenkins at night. During working hours, it acts as an auxiliary source to share the P2P synchronization traffic of the first data source node, avoiding single points of failure. The client layer includes at least one type of client node, with at least one of each type. When each client node first comes online, it initiates a connection request to the first data source node. After authorization, each client node joins the P2P synchronization network. Each client node is both a data consumer and a data provider in the P2P synchronization network; that is, it pulls data to be synchronized from the first data source node, the second data source node, and other synchronized client nodes, and simultaneously acts as a P2P distribution node to synchronize synchronized data to other unsynchronized client nodes. The authorization management service layer includes an authorization management service, which monitors connection requests from the first data source node, performs automated authorization for newly connected client nodes, pushes client node information to the synchronization service instance of the first data source node after successful authorization, completes the binding of the client node to the shared folder, and logs and notifies the client node if authorization fails.

[0022] The following will combine Figure 2 Further explanation and description of steps S110 and S120 are provided.

[0023] In step S110, a connection request is sent to the first data source node so that the first data source node marks the client node as a node to be authorized, and the authorization management service authenticates the client node and obtains the distribution task configuration; wherein, the distribution task configuration includes at least the target branch information corresponding to the client node.

[0024] The first data source node can be the master packer in the data synchronization system. The master packer receives connection requests from client nodes and marks the client node that sent the connection request as a node to be authorized. After the first data source node marks the client node as a node to be authorized, the authorization management service authenticates the client node and determines the task distribution configuration of the client node.

[0025] The following will be explained through the appendix Figure 3 This disclosure provides a further explanation and description of the authentication process for the authorization management service. Specifically, after the authorization management service starts, it continuously polls the pendingDevices interface of the target synchronization service on the first data source node. This pendingDevices interface is one of the REST API interfaces exposed by the target synchronization service, used to query the list of new devices awaiting authorization. The service determines whether a new client node has a connection request on the pendingDevices interface. If no connection request exists, it continuously listens for the pendingDevices interface. When a connection request exists, the authorization management service obtains the device information of the client node sending the connection request and performs authorization verification based on this information. Authorization verification can be performed using a whitelist, i.e., determining whether the device information is in the whitelist. If authorization verification fails, the client node's access is denied and recorded in the denial log. If authorization verification passes, the automatic authorization process begins. This automatic authorization process includes: the authorization management service calling the target synchronization service API of the first data source node, receiving the client node's connection request, adding the client node to the synchronization network, and registering the client node's buffer directory. Once data synchronization is initiated, the target synchronization service of the first data source node begins synchronizing the latest resource packages in the buffer directory of the first data source node to the client node, and pushes an authorization success message to the first client node.

[0026] In one implementation, the client node can send a connection request to the first data source node based on the target synchronization service. The target synchronization service on the client node can pull a target toolkit from an SVN (Subversion) repository and run the tools included in the toolkit. The tools automatically detect whether the target synchronization service is already installed on the client node; if not, they automatically install it. During installation, the version of the target synchronization service included in the toolkit is compared with the running version of the target synchronization service on the client node, and automatically updated if the running version expires. After installing the target synchronization service, a buffer directory can be created on the client node. The buffer directory can include an engine folder, a code folder, and a resource folder. After registering the buffer directory, the target toolkit can also automatically configure the synchronization parameters of the target synchronization service. These synchronization parameters can include transport parameters, discovery service parameters, and intermediary server parameters, which are not specifically limited in this disclosure.

[0027] In one implementation, the target toolkit includes: a synchronization service tool, a service-based installation tool, a registration tool, a configuration management tool, and a directory synchronization tool. Specifically, the target toolkit encapsulates content associated with the target synchronization service, including the synchronization service tool, service-based installation tool, registration tool, configuration management tool, and directory synchronization tool. The synchronization service tool can be a native SYCNThing tool; the service-based installation tool is used to install the synchronization service onto client nodes; the registration tool is used to register the first client node to the synchronization cluster; the configuration management tool is used to manage branch configurations; and the directory synchronization tool is used to synchronize synchronization data from the buffer directory to the local running directory. Furthermore, the target toolkit also includes a hierarchy management tool for handling external directory links.

[0028] In one implementation, installing the target synchronization service includes: The target toolkit is used to detect whether the target synchronization service exists in the client node; When a target synchronization service exists, check whether the running version of the target synchronization service in the client node is the same as the version of the target synchronization service described in the target toolkit. If they are different, update the running version. When the target synchronization service does not exist, the synchronization service tool is installed based on the service-oriented installation tool.

[0029] Specifically, the target toolkit detects the target synchronization service on the client's local machine to determine if the service exists on that client node. If the service exists, it checks if the version of the service running on the client node is the same as the version in the target toolkit. If they are different, the version on the client node is updated. If the service does not exist on the client node, the synchronization service tool is installed using the service-based installation tool.

[0030] In one implementation, before sending the connection request to the first data source node, the method further includes: Register a buffer directory, wherein the buffer directory is used to store the target data to be synchronized.

[0031] Specifically, after installing the target synchronization service on the client node, a buffer directory can be created on the client node. When creating the buffer directory, a branch buffer directory corresponding to each target branch can be created based on the target branch information corresponding to the client node. After establishing the branch buffer directory, each branch buffer directory can be registered. After registration, the buffer directory can store the target data to be synchronized received by the client node.

[0032] After installing the target synchronization service on the client node and creating its buffer directory, the client node sends a connection request to the first data source node in response to the target synchronization service starting. The following describes how... Figure 4 The deployment of the client node disclosed herein will be further explained and described as follows: The client node pulls the target toolkit from SVN. The target toolkit checks if the target synchronization service exists on the client node. If it does, it checks if the client's running version matches the version of the target synchronization service in the target toolkit. If they don't match, the client node's running version is updated. If the target synchronization service doesn't exist on the client node, the Syncthing service is installed on the client node. A branch buffer directory is created on the client node, registered, and its synchronization parameters are automatically configured. The Syncthing service is started, and the client node sends a connection request to the main packager. Deployment is complete, and authorization is awaited.

[0033] After the client node starts the target synchronization service, it sends a connection request to the first data source node so that the first data source node can verify the client node's permissions. If the permission verification is successful, the client node will join the synchronization network.

[0034] In step S120, the second client node receives the target data to be synchronized corresponding to the target branch information from the first data source node or the first client node that has completed data synchronization, and uses the second client node as a distribution node to provide the target data to be synchronized to subsequent client nodes.

[0035] After client node permission verification is successful, the first data source node or the first client node that has completed data synchronization can synchronize the data to be synchronized in its buffer directory to the second client node. This second client node is the node among the client nodes that has not yet received the data to be synchronized. The second client node receives the target data to be synchronized, which corresponds to its target branch information. After receiving this target data to be synchronized, the second client node can act as a distribution node, continuing to synchronize the target data to the client nodes that have not yet received it.

[0036] In one embodiment, the method further includes: Determine the update method of the client node, and synchronize the target data to be synchronized from the buffer directory to the running directory according to the update method.

[0037] Specifically, after the client node receives the target data to be synchronized and stores it in the buffer directory, the target data to be synchronized can be synchronized from the buffer directory to the running directory according to the update method of the client node.

[0038] In one implementation, when the update method is real-time update, synchronizing the target data to be synchronized from the buffer directory to the running directory includes: Add a target link corresponding to the buffer directory to the running directory, and connect the running directory to the buffer directory based on the target link.

[0039] Specifically, when the local update method on the client node is real-time, replication is not performed. Instead, a target link is added to the running directory, directly linking to the buffer directory of the client node. When the client node is running, it will directly read the data from the buffer directory based on this target link.

[0040] In one implementation, when the update method is a non-real-time update, synchronizing the target data to be synchronized from the buffer directory to the running directory includes: Copy the target data to be synchronized from the buffer directory to the running directory.

[0041] Specifically, when the local update method of the client node is non-real-time update, Robocopy image copying is performed to completely copy the buffer directory of the client node to the running directory.

[0042] The following is for reference. Figure 5 As shown, the data synchronization between the first data source node and the client nodes is further explained and illustrated. Specifically, Jenkins executes automated build tasks, generates data to be synchronized, and stores this data in the master directory of the first data source node. This master directory serves as the buffer directory for the first data source node. The target synchronization service in the first data source node monitors changes to the master directory and distributes the data to be synchronized to all authorized client nodes via a P2P synchronization network. Each client node contains a buffer directory and a runtime directory. Synchronizing the data to be synchronized to a client node involves synchronizing the data to the client node's buffer directory. The target synchronization service only operates on the buffer directory in the client node and does not directly modify the runtime directory, preventing file congestion or program crashes during synchronization. Whenever the first data source node is updated, the buffer directory of the client node will automatically synchronize to the latest version via the P2P synchronization network. The runtime directory is where the client node program actually reads resources during runtime; it is physically isolated from the buffer directory, ensuring a stable operating environment. The target synchronization service runs in the second data source node. At night, it pre-synchronizes resource packages from the buffer directory of the first data source node, and during the day, it acts as an auxiliary source to share P2P traffic, preventing single-point failures of the first data source node.

[0043] In this disclosure, a buffer mechanism is used to synchronize data in a separate background directory for the target synchronization service, while the user program runs in its own directory, with the two physically isolated. This allows the buffer to receive push updates while the program is running, and users can choose whether to update to the latest version. It also avoids the program crashes caused by file locking in traditional solutions, ensuring overall operational stability.

[0044] The following is for reference. Figure 6As shown, the process of synchronizing the target data to be synchronized to the running directory is further explained and illustrated. After successful authorization of the client node, the target synchronization service of the first data source node synchronizes the data to be synchronized to the buffer directory of the client node. This synchronization of the data to be synchronized is continuously performed in the background. After starting the main program of the target toolkit in the client node, the buffer directory is determined, and the synchronization data in the buffer directory is synchronized to the running directory according to the local update method of the client node. When the local update method of the client node is real-time update, a target link is added to the running directory, directly linking to the buffer directory of the client node; when the local update method of the client node is non-real-time update, the buffer directory of the client node is completely mirrored and copied to the running directory.

[0045] In one embodiment, the method further includes: The second client node receives the target data to be synchronized corresponding to the target branch information from the second data source node; wherein, the second data source node is used to synchronize the data to be synchronized from the first data source node within a preset time period.

[0046] Specifically, the second client node can also receive target data to be synchronized corresponding to its target branch information from the second data source node. This second data source node can receive data to be synchronized from the first data source node within a preset time period. This preset time period is not specifically limited in this disclosure. After the second data source node receives the data to be synchronized from the first data source node, the second client node can also obtain the data to be synchronized from the second data source node. The first data source node only synchronizes data to the second data source node or the client node, and does not receive data from the second data source node or the client node. The second data source node only synchronizes data to the client node and only receives the data to be synchronized from the first data source node.

[0047] This disclosure also provides a data synchronization method applied to a data source node, see reference. Figure 7 As shown, the method includes: Step S710: Generate data to be synchronized; Step S720: Synchronize the data to be synchronized to the client nodes, so that the first client node that receives the data to be synchronized acts as a distribution node and synchronizes the data to be synchronized to the second client node among the client nodes.

[0048] The following will further explain and illustrate steps S710 and S720. Specifically, after the data source node generates the data to be synchronized, it synchronizes the data to the client node. When the data to be synchronized corresponds to the target branch information of the first client node, the first client node receives the data to be synchronized. After receiving the data to be synchronized, the first client node can act as a distribution node to synchronize the data to be synchronized to the second client node in the client.

[0049] In one implementation, when the first data source node generates data to be synchronized, it collects all the original resource files to be packaged, sorts the original resource files to ensure that the sorting order remains fixed, and encapsulates the sorted original resource files into an uncompressed resource set in a fixed format to ensure that the structure and byte boundaries of each encapsulation are consistent. The zstd small window compression algorithm is used to compress the uncompressed resource set to generate the final res.npk resource package. During the compression process, block boundary markers are preserved. The res.npk resource package is the resource package included in the synchronized data.

[0050] When incremental resource updates occur, the above steps are repeated. Compared to the old version, the newly generated res.npk differs only in localized areas due to sequential packaging ensuring structural stability and small-window compression controlling the diff range. The first data source node pushes the generated res.npk to the target synchronization service. The target synchronization service uses a rolling hash algorithm (sliding window method) to automatically divide data blocks and compare the block hashes of the new and old versions of res.npk, maximizing the reuse of identical data blocks. Through sequential packaging of resource packages and the zstd block compression strategy, combined with the block hash reuse mechanism of the target synchronization service, data synchronization will transmit only the changed data blocks as much as possible, reducing the incremental iteration transmission of large resource packages and improving synchronization efficiency.

[0051] The following will combine Figure 8 The data synchronization method disclosed herein is further explained and illustrated. Specifically, the planner opens an Excel spreadsheet (such as numerical configuration, level configuration, etc.) locally and modifies its contents. The planner clicks "Export" or executes a script command to convert the modified Excel spreadsheet into a binary format file that the program can recognize and automatically commits it to the code repository. The export tool notifies the first data source node, triggering file monitoring on that node. When the first data source node detects a data change, it does not immediately trigger synchronization but waits for 2 seconds. If no new modifications are made within 2 seconds, data synchronization begins. The first data source node synchronizes the data to be synchronized to the client node cluster via a P2P synchronization network. Each client node receives the data to be synchronized in parallel and writes the received data to be synchronized into a buffer directory. The client nodes then hot-reload the data to be synchronized.

[0052] In this disclosure, by combining a file monitoring mechanism (second-level delay triggering) and P2P parallel transmission, the data after the table is imported can be pushed to the buffer directory of all clients within 5-10 seconds. Compared with the traditional solution that requires manual retrieval or timed packaging of patch tasks, this achieves near real-time distribution, accelerates iteration efficiency, and greatly improves efficiency.

[0053] Exemplary embodiments of this disclosure also provide a data synchronization apparatus, with reference to Figure 9 As shown, it includes: The connection request sending module 910 is used to send a connection request to the first data source node, so that the first data source node marks the client node as a node to be authorized, and the authorization management service authenticates the client node and obtains the distribution task configuration; wherein, the distribution task configuration includes at least the target branch information corresponding to the client node; The data receiving module 920 is used for the second client node to receive target data to be synchronized corresponding to the target branch information from the first data source node or the first client node that has completed data synchronization, and to use the second client node as a distribution node to provide the target data to be synchronized to subsequent client nodes.

[0054] Exemplary embodiments of this disclosure also provide a data synchronization apparatus, with reference to Figure 10 As shown, it includes: Data generation module 1010 is used to generate data to be synchronized; The data synchronization module 1020 is used to synchronize the data to be synchronized to the client nodes, so that the first client node that receives the data to be synchronized acts as a distribution node and synchronizes the data to be synchronized to the second client node among the client nodes.

[0055] In one exemplary embodiment, the connection request sending module includes: A buffer directory registration module is used to register a buffer directory, wherein the buffer directory is used to store the target data to be synchronized.

[0056] In one exemplary embodiment, the data receiving module includes: The target data to be synchronized receiving module is used to receive the target data to be synchronized and store the target data to be synchronized in the buffer directory.

[0057] In one exemplary embodiment, the data receiving module includes: The runtime directory synchronization module is used to determine the update method of the client node and, according to the update method, synchronize the target data to be synchronized from the buffer directory to the runtime directory.

[0058] In one exemplary embodiment, when the local update method is real-time update, the running directory synchronization module includes: The target link creation module is used to add a target link corresponding to the buffer directory in the running directory, and connect the running directory to the buffer directory based on the target link.

[0059] In one exemplary embodiment, when the local update method is a non-real-time update, the running directory synchronization module includes: The data synchronization module is used to copy the target data to be synchronized from the buffer directory to the running directory.

[0060] In one exemplary embodiment, the data receiving module includes: The second data source node synchronization module is used for the second client node to receive the target data to be synchronized corresponding to the target branch information from the second data source node; wherein, the second data source node is used to synchronize the data to be synchronized from the first data source node within a preset time period.

[0061] In one exemplary embodiment, the data generation module includes: The original resource file encapsulation module is used to sort the original resource files and encapsulate the sorted original resource files into a resource set according to a preset format. The data compression module is used to compress the resource set to obtain the data to be synchronized.

[0062] The specific details of each part of the above-mentioned device have been described in detail in the method section of the implementation plan. For any undisclosed details, please refer to the implementation plan of the method section, and therefore will not be repeated here.

[0063] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to exemplary embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0064] Furthermore, although the steps of the method in this invention are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0065] Exemplary embodiments of this disclosure also provide a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the data synchronization method described above.

[0066] In one implementation, the computer program product can be a tangible product containing a computer program, such as a computer-readable storage medium storing the computer program. The readable storage medium can be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, including but not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory, hard disk drive (HDD), solid-state drive (SSD), etc. For example, the computer program product can be implemented as a non-volatile storage medium storing a computer program, such as read-only memory, NAND flash memory, etc.

[0067] In one implementation, the computer program product can be an intangible product containing a computer program. For example, the computer program product can be implemented as a virtual digital product, such as an executable file, installation package, or other digital file storing the computer program.

[0068] Computer program code can be written in one or more programming languages. Examples of programming languages ​​include C, Java, and C++. Program code can execute entirely on the user's computing device, partially on the user's computing device, or as a standalone software package. It can also execute partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, such as a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via an internet connection provided by a mobile network operator).

[0069] Computer programs can be carried or transmitted via signals such as electricity, magnetism, light, electromagnetic radiation, and infrared rays. Electronic devices can convert signals carrying computer programs into digital signals, thereby running the computer programs. When a computer program runs on an electronic device, its code is used to cause the electronic device to execute (more specifically, the processor of the electronic device to execute) the method steps of various exemplary embodiments of this disclosure. For example, the above-described data synchronization method can be executed, which includes the following steps: Step S110: Sending a connection request to a first data source node, so that the first data source node marks the client node as an authorized node, and the authorization management service authenticates the client node and obtains the distribution task configuration; wherein, the distribution task configuration includes at least target branch information corresponding to the client node; Step S120: A second client node receives target data to be synchronized corresponding to the target branch information from the first data source node or the first client node that has completed data synchronization, and uses the second client node as a distribution node to provide the target data to be synchronized to subsequent client nodes.

[0070] Implementing the above method steps through a computer program, on the one hand, after receiving the target data to be synchronized, the client node is configured as a distribution node to provide the target data to subsequent client nodes. By having synchronized client nodes participate in data propagation, the target data can be synchronized among multiple client nodes, thereby reducing the concentrated load caused by multiple client nodes simultaneously requesting data from the data source node, lowering the bandwidth and computational pressure on the data source node, and improving overall file synchronization efficiency and client download speed. On the other hand, by having the first data source node uniformly generate and distribute the target data to be synchronized, the target data to be synchronized has a fixed and reliable source. Simultaneously, the authorization management service authenticates client nodes and determines the target branch information of client nodes based on the distribution task configuration, thereby reducing the risk of unauthorized nodes accessing the synchronization network and improving the security and reliability of the data synchronization process.

[0071] In an exemplary embodiment of the present invention, an electronic device capable of implementing the above-described method is also provided. The electronic device may include a processor and a memory. The memory stores executable instructions of the processor, such as a computer program. The processor executes the executable instructions to perform the method steps of various exemplary embodiments of the present disclosure. Furthermore, the electronic device may also include a display for displaying a graphical user interface.

[0072] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: entirely in hardware, entirely in software (including firmware, microcode, etc.), or in a combination of hardware and software, collectively referred to herein as “circuit,” “module,” or “system.”

[0073] The following reference Figure 11 To describe an electronic device 1100 according to this embodiment of the present invention. Figure 11 The electronic device 1100 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0074] like Figure 11 As shown, the electronic device 1100 is manifested in the form of a general-purpose computing device. The components of the electronic device 1100 may include, but are not limited to: at least one processing unit 1110, at least one storage unit 1120, a bus 1130 connecting different system components (including storage unit 1120 and processing unit 1110), and a display unit 1140.

[0075] The storage unit stores program code that can be executed by the processing unit 910, causing the processing unit 1110 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 1110 can perform actions such as... Figure 1 Step S110: A connection request is sent to the first data source node, so that the first data source node marks the client node as a node to be authorized, and the authorization management service authenticates the client node and obtains the distribution task configuration; wherein, the distribution task configuration includes at least the target branch information corresponding to the client node; Step S120: The second client node receives the target data to be synchronized corresponding to the target branch information from the first data source node or the first client node that has completed data synchronization, and uses the second client node as a distribution node to provide the target data to be synchronized to subsequent client nodes.

[0076] Storage unit 1120 may include readable media in the form of volatile storage units, such as random access memory (RAM) 11201 and / or cache memory 11202, and may further include read-only memory (ROM) 11203.

[0077] Storage unit 1120 may also include a program / utility 11204 having a set (at least one) program module 11205, such program module 11205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0078] Bus 1130 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0079] Electronic device 1100 can also communicate with one or more external devices 1200 (e.g., keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with electronic device 1100, and / or any device that enables electronic device 1100 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 1150. Furthermore, electronic device 1100 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1160. As shown, network adapter 1160 communicates with other modules of electronic device 1100 via bus 1130. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1100, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID (Redundant Arrays of Independent Disks) systems, tape drives, and data backup storage systems.

[0080] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, portable hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the method according to the embodiments of the present invention.

[0081] In exemplary embodiments of the present invention, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of the present invention may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of the present invention described in the "Exemplary Methods" section above.

[0082] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0083] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention described herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not invented herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the claims.

Claims

1. A data synchronization method, characterized in that, Applied to client nodes, the method includes: A connection request is sent to the first data source node, so that the first data source node marks the client node as a node to be authorized, and the authorization management service authenticates the client node and obtains the distribution task configuration; wherein, the distribution task configuration includes at least the target branch information corresponding to the client node; The second client node receives the target data to be synchronized corresponding to the target branch information from the first data source node or the first client node that has completed data synchronization, and uses the second client node as a distribution node to provide the target data to be synchronized to subsequent client nodes.

2. The method according to claim 1, characterized in that, Before sending the connection request to the first data source node, the method further includes: Register a buffer directory, wherein the buffer directory is used to store the target data to be synchronized.

3. The method according to claim 2, characterized in that, The receiving of the target data to be synchronized corresponding to the target branch information includes: Receive the target data to be synchronized and store the target data to be synchronized in the buffer directory.

4. The method according to claim 3, characterized in that, The method further includes: Determine the update method of the client node, and synchronize the target data to be synchronized from the buffer directory to the running directory according to the update method.

5. The method according to claim 4, characterized in that, When the update method is real-time update, synchronizing the target data to be synchronized from the buffer directory to the running directory includes: Add a target link corresponding to the buffer directory to the running directory, and connect the running directory to the buffer directory based on the target link.

6. The method according to claim 4, characterized in that, When the update method is non-real-time update, synchronizing the target data to be synchronized from the buffer directory to the running directory includes: Copy the target data to be synchronized from the buffer directory to the running directory.

7. The method according to claim 1, characterized in that, The method further includes: The second client node receives the target data to be synchronized corresponding to the target branch information from the second data source node; wherein, the second data source node is used to synchronize the data to be synchronized from the first data source node within a preset time period.

8. A data synchronization method, characterized in that, Applied to a data source node, the method includes: Generate data to be synchronized; The data to be synchronized is synchronized to the client nodes, so that the first client node that receives the data to be synchronized acts as a distribution node and synchronizes the data to be synchronized to the second client node among the client nodes.

9. The method according to claim 8, characterized in that, The generation of data to be synchronized includes: The original resource files are sorted, and the sorted original resource files are packaged into a resource set according to a preset format; The resource set is compressed to obtain the data to be synchronized.

10. A data synchronization device, characterized in that, Applied to a client node, the device includes: A connection request sending module is used to send a connection request to a first data source node, so that the first data source node marks the client node as a node to be authorized, and the authorization management service authenticates the client node and obtains the distribution task configuration; wherein, the distribution task configuration includes at least the target branch information corresponding to the client node; The data receiving module is used for the second client node to receive target data to be synchronized corresponding to the target branch information from the first data source node or the first client node that has completed data synchronization, and to use the second client node as a distribution node to provide the target data to be synchronized to subsequent client nodes.

11. A data synchronization device, characterized in that, Applied to a data source node, the device includes: The data generation module is used to generate the data to be synchronized. The data synchronization module is used to synchronize the data to be synchronized to the client nodes, so that the first client node that receives the data to be synchronized acts as a distribution node and synchronizes the data to be synchronized to the second client node among the client nodes.

12. A program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method according to any one of claims 1 to 9.

13. An electronic device, characterized in that, include: processor; Memory for storing the executable instructions of the processor; The processor is configured to execute the method of any one of claims 1 to 9 by executing the executable instructions.

Images