Data synchronization method, device, medium and equipment

Abstract

The present disclosure provides a data synchronization method, a data synchronization device, a computer-readable storage medium and a terminal, and relates to the field of cloud storage technology. The method includes: receiving multiple tasks for data synchronization from a source storage space to multiple target storage spaces respectively; determining data information corresponding to the synchronized data according to the tasks, and storing the data information corresponding to the multiple tasks respectively in a message queue, and reading the data information corresponding to the multiple tasks respectively from the message queue in parallel; acquiring the synchronized data from the source storage space based on the data information corresponding to the multiple tasks respectively; and sending the synchronized data corresponding to the multiple tasks respectively to the corresponding target storage space in parallel. This technical solution can effectively improve the availability of cloud storage services and the efficiency of data synchronization.

Description

Technical Field

[0001] The present disclosure relates to the field of cloud storage technology, and in particular, to a data synchronization method, a data synchronization device, and a computer-readable storage medium and a terminal. Background Art

[0002] Cloud storage services can effectively guarantee the security and reliability of stored data. The cross-region replication of cloud storage services explicitly creates a backup in data centers in different regions of the cloud storage service, thereby preventing data from being irreversibly damaged and still having safe and available backup data.

[0003] However, in the cross-region replication process of cloud storage services, the difficulty in synchronizing user data between multiple regions lies in asynchronously and quickly synchronizing the upload, modification, deletion and other operations generated in the source storage space to the target storage space, and generating a copy of the same data object as the source storage space in the target storage space to ensure the consistency of the source storage data and the copied data.

[0004] The data synchronization solution provided by the related art does not support the problem of simultaneously copying data from a single source storage space to multiple target storage spaces. It can be seen that the availability of the data synchronization solution provided by the related art needs to be improved, and there is also the problem of low data synchronization efficiency.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] The purpose of the embodiments of the present disclosure is to provide a data synchronization method, a data synchronization device, a computer-readable storage medium and a terminal, thereby improving the data synchronization efficiency of the cloud storage service at least to a certain extent, thereby improving the availability of the cloud storage service.

[0007] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0008] According to one aspect of an embodiment of the present disclosure, a data synchronization method is provided, including: receiving multiple tasks for performing data synchronization from a source storage space to multiple target storage spaces respectively, and determining data information corresponding to the synchronized data according to the tasks; storing the data information respectively corresponding to the multiple tasks in a message queue; reading the data information respectively corresponding to the multiple tasks from the message queue in parallel, and obtaining the synchronized data from the source storage space based on the data information respectively corresponding to the multiple tasks; and sending the synchronized data respectively corresponding to the multiple tasks in parallel to the corresponding target storage spaces.

[0009] According to one aspect of an embodiment of the present disclosure, a data synchronization device is provided, including: a data information determination module, used to receive multiple tasks for performing data synchronization from a source storage space to multiple target storage spaces respectively, and determine data information corresponding to the synchronized data according to the tasks; a storage module, used to store the data information respectively corresponding to the multiple tasks in a message queue; a synchronized data acquisition module, used to read the data information respectively corresponding to the multiple tasks from the message queue in parallel, and acquire the synchronized data from the source storage space based on the data information respectively corresponding to the multiple tasks; and a parallel sending module, used to send the synchronized data respectively corresponding to the multiple tasks in parallel to the corresponding target storage spaces.

[0010] According to another aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the data synchronization method as described in the above embodiments is implemented.

[0011] According to another aspect of an embodiment of the present disclosure, a terminal is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the data synchronization method as described in the above embodiment is implemented.

[0012] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0013] In the technical solutions provided by some embodiments of the present disclosure, the following processing is performed for multiple tasks that perform data synchronization from the same source storage space to the target storage space respectively: determine the data information corresponding to the synchronized data according to the task, then store the data information corresponding to the multiple tasks respectively into the message queue, and read the data information corresponding to the multiple tasks respectively from the message queue in parallel. It can be seen that the technical solution implements the synchronization processing of the above-mentioned multiple tasks based on the message queue. Furthermore, based on the data information corresponding to the multiple tasks respectively, the synchronized data is obtained from the source storage space, and then the synchronized data corresponding to the above-mentioned multiple tasks respectively are sent in parallel to the corresponding target storage space. Thereby, the data is copied from the same source storage space to multiple target storage spaces at the same time, which effectively improves the data synchronization efficiency and further improves the availability of the cloud storage service.

[0014] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 A scenario diagram schematically illustrates a data synchronization solution for a cloud storage service in the related art.

[0017] Figure 2 A scenario diagram schematically illustrates a data synchronization solution for a cloud storage service in another related technology.

[0018] Figure 3 A scenario diagram of a data synchronization solution for a cloud storage service according to an exemplary embodiment of the present disclosure is schematically shown.

[0019] Figure 4 A scenario diagram of a data synchronization solution of a cloud storage service according to another exemplary embodiment of the present disclosure is schematically shown.

[0020] Figure 5 The schematic diagram shows an architecture diagram of a data synchronization system according to an exemplary embodiment of the present disclosure.

[0021] Figure 6 A flowchart of a data synchronization method according to an exemplary embodiment of the present disclosure is schematically shown.

[0022] Figure 7 A flowchart of a data synchronization method according to another exemplary embodiment of the present disclosure is schematically shown.

[0023] Figure 8 A scenario diagram of a cascade data synchronization solution in an embodiment of the present disclosure is schematically shown.

[0024] Fig. 9 A scenario diagram schematically illustrates a scheme for simultaneously synchronizing data from a same source storage space to multiple target storage spaces in an embodiment of the present disclosure.

[0025] Fig.10 The structure diagram of a data synchronization device according to an exemplary embodiment of the present disclosure is schematically shown.

[0026] Fig.11 The structure diagram of a data synchronization device according to another exemplary embodiment of the present disclosure is schematically shown.

[0027] Fig.12 The structure diagram of a terminal in an exemplary embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0028] In order to make the objectives, technical solutions and advantages of the present disclosure more clear, the embodiments of the present disclosure will be further described in detail below with reference to the accompanying drawings.

[0029] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are only examples of devices and methods consistent with some aspects of the present disclosure as detailed in the attached claims.

[0030] In the description of the present disclosure, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and cannot be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present disclosure can be understood according to specific circumstances. In addition, in the description of the present disclosure, unless otherwise specified, "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previously associated objects are in an "or" relationship.

[0031] With the popularization of mobile Internet and the storage demand for massive data generated by handheld devices, PCs, and wearable products. Especially with the rise of 5G networks, data will grow exponentially. Cloud storage services are a type of data storage service that provides a relatively simple, unified, and convenient data access service that has emerged with the massive storage demand. Cloud storage services can effectively guarantee the security and reliability of stored data.

[0032] The following is an explanation of the terms related to the data synchronization solution of cloud services:

[0033] Region: Indicates the region where the cloud storage service's data center is located, the physical location. Users can choose the region for data storage based on cost, request source, etc. Generally speaking, the region closer to the user has faster access speed.

[0034] Storage space: A storage space (Bucket) is a container for users to store objects (Object). All objects must belong to a storage space. Storage spaces have various configuration attributes, including region, access rights, storage type, etc.

[0035] Cross-region data synchronization / replication: A cross-region data disaster recovery capability provided by cloud storage services. By creating cross-region replication rules, data in a bucket / storage space (Bucket) is automatically and asynchronously replicated to another bucket in a different region under the same account.

[0036] Specifically, data synchronization / replication of cloud services is to explicitly create a backup in the data centers of different regions of the cloud storage service, which can prevent data from being irreversibly damaged and provide safe and available backup data. In addition, access efficiency can be improved by accessing cloud service data copies in multiple regions nearby. For example, you can automatically synchronize user data between multiple regions around the world by configuring cross-region replication, so that when users access data, they can access data in the corresponding region nearby, which can minimize the delay in customer access to data.

[0037] Generally, data synchronization / replication can refer to the replication between the master and slave databases in a database, the replication between multiple clusters in a distributed cluster, or the replication between multiple replicas in a distributed system. The difficulty lies in that data is usually constantly changing, and the changes need to be continuously reflected in multiple copies of data, and these copies must be guaranteed to be completely consistent. For cloud storage services, the difficulty of synchronizing user data between multiple regions lies in asynchronously and quickly synchronizing the upload, modification, deletion and other operations generated by the source storage space to the target storage space, and finally generating a copy of the data object in the target storage space that is the same as the source storage space.

[0038] For cloud service systems, the amount of historical object data in the source storage space that users need to replicate may be very large (tens of billions or even hundreds of billions), and high performance of the cross-region replication process must be guaranteed. In addition, during the cross-region data replication process, when the network transmission delay, bandwidth, and network instability between data centers in different regions are high, it is crucial to ensure high reliability and high availability of data synchronization.

[0039] However, there are some restrictions when users use the cross-region data synchronization solutions of existing cloud storage services. For example, cloud storage service providers restrict the conditions for enabling cross-region synchronization, that is, two buckets for data synchronization are not allowed to synchronize with another bucket. Figure 1 , while data synchronization is being performed between Bucket-A in Region-1 and Bucket-B in Region-2, data synchronization is not allowed between Bucket-A in Region-1 and Bucket-C in Region-3.

[0040] For another example, the migration of the same object data from the source bucket to the target bucket is allowed only once. Figure 2 , after data S stored in Bucket-A of Region-1 is synchronized to Bucket-B of Region-2 through cross-region data synchronization, the data S will not be allowed to be synchronized from Bucket-B of Region-2 to Bucket-C of Region-3.

[0041] The above technical problems existing in the related technologies are not conducive to the synchronization efficiency of data and thus result in low availability of cloud storage services.

[0042] In view of the technical problems existing in the above-mentioned related technologies, a data synchronization solution for a cloud storage service disclosed in the present invention is proposed. Based on the business characteristics of cloud storage services and user needs, the present invention proposes a cross-regional data synchronization solution. This solution can be applied to multi-region backup scenarios, data asynchronous synchronization scenarios, and can also be applied to multi-region data synchronization scenarios, which can effectively improve the reliability and availability of distributed storage systems, multi-active applications, etc.

[0043] Before introducing the data synchronization method, the composition and functions of the data synchronization system are introduced below. The specific implementation of the corresponding functions of each component will be specifically introduced in the method embodiment part.

[0044] in, Figure 5 The following schematically shows the architecture of a data synchronization system according to an exemplary embodiment of the present disclosure. Specifically, Figure 5 The following figure shows the execution process of a synchronization task that can synchronize the data in the storage space Bucket-X to the storage space Bucket-Y, that is, Bucket-X is used as the source storage space of the synchronization task, and Bucket-Y is used as the target storage space of the task. Figure 5 , because Bucket-X is in database 510 of Region-1 and Bucket-Y is in database 520 of Region-2, this task also belongs to cross-region data synchronization. At the same time, it can be seen that database 510 in Region-1 is the source database of the synchronization task, and database 520 in Region-2 is the target database of the synchronization task.

[0045] refer to Figure 5 In order to complete the above synchronization task, it involves: task management component (TaskManager component) 501, log synchronization component (LogSyncer component) 502, synchronization processing component (SyncWorker component) 503, message queue (Message Queue) 504, and data upload interface (S3 Service interface) 505. The execution process of the above synchronization task is as follows:

[0046] The log synchronization component (LogSyncer component) 502 is used to pull out the operation logs in the source database (including information such as addition, modification, deletion and change of the source storage space); illustratively, the pulled operation logs can be stored in a message queue for consumption by the subsequent task management component (TaskManager component) 501.

[0047] The task management component (TaskManager component) 501 is used to receive synchronization tasks and is responsible for decomposing the synchronization tasks into: historical data synchronization subtasks and incremental data synchronization subtasks.

[0048] For the historical data synchronization subtask, the TaskManager component 501 obtains (through a list query) the metadata of the synchronized data in the source storage space Bucket-X, and encapsulates and delivers it to the message queue (MessageQueue) 504. For the incremental data synchronization subtask, the TaskManager component 501 obtains the operation log related to the synchronized data (pulled by the above-mentioned LogSyncer component 502), and then encapsulates and delivers it to the message queue (Message Queue) 504. Therefore, the metadata and operation logs corresponding to the synchronized data in the above-mentioned synchronization task are encapsulated and delivered to the message queue (Message Queue) 504.

[0049] The synchronization processing component (SyncWorker component) 503 is used to read the encapsulated objects corresponding to the synchronization tasks from the message queue (Message Queue) 504 in parallel, and is also used to obtain the synchronized data from the source storage space Bucket-X based on the encapsulated objects.

[0050] Furthermore, the synchronized data corresponding to the synchronization task is uploaded to the target storage space Bucket-Y through the data upload interface (S3 Service interface) 505 .

[0051] The synchronization processing component (SyncWorker component) 503 is also used to perform idempotent processing on the message queue (MessageQueue) 504. The specific principle will be described in detail in the subsequent method embodiments.

[0052] The specific implementation of the functions of each component in the above data synchronization system will also be specifically described in the method embodiment.

[0053] The following will be combined with the attached Figure 6 -Attached Figure 8 , the data synchronization method provided by the embodiment of the present disclosure is introduced in detail.

[0054] in, Figure 6FIG. 1 is a flow chart of a data synchronization method according to an exemplary embodiment of the present disclosure. Figure 6 , the method comprises the following steps:

[0055] S610, receiving a plurality of tasks for performing data synchronization from a source storage space to a plurality of target storage spaces respectively, and determining data information corresponding to the synchronized data according to the tasks;

[0056] S620, storing data information corresponding to the plurality of tasks respectively into a message queue;

[0057] S630, reading data information corresponding to the multiple tasks in parallel from the message queue, and acquiring synchronized data from the source storage space based on the data information corresponding to the multiple tasks; and

[0058] S640, sending the synchronized data corresponding to the multiple tasks to the corresponding target storage space in parallel.

[0059] Figure 6 The technical solution provided by the illustrated embodiment can support the simultaneous copying of data from a single source storage space to multiple target storage spaces (such as Figure 3 As shown in the figure: while data synchronization is performed between Bucket-A in Region-1 and Bucket-B in Region-2, data synchronization is not allowed between Bucket-A in Region-1 and Bucket-C in Region-3, etc.), this technical solution can effectively improve the availability and data synchronization efficiency of cloud storage services. At the same time, it can also improve the data security and access performance (nearby access) of users.

[0060] The following combination Figure 5 The components in the system shown are Figure 6 The specific implementation methods of each step included in the illustrated embodiment are described in detail:

[0061] In S610, a plurality of tasks for performing data synchronization from a source storage space to a plurality of target storage spaces are received.

[0062] In an exemplary embodiment, each of the above tasks may include a synchronized object identifier, a source storage space identifier, and a target storage space identifier. For example, if a user backs up photos currently stored in Bucket-A in Bucket-B, the task issued by the user includes: the name of the photo to be backed up (synchronized object identifier), Bucket-A identifier (source storage space identifier), and Bucket-B identifier (target storage space identifier). The source storage space indicates the storage space where the data in the request task is located before being synchronized, and the target storage space indicates the storage space to which the data in the request task will be synchronized.

[0063] In an exemplary embodiment, the data synchronization task for the same storage space may be determined according to the source storage space identifier in the task. Figure 3 , which shows multiple tasks for synchronizing data from the same source storage space to multiple target storage spaces. Specifically, the source storage space is Bucket-A of source storage space Region-1, and the target storage spaces include: Bucket-B of Region-2, Bucket-C of Region-3, ... The above multiple tasks can synchronize the data in Bucket-A of Region-1 to Bucket-B of Region-2, Bucket-C of Region-3, ... respectively.

[0064] For example, Figure 7 The following schematically shows a flow chart of a data synchronization method according to another exemplary embodiment of the present disclosure. Figure 7 Let's introduce the specific implementation of S610-S640.

[0065] refer to Figure 7 As a specific implementation of "determining data information corresponding to synchronized data according to tasks" in S610: in S710, obtaining meta information of synchronized objects in the source storage space. And, in S710', obtaining operation logs about synchronized objects.

[0066] In an exemplary embodiment, according to the synchronized data identifier carried in the task, the meta information of the synchronized data is obtained in the source storage space, and the operation log about the synchronized data is obtained, and the meta information and the operation log are determined as the data information corresponding to the synchronized data.

[0067] Since the data in the source storage space may be constantly changing, in order to ensure the consistency of the synchronized data between the source storage space and the target storage space, in this exemplary embodiment, each task is decomposed into a historical data synchronization subtask and an incremental data synchronization subtask. Among them, on the one hand, in order to implement the historical data synchronization subtask, the technical solution needs to obtain the metadata of the synchronized object in the source storage space, so as to further determine the historical data corresponding to the above target task based on the above metadata. On the other hand. In order to implement the incremental data synchronization subtask, the technical solution needs to obtain the operation log about the synchronized object, so as to further determine the incremental data corresponding to the above target task based on the above operation log. Among them, the above incremental data is the upload, modification and other operations generated by the source storage space after the cross-region replication task is configured.

[0068] Specifically, regarding the embodiment of obtaining the meta information of the synchronized object in S710 above:

[0069] In an exemplary embodiment, for historical data of upload, modification, and other operations that were not generated in the source storage space after the cross-region replication task was configured, the object listing interface provided by the cloud storage service can be used to query the data metadata list in the source storage space to obtain the metadata of the synchronized objects in the source storage space corresponding to the above-mentioned target task.

[0070] In this embodiment, the TaskManager component 501 can specifically call the cloud storage service interface API of the source storage space according to the above-mentioned target task configured by the user to list the metadata of the synchronized objects in the source storage space (including: storage space information Bucket, object name, entity tag (Entity tag, Etag) verification value, upload time, object size, etc.), thereby obtaining the metadata of the synchronized objects in the above-mentioned target task.

[0071] Regarding the acquisition of the operation log of the synchronized object in the above S710':

[0072] In an exemplary embodiment, the synchronization of incremental data generated by the source storage space can be processed by using a replication log based on the source storage space in the implementation. The replication log records information such as data insertion operations, data modification operations, and data deletion operations on the source storage space. Taking the replication log oplog of the database Mongodb as an example, it records information such as operation time, operation type (including: new / insertion operations, modification operations, deletion operations, etc.), source space of the data, Key, entity tag Etag check value, etc.

[0073] In an exemplary embodiment, the LogSyncer component 502 may specifically pull data from the replication log of the source database, and filter out the operation logs related to the synchronized object according to the cross-region data synchronization task configured by the user. Exemplarily, the pulled operation logs may also be sent to a message queue (it should be noted that the message queue may be a message queue (Message Queue) 504 used to store encapsulated objects later, or may be other message queues). Furthermore, the TaskManager component 501 consumes the operation logs related to the synchronized object produced by the LogSyncer component 502 in the message queue, thereby obtaining the operation logs of the synchronized object corresponding to the target task.

[0074] It should be noted that the above data information also includes historical synchronization information, which indicates that the synchronized data is used for cascade synchronization. More specifically, the above historical synchronization information is included in the operation log, and then the information is used to implement cascade synchronization of data. That is, the migration of the same object data from the source bucket to the target bucket can be multiple times (such as Figure 4 As shown: Data S stored in Bucket-A of Region-1 is synchronized to Bucket-B of Region-2 through cross-region data synchronization. An operation log including historical synchronization information is generated in Bucket-B, indicating that data S is copied from Bucket-A to Bucket-B. Furthermore, when synchronizing the data S from Bucket-B to Bucket-C, the task of synchronizing the data from Bucket-B of Region-2 to Bucket-C of Region-3 is executed based on the above synchronization information), thereby effectively solving the cascading transmission and replication problem between multiple storage spaces, and thus effectively improving the availability and scalability of cloud storage services. The relevant implementation method will be described in Figure 8 The corresponding embodiments are described in detail.

[0075] Continue to refer Figure 7 As a specific implementation of S620: in S720, the meta information is encapsulated to obtain a first encapsulation object, and the first encapsulation object is sent to the message queue; in S720', the operation log is encapsulated to obtain a second encapsulation object, and the second encapsulation object is sent to the message queue.

[0076] Exemplarily, in order to increase the consumption speed of objects in the message queue and thus improve the data synchronization efficiency, the formats of the first encapsulated object and the second encapsulated object are the same. Specifically, for the meta information generated by S710: the TaskManager component 501 encapsulates it into a first encapsulated object in a target format (e.g., Sync Job Message format), and delivers it to the message queue (Message Queue) 504. For the operation log generated by S710': the TaskManager component 501 also encapsulates it into a second encapsulated object in a target format (e.g., Sync Job Message format), and delivers it to the message queue (Message Queue) 504.

[0077] Continue to refer Figure 7 , as a specific implementation of “reading data information corresponding to multiple tasks in parallel from the message queue” in S630: in S730, the first encapsulation object and the second encapsulation object are obtained in parallel from the message queue.

[0078] It should be noted that, on the one hand, encapsulating the above-mentioned operation information and metadata into the same format is conducive to realizing the replication of single-object data, thereby helping to improve the efficiency of data synchronization processing. On the other hand, the technical effect of encapsulating the data information of multiple tasks for data synchronization from the same source storage space to multiple target storage spaces and delivering them to the same message queue is: according to how many replication tasks are set for the source storage space, the encapsulated objects of the above-mentioned target format are determined respectively according to the data information of each task, and the encapsulated objects corresponding to each task are delivered to the above-mentioned message queue, so that the SyncWorker component processes the sealed objects of each task in parallel, so as to achieve the technical effect of automatically copying the source storage space data to the storage spaces in multiple target regions (such as Figure 3 ).

[0079] Among them, the above-mentioned message queue (Message Queue) 504 is an encapsulated object for storing data information of each task in the source storage space. It has high performance and reliability, and can persist the data storage function to ensure that the data in the message queue is not lost, thereby ensuring the high reliability of the data synchronization solution of the cloud storage service. It should be noted that in order to facilitate the maintenance of the message queue, the data in the above-mentioned message queue (Message Queue) 504 does not have to be ordered, that is, the information in the message queue can be repeated. In response to this problem, the technical solution implements idempotent processing of the message queue by the SyncWorker component 503 to ensure the ultimate consistency of the data in the source storage area and the target storage area. Among them, after the idempotent processing, the effect of executing the same operation multiple times is the same as executing it once.

[0080] Continue to refer Figure 6 In S630, the synchronized data is obtained from the source storage space based on the data information corresponding to the multiple tasks.

[0081] In an exemplary embodiment, the replication module deployed in each target storage area can consume the encapsulated objects (including: the first encapsulated object and the second encapsulated object) from the message queue (Message Queue) 504, and then download the synchronized data from the source storage space and upload it to the target storage space. Figure 7 Among them, S740, S740' and S750 can be used as specific implementation methods of obtaining synchronized data in S630.

[0082] In S740, data is downloaded from the source storage space according to the meta information to obtain historical data in the synchronized data.

[0083] In an exemplary embodiment, the SyncWorker component 503 consumes the encapsulated objects in the message queue in parallel. Since the formats of the first encapsulated object and the second encapsulated object can be the same, it is not necessary to pay attention to whether the encapsulated objects contain metadata or operation logs when consuming them, thereby effectively improving data synchronization efficiency.

[0084] For the encapsulated object containing meta information (ie, the first encapsulated object mentioned above), by calling the cloud storage service interface API of the source storage space and downloading data according to the meta information, the historical data in the synchronized data can be obtained.

[0085] In S740', when the operation log is a new operation type, the corresponding synchronized data is obtained from the source storage space based on the data information corresponding to each task, including: downloading the data corresponding to the new operation type to obtain the incremental data in the synchronized data.

[0086] For example, after configuring the cross-region replication task, a new image Y is uploaded to the source storage space, so an operation log y about the modification is generated in the source storage space. In order to ensure the consistency between the source storage space and the target storage space, image Y needs to be downloaded according to the operation log y.

[0087] In S750, when the operation log is of a modification type: target data is determined in the historical data, and the target data is processed according to the modification operation type to obtain incremental data in the synchronized data.

[0088] For example, the synchronized data is picture X, which has been modified after the cross-region replication task is configured (e.g., the pixel value of picture X is changed), so an operation log x about the modification is generated in the source storage space. Then, for picture X, the picture before the modification (i.e., historical data) can be obtained through S740, and then in step S750, the picture X before the modification is first located in the historical data, and then the modification data is recorded in the operation log x to modify it to obtain the picture X after the pixel is changed. This can effectively ensure the consistency of data in the source storage space and the target storage space.

[0089] It should be noted that it is technically feasible to synchronize the deletion operation in the source storage space. However, in order to ensure that user data is not lost due to accidental deletion, the deletion operation in the source storage space is not synchronized in this technical solution.

[0090] Continue to refer Figure 7As a specific implementation of step S640, in S760, the historical data and the incremental data are uploaded to the target storage space. In an exemplary embodiment, the historical data and the operation data are uploaded to the storage space of the target storage area in a streaming manner by calling the data upload interface (S3 Service interface) 505 (wherein the S3 Service interface is a S3 protocol compatible interface provided by a cloud storage service), which can further improve the data synchronization efficiency.

[0091] In an exemplary embodiment, in order to ensure the content integrity and data consistency of the synchronized data, the data content and time sequence of the synchronized data corresponding to the multiple tasks are verified during the upload to the target storage space. If the verification passes, the synchronized data is uploaded to the target storage space.

[0092] Specifically, the entity tag information Etag carried in the meta information and / or the operation log is verified to verify whether the processed data is the latest version of the data to ensure the content integrity of the synchronized data. For the successive uploads of the same key, if disorder occurs and the problem of overwriting data occurs, the conflict-free replicated data types (CRDT) are used to ensure the final consistency of the business data on both sides, and to prevent the erroneous overwriting of data in the target storage area, which is conducive to ensuring the high reliability of the data synchronization solution of the cloud storage service.

[0093] In this technical solution, data consistency between the source storage space and the target storage space is achieved based on CRDT, where CRDT refers to a data structure in a distributed system that can be replicated in parallel in multiple hosts in a network without cooperation and can resolve possible inconsistencies.

[0094] In an exemplary embodiment, an atomic operation is also performed on the entity tag information of the synchronized data based on the upload time of the historical data and / or the update time of the incremental data. Specifically, the compare and swap (CAS) in the atomic operation is performed to ensure the final consistency of the synchronized data, and even if the network is unstable, the message is out of order, or delayed across regions, the final consistency between multiple copies can still be achieved.

[0095] Among them, for distributed replication systems, it is particularly important to ensure the consistency model between multiple copies of data. At present, the main consistency models in the industry are the following types: strict consistency, sequential consistency, causal consistency, and eventual consistency. The differences between the above consistency models are mainly the degree of restriction, implementation complexity, and performance in distributed systems. For systems that need to solve the synchronization of massive data through cross-region asynchronous replication, eventual consistency is generally required.

[0096] In an exemplary embodiment, when downloading data through the SyncWorker component 503, the upload time and Etag check value in the meta information are downloaded. If the metadata with the same relevant information already exists, the copy is skipped, thereby avoiding the problem of circular configuration copy (such as Bucket-A-Bucket-B-Bucket A), and thus unnecessary data processing computing power consumption.

[0097] This technical solution not only realizes the simultaneous copying of data from a single source storage space to multiple target storage spaces, but also enables the migration of the same object data from the source bucket to the target bucket multiple times. Figure 8 A scenario diagram of a cascade data synchronization solution in an exemplary embodiment of the application is schematically shown.

[0098] refer to Figure 8 , the user uploads the storage object to Region-1, in order to achieve the synchronization of the storage object in three regions. This technical solution includes two tasks, Task-1: object synchronization from Region-1 to Region-2, and Task-2: object synchronization to Region-2 from Region-2 to Region-3. The solution to achieve cascade synchronization of data based on this technical solution is as follows:

[0099] For Task-1: The TaskManager-1 component receives Task-1, which contains the synchronized object identifier object, the source storage space identifier Region-1, and the target storage space identifier Region-2. The TaskManager-1 component also obtains the metadata of the object in the source storage space Region-1. Then, the LogSyncer-1 component (not shown) filters out the log information related to the above object from the log information in Region-1 to obtain the operation log of the object. Furthermore, the TaskManager-1 component encapsulates the above operation log and the encapsulated metadata respectively, and delivers the obtained encapsulated objects of the same format to the message queue-1. Then, the SyncWorker-1 component determines the historical data corresponding to Task-1 based on the above metadata in the message queue-1 and determines the incremental data corresponding to Task-1 based on the above operation log. Finally, the above historical data and the above incremental data are uploaded to Region-2 through the S3 Service interface to complete Task-1. Thereby, the object is synchronized from Region-1 to Region-2.

[0100] It should be noted that after the object is synchronized from Region-1 to Region-2, an operation log about the object is generated in Region-2. The operation log contains historical synchronization information, which is used to indicate that the synchronized data object is used for cascade synchronization.

[0101] Then, for Task-2: The TaskManager-2 component receives Task-2, which contains the synchronized object identifier object, the source storage space identifier Region-2, and the target storage space identifier Region-3. The TaskManager-2 component also obtains the metadata of the object in the above-mentioned source storage space Region-2. Then, the LogSyncer-2 component (not shown) filters out the log information related to the above-mentioned object from the log information in Region-2 to obtain the operation log of the object. It should be noted that regarding the operation log containing historical synchronization information generated after the object is synchronized from Region-1 to Region-2, this operation log will be filtered out by the LogSyncer-2 component of Region-2 as the log information related to the above-mentioned object and delivered to the message queue-2 of Region-2.

[0102] Furthermore, the TaskManager-2 component encapsulates the above operation logs and encapsulated meta-information respectively, and delivers the obtained encapsulated objects of the same format to the message queue-2. Then, the SyncWorker-2 component determines the historical data corresponding to Task-2 according to the above meta-information and the incremental data corresponding to Task-2 according to the above operation logs. Among them, for the object encapsulated by the operation log containing historical synchronization information, the object will be downloaded according to the encapsulated object and uploaded to the target storage space Region-3 of the task to complete Task-2.

[0103] It should be noted that after synchronizing data from Region-2 to Region-3, a related operation log will also be generated in the source database of Region-3. The operation log contains historical synchronization information, which is used to indicate that the synchronized data object is used for cascade synchronization. Therefore, the operation log can be used to synchronize the data of the object from Region3 to Region-4. By analogy, the object can be replicated level by level. In this way, the data of the source storage space can be synchronized to the target regions at each level in a cascade manner.

[0104] This technical solution (such as Figure 8 As shown in the figure, it can support the migration of the same object data from the source storage space to the target storage space multiple times, thereby effectively solving the cascade transmission and replication problem between multiple storage spaces. In theory, it can be infinitely expanded in multiple levels and can solve the infinite loop problem of data replication when loop configuration is used for cross-region replication.

[0105] In an exemplary embodiment, Fig. 9 A scenario diagram schematically illustrates a scheme for simultaneously synchronizing data from a same source storage space to multiple target storage spaces in an embodiment of the present disclosure.

[0106] refer to Fig. 9 , the figure shows the data synchronization tasks performed simultaneously from the same source storage space Bucket-X to two target storage spaces Bucket-Y1 and Bucket-Y2: Task (data S1, X-Y1) and Task (data S2, X-Y1). Among them, Task (data S1, X-Y1) means synchronizing data S1 stored in Bucket-X to Bucket-Y1, and Task (data S2, X-Y2) means synchronizing data S2 stored in Bucket-X to Bucket-Y2. It should be noted that data S1 and data S2 are both data stored in Bucket-X, and data S1 and data S2 can be the same data or different data, which is not limited here.

[0107] refer to Fig. 9, the synchronous execution process of the above two tasks is as follows:

[0108] The LogSyncer component pulls out the operation logs in the source database (including information such as addition, modification, deletion, and change of the source storage space Bucket-X); illustratively, the pulled operation logs can be stored in a message queue for subsequent consumption by the TaskManager component.

[0109] Furthermore, the TaskManager component receives the above-mentioned task (data S1, X-Y1) and task (data S2, X-Y1), and decomposes the two tasks into: historical data synchronization subtask and incremental data synchronization subtask. Specifically: split the task (data S1, X-Y1) into a historical data synchronization subtask (data S1', X-Y1) and an incremental data synchronization subtask (data S1", X-Y1); split the task (data S2, X-Y2) into a historical data synchronization subtask (data S2', X-Y2) and an incremental data synchronization subtask (data S2", X-Y2).

[0110] Among them, for the historical data synchronization subtasks (data S1', X-Y1) and (data S2', X-Y2), the TaskManager component obtains the metadata of the synchronized data S1 and S2 in Bucket-X, and encapsulates and delivers them to the Message Queue respectively. For the incremental data synchronization subtasks (data S1", X-Y1) and (data S2", X-Y2), the TaskManager component obtains the operation logs related to the synchronized data S1 and S2 (pulled by the above-mentioned LogSyncer component), and then encapsulates and delivers them to the Message Queue respectively. As a result, the metadata and operation logs corresponding to the synchronized data S1 in the above-mentioned task (data S1, X-Y1) are encapsulated and delivered to the Message Queue. At the same time, the metadata and operation logs corresponding to the synchronized data S2 in the above-mentioned task (data S2, X-Y2) are also encapsulated and delivered to the Message Queue.

[0111] Furthermore, the SyncWorker component reads the encapsulated objects corresponding to the above two tasks from the Message Queue in parallel, and then obtains the synchronized data S1 and S2 from the source storage space Bucket-X based on the encapsulated objects. Among them, the above operation log may contain modification information of the synchronized data S1 and / or S2, but in order to ensure that the synchronized data is not lost, the deletion operation of the synchronized data S1 and S2 is not synchronized in this embodiment. In the case of modification of the synchronized data, the synchronized data is modified and updated accordingly through the relevant operation log. For example, if the operation log contains modification information of the synchronized data S1, the synchronized data S1 is modified accordingly to ensure that the synchronized data remains consistent between the source storage space and the target storage space.

[0112] Furthermore, the synchronized data S1 is uploaded to the storage space Bucket-Y1 in the target database 101 through the data upload interface S3 Service interface, and the synchronized data S2 is uploaded to the storage space Bucket-Y2 in the target database 102 through the upload interface S3 Service interface.

[0113] This technical solution (such as Fig. 9 As shown in the figure, it can support the replication of data from a single source storage space to multiple target storage spaces at the same time, so that cloud storage service users can easily configure the replication of data from a source storage space to multiple target geographical area spaces, achieving the technical effect of uploading in one place, automatically backing up in multiple places, and accessing in multiple places. It is conducive to strengthening the security of user data, access performance and availability of user systems, and also to broadening the capabilities of cloud storage service users.

[0114] The following are embodiments of the device disclosed herein, which can be used to execute the method embodiments disclosed herein. For details not disclosed in the device embodiments disclosed herein, please refer to the method embodiments disclosed herein.

[0115] in, Fig.10 FIG. 1 shows a structural diagram of a data synchronization device according to an exemplary embodiment of the present disclosure. Fig.10 The data synchronization device shown in the figure can be implemented as all or part of the terminal through software, hardware or a combination of both, and can also be integrated on the server as an independent module.

[0116] The data synchronization device 1000 in the embodiment of the present disclosure includes: a data information determination module 1010, a storage module 1020, a synchronized data acquisition module 1030 and a parallel sending module 1040, wherein:

[0117] The above-mentioned data information determination module 1010 is used to receive multiple tasks for synchronizing data from a source storage space to multiple target storage spaces respectively, and determine the data information corresponding to the synchronized data according to the above-mentioned tasks; the above-mentioned storage module 1020 executes the function of the above-mentioned TaskManager component 501, and is used to store the data information corresponding to the above-mentioned multiple tasks in the message queue; the above-mentioned synchronized data acquisition module 1030 executes the function of the above-mentioned SyncWorker component 503, and is used to read the data information corresponding to the above-mentioned multiple tasks in parallel from the above-mentioned message queue, and obtain the synchronized data from the above-mentioned source storage space based on the data information corresponding to the above-mentioned multiple tasks; and the above-mentioned parallel sending module 1040 is used to send the synchronized data corresponding to the above-mentioned multiple tasks in parallel to the corresponding target storage space.

[0118] In an exemplary embodiment, Fig.11 Schematically shows a structural diagram of a data synchronization device according to another exemplary embodiment of the present disclosure. Fig.11 :

[0119] In an exemplary embodiment, the above-mentioned data information includes historical synchronization information, and the above-mentioned historical synchronization information indicates that the above-mentioned synchronized data is used for cascade synchronization, wherein the above-mentioned storage module 1020 is specifically used to: store the above-mentioned historical synchronization information in the above-mentioned message queue; the above-mentioned synchronized data acquisition module 1030 is specifically used to: obtain the synchronized data from the above-mentioned source storage space according to the above-mentioned historical synchronization information.

[0120] In an exemplary embodiment, the data information determination module 1010 is specifically used to: obtain the metadata of the synchronized data in the source storage space according to the synchronized data identifier carried in the task (i.e., execute the function of the TaskManager component 501), and obtain the operation log about the synchronized data, and determine the metadata and the operation log as the data information corresponding to the synchronized data.

[0121] In an exemplary embodiment, the storage module 1020 includes: a first packaging unit 10201 and a second packaging unit 10202. Wherein:

[0122] The above-mentioned first encapsulation unit 10201 executes the function of the above-mentioned TaskManager component 501, and is used to: encapsulate the above-mentioned metadata to obtain a first encapsulation object, and send the above-mentioned first encapsulation object to the above-mentioned message queue; the above-mentioned second encapsulation unit 10202 executes the function of the above-mentioned TaskManager component 501, and is used to: encapsulate the above-mentioned operation log to obtain a second encapsulation object and send the above-mentioned second encapsulation object to the above-mentioned message queue.

[0123] In an exemplary embodiment, the first packaged object and the second packaged object have the same format.

[0124] In an exemplary embodiment, the operation log in the above-mentioned data information includes historical synchronization information, and the above-mentioned historical synchronization information indicates that the above-mentioned synchronized data is used for cascade synchronization, wherein the above-mentioned second encapsulation unit 10202 executes the function of the above-mentioned TaskManager component 501, and is specifically used for: encapsulating the above-mentioned historical synchronization information to obtain a second encapsulation object; the above-mentioned synchronized data acquisition module 1030 executes the function of the above-mentioned SyncWorker component 503, and is specifically used for: reading the above-mentioned first encapsulation object to obtain information for downloading the above-mentioned synchronized data; and, according to the metadata of the above-mentioned synchronized data, downloading the above-mentioned synchronized data from the above-mentioned source storage space to obtain the above-mentioned synchronized data.

[0125] In an exemplary embodiment, the synchronized data acquisition module 1030 executes the function of the SyncWorker component 503, and is further specifically used to: download data from the source storage space according to the meta information to obtain the historical data in the synchronized data.

[0126] In an exemplary embodiment, the synchronized data acquisition module 1030 executes the functions of the SyncWorker component 503, and is also specifically used for: when the operation log is a modification operation type, determining the target data in the historical data, and processing the target data according to the modification operation type to obtain the incremental data in the synchronized data; when the operation log is a new operation type, obtaining the corresponding synchronized data from the source storage space based on the data information corresponding to each of the above tasks, including: downloading the data corresponding to the new operation type to obtain the incremental data in the synchronized data.

[0127] In an exemplary embodiment, the parallel sending module 1040 is specifically used to: verify the data content and time sequence of the synchronized data corresponding to the multiple tasks; if the verification passes, the synchronized data corresponding to the multiple tasks are sent in parallel to the corresponding target storage space.

[0128] In an exemplary embodiment, the parallel sending module 1040 includes: a verification unit 10401 and an upload unit 10402. Among them:

[0129] The verification unit 10401 is used to verify the entity tag information carried in the meta information and / or the operation log; the upload unit 10402 is used to upload the historical data and the incremental data to the target storage space if the verification passes.

[0130] In an exemplary embodiment, the verification unit 10402 is specifically used to perform atomic operations on the entity tag information of the synchronized data based on the upload time of the historical data and / or the update time of the incremental data.

[0131] It should be noted that the data synchronization device provided in the above embodiment only uses the division of the above functional modules as an example when executing the data synchronization method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the data synchronization device provided in the above embodiment belongs to the same concept as the data synchronization method embodiment. Therefore, for details not disclosed in the embodiment of the device disclosed in the present invention, please refer to the embodiment of the data synchronization method disclosed in the present invention, which will not be repeated here.

[0132] The serial numbers of the above-mentioned embodiments of the present disclosure are only for description and do not represent the advantages or disadvantages of the embodiments.

[0133] The embodiments of the present disclosure also provide a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the method of any of the above embodiments are implemented. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0134] The embodiment of the present disclosure also provides a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above-mentioned method embodiments when executing the program.

[0135] Fig.12 The structure diagram of the terminal in an exemplary embodiment of the present disclosure is schematically shown. Fig.12 As shown, the terminal 1200 includes: a processor 1201 and a memory 1202 .

[0136] In the embodiment of the present disclosure, the processor 1201 is the control center of the computer system, which can be the processor of a physical machine or the processor of a virtual machine. The processor 1201 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1201 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1201 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state.

[0137] In the embodiment of the present disclosure, the processor 1201 is specifically configured to:

[0138] Receive multiple tasks for synchronizing data from a source storage space to multiple target storage spaces respectively, and determine data information corresponding to the synchronized data according to the above tasks; store the data information corresponding to the above multiple tasks in a message queue, and read the data information corresponding to the above multiple tasks in parallel from the above message queue; obtain the synchronized data from the above source storage space based on the data information corresponding to the above multiple tasks; send the synchronized data corresponding to the above multiple tasks in parallel to the corresponding target storage spaces.

[0139] Furthermore, the above-mentioned data information includes historical synchronization information, and the above-mentioned historical synchronization information indicates that the above-mentioned synchronized data is used for cascade synchronization, wherein the above-mentioned storing the data information corresponding to the above-mentioned multiple tasks respectively into the message queue includes: storing the above-mentioned historical synchronization information into the above-mentioned message queue; the above-mentioned obtaining the synchronized data from the above-mentioned source storage space based on the data information corresponding to the above-mentioned multiple tasks respectively includes: obtaining the synchronized data from the above-mentioned source storage space according to the above-mentioned historical synchronization information.

[0140] Furthermore, the above-mentioned determination of data information corresponding to the synchronized data according to the above-mentioned task includes: obtaining metadata of the above-mentioned synchronized data in the above-mentioned source storage space according to the synchronized data identifier carried in the above-mentioned task, and obtaining operation logs about the above-mentioned synchronized data, and determining the above-mentioned metadata and the above-mentioned operation logs as the data information corresponding to the above-mentioned synchronized data.

[0141] Furthermore, the above-mentioned storing the data information corresponding to the above-mentioned multiple tasks respectively into the message queue includes: encapsulating the above-mentioned meta-information to obtain a first encapsulation object, and sending the above-mentioned first encapsulation object to the above-mentioned message queue; encapsulating the above-mentioned operation log to obtain a second encapsulation object, and sending the above-mentioned second encapsulation object to the above-mentioned message queue.

[0142] Furthermore, the formats of the first encapsulated object and the second encapsulated object are the same.

[0143] Furthermore, the operation log in the above-mentioned data information includes historical synchronization information, and the above-mentioned historical synchronization information indicates that the above-mentioned synchronized data is used for cascade synchronization, wherein the above-mentioned encapsulation of the above-mentioned operation log to obtain a second encapsulation object includes: encapsulating the above-mentioned historical synchronization information to obtain a second encapsulation object; the above-mentioned reading of the data information corresponding to the above-mentioned multiple tasks in parallel from the above-mentioned message queue includes: reading the above-mentioned first encapsulation object to obtain information for downloading the above-mentioned synchronized data; obtaining the synchronized data from the above-mentioned source storage space based on the data information corresponding to the above-mentioned multiple tasks, including: downloading the above-mentioned synchronized data from the above-mentioned source storage space according to the metadata of the above-mentioned synchronized data.

[0144] Furthermore, the acquiring of synchronized data from the source storage space based on the data information respectively corresponding to the plurality of tasks includes: downloading data from the source storage space according to the meta-information to obtain historical data in the synchronized data.

[0145] Furthermore, the above-mentioned obtaining of synchronized data from the above-mentioned source storage space based on the data information respectively corresponding to the above-mentioned multiple tasks includes: in the case where the above-mentioned operation log is a modification operation type, determining the target data in the above-mentioned historical data, and processing the above-mentioned target data according to the above-mentioned modification operation type to obtain the incremental data in the above-mentioned synchronized data; in the case where the above-mentioned operation log is a new operation type, obtaining the corresponding synchronized data from the above-mentioned source storage space based on the data information corresponding to each of the above-mentioned tasks, including: downloading the data corresponding to the above-mentioned new operation type to obtain the incremental data in the above-mentioned synchronized data.

[0146] Furthermore, the above-mentioned sending the synchronized data corresponding to the above-mentioned multiple tasks respectively to the corresponding target storage space in parallel includes: verifying the data content and time sequence of the synchronized data corresponding to the above-mentioned multiple tasks respectively; if the verification passes, sending the synchronized data corresponding to the above-mentioned multiple tasks respectively to the corresponding target storage space in parallel.

[0147] Furthermore, the synchronized data corresponding to the above-mentioned multiple tasks are sent in parallel to the corresponding target storage space, including: verifying the entity tag information carried in the above-mentioned metadata and / or the above-mentioned operation log; if the verification passes, uploading the above-mentioned historical data and the above-mentioned incremental data to the above-mentioned target storage space.

[0148] Furthermore, uploading the historical data and the incremental data to the target storage space includes: performing an atomic operation on the entity tag information of the synchronized data based on the upload time of the historical data and / or the update time of the incremental data.

[0149] The memory 1202 may include one or more computer-readable storage media, which may be non-transitory. The memory 1202 may also include a high-speed random access memory and a non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments of the present disclosure, the non-transitory computer-readable storage medium in the memory 1202 is used to store at least one instruction, which is used to be executed by the processor 1201 to implement the method in the embodiment of the present disclosure.

[0150] In some embodiments, the terminal 1200 further includes: a peripheral device interface 1203 and at least one peripheral device. The processor 1201, the memory 1202 and the peripheral device interface 1203 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 1203 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a display screen 1204, a camera 1205 and an audio circuit 1206.

[0151] The peripheral device interface 1203 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 1201 and the memory 1202. In some embodiments of the present disclosure, the processor 1201, the memory 1202, and the peripheral device interface 1203 are integrated on the same chip or circuit board; in some other embodiments of the present disclosure, any one or two of the processor 1201, the memory 1202, and the peripheral device interface 1203 may be implemented on a separate chip or circuit board. The embodiments of the present disclosure are not specifically limited to this.

[0152] The display screen 1204 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 1204 is a touch display screen, the display screen 1204 also has the ability to collect touch signals on the surface or above the surface of the display screen 1204. The touch signal can be input to the processor 1201 as a control signal for processing. At this time, the display screen 1204 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments of the present disclosure, the display screen 1204 can be one, and the front panel of the terminal 1200 is set; in other embodiments of the present disclosure, the display screen 1204 can be at least two, which are respectively set on different surfaces of the terminal 1200 or are folded; in some other embodiments of the present disclosure, the display screen 1204 can be a flexible display screen, which is set on the curved surface or folded surface of the terminal 1200. Even, the display screen 1204 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 1204 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0153] Camera 1205 is used to capture images or videos. Optionally, camera 1205 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize the panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments of the present disclosure, camera 1205 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0154] The audio circuit 1206 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 1201 for processing. For the purpose of stereo sound collection or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 1200. The microphone may also be an array microphone or an omnidirectional collection microphone.

[0155] The power supply 1207 is used to power various components in the terminal 1200. The power supply 1207 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 1207 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery charged through a wired line, and a wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0156] The terminal structure block diagram shown in the embodiment of the present disclosure does not constitute a limitation on the terminal 1200. The terminal 1200 may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0157] In the present disclosure, the terms "first", "second", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or order; the term "plurality" refers to two or more, unless otherwise expressly defined. The terms "installed", "connected", "connected", "fixed", etc. should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; "connected" can be a direct connection or an indirect connection through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms in the present disclosure can be understood according to the specific circumstances.

[0158] In the description of the present disclosure, it is necessary to understand that the orientation or positional relationship indicated by the terms "upper" and "lower" etc. is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present disclosure and simplifying the description, rather than indicating or implying that the device or unit referred to must have a specific direction, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present disclosure.

[0159] The above is only a specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, equivalent changes made according to the claims of the present disclosure are still within the scope covered by the present disclosure.

Claims

1. A data synchronization method, characterized in that: include: receiving a plurality of tasks for performing data synchronization from a source storage space to a plurality of target storage spaces respectively, obtaining meta information of the synchronized data in the source storage space according to synchronized data identifiers carried in the tasks, and obtaining operation logs about the synchronized data, and determining the meta information and the operation logs as data information corresponding to the synchronized data; the synchronized data includes historical data and incremental data, the historical data is used to indicate data that has not been modified in the source storage space after receiving the plurality of data synchronization tasks, and the historical data is obtained through the meta information; the incremental data is used to indicate data that has been modified in the source storage space after receiving the plurality of data synchronization tasks, and the incremental data is obtained through the operation log; storing data information corresponding to the plurality of tasks respectively into a message queue; the data information includes historical synchronization information, and the historical synchronization information indicates that the synchronized data is used for cascade synchronization; Reading data information respectively corresponding to the plurality of tasks from the message queue in parallel, and acquiring synchronized data from the source storage space based on the data information respectively corresponding to the plurality of tasks; Sending the synchronized data corresponding to the multiple tasks to the corresponding target storage space in parallel; The storing of the data information respectively corresponding to the plurality of tasks into the message queue comprises: storing the historical synchronization information into the message queue; The acquiring the synchronized data from the source storage space based on the data information respectively corresponding to the multiple tasks includes: acquiring the synchronized data from the source storage space according to the historical synchronization information.

2. The method according to claim 1, characterized in that The storing the data information respectively corresponding to the plurality of tasks into a message queue comprises: Encapsulating the meta-information to obtain a first encapsulated object, and sending the first encapsulated object to the message queue; The operation log is encapsulated to obtain a second encapsulation object, and the second encapsulation object is sent to the message queue.

3. The method according to claim 2, characterized in that The first encapsulated object and the second encapsulated object have the same format.

4. The method according to claim 2, characterized in that: The operation log in the data information includes historical synchronization information, and the historical synchronization information indicates that the synchronized data is used for cascade synchronization, wherein: The encapsulating the operation log to obtain a second encapsulation object includes: encapsulating the historical synchronization information to obtain a second encapsulation object; The reading the data information respectively corresponding to the plurality of tasks from the message queue in parallel includes: reading the first encapsulation object to obtain metadata of the synchronized data; Acquiring the synchronized data from the source storage space based on the data information respectively corresponding to the multiple tasks includes: downloading the synchronized data from the source storage space according to the meta-information of the synchronized data.

5. The method according to any one of claims 1 to 4, characterized in that: The acquiring the synchronized data from the source storage space based on the data information respectively corresponding to the multiple tasks includes: According to the meta information, data is downloaded from the source storage space to obtain historical data in the synchronized data.

6. The method according to claim 5, characterized in that The acquiring the synchronized data from the source storage space based on the data information respectively corresponding to the multiple tasks includes: In the case where the operation log is a modification operation type, target data is determined in the historical data, and the target data is processed according to the modification operation type to obtain incremental data in the synchronized data; In the case where the operation log is a newly added operation type, the corresponding synchronized data is obtained from the source storage space based on the data information corresponding to each task, including: downloading the data corresponding to the newly added operation type to obtain incremental data in the synchronized data.

7. The method according to any one of claims 1 to 4, characterized in that The step of sending the synchronized data corresponding to the plurality of tasks to the corresponding target storage space in parallel includes: Verifying the data content and time sequence of the synchronized data corresponding to the multiple tasks respectively; If the verification is passed, the synchronized data corresponding to the multiple tasks are sent in parallel to the corresponding target storage space.

8. The method according to claim 6, characterized in that The step of sending the synchronized data corresponding to the plurality of tasks to the corresponding target storage space in parallel includes: Verifying entity tag information carried in the meta information and / or the operation log; If the verification passes, the historical data and the incremental data are uploaded to the target storage space.

9. The method according to claim 8, characterized in that The uploading of the historical data and the incremental data to the target storage space includes: Based on the upload time of the historical data and / or the update time of the incremental data, an atomic operation is performed on the entity tag information of the synchronized data.

10. A data synchronization device, characterized in that: include: A data information determination module, used for receiving a plurality of tasks for performing data synchronization from a source storage space to a plurality of target storage spaces respectively, obtaining meta information of the synchronized data in the source storage space according to synchronized data identifiers carried in the tasks, and obtaining operation logs about the synchronized data, and determining the meta information and the operation logs as data information corresponding to the synchronized data; the synchronized data includes historical data and incremental data, the historical data is used for indicating data that has not been modified in the source storage space after receiving the plurality of data synchronization tasks, and the historical data is obtained through the meta information; the incremental data is used for indicating data that has been modified in the source storage space after receiving the plurality of data synchronization tasks, and the incremental data is obtained through the operation log; A storage module, used for storing the data information respectively corresponding to the multiple tasks in a message queue, and reading the data information respectively corresponding to the multiple tasks from the message queue in parallel; the data information includes historical synchronization information, and the historical synchronization information indicates that the synchronized data is used for cascade synchronization; A synchronized data acquisition module, used for acquiring synchronized data from the source storage space based on the data information respectively corresponding to the multiple tasks; A parallel sending module, used for sending the synchronized data corresponding to the plurality of tasks in parallel to the corresponding target storage space; The storing of the data information respectively corresponding to the plurality of tasks into the message queue comprises: storing the historical synchronization information into the message queue; The acquiring the synchronized data from the source storage space based on the data information respectively corresponding to the multiple tasks includes: acquiring the synchronized data from the source storage space according to the historical synchronization information.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the data synchronization method according to any one of claims 1 to 9 are implemented.

12. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the data synchronization method according to any one of claims 1 to 9 is implemented.

Images