Transaction-based file synchronization method and system

Abstract

Disclosed in the present invention are a transaction-based file synchronization method and system. The method comprises: acquiring a file operation record of a first device; performing conflict check on the first device and a cloud on the basis of the file operation record; and updating a file version information table, an operation record table and a cloud transaction identifier of the cloud in chronological order on the basis of the file operation record. The method comprises: acquiring the latest first transaction identifier of a cloud and a second transaction identifier, synchronized last time, of a second device; acquiring a first operation record corresponding to the first transaction identifier; performing conflict check on the cloud and the second device on the basis of the first operation record; and performing a replay operation in a file synchronization disk of the second device in chronological order on the basis of the file operation record to complete synchronization. According to embodiments of the present invention, the operation record writing is based on the transaction identifiers, so that the risk of data loss of the synchronization disk of a user can be reduced, conflict check is carried out on the device terminal and the cloud during synchronization, conflicts can be effectively prevented, and the present invention can be widely applied to the technical field of computers.

Description

Transaction-based file synchronization method and synchronization system Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a transaction-based file synchronization method and synchronization system. Background Art

[0002] With the development of cloud storage applications, the number of scenarios where files are uploaded to the cloud from different terminals is increasing. File uploads from computer devices to the cloud have been further optimized. A new synchronization disk directory can be created on the computer's local disk, eliminating the need for users to manually click upload. This allows for two-way synchronization of local files and cloud files within the synchronization disk directory, and supports multiple terminals on a single cloud. The current pre-reconstruction technology writes all operation records for files on the synchronization disk to the cloud, then retrieves incremental file records from the cloud based on the interval between the current operation time and the last operation time, and replays them on other devices, achieving synchronization between the first device and the cloud, and between the cloud and the second device, as shown in Figure 1.

[0003] This solution can lead to file conflicts. As shown in Figure 2, due to a critical value issue in the query operation record interface, when requesting the interface, the request time range starts at fromRev, which is the last change time + 1, and ends at toRev, which defaults to the current server time at the time of the request. The time interval for querying operation behaviors is fromRev <= operation time <= toRev. If some concurrent operations occur at toRev time but the operations are not yet stored, there is a possibility of omission.

[0004] Furthermore, when a new file is added to the cloud and not synchronized locally, a new file with the same name is added locally, causing a file conflict. Such conflicts will always occur if they are not pre-identified with the cloud. Currently, conflict determination in the industry is based on the modification time of metafile information or the metafile's content. Furthermore, bidirectional synchronization of files between the local and cloud often relies on changes in the MD5 value of the file's content to update the file. This also requires separate bidirectional synchronization of files and folders, resulting in low synchronization efficiency.

[0005] Summary of the Invention

[0006] The present invention aims to solve one of the technical problems in the related art at least to a certain extent. To this end, the present invention provides a transaction-based file synchronization method and synchronization system with high reliability.

[0007] In one aspect, an embodiment of the present invention provides a transaction-based file synchronization method, comprising:

[0008] Acquire a file operation record of a first device on a file synchronization disk; wherein the file operation record is transactional; and the first device is one or more;

[0009] performing a conflict check on the first device and the cloud according to the file operation record, so that the cloud returns conflict information to the first device according to a result of the conflict check;

[0010] If the conflict information is not returned, the file version information table, the operation record table and the cloud transaction identifier in the cloud are updated according to the file operation record and in chronological order.

[0011] Optionally, performing a conflict check on the first device and the cloud based on the file operation record, so that the cloud returns conflict information to the first device based on a result of the conflict check, includes:

[0012] Determining an operation type of the file operation record;

[0013] Determine the conflict situation based on the operation type and in combination with the file version information table on the cloud;

[0014] When a conflict occurs, conflict information is returned.

[0015] Optionally, determining the conflict situation based on the operation type and in combination with the file version information table on the cloud includes at least one of the following:

[0016] When the operation type is adding, the situation where the parent directory of the file to be added is empty, the original file version record does not exist, or the parent folder is deleted on the cloud is determined as a conflict situation;

[0017] When the operation type is update, the situation where the parent directory of the file to be updated is empty, the original file version record does not exist, or the cloud version number is greater than the current version number is determined as a conflict situation;

[0018] When the operation type is rename, the situation where the parent directory of the file to be renamed is empty, the parent folder is deleted in the cloud, the version record of the renamed file exists, the original file version does not exist, or the cloud version number is greater than the current version number is determined as a conflict situation;

[0019] When the operation type is move, a situation where the file version after the move exists but the file version before the move does not exist or the cloud version number is greater than the current version number is determined as a conflict situation;

[0020] When the operation type is restoration, the situation where the original file version of the file to be restored does not exist is determined as a conflict situation.

[0021] Optionally, if the conflict information is not returned, updating the file version information table, operation record table and cloud transaction identifier in the cloud according to the file operation record in a time sequence, including:

[0022] Uploading the file operation records to the cloud through a message queue in a time sequence; wherein the file operation records of the same first device are uploaded through the same message queue;

[0023] Incrementally processing the first cloud transaction identifier according to the file operation record to obtain a second cloud transaction identifier;

[0024] storing the file operation record in an operation record table of a cloud database according to the second cloud transaction identifier;

[0025] Each time a file operation record is stored, the file version corresponding to the file operation record in the file version information table is incremented; wherein, in the case of concurrency, a database row lock is added to the version field of the file version information table.

[0026] On the other hand, an embodiment of the present invention further provides a transaction-based file synchronization method, comprising:

[0027] Obtain the latest first transaction identifier on the cloud and the second transaction identifier of the previous synchronization with the second device;

[0028] When the first transaction identifier is greater than the second transaction identifier, obtaining a first operation record corresponding to the first transaction identifier; wherein the first operation record is stored in the cloud; and the first operation record is transactional;

[0029] Performing a conflict check between the cloud and the second device according to the first operation record;

[0030] When no conflict information is returned, a replay operation is performed in the file synchronization disk of the second device according to the first operation record and in a time sequence to complete synchronization from the cloud to the second device;

[0031] Update the second transaction identifier of the second device according to the first transaction identifier.

[0032] Optionally, performing a conflict check on the cloud and the second device according to the first operation record includes:

[0033] determining an operation type of the first operation record;

[0034] Determine the conflict situation based on the operation type and in combination with the file version information table on the cloud;

[0035] When a conflict occurs, conflict information is returned.

[0036] Optionally, when no conflict information is returned, performing a replay operation in the file synchronization disk of the second device according to the first operation record and in a time sequence to complete synchronization from the cloud to the second device includes:

[0037] Sending the first operation record to the second device through a message queue according to a time sequence;

[0038] According to the first operation record, a replay operation is performed on the files and folders of the file synchronization disk of the second device to complete synchronization from the cloud to the second device.

[0039] On the other hand, an embodiment of the present invention further provides a transaction-based file synchronization system, including:

[0040] File synchronization disk, used to store files and folders that need to be synchronized;

[0041] A synchronization service module, configured to execute the transaction-based file synchronization method of one aspect described above; and also configured to execute the transaction-based file synchronization method of another aspect described above;

[0042] The database module is used to store the operation record table and the file version information table.

[0043] It should be noted that, in some embodiments, the system may further include the following modules:

[0044] The initialization module is used to initialize the file synchronization disk.

[0045] On the other hand, an embodiment of the present invention further provides an electronic device, comprising: a processor and a memory; the memory is used to store programs; and the processor executes the program to implement the method described above.

[0046] On the other hand, an embodiment of the present invention further provides a computer storage medium storing a program executable by a processor. The program executable by the processor is used to implement the method described above when executed by the processor.

[0047] On the other hand, an embodiment of the present invention also provides a computer program product or computer program, which includes computer instructions, and the computer instructions are stored in the computer-readable storage medium mentioned above; the processor of the computer device mentioned above can read the computer instructions from the computer-readable storage medium mentioned above, and the processor executes the computer instructions, so that the computer device executes the transaction-based file synchronization method mentioned above.

[0048] The embodiments of the present invention have the following beneficial effects: the file operation records of the embodiments of the present invention are written in time sequence based on transaction identifiers, and the file operation records saved to the cloud and replayed to the second device can be synchronized with the operation sequence of the first device in operating the synchronization disk, and no operation record behavior will be discarded. The file version information is recorded, which can greatly reduce the risk of data loss on the user's synchronization disk, and conflict checks are performed on the device terminal and the cloud during synchronization, which can effectively prevent conflicts from occurring. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the technical solution of the present invention and do not constitute a limitation to the technical solution of the present invention.

[0050] FIG1 is a timing diagram of an existing file synchronization solution provided by an embodiment of the present invention;

[0051] FIG2 is a schematic diagram of conflicts in an existing file synchronization solution provided by an embodiment of the present invention;

[0052] FIG3 is a schematic diagram of synchronization between multiple terminals and a cloud according to an embodiment of the present invention;

[0053] FIG4 is a diagram showing the steps of a transaction-based file synchronization method according to an aspect of an embodiment of the present invention;

[0054] FIG5 is a flowchart of synchronizing from a first device to the cloud according to an embodiment of the present invention;

[0055] FIG6 is a flow chart of a conflict check according to an embodiment of the present invention;

[0056] 7 is a step diagram of a transaction-based file synchronization method provided by another embodiment of the present invention;

[0057] FIG8 is a flowchart of the synchronization disk initialization according to an embodiment of the present invention;

[0058] FIG9 is a flowchart of bidirectional synchronization between a terminal and a cloud provided by an embodiment of the present invention;

[0059] 10 is a schematic structural diagram of a synchronization service module according to an embodiment of the present invention;

[0060] 11 is a schematic diagram of a transaction-based file synchronization server and terminal module according to an embodiment of the present invention;

[0061] FIG12 is a schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0063] It should be noted that although the system diagrams illustrate functional module divisions and the flowcharts illustrate a logical sequence, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the system or the sequence in the flowcharts. The terms "first / S100," "second / S200," and the like in the specification, claims, and drawings are used to distinguish similar objects and are not necessarily intended to describe a specific sequence or precedence.

[0064] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0065] The following are some of the glossaries involved in the present invention:

[0066] File synchronization disk: The file synchronization disk can be a new synchronization disk directory created on the local disk of the terminal. The files in this directory do not need to be manually uploaded by the user. It is used to realize two-way synchronization of local files and cloud files in the synchronization disk directory. In an embodiment of the present invention, the file synchronization disks of multiple terminals can share a synchronization disk ID. The synchronization disk ID is configured in the cloud, which can support two-way synchronization between one cloud and multiple terminal devices.

[0067] File operation record: A file operation record refers to a record generated by performing file operations on a single file or folder in the file synchronization disk, such as creating, deleting, updating, etc. In an embodiment of the present invention, the file operation record is transactional.

[0068] Transaction: A transaction in the present invention refers to a database transaction, which is a sequence of database operations that access and potentially manipulate various data items. These operations are either all executed or none of them, forming an indivisible unit of work. A transaction consists of all database operations performed between the start and end of the transaction. In the present invention, file operation records are transactional, and a single file operation record can correspond to multiple operations.

[0069] A database row lock is a lock that locks a row of data in a database table to control concurrent access and modification to that row. Row locks prevent multiple transactions from modifying the same row of data simultaneously, thereby ensuring data consistency and integrity.

[0070] Kafka: Kafka is an open source message queue technology framework, mainly used to process message queues in large data states.

[0071] Redis (Remote Dictionary Service): Redis is a message queue and an open-source tool for memory caching. In this embodiment, a single user's operation records are placed in the same Redis queue. Using the Redis queue for multi-threaded processing ensures that user operations are written to the synchronization disk operation record table according to time sequence.

[0072] API service: API service is the basic server-side interface service.

[0073] It should be noted that the "first device" described in the present invention refers to the party representing the terminal in the process of synchronizing from the terminal to the cloud, and the "second device" refers to the party representing the terminal in the process of synchronizing from the cloud to the terminal. The first device and the second device can be one or more. In some embodiments, the first device and the second device can be the same terminal. For ease of understanding, the present invention is introduced with the terminal as the first device or the second device. In other embodiments, the first device or the second device can also be a personal computer, a server computer, a handheld device or a portable device, a tablet device, a multi-processor system, a microprocessor-based system, a set-top box, a programmable consumer electronic device, a network PC, a minicomputer, a mainframe computer, etc.

[0074] Unless otherwise specified, in some descriptions of the present invention, when the noun "file" appears alone, it may include files or folders. "Identifier" refers to the "id" commonly used in databases. Different data items may have different identifiers, i.e., ids.

[0075] In response to at least one problem existing in the prior art, an embodiment of the present invention proposes a transaction-based file synchronization method and synchronization system. Referring to Figure 3, Figure 3 is a schematic diagram of the synchronization of multiple terminals and a cloud provided by an embodiment of the present invention. The method and system can achieve reliable synchronization of multiple terminals and a cloud based on file version information and transaction operation records.

[0076] The following first introduces a transaction-based file synchronization method according to an embodiment of the present invention.

[0077] One aspect of an embodiment of the present invention provides a transaction-based file synchronization method. Referring to Figures 4 and 5, Figure 4 is a step diagram of the transaction-based file synchronization method provided by one aspect of an embodiment of the present invention, and Figure 5 is a flowchart of synchronization from a first device to the cloud provided by an embodiment of the present invention. The method may include but is not limited to the following steps S100 to S300.

[0078] S100: Obtain a file operation record of a first device on a file synchronization disk; wherein the file operation record is transactional; and there are one or more first devices.

[0079] Specifically, the first device operates on files or folders in the local file synchronization disk to generate file operation records, which include at least one of creating a folder, renaming a file / folder, deleting a file / folder, moving a file / folder, and uploading a file.

[0080] S200: Perform a conflict check on the first device and the cloud according to the file operation record, so that the cloud returns conflict information to the first device according to the result of the conflict check.

[0081] First, determine the operation type of the file operation record. The operation type is distinguished according to the operations that the first device can perform on the files in the synchronized disk. According to the operation type, combined with the file version information table in the cloud, determine the conflict situation. When a conflict occurs, return the conflict information.

[0082] First, determine the operation type of the file operation record, and the operation type is distinguished according to the operations that the first device can perform on the files in the synchronization disk; based on the operation type, combined with the file version information table in the cloud, determine the conflict situation; when a conflict occurs, return conflict information.

[0083] More specifically, referring to FIG. 6 , which is a flowchart of a conflict check according to an embodiment of the present invention, various situations involved in step S200 may be handled in at least one of the following ways:

[0084] (1) When the operation type is adding, the situation where the parent directory of the file to be added is empty, the original file version record does not exist, or the parent folder is deleted in the cloud is determined as a conflict situation;

[0085] (2) When the operation type is update, the situation where the parent directory of the file to be updated is empty, the original file version record does not exist, or the cloud version number is greater than the current version number is determined as a conflict situation;

[0086] (3) When the operation type is renaming, the situation where the parent directory of the file to be renamed is empty, the parent folder is deleted in the cloud, the version record of the renamed file exists, the original file version does not exist, or the cloud version number is greater than the current version number is determined as a conflict situation;

[0087] (4) When the operation type is move, a situation where the file version after the move exists, the file version before the move does not exist, or the cloud version number is greater than the current version number is determined as a conflict situation;

[0088] (5) When the operation type is restoration, the situation where the original file version of the file to be restored does not exist is determined as a conflict situation.

[0089] It is understandable that the conflict checking process of the present invention is mainly based on comparing the file version information after the operation with the file version information table to determine whether a conflict occurs. This conflict checking method provides independent conflict judgment and resolution logic. Unlike the original conflict judgment method that only focuses on changes in the file content itself (file operation time or file content), it mainly relies on whether the original folder information (folder ID, etc.) exists, whether the original version record information exists, and whether the original version record version number is greater than the version number of the current operation file to perform conflict judgment. It can be effectively applied to the file synchronization method of the present invention.

[0090] S300: If the conflict information is not returned, the file version information table, operation record table and cloud transaction ID in the cloud are updated according to the file operation record and in chronological order.

[0091] The file version information table of the present invention is stored in the cloud, and can be specifically constructed with reference to the file version meta-information table in Figure 3, which includes data items such as synchronization disk id (repo_id), file path (file_path), file name (file_name), md5 value, file version number (version), etc.

[0092] The operation record table can be constructed with reference to the operation record table in FIG3 , which includes data items such as synchronization disk ID (repo_id), transaction ID (trans_id), operation type (op_type), etc.

[0093] In an embodiment of the present invention, when multiple users, that is, multiple first device terminals need to be synchronized, the synchronization disk ID is configured as the same ID number. For example, it can be applied to synchronize files of different devices (mobile terminals, terminals, etc.) of the same user.

[0094] Specifically, if no conflicts are returned, a unified operation record entry is generated and a message queue containing the file operation records is sent to the cloud in a time-sequential manner. The operation type is determined based on the specific operation. This message queue can use a Kafka message queue, with the user's unique identifier, userId, distinguishing the queues. By assigning userIds to different Redis queues, this ensures that individual users consume in order, thereby ensuring that individual user file operation records are stored in order.

[0095] According to the operation record message queue, query whether the operated file already exists in the file version information table. If so, update the file version, file version + 1, and add a row lock to the database row of the file version information in the database during the update process to ensure data consistency and stability; if not, add a new file version record and set the initial version number. In some embodiments, the initial version number can be set to 1.

[0096] The file operation record is stored in the operation record table, and the cloud transaction ID is incremented by +1 and stored in the database operation record table. The increment process can be performed using the Redis atomic increase command. In Redis, the increase command is used to automatically increment a field.

[0097] Based on this, step S300 may include the following steps S310 to S330.

[0098] S310. Upload the file operation records to the cloud through a message queue according to a time sequence; wherein the file operation records of the same first device are uploaded through the same message queue.

[0099] S320. Increment the first cloud transaction ID according to the file operation record to obtain a second cloud transaction ID.

[0100] S330. According to the second cloud transaction ID, store the file operation record in an operation record table of a cloud database.

[0101] S340. Every time a file operation record is stored, the file version corresponding to the file operation record in the file version information table is incremented; wherein, when concurrent, a database row lock is added to the version field of the file version information table.

[0102] During the process of synchronizing the first device to the cloud, if there is an old file synchronization disk, the old synchronization disk write logic is retained, and the file operation record is also written to the old synchronization disk and saved in the database. After iterating for a period of time in the first device, the data is stripped. This allows the embodiment of the present invention to be compatible with the logical functions of the old synchronization disk.

[0103] Another aspect of the embodiment of the present invention further provides a transaction-based file synchronization method. Referring to FIG. 7 , FIG. 7 is a step diagram of the transaction-based file synchronization method provided by another aspect of the embodiment of the present invention. The method includes the following steps S400 to S800 .

[0104] S400: Obtain the latest first transaction ID on the cloud and the second transaction ID of the previous synchronization with the second device.

[0105] S500. When the first transaction ID is greater than the second transaction ID, obtain a first operation record corresponding to the first transaction ID; wherein the first operation record is stored in the cloud; and the first operation record is transactional.

[0106] The first transaction ID is compared with the second transaction ID. If the first transaction ID is greater than the second transaction ID, it indicates that another terminal has updated the file synchronization disk, and therefore synchronization from the cloud to the second device is required. The first transaction ID is stored in the operation record table of the local database of the second device, and the second transaction ID is stored in the operation record table of the cloud database.

[0107] S600: Perform a conflict check between the cloud and the second device according to the first operation record.

[0108] First, the operation type of the first operation record is determined; based on the operation type and in combination with the file version information table in the cloud, the conflict situation is determined; if a conflict occurs, conflict information is returned.

[0109] For a specific implementation, reference may be made to FIG. 6 and the introduction to step S200 , in which a conflict check is performed based on the file version information according to the operation type of the first operation record, and details thereof will not be repeated here.

[0110] S700: When no conflict information is returned, a replay operation is performed in the file synchronization disk of the second device according to the first operation record and in a time sequence to complete synchronization from the cloud to the second device.

[0111] Specifically, the first operation record and related files are sent to the second device through the message queue in a time sequence; according to the first operation record, the files and folders of the file synchronization disk of the second device are replayed to complete the synchronization from the cloud to the second device.

[0112] Replay refers to the process of restoring data during database failure recovery by replaying the operation records in the log file. Replay is a data recovery technology that helps administrators quickly restore data and maintain data consistency after a database failure. In this invention, the replay operation uses the transaction-based file operation records in the cloud as the basis for restoring data on the second device. This allows the files on the file synchronization disk on the second device to be processed in the same order as the operations on the first device, achieving synchronization.

[0113] S800: Update the second transaction ID of the second device according to the first transaction ID.

[0114] Specifically, the first transaction ID may be assigned to the second transaction ID, thereby updating the second transaction ID of the second device.

[0115] In some embodiments, the file synchronization disk may be initialized before being used. As shown in FIG8 , FIG8 is a flowchart of the synchronization disk initialization according to an embodiment of the present invention. The specific initialization process may be:

[0116] Taking the scenario of syncing between a PC client and the cloud as an example, the PC client initializes the synchronization disk, selects a local folder, and returns the initialization task ID if the setup is successful. An asynchronous operation is performed, carrying the initialization task ID to query whether a synchronization disk already exists in the cloud. If a synchronization disk exists in the cloud, the task ID is used as the key to set the synchronization disk cache information, and the PC client waits for the next call to obtain the synchronization disk information. If a synchronization disk does not exist in the cloud, multi-threaded batch saving is set to the uploaded file version information in the local folder of the synchronization disk, the initial version is set to 1, and an initial synchronization disk operation record is written. The operation type is initialization, the initial transaction ID is 0, and the cloud synchronization disk information is added. The task ID is used as the key to set the synchronization disk cache information, and the PC client waits for the next call to obtain the synchronization disk information. If initialization is complete, the cache value is set to the synchronization value information; if initialization is not complete, the cache value is set to initialization in progress.

[0117] In the case of the old synchronization disk, the PC client queries the result of initializing the synchronization disk, and queries the cloud cache information of the synchronization disk through the synchronization disk cache information. If the cloud cache information is queried, the latest cloud transaction ID is obtained from the database, and the cloud synchronization disk record database table is queried to obtain the file list information of the cloud synchronization disk, and the cloud files are downloaded to the local computer to record the operation; the latest cloud transaction ID is set as the local transaction ID.

[0118] In some embodiments, bidirectional synchronization between the terminal and the cloud can be performed. Referring to FIG9 , which is a flow chart of bidirectional synchronization between the terminal and the cloud provided in an embodiment of the present invention, the synchronization process can be triggered regularly or irregularly, and corresponding synchronization conditions can also be configured so that bidirectional synchronization is performed when the synchronization conditions are met. During the bidirectional synchronization between the terminal and the cloud, steps S100 to S300 and steps S400 to S800 are executed simultaneously. Since they contain conflict handling steps, the bidirectional data without conflict after detection can be merged.

[0119] In some embodiments, referring to FIG3 , multiple terminals can be synchronized with a cloud, for example, multiple terminals of a user, or multiple terminals of different users, to synchronize files through the cloud.

[0120] On the other hand, an embodiment of the present invention provides a transaction-based file synchronization system, as shown in FIG10 . FIG10 is a schematic diagram of the structure of a synchronization service module provided by an embodiment of the present invention, including:

[0121] The file synchronization disk is used to store files and folders that need to be synchronized. Specifically, the file synchronization disk is configured on the terminal and the cloud, and a synchronization disk ID is configured to distinguish different synchronization disks. This can be done by creating a new synchronization disk directory on the local disk of the computer.

[0122] The synchronization service module is used to execute the transaction-based file synchronization method of one aspect described above; and is also used to execute the transaction-based file synchronization method of another aspect described above.

[0123] A database module is used to store an operation log table and a file version information table. The database module can include a cloud database and a local terminal database; both the cloud database and the terminal database can store corresponding file version information tables and operation log tables, wherein the terminal database stores the file version information and operation records of the terminal file synchronization disk.

[0124] It should be noted that, in some embodiments, the system may further include the following modules:

[0125] The initialization module is used to initialize the file synchronization disk.

[0126] Furthermore, the synchronization service module of the transaction-based file synchronization system of the present invention is further introduced as follows:

[0127] Refer to Figure 11, which is a schematic diagram of a transaction-based file synchronization server and terminal module provided by an embodiment of the present invention. The present invention adds a synchronization disk terminal synchronization module to the original old synchronization disk API service, and adds a file synchronization disk operation timing processing module and a synchronization disk microservice. The synchronization disk microservice includes a synchronization disk recording module, a synchronization disk operation management module and a synchronization disk file recording module.

[0128] The functions of each module are described as follows:

[0129] The Synchronous Disk terminal synchronization module includes a basic file creation, deletion, and modification interface. This interface expands upon existing file management capabilities by adding Synchronous Disk conflict detection and providing new file management capabilities, such as creating folders, renaming, deleting, moving, and uploading files. The intermediate logic also includes conflict detection and handling returned by the Synchronous Disk microservice.

[0130] Synchronous disk operation timing processing module: Add a Kafka queue partitioned by a single user dimension. To increase consumption speed, add Redis queue multi-threaded processing and place all single user operation records in the same Redis queue for processing based on userId, ensuring that user operations are written to the synchronous disk operation record table according to time sequence.

[0131] Synchronous Disk Recording Module: This module provides the ability to asynchronously configure the synchronous disk, query the synchronization disk configuration results, and query the synchronization disk information. If an existing synchronous disk system is used, the new synchronous disk system of this invention needs to be compatible with the synchronization history file. Given the time required to synchronize historical files, asynchronous synchronization can be used.

[0132] Synchronous Disk Operation Management Module: This module primarily manages the synchronization disk file operation history, storing synchronization disk file operation records and maintaining incremental transaction IDs for these records. Furthermore, this module provides the ability to query the latest synchronization disk transaction ID, query incremental synchronization disk operation records, write synchronization disk file operation records, and identify synchronization disk operation conflicts in various scenarios.

[0133] Synchronous disk file record module: used to add or update records in the file version information table. When updating, it uses database table row locks to maintain the version field of the file information version to solve problems caused by concurrent operations.

[0134] On the other hand, an embodiment of the present invention further provides an electronic device, as shown in Figure 12. Figure 12 is a schematic diagram of an electronic device provided by an embodiment of the present invention, including: a processor and a memory; the memory is used to store programs; the processor executes the program to implement the method described above.

[0135] On the other hand, an embodiment of the present invention further provides a computer storage medium storing a program executable by a processor. The program executable by the processor is used to implement the method described above when executed by the processor.

[0136] On the other hand, an embodiment of the present invention also provides a computer program product or computer program, which includes computer instructions, and the computer instructions are stored in the computer-readable storage medium mentioned above; the processor of the computer device mentioned above can read the computer instructions from the computer-readable storage medium mentioned above, and the processor executes the computer instructions, so that the computer device executes the transaction-based file synchronization method mentioned above.

[0137] The embodiments of the present invention have the following beneficial effects:

[0138] 1. Because operation records are written based on transaction IDs, the risk of data loss on the user's sync disk is reduced. This transaction ID is automatically atomically incremented by the program based on a number, generated using the Redis increase command. For a single user on the sync disk, each user operation causes the transaction ID to increase. Unlike the MD5 unique identifier method for files, this method targets a single file.

[0139] 2. File version record updates: The file version field version+1 operation is added with a database row lock to reduce the impact of concurrency on the synchronization function of the synchronization disk file, and improve the situation where file version record information errors caused by concurrency.

[0140] 3. Provides independent conflict judgment and resolution logic. Unlike existing conflict judgment methods in the industry that focus on changes in file content itself (file operation time or file content), this embodiment of the present invention relies on whether the original folder ID exists, whether the original file version record information exists, and whether the original file version record version number is greater than the version number of the current operation file to determine conflicts.

[0141] 4. The embodiment of the present invention focuses on the storage and replay of unified operation records of files and folders. The priority of operation records is to keep synchronization according to the user's operation order, and no operation record behavior will be discarded. The synchronization processing method for files and folders is unified. There is no need to separate files and folders separately, nor to pay attention to the download process. The synchronization disk function only needs to maintain the management of file operation record behavior (based on Kafka's single user dimension partition queue to ensure the producer message order, and based on the first-in-first-out principle of Redis queue to ensure the consumer message order consumption), the addition and update of file version record table, and the management of the relationship between users and synchronization disks. And the file operation records are ordered and atomic.

[0142] 5. For folder operations, the server (cloud) only records the current level of operation records and version information management, and the terminal maintains the update of sub-files (deletion, movement, and renaming based on path fuzzy matching). This optimizes the resource usage of the server.

[0143] An application example of an embodiment of the present invention is described below:

[0144] Construct a transaction-based file synchronization system, initialize the file synchronization disk, synchronize the terminal to the cloud at regular intervals, and obtain the file operation record of the first device on the file synchronization disk; wherein the file operation record is transactional; the first device is one or more; based on the file operation record, perform a conflict check on the first device and the cloud, so that the cloud returns conflict information to the first device based on the result of the conflict check; if no conflict is returned, upload the file operation record to the cloud through the message queue in a time sequence; store the file operation record in the operation record table of the cloud database; each time a unit of file operation record is stored, perform an incremental process on the file version and cloud transaction ID corresponding to the file operation record in the file version information table; wherein, in the case of concurrency, add a database row lock to the version field of the file version information table.

[0145] The cloud synchronizes with the terminal to obtain the latest first transaction ID on the cloud and the second transaction ID of the second device last synchronized; when the first transaction ID is greater than the second transaction ID, obtain the first operation record corresponding to the first transaction ID; wherein, the first operation record is stored in the cloud; the first operation record is transactional; according to the first operation record, a conflict check is performed on the cloud and the second device; when no conflict information is returned, according to the first operation record, a replay operation is performed in the file synchronization disk of the second device in accordance with the time sequence to complete the synchronization from the cloud to the second device; according to the first transaction ID, the second transaction ID of the second device is updated.

[0146] When performing a conflict check, first determine the operation type of the file operation record; based on the operation type, determine the conflict situation in combination with the file version information table on the cloud; when a conflict occurs, return conflict information. Specifically, when the operation type is adding, the situation where the parent directory of the file to be added is empty, the original file version record does not exist, or the cloud deletes the parent folder is determined as a conflict situation; when the operation type is updating, the situation where the parent directory of the file to be updated is empty, the original file version record does not exist, or the cloud version number is greater than the current version number is determined as a conflict situation; when the operation type is renaming, the situation where the parent directory of the file to be renamed is empty, the cloud deletes the parent folder, the renamed file version record exists, the original file version does not exist, or the cloud version number is greater than the current version number is determined as a conflict situation; when the operation type is moving, the situation where the file version after moving of the file to be moved exists, the file version before moving does not exist, or the cloud version number is greater than the current version number is determined as a conflict situation; when the operation type is restoring, the situation where the original file version of the file to be restored does not exist is determined as a conflict situation.

[0147] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operation and logic flow presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.

[0148] Furthermore, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise indicated, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It will also be understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present invention. More specifically, given the properties, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be understood within the ordinary skill of an engineer. Therefore, a person skilled in the art using ordinary skill will be able to implement the present invention set forth in the claims without undue experimentation. It will also be understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0149] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0150] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0151] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0152] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0153] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0154] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

[0155] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A transaction-based file synchronization method, characterized in that: include: Acquire a file operation record of a first device on a file synchronization disk; wherein the file operation record is transactional; and the first device is one or more; Performing a conflict check on the first device and the cloud according to the file operation record, so that the cloud returns conflict information to the first device according to a result of the conflict check; If the conflict information is not returned, the file version information table, the operation record table and the cloud transaction identifier in the cloud are updated according to the file operation record in a time sequence.

2. A transaction-based file synchronization method according to claim 1, characterized in that: The performing a conflict check on the first device and the cloud according to the file operation record, so that the cloud returns conflict information to the first device according to a result of the conflict check, includes: Determine the operation type of the file operation record; Determine the conflict situation according to the operation type and in combination with the file version information table on the cloud; When a conflict occurs, conflict information is returned.

3. A transaction-based file synchronization method according to claim 2, characterized in that: The determining of the conflict situation according to the operation type and in combination with the file version information table on the cloud includes at least one of the following: When the operation type is adding, the situation where the parent directory of the file to be added is empty, the original file version record does not exist, or the parent folder is deleted in the cloud is determined as a conflict situation; When the operation type is update, the situation where the parent directory of the file to be updated is empty, the original file version record does not exist, or the cloud version number is greater than the current version number is determined as a conflict situation; When the operation type is renaming, the situation where the parent directory of the file to be renamed is empty, the parent folder is deleted in the cloud, the version record of the renamed file exists, the original file version does not exist, or the cloud version number is greater than the current version number is determined as a conflict situation; When the operation type is move, a situation where a file version of the file to be moved exists after the move, a file version before the move does not exist, or a cloud version number is greater than a current version number is determined as a conflict situation; When the operation type is restoration, the situation that the original file version of the file to be restored does not exist is determined as a conflict situation.

4. The transaction-based file synchronization method according to claim 1, characterized in that: If the conflict information is not returned, the file version information table, the operation record table and the cloud transaction identifier are updated in the cloud according to the file operation record in a time sequence, including: Uploading the file operation records to the cloud through a message queue in a time sequence; wherein the file operation records of the same first device are uploaded through the same message queue; According to the file operation record, the first cloud transaction identifier is incremented to obtain a second cloud transaction identifier; According to the second cloud transaction identifier, storing the file operation record in an operation record table of a cloud database; Each time a file operation record is stored, the file version corresponding to the file operation record in the file version information table is incremented; wherein, in the case of concurrency, a database row lock is added to the version field of the file version information table.

5. A transaction-based file synchronization method, characterized in that: include: Obtain the latest first transaction identifier on the cloud and the second transaction identifier of the last synchronization with the second device; When the first transaction identifier is greater than the second transaction identifier, obtaining a first operation record corresponding to the first transaction identifier; wherein the first operation record is stored in the cloud; and the first operation record is transactional; Performing a conflict check between the cloud and the second device according to the first operation record; When no conflict information is returned, a replay operation is performed in the file synchronization disk of the second device according to the first operation record and in a time sequence to complete synchronization from the cloud to the second device; According to the first transaction identifier, a second transaction identifier of the second device is updated.

6. A transaction-based file synchronization method according to claim 5, characterized in that: The performing a conflict check on the cloud and the second device according to the first operation record includes: Determining an operation type of the first operation record; Determine the conflict situation according to the operation type and in combination with the file version information table on the cloud; When a conflict occurs, conflict information is returned.

7. A transaction-based file synchronization method according to claim 5, characterized in that: When no conflict information is returned, replaying the file synchronization disk of the second device according to the first operation record and in a time sequence to complete synchronization from the cloud to the second device includes: Sending the first operation record to the second device through a message queue according to a time sequence; According to the first operation record, a playback operation is performed on the files and folders of the file synchronization disk of the second device to complete synchronization from the cloud to the second device.

8. A transaction-based file synchronization system, characterized in that: include: File synchronization disk, used to store files and folders that need to be synchronized; A synchronization service module, configured to execute the transaction-based file synchronization method as claimed in claim 1; Also used to perform the transaction-based file synchronization method as described in claim 5; The database module is used to store the operation record table and the file version information table.

9. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 7.

10. A computer storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to implement the method according to any one of claims 1 to 7 when executed by the processor.

Images