City-wide active-active file service file synchronization processing method

Abstract

The invention discloses an intra-city dual-active file service file synchronization processing method, and particularly relates to the field of data storage, and the method comprises the following steps: after a user uploads a file to an A center for storage, a system analyzes a file attribute, generates a hash value and records the hash value in a file service of the A center; the center A tries to be connected with the file service of the center B, and if connection succeeds, whether the file exists in the center B or not is judged by comparing the hash value of the file; if not, the center A needs to obtain an exclusive synchronous lock in a memory; if the file is successfully transmitted, releasing the synchronization lock and ending the transaction, and writing back the synchronization state to the A center; if the transmission fails, entering a B center retry queue; and entering an A center retry queue if connection failure or lock acquisition failure occurs. Finally, the dual-active service can query the file in any center, preferentially obtains the file nearby, and queries the other center if the file is not hit. Through the distributed lock mechanism and the fault-tolerant design of the synchronous task queue, the recovery capability of sudden network interruption and system faults is enhanced.

Description

Technical Field

[0001] This invention relates to the field of data storage technology, and more specifically, to a method for file synchronization processing in a local active-active file service. Background Technology

[0002] Currently, active-active data center technology plays an increasingly important role in ensuring the continuity of critical business operations, especially in scenarios requiring disaster recovery, high availability, and high consistency. Traditional active-active storage solutions, such as Huawei HyperMetro, EMCVplex, and IBM SVC, typically involve building storage clusters in two separate locations and using synchronization mechanisms to ensure data consistency between the primary and backup data centers. These technologies are mostly implemented at the underlying storage device level, providing strong consistency and data redundancy guarantees, along with inter-cluster redundancy backup mechanisms. However, these existing technologies still face significant technical bottlenecks when deployed in different locations, primarily due to latency, jitter, and instability caused by long-distance network transmission. Especially in active-active storage architectures, the real-time requirements between nodes are high, but network conditions between different locations often cannot meet millisecond-level synchronization requirements, easily leading to synchronization delays, inconsistent data versions, and data write conflicts, further causing file access failures or system service anomalies.

[0003] Current solutions often employ a primary-backup deployment architecture, where the primary data center handles the main workload, while the backup data center acts as a disaster recovery node, switching over to continue operation only after the primary center fails. However, this model inherently suffers from switching latency and service interruptions, making it difficult to meet the requirements of high availability and zero data loss. Furthermore, existing technologies suffer from inadequate file redundancy mechanisms, primarily focusing on redundancy only for the file data itself, lacking synchronization and sharing of metadata such as file attributes and access records. This leads to the inability to accurately locate or retrieve target files in abnormal situations. In extreme cases of network outages or storage node failures, a "split-brain" phenomenon can easily occur, where both storage arrays believe themselves to be the primary node, providing independent services and ultimately resulting in data version conflicts and consistency imbalances.

[0004] Therefore, there is an urgent need for a remote file synchronization method with autonomous control capabilities and low latency. Summary of the Invention

[0005] To overcome the aforementioned deficiencies in existing technologies, this invention provides a file synchronization processing method for active-active file services within the same city. This method involves designing a file upload and parsing process, a metadata and hash value management mechanism, and a synchronization process between centers A and B that prioritizes high consistency. Furthermore, it proposes a memory-based exclusive lock mechanism to ensure mutually exclusive execution of synchronous transactions in a concurrent environment, avoiding data version conflicts and the "split-brain" phenomenon, thereby addressing the problems mentioned in the background technology.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A method for file synchronization in a local active-active file service includes the following steps: S1, users upload files to center A through the system.

[0007] S2, the file is stored in the central file storage space of A.

[0008] S3 parses the file, generating file attribute information and hash values.

[0009] S4 writes the file attribute information and hash value records to the file service in center A.

[0010] S5, File Service A begins attempting to connect to File Service B.

[0011] S6. If the connection is successful, proceed to the next step. If the connection fails, save the file synchronization task to the synchronization queue in center A and wait for a retry.

[0012] S7: Determine whether the file exists in center B based on the file hash value. If it exists, mark the file synchronization as complete and end the process; otherwise, proceed to S8.

[0013] S8 proposes a memory-based exclusive lock mechanism and acquires a synchronization lock by checking the lock type in the lock structure of the data object in memory.

[0014] S9. If the synchronization lock is successfully acquired, proceed to S10. If the synchronization lock is not acquired, wait for a retry.

[0015] S10, Start synchronizing transactions.

[0016] S11 will save the file records synchronized from center A to the file service in center B.

[0017] S12, synchronize the file from center A to the file service storage space of center B.

[0018] S13. If S12 synchronization is successful, proceed to the next step. If synchronization fails, save to the synchronization queue in center B and wait for retry.

[0019] S14, release the synchronization lock.

[0020] S15, file synchronization complete, transaction terminated.

[0021] S16, write back the synchronization status of center A.

[0022] S17, the dual-active service can see the complete corresponding files in both centers.

[0023] As a further aspect of the present invention, S1, the user uploads a file to center A through the system, including the following specific content: The user performs a file upload operation through the system front-end interface. The user can access the file service interface through various methods such as browser clients, enterprise clients, or mobile terminals, and select the file to be uploaded. After the file upload request is generated on the client, it is sent to the file service of center A, carrying the content of the file to be uploaded and relevant request parameters (such as identity authentication information, access tokens, etc.) in the form of HTTP, FTP, or a custom data transmission protocol.

[0024] As a further aspect of this invention, S2, storing the file in the file storage space of Center A includes the following specific steps: After receiving a user's upload request, the file service at Center A first performs security verification, permission verification, and parameter checks on the request to confirm the legality of the uploaded file. After successful verification, the file service calls the underlying file storage interface (such as a distributed file system, object storage system, or NAS storage) to write the received file into the dedicated file storage space of Center A in a streaming manner. The storage process may involve mechanisms such as sharding and multi-copy storage to ensure the security and reliability of the file data. During this process, the specific location of the file storage (e.g., storage path, physical address, or logical address of the storage space) is recorded for subsequent file access and synchronization.

[0025] As a further aspect of this invention, S3 involves parsing the file and generating file attribute information and a hash value, including the following specific steps: After the file service completes storage, the file parsing module is immediately invoked to parse the newly uploaded file, including the automatic extraction of file metadata. The file parsing module first obtains basic file attributes through the file system interface, including the filename, file type identified by the extension, file size in bytes, and physical or logical path in the storage system. Simultaneously, the system automatically records relevant information about this file upload operation, including the identity of the uploading user and the system timestamp of the upload. Furthermore, to ensure data consistency across different locations and to facilitate rapid deduplication, the file service uses common hash algorithms (such as MD5 and SHA-256) to generate a unique hash value for the file content, which is used for subsequent synchronization checks.

[0026] As a further aspect of the present invention, S4, the file attribute information and hash value records are written into the A-center file service, including the following specific content: the parsed file attribute information and hash value information are uniformly recorded in the metadata record database or metadata management module of the A-center file service. The metadata record database can maintain and query the metadata of each file in a high-speed and efficient manner, and provides an indexing mechanism for quick location and search.

[0027] As a further aspect of the present invention, in step S5, file service A initiates an attempt to connect to file service B, including the following specific steps: After completing the above steps, to ensure high consistency of file data, file service A activates the synchronization service module and actively establishes a network communication connection with file service B. This connection process may employ a standard TCP connection method, or it may use a secure VPN tunnel or dedicated network link to ensure data transmission security.

[0028] As a further aspect of this invention, in step S6, if the connection is successful, proceed to the next step; if the connection fails, save the file synchronization task to the synchronization queue at center A and wait for retry. This includes the following specific details: The file synchronization service may face network jitter or failures when connecting to center B, requiring a robust failure handling mechanism. When a connection attempt fails, the system does not abandon the synchronization task but records the task information (including the file's hash value, file path, number of synchronization attempts, synchronization initiation time, etc.) in a synchronization queue. The synchronization queue is implemented using a message queue (such as Kafka, RabbitMQ) or a memory queue, providing fault tolerance and retry mechanisms.

[0029] As a further aspect of the present invention, in step S7, the existence of the file in center B is determined based on the file hash value. If the file exists, the file synchronization is marked as complete, and the process ends. If the file does not exist, proceed to step S8, which includes the following specific steps: If the connection is successfully established, the file service in center A quickly queries the metadata record of the file service in center B using the hash value. If center B already has a file with the same hash value, it means the file has already been synchronized, and there is no need to transmit it again; simply mark it as successfully synchronized. If center B does not have the file, proceed to the next step to synchronize the file content.

[0030] As a further aspect of this invention, S8 proposes a memory-based exclusive lock mechanism and acquires a synchronization lock. The lock type in the lock structure of the data object is checked in memory, including the following specific details: To address the file data conflict problem that may occur due to concurrent operations in remote active-active file synchronization, a memory-based exclusive lock mechanism is proposed. This mechanism achieves unique control over file synchronization operations, ensuring that only one transaction can perform synchronization tasks on a specific file at any given time, including: S81, when the system starts, the lock management module is initialized in the file service instances of center A and center B respectively. The lock management module includes LockTable and FileLockStruct. LockTable is a hash table with file hash value (FileHash) as the key and lock structure FileLockStruct as the value. FileLockStruct represents the lock state of a single file, which includes lockType (lock type), transactionId (unique identifier ID of the current lock-holding transaction), timestamp (timestamp of the last lock acquisition), and timeout (maximum lifespan of the lock).

[0031] S82, when a file needs to be synchronized to center B, the system enters the synchronization transaction preparation phase. Before starting file synchronization, an exclusive lock on the file must be acquired.

[0032] S83, call the lock management module to query the FileLockStruct corresponding to the file hash value in LockTable. If it does not exist, proceed to S84 to create a new lock structure. If it exists: 1. The lock type is "NONE", which means it is not occupied by any transaction, proceed to S85; if it is "EXCLUSIVE", proceed to S86 to handle lock conflicts.

[0033] S84. If the query result is empty, it indicates that this is the first synchronization request for that file. The system creates a new FileLockStruct with the following settings: lockType=NONE, transactionId=null, timestamp=0, timeout=default value. Finally, the FileLockStruct is inserted into the LockTable.

[0034] S85, if the current lock state is "NONE", the system attempts to acquire the lock: update lockType to "EXCLUSIVE"; write the ID of the current transaction to transactionId; set timestamp to the current system timestamp; return "lock acquired successfully", and continue the file synchronization transaction process.

[0035] S86, if the current lock state is "EXCLUSIVE", the system checks the lock timeout: get the current timestamp now; compare now-timestamp and timeout: if now-timestamp < timeout, no timeout, it means that another transaction is synchronizing, and the current transaction needs to wait for a period of time before retrying, i.e., enter S89; if now-timestamp > timeout, it means that the transaction has timed out, it means that the transaction terminated abnormally after the last lock acquisition, and enter S87.

[0036] S87, the lock is determined to be invalid due to timeout, and the system executes the recycling process: reset the lockType to "NONE"; clear the transactionId; reset the timestamp, and enter S85 to try to acquire the lock again.

[0037] In S88, for each lock acquisition, release, failure, or timeout recovery, the system records the operation event in the log center. The content includes: operation type (lock, release, failure, recovery); file hash; current transaction ID; timestamp; and operation result.

[0038] S89. If locking fails, the current transaction enters the retry queue.

[0039] S810: After a transaction completes file synchronization (including writing synchronization records, copying files, etc.), the system should release the file lock in a timely manner: set lockType=NONE; clear transactionId; update timestamp.

[0040] S811, the system background maintains a "lock recycling daemon thread" that periodically (every minute) traverses the LockTable.

[0041] S812 uses Redis and Zookeeper to build a global distributed lock in multi-node or distributed deployment scenarios (e.g., multiple file service instances in center A).

[0042] S813 sets a 3-second lock timeout for frequently accessed files; the lock table supports the LRU eviction mechanism to clean up locks that have not been accessed for a long time; the locking process uses CAS to ensure concurrency safety.

[0043] As a further aspect of the present invention, in step S9, if the synchronization lock is successfully acquired, proceed to step S10; if the synchronization lock is not acquired, wait for a retry, including the following specific content: when the exclusive lock is occupied by other transactions, this transaction will be unable to acquire the lock, enter a waiting state, and be added to the retry queue until it has another chance to acquire the lock, ensuring that the synchronization operation is eventually completed.

[0044] As a further aspect of the present invention, S10, starting the synchronization transaction includes the following specific content: after acquiring the exclusive lock, the file synchronization service officially starts the transaction mode to ensure the atomicity and consistency of the synchronization process.

[0045] As a further aspect of the present invention, S11, the file records synchronized from center A are saved to the file service of center B, including the following specific content: the file service of center B receives file metadata (including hash value, file path, file attributes, upload time and other information) and saves it to its own metadata management database to ensure that the file metadata records are highly consistent in the two locations.

[0046] As a further aspect of the present invention, S12, synchronizing files from center A to the file service storage space of center B, includes the following specific content: the file service calls the file transfer module (such as a streaming protocol or a block transfer protocol) to transfer the file data stored in center A to center B one by one and store them in the corresponding locations.

[0047] As a further aspect of the present invention, in step S13, if synchronization in S12 is successful, proceed to the next step; if synchronization fails, save the data to the synchronization queue in center B and wait for retry. This includes the following specific content: if a network anomaly or other fault occurs during transmission, the system will automatically record the anomaly in the synchronization queue and wait for retry until synchronization is successfully completed.

[0048] As a further aspect of the present invention, S14, releasing the synchronization lock includes the following specific content: after the file synchronization is successful, the transaction automatically ends, the system releases the exclusive lock on the file, allowing other subsequent transactions to access or synchronize the file.

[0049] As a further aspect of the present invention, S15, file synchronization ends, and the transaction ends, including the following specific contents: marking the end of the transaction and confirming the integrity of the transaction.

[0050] As a further aspect of the present invention, S16, writing back the synchronization status of center A includes the following specific content: updating the synchronization status field in the file metadata record of the file service in center A, marking that the file has been successfully synchronized to center B.

[0051] As a further aspect of the present invention, S17, the active-active service sees the complete corresponding file in both centers, including the following specific content: the active-active service can see the complete corresponding file in both centers. When there is a need to query the file, it can first obtain it from the nearest center file service. If it cannot obtain it, it can obtain it from the second center.

[0052] The technical effects and advantages of this invention's file synchronization processing method for a local dual-active file service are as follows: This invention ensures bidirectional real-time consistency of files and their metadata by designing a file upload and parsing process, a metadata and hash value management mechanism, and a synchronization process between center A and center B for high consistency. It introduces a synchronization lock control strategy based on a memory exclusive lock mechanism, guaranteeing mutually exclusive execution of synchronization transactions in a concurrent environment, effectively avoiding data version conflicts and "split-brain" phenomena, and improving system stability and security. Simultaneously, this invention enhances the recovery capability against sudden network interruptions and system failures through a distributed lock mechanism and a fault-tolerant design for the synchronization task queue, achieving highly reliable, high-concurrency, and low-latency file synchronization operations. Ultimately, this enables the dual-center system to maintain a complete and consistent file view without service interruption, improving the continuous availability of file services and user access experience. Attached Figure Description

[0053] Figure 1 This is a flowchart of a file synchronization processing method for a local dual-active file service according to the present invention. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] Example 1 like Figure 1 The flowchart shown illustrates a method for file synchronization in a local active-active file service, comprising the following steps: S1, users upload files to center A through the system.

[0056] S2, the file is stored in the central file storage space of A.

[0057] S3 parses the file, generating file attribute information and hash values.

[0058] S4 writes the file attribute information and hash value records to the file service in center A.

[0059] S5, File Service A begins attempting to connect to File Service B.

[0060] S6. If the connection is successful, proceed to the next step. If the connection fails, save the file synchronization task to the synchronization queue in center A and wait for a retry.

[0061] S7: Determine whether the file exists in center B based on the file hash value. If it exists, mark the file synchronization as complete and end the process; otherwise, proceed to S8.

[0062] S8 proposes a memory-based exclusive lock mechanism and acquires a synchronization lock by checking the lock type in the lock structure of the data object in memory.

[0063] S9. If the synchronization lock is successfully acquired, proceed to S10. If the synchronization lock is not acquired, wait for a retry.

[0064] S10, Start synchronizing transactions.

[0065] S11 will save the file records synchronized from center A to the file service in center B.

[0066] S12, synchronize the file from center A to the file service storage space of center B.

[0067] S13. If S12 synchronization is successful, proceed to the next step. If synchronization fails, save to the synchronization queue in center B and wait for retry.

[0068] S14, release the synchronization lock.

[0069] S15, file synchronization complete, transaction terminated.

[0070] S16, write back the synchronization status of center A.

[0071] S17, the dual-active service can see the complete corresponding files in both centers.

[0072] Further, in S1, users upload files to center A through the system, including: users performing file upload operations through the system front-end interface. Users can access the file service interface through various methods such as browser clients, enterprise clients, or mobile terminals, and select the file to be uploaded. After the file upload request is generated on the client, it is sent to the file service in center A via HTTP, FTP, or a custom data transmission protocol, carrying the content of the file to be uploaded and relevant request parameters (such as authentication information, access tokens, etc.).

[0073] Further, S2, storing the file in the file storage space of Center A includes: After receiving a user's upload request, the file service at Center A first performs security verification, permission verification, and parameter checks on the request to confirm the legitimacy of the uploaded file. After successful verification, the file service calls the underlying file storage interface (such as a distributed file system, object storage system, or NAS storage) to write the received file into the dedicated file storage space of Center A in a streaming manner. The storage process may involve mechanisms such as sharding and multi-copy storage to ensure the security and reliability of the file data. During this process, the specific location of the file storage (e.g., storage path, physical address, or logical address of the storage space) is recorded for subsequent file access and synchronization.

[0074] Furthermore, S3 parses the file, generating file attribute information and a hash value. This includes: immediately after the file service finishes storing the file, it immediately calls the file parsing module to parse the newly uploaded file, including the automatic extraction of file metadata. The file parsing module first obtains the basic file attributes through the file system interface, including the file name, file type identified by the extension, file size in bytes, and physical or logical path in the storage system. Simultaneously, the system automatically records relevant information about this file upload operation, including the identity of the uploading user and the system timestamp of the upload. In addition, to ensure cross-regional data consistency and rapid deduplication, the file service uses common hash algorithms (such as MD5 and SHA-256) to generate a unique hash value for the file content, used for subsequent synchronization checks.

[0075] Furthermore, in step S4, the file attribute information and hash value records are written to the A-center file service. This includes: the parsed file attribute information and hash value information are uniformly recorded in the metadata record database or metadata management module of the A-center file service. The metadata record database can maintain and query the metadata of each file in a high-speed and efficient manner, and provides an indexing mechanism for quick location and search.

[0076] Furthermore, in step S5, the file service at center A begins attempting to connect to the file service at center B. This includes: after completing the above steps, to ensure high consistency of file data, the file service at center A activates its synchronization service module and actively establishes a network communication connection with the file service at center B. This connection process may use a standard TCP connection method, or it may use a secure VPN tunnel or dedicated network link to ensure data transmission security.

[0077] Furthermore, in step S6, if the connection is successful, proceed to the next step; if the connection fails, the file synchronization task is saved to the synchronization queue at center A and awaits retry. This includes the fact that the file synchronization service may encounter network jitter or failures when connecting to center B, requiring a robust failure handling mechanism. When a connection attempt fails, the system does not abandon the synchronization task but records the task information (including the file's hash value, file path, number of synchronization attempts, synchronization initiation time, etc.) in a synchronization queue. The synchronization queue is implemented using a message queue (such as Kafka, RabbitMQ) or an in-memory queue, providing fault tolerance and retry mechanisms.

[0078] Further, in step S7, the system determines whether the file exists in center B based on the file hash value. If it exists, the file synchronization is marked as complete, and the process ends. If it does not exist, the process proceeds to step S8, which includes: if the connection is successfully established, the file service in center A quickly queries the metadata record of the file service in center B using the hash value. If center B already has a file with the same hash value, it means the file has already been synchronized, so there is no need to transmit it again; simply mark it as successfully synchronized. If center B does not have this file, the process proceeds to the next step to synchronize the file content.

[0079] Furthermore, S8 proposes a memory-based exclusive lock mechanism and acquires a synchronization lock by checking the lock type in the lock structure of the data object in memory. This includes: To address the file data conflict problem that may occur due to concurrent operations in remote active-active file synchronization, a memory-based exclusive lock mechanism is proposed, achieving unique control over file synchronization operations and ensuring that only one transaction can perform synchronization tasks on a specific file at any given time. S81, when the system starts, the lock management module is initialized in the file service instances of center A and center B respectively. The lock management module includes LockTable and FileLockStruct. LockTable is a hash table with file hash value (FileHash) as the key and lock structure FileLockStruct as the value. FileLockStruct represents the lock state of a single file, which includes lockType (lock type), transactionId (unique identifier ID of the current lock-holding transaction), timestamp (timestamp of the last lock acquisition), and timeout (maximum lifespan of the lock).

[0080] S82, when a file needs to be synchronized to center B, the system enters the synchronization transaction preparation phase. Before starting file synchronization, an exclusive lock on the file must be acquired.

[0081] S83, call the lock management module to query the FileLockStruct corresponding to the file hash value in LockTable. If it does not exist, proceed to S84 to create a new lock structure. If it exists: 1. The lock type is "NONE", which means it is not occupied by any transaction, proceed to S85; if it is "EXCLUSIVE", proceed to S86 to handle lock conflicts.

[0082] S84. If the query result is empty, it indicates that this is the first synchronization request for that file. The system creates a new FileLockStruct with the following settings: lockType=NONE, transactionId=null, timestamp=0, timeout=default value. Finally, the FileLockStruct is inserted into the LockTable.

[0083] S85, if the current lock state is "NONE", the system attempts to acquire the lock: update lockType to "EXCLUSIVE"; write the ID of the current transaction to transactionId; set timestamp to the current system timestamp; return "lock acquired successfully", and continue the file synchronization transaction process.

[0084] S86, if the current lock state is "EXCLUSIVE", the system checks the lock timeout: get the current timestamp now; compare now-timestamp and timeout: if now-timestamp < timeout, no timeout, it means that another transaction is synchronizing, and the current transaction needs to wait for a period of time before retrying, i.e., enter S89; if now-timestamp > timeout, it means that the transaction has timed out, it means that the transaction terminated abnormally after the last lock acquisition, and enter S87.

[0085] S87, the lock is determined to be invalid due to timeout, and the system executes the recycling process: reset the lockType to "NONE"; clear the transactionId; reset the timestamp, and enter S85 to try to acquire the lock again.

[0086] In S88, for each lock acquisition, release, failure, or timeout recovery, the system records the operation event in the log center. The content includes: operation type (lock, release, failure, recovery); file hash; current transaction ID; timestamp; and operation result.

[0087] S89. If locking fails, the current transaction enters the retry queue. The system uses a fixed-interval polling strategy (e.g., retrying every 200ms) or an exponential backoff strategy (e.g., retrying at exponential intervals until the maximum waiting time of 3 seconds). If the lock still cannot be acquired after multiple retries, the system can display "File is being synchronized" or move the transaction to the background retry mechanism.

[0088] S810: After a transaction completes file synchronization (including writing synchronization records, copying files, etc.), the system should release the file lock in a timely manner: set lockType=NONE; clear transactionId; update timestamp.

[0089] S811, the system background maintains a "lock reclamation daemon thread" that periodically (every minute) traverses the LockTable: checks whether the timestamp of each FileLockStruct exceeds the timeout; if it does, and the transaction state is invalid or the transaction record does not exist, then the lock is released.

[0090] In multi-node or distributed deployment scenarios (e.g., multiple file service instances in data center A), S812 uses Redis and Zookeeper to build a global distributed lock. Each instance calls the distributed lock API to implement file locking logic, maintaining lock consistency across the entire data center.

[0091] S813 sets a 3-second lock timeout for frequently accessed files; the lock table supports the LRU eviction mechanism to clean up locks that have not been accessed for a long time; the locking process uses CAS to ensure concurrency safety.

[0092] Furthermore, in S9, if the synchronization lock is successfully acquired, proceed to S10; if the synchronization lock is not acquired, wait for a retry. This includes situations where, if the exclusive lock is held by another transaction, this transaction will be unable to acquire the lock, enter a waiting state, and be added to the retry queue until it has another chance to acquire the lock, ensuring that the synchronization operation is eventually completed.

[0093] Furthermore, in S10, the synchronization transaction begins, including: after acquiring the exclusive lock, the file synchronization service officially starts the transaction mode to ensure the atomicity and consistency of the synchronization process.

[0094] Furthermore, in S11, the file records synchronized from center A will be saved to the file service in center B. This includes: the file service in center B receiving file metadata (including hash value, file path, file attributes, upload time, etc.) and saving it to its own metadata management database to ensure that the file metadata records are highly consistent between the two locations.

[0095] Further, in step S12, the files are synchronized from center A to the file service storage space in center B, including: the file service calls the file transfer module (such as a streaming protocol or a block transfer protocol) to transfer the file data stored in center A to center B one by one and store it in the corresponding location.

[0096] Furthermore, in step S13, if synchronization in S12 is successful, proceed to the next step; if synchronization fails, save the data to the synchronization queue in center B and wait for retry. This includes: if network anomalies or other failures occur during transmission, the system will automatically record the anomaly in the synchronization queue and wait for retry until synchronization is successfully completed.

[0097] Furthermore, in S14, the synchronization lock is released, including: after the file synchronization is successful, the transaction automatically ends, the system releases the exclusive lock on the file, and allows other subsequent transactions to access or synchronize the file.

[0098] Further, in S15, file synchronization ends, the transaction ends, including: marking the end of the transaction and confirming the integrity of the transaction.

[0099] Further, in S16, the synchronization status of center A is written back, including the synchronization status field in the file metadata record updated by the file service of center A, marking that the file has been successfully synchronized to center B.

[0100] Furthermore, in S17, the active-active service sees the complete corresponding file in both data centers. This includes: the active-active service can see the complete corresponding file in both data centers. When there is a file query requirement, it can first retrieve the file from the nearest data center. If the file cannot be retrieved from the nearest data center, it will retrieve it from the second data center. When a user subsequently requests a file, the load balancing mechanism of the dual data centers automatically directs the request to the service center closer to the user. The file service first responds quickly locally. If the file is not found locally (e.g., due to temporary storage failure), the request is automatically routed to the other data center, ensuring continuous, stable, and efficient user access.

[0101] This invention designs a file upload and parsing process, a metadata and hash value management mechanism, and a synchronization process between center A and center B for high consistency. This method ensures bidirectional real-time consistency of files and their metadata. A synchronization lock control strategy based on a memory exclusive lock mechanism is introduced to guarantee mutually exclusive execution of synchronous transactions in a concurrent environment, effectively avoiding data version conflicts and "split-brain" phenomena, thus improving system stability and security. Simultaneously, this invention enhances the recovery capability from sudden network interruptions and system failures through a distributed lock mechanism and a fault-tolerant design for the synchronization task queue. It achieves highly reliable, high-concurrency, and low-latency processing of file synchronization operations, ultimately enabling the dual-center system to maintain a complete and consistent file view without service interruption, improving the continuous availability of file services and the user access experience.

[0102] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0103] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for file synchronization processing in a local active-active file service, characterized in that, Includes the following steps: S1, users upload files to center A through the system; S2, the file is stored in the file storage space of center A; S3 parses the file, generating file attribute information and hash values; S4, write the file attribute information and hash value record to the file service in center A; S5, File Service A at Center B begins attempting to connect to File Service B. S6. If the connection is successful, proceed to the next step. If the connection fails, save the file synchronization task to the synchronization queue in center A and wait for a retry. S7: Determine whether the file exists in center B based on the file hash value. If it exists, mark the file synchronization as complete and end the process; otherwise, proceed to S8. S8 proposes a memory-based exclusive lock mechanism and acquires a synchronization lock by checking the lock type in the lock structure of the data object in memory. S9. If the synchronization lock is successfully acquired, proceed to S10. If the synchronization lock is not acquired, wait to retry. S10, Start synchronizing transactions; S11, save the file records synchronized from center A to the file service in center B; S12, synchronize the file from center A to the file service storage space of center B; S13. If S12 synchronization is successful, proceed to the next step; if synchronization fails, save to the synchronization queue in center B and wait for retry. S14, release the synchronization lock; S15, File synchronization complete, transaction terminated; S16, Write back the synchronization status of center A; S17, the dual-active service can see the complete corresponding files in both centers.

2. The method for file synchronization processing in a local active-active file service according to claim 1, characterized in that... The aforementioned memory-based exclusive lock mechanism ensures that only one transaction can perform synchronization tasks on a specific file at any given time, and includes the following steps: S81, When the system starts, the lock management module is initialized in the file service instances of center A and center B respectively. The lock management module includes LockTable and FileLockStruct. S82, when a file needs to be synchronized to center B, the system enters the synchronization transaction preparation phase; S83, call the lock management module to query the FileLockStruct corresponding to the file hash value in LockTable. If it does not exist, proceed to S84 to create a new lock structure. If it exists:

1. The lock type is "NONE", which means it is not occupied by any transaction, proceed to S85; if it is "EXCLUSIVE", proceed to S86 to handle lock conflicts. S84. If the query result is empty, it means that this is the first synchronization request for the file. The system creates a new FileLockStruct with the following settings: lockType=NONE, transactionId=null, timestamp=0, timeout=default value. Finally, the FileLockStruct is inserted into the LockTable. S85, if the current lock state is "NONE", the system attempts to acquire the lock: update lockType to "EXCLUSIVE"; write the ID of the current transaction to transactionId; set timestamp to the current system timestamp; return "lock acquired successfully" and continue the file synchronization transaction process; S86, if the current lock state is "EXCLUSIVE", the system checks the lock timeout: get the current timestamp now; compare now-timestamp and timeout: if now-timestamp < timeout, no timeout, it means another transaction is synchronizing, and the current transaction needs to wait for a period of time before retrying, i.e., enter S89; if now-timestamp > timeout, it means the timeout has occurred, it means the transaction terminated abnormally after the last lock acquisition, enter S87; S87, the lock is determined to be invalid due to timeout, and the system executes the recycling process: reset the lockType to "NONE"; clear the transactionId; reset the timestamp, and enter S85 to try to acquire the lock again; S88: For each lock acquisition, release, failure, and timeout recovery, the system records the operation events in the log center. S89, If locking fails, the current transaction enters the retry queue; S810: After a transaction completes file synchronization, the system should promptly release the file lock: set lockType=NONE; clear transactionId; update timestamp; S811, the system background maintains a "lock reclamation daemon thread" that periodically (every minute) traverses the LockTable; S812 uses Redis and Zookeeper to build a global distributed lock in multi-node or distributed deployment scenarios; S813 sets a 3-second lock timeout for frequently accessed files; the lock table supports LRU eviction mechanism to clean up locks that have not been accessed for a long time; the locking process uses CAS to ensure concurrency safety.

3. The method for file synchronization processing in a local active-active file service according to claim 1, characterized in that... The lock management module includes LockTable and FileLockStruct. LockTable is a hash table with file hash values ​​as keys and lock structures FileLockStruct as values. FileLockStruct represents the lock state of a single file and includes lockType, transactionId, timestamp, and timeout.

4. The method for file synchronization processing in a local active-active file service according to claim 1, characterized in that... In step S1, users access the file service interface through a browser client, enterprise client, or mobile terminal to upload files using HTTP, FTP, or a custom protocol, carrying identity authentication information and access tokens.

5. The method for file synchronization processing in a local active-active file service according to claim 1, characterized in that... In S2, the A-center file service performs security verification, permission verification, and parameter checks on upload requests. After successful verification, it calls the underlying storage interface to write the file to a dedicated storage space in a streaming manner, records the storage path and physical or logical address, and adopts a sharded storage or multi-copy storage mechanism.

6. The method for file synchronization processing in a local active-active file service according to claim 1, characterized in that... In S4, the file attribute information and hash value records are stored in the metadata record database or metadata management module of the A center file service. The metadata record database has an indexing mechanism to enable querying.

7. The method for file synchronization processing in a local active-active file service according to claim 1, characterized in that... In S5, the A-center file service establishes a communication connection with the B-center file service through a TCP connection, VPN tunnel, or dedicated network link.

8. The method for file synchronization processing in a local active-active file service according to claim 1, characterized in that... The synchronization queue in S6 is implemented through Kafka, RabbitMQ or a memory queue, and records the file hash value, file path, number of synchronization attempts and initiation time, and has fault tolerance and retry mechanism.

9. A method for file synchronization processing in a local active-active file service according to claim 1, characterized in that... In step S7, center A queries the metadata records of center B through file hash values. Files with the same hash value are determined to be synchronized and do not need to be transmitted again.

10. A method for file synchronization processing in a local active-active file service according to claim 1, characterized in that... Transactions that fail to acquire the synchronization lock in S9 are added to the retry queue and retried using a fixed interval of 200ms polling or an exponential backoff strategy, with a maximum waiting time of 3 seconds.

Images