File descriptor operation method and system for access layer of multi-thread file system

Abstract

The invention provides a file descriptor operation method and a file descriptor operation system for an access layer of a multi-thread file system, which belong to the field of cloud storage, and respectively record an I / O operation state and a to-be-processed file descriptor state by introducing two atomic counters of iowork and fdwork. And synchronous control is carried out through an atomic operation counter and a mutual exclusion lock, so that opening, closing and related I / O (Input / Output) operation of the file descriptor are safely and effectively managed. Through atomic operation and a mutual exclusion lock mechanism, it is ensured that concurrent access can be effectively processed when file descriptors are recycled, and unnecessary resource scrambling is avoided. The two key atomic operation functions provided by the invention are used for safely and atomically increasing the counters under specific conditions and obtaining the mutual exclusion lock when the counters are decreased to zero, and through the combined action of the two functions, the synchronization problem of shared resource access in a multi-thread environment is effectively solved, and the system performance and stability are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of cloud storage, in particular to a file storage gateway in the field of cloud storage systems, and more particularly to a file descriptor operation method and system for a multi-threaded file system access layer. Background Art

[0002] In modern computer systems, especially multi-core processors and distributed computing environments, multi-threaded programming has become the norm. In this context, how to effectively manage concurrent threads' access to file system resources has become an important challenge. Especially when it comes to opening, closing, reading and writing file descriptors (fd), since these operations involve complex system resource allocation and state management, it is necessary to ensure that race conditions and deadlocks are avoided during the collaborative work between multiple threads to maintain the correctness and stability of the file system.

[0003] Traditional methods often fail to fully guarantee the consistency and integrity of file descriptors during concurrent execution, which can easily lead to data corruption, performance bottlenecks, and even system crashes. Therefore, a new solution is urgently needed to safely and efficiently manage various operations of file descriptors in a multi-threaded environment. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a file descriptor operation method for a multi-threaded file system access layer. The method cleverly utilizes atomic operations and mutex mechanisms, introduces two atomic counters, io_work and fd_work, and a series of matching synchronization primitives, to accurately track and control the I / O operation status and file descriptor status change tasks of each file descriptor. In particular, the present invention takes into account the complex situation when reclaiming file descriptor resources, ensures that potential conflicts can still be accurately identified and handled in a concurrent execution state, thereby greatly improving the reliability, security and efficiency of file access in multi-threaded applications.

[0005] The technical solution of the present invention is:

[0006] A file descriptor operation method for a multi-threaded file system access layer, by introducing two atomic counters, io_work and fd_work, to record the I / O operation status and the status of the file descriptor to be processed respectively;

[0007] The opening and closing of file descriptors and related I / O operations are managed through atomic operation counters and synchronization control using mutex locks.

[0008] Furthermore,

[0009] For the recovery operation of file descriptors, use atomic operations and mutex lock mechanisms;

[0010] Two atomic operation functions are used to safely and atomically increase the counter under specific conditions and acquire the mutex lock when decrementing the counter to zero. These two functions work together to solve the synchronization problem of shared resource access in a multi-threaded environment.

[0011] Going further,

[0012] a. Initialize and maintain two atomic counters, io_work and fd_work. io_work is used to track the I / O operations currently in progress or pending, and fd_work is used to track the file descriptor status change tasks to be executed;

[0013] b. Before initiating a new I / O operation, atomically increment the io_work counter;

[0014] c. When there is a task to change the status of a file descriptor, the fd_work counter is atomically incremented, and the status of io_work is checked after acquiring the mutex lock. The status change operation is performed only when io_work is 0;

[0015] d. When executing the file descriptor status change operation, first obtain the write lock to ensure atomicity;

[0016] e. After the state change is completed, the fd_work counter is reduced by an atomic decrement operation, and the condition variable signal is sent under appropriate conditions to notify the waiting I / O thread;

[0017] During the waiting period, the fI / O thread monitors the condition variable and releases the suspended state after receiving the signal, and continues to perform subsequent I / O operations.

[0018] It further includes ensuring that the read-write lock is correctly locked and unlocked before and after each file descriptor state change operation to ensure the security of concurrent access between different threads.

[0019] In addition, the present invention also provides a system for implementing the file descriptor operation management method according to claim 1, the system comprising:

[0020] a. Data structure used to store and update io_work and fd_work counters;

[0021] b. Atomic operation-based counter management module, used to implement counter increment, decrement, and locking and unlocking under specific conditions;

[0022] c. A set of mutex locks and condition variables to synchronize the relationship between file descriptor state change operations and I / O operations to ensure that the two are not executed concurrently.

[0023] Furthermore,

[0024] It also includes a controller for monitoring and scheduling file descriptor operations. The controller is responsible for deciding whether to allow new I / O operations or state change tasks based on the current values ​​of io_work and fd_work.

[0025] Furthermore,

[0026] It also specifically stipulates that after the state change operation is completed, if there is a suspended I / O thread, the thread will be reactivated according to a certain priority strategy to optimize system resource allocation and performance.

[0027] The beneficial effects of the present invention are

[0028] The file descriptor operation management system and concurrency control method proposed in the present invention significantly improve the security of file descriptor operations in a concurrent environment, effectively solve the problem of data inconsistency, optimize resource scheduling, and improve the overall performance of the system. Through rigorous counter control and lock mechanism, the orderly operation of file descriptor operations is ensured, and the stability and reliability of the system are enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flowchart of I / O operations;

[0030] Figure 2 It is a flowchart of the file descriptor operation process. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0032] The present invention proposes a novel file descriptor operation management method for implementing in a multi-threaded file system access layer (Filesystem Abstraction Layer, FSAL). The method cleverly utilizes atomic operations and mutex mechanisms, introduces two atomic counters, io_work and fd_work, and a series of matching synchronization primitives, to accurately track and control the I / O operation status and file descriptor status change tasks of each file descriptor. In particular, the present invention takes into account the complex situation when reclaiming file descriptor resources, ensures that potential conflicts can still be accurately identified and handled in a concurrent execution state, thereby greatly improving the reliability, security and efficiency of file access in multi-threaded applications.

[0033] In addition, the present invention also proposes two key atomic operation functions, namely

[0034] atomic_add_unless_int32_t and atomic_dec_int32_t_and_lock are used to atomically increase and decrease counters safely under certain conditions, and to intelligently acquire a mutex lock when the counter is decremented to zero. The design and application of these two atomic operation functions not only solve the synchronization problem of shared resource access in a concurrent environment, but also significantly reduce the risk of resource misoperation caused by race conditions, ensure the correct release of resources and access synchronization, and effectively improve the overall performance and stability of the system.

[0035] The specific steps include:

[0036] 1. Design and introduce two atomic counters, io_work and fd_work, to record and track the working status of I / O operations and the number of pending file descriptor status change tasks respectively.

[0037] 2. If the current thread is processing a read or write I / O operation, use the io_work field as an atomic integer to indicate that an I / O operation is in progress or is attempting to start.

[0038] a) Increment the id_work counter through atomic operations. Loop to check whether there are other threads operating the file descriptor. When it is detected that fd_work is non-zero, perform the following operations: 1) Decrement the io_work counter through atomic operations to temporarily stop I / O operations and make way for the file descriptor work. 2) Use the mutex work_mutex for synchronization. If the current thread successfully obtains the mutex, send a signal to the thread waiting for the file descriptor work to inform it that it can continue; otherwise, directly lock the mutex. 3) Use the conditional variable work_cond to wait for fd_work to become zero, indicating that the management of the file descriptor has been completed.

[0039] b) Under the protection of the mutex, check whether the open mode of the current file descriptor matches the required open mode. If it does not match and reopening is not allowed, unlock the mutex and return an error status. If it matches, increase the fd_work counter to indicate that work needs to be performed on the file descriptor, and reopen the file descriptor. Regardless of whether the file descriptor is reopened, the io_work counter must be incremented again to ensure that the file descriptor is not closed in other threads when preparing for I / O operations. Finally, unlock the mutex to allow other threads to continue execution.

[0040] Through this method, the opening and closing of file descriptors and related I / O operations can be safely managed in a concurrent environment, avoiding the competition conditions caused by multi-threaded operations, ensuring the correctness and integrity of file system operations, and thus improving the reliability of file access in multi-threaded applications.

[0041] 3. In the present invention, we propose a method for managing the open / close operation of file descriptors (fd) in the file system abstraction layer (FSAL). The method aims to ensure safe and effective operation of file descriptors in a multi-threaded environment, especially when it is necessary to reclaim file descriptor resources.

[0042] a) First, the fd_work counter is incremented through atomic operations to indicate that the management work for the file descriptor is about to begin. The mutex work_mutex in the fsal_fd structure is locked to prevent other threads from interfering during the execution of the operation.

[0043] b) Check whether there is an ongoing or about to start I / O operation, and determine whether it is in the file descriptor recycling stage. If there is an ongoing I / O operation and recycling is in progress, then skip the current file descriptor, move its position in the LRU (least recently used) list forward, release the mutex, and return the status code EBUSY, indicating that it is not suitable for recycling operation at present. When there is no recycling operation, enter a loop waiting until the io_work counter is detected to be 0, indicating that there is no I / O operation in progress. During this period, if it is in recycling mode again and there is an active I / O operation, the same measures of skipping and returning the EBUSY status code are taken.

[0044] c) When the io_work counter is detected to be 0, it indicates that the file descriptor can be opened / closed safely. After the operation is completed, the fd_work atomic counter is decremented, the mutex is released, and the I / O thread that was suspended due to waiting for fd_work to be cleared is awakened, so that it can resume subsequent I / O operations.

[0045] Through this method, the present invention can ensure that the management of file descriptors is carried out in an orderly manner in a multi-threaded environment, avoid resource conflicts, and properly manage the recovery process of file descriptors when necessary, thereby improving the stability of the system and resource utilization.

[0046] 4. In order to ensure the atomicity and integrity of the file descriptor state change operation, in this invention, we propose an innovative method for atomic management and synchronization protection of shared resource access in a multi-threaded environment. Specifically, it is manifested in two key atomic operation functions:

[0047] a) Atomic conditional increment function: A function named atomic_add_unless_int32_t is designed to solve the problem of safe increment of a shared 32-bit integer counter in a concurrent environment. The function uses the atomic function feature of GCC to perform the increment operation only when the current value of the counter is not equal to a specific flag value while maintaining atomicity. In this way, the present invention effectively avoids the risk of resource misoperation caused by concurrency when specific conditions are not met.

[0048] b) Atomic decrement and mutex locking function: another key function

[0049] atomic_dec_int32_t_and_lock is designed to decrement a shared 32-bit integer reference counter in a multithreaded scenario and automatically acquire a mutex lock when it is decremented to zero. This function first attempts to atomically decrement the counter. If the counter is not zero after decrement, no processing is done; if the counter happens to become zero after decrement, the mutex lock associated with it is immediately locked to ensure exclusive access to resources and necessary cleanup operations when the counter is zero.

[0050] In summary, the present invention greatly enhances the accuracy and security of shared resource access control through these two carefully designed atomic operation functions, effectively prevents common race conditions in multi-threaded programming, and ensures the correct release and access synchronization of resources, which is of great significance for improving the overall performance and stability of the system.

[0051] The above description is only a preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, and is not used to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A file descriptor operation method for a multi-threaded file system access layer, characterized in that: By introducing two atomic counters, io_work and fd_work, the I / O operation status and the status of the file descriptor to be processed are recorded respectively; The opening and closing of file descriptors and related I / O operations are managed through atomic operation counters and synchronization control using mutex locks.

2. The method according to claim 1, characterized in that For the recovery operation of file descriptors, use atomic operations and mutex lock mechanisms; Two atomic operation functions are used to safely and atomically increase the counter under specific conditions and acquire the mutex lock when decrementing the counter to zero. These two functions work together to solve the synchronization problem of shared resource access in a multi-threaded environment.

3. The method according to claim 2, characterized in that The specific steps include: a. Initialize and maintain two atomic counters, io_work and fd_work. io_work is used to track the I / O operations currently in progress or pending, and fd_work is used to track the file descriptor status change tasks to be executed; b. Before initiating a new I / O operation, atomically increment the io_work counter; c. When there is a task to change the status of a file descriptor, the fd_work counter is atomically incremented, and the status of io_work is checked after acquiring the mutex lock. The status change operation is performed only when io_work is 0; d. When executing the file descriptor status change operation, first obtain the write lock to ensure atomicity; e. After the state change is completed, the fd_work counter is reduced by an atomic decrement operation, and the condition variable signal is sent to notify the waiting I / O thread; f. During the waiting period, the I / O thread monitors the condition variable and releases the suspended state after receiving the signal, and continues to execute subsequent I / O operations.

4. The method according to claim 3, characterized in that Ensure that the read-write lock is correctly locked and unlocked before and after each file descriptor status change operation to ensure the safety of concurrent access between different threads.

5. The method according to any one of claims 1 to 4, characterized in that: After the state change operation is completed, if there are suspended I / O threads, the threads are reactivated according to the priority policy to optimize system resource allocation and performance.

6. A system for implementing the file descriptor operation management method according to claim 1, characterized in that: include: a. Data structure used to store and update io_work and fd_work counters; b. Atomic operation-based counter management module, used to implement counter increment, decrement, and locking and unlocking under specific conditions; c. A set of mutex locks and condition variables to synchronize the relationship between file descriptor state change operations and I / O operations to ensure that the two are not executed concurrently.

7. The system according to claim 6, characterized in that It also includes a controller for monitoring and scheduling file descriptor operations. The controller is responsible for deciding whether to allow new I / O operations or state change tasks based on the current values ​​of io_work and fd_work.

8. The system according to claim 7, characterized in that Ensure that the read-write lock is correctly locked and unlocked before and after each file descriptor status change operation to ensure the safety of concurrent access between different threads.

9. The system according to any one of claims 6 to 8, characterized in that After the state change operation is completed, if there are suspended I / O threads, the threads are reactivated according to the priority policy to optimize system resource allocation and performance.

Images