Bcache cache system online replacement and automatic reconstruction method and system

Abstract

This invention provides a method and system for online replacement and automatic reconstruction of a bcache caching system, belonging to the field of computer storage technology. The method adds a replica management layer, a device management layer, and a reconstruction engine layer to the bcache kernel module, and constructs a distributed replica mapping table to record the replica location, status, and version information of each cached data unit on multiple cache devices. During write operations, the replica management layer distributes data in parallel to multiple replicas and waits for all data to be persisted; during read operations, a failover method is used to read data from available replicas. The device management layer monitors device status and triggers online addition, online removal, or fault handling processes; the reconstruction engine layer dynamically adjusts the priority, parallelism, and rate of reconstruction tasks according to system load, supporting breakpoint resumption. This invention eliminates the single point of failure risk of the bcache caching layer, realizes seamless online replacement and automatic reconstruction of cache devices, ensures high data availability and business continuity, and has minimal impact on the original read and write performance.

Description

Technical Field

[0001] This invention relates to the field of computer storage technology, and in particular to a method and system for online replacement and automatic reconstruction of a bcache caching system. More specifically, this invention relates to a technical solution for implementing a data copy redundancy mechanism among multiple caching devices, seamless online replacement of faulty devices, and automatic data reconstruction and recovery in a bcache architecture. Background Technology

[0002] bcache is a block device caching system widely adopted by the Linux kernel mainline since version 3.10. Its core function is to use high-speed storage devices (such as SSDs) as a transparent read / write cache for slower storage devices (such as HDDs), managing cached data through a B+ tree index structure, thereby significantly improving the input / output (I / O) performance of the storage system. bcache has been widely used in scenarios with extremely high storage performance requirements, such as cloud computing and virtualization.

[0003] However, existing bcache implementations have the following significant technical shortcomings:

[0004] First, the lack of data redundancy protection poses a single point of failure risk. While existing technologies support configuring multiple caching devices, these devices operate only in a simple aggregation or cascading manner, without data replication relationships between them. If any caching device fails, all cached data on it will be immediately lost. Especially in "write-back" mode, data not yet synchronized to slower backend devices will be permanently lost, directly leading to file system inconsistencies and seriously threatening data integrity and business continuity.

[0005] Secondly, the equipment management is rigid, making online operation and maintenance impossible. Under the existing bcache architecture, the addition, removal, or replacement of cache devices lacks standardized online procedures. Performing such maintenance tasks usually requires pausing business operations, unloading the file system, or executing a series of complex and error-prone manual commands, which completely fails to meet the stringent requirements of modern data centers for high service availability (requiring over 99.99%).

[0006] Furthermore, fault recovery relies entirely on manual intervention, which is inefficient. When a caching device fails, the system cannot automatically detect and trigger the data recovery process. System administrators must manually identify the fault, remove it from the cache pool, introduce a new device, and manually trigger data reconstruction. The entire recovery process is time-consuming, complex, and highly error-prone. During this period, the system not only operates in a degraded state with severely degraded performance, but the risk window for the data is also greatly extended.

[0007] Finally, there is uneven resource utilization. The lack of intelligent data distribution and balancing strategies among multiple caching devices can easily lead to hot data being concentrated on specific devices, creating performance bottlenecks, while the storage and performance potential of other devices remains idle.

[0008] In summary, the core flaw of existing bcache technology lies in its "single point of failure" model and "static management" approach, making its cache layer itself a weak link in the reliability of the entire storage system. Therefore, there is an urgent need in the field for a technical solution that can integrate data redundancy, online replacement, and automatic reconstruction capabilities into the bcache system without affecting the original high-performance read / write path and compatibility.

[0009] The information disclosed in this background section is intended only to enhance the understanding of the overall background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0010] The purpose of this invention is to provide a technical solution that integrates data redundancy, online replacement, and automatic reconstruction capabilities into the bcache system without affecting the original high-performance read / write path and compatibility.

[0011] To achieve the above objectives, the present invention provides the following solution:

[0012] A method for online replacement and automatic reconstruction of a bcache caching system includes:

[0013] In the bcache kernel module, a replica management layer, a device management layer, and a rebuild engine layer are added, and a distributed replica mapping table is constructed. The replica mapping table records the replica location, status, and version information of each cached data unit on multiple cache devices.

[0014] In the write operation path, the replica management layer distributes the write data in parallel to multiple replicas of multiple cache devices according to the replica mapping table, and waits for all configuration replicas to be persisted before confirming the write success.

[0015] In the read operation path, the replica management layer obtains multiple replica locations based on the replica mapping table and reads data from available replicas using a failover method.

[0016] The device management layer monitors the status of cache devices. When a change in device status is detected, it triggers online addition, online removal, or fault handling processes, and the reconstruction engine layer performs data migration or reconstruction tasks.

[0017] The reconstruction engine layer dynamically adjusts the priority, parallelism, and rate of reconstruction tasks based on preset redundancy strategies and the current system load, and supports breakpoint resumption of reconstruction tasks.

[0018] Optionally, the distributed replica mapping table adopts a two-level index structure: the first level is the mapping from the original bcache B+ tree key to the replica set descriptor, and the second level is the replica set descriptor, which contains multiple replica entries. Each replica entry records the device identifier, physical address, replica status, and version number.

[0019] Optionally, the redundancy strategy includes at least one of the following: N-replica mirroring mode, which completely copies the data N times and stores it on N different devices; erasure coding mode, which divides the data into N data blocks and calculates M check blocks, and stores them distributed across different devices; and local replica mode, which uses multi-replica mirroring for metadata and erasure coding for user data.

[0020] Optionally, configurable consistency levels are also included: in strong consistency mode, write operations are only confirmed after they are persisted on all configured replicas; in eventual consistency mode, write operations are confirmed after they are persisted on the primary replica, and other replicas are updated asynchronously in the background.

[0021] Optionally, the online joining process includes: after the device management layer discovers a new device, it sets its status to joining, triggers the rebuild engine layer to start a rebalancing task, which scans the replica mapping table, migrates existing data replicas to the new device and atomically updates the replica mapping table, and changes the status of the new device to online after completion.

[0022] Optionally, the online removal process includes: in response to a planned removal instruction, setting the target device status to "draining", stopping the replica management layer from assigning new replicas to it, initiating a migration task at the rebuild engine layer, migrating all replicas on the device to other online devices and updating the mapping table, and setting the device status to "drained" and removing it from the device group after all migrations are completed.

[0023] Optionally, the fault handling process includes: after the device management layer confirms the device fault through IO timeout, sentinel read / write test failure or driver report error, the device status is marked as faulty, all IO requests to the device are immediately routed to other available replicas, the replica management layer marks the replica items on the faulty device as invalid, and for data units with fewer than the number of available replicas, the reconstruction engine adds them to the high-priority reconstruction queue.

[0024] Optionally, the reconstruction engine layer maintains multiple priority queues, including urgent, high, medium, and low queues. The scheduler dynamically determines the task execution order based on queue priorities and the current system load. Furthermore, the reconstruction engine layer continuously monitors system-level IO latency, device-level IO queue depth, and CPU idle ratio. Through feedback control loops, it dynamically adjusts the number of reconstruction worker threads, data block size, and request interval to achieve adaptive flow control.

[0025] An online replacement and automatic rebuilding system for a bcache caching system includes:

[0026] The replica management layer is used to maintain a global distributed replica mapping table, record the physical replica location, status and version information of each cached data unit on multiple cache devices, and execute the configured redundancy policy. During write operations, write data is distributed to multiple replicas in parallel, and during read operations, an available replica is selected from multiple replicas for failover reading.

[0027] The device management layer is used to continuously monitor the status of all cache devices, manage the relationship between device pool members, and trigger the corresponding online addition, online removal or fault handling protocol when a device status change is detected.

[0028] The reconstruction engine layer is used to perform data reconstruction or rebalancing tasks when the replica management layer reports that the number of replicas is lower than the configured threshold or the device management layer notifies the device to be added or removed. It also dynamically adjusts the priority, parallelism, and rate of reconstruction tasks according to the system load and supports breakpoint resumption.

[0029] The system is an enhancement of the Linux kernel bcache module, retaining the original application programming interface and management toolchain.

[0030] Optionally, the replica management layer uses read-copy-update lock-free technology to access the replica mapping table on the critical path of data read and write; the device management layer designs device status monitoring and partial metadata updates to be executed asynchronously; the reconstruction engine layer uses an independent work queue and an adjustable thread pool, isolated from the thread pool that processes front-end business requests.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] Traditional bcache suffers from a single point of failure risk; the failure of any cache device will lead to data loss and service interruption. This invention transforms the cache layer into a fault-tolerant, highly available component by maintaining data copies across multiple cache devices. With proper configuration, it can achieve a reliability level comparable to the backend primary storage, and provides robust protection for data security, especially in write-back mode.

[0033] Existing bcache device maintenance requires downtime, impacting business continuity. This invention supports hot-swapping, online addition, online removal, and online replacement of cache devices. All operations are transparent to upper-layer applications, requiring no service interruption or unloading of storage volumes, thus meeting the extreme high availability requirements of cloud data centers.

[0034] This invention automatically detects and isolates faulty devices at the device management level, and the reconstruction engine automatically initiates data reconstruction, freeing administrators from tedious manual operations. A unified policy framework and enhanced management tools make managing large-scale cache device configurations possible, significantly reducing operating costs.

[0035] Under normal operation, the multi-replica mechanism has a positive effect on read performance, keeping the increase in write latency within 15%. During fault recovery, adaptive flow control ensures that rebuilding operations do not interfere with production operations, achieving an optimal balance of resource usage.

[0036] This invention enhances, rather than replaces, the existing bcache kernel module, fully retaining the original API and management toolchain. Existing applications can run without modification. The modular policy engine supports pluggable configuration, adapting to different hardware and business scenarios. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a schematic diagram of the architecture provided for an embodiment of the present invention.

[0039] Figure 2 This is a schematic diagram of the writing process provided in an embodiment of the present invention.

[0040] Figure 3 This is a schematic diagram of the reading process provided in an embodiment of the present invention.

[0041] Figure 4 A flowchart illustrating the online caching device replacement process provided in this embodiment of the invention.

[0042] Figure 5 A flowchart of an automatic reconstruction engine provided for an embodiment of the present invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] The purpose of this invention is to provide a technical solution that integrates data redundancy, online replacement, and automatic reconstruction capabilities into the bcache system without affecting the original high-performance read / write path and compatibility.

[0045] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] Example 1: System Architecture and Deployment

[0047] This embodiment describes the overall architecture and deployment method of the online replacement and automatic reconstruction system for the bcache caching system provided by the present invention.

[0048] like Figure 1 As shown, this invention introduces three new core logical layers into the standard bcache architecture: the replica management layer, the device management layer, and the rebuild engine layer. These new layers work in conjunction with the original bcache's data path and control path to form a complete enhanced system.

[0049] The replica management layer is the core control unit of the system, responsible for maintaining the global replica mapping relationship. This layer implements a distributed metadata management system, recording the location information of all physical replicas corresponding to each logical cache unit. The replica management layer is also responsible for executing the configured redundancy policies, such as ensuring that each write operation is distributed synchronously or asynchronously to two different devices for two-replica mirrors; and calculating and distributing data blocks and checksum blocks for erasure coding policies.

[0050] The device management layer is the system's status monitoring and coordination unit. It continuously monitors the status of all cache devices (including online, offline, degraded, and faulty) and manages the membership relationships of the device pool. When a device status change is detected, this layer triggers the corresponding protocol process, working in conjunction with the replica management layer and the rebuild engine layer to coordinate data migration or replica quantity adjustment operations.

[0051] The rebuild engine layer is the system's data repair and rebalancing unit. It is activated to perform rebuild tasks when the replica management layer reports that the number of replicas for a data unit is below a configured threshold, or when the device management layer notifies that a new device has joined and data rebalancing is required. It reads data from surviving replicas, recalculates redundancy information, writes the data to a new target location, and updates the replica mapping table.

[0052] The original bcache read / write path has been modified as follows: upon receiving a read / write request, the replica mapping table is first queried to obtain the location of one or more replicas of the target data. Then, based on the read / write type and the configured consistency level, one or more replicas are selected for access operations.

[0053] The system implementing this invention consists of two parts: a modified Linux kernel bcache module and a supporting user space management toolset. The deployment process includes the following key steps:

[0054] First, the bcache kernel module patch, which includes the improvements made in this invention, is integrated into the target Linux kernel source code, and the new kernel is recompiled and installed. This step ensures that the underlying operating system supports enhanced copy management and automatic rebuilding capabilities. Next, the accompanying user-space management tools are installed, providing a user-friendly command-line interface and configuration management functionality.

[0055] Use enhanced management commands to create cache device groups that support replication. For example, the command format for creating a three-replica mirrored cache system is as follows: By specifying parameters such as replication mode, redundancy policy, number of replicas, cache device list, and backend devices, the system automatically initializes the device group, establishes replica mapping relationships, and formats a unified block device for use by upper-layer applications. The entire configuration process is highly automated, reducing the possibility of manual intervention and configuration errors.

[0056] Example 2: Replica Redundancy Mechanism and Read / Write Process.

[0057] This embodiment describes in detail the copy redundancy mechanism of the present invention, as well as the specific process of data writing and reading.

[0058] like Figure 2 As shown, this invention extends the disk superblock and memory index structure of bcache. On the disk, each cache device, in addition to storing the original superblock information, also adds a storage device group descriptor, which contains a group unique identifier, a list of devices, configured redundancy policies and parameters, etc.

[0059] In memory, the core data structure is a distributed replica mapping table. This table uses a two-level index design: the first level is the mapping from the original B+ tree key to the "replica set descriptor"; the second level is the "replica set descriptor" itself, which is a dynamic array containing multiple "replica entries". Each "replica entry" records detailed replica information, including device identifier, physical address, replica status (valid / obsolete / under writing), version number or timestamp, etc.

[0060] Taking a synchronous dual-replica mirror as an example, the specific steps of the data writing process are as follows (see Figure 2After the upper-layer application initiates a write request, bcache determines the cache key and corresponding bucket to which the data should be written based on its existing logic. Then, the replica management layer intervenes, querying the replica mapping table based on the key to find the two corresponding target devices and their physical addresses. The system simultaneously dispatches the write request to the driver queues of both target devices; these two dispatch operations are parallel and atomic. The system waits for both devices' write operations to complete before marking the write operation as successful, updating the status of these two replica entries in the replica mapping table to valid, and incrementing the version number. If either write fails, the system will retry according to a preset strategy or mark the replica as downgraded, potentially triggering a rebuild process.

[0061] like Figure 3 As shown, the data reading process enables intelligent failover: After the upper-layer application initiates a read request, the system queries the replica mapping table based on the key to obtain the location information of all valid replicas. The system adopts a configurable read strategy, with the default strategy being "priority read," which means initiating a read request to the replica considered to have the lowest latency. If this replica does not respond or returns an error within a preset time, a read request is immediately initiated to another replica, achieving rapid failover. Data is returned upon successful read. If all replicas fail to read, it is considered a cache miss, and the system switches to reading from the slower backend device, potentially triggering an emergency rebuild of the cache unit.

[0062] The redundancy strategies supported by this invention include multiple modes: N-replica mirroring mode completely replicates the data N times and stores it on N different devices, which can tolerate the simultaneous failure of up to N-1 devices and features high read and write performance but low storage efficiency; erasure coding mode (such as N+M) divides the data into N equal-length data blocks, generates M check blocks through encoding calculations, and these blocks are distributed and stored on different devices, which can tolerate the loss of up to M arbitrary blocks, with high storage efficiency but large computational overhead; local replica mode combines the above two strategies, using multiple replica mirroring for key small data such as metadata and indexes, and using erasure coding for large batches of user data, to achieve an optimized balance between reliability, performance and cost.

[0063] Example 3: Online Device Replacement and Data Migration Protocol

[0064] This embodiment describes the specific process for adding new equipment online, removing old equipment in a planned manner, and automatically handling faulty equipment.

[0065] like Figure 4As shown, the online addition process for a new device includes the following steps: The device management layer discovers the new device through kernel events or management tool commands. The system verifies whether its capacity and performance meet the requirements of the device group and assigns it a unique device identifier. The new device is officially added to the member list of the device group, but its initial status is set to "joining". At this time, the replica mapping table has not yet allocated any data to it. Then, the rebuild engine layer is triggered, starting a background "rebalancing" task. This task scans the current replica mapping table and calculates which existing data replicas should be migrated to the new device to optimize load distribution based on the configured data distribution strategy. The rebalancing task reads data from the source device, writes it to the new device, and atomically updates the replica mapping table in small data blocks, adding the new device as the new replica position for that data unit. It can also optionally delete an old replica. When the rebalancing task is completed, or when the amount of data on the new device reaches a stable state, the device status changes from "joining" to "online", and it officially begins to receive new read and write requests.

[0066] The planned removal process for legacy devices follows these steps: An administrator initiates a removal command, or the system automatically triggers it based on a predictive maintenance alarm, changing the target device's status to "Draining". The replica management layer stops assigning any new data replicas to the device. The rebuild engine layer initiates a migration task, scanning the replica mapping table to identify all replicas located on the device. For each replica, the system creates a new replica on other online devices and updates the mapping table; this process is incremental and performed in the background. Once all data replicas have been migrated from the device, the system sets its status to "Drained". The device management layer then removes it from the device group member list and notifies the underlying driver that the device is safe to unload. The entire drainage process is conducted online, and read / write requests from upper-layer applications are transparently redirected to the new data location.

[0067] The automated fault recovery process for faulty equipment achieves automated fault recovery: It comprehensively assesses equipment faults through multiple mechanisms, including timeouts in device I / O operations, failures in small sentinel read / write tests, device driver errors, or physical link interruptions. Once a fault is confirmed, the device management layer immediately marks the device as "faulty," and all new I / O requests to that device are immediately routed to other available replicas. The replica management layer traverses the replica mapping table, marking all replica entries on the faulty device as invalid or downgraded, and the system calculates the actual number of available replicas for each data unit. For data units with fewer available replicas than the configured minimum safety threshold, the reconstruction engine adds them to a high-priority reconstruction queue. For other data units with a safe number of replicas, reconstruction may be delayed or performed on demand. After physically replacing the faulty device, the new device will act as a replacement, integrating into the system through the "new device online addition process," and prioritizing the reconstruction of high-priority data units.

[0068] Example 4: Automatic Reconstruction Engine Design.

[0069] This embodiment describes in detail the intelligent scheduling mechanism, adaptive flow control, and breakpoint resume function of the automatic reconstruction engine.

[0070] like Figure 5 As shown, the reconstruction engine employs an intelligent scheduling mechanism, maintaining multiple priority queues (such as urgent, high, medium, and low). The urgent queue is used for data reconstruction when the number of replicas is below a safe threshold; the high, medium, and low queues are used for rebalancing tasks caused by device additions or removals, or for preventative replica replenishment. The scheduler dynamically determines which queue to retrieve tasks from for execution based on queue priority and the current system load (such as CPU utilization and I / O latency), ensuring that critical business operations are not affected.

[0071] The system implements adaptive flow control to ensure that the reconstruction process operates as a "good neighbor," not preempting resources required for critical business inputs and outputs. The engine continuously monitors key metrics such as system-level input / output latency, device-level input / output queue depth, and CPU idle rate. Based on these metrics, it dynamically adjusts the number of reconstruction worker threads, the size of each data block read, and the latency between requests through a feedback control loop. When the business load increases, the reconstruction speed is automatically reduced; when the system is idle, it rebuilds at full speed, achieving an optimal balance in resource utilization.

[0072] The reconstruction task is broken down into numerous independent subtasks, each responsible for reconstructing a single data unit. These subtasks can be executed in parallel by multiple kernel threads, fully utilizing the bandwidth of multi-core CPUs and multiple storage devices. The progress of each subtask is persistently recorded in a dedicated log. If the system crashes or restarts during the reconstruction process, the reconstruction engine can resume execution from the last recorded breakpoint without starting from scratch, ensuring the reliability and efficiency of the reconstruction process.

[0073] Example 5: Consistency, Reliability and Performance Assurance.

[0074] This embodiment describes the configurable consistency levels, data integrity protection mechanisms, and performance optimization measures supported by the present invention.

[0075] This invention supports configurable consistency levels, including strong consistency and eventual consistency modes. In strong consistency mode, write operations are only confirmed after persistence on all configured replicas, ensuring that the latest data is obtained from any replica; this is the system's default operating mode. In eventual consistency mode, write operations are confirmed after persistence on the primary replica, with other replicas updating asynchronously in the background. This provides higher write performance, but there is a possibility of reading old data from different replicas within a short time window, making it suitable for scenarios with relatively relaxed consistency requirements.

[0076] In terms of fault recovery and data integrity, the system implements a multi-layered protection mechanism. It employs write operation log technology, recording changes to a persistent log before updating critical metadata such as the replica mapping table, preventing metadata inconsistencies caused by system crashes. A cyclic redundancy checksum is stored for each data block, verified during read and reconstruction processes to ensure data integrity and detect quiescent data corruption. A background data inspection mechanism is implemented, periodically scanning all data replicas, calculating and comparing checksums, proactively identifying and repairing data corruption caused by underlying storage media issues, achieving preventative maintenance.

[0077] To minimize the impact on the original system's performance, this invention employs several optimization techniques. On critical data read / write paths, lock-free techniques such as read-copy-update are used to access the replica mapping table, avoiding performance bottlenecks caused by lock contention. Operations such as device status monitoring, partial metadata updates, and synchronization of non-critical replicas are designed for asynchronous execution, without blocking the main input / output path, ensuring the responsiveness of front-end services. The rebuild engine uses an independent work queue and an adjustable thread pool, isolated from the thread pool handling front-end service requests, achieving effective partitioning and priority management of computing resources.

[0078] Example 6: Typical operation and maintenance example.

[0079] This embodiment provides two specific operation and maintenance examples to illustrate the practical application of the present invention.

[0080] Scenario 1: Replacing a Degraded Cache Device. When an administrator discovers an abnormally high latency in an NVMe cache device through monitoring tools, an online replacement operation can be performed. First, a secure removal process is initiated via management commands. The system sets the target device's status to "draining" and migrates its data replicas to other devices within the device group in the background. The administrator can monitor the migration progress in real time using status query commands. After the data migration is complete, the old device is physically removed and the new storage device is inserted. Next, a device addition command is executed to add the new device to the cache pool. The system incorporates the new device into the device group and triggers a background rebalancing process to restore the data distribution to an optimized state. Throughout the entire operation, the database service mounted on the cache device remains completely transparent, ensuring business continuity.

[0081] Scenario 2: Handling Sudden Cache Device Failure. When the system detects a cache device failure via I / O timeout, the device management layer automatically marks it as faulty and immediately routes all read / write requests to the remaining healthy replica devices. The rebuild engine starts simultaneously, prioritizing the rebuilding of data units with insufficient replicas due to the failure. It recalculates the missing replicas using the remaining replicas and writes them to other healthy devices within the group. Administrators will receive a system alert but do not need to respond immediately; they can replace the faulty hardware at a convenient time. After hardware replacement, a standard new device addition process completes full recovery. This automated fault handling significantly reduces the mean time to recovery and improves system availability.

[0082] Example 7: Monitoring and Diagnosis.

[0083] This embodiment describes the monitoring and diagnostic interface provided by the present invention.

[0084] This invention extends the sysfs interface, adding richer runtime information output to the existing bcache monitoring system. The system exposes multi-dimensional monitoring data, including replica status, policy configuration, rebuild progress, and degradation object statistics, under a dedicated file system path.

[0085] The replica status interface displays detailed status information for each device in the device group, including device identifier, operating status, role in the device group, and health assessment. The policy configuration interface displays currently active redundancy policies and all their parameter settings, facilitating administrator verification of configuration correctness. The rebuild progress interface provides detailed information on currently ongoing rebuild or rebalancing tasks, including overall progress percentage, real-time data transfer rate, and estimated completion time, helping administrators monitor system recovery status. The degraded object statistics interface displays the number of data objects whose current replica count is lower than the expected configuration, a key indicator for assessing system risk level.

[0086] User space management tools can parse this rich runtime information and provide comprehensive monitoring dashboards, either graphically or via command line. These tools can also integrate with enterprise-level monitoring systems (such as Prometheus) to achieve centralized monitoring, alerting, and performance analysis, meeting the management needs of large-scale production environments.

[0087] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0088] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for online replacement and automatic reconstruction of a bcache caching system, characterized in that, include: In the bcache kernel module, a replica management layer, a device management layer, and a rebuild engine layer are added, and a distributed replica mapping table is constructed. The replica mapping table records the replica location, status, and version information of each cached data unit on multiple cache devices. In the write operation path, the replica management layer distributes the write data in parallel to multiple replicas of multiple cache devices according to the replica mapping table, and waits for all configuration replicas to be persisted before confirming the write success. In the read operation path, the replica management layer obtains multiple replica locations based on the replica mapping table and reads data from available replicas using a failover method. The device management layer monitors the status of cache devices. When a change in device status is detected, it triggers online addition, online removal, or fault handling processes, and the reconstruction engine layer performs data migration or reconstruction tasks. The reconstruction engine layer dynamically adjusts the priority, parallelism, and rate of reconstruction tasks based on preset redundancy strategies and the current system load, and supports breakpoint resumption of reconstruction tasks.

2. The online replacement and automatic reconstruction method for the bcache caching system according to claim 1, characterized in that, The distributed replica mapping table adopts a two-level index structure: the first level is the mapping from the original bcache B+ tree key to the replica set descriptor, and the second level is the replica set descriptor, which contains multiple replica entries. Each replica entry records the device identifier, physical address, replica status, and version number.

3. The online replacement and automatic reconstruction method for the bcache cache system according to claim 1, characterized in that, The redundancy strategy includes at least one of the following: N-replica mirroring mode, which copies the data completely N times and stores it on N different devices; erasure coding mode, which divides the data into N data blocks and calculates M check blocks, and stores them distributed across different devices; and local replica mode, which uses multi-replica mirroring for metadata and erasure coding for user data.

4. The online replacement and automatic reconstruction method for the bcache caching system according to claim 1, characterized in that, It also includes configurable consistency levels: in strong consistency mode, write operations are only confirmed after they are persisted on all configured replicas; in eventual consistency mode, write operations are confirmed after they are persisted on the primary replica, and other replicas are updated asynchronously in the background.

5. The online replacement and automatic reconstruction method for the bcache cache system according to claim 1, characterized in that, The online joining process includes: after the device management layer discovers a new device, it sets its status to joining, triggers the rebuild engine layer to start a rebalancing task, which scans the replica mapping table, migrates existing data replicas to the new device and atomically updates the replica mapping table, and changes the status of the new device to online after completion.

6. The online replacement and automatic reconstruction method for the bcache cache system according to claim 1, characterized in that, The online removal process includes: in response to a planned removal instruction, setting the target device status to "draining", stopping the replica management layer from assigning new replicas to it, rebuilding the engine layer to start a migration task, migrating all replicas on the device to other online devices and updating the mapping table, and after all migrations are completed, setting the device status to "drained" and removing it from the device group.

7. The online replacement and automatic reconstruction method for the bcache cache system according to claim 1, characterized in that, The fault handling process includes: after the device management layer confirms the device fault through IO timeout, sentinel read / write test failure or driver report error, the device status is marked as faulty. All IO requests to the device are immediately routed to other available replicas. The replica management layer marks the replica items on the faulty device as invalid. For data units with fewer available replicas than the safety threshold, the reconstruction engine adds them to the high-priority reconstruction queue.

8. The online replacement and automatic reconstruction method for the bcache cache system according to claim 1, characterized in that, The reconstruction engine layer maintains multiple priority queues, including urgent, high, medium, and low queues. The scheduler dynamically determines the task execution order based on queue priority and the current system load. Furthermore, the reconstruction engine layer continuously monitors system-level I / O latency, device-level I / O queue depth, and CPU idle ratio. Through feedback control loops, it dynamically adjusts the number of reconstruction worker threads, data block size, and request interval to achieve adaptive flow control.

9. A bcache caching system online replacement and automatic reconstruction system, characterized in that, include: The replica management layer is used to maintain a global distributed replica mapping table, record the physical replica location, status and version information of each cached data unit on multiple cache devices, and execute the configured redundancy policy. During write operations, write data is distributed to multiple replicas in parallel, and during read operations, an available replica is selected from multiple replicas for failover reading. The device management layer is used to continuously monitor the status of all cache devices, manage the relationship between device pool members, and trigger the corresponding online addition, online removal or fault handling protocol when a device status change is detected. The reconstruction engine layer is used to perform data reconstruction or rebalancing tasks when the replica management layer reports that the number of replicas is lower than the configured threshold or the device management layer notifies the device to be added or removed. It also dynamically adjusts the priority, parallelism, and rate of reconstruction tasks according to the system load and supports breakpoint resumption. The system is an enhancement of the Linux kernel bcache module, retaining the original application programming interface and management toolchain.

10. The online replacement and automatic reconstruction system for the bcache caching system according to claim 9, characterized in that, The replica management layer uses read-copy-update lock-free technology to access the replica mapping table on the critical path of data read and write; the device management layer designs device status monitoring and some metadata updates to be executed asynchronously; the reconstruction engine layer uses an independent work queue and an adjustable thread pool, isolated from the thread pool that processes front-end business requests.

Images