CEPH block equipment snapshot deletion system, electronic equipment and storage medium

Abstract

The invention provides a CEPH block equipment snapshot deletion system, electronic equipment and a storage medium. The system comprises a CEPH management cluster, an Mgr layer and an OSD layer, the CEPH management cluster is responsible for maintaining snapshot metadata and providing an Mgr main selection service at the same time; the Mgr is responsible for managing a snapshot deletion task; and the OSD layer is responsible for processing a snapshot deletion request issued by the Mgr and executing snapshot data cleaning work. According to the method, the execution rate of the background snapshot deletion task can be controlled, excessive pressure on a system is avoided, meanwhile, the deletion granularity of the snapshot object is controlled, and the influence on service request time delay is reduced.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a CEPH block device snapshot deletion system, an electronic device, and a storage medium. Background Art

[0002] CEPH Figure 3 The shown figure is an open source distributed storage system. As a powerful, flexible and scalable distributed storage system, it has been widely used and recognized in various application scenarios.

[0003] Snapshot technology is a key feature in Ceph for data protection, backup, and recovery. Ceph snapshots allow users to create a read-only copy of file system, block device, or object storage data at a certain moment in time. These snapshots provide an efficient way to backup and restore data without significantly affecting the running system.

[0004] However, snapshot deletion in current block device usage scenarios still faces some problems and challenges:

[0005] The current snapshot object deletion granularity is the entire object. The deletion process involves multiple metadata read and write operations, which in turn increases the latency of business requests.

[0006] Snapshot deletion is spread to all OSD services in the storage cluster through OSDMap change requests, and then deleted asynchronously by the OSD service background. This asynchronous method will affect the cluster business and cannot be effectively controlled.

[0007] Therefore, how to provide a CEPH block device snapshot deletion system, electronic device and storage medium has become a technical problem that needs to be solved urgently in the field. Summary of the invention

[0008] The purpose of the present invention is to provide a CEPH block device snapshot deletion system, electronic equipment and storage medium.

[0009] The first aspect of the present invention discloses a CEPH block device snapshot deletion system; the system comprises: a CEPH management cluster, an Mgr and an OSD layer;

[0010] The CEPH management cluster is responsible for maintaining snapshot metadata and providing Mgr master election services;

[0011] The Mgr is responsible for the management of snapshot deletion tasks;

[0012] The OSD layer is responsible for processing the snapshot deletion request sent by the Mgr and performing the cleanup of the snapshot data.

[0013] According to the system of the first aspect of the present invention, the management of the snapshot deletion task by the Mgr is completed by the rbd_snap_support module, which specifically includes:

[0014] Get the deleted snapshot information according to OSDMap;

[0015] Select the PG list of snapshots to be deleted and set the maximum number of PGs that can concurrently delete snapshots;

[0016] Send a kick_trim request to the primary OSD where the PG is located, with the request parameter carrying the snapshot information to be deleted, so that the PG enters the snap trim state machine;

[0017] After PG completes snapshot deletion, it notifies Mgr to update the status.

[0018] According to the system of the first aspect of the present invention, the PG list of the snapshot to be deleted is selected including:

[0019] Select the PG list of snapshots to be deleted based on the parameter max_concurrent_snap_trim_pgs and the load of the cluster OSD.

[0020] According to the system of the first aspect of the present invention, the OSD layer performs the cleaning of snapshot data including:

[0021] Mark the snapshots that need to be deleted in the internal data structure;

[0022] After marking is completed, the OSD layer will enter the Snap Trim state machine;

[0023] The Snap Trim state machine enters the initialization phase and checks the snapshot to be deleted and its associated objects;

[0024] The OSD layer enters the snapshot deletion preparation phase, and the Snap Trim state machine constructs a list of objects to be deleted, which includes all objects related to the snapshot to be deleted;

[0025] Entering the actual deletion phase, the Snap Trim state machine gradually deletes the objects in the object list;

[0026] When all objects are deleted, the Snap Trim state machine enters the completion phase;

[0027] The OSD layer updates the status of the snapshot, marks the deleted snapshot as deleted, and notifies the Mgr.

[0028] According to the system of the first aspect of the present invention, the checking of the snapshot to be deleted and its associated objects comprises:

[0029] The OSD layer scans the snapshot metadata and identifies all objects and data blocks associated with the snapshot.

[0030] According to the system of the first aspect of the present invention, the Snap Trim state machine gradually deletes the objects in the object list, including:

[0031] The step-by-step deletion is performed in multiple small steps;

[0032] In each of the small steps, the OSD layer deletes a predefined portion of objects. Specifically, the deletion of each object is performed in a data sharding manner, that is, only a predefined portion of data of the object is deleted each time, and then it sleeps for a period of time.

[0033] According to the system of the first aspect of the present invention, the deletion operation of deleting the predefined part object includes:

[0034] Delete data object: delete the actual data object associated with the snapshot;

[0035] Update metadata: Update metadata to remove references to deleted objects;

[0036] Resource recycling: Reclaim the storage space occupied by deleted objects.

[0037] According to the system of the first aspect of the present invention, when all objects are deleted, the Snap Trim state machine enters the completion phase, including:

[0038] OSD layer cleanup deletes temporary data generated during the snapshot process and ensures metadata consistency.

[0039] In a second aspect, the present application shows an electronic device, which includes: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the method described in any of the above aspects.

[0040] In a third aspect, the present application shows a non-temporary computer-readable storage medium, when the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method described in any of the above aspects.

[0041] In a fourth aspect, the present application illustrates a computer program product. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device is enabled to execute the method described in any of the above aspects.

[0042] The technical solution provided by the present application may include the following beneficial effects: being able to control the execution rate of background snapshot deletion tasks to avoid excessive pressure on the system, while controlling the deletion granularity of snapshot objects to reduce the impact on service request latency. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0044] Figure 1 A structural diagram of a CEPH block device snapshot deletion system provided according to an embodiment;

[0045] Figure 2 A workflow diagram for cleaning up snapshot data, in which the OSD layer is responsible for processing the snapshot deletion request sent by the Mgr according to an embodiment of the present invention;

[0046] Figure 3 CEPH architecture diagram according to an embodiment of the present invention;

[0047] Figure 4 is a block diagram of an electronic device according to an embodiment of the present invention;

[0048] Figure 5 It is a block diagram of a storage medium shown in this application. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0050] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0051] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0052] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0053] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0054] Glossary:

[0055] PG: The full name is Placement Group. Placement group is an important concept used for data distribution and management in Ceph. Each object belongs to a placement group. Ceph achieves data distribution and load balancing by distributing data to different placement groups.

[0056] CEPH: is an open source distributed storage system that originated in 2004 and was developed by Inktank and contributed to the open source community. It is a unified storage system that supports block storage, file storage, and object storage, with excellent performance, reliability, and scalability.

[0057] Embodiment 1:

[0058] The first aspect of the present invention discloses a CEPH block device snapshot deletion system. Figure 1 This is a structural diagram of a CEPH block device snapshot deletion system according to an embodiment of the present invention. The present application designs a new snapshot deletion system, which can control the execution rate of background snapshot deletion tasks to avoid excessive pressure on the system, and at the same time control the deletion granularity of snapshot objects to reduce the impact on service request latency; the present application first proposes a snapshot deletion system that can be applied to the Ceph block storage system, which can effectively control the impact of a large number of snapshot deletions in the cluster on normal services. The entire snapshot deletion system device is composed of three parts, from top to bottom:

[0059] 1. Ceph manages the cluster, is responsible for maintaining snapshot metadata, and provides Mgr master election services.

[0060] 2.Mgr is responsible for the management of snapshot deletion tasks. In order to achieve high availability, a one-master-multiple-standby deployment is generally adopted, and only the master Mgr provides services at the same time.

[0061] 3. The OSD layer is responsible for processing snapshot deletion requests issued by the Mgr and performing snapshot data cleanup.

[0062] Specific as Figure 1 As shown, the system includes: a CEPH management cluster, an Mgr and an OSD layer;

[0063] The CEPH management cluster is responsible for maintaining snapshot metadata and providing Mgr master election services;

[0064] The Mgr is responsible for the management of snapshot deletion tasks;

[0065] Mgr stands for Manager Daemon, which is an important component in the CEPH cluster. It is mainly used to monitor and manage the status and performance of the cluster. It complements the functions of the CEPH Monitor (MON) node and provides more monitoring and management capabilities for the cluster.

[0066] OSD is a storage service responsible for storing and managing data. The full name of OSD is Object Storage Device. In the computer field, it refers to a device based on object storage technology. Object-based storage is a new network storage architecture that combines the advantages of storage area network (SAN) and network attached storage (NAS), providing a high-performance, high-reliability, cross-platform and secure data sharing storage architecture.

[0067] The OSD layer is responsible for processing the snapshot deletion request sent by the Mgr and performing the cleanup of the snapshot data.

[0068] The Mgr is responsible for the management of snapshot deletion tasks. In order to achieve high availability, a one-master-multiple-standby deployment is generally adopted, and only the master Mgr provides services at the same time.

[0069] In some embodiments, the management of snapshot deletion tasks by the Mgr is completed by the rbd_snap_support (snapshot management) module, specifically including:

[0070] Get the deleted snapshot information according to OSDMap;

[0071] OSDMap is used to manage and track all OSD daemons in the cluster as well as their status and location information. It also records user metadata such as storage pools and snapshots.

[0072] Select the PG list of snapshots to be deleted and set the maximum number of PGs that can concurrently delete snapshots;

[0073] Send a kick_trim request to the primary OSD where the PG is located, with the request parameter carrying the snapshot information to be deleted, so that the PG enters the snap trim state machine;

[0074] After PG completes snapshot deletion, it notifies Mgr to update the status.

[0075] The PG list of snapshots to be deleted includes:

[0076] Select the PG list of snapshots to be deleted based on the parameter max_concurrent_snap_trim_pgs (the maximum number of concurrent snapshot deletions) and the load of the cluster OSD.

[0077] The OSD layer is responsible for processing the snapshot deletion request sent by the Mgr and performing the cleanup of the snapshot data.

[0078] In some embodiments, Figure 2 As shown in the figure, the OSD layer performs the following tasks to clean up snapshot data:

[0079] Mark the snapshots that need to be deleted in an internal data structure; this is to ensure that the system knows which snapshots to delete before the actual deletion operation begins;

[0080] After marking is completed, the OSD layer will enter the Snap Trim state machine; this state machine is an asynchronously executed task responsible for managing the entire process of snapshot deletion;

[0081] Snap Trim in the field of computer storage, especially when using the Ceph storage system, refers to an optimization operation used to free up data block space that is no longer in use.

[0082] Specifically, Snap Trim is a form of the TRIM command that allows the operating system to inform the solid-state drive (SSD) which data blocks are no longer needed and can be internally erased. This operation helps the SSD to perform garbage collection more efficiently, reducing future write operations to these blocks, thereby increasing write speeds and extending the life of the SSD.

[0083] The Snap Trim state machine enters the initialization phase and checks the snapshot to be deleted and its associated objects;

[0084] The OSD layer enters the snapshot deletion preparation phase, and the Snap Trim state machine constructs a list of objects to be deleted, which includes all objects related to the snapshot to be deleted;

[0085] Entering the actual deletion phase, the Snap Trim state machine gradually deletes the objects in the object list;

[0086] When all objects are deleted, the Snap Trim state machine enters the completion phase;

[0087] The OSD layer updates the status of the snapshot, marks the deleted snapshot as deleted, and notifies the Mgr.

[0088] The snapshot to be deleted and its associated objects are checked as follows:

[0089] The OSD layer scans the snapshot metadata and identifies all objects and data blocks associated with the snapshot.

[0090] The Snap Trim state machine gradually deletes the objects in the object list including:

[0091] The step-by-step deletion is performed in multiple small steps;

[0092] In each of the small steps, the OSD layer will delete a predefined part of the objects. Specifically, the deletion of each object is performed in a data sharding manner, that is, only a predefined part of the object's data is deleted each time, and then it sleeps for a period of time to avoid excessive pressure on the foreground I / O operation.

[0093] In a preferred implementation, the sleep time can be set in the range of 1-30 minutes. It should be noted that this is only a preferred implementation provided by the present application and should not be construed as a limitation of the present application. In actual applications, it can be reasonably set according to specific circumstances.

[0094] The deletion operations for deleting predefined partial objects include:

[0095] Delete data object: delete the actual data object associated with the snapshot;

[0096] Update metadata: Update metadata to remove references to deleted objects;

[0097] Resource recycling: Reclaim the storage space occupied by deleted objects.

[0098] When all objects are deleted, the Snap Trim state machine enters the completion phase including:

[0099] OSD layer cleanup deletes temporary data generated during the snapshot process and ensures metadata consistency.

[0100] This application is applicable to the block storage system product of ceph. This application can effectively solve the impact of snapshot deletion on normal business by using a new system device, and greatly improve the user experience of the product.

[0101] In summary, the technical solutions of various aspects of the present invention have the following advantages compared with the prior art: effectively controlling the number of PGs of concurrent snapshot deletions in the cluster, thereby reducing the impact on cluster performance and stability, and at the same time reducing the load of a single deletion operation by decomposing a single snapshot object deletion operation into multiple small fragments and deleting them step by step.

[0102] 1) The solution is concise and clear, with a relatively low degree of coupling with the CEPH system, which reduces the complexity of the system and is especially suitable for complex storage systems such as CEPH.

[0103] 2) Flexible deletion policy management: Users can formulate snapshot deletion policies as needed to adapt to different scenarios.

[0104] 3) Adaptive adjustment: Mgr can dynamically schedule snapshot deletion tasks based on OSD load conditions.

[0105] Embodiment 2:

[0106] Based on Example 1, a CEPH block device snapshot deletion system specifically includes:

[0107] 1. The Mgr service adds the rbd_snap_support module. The main logic of this module is:

[0108] a) Get the deleted snapshot information according to OSDMap.

[0109] b) Select the PG list of snapshots to be deleted and set the maximum number of PGs that can concurrently delete snapshots based on the parameter max_concurrent_snap_trim_pgs and the load of the cluster OSD.

[0110] c) Send a kick_trim request to the primary OSD where the PG is located. The request parameter carries the snapshot information to be deleted, allowing the PG to enter the snap trim state machine.

[0111] d) After PG completes snapshot deletion, it notifies Mgr to update the status.

[0112] 2. OSD internal snapshot deletion logic:

[0113] a) Mark for deletion

[0114] When an OSD receives a snapshot deletion request, it first marks the snapshots to be deleted in an internal data structure. This is to ensure that the system knows which snapshots to delete before the actual deletion operation begins.

[0115] b) Enter the Snap Trim state machine

[0116] After marking is completed, the OSD will enter the Snap Trim state machine. This state machine is an asynchronously executed task that manages the entire process of snapshot deletion.

[0117] c) Initialization

[0118] The state machine first enters the initialization phase to check for snapshots to be deleted and their associated objects. During the initialization process, the OSD scans the snapshot metadata and identifies all objects and data blocks associated with the snapshot.

[0119] d) Preparing Trim

[0120] In the prepare-to-delete phase, the OSD prepares the relevant deletion tasks. Specifically, it builds a list of objects to be deleted, which includes all objects related to the snapshot.

[0121] 3. OSD internal snapshot deletion logic:

[0122] a) Trimming Snapshots

[0123] Entering the actual deletion phase, the state machine will gradually delete the objects in the list. In order to reduce the impact on system performance, the deletion operation is usually divided into multiple small steps. In each small step, the OSD will delete a part of the objects. Specifically, the deletion of each object is carried out in a data sharding manner, that is, only a part of the object's data is deleted each time, and then it sleeps for a period of time to avoid excessive pressure on the foreground I / O operation.

[0124] In a preferred implementation, the sleep time can be set in the range of 1-30 minutes. It should be noted that this is only a preferred implementation provided by the present application and should not be construed as a limitation of the present application. In actual applications, it can be reasonably set according to specific circumstances.

[0125] The specific deletion operations include:

[0126] 1. Delete data object: Delete the actual data object associated with the snapshot.

[0127] 2. Update metadata: Update metadata to remove references to deleted objects.

[0128] 3. Resource recycling: Reclaim the storage space occupied by deleted objects.

[0129] f)Completing Trim

[0130] When all related objects are deleted, the state machine enters the completion phase. In this phase, the OSD cleans up the temporary data generated during the deletion process and ensures the consistency of the metadata.

[0131] g) Update status

[0132] Finally, the OSD updates the snapshot status, marks it as deleted, and notifies the Mgr.

[0133] This application proposes for the first time a snapshot deletion system that can be applied to the Ceph block storage system. The system can effectively control the impact of large-scale snapshot deletions in the cluster on normal business. The solution is concise and clear, with a relatively low degree of coupling with the Ceph system, which reduces the complexity of the system, and is especially suitable for complex storage systems such as Ceph. This application can flexibly manage deletion policies: users can formulate snapshot deletion policies as needed to adapt to different scenarios. This application has strong adaptive adjustment capabilities, and the Mgr can dynamically schedule snapshot deletion tasks based on the OSD load.

[0134] Embodiment 3:

[0135] An embodiment of the present application also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0136] Figure 4 The present application shows a block diagram of an electronic device 800. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0137] Reference Figure 4 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0138] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0139] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, images, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0140] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0141] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0142] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0143] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0144] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also detect the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0145] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast operation information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0146] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0147] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by a processor 820 of an electronic device 800 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0148] Embodiment 4:

[0149] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0150] Figure 5 19 is a block diagram of a computer-readable storage medium shown in the present application. For example, the storage medium 1900 may be provided as a server.

[0151] Reference Figure 5, the storage medium 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions, such as an application, that can be executed by the processing component 1922. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0152] The storage medium 1900 may also include a power supply component 1926 configured to perform power management of the storage medium 1900, a wired or wireless network interface 1950 configured to connect the storage medium 1900 to a network, and an input / output (I / O) interface 1958. The storage medium 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.

[0153] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0154] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0155] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0156] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0157] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0158] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0159] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0160] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0161] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0162] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A CEPH block device snapshot deletion system, characterized in that: The system includes: a CEPH management cluster, an Mgr component and an OSD layer; The CEPH management cluster is responsible for maintaining snapshot metadata and providing Mgr master election services; The Mgr is responsible for the management of snapshot deletion tasks; The OSD layer is responsible for processing the snapshot deletion request sent by the Mgr and performing the cleanup of the snapshot data.

2. A CEPH block device snapshot deletion system according to claim 1, characterized in that: The management of snapshot deletion tasks by the Mgr is completed by the rbd_snap_support module, which specifically includes: Get the deleted snapshot information according to OSDMap; Select the PG list of snapshots to be deleted and set the maximum number of PGs that can concurrently delete snapshots; Send a kick_trim request to the primary OSD where the PG is located, with the request parameter carrying the snapshot information to be deleted, so that the PG enters the snap trim state machine; After PG completes snapshot deletion, it notifies Mgr to update the status.

3. A CEPH block device snapshot deletion system according to claim 2, characterized in that: The PG list of snapshots to be deleted includes: Select the PG list of snapshots to be deleted based on the parameter max_concurrent_snap_trim_pgs and the load of the cluster OSD.

4. A CEPH block device snapshot deletion system according to claim 1, characterized in that: The OSD layer performs the following tasks to clean up snapshot data: Mark the snapshots that need to be deleted in the internal data structure; After marking is completed, the OSD layer will enter the Snap Trim state machine; The Snap Trim state machine enters the initialization phase and checks the snapshot to be deleted and its associated objects; The OSD layer enters the snapshot deletion preparation phase, and the Snap Trim state machine constructs a list of objects to be deleted, which includes all objects related to the snapshot to be deleted; Entering the actual deletion phase, the Snap Trim state machine gradually deletes the objects in the object list; When all objects are deleted, the Snap Trim state machine enters the completion phase; The OSD layer updates the status of the snapshot, marks the deleted snapshot as deleted, and notifies the Mgr.

5. A CEPH block device snapshot deletion system according to claim 4, characterized in that: The snapshot to be deleted and its associated objects are checked as follows: The OSD layer scans the snapshot metadata and identifies all objects and data blocks associated with the snapshot.

6. A CEPH block device snapshot deletion system according to claim 4, characterized in that: The Snap Trim state machine gradually deletes the objects in the object list including: The step-by-step deletion is performed in multiple small steps; In each of the small steps, the OSD layer deletes a predefined portion of objects. Specifically, the deletion of each object is performed in a data sharding manner, that is, only a predefined portion of data of the object is deleted each time, and then sleeps for a set period of time.

7. A CEPH block device snapshot deletion system according to claim 6, characterized in that: The deletion operations for deleting predefined partial objects include: Delete data object: delete the actual data object associated with the snapshot; Update metadata: Update metadata to remove references to deleted objects; Resource recycling: Reclaim the storage space occupied by deleted objects.

8. A CEPH block device snapshot deletion system according to claim 4, characterized in that: When all objects are deleted, the Snap Trim state machine enters the completion phase including: OSD layer cleanup deletes temporary data generated during the snapshot process and ensures metadata consistency.

9. An electronic device, characterized in that: The system comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, a method in a CEPH block device snapshot deletion system as claimed in any one of claims 1 to 8 is executed.

10. A storage medium, characterized in that: The computer program stored in the storage medium can be executed by one or more processors and can be used to implement the method in the CEPH block device snapshot deletion system as described in any one of claims 1 to 8.

Images