Online simplified migration method of Ceph RBD type volume based on OpenStack

Abstract

The invention discloses an online simplified migration method of a Ceph RBD type volume based on OpenStack, and relates to the technical field of OpenStack cloud computing, and the method comprises the following steps: a user requests to modify the type of an in-use volume on a virtual machine through a Cinder API, and provides a UUID, a new type and a migration strategy; the cinder volume service judges whether a new volume needs to be created or not and notifies the nova-api to carry out copy switching; the nova-api verifies the UUID and queries the computing node, and the new volume information is sent to the nova-component service; the nova-complex service configures a deject-zeros option according to the drive list, and the configuration is modified to be off after volume migration is completed; and finally, calling the Cinder API to mount the new volume and unload the old volume. According to the invention, after the volume type switching is completed, the storage space actually occupied by the virtual block device is not obviously increased, and the purpose of saving space is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of OpenStack cloud computing, and in particular to an online slim migration method of a CephRBD type volume based on OpenStack. Background Art

[0002] OpenStack, as a leader in the field of cloud computing, has developed rapidly since its birth and has become the first choice for private cloud solutions. It is not only an open source cloud management platform software, but also the preferred solution in private cloud scenarios. By using OpenStack to manage and optimize public cloud and private cloud resources, enterprises can provide services such as virtual machines, virtual disks, and virtual VPCs, improve data center resource utilization, and reduce costs.

[0003] Under the management of OpenStack, the virtual disk management service (cinder) is the core of virtual disk management. It not only supports online switching of volume types, allowing volumes to be seamlessly converted from one type to another, but also allows online migration of volumes. Online volume migration mainly occurs when the volume backend is switched or the volume type is switched. By copying data to the new volume in real time, it ensures that the virtual machine continues to run after data synchronization. At the same time, volume simplification is also an important feature, which allows thick volumes to be converted to thin volumes. Thick volumes refer to virtual block devices with a capacity equal to the actual storage capacity allocated to them, while the initial capacity of thin volumes is less than the actual allocated storage capacity and grows dynamically according to write demand. The advantage of this mechanism is that it can provide users with virtual disk space that nominally exceeds the actual physical storage capacity.

[0004] However, the current online volume type switching technology faces a challenge: it depends on whether the storage device supports thin writes. Thin writes mean that when zeros are written to the end of a block device, the storage device does not actually allocate new storage capacity. When using Ceph as a storage device, even if zeros are written to the end of a logical block device, the corresponding storage space is still allocated. This results in a significant increase in the storage space occupied by the virtual disk before and after the volume type switch. Summary of the invention

[0005] In view of the needs and shortcomings of current technology development, the present invention provides an online slim migration method for CephRBD type volumes based on OpenStack, which ensures that after the volume type is switched, the storage space actually occupied by the virtual block device will not increase significantly, thereby achieving the purpose of saving space.

[0006] The present invention provides an online slim migration method for Ceph RBD type volumes based on OpenStack, and the technical solution adopted to solve the above technical problems is as follows:

[0007] An online thin migration method for Ceph RBD type volumes based on OpenStack, comprising the following steps:

[0008] S1. The user selects an in-use volume mounted on a virtual machine and initiates a request to modify the volume type through the Cinder API. The request includes the UUID of the selected volume, the new volume type, and the volume migration policy.

[0009] S2. Cinder API finds the backend storage and sends a request to the cinder volume service to determine whether the new and old volume types correspond to the same backend. If they are the same, the database record is directly updated. Otherwise, a new volume is created and nova-api is notified to perform volume copy and switch operations.

[0010] S3, nova-api verifies the UUID and queries the compute node where the virtual machine is located, and sends the new volume information and its correspondence with the selected volume to the nova-compute service;

[0011] S4. The nova-compute service adds the detect-zeros option to the API of the virtual machine manager's volume online copy and switch process;

[0012] S5. The storage backend corresponding to the driver in the driver list to be processed cannot support simplified volume migration. The nova-compute service determines whether the driver of the new volume is in the driver list to be processed. If so, the detect-zeroes option is not configured during volume online copy and switching. If not, the detect-zeroes option is configured during volume online copy and switching.

[0013] S6. When the driver of the new volume is in the list of drivers to be processed, further query the version of the virtual machine manager and determine whether the version meets the requirements. a) If the requirements are met, configure the detect-zeroes attribute of the virtual disk to on; b) If the requirements are not met, do not configure the detect-zeroes option when performing online copy and switch of the volume.

[0014] S7, the nova-compute service waits for the virtual machine manager to complete the volume migration operation, and then calls the method of the virtual machine manager to change the detect-zeroes configuration of the corresponding disk to off;

[0015] S8. The nova-compute service calls the Cinder API interface to complete the mounting of the new volume and notifies the cinder service to unmount the selected volume.

[0016] Optionally, in step S1, after the user initiates a request through the Cinder API, the Cinder API service performs the following checks:

[0017] Whether the UUIDD of the selected volume is valid and exists in the database;

[0018] Whether the new volume type exists in the database;

[0019] Whether the volume migration policy is "Do not perform migration" or "Perform volume migration";

[0020] After the inspection passes, continue with the subsequent operations.

[0021] Optionally, the step S2 involved specifically includes:

[0022] The Cinder API finds the corresponding backend storage based on the UUID of the volume selected by the user and sends the request to the corresponding cinder volume service backend for processing;

[0023] The cinder volume service first determines whether the new volume type requested by the user corresponds to the same backend as the current type of the selected volume.

[0024] a) If yes, directly modify the volume record in the database, update the volume type in the volume record to the new volume type requested by the user, and return success.

[0025] b) If not, the backend migration is performed. The cinder volume service creates a new volume of the same type as the new volume type requested by the user, and then sends the UUID of the selected volume, virtual machine, and new volume to the nova-api service to notify it to perform the volume copy and switch operations.

[0026] Optionally, the step S3 involved specifically includes:

[0027] nova-api verifies the UUIDs of the selected volume, virtual machine, and new volume;

[0028] After the verification is passed, the compute node where the virtual machine is located is queried based on the UUID of the virtual machine, and the new volume information and its correspondence with the selected volume are sent to the nova-compute service of the compute node.

[0029] Optionally, nova-api verifies the UUIDs of the selected volume, virtual machine, and new volume. The specific verification operations include:

[0030] Confirm that the UUID of the virtual machine is a valid UUID string and the virtual machine exists;

[0031] Verify that the selected volume and the new volume exist.

[0032] Optionally, step S4 is performed, during the copying process, the virtual machine manager reads the data segment of the selected volume and determines whether its content is all 0:

[0033] a) If yes, call the discard command of the virtual machine manager to release the part of the content with the given offset and length in the new volume;

[0034] b) If not, execute the write logic.

[0035] Optionally, execute step S7, the nova-compute service calls the virtual machine manager method to modify the detect-zeroes configuration of the corresponding disk to off, thereby turning off the storage device's detection of whether the written data is 0, so that when the virtual machine's application system writes new data, the virtual machine manager no longer determines whether the data is all zero.

[0036] Optionally, when the cinder service unmounts the selected volume, perform the following steps:

[0037] The cinder service verifies the parameters passed in to ensure that the UUID, the selected volume, and the new volume information are legal;

[0038] After confirming that the parameters are legal, the cinder service copies the metadata information of the selected volume except the UUID to the new volume;

[0039] The cinder service deletes the selected volume, including removing the data and metadata of the selected volume from the storage system and releasing related resources;

[0040] After deleting the selected volume, the cinder service continues to empty the contents of the selected volume;

[0041] The cinder service changes the UUID of the new volume to the UUID of the selected volume.

[0042] The online slim migration method of Ceph RBD type volume based on OpenStack of the present invention has the following beneficial effects compared with the prior art:

[0043] 1. The present invention realizes that after the volume type is switched, the storage space actually occupied by the virtual block device will not increase significantly, thereby achieving the purpose of saving space. It can support the maintenance of the streamlined characteristics of the new volume after the volume online migration is completed, thereby improving the migration speed and saving the use of resources; it is applicable to a variety of volume types and migration strategies, provides a flexible volume management method, and can be expanded and adjusted according to different needs;

[0044] 2. The present invention can effectively reduce the waste of storage space through online streamlined migration; the entire migration process is carried out online without interrupting the service of the virtual machine. Users can complete the conversion of volume types without downtime, ensuring the continuity and stability of the business; by verifying the UUID and querying the computing node, the safety and accuracy of the operation can be ensured. At the same time, the inspection of the new volume driver and the virtual machine manager version can ensure the compatibility and reliability of the migration process; according to the driver list and the virtual machine manager version, the detect-zeros option can be configured to optimize the migration performance if supported, and further improve the migration efficiency; after the new volume is mounted, the selected volume will be unmounted, releasing resources that are no longer used, which is helpful to better manage system resources and improve resource utilization; because the detect-zeros option is used in the migration process, it can ensure that the data is consistent before and after the migration, avoiding problems caused by data inconsistency;

[0045] 3. The present invention is compatible with different versions of virtual machine managers and different types of drivers, and no processing is required for drivers that already support streamlined migration; this saves backend storage resource usage, avoids the situation where previously migrated sparse volumes also occupy the same capacity as the virtual size of the volume, saves storage resources, and reduces the operating costs of the data center. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Attached Figure 1 It is a flow chart of the method of embodiment 1 of the present invention. DETAILED DESCRIPTION

[0047] In order to make the technical solution, the technical problem solved and the technical effect of the present invention more clearly understood, the technical solution of the present invention is clearly and completely described below in conjunction with specific embodiments.

[0048] The English words involved in the embodiments are explained as follows:

[0049] OpenStack: is an open source cloud computing management platform project that provides a set of APIs for building and managing public and private clouds. OpenStack can control a large number of computing resources, network resources, and storage resources, and provide users with a graphical interface through the dashboard to manage and control these resources.

[0050] Ceph: is a distributed object storage system that provides high-performance, high-availability, and high-scalability storage solutions. Ceph can be used as block storage, file storage, or object storage.

[0051] RBD: RBD stands for Rados Block Device, which is a block storage interface provided by Ceph. It allows users to create and manage virtual disks on Ceph clusters, which can be used by the operating system like local disks.

[0052] Cinder API: is a set of RESTful APIs that define the specific methods and paths for interacting with Cinder services. It provides developers with a programming interface that enables them to interact with Cinder services directly through code.

[0053] Cinder API service: As the front-end interface of the OpenStack block storage service, it is responsible for receiving external requests and processing them. These requests include operations such as volume creation, deletion, and mounting. The Cinder API service usually includes three main components: cinder-api, cinder-scheduler, and cinder-volume. These components work together to implement the complete process from receiving API requests to completing volume operations.

[0054] Embodiment 1:

[0055] Combined with Figure 1 This embodiment proposes an online thin migration method for Ceph RBD type volumes based on OpenStack, which includes the following steps:

[0056] S1. The user selects an in-use volume mounted on a virtual machine and initiates a request to modify the volume type through the Cinder API. The request includes the UUID of the selected volume, the new volume type, and the volume migration policy.

[0057] The Cinder API service then performs the following checks:

[0058] i) Whether the UUIDD of the selected volume is valid and exists in the database;

[0059] ii) whether the new volume type exists in the database;

[0060] iii) Whether the volume migration policy is "Do not perform migration" or "Perform volume migration"; it should be added that: not performing volume migration means that the volume does not need to change the backend when switching from the previous volume type to the new volume type, while for the new volume type corresponding to the different types of volume backends, it is necessary to select the "Perform volume migration" option.

[0061] After the inspection passes, continue with the subsequent operations.

[0062] S2. Cinder API finds the corresponding backend storage based on the UUID of the volume selected by the user and sends the request to the corresponding cinder volume service backend for processing;

[0063] The cinder volume service first determines whether the new volume type requested by the user corresponds to the same backend as the current type of the selected volume:

[0064] a) If yes, directly modify the volume record in the database, update the volume type in the volume record to the new volume type requested by the user, and return success;

[0065] b) If not, a backend migration is performed. The cinder volume service creates a new volume as the target volume. The type of the new volume is the same as the new volume type requested by the user. The UUID of the selected volume, the virtual machine, and the new volume is then sent to the nova-api service to notify it to perform the volume copy and switch operations.

[0066] S3, nova-api verifies the UUIDs of the selected volume, virtual machine, and new volume, including: confirming that the UUID of the virtual machine is a legal UUID string and that the virtual machine exists, and confirming that the selected volume and new volume exist;

[0067] After the verification is passed, the compute node where the virtual machine is located is queried based on the UUID of the virtual machine, and the new volume information and its correspondence with the selected volume are sent to the nova-compute service of the compute node.

[0068] S4. The nova-compute service adds the detect-zeros option to the API of the virtual machine manager's volume online copy and switching process.

[0069] During the copy process, the virtual machine manager reads the data segment of the selected volume and determines whether its content is all zeros:

[0070] a) If yes, call the discard command of the virtual machine manager to release the part of the content with the given offset and length in the new volume;

[0071] b) If not, execute the write logic.

[0072] S5. The storage backend corresponding to the driver in the driver list to be processed cannot support simplified volume migration. The nova-compute service determines whether the driver of the new volume is in the driver list to be processed:

[0073] If so, do not configure the detect-zeroes option when performing volume online copy and switch;

[0074] If not, configure the detect-zeroes option when performing volume online copy and switchover.

[0075] S6. When the driver of the new volume is in the list of drivers to be processed, further query the version of the virtual machine manager and determine whether the version meets the requirements:

[0076] a) If the requirements are met, the detect-zeroes attribute of the virtual disk is configured to be on; the purpose of configuring it to be on is to use the capability of the virtual machine manager mentioned in step S4, and when the input data is all zero, call the write-zero method of the virtual machine manager to release the data in the storage area corresponding to the new volume, thereby saving storage space;

[0077] b) If the requirements are not met, do not configure the detect-zeroes option when performing online volume copy and switch.

[0078] It should be noted that common virtual machine managers such as QEMU provide a QMP API, which enables the cloud platform management program to obtain the current virtual machine manager version of the virtual machine.

[0079] S7. The nova-compute service waits for the virtual machine manager to complete the volume migration operation, and then calls the method of the virtual machine manager to change the detect-zeroes configuration of the corresponding disk to off, turning off the storage device's detection of whether the written data is 0. When the application system of the virtual machine writes new data, the virtual machine manager no longer determines whether the data is all zero, thereby saving computing resources of the computing node.

[0080] S8. The nova-compute service calls the Cinder API interface to complete the mounting of the new volume and notifies the cinder service to unmount the selected volume.

[0081] When the cinder service unmounts the selected volume, it performs the following steps:

[0082] (i) The cinder service verifies the parameters passed in. The parameters passed in refer to the data information that the cinder service needs to receive and process when performing the uninstallation operation, which usually includes the UUID, the selected volume, and the new volume information. The verification determines that the UUID, the selected volume, and the new volume information are legal;

[0083] (ii) After confirming that the parameters are legal, the cinder service copies the metadata information of the selected volume except the UUID to the new volume;

[0084] (iii) The cinder service deletes the selected volume, including removing the data and metadata of the selected volume from the storage system and releasing related resources;

[0085] (iv) After deleting the selected volume, the cinder service continues to clear the contents of the selected volume;

[0086] (v) The cinder service changes the UUID of the new volume to the UUID of the selected volume.

[0087] In summary, by adopting the online slim migration method of Ceph RBD type volume based on OpenStack of the present invention, after the volume type is switched, the storage space actually occupied by the virtual block device will not increase significantly, thereby achieving the purpose of saving space, and supporting the maintenance of the slim feature of the new volume after the volume online migration is completed, thereby improving the migration speed and saving the use of resources.

[0088] The present invention is applicable to a variety of volume types and migration strategies, provides a flexible volume management method, and can be expanded and adjusted according to different requirements.

[0089] The above specific examples are used to explain the principles and implementation methods of the present invention in detail. These examples are only used to help understand the core technical content of the present invention. Based on the above specific embodiments of the present invention, any improvements and modifications made by technicians in this technical field without departing from the principles of the present invention should fall within the scope of patent protection of the present invention.

Claims

1. An online thin migration method for Ceph RBD type volumes based on OpenStack, characterized in that: The steps include: S1. The user selects an in-use volume mounted on a virtual machine and initiates a request to modify the volume type through the Cinder API. The request includes the UUID of the selected volume, the new volume type, and the volume migration policy. S2. Cinder API finds the backend storage and sends a request to the cinder volume service to determine whether the new and old volume types correspond to the same backend. If they are the same, the database record is directly updated. Otherwise, a new volume is created and nova-api is notified to perform volume copy and switch operations. S3, nova-api verifies the UUID and queries the compute node where the virtual machine is located, and sends the new volume information and its correspondence with the selected volume to the nova-compute service; S4. The nova-compute service adds the detect-zeros option to the API of the virtual machine manager's volume online copy and switch process; S5. The storage backend corresponding to the driver in the driver list to be processed cannot support simplified volume migration. The nova-compute service determines whether the driver of the new volume is in the driver list to be processed. If so, the detect-zeroes option is not configured during volume online copy and switching. If not, the detect-zeroes option is configured during volume online copy and switching. S6. When the driver of the new volume is in the list of drivers to be processed, further query the version of the virtual machine manager and determine whether the version meets the requirements. a) If the requirements are met, configure the detect-zeroes attribute of the virtual disk to on; b) If the requirements are not met, do not configure the detect-zeroes option when performing online copy and switch of the volume. S7, the nova-compute service waits for the virtual machine manager to complete the volume migration operation, and then calls the method of the virtual machine manager to change the detect-zeroes configuration of the corresponding disk to off; S8. The nova-compute service calls the Cinder API interface to complete the mounting of the new volume and notifies the cinder service to unmount the selected volume.

2. According to claim 1, a Ceph RBD type volume online thin migration method based on OpenStack is characterized in that: In step S1, after the user initiates a request through the Cinder API, the Cinder API service performs the following checks: Whether the UUIDD of the selected volume is valid and exists in the database; Whether the new volume type exists in the database; Whether the volume migration policy is "Do not perform migration" or "Perform volume migration"; After the inspection passes, continue with the subsequent operations.

3. According to the method for online thin migration of Ceph RBD type volumes based on OpenStack in claim 1, it is characterized in that: The step S2 specifically includes: The Cinder API finds the corresponding backend storage based on the UUID of the volume selected by the user and sends the request to the corresponding cindervolume service backend for processing; The cinder volume service first determines whether the new volume type requested by the user corresponds to the same backend as the current type of the selected volume. a) If yes, directly modify the volume record in the database, update the volume type in the volume record to the new volume type requested by the user, and return success. b) If not, the backend migration is performed. The cinder volume service creates a new volume of the same type as the new volume type requested by the user, and then sends the UUID of the selected volume, virtual machine, and new volume to the nova-api service to notify it to perform the volume copy and switch operations.

4. The online thin migration method of Ceph RBD type volume based on OpenStack according to claim 1, characterized in that: The step S3 specifically includes: nova-api verifies the UUIDs of the selected volume, virtual machine, and new volume; After the verification is passed, the compute node where the virtual machine is located is queried based on the UUID of the virtual machine, and the new volume information and its correspondence with the selected volume are sent to the nova-compute service of the compute node.

5. The online thin migration method of Ceph RBD type volume based on OpenStack according to claim 4, characterized in that: nova-api verifies the UUIDs of the selected volume, virtual machine, and new volume. The specific verification operations include: Confirm that the UUID of the virtual machine is a valid UUID string and the virtual machine exists; Verify that the selected volume and the new volume exist.

6. The online thin migration method of Ceph RBD type volume based on OpenStack according to claim 1, characterized in that: Execute step S4. During the copy process, the virtual machine manager reads the data segment of the selected volume and determines whether its content is all 0: a) If yes, call the discard command of the virtual machine manager to release the part of the content with the given offset and length in the new volume; b) If not, execute the write logic.

7. The online thin migration method of Ceph RBD type volume based on OpenStack according to claim 1, characterized in that: Execute step S7, the nova-compute service calls the method of the virtual machine manager, and changes the detect-zeroes configuration of the corresponding disk to off, that is, the storage device is turned off to detect whether the written data is 0, so that when the application system of the virtual machine writes new data, the virtual machine manager no longer determines whether the data is all zero.

8. The online thin migration method of Ceph RBD type volume based on OpenStack according to claim 1, characterized in that: When the cinder service unmounts the selected volume, it performs the following steps: The cinder service verifies the parameters passed in to ensure that the UUID, the selected volume, and the new volume information are legal; After confirming that the parameters are legal, the cinder service copies the metadata information of the selected volume except the UUID to the new volume; The cinder service deletes the selected volume, including removing the data and metadata of the selected volume from the storage system and releasing related resources; After deleting the selected volume, the cinder service continues to empty the contents of the selected volume; The cinder service changes the UUID of the new volume to the UUID of the selected volume.

Images