Object storage service system

Abstract

An embodiment of the present invention relates to an object storage service system, including: a Gateway Server component, an object storage management module, multiple object storage node servers, and an Etcd storage database; the Gateway Server component is connected to an object storage interface and is used to receive an object storage request sent by the object storage interface; the object storage management module is connected to the Gateway Server component and is used to receive the object storage request transmitted by the Gateway Server component; the object storage management module is connected to the multiple object storage node servers and is used to manage the multiple object storage node servers. Thus, the system can support Openstack Swift interface and AWS Swift interface, support EC erasure code and replica management, use strong consistency to ensure data consistency, and introduce the Etcd storage database as cluster management and database management, thereby improving the function, performance and maintainability of the object storage system.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of object storage, and in particular, to an object storage service system. Background Art

[0002] In the field of distributed storage, Openstack Swift and AWS S3 are two different object storage interfaces, which are widely used in the storage field. In the object storage service, Ceph provides three block, file and object storage services: Ceph RBD, Ceph FS and RGW. RGW is an object storage service that supports the API interfaces of Swift and AWSS3 and provides two different object storage interfaces. In addition, in the open source object storage service, Openstack Swfit is an object storage service, which is an object storage management in the Openstack system and provides the Swfit API interface; Minio is an object storage management management compatible with AWS S3.

[0003] Ceph's RGW object storage service is compatible with Swfit and AWS S3 interfaces. However, since Ceph's RBD and Ceph FS use a unified Librados wrapper, it is difficult to optimize object storage in terms of performance, usability, etc. At the same time, the entire Ceph is also limited by the overall Ceph system in deployment and use, and the entire framework is bloated; Openstack Swift can only provide Swift interfaces and adopts an eventual consistency model, which cannot guarantee strong data consistency in some scenarios; Minio is an object storage service written in Go language. Minio only supports AWS S3 object storage service, which only supports EC erasure codes but not replicas, and is functionally missing. Summary of the invention

[0004] In view of this, in order to solve the above technical problems or part of the technical problems, an embodiment of the present invention provides an object storage service system.

[0005] In a first aspect, an embodiment of the present invention provides an object storage service system, including: a Gateway Server component, an object storage management module, a plurality of object storage node servers, and an Etcd storage database;

[0006] The Gateway Server component is connected to the object storage interface and is used to receive the object storage request sent by the object storage interface;

[0007] The object storage management module is connected to the Gateway Server component and is used to receive the object storage request transmitted by the Gateway Server component;

[0008] The object storage management module is connected to the multiple object storage node servers and is used to manage the multiple object storage node servers.

[0009] Optionally, the system further includes: an authentication server;

[0010] The authentication server is used to receive the account authentication request sent by the Gateway Server component and authenticate the access rights of the account.

[0011] The optional object storage management module includes: a container management module, an account management module, and an object management module;

[0012] The container management module is used to manage the status information of the container;

[0013] The account management module is used to manage the status information of the account;

[0014] The object management module is used to manage storage objects.

[0015] Optionally, the Etcd storage database is connected to the container management module, the account management module and the object management module;

[0016] The Etcd storage database is used to receive account status information, container status information, and metadata information of storage objects.

[0017] Optionally, the object management module is used to manage the physical disk information of the multiple object storage node servers, and uses a distributed storage method to store the object information to be stored in the target physical disk.

[0018] Optionally, the system further includes: an AWS S3 module, used for being compatible with the AWS S3 object storage interface;

[0019] The AWS S3 object storage interface is connected to the Gateway Server component and is used to send an AWS S3 object storage request to the Gateway Server component.

[0020] Optionally, the system further includes: an Openstack Swift module, used for being compatible with the Openstack Swift object storage interface;

[0021] The Openstack Swift object storage interface is connected to the Gateway Server component and is used to send an Openstack Swift object storage request to the Gateway Server component.

[0022] Optionally, the system further includes: an Ext4 journal file system;

[0023] The Ext4 log file system is connected to the Etcd storage database and is used to store the account status information, container status information and metadata information of the storage object sent by the Etcd storage database.

[0024] Optionally, the system further includes: a user terminal;

[0025] The user client sends an object storage request through the AWS S3 object storage interface and the Openstack Swift object storage interface.

[0026] The object storage service system provided by the embodiment of the present invention includes: a Gateway Server component, an object storage management module, multiple object storage node servers, and an Etcd storage database; the Gateway Server component is connected to the object storage interface and is used to receive the object storage request sent by the object storage interface; the object storage management module is connected to the Gateway Server component and is used to receive the object storage request transmitted by the Gateway Server component; the object storage management module is connected to the multiple object storage node servers and is used to manage the multiple object storage node servers. Therefore, the system can support Openstack Swift interface and AWS Swift interface, support EC erasure code and replica management, use strong consistency to ensure data consistency, and introduce the Etcd storage database as cluster management and database management to improve the function, performance and maintainability of the object storage system. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a schematic diagram of the architecture of the object storage service system. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions.

[0030] Although the embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents, and the embodiments do not constitute limitations on the embodiments of the present invention.

[0031] To facilitate understanding of the embodiments of the present invention, specific embodiments will be further explained below in conjunction with the accompanying drawings. The embodiments do not constitute a limitation on the embodiments of the present invention.

[0032] Figure 1 This is a schematic diagram of the architecture of the object storage service system. Figure 1 As shown, the object storage service system includes: a Gateway Server component, an object storage management module (Object Storage Manager), multiple object storage node servers, and an Etcd storage database, wherein the Gateway Server component is connected to the object storage interface and is used to receive object storage requests sent by the object storage interface; the object storage management module is connected to the Gateway Server component and is used to receive object storage requests transmitted by the Gateway Server component; the object storage management module is connected to multiple object storage node servers and is used to manage multiple object storage node servers.

[0033] Furthermore, the object storage service system also includes: an authentication server, which is used to receive the account authentication request sent by the Gateway Server component and authenticate the access rights of the account. The Gateway Server component receives the object storage request sent by the user end according to the AWS S3 and Openstak swift interfaces provided by the object storage service system, and first sends the authentication to the Authentication Server to authenticate the user end's access rights, supporting Openstack Keystone authentication; after meeting the conditions, it is sent to the object storage management module according to the interface request.

[0034] Optionally, the object storage management module includes: a container management module, an account management module, and an object management module; the container management module is used to manage the status information of the container; the account management module is used to manage the status information of the account; and the object management module is used to manage the storage objects.

[0035] The object storage management module manages the information of all object storage node servers, and maintains the online and offline functions of the object storage node servers. It manages the status information of other object storage management modules through the Etcd storage database as a cluster discovery function. According to the access provided by the Gateway Server component, the three modules of container management module (Container Manager), account management module (Account Manager), and object management module (ObjectManager) can be set separately. If it is account creation, deletion, query and other information, it will be sent to Account Manager account management, if it is container creation, deletion, query and other information, it will be sent to Container Manager container management module.

[0036] Optionally, the object management module is also used to manage the physical disk information of multiple object storage node servers, and use a distributed storage method (for example, a DHT distributed storage method) to store the object information to be stored in a target physical disk (Disk).

[0037] Optionally, the Etcd storage database is connected to the container management module, the account management module and the object management module; the Etcd storage database is used to receive account status information, container status information and metadata information of storage objects.

[0038] Optionally, the object storage service system also includes: an Ext4 log file system, which is connected to the Etcd storage database and is used to store account status information, container status information, and metadata information of storage objects sent by the Etcd storage database.

[0039] Optionally, the object storage service system also includes: an AWS S3 module, which is used to be compatible with the AWS S3 object storage interface, wherein the AWS S3 object storage interface is connected to the Gateway Server component and is used to send an AWS S3 object storage request to the Gateway Server component.

[0040] Optionally, the object storage service system also includes: an Openstack Swift module, which is used to be compatible with the OpenstackSwift object storage interface, wherein the Openstack Swift object storage interface is connected to the Gateway Server component and is used to send an Openstack Swift object storage request to the Gateway Server component.

[0041] Optionally, the object storage service system further includes: a user end, which sends an object storage request to the Gateway Server component through the AWS S3 object storage interface and the Openstack Swift object storage interface.

[0042] The object storage service system provided by the embodiment of the present invention includes: a Gateway Server component, an object storage management module, multiple object storage node servers, and an Etcd storage database; the Gateway Server component is connected to the object storage interface and is used to receive the object storage request sent by the object storage interface; the object storage management module is connected to the Gateway Server component and is used to receive the object storage request transmitted by the Gateway Server component; the object storage management module is connected to the multiple object storage node servers and is used to manage the multiple object storage node servers. Therefore, the system can support Openstack Swift interface and AWS Swift interface, support EC erasure code and replica management, use strong consistency to ensure data consistency, and introduce the Etcd storage database as cluster management and database management to improve the function, performance and maintainability of the object storage system.

Claims

1. An object storage service system, characterized in that: include: Gateway Server components, object storage management module, multiple object storage node servers, and Etcd storage database; The Gateway Server component is connected to the object storage interface and is used to receive the object storage request sent by the object storage interface; The object storage management module is connected to the Gateway Server component and is used to receive the object storage request transmitted by the Gateway Server component; The object storage management module is connected to the multiple object storage node servers and is used to manage the multiple object storage node servers; The object storage management module includes: a container management module, an account management module, and an object management module; The container management module is used to manage the status information of the container; The account management module is used to manage the status information of the account; The object management module is used to manage storage objects; The Etcd storage database is connected to the container management module, the account management module and the object management module; The Etcd storage database is used to receive account status information, container status information and metadata information of storage objects; The object management module is used to manage the physical disk information of the multiple object storage node servers, and uses a distributed storage method to store the object information to be stored in the target physical disk.

2. The system according to claim 1, characterized in that The system also includes: an authentication server; The authentication server is used to receive the account authentication request sent by the Gateway Server component and authenticate the access rights of the account.

3. The system according to claim 1, characterized in that The system also includes: an AWS S3 module for being compatible with the AWS S3 object storage interface; The AWS S3 object storage interface is connected to the Gateway Server component and is used to send an AWS S3 object storage request to the Gateway Server component.

4. The system according to claim 1, characterized in that The system further includes: an Openstack Swift module, which is used to be compatible with the Openstack Swift object storage interface; The Openstack Swift object storage interface is connected to the Gateway Server component and is used to send an Openstack Swift object storage request to the Gateway Server component.

5. The system according to claim 1, characterized in that The system also includes: an Ext4 journal file system; The Ext4 log file system is connected to the Etcd storage database and is used to store the account status information, container status information and metadata information of the storage object sent by the Etcd storage database.

6. The system according to claim 3 or 4, characterized in that: The system further comprises: a user terminal; The user client sends an object storage request through the AWS S3 object storage interface and the Openstack Swift object storage interface.

Images