Data download method, data upload method and distributed storage system

Abstract

The present invention discloses a data download method, an upload method and a distributed storage system. The method comprises: a first object storage gateway receives an access request; the first object storage gateway obtains a target file identifier in the access request; the first object storage gateway determines a target relational database corresponding to the target file identifier from a relational database cluster, and queries metadata of a target file corresponding to the target file identifier from the target relational database, wherein the metadata is description information of the target file; the first object storage gateway determines a target distributed storage cluster corresponding to the metadata and a second object storage gateway in the target distributed cluster from a plurality of distributed storage clusters corresponding to the first object storage gateway; the first object storage gateway forwards the access request to the second object storage gateway in the target distributed cluster to obtain the target file from the second object storage gateway.

Description

Technical Field

[0001] The present invention relates to the field of data storage, and in particular to a data downloading method, an uploading method and a distributed storage system. Background Art

[0002] Ceph (distributed file system) is a very popular open source distributed storage system with a good design concept. Its architectural design avoids single point failure of each component and provides good scalability and performance. As a PB-level object storage system, it can be used in production environments without much improvement in the underlying architecture. However, there are scalability issues in the existing Ceph system, and the performance of Ceph will be affected when a large number of files are stored.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present invention provide a data download method, an upload method and a distributed storage system to at least solve the technical problem that the number of files stored in each RGW bucket is too large and the performance is affected due to the metadata of the files being stored through the RGW.

[0005] According to one aspect of an embodiment of the present invention, a data download method is provided, which is applied to a distributed storage system, wherein the distributed storage system includes: a first object storage gateway, a distributed storage cluster and a relational database cluster, and the method includes: the first object storage gateway receives an access request; the first object storage gateway obtains a target file identifier in the access request; the first object storage gateway determines a target relational database corresponding to the target file identifier from the relational database cluster, and queries metadata of the target file corresponding to the target file identifier from the target relational database, wherein the metadata is description information of the target file; the first object storage gateway determines a target distributed storage cluster corresponding to the metadata and a second object storage gateway in the target distributed cluster from a plurality of distributed storage clusters corresponding to the first object storage gateway; the first object storage gateway forwards the access request to the second object storage gateway in the target distributed cluster to obtain the target file from the second object storage gateway.

[0006] Optionally, the distributed storage system includes: a load balancing gateway; and the first object storage gateway receives an access request, including: the first object storage gateway receives the access request forwarded from the load balancing gateway.

[0007] Optionally, before the first object storage gateway receives an access request forwarded from a load balancing gateway, the method also includes: the load balancing gateway receives multiple access requests; selecting the first object storage gateway from a preset gateway list according to a preset weight, wherein the preset weight is the weight of each gateway in the preset gateway list, and the greater the weight, the greater the number of access requests allocated for parallel processing.

[0008] Optionally, the first object storage gateway determines the target distributed storage cluster corresponding to the metadata and the second object storage gateway in the target distributed cluster from multiple distributed storage clusters corresponding to the first object storage gateway, including: the first object storage gateway determines the target distributed storage cluster corresponding to the metadata from the cluster information table in the target relational database; determines the second object storage gateway from the metadata information table in the target relational database.

[0009] Optionally, the cluster information table includes at least one of the following: cluster identification, cluster read / write status, cluster address information; the metadata information table includes at least one of the following: cluster identification where the target file is located, file information of the target file, file write status, and file size.

[0010] Optionally, the readable and writable state of the cluster and the write state of the file are determined in the following manner: the relational database in the relational database cluster modifies the readable and writable state of the cluster and the write state of the file in response to a modification request of the target object; wherein, when a trigger event is detected, an operation corresponding to the modified readable and writable state of the cluster and the write state of the file is triggered to be performed on the file stored in the second object storage gateway.

[0011] According to another aspect of an embodiment of the present invention, a data uploading method is provided, which is applied to a distributed storage system, wherein the distributed storage system includes: a first object storage gateway, a distributed storage cluster and a relational database cluster, and the method includes: the first object storage gateway receives a file upload request; the first object storage gateway obtains metadata of a file to be uploaded corresponding to the file upload request; the first object storage gateway sends the metadata to the relational database cluster for storage, and sends the data to be uploaded to a second object storage gateway in the distributed storage cluster.

[0012] Optionally, there are multiple file upload requests; before sending the data to be uploaded to the second object storage gateway in the distributed storage cluster, the method also includes: the load balancing gateway receives multiple file upload requests; selects the first object storage gateway from the preset gateway list according to the preset weight, wherein the preset weight is the weight of each gateway in the preset gateway list, and the larger the weight, the larger the number of file upload requests allocated for parallel processing.

[0013] Optionally, the method also includes: when the upload type of the file to be uploaded is multi-slice upload, the first object storage gateway obtains the first type of metadata from the file upload request; and after all the segments corresponding to the file to be uploaded are uploaded, sending a request message to the second object storage gateway, and receiving the second type of metadata corresponding to the request message fed back by the second object storage gateway; when the upload type of the file to be uploaded is uploaded in the form of a form, obtaining the first type of metadata from the data to be uploaded.

[0014] Optionally, the first type of metadata includes: a cluster identifier corresponding to the file to be uploaded, file information of the file to be uploaded, and a writing status of the file; the second type of metadata includes: a file size.

[0015] According to another aspect of an embodiment of the present invention, a distributed storage system is provided, which includes: a first object storage gateway, a distributed storage cluster and a relational database cluster; wherein the first object storage gateway is used to receive an access request; obtain a target file identifier in the access request; determine a target relational database corresponding to the target file identifier from the relational database cluster, and query metadata of the target file corresponding to the target file identifier from the target relational database, wherein the metadata is description information of the target file; determine a target distributed storage cluster corresponding to the metadata and a second object storage gateway in the target distributed cluster from multiple distributed storage clusters corresponding to the first object storage gateway; and forward the access request to the second object storage gateway in the target distributed cluster to obtain the target file from the second object storage gateway; the relational database cluster is used to store the metadata; and the distributed storage cluster is used to store the target file.

[0016] According to another aspect of an embodiment of the present invention, a data download device is provided, which is applied to a first object storage gateway in a distributed storage system, the distributed storage system including: a first object storage gateway, a distributed storage cluster and a relational database cluster, the device including: a first processing module, used to receive an access request; an acquisition module, used to acquire a target file identifier in the access request; a first search module, used to determine a target relational database corresponding to the target file identifier from the relational database cluster, and query metadata of the target file corresponding to the target file identifier from the target relational database, wherein the metadata is description information of the target file; a second search module determines a target distributed storage cluster corresponding to the metadata and a second object storage gateway in the target distributed cluster from a plurality of distributed storage clusters corresponding to the first object storage gateway; and the second processing module forwards the access request to the second object storage gateway in the target distributed cluster to acquire the target file from the second object storage gateway.

[0017] According to another aspect of an embodiment of the present invention, a data upload device is provided, characterized in that the device is applied to a first object storage gateway in a distributed storage system, the distributed storage system includes: a first object storage gateway, a distributed storage cluster and a relational database cluster, the device includes: a processing module, used to receive a file upload request; an acquisition module, used to obtain metadata of a file to be uploaded corresponding to the file upload request; a sending module, used to send the metadata to the relational database cluster for storage, and send the data to be uploaded to the second object storage gateway in the distributed storage cluster.

[0018] According to another aspect of an embodiment of the present invention, a nonvolatile storage medium is provided. The nonvolatile storage medium includes a stored program, wherein when the program is running, a device where the nonvolatile storage medium is located is controlled to execute a data download method.

[0019] In an embodiment of the present invention, a first object storage gateway, a distributed storage cluster and a relational database cluster are used, and the method includes: the first object storage gateway receives an access request; the first object storage gateway obtains a target file identifier in the access request; the first object storage gateway determines a target relational database corresponding to the target file identifier from the relational database cluster, and queries metadata of the target file corresponding to the target file identifier from the target relational database, wherein the metadata is description information of the target file; the first object storage gateway determines a target distributed storage cluster corresponding to the metadata and a second object storage gateway in the target distributed cluster from a plurality of distributed storage clusters corresponding to the first object storage gateway; the first object storage gateway forwards the access request to the second object storage gateway in the target distributed cluster, obtains the target file from the second object storage gateway, and stores the metadata in the relational database cluster, thereby achieving the purpose of avoiding storing the metadata of the file in the RGW, thereby achieving the technical effect of improving database performance, and further solving the technical problem that the number of files stored in each RGW bucket is too large and the performance is affected due to the metadata of the file being stored in the RGW.

[0020] In addition, since the capacity expansion is performed in units of Ceph clusters in the embodiment of the present invention, the problem of data balance caused by capacity expansion is avoided. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0022] Figure 1 It is an architectural diagram of a distributed storage system according to the relevant technology;

[0023] Figure 2 is an architecture diagram of a distributed storage system according to an embodiment of the present application;

[0024] Figure 3 is a flow chart of a data downloading method according to an embodiment of the present application;

[0025] Figure 4 is a flow chart of a data uploading method according to an embodiment of the present application;

[0026] Figure 5 is a structural schematic diagram of a data downloading device according to an embodiment of the present application;

[0027] Figure 6 is a structural schematic diagram of a data uploading device according to an embodiment of the present application;

[0028] Figure 7 The present invention is a flowchart of a storage gateway processing a request according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] In order to better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application are briefly described as follows:

[0032] Metadata, also known as intermediary data or relay data, is data about data. It is mainly information describing data properties, used to support functions such as indicating storage location, historical data, resource search, and file records. Metadata is a kind of electronic catalog. In order to achieve the purpose of cataloging, it is necessary to describe and collect the content or characteristics of the data, thereby achieving the purpose of assisting data retrieval.

[0033] CRUSH is an algorithm based on pseudo-random control of data distribution and replication. CRUSH is designed to solve the problem of balanced data distribution, load, maximizing system performance, system expansion and hardware fault tolerance. In a Ceph cluster, CRUSH requires a concise and clearly hierarchical device description, including storage clusters and replica placement strategies, so as to effectively map data objects to large storage devices, and this process is completed in a distributed manner.

[0034] Kubernetes, or K8s for short, is an open source containerized application management platform for multiple hosts. Kubernetes provides a mechanism for application deployment, planning, updating, and maintenance by deploying containers. Specifically, each container is isolated from each other, each container has its own file system, and the processes between containers will not affect each other, and computing resources can be distinguished. Compared with virtual machines, containers can be deployed quickly. In addition, because containers are decoupled from the underlying facilities and machine file systems, containers can be migrated between different clouds and different versions of operating systems.

[0035] OpenResty is a software platform based on nginx and various extension modules. Since OpenResty is packaged with lua / luajit, you can use lua scripts on the openresty platform and use nginx's asynchronous non-blocking function to use lua to asynchronously and concurrently access backend relational database clusters, such as MySQL, PostgreSQL, Memcached, Redis, and other services; you can also reduce the number of browser http connections and asynchronously and concurrently access backend Java / PHP / Python and other interfaces.

[0036] Sharding-Sphere is an ecosystem composed of a set of open source distributed database middleware solutions, which consists of three parts: Sharding-JDBC, Sharding-Proxy and Sharding-Sidecar. Sharding-Sphere can provide standardized data sharding, distributed transactions and database governance functions, and can be applied to various application scenarios such as Java isomorphism, heterogeneous languages, containers, cloud native, etc. In addition, Sharding-Sphere is positioned as a relational database middleware, which is used to fully and reasonably utilize the computing and storage capabilities of relational databases in distributed scenarios, rather than realizing a new relational database. It coexists with NoSQL and NewSQL rather than being mutually exclusive.

[0037] Example 1

[0038] Figure 1 This is an architecture diagram of a distributed storage system based on related technologies, where Rack in crushmap (data distribution map) corresponds to the rack in the physical environment; Node is the Ceph storage server; RGW is the object storage gateway component of Ceph; Mon is the monitor component of Ceph; OSD is the data storage component of Ceph; Loadbalancer is the external traffic entrance, which reversely proxies external http requests to RGW. It should be noted that each physical server can deploy multiple components of different types at the same time.

[0039] However, this architecture has scalability issues. Generally, there are two ways to expand it: 1. Add Node to each Rack to rebalance the data of the entire cluster according to the crushmap; 2. Add Node to each Rack, create a new Crush Rule, and then use the new Crush Rule to create a new Pool to achieve capacity expansion.

[0040] The above two expansion solutions have the following problems:

[0041] For Solution 1, each capacity expansion will lead to large-scale data migration, which will affect the performance of the cluster and is not suitable for storage environments above PB level. Although Solution 2 avoids data migration and achieves cluster capacity expansion to a certain extent, it does not expand the capacity of the bucket previously created by the user. The user needs to create a new bucket to use the expanded space.

[0042] In addition, affected by the file metadata read and write performance, if the number of files stored in each RGW bucket exceeds 50 million, the performance will be greatly affected.

[0043] In order to solve the above technical problems, an embodiment of the present application provides a new distributed storage system architecture, in which a first object storage gateway is newly added, which is used to receive an access request; obtain a target file identifier in the access request; determine a target relational database corresponding to the target file identifier from a relational database cluster, and query metadata of the target file corresponding to the target file identifier from the target relational database, wherein the metadata is description information of the target file; determine a target distributed storage cluster corresponding to the metadata and a second object storage gateway in the target distributed cluster from multiple distributed storage clusters corresponding to the first object storage gateway; and forward the access request to the second object storage gateway in the target distributed cluster to obtain the target file from the second object storage gateway.

[0044] Specifically, Figure 2 As shown, the distributed storage system provided by the embodiment of the present application includes: a first object storage gateway 20, a distributed storage cluster 22 and a relational database cluster 24; wherein the first object storage gateway 20 is used to receive an access request; obtain a target file identifier in the access request; determine a target relational database 240 corresponding to the target file identifier from the relational database cluster 24, and query metadata of the target file corresponding to the target file identifier from the target relational database 240, wherein the metadata is description information of the target file; determine a target distributed storage cluster 22 corresponding to the metadata and a second object storage gateway 220 in the target distributed cluster 22 from a plurality of distributed storage clusters 22 corresponding to the first object storage gateway 20; and forward the access request to the second object storage gateway 220 in the target distributed cluster 22 to obtain the target file from the second object storage gateway 220; the relational database cluster 24 is used to store metadata; and the distributed storage cluster 22 is used to store target files.

[0045] In some embodiments of the present application, the second object storage gateway 220 may be an RGW, the first object storage gateway 20 may be a load gateway composed of openresty+lua, the target relational database 240 may be a MySQL database, and the distributed storage cluster 22 may be a Ceph cluster.

[0046] In some embodiments of the present application, the workflow of the above distributed storage system when processing a specific request is as follows: Figure 7 As shown. Among them, Figure 7 The flowchart shown describes the 11 stages involved in the operation of the first object storage gateway 20 and the instructions (Order of lua Nginx Module Directives) corresponding to each stage.

[0047] Specifically, the init_by_lua stage and the init_worker_by_lua stage are used to implement the initialization settings of the first object storage gateway 20, wherein the init_by_lua stage is usually used to initialize the global configuration / preload the Lua module; the init_worker_by_lua stage is usually used to periodically pull configuration and data, or perform health checks on upstream services. The first type of object storage gateway 20 periodically obtains Ceph cluster endpoint information and bucket-related information from MySQL at this stage.

[0048] When accessing the distributed storage system through the first object storage gateway 20, the following stages are involved: the ssl_certificate_by_lua stage is usually used to perform special processing on SSL, such as version restrictions; the set_by_lua stage is used to set nginx variables, which can implement complex assignment logic; the rewrite_by_lua stage is used to implement complex forwarding, redirection, caching and other functions (for example, proxying specific requests to the external network); the access_by_lua stage is used to centrally handle IP access, interface permissions and other situations.

[0049] When the target user operates the data in the distributed storage system through the first object storage gateway 20, the first object storage gateway 20 will involve the following working stages: content_by_lua stage, the first object gateway 20 can be regarded as a content processor, which is used to receive request processing and output response. The first type of object storage gateway 20 implements core functions in this stage (such as initiating http requests to the second type of object storage gateway; obtaining file metadata from MySQL, and updating file metadata); balancer_by_lua stage is used for dynamic load balancing; header_filter_by_lua stage is used to customize the response header and cookie of http requests; body_filter_by_lua stage is used to customize the response body of http requests; log_by_lua stage is used to record the number of visits and count the average response time. At present, the first type of object storage gateway writes the file metadata less than 512KB to kafka in this stage, so that the merge program can merge small files based on this, thereby improving the utilization of Ceph storage space.

[0050] In some embodiments of the present application, the first object storage gateway 20 mainly embeds lua code (a scripting language) in the two stages (i.e., two processes) of init_worker_by_lua and content_by_lua to expand its functions. The following is an introduction to the extended functions of the two stages of init_worker_by_lua and content_by_lua.

[0051] Among them, in the init_worker_by_lua stage, the storage gateway mainly obtains the information of the backend ceph cluster from MySQL and passes it to the content_by_lua stage in the form of package variables. The package variable is at the worker level and is shared by all requests processed by the worker.

[0052] In the content_by_lua stage, the storage gateway mainly parses each http request and assigns different access paths to different requests.

[0053] In some embodiments of the present application, in order to achieve elastic scaling of the number of instances of the first object storage gateway 20, the deployment mode of the above distributed storage system is deployment (also an object). The role of this object is to create a POD (also an object, the smallest unit of scheduling) with a built-in controller, thereby creating a gateway instance in a development node.

[0054] In some embodiments of the present application, the above-mentioned distributed storage system uses the ceph cluster as a storage unit when expanding capacity, that is, expands capacity in units of ceph clusters, and when creating a bucket or formulating a bucket deletion policy, and performing access rights setting and authentication processing and other operations on the bucket, the operation content will be broadcast to all Ceph clusters in the backend.

[0055] In some embodiments of the present application, the above-mentioned distributed storage system further includes a load balancing gateway 26 , and the access request received by the first object storage gateway 20 is the access request forwarded from the load balancing gateway 26 .

[0056] When performing load balancing, you can perform load balancing from the L4 or L7 layer, that is, when performing load balancing on the backend server, you can decide how to forward traffic based on the information of the OSI (Open Systems Interconnection reference) layer 4 or layer 7. For example, the load balancing of layer 4 is to determine which traffic needs to be load balanced by publishing the IP address (VIP) of layer 3 and then adding the port number of layer 4, and then perform NAT (Network Address Translation) on the traffic that needs to be processed, forward it to the backend server, and record which server processes this TCP or UDP traffic. All subsequent traffic of this connection is also forwarded to the same server for processing. The load balancing of layer 7 is to consider the characteristics of the application layer on the basis of layer 4. For example, for the load balancing of the same Web server, in addition to determining whether the traffic needs to be processed based on VIP plus port 80, you can also decide whether to perform load balancing based on the URL, browser category, and language of layer 7. For example, if the Web servers are divided into two groups, one for Chinese language and the other for English language, then the seven-layer load balancing can automatically identify the user language when the user visits the domain name, and then select the corresponding language server group for load balancing.

[0057] In some embodiments of the present application, the load balancing gateway 26 may be an L4 load balancer (Loadbalancer), wherein the L4 Loadbalancer may be horizontally scalable.

[0058] In some embodiments of the present application, Figure 2 As shown, and Figure 1 Compared with the distributed storage system shown in FIG. 1 , the distributed storage system described in the embodiment of the present application is managed by Kubernetes.

[0059] exist Figure 2 Under the operating environment shown, an embodiment of the present application provides a method embodiment of a data downloading method. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0060] Figure 3 is a data download method according to an embodiment of the present invention, such as Figure 3 As shown, the method is applied in a distributed storage system, wherein the distributed storage system includes: a first object storage gateway, a distributed storage cluster and a relational database cluster, and includes the following steps:

[0061] Step S302: The first object storage gateway receives an access request;

[0062] Specifically, the first object storage gateway may be a load gateway composed of openresty+lua.

[0063] In some embodiments of the present application, before the first object storage gateway receives the access request forwarded from the load balancing gateway, the load balancing gateway will also receive multiple access requests and select the first object storage gateway from the preset gateway list according to the preset weight, wherein the preset weight is the weight of each gateway in the preset gateway list, and the greater the weight, the greater the number of access requests allocated for parallel processing. In other words, the load balancing gateway can also perform load balancing processing on the access requests during the download process according to the weighted polling algorithm.

[0064] Step S304: the first object storage gateway obtains the target file identifier in the access request;

[0065] In some embodiments of the present application, when the openresty+lua storage gateway receives an access request, the storage gateway parses the request, obtains the file's unique ID, and queries the file's metadata from the MySQL cluster.

[0066] Step S306: the first object storage gateway determines a target relational database corresponding to the target file identifier from the relational database cluster, and queries metadata of the target file corresponding to the target file identifier from the target relational database, wherein the metadata is description information of the target file;

[0067] Specifically, when the first object storage gateway receives the access request forwarded from the load balancing gateway, the storage gateway forwards the request to the RGW component of the corresponding Ceph cluster for processing.

[0068] In some embodiments of the present application, the above-mentioned distributed storage system stores file metadata through MySQL (used to store relational data), and expands MySQL through sharding-sphere to support metadata storage of massive files.

[0069] It should be noted that sharding-sphere can be connected to multiple MySQL servers or proxy SQL servers.

[0070] In some embodiments of the present application, the storage gateway backend MySQL database table mainly includes a cluster_infos table storing Ceph cluster information and a coss_files table storing file metadata, wherein the key fields of the cluster_infos table are as follows:

[0071] `cluster` int(10) NOT NULL COMMENT 'cluster id';

[0072] `cluster_name`varchar(64)NOT NULL COMMENT 'cluster name';

[0073] `rw_status`tinyint(2)NOT NULL DEFAULT'-1'COMMENT'Cluster readable and writable status, 1 is readable and writable, 0 is read-only, -1 is not readable and writable';

[0074] `rgw_url`varchar(255)NOT NULL COMMENT'Cluster rgw address, <ip> : <port>'.

[0075] The core fields of the coss_files table are:

[0076] `cluster` int(10) unsigned NOT NULL COMMENT 'The file's current storage cluster ID';

[0077] `bucket`int(10)unsigned NOT NULL COMMENT'bucket id';

[0078] `filekey`varchar(255)NOT NULL COMMENT'File unique identifier';

[0079] `file_type`varchar(128)NOT NULL COMMENT'file type';

[0080] `file_size` bigint(20) unsigned NOT NULL COMMENT 'file size';

[0081] `w_status` tinyint(2) NOT NULL DEFAULT '0' COMMENT 'File writing status, 0 is marked for deletion; 1 is writing completed; 2 is writing in progress; 3 is marked for update; 4 is pending for merging; 5 is uploading multiple parts.'

[0082] In some embodiments of the present application, when the above-mentioned distributed storage system needs to delete a file, it only needs to modify the w_status field of the coss_files table where the file is stored in MySQL to mark the file to be deleted. The files will be deleted uniformly later through a scheduled task.

[0083] Step S308, the first object storage gateway determines a target distributed storage cluster corresponding to the metadata and a second object storage gateway in the target distributed cluster from a plurality of distributed storage clusters corresponding to the first object storage gateway;

[0084] In some embodiments of the present application, the specific process for the above-mentioned first object storage gateway to determine the target distributed storage cluster corresponding to the metadata and the second object storage gateway in the target distributed cluster from multiple distributed storage clusters corresponding to the first object storage gateway is: the first object storage gateway determines the target distributed storage cluster corresponding to the metadata from the cluster information table in the target relational database; determines the second object storage gateway from the metadata information table in the target relational database.

[0085] In some embodiments of the present application, the cluster information table includes: cluster identification, cluster read / write status, and cluster address information; the metadata information table includes the cluster identification of the target file, the file information of the target file, and the write status of the file. The file information of the target file includes the identification, name, file type, file size, and file write status of the target file.

[0086] In some embodiments of the present application, the readable and writable status of the above-mentioned cluster and the write status of the file are determined in the following manner: the relational database in the relational database cluster modifies the readable and writable status of the cluster and the write status of the file in response to a modification request of the target object; wherein, when a trigger event is detected, an operation corresponding to the modified readable and writable status of the cluster and the write status of the file is triggered to be performed on the file stored in the second object storage gateway.

[0087] Specifically, when the cluster is under maintenance, the cluster can be marked as unreadable and unwritable; when the remaining storage space of the cluster is insufficient, the cluster can be set to read-only status.

[0088] Step S310: The first object storage gateway forwards the access request to the second object storage gateway in the target distributed cluster to obtain the target file from the second object storage gateway.

[0089] By adopting the solution provided in the above embodiment, since during the file download process, the first object storage gateway processes the access request, and before the first object storage gateway processes the access request, a load balancing gateway will distribute the access request to the corresponding first object storage gateway according to the preset weight, it is possible to reduce the workload of RGW and improve the working performance of the distributed database.

[0090] Example 2

[0091] According to an embodiment of the present invention, a method embodiment of a data uploading method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0092] Figure 4 is a data uploading method according to an embodiment of the present invention, such as Figure 4 As shown, the method is applied in a distributed storage system, wherein the distributed storage system includes: a first object storage gateway, a distributed storage cluster and a relational database cluster, and includes the following steps:

[0093] Step S402: The first object storage gateway receives a file upload request;

[0094] Step S404, the first object storage gateway obtains metadata of the file to be uploaded corresponding to the file upload request;

[0095] Step S406: The first object storage gateway sends the metadata to the relational database cluster for storage, and sends the data to be uploaded to the second object storage gateway in the distributed storage cluster.

[0096] In some embodiments of the present application, the above-mentioned file upload request is multiple.

[0097] In some embodiments of the present application, before sending the data to be uploaded to the second object storage gateway in the distributed storage cluster, the load balancing gateway receives multiple file upload requests, and then selects the first object storage gateway from the preset gateway list according to the preset weight, wherein the preset weight is the weight of each gateway in the preset gateway list, and the larger the weight, the larger the number of file upload requests allocated for parallel processing.

[0098] In some embodiments of the present application, when the upload type of the file to be uploaded is multi-slice upload, the first object storage gateway will obtain the first type of metadata from the file upload request; and after all the slicing corresponding to the file to be uploaded is uploaded, a request message is sent to the second object storage gateway, and the second type of metadata corresponding to the request message is fed back by the second object storage gateway; and when the upload type of the file to be uploaded is uploaded in the form of a form, the first object storage gateway obtains the first type of metadata from the data to be uploaded.

[0099] The first type of metadata includes: the cluster identifier corresponding to the file to be uploaded, the file information of the file to be uploaded, and the writing status of the file; the second type of metadata includes: the file size.

[0100] Specifically, when the file size is large and needs to be uploaded in pieces, the uploaded pieces only contain the file content but no metadata. In this case, the file type and identifier of the file can be obtained from the http request, while the file size needs to be obtained by sending a request to RGW. When the uploaded file is a form uploaded through a client or APP, the uploaded data will contain metadata and data content.

[0101] To facilitate understanding of the above file upload steps, the following is an explanation of the above steps with a specific example. When uploading files, the upper-layer application accesses the object storage gateway through the domain name bucketname.oss.demo.com, and is resolved to a certain L4 Loadbalancer through DNS; then the request is forwarded to the openresty+lua storage gateway, and the storage gateway forwards the request to the RGW of a certain Ceph cluster in the backend according to the weighted polling algorithm; finally, the openresty+lua storage gateway writes the file metadata to the MySQL cluster and responds to the client.

[0102] With the solution provided in the above embodiment, since during the file upload process, the load balancing gateway will receive multiple file upload requests, and then select the first object storage gateway from the preset gateway list according to the preset weight, it is possible to reduce the workload of the RGW and thus improve the working efficiency of the distributed storage system.

[0103] Example 3

[0104] According to an embodiment of the present invention, there is also provided a Figure 5 The data download device shown is applied to the first object storage gateway in the distributed storage system, and is used to perform the following steps: Figure 2 The data download method shown, wherein the data download device includes: a first processing module 50, used to receive an access request; an acquisition module 52, used to obtain a target file identifier in the access request; a first search module 54, used to determine a target relational database corresponding to the target file identifier from a relational database cluster, and query metadata of the target file corresponding to the target file identifier from the target relational database, wherein the metadata is description information of the target file; a second search module 56, used to determine a target distributed storage cluster corresponding to the metadata and a second object storage gateway in the target distributed cluster from a plurality of distributed storage clusters corresponding to the first object storage gateway; a second processing module 58, used to forward the access request to the second object storage gateway in the target distributed cluster to obtain the target file from the second object storage gateway.

[0105] Because Figure 5 The data downloading device shown is used to execute the following Figure 3 The data downloading method shown, therefore, the relevant explanations in Example 1 are also applicable to this embodiment.

[0106] Example 4

[0107] According to an embodiment of the present invention, there is also provided a Figure 6 The data download device shown is applied to the first object storage gateway in the distributed storage system, and is used to perform the following steps: Figure 4 The data uploading method shown, wherein the data uploading device includes: a processing module 60, used to receive a file upload request; an acquisition module 62, used to obtain the metadata of the file to be uploaded corresponding to the file upload request; a sending module 64, used to send the metadata to a relational database cluster for storage, and send the data to be uploaded to a second object storage gateway in the distributed storage cluster.

[0108] Because Figure 6 The data downloading device shown is used to execute the following Figure 4 The data uploading method shown, therefore, the relevant explanations in Example 1 are also applicable to this embodiment.

[0109] Example 5

[0110] In some embodiments of the present application, a non-volatile storage medium is also provided, wherein the non-volatile storage medium includes a stored program, and when the program is running, controls the device where the non-volatile storage medium is located to execute the following data download method: the first object storage gateway receives an access request; the first object storage gateway obtains a target file identifier in the access request; the first object storage gateway determines a target relational database corresponding to the target file identifier from a relational database cluster, and queries metadata of the target file corresponding to the target file identifier from the target relational database, wherein the metadata is description information of the target file; the first object storage gateway determines a target distributed storage cluster corresponding to the metadata and a second object storage gateway in the target distributed cluster from a plurality of distributed storage clusters corresponding to the first object storage gateway; the first object storage gateway forwards the access request to the second object storage gateway in the target distributed cluster to obtain the target file from the second object storage gateway.

[0111] Example 6

[0112] In some embodiments of the present application, a non-volatile storage medium is also provided, wherein the non-volatile storage medium includes a stored program, and when the program is running, controls the device where the non-volatile storage medium is located to execute the following data upload method: the first object storage gateway receives a file upload request; the first object storage gateway obtains metadata of the file to be uploaded corresponding to the file upload request; the first object storage gateway sends the metadata to a relational database cluster for storage, and sends the data to be uploaded to a second object storage gateway in a distributed storage cluster.

[0113] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0114] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0115] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0116] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0117] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0118] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0119] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.< / port> < / ip>

Claims

1. A data downloading method, characterized in that: The method is applied to a distributed storage system, wherein the distributed storage system includes: a first object storage gateway, a distributed storage cluster, and a relational database cluster, and the method includes: The first object storage gateway receives an access request; The first object storage gateway obtains a target file identifier in the access request; The first object storage gateway determines a target relational database corresponding to the target file identifier from the relational database cluster, and queries metadata of the target file corresponding to the target file identifier from the target relational database, wherein the metadata is description information of the target file; The first object storage gateway determines, from a plurality of distributed storage clusters corresponding to the first object storage gateway, a target distributed storage cluster corresponding to the metadata and a second object storage gateway in the target distributed storage cluster; The first object storage gateway determines a target distributed storage cluster corresponding to the metadata and a second object storage gateway in the target distributed storage cluster from a plurality of distributed storage clusters corresponding to the first object storage gateway, including: the first object storage gateway determines the target distributed storage cluster corresponding to the metadata from a cluster information table in the target relational database; determines the second object storage gateway from the metadata information table in the target relational database; The first object storage gateway forwards the access request to the second object storage gateway in the target distributed storage cluster to obtain the target file from the second object storage gateway.

2. The method according to claim 1, characterized in that The distributed storage system includes: a load balancing gateway; the first object storage gateway receives an access request, including: The first object storage gateway receives the access request forwarded by the load balancing gateway.

3. The method according to claim 2, characterized in that Before the first object storage gateway receives the access request forwarded by the load balancing gateway, the method further includes: The load balancing gateway receives multiple access requests; selects the first object storage gateway from the preset gateway list according to the preset weight, wherein the preset weight is the weight of each gateway in the preset gateway list, and the larger the weight, the larger the number of access requests allocated for parallel processing.

4. The method according to claim 1, characterized in that: The cluster information table includes at least one of the following: cluster identification, cluster read / write status, and cluster address information; the metadata information table includes at least one of the following: cluster identification where the target file is located, file information of the target file, file writing status, and file size.

5. The method according to claim 4, characterized in that The readable and writable state of the cluster and the write state of the file are determined in the following manner: the relational database in the relational database cluster modifies the readable and writable state of the cluster and the write state of the file in response to a modification request of the target object; wherein, when a triggering event is detected, an operation corresponding to the modified readable and writable state of the cluster and the write state of the file is triggered to be performed on the file stored in the second object storage gateway.

6. A data uploading method, characterized in that: The method is applied to a distributed storage system, wherein the distributed storage system includes: a first object storage gateway, a distributed storage cluster, and a relational database cluster, and the method includes: The first object storage gateway receives a file upload request; The first object storage gateway obtains metadata of the file to be uploaded corresponding to the file upload request; The first object storage gateway sends the metadata to a relational database cluster for storage, and sends the data to be uploaded to a second object storage gateway in a distributed storage cluster; When the upload type of the file to be uploaded is multi-slice upload, the first object storage gateway obtains the first type of metadata from the file upload request; and after all the slicings corresponding to the file to be uploaded are uploaded, sends a request message to the second object storage gateway, and receives the second type of metadata corresponding to the request message fed back by the second object storage gateway; When the upload type of the file to be uploaded is uploading in a form, the first type of metadata is obtained from the data to be uploaded.

7. The method according to claim 6, characterized in that The file upload request is multiple; before sending the data to be uploaded to the second object storage gateway in the distributed storage cluster, the method further includes: The load balancing gateway receives multiple file upload requests; selects the first object storage gateway from the preset gateway list according to the preset weight, wherein the preset weight is the weight of each gateway in the preset gateway list, and the larger the weight, the larger the number of file upload requests allocated for parallel processing.

8. The method according to claim 6, characterized in that The first type of metadata includes: a cluster identifier corresponding to the file to be uploaded, file information of the file to be uploaded, and a writing status of the file; the second type of metadata includes: a file size.

9. A distributed storage system, characterized in that: The distributed storage system includes: a first object storage gateway, a distributed storage cluster and a relational database cluster; wherein, The first object storage gateway is used to receive an access request; obtain a target file identifier in the access request; determine a target relational database corresponding to the target file identifier from the relational database cluster, and query metadata of the target file corresponding to the target file identifier from the target relational database, wherein the metadata is description information of the target file; determine a target distributed storage cluster corresponding to the metadata and a second object storage gateway in the target distributed storage cluster from a plurality of distributed storage clusters corresponding to the first object storage gateway; and forward the access request to the second object storage gateway in the target distributed storage cluster to obtain the target file from the second object storage gateway; The first object storage gateway determines a target distributed storage cluster corresponding to the metadata and a second object storage gateway in the target distributed storage cluster from a plurality of distributed storage clusters corresponding to the first object storage gateway, including: the first object storage gateway determines the target distributed storage cluster corresponding to the metadata from a cluster information table in the target relational database; determines the second object storage gateway from the metadata information table in the target relational database; The relational database cluster is used to store the metadata; The distributed storage cluster is used to store the target file.

10. A data downloading device, characterized in that: The device is applied to a first object storage gateway in a distributed storage system, wherein the distributed storage system includes: a first object storage gateway, a distributed storage cluster and a relational database cluster, and the device includes: A first processing module, configured to receive an access request; An acquisition module, used to acquire a target file identifier in the access request; A first search module is used to determine a target relational database corresponding to the target file identifier from the relational database cluster, and query metadata of the target file corresponding to the target file identifier from the target relational database, wherein the metadata is description information of the target file; A second search module, configured to determine a target distributed storage cluster corresponding to the metadata and a second object storage gateway in the target distributed storage cluster from a plurality of distributed storage clusters corresponding to the first object storage gateway; The first object storage gateway determines a target distributed storage cluster corresponding to the metadata and a second object storage gateway in the target distributed storage cluster from a plurality of distributed storage clusters corresponding to the first object storage gateway, including: the first object storage gateway determines the target distributed storage cluster corresponding to the metadata from a cluster information table in the target relational database; determines the second object storage gateway from the metadata information table in the target relational database; The second processing module is used to forward the access request to the second object storage gateway in the target distributed storage cluster to obtain the target file from the second object storage gateway.

11. A data uploading device, characterized in that: The device is applied to a first object storage gateway in a distributed storage system, wherein the distributed storage system includes: a first object storage gateway, a distributed storage cluster and a relational database cluster, and the device includes: A processing module, used for receiving a file upload request; An acquisition module, used to acquire metadata of the file to be uploaded corresponding to the file upload request; A sending module, used to send the metadata to a relational database cluster for storage, and to send the data to be uploaded to a second object storage gateway in a distributed storage cluster; When the upload type of the file to be uploaded is multi-slice upload, the first object storage gateway obtains the first type of metadata from the file upload request; and after all the slicings corresponding to the file to be uploaded are uploaded, sends a request message to the second object storage gateway, and receives the second type of metadata corresponding to the request message fed back by the second object storage gateway; When the upload type of the file to be uploaded is uploading in a form, the first type of metadata is obtained from the data to be uploaded.

12. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored program, wherein when the program is executed, the device where the non-volatile storage medium is located is controlled to execute the data downloading method according to any one of claims 1 to 5.

Images