Object storage service method and cloud platform, storage medium

Abstract

The present disclosure provides an object storage service method and cloud platform, and a storage medium. The method is applied to a cloud platform and comprises: acquiring, by a control node of the cloud platform, a full-link access log of an object storage service; wherein the full-link access log comprises access logs of service nodes at each level covered by the link; determining, by the control node, service index data corresponding to each level according to the access logs of the service nodes at each level; determining, by a monitoring node of the cloud platform, a level that does not meet the service according to the service index data at each level; and scheduling, by a scheduling node of the cloud platform, an object storage service task of the level that does not meet the service. Through the above method, the level that does not meet the service can be quickly located to perform the scheduling task of the level.

Description

Technical Field

[0001] This disclosure relates to, but is not limited to, the field of computer science, and particularly to an object storage service method, a cloud platform, and a storage medium. Background Technology

[0002] As people's demand for data volume changes, the requirements for data storage capacity and data security are also gradually increasing. Cloud-based storage, as an important component of cloud computing networks that have emerged in recent years, eliminates the need for portable storage devices, allowing users to store and manage data anytime and anywhere, thus improving ease of use.

[0003] In related technologies, when service quality is substandard, it's difficult to intuitively reflect what exactly happened on the server side. There's a lack of efficient solutions for quickly identifying and promptly restoring service quality in such situations. Summary of the Invention

[0004] In view of this, the present disclosure aims to provide an object storage service method, a cloud platform, and a storage medium.

[0005] In a first aspect, embodiments of this disclosure provide an object storage service method applied in a cloud platform, the method comprising:

[0006] The cloud platform's control node obtains the object storage service's full-link access logs; wherein, the full-link access logs include access logs of service nodes at each level covered by the link.

[0007] The control node determines the service indicator data corresponding to each level based on the access logs of the service nodes at each level.

[0008] The monitoring nodes of the cloud platform determine the levels where services are not up to standard based on service indicator data at each level.

[0009] The cloud platform's scheduling nodes schedule the object storage service tasks for the substandard levels.

[0010] In some embodiments, determining the level of service non-compliance through the monitoring nodes of the cloud platform based on service indicator data at each level includes:

[0011] For each level, the service indicator data of the level is compared with the preset indicator data threshold corresponding to the level through the monitoring node to obtain the comparison result;

[0012] The monitoring nodes determine whether a given level is not meeting service standards based on the comparison results of each level.

[0013] In some embodiments, the service indicator data includes multiple service indicator data; each service indicator data includes data under multiple dimensions; the preset indicator data threshold corresponding to the level includes the threshold corresponding to the data under each dimension in the multiple service indicator data of the level;

[0014] The step of comparing the service indicator data of the level with the preset indicator data threshold corresponding to the level through the monitoring node to obtain the comparison result includes:

[0015] For each service indicator data at the aforementioned level, the monitoring node compares the data under each dimension of the service indicator data at the aforementioned level with the threshold corresponding to the dimension to obtain the comparison result;

[0016] The step of determining whether a level's service is substandard based on comparison results at each level through the monitoring node includes:

[0017] The monitoring nodes determine whether a given level of service is substandard by comparing the data across different dimensions of the service metrics at each level.

[0018] In some embodiments, the layers include: a metadata layer, a data layer, an access layer, and a load balancing layer; the object storage service tasks include: garbage collection tasks, lifecycle management tasks, silent data verification tasks, data repair tasks, and metadata repair tasks;

[0019] The scheduling of the substandard object storage service tasks through the scheduling node of the cloud platform includes:

[0020] If the services of the metadata layer are not up to standard, the scheduling node will schedule the garbage collection task, the lifecycle management task, and the metadata repair task of the metadata layer.

[0021] If the service of the data layer is not up to standard, the scheduling node will schedule the garbage collection task, the silent data verification task and the data repair task of the data layer.

[0022] If the services of the access layer are not up to standard, the scheduling node will schedule the garbage collection task, the lifecycle management task, the silent data verification task, the data repair task and the metadata repair task of the access layer.

[0023] If the service of the load balancing layer is not up to standard, the silent data verification task of the load balancing layer is scheduled through the scheduling node.

[0024] In some embodiments, scheduling the object storage service tasks at the substandard level through the scheduling node of the cloud platform includes:

[0025] The scheduling node suspends the object storage service tasks at the substandard level.

[0026] Once the service at the substandard level meets the standards, the object storage service task at that level is restored through the scheduling node.

[0027] In some embodiments, the step of restoring the object storage service task of the layer through the scheduling node includes:

[0028] If the service metric data of the aforementioned level reaches the preset optimal metric data threshold, the object storage service task of the aforementioned level is restored through the scheduling node.

[0029] In some embodiments, the step of restoring the object storage service task of the layer through the scheduling node includes:

[0030] Within a preset recovery count threshold, the object storage service task of the specified level is restored through the scheduling node until the object storage service task of the specified level is successfully restored.

[0031] In some embodiments, the non-compliant levels include multiple non-compliant levels;

[0032] The scheduling of the substandard object storage service tasks through the scheduling node of the cloud platform includes:

[0033] The scheduling node determines the lowest level among the multiple non-compliant levels based on the order of all levels covered by the link.

[0034] The scheduling node schedules the lowest-level object storage service tasks.

[0035] Secondly, embodiments of this disclosure provide a cloud platform, the cloud platform comprising:

[0036] A control node is used to obtain the full-link access logs of the object storage service and determine the service indicator data corresponding to each level based on the access logs of the service nodes at each level; wherein, the full-link access logs include the access logs of the service nodes at each level covered by the link.

[0037] Monitoring nodes are used to determine the levels where services are not up to standard based on service indicator data at each level.

[0038] The scheduling node is used to schedule object storage service tasks at the substandard level.

[0039] Thirdly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the object storage service method applied in a cloud platform as described in the first aspect.

[0040] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0041] In the embodiments of this disclosure, the cloud platform determines the service indicator data corresponding to each level based on the full-link access logs through the control node, thereby identifying the levels where services are substandard, and then scheduling object storage service tasks for the substandard levels. Obtaining service indicators for each level based on the full-link access logs enables tracking the quality of service provided at each level, allowing for rapid identification of substandard levels and execution of scheduling tasks for those levels.

[0042] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0044] Figure 1 A schematic diagram of the implementation flow of an object storage service method provided in this embodiment of the disclosure. Figure 1 ;

[0045] Figure 2 An architectural block diagram of an object storage service system provided in this disclosure embodiment;

[0046] Figure 3 A schematic diagram illustrating the linkage between service metrics and task scheduling provided in this embodiment of the disclosure;

[0047] Figure 4 A schematic diagram of the implementation flow of an object storage service method provided in this embodiment of the disclosure. Figure 2 ;

[0048] Figure 5 This is a block diagram of a cloud platform provided in an embodiment of the present disclosure. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this disclosure clearer, the technical solutions of this disclosure are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this disclosure. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0050] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0051] The terms “first / second / third” used in this disclosure are merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that “first / second / third” may be interchanged in a specific order or sequence where permitted, so that the embodiments of this disclosure described herein can be implemented in an order other than that illustrated or described herein.

[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. The terminology used herein is for descriptive purposes only and is not intended to limit the scope of this disclosure.

[0053] Figure 1 A schematic diagram of the implementation flow of an object storage service method provided in this embodiment of the disclosure. Figure 1 Applied in cloud platforms, such as Figure 1 As shown, the method includes the following steps:

[0054] S101. Obtain the full-link access log of the object storage service through the control node of the cloud platform; wherein, the full-link access log includes the access logs of the service nodes at each level covered by the link;

[0055] S102. The control node determines the service indicator data corresponding to each level based on the access logs of the service nodes at each level.

[0056] S103. Based on the service indicator data of each level, determine the level where the service is not up to standard through the monitoring nodes of the cloud platform;

[0057] S104. The scheduling node of the cloud platform schedules the object storage service tasks of the substandard level.

[0058] The object storage service method of this disclosure is applied to a cloud platform. Object storage services have a wide range of applications in cloud computing service scenarios and can be used to store unstructured data. Cloud computing service types include public cloud, private cloud, hybrid cloud, multi-cloud, etc. For example, the cloud platform can be OpenStack, Kubernetes, CloudStack, Hadoop, Amazon Web Services, Alibaba Cloud, Sina Cloud, etc.

[0059] In this embodiment of the disclosure, the cloud platform provides object storage services through nodes. These nodes may include control nodes, service nodes, monitoring nodes, scheduling nodes, etc., and can be workstations, personal computers, servers, and other devices capable of creating, editing, and running virtual machines. The service node layer can be divided into a load balancing layer, an access layer, a metadata layer, and a data layer. The load balancing layer may include Layer 4 load balancing and Layer 7 load balancing, corresponding to Layer 4 and Layer 7 of the Open System Interconnection Reference Model (OSI), respectively. For example, Layer 4 load balancing can be Server Load Balancing (SLB), and Layer 7 load balancing can be implemented based on APISIX.

[0060] In this embodiment, the cloud platform can construct a traceable end-to-end access log through service nodes. This end-to-end access log includes access logs from service nodes at each level covered by the link. When a user accesses data in object storage through the cloud platform, the object storage service's access request leaves corresponding access logs at each level of the link. All these access logs at all levels constitute the end-to-end access log for the object storage service. To ensure that the access logs for a single request can be chained together across all levels, the cloud platform needs to locate the access logs generated by the same request at different levels.

[0061] For example, a cloud platform can provide an access interface to data in object storage based on the Hypertext Transfer Protocol (HTTP). Using the APISIX request-id plugin, it can add a request header `x-request-id` to requests accessing object storage services via HTTP through a rewrite method. The value of `x-request-id` can be represented by a 32-bit universally unique identifier (UUID). The relevant settings of the request-id plugin are saved in APISIX's global rules, ensuring that `x-request-id` applies to all routed requests. Subsequently, the load balancing layer forwards the request with the added header to the access layer. Upon receiving the request, the access layer retrieves the value of the `x-request-id` header and adds this value to all subsequent requests accessing the metadata and data layers, ensuring that the `x-request-id` value is always carried, thus completing the end-to-end tracing of a request.

[0062] The interaction between the access layer and metadata involves requests such as retrieving user information, retrieving bucket information, and retrieving object metadata. Data from each request is recorded and ultimately fed back to the access layer's access log. The interaction between the access layer and the data layer involves requests such as writing and reading data shards. If erasure coding (EC) algorithms are used to encode the data in object storage to achieve redundancy, the data related to requests for the required number of erasure coding shards to meet read / write requirements will also be recorded and ultimately fed back to the access layer's access log.

[0063] In this embodiment, the cloud platform can write end-to-end access logs to the log storage and analysis engine via a control node. For example, the log storage and analysis engine can be a storage engine such as ClickHouse, Elasticsearch, or Loki. Logs from all levels are accessed to the log storage and analysis engine primarily in two ways: one is by writing directly via a Hypertext Transfer Protocol (HTTP) interface, such as logs from the load balancing layer; the other is by calling the log storage and analysis engine's Software Development Kit (SDK) to write logs via native method interfaces, such as logs from the access layer, metadata layer, and data layer. Specifically, the cloud platform can listen for HTTP requests based on ClickHouse. The load balancing layer's APISIX uses the ClickHouse-logger plugin to match all fields of the requested logs with the column attributes of the tables in the log storage and analysis engine. Furthermore, it embeds Structured Query Language (SQL) statements into the body of the HTTP request, sends an HTTP request to ClickHouse, and writes the logs to ClickHouse. The access layer, metadata layer, and data layer write logs to ClickHouse via native interfaces by calling the ClickHouse-sdk. Among them, the Hypertext Transfer Protocol includes Hypertext Transfer Protocol (HTTP), HTTP Secure (HTTPS), HTTP / 1.1, HTTP / 2, and HTTP / 3.

[0064] It should be noted that both the data accessor and the data provider of the object storage service must comply with the relevant Service Level Agreement (SLA). Each object storage service provider has its own service-related SLA terms; for example, Amazon Web Services' EC2 and S3 services have corresponding SLA terms. Defining an enforceable SLA requires a good Service Level Objective (SLO) and a Service Level Indicator (SLI). Here, the SLI is the measurement metric for the object storage service, and the SLO is the benchmark for providing the SLA, offering directions for service improvement. In this embodiment, the service metric data corresponding to each level is the measurement metric for the object storage service. This service metric data may include one or more service metric data such as throughput (Requests Per Second, TPS), bandwidth, and latency.

[0065] In this embodiment, the cloud platform determines the service indicator data corresponding to each level based on the access logs of service nodes at each level through the control node. In this embodiment, the cloud platform can use a log storage and analysis engine to store the access logs of each level covered by the link into different tables according to the time sequence of log generation, and statistically analyze the service indicator data corresponding to each level according to a preset statistical time. Specifically, the cloud platform can use the control node to use aggregation functions based on the time-series analysis engine in ClickHouse to query data such as throughput, bandwidth, and latency within a preset query period at each statistical time, and then store the calculated results in the corresponding tables in chronological order for subsequent querying and display. The preset query period can be the most recent 5 minutes of each statistical time, or other time periods.

[0066] In this embodiment of the disclosure, the cloud platform can display service metric data at each level through monitoring nodes. For example, the cloud platform can use the Grafana-Clickhouse-datasource plugin to obtain service metric data at each level in Clickhouse based on Grafana or other monitoring platforms through monitoring nodes, and then display it.

[0067] In this embodiment of the disclosure, the cloud platform determines the service level that is not up to standard by monitoring nodes based on service indicator data at each level. It is understood that if each level has multiple service indicator data, then for any service indicator data at that level that is not up to standard, the service at that level is considered substandard.

[0068] After determining the level of service non-compliance, in this embodiment, the cloud platform obtains the determined level of service non-compliance through a scheduling node. Specifically, the cloud platform can use monitoring nodes to issue alerts for non-compliant service indicator data at the non-compliant level based on the alarm rules configured in the alarm module of the Grafana monitoring platform, and display the alerts. Alarm information can also be sent to a designated address via email or webhook. The alarm rules can be configured based on existing service indicator data for each level, or they can be preset.

[0069] In this embodiment, the cloud platform schedules object storage service tasks at substandard service levels through scheduling nodes. Specifically, the cloud platform can monitor alarm events via scheduling nodes using management tools such as Argo or Flux. Once an alarm event is received, the corresponding task can be scheduled using CronJob or other timed management tasks. Typically, the scheduling tasks of the scheduling nodes include tasks such as garbage collection (GC), lifecycle (LC) management, silent data verification, metadata repair, and data repair. It should be noted that executing scheduling tasks consumes resources such as the central processing unit (CPU), memory, network, and hard disk of the corresponding level of the node. If the service at a certain level is substandard, the scheduling tasks at that level can be paused, thereby releasing the resources occupied by the scheduling tasks at that level.

[0070] In related technologies, the service operation status of the client is obtained by installing a proxy on the client side. However, in scenarios where service quality is substandard, it is not possible to intuitively reflect what is happening on the server side. The server side can include the access layer, metadata layer, and data layer. For object storage services, there are many reasons for substandard service quality, such as excessive resource utilization in the access layer, hotspots in the metadata layer, and poor network quality in the data layer. How to quickly locate and promptly guarantee service quality in scenarios where service quality is substandard has become a challenge.

[0071] In this embodiment, the cloud platform uses a control node to determine service metric data for each level based on end-to-end access logs, thereby identifying the levels where service is substandard. Then, it schedules object storage service tasks for these substandard levels. By obtaining service metrics for each level from end-to-end access logs, the quality of service provided at each level can be tracked, enabling rapid identification of substandard levels for execution of scheduled tasks at those levels.

[0072] In some embodiments, determining the level of service non-compliance through the monitoring nodes of the cloud platform based on service indicator data at each level includes:

[0073] For each level, the service indicator data of the level is compared with the preset indicator data threshold corresponding to the level through the monitoring node to obtain the comparison result;

[0074] The monitoring nodes determine whether a given level is not meeting service standards based on the comparison results of each level.

[0075] It should be noted that the tasks performed at each level are different, and the corresponding Service Level Objectives (SLOs) for each level are also different. In this embodiment, the SLO for each level can be represented by a preset indicator data threshold corresponding to that level. It is understood that the cloud platform compares the service indicator data of each level with the preset indicator data threshold corresponding to that level through monitoring nodes, and can determine whether the service at each level meets the standards.

[0076] In this embodiment of the disclosure, for each level, the cloud platform compares the service indicator data of the level with the preset indicator data threshold corresponding to the level through monitoring nodes, obtains the comparison result, and then determines whether the service of the level is substandard based on the comparison result of each level. For example, if the preset indicator data threshold is an upper limit value, when the service indicator data of the level is greater than the preset indicator data threshold corresponding to the level, the service of that level is substandard; if the preset indicator data threshold is a lower limit value, when the service indicator data of the level is less than the preset indicator data threshold corresponding to the level, the service of that level is substandard.

[0077] In this embodiment, the cloud platform compares the service indicator data of each level with the preset indicator data thresholds corresponding to each level through monitoring nodes to determine the levels where service is substandard. The preset indicator data thresholds reflect the service level targets, and judging whether a level is substandard by using the preset indicator data thresholds can yield more accurate results.

[0078] In some embodiments, the service indicator data includes multiple service indicator data; each service indicator data includes data under multiple dimensions; the preset indicator data threshold corresponding to the level includes the threshold corresponding to the data under each dimension in the multiple service indicator data of the level;

[0079] The step of comparing the service indicator data of the level with the preset indicator data threshold corresponding to the level through the monitoring node to obtain the comparison result includes:

[0080] For each service indicator data at the aforementioned level, the monitoring node compares the data under each dimension of the service indicator data at the aforementioned level with the threshold corresponding to the dimension to obtain the comparison result;

[0081] The step of determining whether a level's service is substandard based on comparison results at each level through the monitoring node includes:

[0082] The monitoring nodes determine whether a given level of service is substandard by comparing the data across different dimensions of the service metrics at each level.

[0083] In this embodiment of the disclosure, the service indicator data includes multiple service indicator data, and each service indicator data includes data under multiple dimensions. For each indicator, there are multiple evaluation criteria across multiple dimensions. In this embodiment of the disclosure, the preset indicator data threshold corresponding to the level includes the threshold corresponding to the data under each dimension of the multiple service indicator data at that level.

[0084] For example, the layers covered by the link may include: metadata layer, data layer, access layer, and load balancing layer; multiple service metrics data may include throughput, bandwidth, latency, etc.; throughput may include data under dimensions such as error, compliance rate, and performance fluctuation; bandwidth may include data under dimensions such as mean error and error; latency may include data under dimensions such as error, compliance rate, and percentile performance. The service metrics data of each layer covered by the above link, including the metadata layer, data layer, access layer, and load balancing layer, are shown in Table 1:

[0085] Table 1 Service Indicator Data Table

[0086]

[0087] Where M represents the metadata layer, D represents the data layer, A represents the access layer, LB represents the load balancing layer, and E represents the data layer. T For the throughput error of the hierarchy; O T For the throughput compliance rate of the hierarchy; PF T E represents the throughput performance fluctuation value at the tier level. B The mean bandwidth error at each level; E Bβ For the bandwidth error of the layer; E L For the time delay error at the hierarchical level; O L Latency compliance rate for the hierarchy; PC L This represents the percentile performance value for the tier.

[0088] For example, the cloud platform obtains throughput error, throughput compliance rate, and throughput performance fluctuation values ​​at a certain level through the control node, as shown in equations (1), (2), and (3) below:

[0089] E T =(T actual -T target ) / T target *100% (1)

[0090] O T =F(δ1≤T≤δ2)*100% (2)

[0091] PF T =SD(T) actual (k)) / T target *100% (3)

[0092] Among them, E T For the throughput error of the hierarchy; T actual T represents the actual throughput. target Target throughput; O T is the throughput compliance rate of the hierarchy; F() is the correlation function of the throughput compliance rate of the hierarchy; δ1 and δ2 are the allowable fluctuation ranges; PF T SD represents the throughput performance fluctuation value at the tier level; SD is the performance standard deviation, used to indicate performance fluctuation; T actual (k) represents the k-th actual throughput value within the preset query period.

[0093] For example, the cloud platform obtains the average bandwidth error and bandwidth error of a certain level through the control node, as shown in equations (4) and (5) below:

[0094] E B = (B avg -B target ) / B target *100% (4)

[0095]

[0096] Among them, E B B represents the average bandwidth error at each level. avg B is the average bandwidth; target For target bandwidth; E Bβ The bandwidth error is at the hierarchical level; [t1, t2] represents the preset query time period; B i This is the bandwidth value measured in the i-th second of the time interval [t1, t2].

[0097] For example, the cloud platform obtains latency error, latency compliance rate, and latency percentile performance at a certain level through the control node, as shown in equations (6), (7), and (8) below:

[0098] E L =(L avg -L target ) / L target *100%(6)

[0099] O L =F(δ3≤L≤δ4)*100%(7)

[0100] PC L =F -1 (X%), 0≤X≤100(8)

[0101] Among them, E L For the time delay error at the hierarchical level; L avg L represents the average delay. target For target delay; OL δ is the latency compliance rate of the hierarchy; F() is the correlation function of the latency compliance rate of the hierarchy; δ3 and δ4 are the allowable fluctuation ranges; PC L X represents the percentile performance value for the level; X represents the tail delay value.

[0102] In this embodiment, for each service indicator data point at a given level, the cloud platform compares the data for each dimension of the service indicator data with the corresponding threshold values ​​through monitoring nodes to obtain comparison results. Based on these results, it then determines whether the service at that level is substandard. It is understood that if any dimension of any service indicator data is substandard, the service at the corresponding level is substandard. For example, referring to Table 1, ideally, performance always meets the standards; in this case, the error E... T E B E L Approaching 0, the rate of compliance O T O L Approaching 100%, PF T It tends towards 0. If performance fluctuations occur, the error will definitely be greater than 0, and the compliance rate will be less than 100%. If MO T Below 90% or MO L Below 90% or M-PF T If it exceeds 20%, the metadata layer service is substandard; if DO T Below 60% or DO L Below 60% or D-PF T If it exceeds 20%, the metadata layer service is substandard; if AO T Below 90% or AO L Below 90% or A-PF T If it exceeds 20%, the access layer service is substandard; if LB-O T Below 90% or LB-O L Below 90% or LB-PF T If the load balancing layer's service level exceeds 20%, then the service is substandard.

[0103] In this embodiment of the disclosure, the cloud platform compares the data under each dimension of the various service indicator data at each level with the corresponding thresholds of the dimensions through monitoring nodes to determine whether the service at a level is substandard. It is understood that multiple service indicator data can accurately determine whether the corresponding level is substandard, and each service indicator data includes data under multiple dimensions, which can further improve the accuracy of determining whether the corresponding level is substandard.

[0104] In some embodiments, the layers include: a metadata layer, a data layer, an access layer, and a load balancing layer; the object storage service tasks include: a garbage collection task, a lifecycle management task, a silent data verification task, a data repair task, and a metadata repair task;

[0105] The scheduling of the substandard object storage service tasks through the scheduling node of the cloud platform includes:

[0106] If the services of the metadata layer are not up to standard, the scheduling node will schedule the garbage collection task, the lifecycle management task, and the metadata repair task of the metadata layer.

[0107] If the service of the data layer is not up to standard, the scheduling node will schedule the garbage collection task, the silent data verification task and the data repair task of the data layer.

[0108] If the services of the access layer are not up to standard, the scheduling node will schedule the garbage collection task, the lifecycle management task, the silent data verification task, the data repair task and the metadata repair task of the access layer.

[0109] If the service of the load balancing layer is not up to standard, the silent data verification task of the load balancing layer is scheduled through the scheduling node.

[0110] In this embodiment, the layers include: a metadata layer, a data layer, an access layer, and a load balancing layer; the object storage service tasks include: garbage collection tasks, lifecycle management tasks, silent data verification tasks, data repair tasks, and metadata repair tasks. The execution of these tasks typically consumes certain cluster resources, and the resource consumption varies depending on the task and the corresponding layer. For example, garbage collection tasks typically require resources from the metadata and data layers; lifecycle management tasks require resources from the access and metadata layers; silent data verification tasks require resources from the data layer; metadata repair tasks require resources from the metadata layer; and data repair tasks require resources from the data layer.

[0111] If a service level fails to meet standards, the cloud platform can release resources occupied by the failing level through scheduling nodes to ensure the service level target for that level is met. It should be noted that, to avoid impacting other services on the cloud platform, task scheduling is typically performed during off-peak hours. In this embodiment, if the metadata layer service fails to meet standards, the scheduling node schedules garbage collection tasks, lifecycle management tasks, and metadata repair tasks for the metadata layer; if the data layer service fails to meet standards, the scheduling node schedules garbage collection tasks, silent data verification tasks, and data repair tasks for the data layer; if the access layer service fails to meet standards, the scheduling node schedules garbage collection tasks, lifecycle management tasks, silent data verification tasks, data repair tasks, and metadata repair tasks for the access layer; if the load balancing layer service fails to meet standards, the scheduling node schedules silent data verification tasks for the load balancing layer.

[0112] It should be noted that each dimension may correspond to multiple thresholds, and the task scheduling for scenarios where services fail to meet standards under different thresholds may differ. For example, refer to the service metric data table (Table 1 above). If the data in any dimension of the metadata layer fails to meet the standard, the cloud platform will suspend the metadata repair task through the scheduling node; if MO... T Or MO L If the rate is below 90%, then the lifecycle management tasks of the metadata layer are paused. If MO T Or MO L If the percentage is below 60%, then pause garbage collection at the metadata layer. If M-PF T If the threshold exceeds 20%, then tasks such as garbage collection, lifecycle management, and metadata repair in the metadata layer will be directly suspended. If data in any dimension of the data layer fails to meet the standards, the cloud platform will suspend silent data verification and data repair tasks in the data layer through scheduling nodes; if DO T Or DO L If the percentage is below 60%, then pause garbage collection at the data layer. If D-PF T If the data rate exceeds 20%, then garbage collection, silent data verification, and data repair tasks in the data layer will be suspended directly. If data in any dimension of the access layer fails to meet the standards, the cloud platform will suspend lifecycle management, silent data verification, data repair, and metadata repair tasks in the access layer through scheduling nodes; if AO... T Or AO L If the AO rate is below 90%, then the access layer lifecycle management tasks are paused; if the AO rate is below 90%, then the access layer lifecycle management tasks are paused. T Or AO L If the efficiency is below 60%, the garbage collection task at the access layer will be paused; if A-PF TIf the threshold exceeds 20%, all scheduling tasks at the access layer will be suspended immediately. If the data in any dimension of the load balancing layer fails to meet the standard, the cloud platform will suspend the silent data verification task of the load balancing layer through the scheduling node.

[0113] In this embodiment of the disclosure, the cloud platform executes different scheduling tasks for different service failure levels through scheduling nodes. It can be understood that each level has a corresponding scheduling strategy. Through the above method, task scheduling can be automated, and the task scheduling strategy under the current failure scenario can be quickly determined, thereby improving the efficiency of task scheduling.

[0114] In some embodiments, scheduling the object storage service tasks at the substandard level through the scheduling node of the cloud platform includes:

[0115] The scheduling node suspends the object storage service tasks at the substandard level.

[0116] Once the service at the substandard level meets the standards, the object storage service task at that level is restored through the scheduling node.

[0117] In this embodiment of the disclosure, the cloud platform suspends object storage service tasks at substandard levels through scheduling nodes. After the object storage service tasks at substandard levels are suspended, the cloud platform monitors whether the service at that level meets the standards through a monitoring platform.

[0118] After the service at a previously non-compliant level meets the required standards, in this embodiment of the disclosure, the cloud platform restores the object storage service task for that level through a scheduling node. It is understood that if the scheduling task for that level continues to be suspended after the service at a previously non-compliant level meets the required standards, it may impact the object storage service. Restoring the object storage service task for that level ensures that the object storage service can operate normally.

[0119] In some embodiments, the step of restoring the object storage service task of the layer through the scheduling node includes:

[0120] If the service metric data of the aforementioned level reaches the preset optimal metric data threshold, the object storage service task of the aforementioned level is restored through the scheduling node.

[0121] It's important to note that for object storage services, workload typically fluctuates over time, with peaks and troughs. Understandably, during peak periods, the workload is high, and service metrics for the corresponding tier may exceed preset thresholds. Conversely, during troughs, the workload is low, and service metrics for the corresponding tier remain within the preset thresholds. However, when a tier's service improves from substandard to compliant, the service may not necessarily be in a trough. If object storage service tasks for that tier are resumed at this point, the service at that tier could very well fall below compliance again.

[0122] In this embodiment, after a substandard service level meets the required standards, if the service metric data for that level reaches a preset optimal metric data threshold, the object storage service task for that level is restored via a scheduling node. The preset optimal metric data threshold corresponds to a low-pressure period in the business, where business pressure is relatively low. It should be noted that if a level has multiple service metric data points, the object storage service task restored via the scheduling node corresponds to the service metric data that reaches the preset optimal metric data threshold.

[0123] During the recovery of the object storage service task at the tier, in this embodiment of the disclosure, the cloud platform continues to monitor the service indicator data of the tier through monitoring nodes. If the service of the tier does not meet the standards, the recovery of the object storage service task at that tier is suspended.

[0124] In this embodiment of the disclosure, when the service indicator data at the tier reaches a preset optimal indicator data threshold, the cloud platform restores the object storage service task at the tier through scheduling nodes. When business pressure is low, there are sufficient resources to restore the object storage service task at the tier, enabling the object storage service to execute normally.

[0125] In some embodiments, the step of restoring the object storage service task of the layer through the scheduling node includes:

[0126] Within a preset recovery count threshold, the object storage service task of the specified level is restored through the scheduling node until the object storage service task of the specified level is successfully restored.

[0127] In this embodiment of the disclosure, during the recovery of the object storage service task at the recovery level, the cloud platform records the number of times the recovery is performed through the scheduling node.

[0128] In this embodiment of the disclosure, within a preset recovery count threshold, the cloud platform restores the hierarchical object storage service task by scheduling nodes until the hierarchical object storage service task is successfully restored. The preset recovery count threshold can be 3 times or other values. For example, if three consecutive recovery attempts fail, no further recovery will be attempted, and the operations and maintenance personnel can be notified that the cloud platform's cluster processing capacity is limited and requires corresponding resource expansion.

[0129] In this embodiment of the disclosure, if the number of recovery attempts exceeds the threshold range, the cloud platform will not perform the recovery of the hierarchical object storage service task, and will release the resources occupied by the recovery task operation so that other tasks can be executed normally.

[0130] In some embodiments, the non-compliant levels include multiple non-compliant levels;

[0131] The scheduling of the substandard object storage service tasks through the scheduling node of the cloud platform includes:

[0132] The scheduling node determines the lowest level among the multiple non-compliant levels based on the order of all levels covered by the link.

[0133] The scheduling node schedules the lowest-level object storage service tasks.

[0134] In this embodiment of the disclosure, the non-compliant layers include multiple non-compliant layers, and these multiple non-compliant layers have a corresponding order. The object storage service is a service provided to customers, and all layers covered by the link can be arranged from the bottom layer to the top layer in order of distance from the client. For example, from bottom to top, the layers are metadata layer, data layer, access layer, and load balancing layer.

[0135] It should be noted that, typically, a failure at the lower layer will lead to a failure at the upper layer, because the service level of the upper layer depends on the service quality provided by the lower layer. In this embodiment, the cloud platform, through its scheduling nodes, determines the lowest level among the multiple substandard layers based on the order of all layers covered by the links, and schedules the object storage service tasks for the lowest level. For example, if the services of the metadata layer and the data layer are up to standard, but the services of the load balancing layer and the access layer are substandard, the tasks of the access layer are scheduled.

[0136] In this embodiment of the disclosure, the cloud platform schedules the lowest-level object storage service tasks through scheduling nodes, which can save the cost of resources occupied by other levels of scheduling tasks, and the saved resources are used for the current scheduling task.

[0137] Figure 2An architectural block diagram of an object storage service system provided in this disclosure embodiment is shown below. Figure 2 As shown in this embodiment, the object storage service system includes a cloud platform and nodes. Nodes include: client L201 nodes, control nodes, service nodes, monitoring nodes, scheduling nodes, etc. The service nodes are located in layers that can be divided into a load balancing layer L202, an access layer L203, a metadata layer L204, and a data layer L205. The scheduling nodes can be used to schedule tasks such as garbage collection, lifecycle management, silent data verification, metadata repair, and data repair. In this embodiment, each layer corresponds to different scheduling tasks. The cloud platform can obtain the full-link access logs of the service nodes through the control node on the control plane L207. Then, it calculates the service indicator data corresponding to each layer covered by the link based on the full-link access logs and displays it based on the Grafana monitoring platform.

[0138] In this embodiment of the disclosure, the cloud platform determines the service failure level by the monitoring node of the control plane L207 based on the service indicator data of each level, and then schedules the object storage service tasks of the service failure level by the node (scheduling node) of the scheduling layer L206.

[0139] In the embodiments disclosed herein, the cloud platform obtains service metrics for each level based on the full-link access logs, enabling it to track the quality of service provision at each level and quickly locate the level where the service is substandard in order to execute the scheduling task for that level.

[0140] Figure 3 This is a schematic diagram illustrating the linkage between service metrics and task scheduling, as provided in an embodiment of this disclosure. Figure 3As shown in this embodiment, the service node's layer can be divided into a load balancing layer, an access layer, a metadata layer, and a data layer. The cloud platform can determine the service indicator data corresponding to each layer based on the access logs of the service nodes at each layer through the control node of the control plane L301, and construct service level indicators L3011 for the load balancing layer, L3012 for the access layer, L3013 for the metadata layer, and L3014 for the data layer. By comparing the service indicator data of each layer with the service level target values ​​corresponding to the service indicator data in the service level target module L302, it can determine whether the service indicator data of each layer meets the standards. Then, the cloud platform, through the monitoring node of the control plane L301, determines the layer with substandard service indicator data based on the substandard service indicator data, and issues an alarm for the substandard service indicator data of the substandard layer through the alarm module L3015. Meanwhile, the cloud platform listens for alarm events through nodes (scheduling nodes) in the scheduling layer L303 based on the Argo management tool L3011. Once an alarm event notification is received, the platform schedules object storage service tasks for the substandard level through the garbage collection task module L3032, lifecycle management task module L3033, silent data verification task module L3034, data repair task module L3035, and metadata repair task module L3036.

[0141] In the embodiments disclosed herein, the cloud platform obtains service metrics for each level based on the full-link access logs, enabling it to track the quality of service provision at each level and quickly locate the level where the service is substandard in order to execute the scheduling task for that level.

[0142] Figure 4 A schematic diagram of the implementation flow of an object storage service method provided in this embodiment of the disclosure. Figure 2 Applied in cloud platforms, such as Figure 4 As shown, it includes the following steps:

[0143] S401. The cloud platform constructs object storage for full-link access logs through service nodes, including load balancing layer, access layer, metadata layer, and data layer.

[0144] S402. Store the entire link access log to the log storage and analysis engine through the control node;

[0145] S403. By using the aggregation task through the control node, the data in the log storage analysis engine is calculated to obtain service metrics at various levels.

[0146] S404. By constructing a linkage between service indicator fluctuations and task scheduling at the scheduling layer through monitoring nodes, pause and resume signals are sent when service indicators are abnormal and when indicators are at their optimal levels, respectively.

[0147] S405. The scheduling layer executes the corresponding task scheduling according to the received linkage signal.

[0148] The object storage service method of this disclosure is applied to a cloud platform, which provides object storage services through nodes. In this disclosure, nodes may include: control nodes, service nodes, monitoring nodes, scheduling nodes, etc.

[0149] In this embodiment of the disclosure, the cloud platform can construct a traceable full-link access log through service nodes. The full-link access log includes access logs of service nodes at each level covered by the link, including the load balancing layer, access layer, metadata layer, and data layer.

[0150] In this embodiment, the cloud platform can write end-to-end access logs into the log storage and analysis engine via the control node, and use aggregation functions to calculate the data in the log storage and analysis engine to determine the service metric data corresponding to each level. The service metric data may include multiple service metrics such as throughput, bandwidth, and latency. Each service metric data may include data from multiple dimensions.

[0151] In this embodiment of the disclosure, the cloud platform can establish a linkage between service indicator fluctuations and scheduling tasks through monitoring nodes and nodes in the scheduling layer (scheduling nodes). When the data in any dimension fails to meet the standard, a signal to suspend the scheduling task is sent through the monitoring node; when the service at the level meets the standard and the service indicator data reaches the preset optimal indicator data threshold, a signal to resume the scheduling task is sent through the monitoring node.

[0152] In this embodiment of the disclosure, the cloud platform schedules the corresponding level of object storage service tasks according to the received linkage signal through the scheduling node. By scheduling the tasks through the scheduling layer, the service level target of object storage is guaranteed in a timely manner.

[0153] In the embodiments disclosed herein, the cloud platform obtains service metrics for each level based on the full-link access logs, enabling it to track the quality of service provision at each level and quickly locate the level where the service is substandard in order to execute the scheduling task for that level.

[0154] Figure 5 This is a block diagram of a cloud platform provided in an embodiment of the present disclosure, such as... Figure 5 As shown, the cloud platform 500 includes:

[0155] Control node 501 is used to obtain the full-link access log of the object storage service and determine the service indicator data corresponding to each level based on the access log of the service nodes at each level; wherein, the full-link access log includes the access log of the service nodes at each level covered by the link.

[0156] Monitoring node 502 is used to determine the level of service that is not up to standard based on the service indicator data of each level;

[0157] Scheduling node 503 is used to schedule object storage service tasks at the substandard level.

[0158] In some embodiments, the monitoring node 502 is further configured to compare the service indicator data of each level with the preset indicator data threshold corresponding to the level to obtain a comparison result; and determine whether the level is substandard based on the comparison result of each level.

[0159] In some embodiments, the service indicator data includes multiple service indicator data; each service indicator data includes data under multiple dimensions; the preset indicator data threshold corresponding to the level includes the threshold corresponding to the data under each dimension in the multiple service indicator data of the level;

[0160] The monitoring node 502 is also used to compare the data under each dimension of the service indicator data of the level with the threshold corresponding to the dimension for each service indicator data of the level, and obtain the comparison result; based on the comparison result corresponding to the data under each dimension of the service indicator data of each level, determine whether the level is substandard.

[0161] In some embodiments, the layers include: a metadata layer, a data layer, an access layer, and a load balancing layer; the object storage service tasks include: a garbage collection task, a lifecycle management task, a silent data verification task, a data repair task, and a metadata repair task;

[0162] The scheduling node 503 is further configured to schedule the garbage collection task, lifecycle management task, and metadata repair task of the metadata layer if the service of the metadata layer is substandard; schedule the garbage collection task, silent data verification task, and data repair task of the data layer if the service of the data layer is substandard; schedule the garbage collection task, lifecycle management task, silent data verification task, data repair task, and metadata repair task of the access layer if the service of the access layer is substandard; and schedule the silent data verification task of the load balancing layer if the service of the load balancing layer is substandard.

[0163] In some embodiments, the scheduling node 503 is further configured to suspend the object storage service tasks of the substandard level; and resume the object storage service tasks of the level after the service of the substandard level meets the requirements.

[0164] In some embodiments, the scheduling node 503 is further configured to restore the object storage service task of the level if the service indicator data of the level reaches a preset optimal indicator data threshold.

[0165] In some embodiments, the scheduling node 503 is further configured to restore the object storage service task of the level within a preset recovery count threshold range until the object storage service task of the level is successfully restored.

[0166] In some embodiments, the non-compliant levels include multiple non-compliant levels;

[0167] The scheduling node 503 is further configured to determine the lowest level among the multiple non-compliant levels based on the order of the multiple non-compliant levels and all levels covered by the link; and to schedule the object storage service task of the lowest level.

[0168] Correspondingly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements some or all of the steps in the above method.

[0169] It should be noted that the descriptions of the storage medium and device embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this disclosure, please refer to the descriptions of the method embodiments of this disclosure for understanding.

[0170] It should be understood that the phrase "an embodiment" or "one embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this disclosure. Therefore, "in one embodiment" or "one embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this disclosure, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure. The sequence numbers of the above-described embodiments are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0171] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0172] In the several embodiments provided in this disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components may be combined, or integrated into another system, or some features may be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0173] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0174] In addition, each functional unit in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0175] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0176] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, magnetic disks, or optical disks.

[0177] The above description is merely an embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A method for storing object storage services, characterized in that, When applied to a cloud platform, the method includes: The cloud platform's control node obtains the object storage service's full-link access logs; wherein, the full-link access logs include access logs of the object storage service's access requests at each level of service nodes covered by the link, the levels including metadata layer, data layer, access layer, and load balancing layer; The control node determines the service indicator data corresponding to each level based on the access logs of the service nodes at each level. The monitoring nodes of the cloud platform determine the levels where services are not up to standard based on service indicator data at each level. The object storage service tasks at the substandard level are paused by the scheduling node; Once the service at the substandard level meets the standards, the object storage service task at that level is restored through the scheduling node.

2. The method according to claim 1, characterized in that, The process of determining the substandard service levels through monitoring nodes on the cloud platform based on service indicator data at each level includes: For each level, the service indicator data of the level is compared with the preset indicator data threshold corresponding to the level through the monitoring node to obtain the comparison result; The monitoring nodes determine whether a given level is not meeting service standards based on the comparison results of each level.

3. The method according to claim 2, characterized in that, The service indicator data includes multiple service indicator data; each of the service indicator data includes data under multiple dimensions; the preset indicator data threshold corresponding to the level includes the threshold corresponding to the data under each dimension in the multiple service indicator data of the level; The step of comparing the service indicator data of the level with the preset indicator data threshold corresponding to the level through the monitoring node to obtain the comparison result includes: For each service indicator data at the aforementioned level, the monitoring node compares the data under each dimension of the service indicator data at the aforementioned level with the threshold corresponding to the dimension to obtain the comparison result; The step of determining whether a level's service is substandard based on comparison results at each level through the monitoring node includes: The monitoring nodes determine whether a given level of service is substandard by comparing the data across different dimensions of the service metrics at each level.

4. The method according to claim 1, characterized in that, The layers include: metadata layer, data layer, access layer, and load balancing layer; the object storage service tasks include: garbage collection task, lifecycle management task, silent data verification task, data repair task, and metadata repair task; The step of suspending the object storage service task at the substandard level through the scheduling node includes: If the service of the metadata layer is not up to standard, the garbage collection task, the lifecycle management task, and the metadata repair task of the metadata layer shall be suspended through the scheduling node; If the service of the data layer is not up to standard, the garbage collection task, the silent data verification task, and the data repair task of the data layer are suspended through the scheduling node. If the services of the access layer are not up to standard, the garbage collection task, lifecycle management task, silent data verification task, data repair task and metadata repair task of the access layer shall be suspended through the scheduling node; If the service of the load balancing layer is not up to standard, the silent data verification task of the load balancing layer is suspended through the scheduling node.

5. The method according to claim 1, characterized in that, The task of restoring the object storage service of the specified level through the scheduling node includes: If the service metric data of the aforementioned level reaches the preset optimal metric data threshold, the object storage service task of the aforementioned level is restored through the scheduling node.

6. The method according to claim 1, characterized in that, The task of restoring the object storage service of the specified level through the scheduling node includes: Within a preset recovery count threshold, the object storage service task of the specified level is restored through the scheduling node until the object storage service task of the specified level is successfully restored.

7. The method according to any one of claims 1 to 6, characterized in that, The substandard levels include multiple substandard levels; The step of suspending the object storage service task at the substandard level through the scheduling node includes: The scheduling node determines the lowest level among the multiple non-compliant levels based on the order of all levels covered by the link. The scheduling node suspends the lowest-level object storage service task.

8. A cloud platform, characterized in that, The cloud platform includes: A control node is used to obtain the full-link access log of the object storage service and determine the service indicator data corresponding to each level based on the access log of the service nodes at each level. The full-link access log includes the access log of the access request of the object storage service at each level of the service nodes covered by the link. The levels include the metadata layer, the data layer, the access layer, and the load balancing layer. Monitoring nodes are used to determine the levels where services are not up to standard based on service indicator data at each level. A scheduling node is used to suspend the object storage service tasks of the substandard level; and to resume the object storage service tasks of the substandard level after the service meets the standards.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements any one of claims 1 to 7, an object storage service method applied to a cloud platform.

Images