Cluster capacity management method and device, storage medium and electronic equipment

Abstract

The invention provides a cluster capacity management method and device, a storage medium and electronic equipment, and the method comprises the steps: determining a target data popularity division model based on a data popularity division model indicated by a data bucket creation operation when the data bucket creation operation is detected, the target data popularity division model is any data popularity division model in a plurality of data popularity division models; creating a target data bucket based on the target data popularity division model; and when a data migration task for the target data bucket is detected, performing data popularity division on the data in the target data bucket according to the target data popularity division model to obtain a target data popularity division result, and performing data migration on the data in the target data bucket according to the target data popularity division result. According to the embodiment of the invention, cluster capacity management can be flexibly carried out.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular, to a method, device, storage medium, and electronic device for cluster capacity management. Background Art

[0002] Currently, cluster distributed object storage has been widely used; however, related technologies usually perform data heat level classification at the granularity of the entire cluster, resulting in poor flexibility in cluster capacity management. Based on this, there is currently no good solution for how to flexibly perform cluster capacity management. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a method, device, storage medium, and electronic device for cluster capacity management to solve problems such as poor flexibility in cluster capacity management caused by related technologies. That is to say, embodiments of the present invention can implement refined heat management at the bucket granularity by setting data heat division models for different data buckets (buckets), so as to perform intelligent layering of business bucket storage, and can perform intelligent layering of hot data, cold data, etc. according to the business to achieve fine-grained resource capacity management, that is, can flexibly perform cluster capacity management.

[0004] According to one aspect of the present invention, there is provided a method for cluster capacity management, the method comprising:

[0005] When detecting a data bucket creation operation, determining a target data heat division model based on the data heat division model indicated by the data bucket creation operation, the target data heat division model being any one of a plurality of data heat division models;

[0006] Based on the target data heat division model, creating a target data bucket;

[0007] When detecting a data migration task for the target data bucket, performing data heat division on the data in the target data bucket according to the target data heat division model to obtain a target data heat division result, and performing data migration on the data in the target data bucket according to the target data heat division result.

[0008] According to another aspect of the present invention, there is provided a device for cluster capacity management, the device comprising:

[0009] A determination unit, configured to determine a target data heat division model based on the data heat division model indicated by the data bucket creation operation when detecting a data bucket creation operation, the target data heat division model being any one of a plurality of data heat division models;

[0010] A processing unit, configured to create a target data bucket based on the target data heat division model;

[0011] The processing unit is further configured to, when detecting a data migration task for the target data bucket, perform data heat division on the data in the target data bucket according to the target data heat division model to obtain a target data heat division result, and perform data migration on the data in the target data bucket according to the target data heat division result.

[0012] According to another aspect of the present invention, there is provided an electronic device, which includes a processor and a memory storing a program, wherein the program includes instructions that, when executed by the processor, cause the processor to execute the method mentioned above.

[0013] According to another aspect of the present invention, there is provided a non-transitory computer-readable storage medium storing computer instructions, and the computer instructions are used to cause a computer to execute the method mentioned above.

[0014] In the embodiments of the present invention, when detecting a data bucket creation operation, the target data heat division model can be determined based on the data heat division model indicated by the data bucket creation operation, and the target data heat division model is any one of multiple data heat division models; and a target data bucket is created based on the target data heat division model. Based on this, when detecting a data migration task for the target data bucket, the data in the target data bucket can be subjected to data heat division according to the target data heat division model to obtain a target data heat division result, and the data in the target data bucket can be subjected to data migration according to the target data heat division result. It can be seen that the embodiments of the present invention can achieve fine-grained heat management according to the bucket granularity by setting different data heat division models for data buckets, so as to perform intelligent layering on the business bucket storage, and can perform intelligent layering of hot data, cold data, etc. according to the business to achieve fine-grained resource capacity management, that is, the cluster capacity can be managed flexibly. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In the following description of exemplary embodiments with reference to the accompanying drawings, more details, features, and advantages of the present invention are disclosed. In the drawings:

[0016] Figure 1 A flowchart showing a method for cluster capacity management according to an exemplary embodiment of the present invention;

[0017] Figure 2 A flowchart showing another method for cluster capacity management according to an exemplary embodiment of the present invention;

[0018] Figure 3Shows a schematic diagram of data synchronization according to an exemplary embodiment of the present invention;

[0019] Figure 4 Shows a schematic diagram of index statistical data according to an exemplary embodiment of the present invention;

[0020] Figure 5 Shows a schematic diagram of a capacity index value according to an exemplary embodiment of the present invention;

[0021] Figure 6 Shows a schematic diagram of a service resource index value according to an exemplary embodiment of the present invention;

[0022] Figure 7 Shows a schematic diagram of bucket trend change display index data according to an exemplary embodiment of the present invention;

[0023] Figure 8 Shows a schematic diagram of a monitoring index trend chart according to an exemplary embodiment of the present invention;

[0024] Figure 9 Shows a schematic block diagram of a cluster capacity management device according to an exemplary embodiment of the present invention;

[0025] Figure 10 Shows a structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present invention. Detailed Embodiments

[0026] Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Instead, these embodiments are provided to more thoroughly and completely understand the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not used to limit the scope of protection of the present invention.

[0027] It should be understood that the various steps recited in the method embodiments of the present invention can be executed in a different order and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this regard.

[0028] As used herein, the term "including" and its variations are open-ended, that is, "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description. It should be noted that the concepts such as "first", "second", etc. mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the order or mutual dependency relationship of the functions performed by these devices, modules or units.

[0029] It should be noted that the modifications of "one" and "a plurality of" mentioned in the present invention are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly specified in the context, it should be understood as "one or more".

[0030] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0031] It should be noted that the execution subject of the cluster capacity management method provided in the embodiments of the present invention can be one or more electronic devices, and the present invention does not make any limitation in this regard; among them, the electronic device can be a terminal (i.e., a client) or a server. Then, when the execution subject includes multiple electronic devices, and at least one terminal and at least one server are included in the multiple electronic devices, the cluster capacity management method provided in the embodiments of the present invention can be jointly executed by the terminal and the server. Correspondingly, the terminal mentioned herein can include, but is not limited to: smart phones, tablet computers, laptop computers, desktop computers, smart watches, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, and so on. The server mentioned herein can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms, and so on.

[0032] Optionally, the execution subject of the cluster capacity management method may also be referred to as a cluster capacity management platform. That is to say, the execution subject of the cluster capacity management method provided in the embodiments of the present invention may be one or more electronic devices constituting the cluster capacity management platform. Then, the cluster capacity management method provided in the embodiments of the present invention may be executed by the cluster capacity management platform. Optionally, the cluster mentioned in the embodiments of the present invention may be a BOS (Business & Operation Support) cluster, etc.; the embodiments of the present invention do not limit this.

[0033] Based on the above description, an embodiment of the present invention proposes a cluster capacity management method. This cluster capacity management method may be executed by one or more of the above-mentioned electronic devices, that is, this cluster capacity management method may be executed by the cluster capacity management platform. For the convenience of description, hereinafter, it will be described by taking an electronic device executing this cluster capacity management method as an example; as Figure 1 shown, this cluster capacity management method may include the following steps S101 - S103:

[0034] S101, when a data bucket creation operation is detected, determine a target data heat division model based on the data heat division model indicated by the data bucket creation operation. The target data heat division model is any one of multiple data heat division models.

[0035] Among them, a data bucket can also be expressed as a bucket, which is provided for users to use and can be understood as a virtual file directory. Users can use the bucket with an authorized account. Optionally, the storage method of a data bucket may include but is not limited to at least one of the following: standard storage, low-frequency storage, and cold storage, etc.; the embodiments of the present invention do not limit this. Among them, standard storage can be used to store extremely hot data, low-frequency storage can be used to store hot data, cold storage can be used to store cold data, etc.

[0036] Optionally, multiple data heat division models can be set according to experience or according to actual needs. The embodiments of the present invention do not limit this. Exemplarily, multiple data heat division models may include Model A, Model B, and Model C; among them, Model A can be used to indicate that data that has not been accessed for more than 1 year is cold data, data that has been accessed within 1 year and the number of accesses does not exceed 36,000 is hot data, and data that has been accessed more than 36,000 times within 1 year is extremely hot data; Model B can be used to indicate that data that has not been accessed for more than 2 years is cold data, data that has been accessed within 2 years and the number of accesses does not exceed 72,000 is hot data, and data that has been accessed more than 72,000 times within 2 years is extremely hot data; Model C can be used to indicate that data that has not been accessed for more than 3 years is cold data, data that has been accessed within 3 years and the number of accesses does not exceed 108,000 is hot data, and data that has been accessed more than 108,000 times within 3 years is extremely hot data, etc.

[0037] In an embodiment of the present invention, a user (i.e., any user) can perform a data bucket creation operation, and then the electronic device can detect the data bucket creation operation. For example, detecting a data bucket creation instruction can be used to achieve detecting the data bucket creation operation, and so on. It should be noted that the embodiments of the present invention do not limit the specific implementation manner of the data bucket creation operation. Exemplarily, when a user performs a data bucket creation operation, the user can select a required data heat division model from multiple data heat division models, or can input the model identifier (such as model name or model number, etc.) of the data heat division model, and so on. Optionally, when a user performs a data bucket creation operation, the user may not set a data heat division model. In this case, the target data heat division model can be a specified policy model, that is, the specified policy model can be used as the target data heat division model (i.e., the data heat division model of the target data bucket). Optionally, the specified policy model can be one of the data heat division models in multiple data heat division models, or may not be a data heat division model in multiple data heat division models. The embodiments of the present invention do not limit this. Based on this, the specified policy model can be set according to experience or according to actual requirements. The embodiments of the present invention do not limit this. Correspondingly, when the data bucket creation operation is set with a data heat division model, the set data heat division model (i.e., the data heat division model indicated by the data bucket creation operation) can replace the specified policy model to be used as the target data heat division model. Based on this, when determining the target data heat division model based on the data heat division model indicated by the data bucket creation operation, the data heat division model indicated by the data bucket creation operation can be used as the target data heat division model. At this time, the data heat division model indicated by the data bucket creation operation is not empty; or, when the data heat division model indicated by the data bucket creation operation is empty, the specified policy model can be used as the target data heat division model, and so on.

[0038] Optionally, a user can perform a data bucket creation operation through noahee (an enterprise-level operation and maintenance platform, that is, the cluster capacity management platform can be noahee). Then the electronic device can detect the data bucket creation operation and can determine the data heat division model indicated by the data bucket creation operation and the bucket creation information of the target data bucket, etc. Optionally, the bucket creation information of a data bucket can include but is not limited to at least one of the following: the bucket name of the corresponding data bucket, the owner, the affiliated department, the storage method, etc. The embodiments of the present invention do not limit this.

[0039] Optionally, the electronic device may also respond to the detected machine management display operation and display the model display information of each model in the target cluster; optionally, the target cluster may be any cluster, and the embodiments of the present invention do not limit this. Optionally, the model display information of a model may include, but is not limited to, at least one of the following: the model identifier of the corresponding model (such as the model name or model number, etc., which can be used to indicate the package type, such as for indicating memory, disk, and CPU (Central Processing Unit)), the disk occupancy ratio ((the disk size of this model × the number of disks)), the quantity (i.e., the package quantity, which is also the number of models), the unit price, the usage (editable mode, facilitating data change), and so on; the embodiments of the present invention do not limit this.

[0040] S102. Create a target data bucket based on the target data heat division model.

[0041] In the embodiments of the present invention, the electronic device may use the target data heat division model as the data heat division model of the target data bucket, that is, the target data heat division model may be the data heat division model of the target data bucket, so as to perform data heat division (i.e., heat stratification) on the data in the target data bucket according to the target data heat division model during subsequent data heat division.

[0042] Optionally, the target data bucket may be any data bucket in the target cluster, and the embodiments of the present invention do not limit this.

[0043] S103. When a data migration task for the target data bucket is detected, perform data heat division on the data in the target data bucket according to the target data heat division model to obtain a target data heat division result, and perform data migration on the data in the target data bucket according to the target data heat division result.

[0044] Optionally, the data migration task may be a timed task, that is, a data migration task may be triggered every preset data migration duration; optionally, the preset data migration duration may be set according to experience or according to actual requirements, and the embodiments of the present invention do not limit this. Optionally, the electronic device may also respond to the detected data migration task trigger operation to detect the data migration task. Optionally, the data migration task may be a data migration task for all data buckets in the target cluster. Then, when the data migration task is detected, it can be determined that a data migration task for the target data bucket is detected; or when the user performs a data migration operation on the target data bucket, the electronic device can also detect the data migration task for the target data bucket, and so on; the embodiments of the present invention do not limit this. For the convenience of description, subsequent descriptions will all take the data migration task as a timed task for all data buckets in the target cluster as an example.

[0045] Optionally, a data heat division result may include, but is not limited to, at least one of the following: at least one extremely hot data, at least one hot data, and at least one cold data, etc., which are not limited in the embodiments of the present invention. Among them, a piece of data may also be referred to as a file (i.e., file data), and one service may correspond to at least one file. Optionally, when the storage method of the target data bucket is standard storage, data migration can be performed on the hot data and / or cold data in the target data bucket; when the storage method of the target data bucket is low-frequency storage, data migration can be performed on the extremely hot data and / or cold data in the target data bucket; when the storage method of the target data bucket is cold storage, data migration can be performed on the extremely hot data and / or hot data in the target data bucket, so as to achieve data migration of the data in the target data bucket according to the target data heat division result, so that the extremely hot data in the target data bucket is stored in the data bucket with standard storage, the hot data is stored in the data bucket with low-frequency storage, and the cold data is stored in the data bucket with cold storage, etc. In the embodiments of the present invention, the cost of the machine for low-frequency storage is less than or equal to the cost of the machine for standard storage, and the cost of the machine for cold storage is less than or equal to the cost of the machine for low-frequency storage.

[0046] Based on this, the embodiments of the present invention can perform storage according to data heat through data layering, thereby improving data storage efficiency and reducing data storage costs. That is to say, the embodiments of the present invention can realize the automatic management of the BOS cluster capacity to cope with the capacity risk of the bos cluster, improve the stability of the bos cluster, and realize the intelligent layering of business storage through the intelligent layering of data buckets, that is, the intelligent layering of hot data, cold data, etc. according to the service to achieve fine-grained resource capacity management (i.e., cluster capacity management).

[0047] Optionally, the electronic device can also monitor the capacity utilization rate (i.e., the percentage of capacity used) of each data bucket in the target cluster in real time, and perform a capacity warning operation on any data bucket when the capacity utilization rate of any data bucket in the target cluster reaches the preset capacity utilization rate threshold, so as to realize the automatic warning function and quickly locate the specific resources in the face of a sudden increase in cluster capacity. Optionally, the electronic device can perform a capacity utilization rate monitoring every preset capacity usage monitoring duration; optionally, the preset capacity usage monitoring duration can be set according to experience or according to actual needs, which is not limited in the embodiments of the present invention; based on this, when the preset capacity usage monitoring duration is at the minute level, the embodiments of the present invention can detect the risk of sudden increase in capacity at the minute level to achieve resource location at the minute level, etc.

[0048] In an embodiment of the present invention, when a data bucket creation operation is detected, a target data heat division model can be determined based on the data heat division model indicated by the data bucket creation operation. The target data heat division model is any one of multiple data heat division models; and based on the target data heat division model, a target data bucket is created. Based on this, when a data migration task for the target data bucket is detected, the data in the target data bucket can be divided according to the target data heat division model to obtain a target data heat division result, and the data in the target data bucket can be migrated according to the target data heat division result. It can be seen that the embodiment of the present invention can achieve fine-grained heat management at the bucket granularity by setting different data heat division models for different data buckets, so as to perform intelligent stratification of business bucket storage, and can perform intelligent stratification of multiple layers such as hot data and cold data according to the business to achieve fine-grained resource capacity management, that is, the cluster capacity can be managed flexibly.

[0049] Based on the above description, an embodiment of the present invention further proposes a more specific cluster capacity management method. Correspondingly, this cluster capacity management method can be executed by the above-mentioned electronic device (terminal or server), that is, this cluster capacity management method can be executed by the cluster capacity management platform. For the convenience of description, hereinafter, it will be described by taking the electronic device executing this cluster capacity management method as an example; please refer to Figure 2 , this cluster capacity management method may include the following steps S201 - S205:

[0050] S201, when a data bucket creation operation is detected, determine a target data heat division model based on the data heat division model indicated by the data bucket creation operation. The target data heat division model is any one of multiple data heat division models.

[0051] S202, create a target data bucket based on the target data heat division model.

[0052] Optionally, the electronic device may also respond to the detected modification operation of the data bucket heat division model, and determine the data bucket to be modified and the data division model to be modified indicated by the modification operation of the data bucket heat division model; based on this, the data division model to be modified can be updated to the data heat division model of the data bucket to be modified. It should be noted that the embodiments of the present invention do not limit the specific implementation manner of the modification operation of the data bucket heat division model; based on this, the embodiments of the present invention can modify the data heat division model of any data bucket through the modification operation of the data bucket heat division model. Optionally, the data bucket to be modified can be any data bucket, and the data division model to be modified can be any data heat division model among multiple data heat division models, and the embodiments of the present invention do not limit this. Among them, a data heat division model can be used to indicate settlement rules and / or floating rules at the bucket level, etc.

[0053] Optionally, after detecting the settlement start operation performed by a target object (such as an administrator, etc.), the electronic device may allow the user to perform operations such as modifying the data bucket heat division model; it should be noted that the embodiments of the present invention do not limit the specific implementation manner of the settlement start operation, such as the settlement start operation can be performed through the switch for settlement start, etc. Optionally, the electronic device may perform policy distribution to distribute the data heat division model of the data bucket to be modified and / or the data heat division model of the target data bucket, so as to store it in the database; optionally, the database may create a new table to store the model identifier of the data heat division model of the data bucket, so as to implement the storage of the data heat division model of the data bucket. In the embodiments of the present invention, the data heat division model may also be referred to as a data heat division strategy, etc.

[0054] S203. When detecting a data migration task for a target data bucket, perform data heat division on the data in the target data bucket according to the target data heat division model to obtain a target data heat division result, and perform data migration on the data in the target data bucket according to the target data heat division result.

[0055] Optionally, the data migration task can be an offline computing task. Correspondingly, when the data heat division models of each data bucket in the target cluster are stored in the database (i.e., the model identifiers of the data heat division models of each data bucket are stored in the database), the electronic device can read the data heat division models of each data bucket from the database to implement reading the target data heat division model (i.e., the data heat division model of the target data bucket), so as to respectively perform data heat division on the data in the corresponding data bucket according to the data heat division models of each data bucket, obtain the data heat division results of each data bucket, and then respectively perform data migration on the data in each data bucket. Optionally, when the data heat division model of a data bucket is not stored in the database (i.e., the data heat division model of this data bucket is empty), the specified policy model can be determined as the data heat division model of the corresponding data bucket.

[0056] Optionally, when performing data heat division on the data in the target data bucket according to the target data heat division model to obtain the target data heat division result (i.e., the data heat division result of the target data bucket), the electronic device can determine multiple file heat rules (such as reading the file heat rule configuration file to determine multiple file heat rules), and respectively determine the file heat indication information of each data in the target data bucket under each file heat rule in the multiple file heat rules, such as determining the file heat indication information of each data under each file heat rule (i.e., the specific values of each file heat rule) through the file heat rule configuration file; based on this, the data in the target data bucket can be subjected to data heat division based on the file heat indication information of each data under each file heat rule and the target data heat division model to obtain the target data heat division result.

[0057] Optionally, multiple file heat rules may include, but are not limited to, the last access time of the file (i.e., the most recent access time of the file) and the file access frequency (i.e., the number of accesses), etc. The embodiments of the present invention do not limit this. Optionally, the file heat rule configuration file may be stored in an offline bucket. Exemplarily, for any data in the target data bucket, assuming that the target data heat division model is the above-mentioned Model A, and the file heat indication information of any data under the last access time of the file (i.e., the last access time of the file of any data) is within one year, and the file heat indication information of any data under the file access frequency (i.e., the file access frequency of any data) is 3,000 times within one year, then it can be determined that any data is hot data. For example, any data is added to the hot data in the target data bucket to obtain the target data heat division result. The target data heat division result may include extremely hot data, hot data, and cold data in the target data bucket. Optionally, the extremely hot data, hot data, or cold data in the target data bucket may be empty (for example, if the cold data in the target data bucket is empty, it means that the target data bucket does not include cold data at this time), or the number of extremely hot data, hot data, or cold data in the target data bucket may be one or more, etc.; the embodiments of the present invention do not limit this.

[0058] Optionally, the electronic device may include an intelligent sedimentation function module; in this case, when migrating the data in the target data bucket according to the target data heat division result, the data to be sedimented in the target heat division result can be uploaded to the intelligent sedimentation function module, and the data to be sedimented is migrated to achieve the migration of the data in the target data bucket according to the target data heat division result. Optionally, when the storage method of the target data bucket is standard storage, the data to be sedimented may include the hot data and / or cold data in the target data bucket, so as to sink the hot data and / or cold data in the target data bucket to achieve data migration; when the storage method of the target data bucket is low-frequency storage, the data to be sedimented may include the extremely hot data and / or cold data in the target data bucket, so as to float the extremely hot data in the target data bucket and / or sink the cold data in the target data bucket; when the storage method of the target data bucket is cold storage, the data to be sedimented may include the extremely hot data and / or hot data in the target data bucket, so as to float the extremely hot data and / or hot data in the target data bucket to achieve data migration, etc.

[0059] Optionally, when migrating the data in the target data bucket according to the result of dividing the target data by heat, the electronic device may obtain disaster recovery data indication information, and respectively traverse each piece of cold data in the target data bucket according to the result of dividing the target data by heat, and use the currently traversed cold data as the current cold data. Based on this, it is possible to determine whether the current cold data exists in the disaster recovery data based on the disaster recovery data indication information. If the current cold data exists in the disaster recovery data, the current cold data is migrated to the offline data bucket corresponding to the target data bucket. At this time, the target data bucket can be an online data bucket, and an online data bucket can be a data bucket of standard storage or low-frequency storage, that is, the storage method of the target data bucket can be standard storage or low-frequency storage at this time, so as to realize sinking the cold data in the target data bucket to the corresponding offline data bucket. After traversing each piece of cold data in the target data bucket, the data migration of the cold data in the target data bucket is realized. Correspondingly, if the current cold data does not exist in the disaster recovery data, the current cold data may not be migrated to the offline data bucket corresponding to the target data bucket. Optionally, an offline data bucket can be a data bucket with a cold storage method. Based on this, the embodiment of the present invention can ensure the stability of the cluster through disaster recovery data, sink the cold data existing in the disaster recovery data to the offline data bucket, and effectively reduce the cost of storage resources while ensuring the stability of the cluster. Optionally, a data bucket of standard storage can correspond to at least one data bucket of low-frequency storage and at least one data bucket of cold storage (i.e., offline data bucket). For example, at least one data bucket of standard storage, at least one data bucket of low-frequency storage, and at least one data bucket of cold storage under one service can correspond to each other. Optionally, the data buckets under one service can be created according to experience or according to actual needs, and the embodiment of the present invention does not limit this.

[0060] Optionally, the disaster recovery data indication information can be used to indicate disaster recovery data. Then, it can be determined whether the current cold data exists in the disaster recovery data by judging whether the data identifier of the current cold data (such as data number or data index, etc., which can also be called file identifier, etc.) exists in the disaster recovery data indication information. Optionally, the disaster recovery data indication information can include but is not limited to at least one of the following: disaster recovery data, data identifiers of each data in the disaster recovery data, etc. The embodiments of the present invention do not make any limitations in this regard. Optionally, the electronic device can obtain the disaster recovery data indication information into the online data bucket to trigger the execution of traversing each cold data in the target data bucket according to the target data heat division result; or, the disaster recovery data indication information can also be synchronized to the target offline data bucket to trigger the execution of traversing each cold data in the target data bucket according to the target data heat division result, and so on. Optionally, the target offline data bucket can be any offline data bucket, that is, the target offline data bucket can be set according to experience or actual requirements. The embodiments of the present invention do not make any limitations in this regard. Optionally, the disaster recovery data indication information can be the content in the BOS metadata. The electronic device can synchronize the BOS metadata in the drds (a kind of database) in the BOS to the online data bucket and / or the offline data bucket through a java (a programming language) script to obtain the disaster recovery data indication information, such as Figure 3 as shown, and so on. Among them, Figure 3 MapReduce Hive in it can represent a data warehouse tool for performing data analysis and processing, etc. Access can represent database management. For example, Access information can include but is not limited to the access times of all data within each day in the specified time range, etc., so that the access times, etc. can be associated with the metadata, and so on. Optionally, the BOS metadata (which can also be called metadata information) can also include but is not limited to the underlying resource information of each data bucket in the target cluster (such as quota (that is, the maximum capacity that can be stored), etc.), and so on. The embodiments of the present invention do not make any limitations in this regard. Optionally, the specified time range can be set according to experience or actual requirements, or can be determined according to the data heat division model of the data bucket where the corresponding data is located (in this case, the specified time ranges corresponding to different data can be affected by the data heat division model of the data bucket where they are located), and so on. The embodiments of the present invention do not make any limitations in this regard.

[0061] Optionally, the electronic device can also collect the log data in bos-nginx (a high-performance HTTP (HyperText Transfer Protocol) and reverse proxy server, which can be used to store disaster recovery data) into the online data bucket of the BOS through a data collection component to obtain the Access information; optionally, the log data or the Access information can also be synchronized to the offline data bucket to store the access times of each data at historical times, and so on.

[0062] Optionally, when the storage mode of the target data bucket is cold storage, the electronic device can also perform data migration on the extremely hot data and / or hot data in the target data bucket according to the target data heat division result, so as to migrate the extremely hot data in the target data bucket to the standard storage data bucket corresponding to the target data bucket (which can be simply referred to as the standard storage data bucket), and / or migrate the hot data in the target data bucket to the low-frequency storage data bucket corresponding to the target data bucket (which can be simply referred to as the low-frequency storage data bucket), so as to realize floating the extremely hot data and / or hot data in the target data bucket to the corresponding online data bucket; or, when the storage mode of the target data bucket is low-frequency storage, data migration can also be performed on the extremely hot data and / or cold data in the target data bucket, and so on. In the embodiments of the present invention, the access frequency of the extremely hot data in a data bucket is greater than that of the hot data, and the access frequency of the hot data in a data bucket is greater than that of the cold data; correspondingly, the standard storage can be used to store the extremely hot data, the low-frequency storage can be used to store the hot data, the cold storage can be used to store the cold data, and so on.

[0063] S204. When a timing statistical task is detected, read the file heat rule configuration file and group at least one data bucket to obtain at least one grouping result.

[0064] Optionally, the timing statistical task can be an offline calculation task. In the embodiments of the present invention, the electronic device can trigger a timing statistical task every preset statistical task interval duration to detect the timing statistical task; optionally, the preset statistical task interval duration can be set according to experience or according to actual requirements, and the embodiments of the present invention do not limit this.

[0065] Optionally, the above at least one data bucket may include all data buckets in the target cluster.

[0066] Optionally, a data bucket may belong to a group; optionally, the group to which a data bucket belongs can be set by the user, or data buckets under the same service can be divided into the same group (that is, divided into the same grouping result), and so on, and the embodiments of the present invention do not limit this.

[0067] S205. According to the file heat rule configuration file and at least one statistical indicator, perform grouped statistics on each grouping result in at least one grouping result to obtain the indicator statistical data of each grouping result under each statistical indicator in at least one statistical indicator.

[0068] Optionally, at least one statistical metric may include, but is not limited to, at least one of the following: storage type, number of files, storage space, file popularity, etc., which are not limited in the embodiments of the present invention; based on this, the embodiments of the present invention can implement statistical storage type, statistical number of files, statistical storage space, and statistical file popularity, etc. Exemplarily, the metric statistical data of a grouping result under the storage type may include the number of online data and the number of offline data in all data buckets in the corresponding grouping result (the storage type may include online data and offline data), or may include the number of standard files (i.e., the number of data stored in the standard), the number of low-frequency files (i.e., the number of data stored at low frequency), and the number of cold files (i.e., the number of data stored in the cold), etc. in all data buckets in the corresponding grouping result; the metric statistical data of a grouping result under the number of files may include the number of files (i.e., the number of data) in all data buckets in the corresponding grouping result; the metric statistical data of a grouping result under the storage space may include the number of files in at least one storage location of all data buckets in the corresponding grouping result, and / or include the storage space of standard storage (i.e., the storage space of standard files (which may also be referred to as standard storage files)), the storage space of low-frequency storage (i.e., the storage space of low-frequency files (which may also be referred to as low-frequency storage files)), and the storage space of cold storage (i.e., the storage space of cold files (which may also be referred to as cold storage files)) in all data buckets in the corresponding grouping result. A storage space can be represented by a storage space indication information (such as storage location information, etc.), then the storage space indication information of the corresponding storage space may be included, and so on; the metric statistical data of a grouping result under the file popularity may include the file popularity indication information (such as the last access time and access frequency) of each data in all data buckets in the corresponding grouping result, and so on. Optionally, online data may include standard storage data and low-frequency storage data, and offline data may include cold storage data, then the storage type may include standard storage, low-frequency storage, and cold storage, etc. Based on this, the embodiments of the present invention do not limit the metric statistical data under a statistical metric. Optionally, the electronic device may also perform statistics on each data bucket in the target cluster respectively to obtain the metric statistical data of each data bucket under each statistical metric, and so on. Exemplarily, taking the storage type including standard storage and cold storage as an example for illustration, the metric statistical data of a data bucket under the storage type and file popularity may be as Figure 4 shown.

[0069] Optionally, the electronic device may generate a statistical result through the metric statistical data of each grouping result under each statistical metric in at least one statistical metric, and upload the statistical result to a specified data bucket for the user to download; optionally, the specified data bucket may be set according to experience or according to actual requirements, which is not limited in the embodiments of the present invention.

[0070] Optionally, the electronic device may also respond to a detected capacity management display operation, and determine the storage capacity management type to be displayed indicated by the capacity management display operation; based on this, each capacity management indicator in at least one capacity management indicator may be respectively determined with a capacity indicator value under the storage capacity management type to be displayed, and the capacity indicator values of each capacity management indicator under the storage capacity management type to be displayed may be displayed. Optionally, the storage capacity management type to be displayed may be any storage capacity management type in the set of storage capacity management types; optionally, the set of storage capacity management types may include but is not limited to at least one of the following: standard storage - common resource pool, online independent resource pool, cold storage - common resource pool, low - frequency storage - common resource pool, standard storage - online independent resource pool, etc., and the embodiments of the present invention do not limit this. It should be noted that the embodiments of the present invention do not limit the specific implementation manner of the capacity management display operation; by way of example, the user may perform a storage type selection operation in the capacity management display interface to implement the execution of the capacity management display operation. Then, when the selected storage type by the storage type selection operation is the standard storage - common resource pool, the storage capacity management type to be displayed indicated by the capacity management display operation may be the standard storage - common resource pool, as Figure 5 shown, etc. Among them, Figure 5 the PB in it may represent petabyte (a storage capacity unit).

[0071] Optionally, at least one capacity management indicator may include but is not limited to at least one of the following: physical capacity (i.e., underlying storage capacity) and logical capacity (i.e., the capacity divided by the administrator). Optionally, the capacity indicator value of the physical capacity under a storage capacity management type may include but is not limited to at least one of the following: the total physical capacity, space usage, and remaining space of the physical capacity under the corresponding storage capacity management type, etc., and the embodiments of the present invention do not limit this; correspondingly, the capacity indicator value of the logical capacity under a storage capacity management type may include but is not limited to at least one of the following: the allocated capacity (i.e., quota), space usage, and remaining space of the logical capacity under the corresponding storage capacity management type, etc., and the embodiments of the present invention do not limit this. Optionally, the electronic device may obtain the used capacity of the storage bucket through the GetBucketQuota (an interface) of the BOS, and may query the data interface from the bcm (a monitoring platform) to obtain it.

[0072] Optionally, the electronic device may also respond to a detected business resource management display operation, and determine the business resource management type to be displayed indicated by the business resource management display operation; based on this, each business resource management indicator in at least one business resource management indicator may be respectively determined with a business resource indicator value under the business resource management type to be displayed, and the business resource indicator values of each business resource management indicator under the business resource management type to be displayed may be displayed, such asFigure 6 As shown. Optionally, the business resource management type to be displayed may be any business resource management type in the set of business resource management types; optionally, the set of business resource management types may include, but is not limited to, at least one of the following: standard storage - common resource pool, online independent resource pool, cold storage - common resource pool, etc., and the embodiments of the present invention do not limit this. It should be noted that the embodiments of the present invention do not limit the specific implementation manner of the business resource management display operation.

[0073] Optionally, the at least one business resource management metric may include, but is not limited to, at least one of the following: offline services and online services, etc., and the embodiments of the present invention do not limit this. Optionally, the business resource metric values of a business resource management metric under a business resource management type may include, but is not limited to, at least one of the following: the quota application volume, quota usage volume, and quota remaining volume of the corresponding business resource management metric under the corresponding business resource management type, etc., and the embodiments of the present invention do not limit this.

[0074] Optionally, after detecting the business resource management display operation, the electronic device may also display a quota trend graph of each business resource management metric under the business resource management type to be displayed, such as Figure 6 the offline service quota trend and the online service quota trend in it, where MB may represent megabyte (a storage unit). Optionally, a quota trend graph may represent the trend of the amount change (such as the change in the application volume and the change in the usage volume) of a business resource management metric under the business resource management type to be displayed within a time range such as 30 days.

[0075] Optionally, the electronic device may also respond to the detected bucket trend change display operation, and determine the bucket screening condition indicated by the bucket trend change display operation; based on this, at least one screened data bucket may be determined according to the bucket screening condition, and the bucket trend change display metric data of each screened data bucket in the at least one screened data bucket may be displayed. The bucket trend change display metric data of a screened data bucket includes the usage increase rate of the corresponding screened data bucket. It should be noted that the embodiments of the present invention do not limit the specific implementation manner of the bucket trend change display operation. Optionally, the electronic device may obtain the corresponding metric from the bucket management platform.

[0076] Optionally, the bucket screening conditions may include, but are not limited to, at least one of the following: standard storage - common resource pool (which can screen out the standard storage in the common resource pool, that is, screen out the data buckets with the storage method of standard storage), standard storage - online resource pool (which can screen out the standard storage in the online resource pool), cold storage (which can screen out cold storage, that is, screen out all data buckets with the storage method of cold storage), etc. The embodiments of the present invention do not limit this. Optionally, a bucket trend change display index data for screening data buckets may also include, but are not limited to, at least one of the following: the bucket name of the corresponding screened data bucket, total usage / total quota, owner, owner department, and operations, etc., as Figure 7 shown. Among them, Figure 7 in Figure 7 , TB can represent terabyte (a storage capacity unit), and GB can represent gigabyte (a storage capacity unit).

[0077] Among them, the usage increase rate can be the latest usage - historical usage. Optionally, the user can also perform an increase rate window selection operation on the bucket trend change display interface, then the electronic device can determine the usage increase rate of any screened data bucket according to the increase rate window indicated by the increase rate window selection operation; exemplarily, when the increase rate window is 7 days, the usage increase rate of any screened data bucket can be the current usage of any screened data bucket - the usage of any screened data bucket 7 days ago, etc. Based on this, the embodiments of the present invention can allow the user to make decisions on the real-time nature of the data and / or comparison at the daily level, etc.

[0078] Optionally, the electronic device can also respond to the detected single data bucket (also called storage bucket) monitoring data display operation, and determine the data bucket to be displayed indicated by the single data bucket monitoring data display operation; based on this, the monitoring index data of the data bucket to be displayed under each data bucket monitoring index in at least one data bucket monitoring index can be determined, and a monitoring index trend graph of the data bucket to be displayed can be generated according to the monitoring index data of the data bucket to be displayed under each data monitoring index, so as to display this monitoring index trend graph (also called data bucket monitoring chart), as Figure 8 shown; among them, Figure 8The "bit" in it can represent a bit. Optionally, at least one data bucket monitoring metric may include but is not limited to at least one of the following: data bucket storage usage trend (such as 90 days, etc.), number of read and write requests for the data bucket, amount of read and write data for the data bucket, status code statistics, number of requests with excessive backend processing time, and quota increase, etc. The embodiments of the present invention do not limit this. Optionally, the monitoring metric data of the data bucket to be displayed under the data bucket storage usage trend may include but is not limited to at least one of the following: standard storage and cold storage (i.e., standard storage usage and cold storage usage) of the data bucket to be displayed within the target time range, etc. The embodiments of the present invention do not limit this. Optionally, the monitoring metric data of the data bucket to be displayed under the number of read and write requests for the data bucket may include but is not limited to at least one of the following: number of standard storage read and write requests, number of low-frequency storage read and write requests, and number of cold storage read and write requests of the data bucket to be displayed within the target time range, etc. The embodiments of the present invention do not limit this. Optionally, the monitoring metric data of the data bucket to be displayed under the amount of read and write data for the data bucket may include but is not limited to at least one of the following: amount of internal network read data per second and amount of internal network write data per second of the data bucket to be displayed within the target time range, etc. The embodiments of the present invention do not limit this. Optionally, the monitoring metric data of the data bucket to be displayed under the status code statistics may include but is not limited to at least one of the following: number of requests with status code 4xx, number of requests with status code 5xx, and number of requests with status code 2xx of the data bucket to be displayed within the target time range, etc.; the embodiments of the present invention do not limit this. Optionally, the monitoring metric data of the data bucket to be displayed under the number of requests with excessive backend processing time may include but is not limited to at least one of the following: read and write requests with processing time exceeding 1 s (second) and read and write requests with processing time exceeding 5 s of the data bucket to be displayed within the target time range, etc. The embodiments of the present invention do not limit this. Optionally, the monitoring metric data of the data bucket to be displayed under the quota increase may include but is not limited to at least one of the following: quota increase (such as quota increase for 7 days or 15 days) of the data bucket to be displayed within the target time range, etc. The embodiments of the present invention do not limit this. Optionally, the target time range can be any time range, and the embodiments of the present invention do not limit this.

[0079] Optionally, the electronic device may also display the usage amount of the recycle bin within each hour, etc. It should be noted that Figures 5 - 8 etc. only exemplarily represent the display content, and the embodiments of the present invention do not limit this; for example, Figure 8 each of the charts in it can be generated from actual data, Figure 8 etc. all exemplarily illustrate that the embodiments of the present invention can respectively represent the changes of each monitoring metric data in the form of charts, etc.; for another example, Figure 8 the monitoring metric data under the number of read and write requests for the data bucket in it may also include standard storage write requests, standard storage read requests, etc., etc.

[0080] Based on this, embodiments of the present invention can display various distribution charts, trend charts, etc. (such as storage capacity distribution charts, usage trend charts, etc.), predict resource growth and resource utilization rates, and count the amount of data read and written per second by data buckets. That is to say, embodiments of the present invention can display the physical and logical resource utilization rates of real-time user-side data buckets, display the usage amount of the recycle bin at the hourly level, view the capacity and capacity utilization percentage at any time point, and collect and display the resource utilization amount and quota increase trend chart of a single data bucket, etc. Furthermore, it can provide a convenient and powerful query function; and can detect capacity risks at the minute level, so as to give early warnings in a timely manner, and improve the stability of the cluster.

[0081] When detecting a data bucket creation operation, embodiments of the present invention can determine a target data heat division model based on the data heat division model indicated by the data bucket creation operation. The target data heat division model is any one of multiple data heat division models; and create a target data bucket based on the target data heat division model. Based on this, when detecting a data migration task for the target data bucket, divide the data in the target data bucket according to the target data heat division model to obtain a target data heat division result, and perform data migration on the data in the target data bucket according to the target data heat division result. Correspondingly, when detecting a timed statistics task, read the file heat rule configuration file, and group at least one data bucket to obtain at least one grouping result; thus, perform grouped statistics on each grouping result in at least one grouping result according to the file heat rule configuration file and at least one statistical indicator to obtain the indicator statistical data of each grouping result under each statistical indicator in at least one statistical indicator. It can be seen that embodiments of the present invention can flexibly manage the cluster capacity, perform real-time capacity statistics for specific resources, perform real-time capacity statistics for specific resources, so as to grasp the online risks in real time, quickly locate specific resources and give early warnings when encountering capacity risks, and effectively improve the stability of the cluster.

[0082] Based on the description of the related embodiments of the above cluster capacity management method, embodiments of the present invention also propose a cluster capacity management device. The cluster capacity management device can be a computer program (including program code) running in an electronic device; as Figure 9 shown, the cluster capacity management device can include a determination unit 901 and a processing unit 902. The cluster capacity management device can execute Figure 1 or Figure 2 shown cluster capacity management method, that is, the cluster capacity management device can run the above units:

[0083] A determination unit 901, configured to, when detecting a data bucket creation operation, determine a target data heat division model based on the data heat division model indicated by the data bucket creation operation, where the target data heat division model is any one of multiple data heat division models;

[0084] A processing unit 902, configured to create a target data bucket based on the target data heat division model;

[0085] The processing unit 902 is further configured to, when detecting a data migration task for the target data bucket, perform data heat division on the data in the target data bucket according to the target data heat division model to obtain a target data heat division result, and perform data migration on the data in the target data bucket according to the target data heat division result.

[0086] In one implementation manner, the determination unit 901 may further be configured to:

[0087] Respond to a detected data bucket heat division model modification operation, and determine a data bucket to be modified and a data division model to be modified indicated by the data bucket heat division model modification operation;

[0088] The processing unit 902 may further be configured to:

[0089] Update the data division model to be modified to the data heat division model of the data bucket to be modified.

[0090] In another implementation manner, when the processing unit 902 performs data heat division on the data in the target data bucket according to the target data heat division model to obtain a target data heat division result, it may specifically be configured to:

[0091] Determine multiple file heat rules, and respectively determine file heat indication information of each data in the target data bucket under each file heat rule in the multiple file heat rules;

[0092] Perform data heat division on the data in the target data bucket based on the file heat indication information of each data under each file heat rule and the target data heat division model to obtain a target data heat division result.

[0093] In another implementation manner, when the processing unit 902 performs data migration on the data in the target data bucket according to the target data heat division result, it may specifically be configured to:

[0094] Obtain disaster recovery data indication information, and respectively traverse each cold data in the target data bucket according to the target data heat division result, and use the currently traversed cold data as the current cold data;

[0095] Based on the disaster recovery data indication information, determine whether the current cold data exists in the disaster recovery data;

[0096] If the current cold data exists in the disaster recovery data, migrate the current cold data to the offline data bucket corresponding to the target data bucket;

[0097] After traversing each piece of cold data in the target data bucket, implement data migration of the cold data in the target data bucket.

[0098] In another implementation, the processing unit 902 can also be used for:

[0099] When detecting a timed statistical task, read the file heat rule configuration file and group at least one data bucket to obtain at least one grouping result;

[0100] According to the file heat rule configuration file and at least one statistical indicator, perform grouped statistics on each grouping result in the at least one grouping result to obtain the indicator statistical data of each grouping result under each statistical indicator in the at least one statistical indicator.

[0101] In another implementation, the determination unit 901 can also be used for:

[0102] In response to detecting a capacity management display operation, determine the storage capacity management type to be displayed indicated by the capacity management display operation;

[0103] Respectively determine the capacity index values of each capacity management indicator in at least one capacity management indicator under the storage capacity management type to be displayed;

[0104] The processing unit 902 can also be used for:

[0105] Display the capacity index values of each capacity management indicator under the storage capacity management type to be displayed.

[0106] In another implementation, the determination unit 901 can also be used for:

[0107] In response to detecting a bucket trend change display operation, determine the bucket screening conditions indicated by the bucket trend change display operation;

[0108] The processing unit 902 can also be used for:

[0109] According to the bucket screening conditions, determine at least one screened data bucket, and display the bucket trend change display indicator data of each screened data bucket in the at least one screened data bucket. The bucket trend change display indicator data of a screened data bucket includes the usage increase rate of the corresponding screened data bucket.

[0110] According to an embodiment of the present invention, Figure 9 Each unit in the shown cluster capacity management device can be respectively or all combined into one or several other units to form, or a certain (some) unit among them can also be further split into multiple smaller units in terms of function to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present invention. The above units are divided based on logical functions. In practical applications, the function of one unit can also be realized by multiple units, or the functions of multiple units can be realized by one unit. In other embodiments of the present invention, any cluster capacity management device can also include other units. In practical applications, these functions can also be assisted by other units and can be realized by the cooperation of multiple units.

[0111] According to another embodiment of the present invention, it can be achieved by running a computer program (including program code) capable of executing the respective steps involved in the corresponding method shown in Figure 1 or Figure 2 on a general-purpose electronic device such as a computer including processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM), to construct the cluster capacity management device shown in Figure 9 and to implement the cluster capacity management method of the embodiments of the present invention. The computer program can be recorded on, for example, a computer storage medium, loaded into the above-mentioned electronic device through the computer storage medium, and run therein.

[0112] In the embodiments of the present invention, when a data bucket creation operation is detected, a target data heat division model can be determined based on the data heat division model indicated by the data bucket creation operation. The target data heat division model is any one of multiple data heat division models; and based on the target data heat division model, a target data bucket is created. Based on this, when a data migration task for the target data bucket is detected, the data in the target data bucket can be divided according to the target data heat division model to obtain a target data heat division result, and the data in the target data bucket can be migrated according to the target data heat division result. It can be seen that the embodiments of the present invention can achieve fine-grained heat management according to bucket granularity by setting different data heat division models for data buckets, so as to perform intelligent stratification of business bucket storage, and can perform intelligent stratification of hot data, cold data, etc. according to the business to achieve fine-grained resource capacity management, that is, flexible cluster capacity management can be performed.

[0113] Based on the descriptions of the above method embodiments and apparatus embodiments, an exemplary embodiment of the present invention further provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, it is configured to cause the electronic device to execute the method according to the embodiments of the present invention.

[0114] An exemplary embodiment of the present invention further provides a non-transitory computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor of a computer, it is configured to cause the computer to execute the method according to the embodiments of the present invention.

[0115] An exemplary embodiment of the present invention further provides a computer program product, including a computer program, wherein when the computer program is executed by a processor of a computer, it is configured to cause the computer to execute the method according to the embodiments of the present invention.

[0116] Referring to Figure 10 , a block diagram of an electronic device 1000 that can be a server or a client of the present invention will now be described. It is an example of a hardware device applicable to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0117] As Figure 10 shown, the electronic device 1000 includes a computing unit 1001, which can execute various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the electronic device 1000 can also be stored. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0118] Multiple components in the electronic device 1000 are connected to the I / O interface 1005, including: an input unit 1006, an output unit 1007, a storage unit 1008, and a communication unit 1009. The input unit 1006 can be any type of device capable of inputting information into the electronic device 1000. The input unit 1006 can receive input digital or character information and generate key signal inputs related to user settings and / or function controls of the electronic device. The output unit 1007 can be any type of device capable of presenting information and can include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 1008 can include, but is not limited to, a magnetic disk and an optical disk. The communication unit 1009 allows the electronic device 1000 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks and can include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth™ device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0119] The computing unit 1001 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 executes the various methods and processes described above. For example, in some embodiments, the cluster capacity management method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 1000 via the ROM 1002 and / or the communication unit 1009. In some embodiments, the computing unit 1001 can be configured to execute the cluster capacity management method by any other suitable means (e.g., by means of firmware).

[0120] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, executed partially on the machine and partially on a remote machine as an independent software package, or executed entirely on a remote machine or server.

[0121] In the context of the present invention, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0122] As used in the present invention, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., a disk, an optical disk, a memory, a programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0123] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0124] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0125] A computer system can include a client and a server. The client and the server are generally far from each other and typically interact through a communication network. The client-server relationship is created by computer programs that run on the respective computers and have a client-server relationship with each other.

[0126] Moreover, it should be understood that the above-disclosed is only a preferred embodiment of the present invention, and of course it cannot be used to limit the scope of the rights of the present invention. Therefore, equivalent changes made according to the claims of the present invention still fall within the scope covered by the present invention.

Claims

1. A cluster capacity management method, characterized in that: include: When a data bucket creation operation is detected, a target data heat partition model is determined based on the data heat partition model indicated by the data bucket creation operation, wherein the target data heat partition model is any one of multiple data heat partition models; Based on the target data heat partition model, create a target data bucket; When a data migration task for the target data bucket is detected, data heat partitioning is performed on the data in the target data bucket according to the target data heat partitioning model to obtain a target data heat partitioning result, and data migration is performed on the data in the target data bucket according to the target data heat partitioning result.

2. The method according to claim 1, characterized in that The method further comprises: In response to the detected data bucket heat partition model modification operation, determine the data bucket to be modified and the data partition model to be modified indicated by the data bucket heat partition model modification operation; The data partition model to be modified is updated to the data heat partition model of the data bucket to be modified.

3. The method according to claim 1 or 2, characterized in that: The step of performing data heat partitioning on the data in the target data bucket according to the target data heat partitioning model to obtain a target data heat partitioning result includes: Determine a plurality of file heat rules, and respectively determine file heat indication information of each data in the target data bucket under each file heat rule in the plurality of file heat rules; Based on the file heat indication information of each data under each file heat rule and the target data heat partition model, data heat partition is performed on the data in the target data bucket to obtain a target data heat partition result.

4. The method according to claim 1 or 2, characterized in that: The migrating the data in the target data bucket according to the target data heat classification result includes: Obtain disaster recovery data indication information, and traverse each cold data in the target data bucket according to the target data heat division result, and use the currently traversed cold data as the current cold data; Based on the disaster recovery data indication information, determining whether the current cold data exists in the disaster recovery data; If the current cold data exists in the disaster recovery data, the current cold data is migrated to the offline data bucket corresponding to the target data bucket; After traversing all the cold data in the target data bucket, data migration of the cold data in the target data bucket is implemented.

5. The method according to claim 1 or 2, characterized in that: The method further comprises: When a scheduled statistics task is detected, the file heat rule configuration file is read, and at least one data bucket is grouped to obtain at least one grouping result; According to the file heat rule configuration file and at least one statistical indicator, group statistics are performed on each group result in the at least one group result to obtain indicator statistical data of each group result under each statistical indicator in the at least one statistical indicator.

6. The method according to claim 1 or 2, characterized in that: The method further comprises: responding to the detected capacity management display operation and determining a storage capacity management type to be displayed indicated by the capacity management display operation; The capacity indicator value of each capacity management indicator of at least one capacity management indicator under the storage capacity management type to be displayed is determined respectively, and the capacity indicator value of each capacity management indicator under the storage capacity management type to be displayed is displayed.

7. The method according to claim 1 or 2, characterized in that: The method further comprises: responding to the detected bucket trend change display operation, and determining a bucket screening condition indicated by the bucket trend change display operation; At least one filtered data bucket is determined according to the bucket filtering condition, and bucket trend change display indicator data of each filtered data bucket in the at least one filtered data bucket is displayed, and the bucket trend change display indicator data of a filtered data bucket includes the usage increase of the corresponding filtered data bucket.

8. A cluster capacity management device, characterized in that: The device comprises: A determination unit, configured to determine a target data heat partition model based on the data heat partition model indicated by the data bucket creation operation when a data bucket creation operation is detected, wherein the target data heat partition model is any one of the multiple data heat partition models; A processing unit, configured to create a target data bucket based on the target data heat partitioning model; The processing unit is also used to, when a data migration task for the target data bucket is detected, perform data heat partitioning on the data in the target data bucket according to the target data heat partitioning model, obtain the target data heat partitioning result, and perform data migration on the data in the target data bucket according to the target data heat partitioning result.

9. An electronic device, characterized in that: include: processor; as well as Memory for storing programs, The program includes instructions, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to make a computer execute the method according to any one of claims 1-7.

Images