Multi-cloud storage cluster management method and device, equipment and storage medium

Abstract

The invention discloses a multi-cloud storage cluster management method, device and equipment and a storage medium, is applied to a preset management system, and relates to the technical field of creativity creativity, and the method comprises the steps: monitoring the business load condition, business data and database copy operation condition of each storage cluster in a multi-cloud storage cluster on a creativity creativity server; the current load information is obtained; performing index analysis based on the load information by using a current database copy storage resource state and a database copy storage resource use condition to obtain a corresponding index analysis result; and determining a corresponding target resource scheduling strategy based on a preset scheduling decision model, a preset intelligent learning mechanism, the load information and the index analysis result, and triggering a resource allocation operation corresponding to the multi-cloud storage cluster by using a preset interface and the target resource scheduling strategy. Intelligent dynamic allocation and efficient management of storage resources can be realized.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a multi-cloud storage cluster management method, device, equipment and storage medium. Background Art

[0002] In the era of cloud computing and big data, enterprises have a growing demand for data storage and management. However, traditional multi-cloud storage management methods are often based on static preset rules, which cannot be dynamically adjusted according to real-time changes in business loads, are not flexible enough, and will cause resource waste; and scheduling often relies on manual intervention or simple rule matching, which is inefficient and prone to inaccurate resource scheduling due to human errors or imperfect rules. Summary of the invention

[0003] In view of this, the purpose of the present invention is to provide a multi-cloud storage cluster management method, device, equipment and storage medium, which can realize intelligent dynamic allocation and efficient management of storage resources, improve the utilization and adaptability of storage resources, and thus reduce operation and maintenance costs. The specific scheme is as follows:

[0004] In a first aspect, the present application provides a multi-cloud storage cluster management method, which is applied to a preset management system, including:

[0005] Monitor the business load, business data, and database replica operation status of each storage cluster in the multi-cloud storage cluster on the trusted server to obtain the current load information;

[0006] Based on the load information and using the current database replica storage resource state and database replica storage resource usage, an indicator analysis is performed to obtain a corresponding indicator analysis result;

[0007] Based on a preset scheduling decision model, a preset intelligent learning mechanism, the load information and the indicator analysis result, a corresponding target resource scheduling strategy is determined, and a resource allocation operation corresponding to the multi-cloud storage cluster is triggered by using a preset interface and the target resource scheduling strategy.

[0008] Optionally, the business load, business data, and database copy operation status of each storage cluster in the multi-cloud storage cluster on the trusted server are monitored to obtain current load information, including:

[0009] Obtain the business load information and corresponding business data of each storage cluster in the multi-cloud storage cluster on the current Xinchuang server;

[0010] Classifying the acquired current business data, and filtering the classified business data to determine corresponding filtered data;

[0011] Counting the read and write frequencies and average read and write delays when the business application accesses each database copy in the multi-cloud storage cluster in a running state, so as to obtain corresponding database copy operation information;

[0012] Current load information is determined based on the business load information, the filtered data, and the database copy operation information.

[0013] Optionally, the performing an indicator analysis based on the load information and using the current database replica storage resource state and database replica storage resource usage to obtain a corresponding indicator analysis result includes:

[0014] For the target indicators of the database copy storage resources, analysis is performed based on the load information, the current database copy storage resource status and the database copy storage resource usage to obtain corresponding indicator analysis results; wherein the target indicators include access idle rate, usage rate, type and capacity.

[0015] Optionally, the determining a corresponding target resource scheduling strategy based on a preset scheduling decision model, a preset intelligent learning mechanism, the load information, and the indicator analysis result includes:

[0016] If the database replica in operation in the current multi-cloud storage cluster does not meet the target access latency of the business application, the preset reinforcement learning model in the preset intelligent learning mechanism is called;

[0017] Based on the preset reinforcement learning model, the load information and the indicator analysis result, a target resource scheduling strategy corresponding to the database copy in the running state is determined; wherein the target resource scheduling strategy includes the corresponding number of database copies to be increased / the number of database copies to be reduced, and the number of database copies to be dynamically migrated.

[0018] Optionally, the determining, based on the preset reinforcement learning model, the load information, and the indicator analysis result, a target resource scheduling strategy corresponding to the database replica in the running state includes:

[0019] Based on the load information and the indicator analysis results, the resource utilization rate in each storage cluster, the read and write pressure of each database copy, and the read latency and write latency of the database copy closest to the business application access distance are input into the Actor current network in the preset reinforcement learning model to obtain the target resource scheduling strategy output by the Actor current network after adding random noise.

[0020] Optionally, after determining the target resource scheduling strategy corresponding to the database replica in the running state based on the preset reinforcement learning model, the load information, and the indicator analysis result, the method further includes:

[0021] Acquire the experience value obtained after executing the target resource scheduling strategy through the preset reinforcement learning model, and save the state transition information quadruple determined by using the experience value into the experience replay pool;

[0022] A preset number of samples are randomly sampled from the experience replay pool, and the parameters of the Critic current network and the Actor current network in the preset reinforcement learning model are updated by gradient descent to obtain the updated preset reinforcement learning model.

[0023] Optionally, also include:

[0024] Run the received multi-cloud storage resource definition file, and after listening to the multi-cloud storage resource creation event of the Kubernetes API Server, enter the database replica deployment phase to complete the corresponding multi-cloud storage resource creation event based on the number of database replicas, database type, multiple target storage clusters used for deployment, and the enabling state of the adaptive scheduling function in the multi-cloud storage resource definition file;

[0025] In the process of determining the resource scheduling strategy, when the adaptive scheduling function in the multi-cloud storage resource definition file is enabled, the first adaptive scheduling operation of the corresponding database copy is triggered for the deployed and non-running database copy;

[0026] When the adaptive scheduling function is turned off, for the deployed database replica that is not in operation, a corresponding fixed database resource is created in the corresponding target storage cluster according to the multi-cloud storage resource definition file to complete the first fixed cluster scheduling operation of the corresponding database replica.

[0027] In a second aspect, the present application provides a multi-cloud storage cluster management device, which is applied to a preset management system, including:

[0028] The cluster monitoring module is used to monitor the business load, business data, and database copy operation of each storage cluster in the multi-cloud storage cluster on the Xinchuang server to obtain the current load information;

[0029] An indicator analysis module, used to perform indicator analysis based on the load information and using the current database copy storage resource state and database copy storage resource usage to obtain corresponding indicator analysis results;

[0030] The resource scheduling module is used to determine the corresponding target resource scheduling strategy based on the preset scheduling decision model, the preset intelligent learning mechanism, the load information and the indicator analysis result, and use the preset interface and the target resource scheduling strategy to trigger the resource allocation operation corresponding to the multi-cloud storage cluster.

[0031] In a third aspect, the present application provides an electronic device, including:

[0032] Memory, used to store computer programs;

[0033] The processor is used to execute the computer program to implement the steps of the aforementioned multi-cloud storage cluster management method.

[0034] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program implements the steps of the aforementioned multi-cloud storage cluster management method when executed by a processor.

[0035] It can be seen that in this application, the preset management system monitors the business load, business data, and database copy operation of each storage cluster in the multi-cloud storage cluster on the xinchuang server to obtain the current load information; based on the load information and using the current database copy storage resource status and database copy storage resource usage to perform index analysis to obtain the corresponding index analysis results; based on the preset scheduling decision model, the preset intelligent learning mechanism, the load information and the index analysis results, the corresponding target resource scheduling strategy is determined, and the preset interface and the target resource scheduling strategy are used to trigger the resource allocation operation corresponding to the multi-cloud storage cluster. That is, this application monitors each storage cluster in the multi-cloud storage cluster on the xinchuang server in real time, and uses the load information obtained by monitoring and the current status and usage of the database copy for analysis, and then uses the index analysis results and load information to determine the target resource scheduling strategy, and combines the preset interface for resource allocation. In this way, it is possible to achieve intelligent dynamic allocation and efficient management of storage resources, and improve the utilization and adaptability of storage resources, thereby reducing operation and maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0037] Figure 1 A flow chart of a multi-cloud storage cluster management method provided in this application;

[0038] Figure 2 A specific reinforcement learning process diagram provided for this application;

[0039] Figure 3 A schematic diagram of a multi-cloud storage resource deployment process for a preset management system provided in this application;

[0040] Figure 4 A schematic diagram of the structure of a multi-cloud storage cluster management device provided in this application;

[0041] Figure 5 A structural diagram of an electronic device provided for this application. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] Traditional multi-cloud storage management methods are often based on static preset rules, which cannot be adjusted dynamically according to real-time changes in business loads, are not flexible enough, and will cause waste of resources; and scheduling often relies on manual intervention or simple rule matching, which is inefficient and prone to inaccurate resource scheduling due to human errors or imperfect rules. To this end, this application provides a multi-cloud storage cluster management solution that can realize intelligent dynamic allocation and efficient management of storage resources, improve the utilization and adaptability of storage resources, and thus reduce operation and maintenance costs.

[0044] See also Figure 1 As shown, an embodiment of the present invention discloses a multi-cloud storage cluster management method, which is applied to a preset management system, including:

[0045] Step S11: Monitor the business load, business data, and database copy operation status of each storage cluster in the multi-cloud storage cluster on the trusted computing server to obtain the current load information.

[0046] Specifically, in this embodiment, various types of business data will be collected and analyzed for each storage cluster in the multi-cloud storage cluster on the trusted server, and the operation of the user business load and the database copy will be monitored in real time. That is, the business load, business data, and database copy operation of each storage cluster in the multi-cloud storage cluster on the trusted server are monitored to obtain the current load information, including: obtaining the business load information and corresponding business data of each storage cluster in the multi-cloud storage cluster on the current trusted server; classifying the acquired current business data, and filtering the classified business data to determine the corresponding filtered data; counting the read and write frequency and average read and write delay when the business application accesses each database copy in the multi-cloud storage cluster in operation to obtain the corresponding database copy operation information; determining the current load information based on the business load information, the filtered data, and the database copy operation information.

[0047] Step S12: Based on the load information and using the current database copy storage resource status and database copy storage resource usage, an indicator analysis is performed to obtain a corresponding indicator analysis result.

[0048] In this embodiment, the indicator analysis is performed based on the load information and using the current database replica storage resource state and the database replica storage resource usage to obtain the corresponding indicator analysis results, including: for the target indicator of the database replica storage resource, the analysis is performed based on the load information, the current database replica storage resource state and the database replica storage resource usage to obtain the corresponding indicator analysis results; wherein the target indicator includes access idle rate, usage rate, type, and capacity. That is, after obtaining the load information, the key indicators such as the access idle rate and usage rate of the database replica storage resource will be analyzed in combination with the current state and usage of the database replica storage resource.

[0049] Step S13: determining a corresponding target resource scheduling strategy based on a preset scheduling decision model, a preset intelligent learning mechanism, the load information, and the indicator analysis result, and triggering a resource allocation operation corresponding to the multi-cloud storage cluster using a preset interface and the target resource scheduling strategy.

[0050] It should be understood that in this embodiment, when the running database copy does not meet the expected access delay of the business application, a multi-cloud storage resource rescheduling decision will be made for the running database copy based on the data obtained in the above steps by calling the preset reinforcement learning model in the preset intelligent learning mechanism. That is, the corresponding target resource scheduling strategy is determined based on the preset scheduling decision model, the preset intelligent learning mechanism, the load information and the index analysis result, including: if the database copy in the current multi-cloud storage cluster in operation does not meet the target access delay of the business application, the preset reinforcement learning model in the preset intelligent learning mechanism is called; based on the preset reinforcement learning model, the load information and the index analysis result, the target resource scheduling strategy corresponding to the database copy in operation is determined; wherein the target resource scheduling strategy includes the corresponding number of database copies to be increased / the number of database copies to be reduced, and the number of database copies to be dynamically migrated. It can be understood that if the database copy to be dynamically migrated exists in the strategy, the database copy in the cluster with lower access frequency will be migrated to the cluster closer to the business application to achieve dynamic migration.

[0051] Furthermore, in this embodiment, the target resource scheduling strategy corresponding to the database copy in the running state is determined based on the preset reinforcement learning model, the load information and the indicator analysis result, including: based on the load information and the indicator analysis result, the resource utilization rate in each storage cluster, the read and write pressure of each database copy, and the read latency and write latency of the database copy with the closest access distance for business applications are input into the Actor current network in the preset reinforcement learning model to obtain the target resource scheduling strategy output by the Actor current network after adding random noise.

[0052] It should be understood that after receiving the strategy fed back by the preset intelligent learning mechanism, the strategy will be executed. If the database access read latency and write latency of the business application meet expectations after execution, the rescheduling process ends and a positive reward value is sent to the preset reinforcement learning model in the preset intelligent learning mechanism. The preset reinforcement learning model will save the reward value R(t) to the current scheduling state, and send the monitoring values ​​of the business application status, cluster load status and database replica load pressure to the preset intelligent learning mechanism. If the expected latency is not met, first send a negative reward value to the preset reinforcement learning model in the preset intelligent learning mechanism. It is also necessary to send the monitoring values ​​of the business application status, cluster load status and database replica load pressure to the preset intelligent learning mechanism, and then re-read the load information and the indicator analysis results, and continue to call the preset reinforcement learning model in the preset intelligent learning mechanism to re-execute the decision result until the database access read latency and write latency of the business application meet expectations.

[0053] It can be understood that in this embodiment, the model in the preset intelligent learning mechanism can continuously optimize the parameters and algorithms of the model according to the business application status, cluster load conditions and database replica load pressure to improve the accuracy and efficiency of database resource scheduling. That is, after determining the target resource scheduling strategy corresponding to the database replica in operation based on the preset reinforcement learning model, the load information and the indicator analysis results, it also includes: obtaining the experience value obtained after executing the target resource scheduling strategy through the preset reinforcement learning model, and saving the state transition information quadruple determined by the experience value to the experience replay pool; randomly sampling a preset number of samples from the experience replay pool, and using the gradient descent method to update the parameters of the Critic current network and the Actor current network in the preset reinforcement learning model to obtain the updated preset reinforcement learning model.

[0054] Regarding the preset intelligent learning mechanism, this mechanism integrates the database replica adaptive scheduling Data DDPG algorithm based on DDPG (Deep Deterministic Policy Gradient, an algorithm based on deep reinforcement learning). The execution process of the algorithm is shown in Figure 2 The algorithm inputs the resource utilization of each cluster, the read pressure of each database replica, the write pressure of each database replica, the read latency of the database replica that the business application accesses to the nearest, and the write latency of the database replica that the business application accesses to the nearest to it into the Actor's current network, and outputs a rescheduling decision. The rescheduling decision includes adding a database replica, reducing a database replica, and dynamically migrating a database replica. After the Data DDPG algorithm outputs the decision result, it feeds it back to the decision scheduling module and waits for the decision scheduling module to return the reward value R(t). After obtaining the reward value R(t), the Data DDPG algorithm calls and saves the state transition information quadruple ((F_S(t), A(t), R(t), F_S(t+1)) into the experience replay pool, where A(t) is the decision result output by the Actor's current network at time t, R(t) is the decision reward value at time t, and F_S(t) and F_S(t+1) are the model input vectors at time t and t+1 respectively. Actor Optimizer is the optimizer of the Actor's current network. Critic Optimizer is the optimizer of the Critic's current network. After each run of the Data DDPG algorithm, a certain number of samples need to be randomly sampled from the experience replay pool, and the parameters of the Critic's current network and the Actor's current network need to be updated by gradient descent.

[0055] In addition, in order to increase random exploration, the Data DDPG algorithm will adopt the e-greedy strategy when running. It adds random noise to the value of the Actor's current network output with a certain probability as the final decision result. The present invention uses two commonly used Gaussian noises and Ornstein-Uhlenbeck noises as random noise options, and compares their effects respectively.

[0056] In this embodiment, after obtaining the target resource scheduling strategy, the storage resources in the multi-cloud storage cluster are dynamically reallocated by calling the API (Application Programming Interface) provided by Kubernetes (cluster management platform). The allocation process can be adding new database copies, dynamically migrating database copies, reducing database copies to release idle resources, etc.

[0057] At the same time, in this embodiment, the creation of new resources is described. The multi-cloud storage resource definition file created by the user is received, the file is applied using the kubectl command, and after monitoring the multi-cloud storage resource creation event of the Kubernetes API Server, the database copy deployment phase is entered to complete the corresponding multi-cloud storage resource creation event based on the scheduling status and data synchronization status monitoring function enablement status in the multi-cloud storage resource definition file, the number of database copies, the database type (such as Mysql, Postgresql, Elasticsearch, etc.), the multiple target storage clusters used for deployment, and the adaptive scheduling function enablement status; in the process of determining the resource scheduling strategy, when the adaptive scheduling function enablement status in the multi-cloud storage resource definition file is enabled, the first adaptive scheduling operation of the corresponding database copy is triggered for the deployed and non-running database copy; when the adaptive scheduling function enablement status is disabled, the corresponding fixed database resource is created in the corresponding target storage cluster according to the multi-cloud storage resource definition file for the deployed and non-running database copy, so as to complete the first fixed cluster scheduling operation of the corresponding database copy. That is, when the multi-cloud storage resource created for the first time is scheduled. If multi-cloud storage resources require adaptive scheduling, the scheduling decision module will count the resource idle rates of each cluster used for scheduling each time a database copy is scheduled, and then bind and schedule the database copy to the cluster with the optimal resource idle rate; if multi-cloud storage resources require fixed cluster scheduling, a fixed copy database resource will be created in the corresponding cluster according to the number of database copies defined for each cluster and the definition file.

[0058] It can be seen that in the embodiment of the present application, the preset management system monitors the business load, business data, and database copy operation of each storage cluster in the multi-cloud storage cluster on the xinchuang server to obtain the current load information; based on the load information and using the current database copy storage resource status and database copy storage resource usage to perform index analysis to obtain the corresponding index analysis results; based on the preset scheduling decision model, the preset intelligent learning mechanism, the load information and the index analysis results, the corresponding target resource scheduling strategy is determined, and the preset interface and the target resource scheduling strategy are used to trigger the resource allocation operation corresponding to the multi-cloud storage cluster. That is, the present application monitors each storage cluster in the multi-cloud storage cluster on the xinchuang server in real time, and uses the load information obtained by monitoring and the current status and usage of the database copy for analysis, and then uses the index analysis results and load information to determine the target resource scheduling strategy, and combines the preset interface for resource allocation. In this way, it is possible to achieve intelligent dynamic allocation and efficient management of storage resources, and improve the utilization and adaptability of storage resources, thereby reducing operation and maintenance costs.

[0059] Combine the following Figure 3 The schematic diagram disclosed in the figure specifically illustrates the technical solution of the embodiment of the present application.

[0060] In this embodiment, the preset management system includes a CRD (Cluster Resource Description, i.e., custom resource definition, which is a powerful interface extension mechanism provided by Kubernetes (container management platform)) resource control module, a business monitoring module, a resource analysis module, a scheduling decision module, a resource allocation module, and an intelligent learning module.

[0061] Specifically, regarding the CRD resource control module, this module receives the multi-cloud storage resource definition file created by the user to create multi-cloud storage resources, and based on whether adaptive scheduling is required, sends a scheduling notification to the scheduling decision module to implement the scheduling of the multi-cloud storage resources created for the first time. Regarding the business monitoring module, this module monitors the business load in the cluster in real time, including collecting and analyzing various business data and the operation of the running database copies to generate load data. The load data is then sent to the resource analysis module.

[0062] At the same time, regarding the resource analysis module, the module analyzes key indicators such as access idle rate and usage rate of database copy storage resources based on the load data, and sends these indicators together with the load data to the scheduling decision module.

[0063] Furthermore, regarding the scheduling decision module, this module schedules the multi-cloud storage resources created for the first time. A scheduling strategy is also formulated based on the indicators sent by the resource analysis module and the load data. And when necessary, for example, the running database copy does not meet the expected access delay of the business application, the preset reinforcement learning model in the intelligent learning module is called to determine the target resource scheduling strategy. Regarding the resource allocation module, this module dynamically reallocates storage resources in the multi-cloud storage cluster according to the target resource scheduling strategy sent by the scheduling decision module and using the API interface provided by Kubernetes. In addition, the intelligent learning module can continuously optimize its own mechanism according to the business application status, cluster load conditions and database copy load pressure.

[0064] It should be understood that in this embodiment, the preset management system also includes a user interaction interface for displaying information such as business load, storage resource allocation, and scheduling decision results, and provides a user operation interface to support user-defined settings and adjustments. In addition, the preset management system also supports integration with other enterprise systems to achieve data sharing and exchange, and improve the efficiency and effectiveness of overall resource management.

[0065] See also Figure 4 As shown, the embodiment of the present application also discloses a multi-cloud storage cluster management device, which is applied to a preset management system, including:

[0066] The cluster monitoring module 11 is used to monitor the business load, business data, and database copy operation status of each storage cluster in the multi-cloud storage cluster on the Xinchuang server to obtain the current load information;

[0067] The indicator analysis module 12 is used to perform indicator analysis based on the load information and using the current database copy storage resource state and database copy storage resource usage to obtain corresponding indicator analysis results;

[0068] The resource scheduling module 13 is used to determine the corresponding target resource scheduling strategy based on the preset scheduling decision model, the preset intelligent learning mechanism, the load information and the indicator analysis result, and use the preset interface and the target resource scheduling strategy to trigger the resource allocation operation corresponding to the multi-cloud storage cluster.

[0069] Among them, for more specific working processes of the above-mentioned modules, please refer to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0070] It can be seen that this application monitors each storage cluster in the multi-cloud storage cluster on the Xinchuang server in real time, and uses the load information obtained from the monitoring and the current status and usage of the database copy for analysis, and then uses the indicator analysis results and load information to determine the target resource scheduling strategy, and combines the preset interface to allocate resources. In this way, it is possible to achieve intelligent dynamic allocation and efficient management of storage resources, improve the utilization and adaptability of storage resources, and thus reduce operation and maintenance costs.

[0071] In some specific embodiments, the cluster monitoring module 11 may specifically include:

[0072] A data acquisition unit is used to obtain the business load information and corresponding business data of each storage cluster in the multi-cloud storage cluster on the current Xinchuang server;

[0073] A classification and filtering unit, used to classify the acquired current business data and filter the classified business data to determine corresponding filtered data;

[0074] A replica statistics unit, used to count the read and write frequencies and average read and write delays when the business application accesses each database replica in the multi-cloud storage cluster in a running state, so as to obtain corresponding database replica operation information;

[0075] An information determination unit is used to determine current load information based on the business load information, the filtered data and the database copy operation information.

[0076] In some specific embodiments, the indicator analysis module 12 may specifically include:

[0077] The indicator analysis unit is used to analyze the target indicators of the database copy storage resources based on the load information, the current database copy storage resource status and the database copy storage resource usage to obtain corresponding indicator analysis results; wherein the target indicators include access idle rate, usage rate, type and capacity.

[0078] In some specific embodiments, the resource scheduling module 13 may specifically include:

[0079] A model calling unit, configured to call a preset reinforcement learning model in a preset intelligent learning mechanism if the database replica in operation in the current multi-cloud storage cluster does not meet the target access latency of the business application;

[0080] An analysis result processing unit is used to determine a target resource scheduling strategy corresponding to the database copy in a running state based on the preset reinforcement learning model, the load information and the indicator analysis result; wherein the target resource scheduling strategy includes the corresponding number of database copies to be increased / the number of database copies to be reduced, and the number of database copies to be dynamically migrated.

[0081] In some specific embodiments, the analysis result processing unit can be specifically used to: input the resource utilization rate in each storage cluster, the read and write pressure of each database copy, and the read latency and write latency of the database copy closest to the business application access distance into the Actor current network in the preset reinforcement learning model based on the load information and the indicator analysis results, so as to obtain the target resource scheduling strategy output by the Actor current network after adding random noise.

[0082] In some specific embodiments, the multi-cloud storage cluster management device can also be used to: obtain the experience value obtained after executing the target resource scheduling strategy through the preset reinforcement learning model, and save the state transition information quadruple determined by the experience value into the experience replay pool; randomly sample a preset number of samples from the experience replay pool, and use gradient descent to update the parameters of the Critic current network and the parameters of the Actor current network in the preset reinforcement learning model to obtain the updated preset reinforcement learning model.

[0083] In some specific embodiments, the multi-cloud storage cluster management device may further include:

[0084] A resource creation unit is used to run the received multi-cloud storage resource definition file, and after listening to the multi-cloud storage resource creation event of the Kubernetes API Server, enter the database copy deployment phase to complete the corresponding multi-cloud storage resource creation event based on the number of database copies, database type, multiple target storage clusters used for deployment, and the enabling state of the adaptive scheduling function in the multi-cloud storage resource definition file;

[0085] A first resource scheduling unit is configured to trigger a first adaptive scheduling operation of a corresponding database copy for a deployed and non-running database copy when the adaptive scheduling function in the multi-cloud storage resource definition file is enabled in the process of determining the resource scheduling strategy;

[0086] The second resource scheduling unit is used to create a corresponding fixed database resource in the corresponding target storage cluster according to the multi-cloud storage resource definition file for the deployed and non-running database copy when the adaptive scheduling function is turned off, so as to complete the first fixed cluster scheduling operation of the corresponding database copy.

[0087] Furthermore, the present application also discloses an electronic device. Figure 5 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0088] Figure 5 A schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the multi-cloud storage cluster management method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0089] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0090] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0091] The operating system 221 is used to manage and control the hardware devices on the electronic device 20 and the computer program 222, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the multi-cloud storage cluster management method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0092] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the multi-cloud storage cluster management method disclosed above is implemented. For the specific steps of the method, reference may be made to the corresponding contents disclosed in the above embodiments, and no further description will be given here.

[0093] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0094] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0095] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0096] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0097] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A multi-cloud storage cluster management method, characterized in that: Applied to preset management systems, including: Monitor the business load, business data, and database replica operation status of each storage cluster in the multi-cloud storage cluster on the trusted server to obtain the current load information; Based on the load information and using the current database replica storage resource state and database replica storage resource usage, an indicator analysis is performed to obtain a corresponding indicator analysis result; Based on a preset scheduling decision model, a preset intelligent learning mechanism, the load information and the indicator analysis result, a corresponding target resource scheduling strategy is determined, and a resource allocation operation corresponding to the multi-cloud storage cluster is triggered by using a preset interface and the target resource scheduling strategy.

2. The multi-cloud storage cluster management method according to claim 1, characterized in that: The business load, business data, and database copy operation status of each storage cluster in the multi-cloud storage cluster on the Xinchuang server are monitored to obtain current load information, including: Obtain the business load information and corresponding business data of each storage cluster in the multi-cloud storage cluster on the current Xinchuang server; Classifying the acquired current business data, and filtering the classified business data to determine corresponding filtered data; Counting the read and write frequencies and average read and write delays when the business application accesses each database copy in the multi-cloud storage cluster in a running state, so as to obtain corresponding database copy operation information; Current load information is determined based on the business load information, the filtered data, and the database copy operation information.

3. The multi-cloud storage cluster management method according to claim 1, characterized in that: The index analysis is performed based on the load information and using the current database copy storage resource state and database copy storage resource usage to obtain corresponding index analysis results, including: For the target indicators of the database copy storage resources, analysis is performed based on the load information, the current database copy storage resource status and the database copy storage resource usage to obtain corresponding indicator analysis results; wherein the target indicators include access idle rate, usage rate, type and capacity.

4. The multi-cloud storage cluster management method according to claim 1, characterized in that: The determining of the corresponding target resource scheduling strategy based on the preset scheduling decision model, the preset intelligent learning mechanism, the load information and the indicator analysis result includes: If the database replica in operation in the current multi-cloud storage cluster does not meet the target access latency of the business application, the preset reinforcement learning model in the preset intelligent learning mechanism is called; Based on the preset reinforcement learning model, the load information and the indicator analysis result, a target resource scheduling strategy corresponding to the database copy in the running state is determined; wherein the target resource scheduling strategy includes the corresponding number of database copies to be increased / the number of database copies to be reduced, and the number of database copies to be dynamically migrated.

5. The multi-cloud storage cluster management method according to claim 4, characterized in that: The determining, based on the preset reinforcement learning model, the load information, and the indicator analysis result, a target resource scheduling strategy corresponding to the database copy in the running state includes: Based on the load information and the indicator analysis results, the resource utilization rate in each storage cluster, the read and write pressure of each database copy, and the read latency and write latency of the database copy closest to the business application access distance are input into the Actor current network in the preset reinforcement learning model to obtain the target resource scheduling strategy output by the Actor current network after adding random noise.

6. The multi-cloud storage cluster management method according to claim 5, characterized in that: After determining the target resource scheduling strategy corresponding to the database copy in the running state based on the preset reinforcement learning model, the load information and the indicator analysis result, the method further includes: Acquire the experience value obtained after executing the target resource scheduling strategy through the preset reinforcement learning model, and save the state transition information quadruple determined by using the experience value into the experience replay pool; A preset number of samples are randomly sampled from the experience replay pool, and the parameters of the Critic current network and the Actor current network in the preset reinforcement learning model are updated by gradient descent to obtain the updated preset reinforcement learning model.

7. The multi-cloud storage cluster management method according to any one of claims 1 to 6, characterized in that: Also includes: Run the received multi-cloud storage resource definition file, and after listening to the multi-cloud storage resource creation event of the Kubernetes API Server, enter the database replica deployment phase to complete the corresponding multi-cloud storage resource creation event based on the number of database replicas, database type, multiple target storage clusters used for deployment, and the enabling state of the adaptive scheduling function in the multi-cloud storage resource definition file; In the process of determining the resource scheduling strategy, when the adaptive scheduling function in the multi-cloud storage resource definition file is enabled, the first adaptive scheduling operation of the corresponding database copy is triggered for the deployed and non-running database copy; When the adaptive scheduling function is turned off, for the deployed database replica that is not in operation, a corresponding fixed database resource is created in the corresponding target storage cluster according to the multi-cloud storage resource definition file to complete the first fixed cluster scheduling operation of the corresponding database replica.

8. A multi-cloud storage cluster management device, characterized in that: Applied to preset management systems, including: The cluster monitoring module is used to monitor the business load, business data, and database copy operation of each storage cluster in the multi-cloud storage cluster on the Xinchuang server to obtain the current load information; An indicator analysis module, used to perform indicator analysis based on the load information and using the current database copy storage resource state and database copy storage resource usage to obtain corresponding indicator analysis results; The resource scheduling module is used to determine the corresponding target resource scheduling strategy based on the preset scheduling decision model, the preset intelligent learning mechanism, the load information and the indicator analysis result, and use the preset interface and the target resource scheduling strategy to trigger the resource allocation operation corresponding to the multi-cloud storage cluster.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the multi-cloud storage cluster management method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, which, when executed by a processor, implements the multi-cloud storage cluster management method according to any one of claims 1 to 7.

Images