Cloud platform resource scheduling method and device, electronic equipment and storage medium

Abstract

The invention provides a cloud platform resource scheduling method and device, electronic equipment and a storage medium, and the method comprises the steps: constructing a cloud adapter composed of a plurality of resource agents, and collecting the resource information of a plurality of cloud platforms; through a cloud adapter, according to a preset resource description standard, performing standardized conversion on resource information belonging to different cloud platforms to form a unified resource data model; analyzing an API interface protocol of each cloud platform, and mapping a standardized resource request of a user into a native API call of the corresponding cloud platform; and matching a target resource meeting the standardized resource request from the resource data model, constructing a resource allocation scheme according to the network overhead corresponding to the cloud platform, and sending a resource scheduling instruction to a corresponding target cloud platform through a resource agent according to the resource allocation scheme to call the target resource. Standardized description, intelligent matching, optimal scheduling and cross-cloud collaborative execution of heterogeneous cloud resources can be realized, and efficient management and intelligent scheduling of the cloud resources are realized.

Description

Technical Field

[0001] The present disclosure relates to the field of cloud service technology, and in particular to a cloud platform resource scheduling method, device, electronic device and storage medium. Background Art

[0002] With the development of cloud computing technology, more and more enterprises and organizations are using cloud computing platforms to provide computing, storage, network and other infrastructure services. However, due to the large differences in resource management, scheduling mechanisms, interface protocols, etc. among different cloud platforms (such as AWS, Azure, Google Cloud, Alibaba Cloud, etc.), cross-cloud resource scheduling and unified management have become extremely challenging.

[0003] At present, there are large differences in resource naming, specifications, storage methods, network topology, etc. among different cloud platforms, which makes it difficult to standardize resource descriptions and makes resource migration and scheduling between different platforms complex. The APIs and management interfaces provided by cloud service providers vary, and there are differences in calling methods, authentication mechanisms, and data formats, making it difficult to uniformly call and manage cross-cloud resources. Existing cross-cloud task scheduling strategies are usually based on static rules or simple load balancing algorithms, which fail to fully consider factors such as network latency, computing load, and data transmission costs, resulting in low resource utilization and low scheduling efficiency. Summary of the invention

[0004] The embodiments of the present disclosure at least provide a cloud platform resource scheduling method, device, electronic device and storage medium, which can realize standardized description, intelligent matching, optimized scheduling and cross-cloud collaborative execution of heterogeneous cloud resources, and realize efficient management and intelligent scheduling of cloud resources.

[0005] The present disclosure provides a cloud platform resource scheduling method, including: Constructing a cloud adapter consisting of multiple resource agents and collecting resource information of multiple cloud platforms, wherein each resource agent has at least one resource processing function; By means of the cloud adapter, the resource information belonging to different cloud platforms is standardized and converted according to a preset resource description standard to form a unified resource data model; Parse the API interface protocol of each cloud platform and map the user's standardized resource request to a native API call corresponding to the cloud platform; Match target resources that meet the standardized resource request from the resource data model, build a resource allocation plan based on the network overhead corresponding to the cloud platform, and send resource scheduling instructions to the corresponding target cloud platform through the resource agent according to the resource allocation plan to retrieve the target resources.

[0006] In an optional implementation, the resource agent at least includes a resource awareness agent, a resource description agent, a resource integration agent, a resource management agent, a resource control agent, and a communication agent; The resource perception agent is used to collect the operating environment and operating process information corresponding to the resource information; the resource description agent is used to provide characteristic attribute information corresponding to the resource information to enhance the encapsulation capability of the resource information; the resource integration agent is used to manage the resource information library corresponding to the resource information; the resource management agent is used to manage the access and exit of the cloud service corresponding to the cloud platform and maintain the service status of the cloud service; the resource control agent is used to schedule the task execution process of the resource information; the communication agent is used to realize the communication connection within the cloud adapter and between different cloud adapters.

[0007] In an optional implementation, parsing the API interface protocol of each of the cloud platforms and mapping the user's standardized resource request to a native API call corresponding to the cloud platform specifically includes: Parsing the resource configuration information input by the user, and extracting the standardized resource request corresponding to the cloud platform; Matching corresponding resource parameters according to the API interface protocol and converting the resource parameters into a standardized resource description format; The API interface protocol is parsed based on the standardized resource description format, and an executable API request instruction is generated according to the API calling rule corresponding to the API interface protocol.

[0008] In an optional implementation manner, a resource allocation scheme is constructed according to the network overhead corresponding to the cloud platform, and the method specifically includes: Determine the host load and the host load mean corresponding to the cloud platform, and determine the load stability representation corresponding to the cloud platform according to the host load and the host load mean; Determine the host overload threshold value according to the load stability characterization and the preset safety factor; When the quadratic exponential smoothing prediction value corresponding to the host load is greater than the host overload threshold, it is determined that the current host is in an overloaded state, and the task processing virtual machine is migrated to a target host that is not overloaded.

[0009] In an optional implementation manner, the target host is determined based on the following steps: Determine the resource memory and memory dirty page rate corresponding to the target resource; Determining a migration overhead corresponding to the target resource according to the resource memory and the memory dirty page rate; Determine the overall network overhead corresponding to the cloud platform according to the communication traffic between the task processing virtual machines and the communication delay between the hosts; Determine the total migration task cost according to the virtual machine network cost corresponding to the task processing virtual machine and the overall network cost; A host with the smallest total overhead of the migration task among the non-overloaded hosts is selected as the target host.

[0010] In an optional implementation, after the resource agent sends a resource scheduling instruction to the corresponding target cloud platform according to the resource allocation plan to retrieve the target resource, the method further includes: During the operation of the cloud adapter, recording the operation status attribute information corresponding to each service operation process; The running state attribute information is used as a basic network element and the service operation corresponding to the cloud adapter is used as a connecting edge to construct a semantic directed graph for storing the knowledge representation and accumulation corresponding to the cloud adapter.

[0011] In an optional implementation manner, the method further includes: When the new cloud adapter is connected to the cloud platform, the resource management agent is controlled by the cloud adapter to send information acquisition requests to the resource perception agent, the resource description agent and the resource integration agent respectively; Control the resource awareness agent, the resource description agent, and the resource integration agent to receive the information acquisition request and execute the corresponding resource processing function, and return the execution result to the resource management agent; The resource management agent is controlled to listen to the control instruction issued by the communication agent, and executes the corresponding instruction operation according to the control instruction, and feeds back the operation behavior corresponding to the instruction operation to the communication agent.

[0012] The present disclosure also provides a cloud platform resource scheduling device, including: A cloud adapter construction module is used to construct a cloud adapter composed of multiple resource agents and collect resource information of multiple cloud platforms, wherein each of the resource agents has at least one resource processing function; A data standardization module, used to perform standardized conversion on the resource information belonging to different cloud platforms according to a preset resource description standard through the cloud adapter to form a unified resource data model; An interface calling module, used to parse the API interface protocol of each cloud platform and map the user's standardized resource request to a native API call corresponding to the cloud platform; The resource retrieval module is used to match the target resources that meet the standardized resource request from the resource data model, build a resource allocation plan according to the network overhead corresponding to the cloud platform, and send resource scheduling instructions to the corresponding target cloud platform through the resource agent according to the resource allocation plan to retrieve the target resources.

[0013] An embodiment of the present disclosure also provides an electronic device, including: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the above-mentioned cloud platform resource scheduling method, or steps in any possible implementation of the above-mentioned cloud platform resource scheduling method are executed.

[0014] The embodiments of the present disclosure also provide a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program executes the above-mentioned cloud platform resource scheduling method, or the steps in any possible implementation of the above-mentioned cloud platform resource scheduling method.

[0015] The embodiments of the present disclosure also provide a computer program product, including a computer program / instruction, which, when executed by a processor, implements the above-mentioned cloud platform resource scheduling method, or the steps in any possible implementation of the above-mentioned cloud platform resource scheduling method.

[0016] The disclosed embodiment provides a cloud platform resource scheduling method, device, electronic device and storage medium, which constructs a cloud adapter composed of multiple resource agents and collects resource information of multiple cloud platforms, wherein each resource agent has at least one resource processing function; through the cloud adapter, according to the preset resource description standard, the resource information belonging to different cloud platforms is standardized and converted to form a unified resource data model; the API interface protocol of each cloud platform is parsed, and the user's standardized resource request is mapped to the native API call corresponding to the cloud platform; the target resource that meets the standardized resource request is matched from the resource data model, and a resource allocation scheme is constructed according to the network overhead corresponding to the cloud platform, and the resource agent sends a resource scheduling instruction to the corresponding target cloud platform according to the resource allocation scheme to retrieve the target resource. The standardized description, intelligent matching, optimized scheduling and cross-cloud collaborative execution of heterogeneous cloud resources can be realized, and efficient management and intelligent scheduling of cloud resources can be realized.

[0017] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0019] Figure 1 A flow chart of a cloud platform resource scheduling method provided by an embodiment of the present disclosure is shown; Figure 2 A schematic diagram of the structure of a cloud adapter provided by an embodiment of the present disclosure is shown; Figure 3 A schematic diagram of a cloud platform resource scheduling device provided by an embodiment of the present disclosure is shown; Figure 4 A schematic diagram of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0021] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0022] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.

[0023] Research has found that, at present, there are large differences in resource naming, specifications, storage methods, network topology, etc. among different cloud platforms, which makes it difficult to describe resources in a standardized manner and makes resource migration and scheduling between different platforms complex. The APIs and management interfaces provided by cloud service providers vary, and there are differences in calling methods, authentication mechanisms, and data formats, making it difficult to uniformly call and manage cross-cloud resources. Existing cross-cloud task scheduling strategies are usually based on static rules or simple load balancing algorithms, and fail to fully consider factors such as network latency, computing load, and data transmission costs, resulting in low resource utilization and low scheduling efficiency.

[0024] Based on the above research, the present disclosure provides a cloud platform resource scheduling method, device, electronic device and storage medium, constructs a cloud adapter composed of multiple resource agents, and collects resource information of multiple cloud platforms, wherein each resource agent has at least one resource processing function; through the cloud adapter, according to the preset resource description standard, the resource information belonging to different cloud platforms is standardized and converted to form a unified resource data model; the API interface protocol of each cloud platform is parsed, and the user's standardized resource request is mapped to the native API call corresponding to the cloud platform; the target resource that meets the standardized resource request is matched from the resource data model, and a resource allocation plan is constructed according to the network overhead corresponding to the cloud platform, and the resource agent sends a resource scheduling instruction to the corresponding target cloud platform according to the resource allocation plan to retrieve the target resource. It can realize the standardized description, intelligent matching, optimized scheduling and cross-cloud collaborative execution of heterogeneous cloud resources, and realize efficient management and intelligent scheduling of cloud resources.

[0025] To facilitate understanding of this embodiment, a cloud platform resource scheduling method disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the cloud platform resource scheduling method provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities, and the computer device includes, for example: a terminal device or a server or other processing device, and the terminal device can be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the cloud platform resource scheduling method can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0026] See also Figure 1 FIG. 1 is a flowchart of a cloud platform resource scheduling method provided by an embodiment of the present disclosure, wherein the method includes steps S101 to S104, wherein: S101. Construct a cloud adapter consisting of multiple resource agents, and collect resource information of multiple cloud platforms, wherein each of the resource agents has at least one resource processing function.

[0027] In the specific implementation, in the resource scheduling process of the cloud platform, due to the heterogeneity of resource management methods, interface protocols, data formats, etc. of different cloud platforms, in order to achieve unified management across platforms, it is necessary to introduce a cloud adapter. The cloud adapter consists of multiple resource agents, each of which is responsible for a specific resource processing task, thereby realizing the collection, management, conversion and scheduling of heterogeneous resources.

[0028] Here, the cloud adapter is composed of multiple resource agents, each of which undertakes a specific resource processing function. Resource agents include but are not limited to resource awareness agents, resource description agents, resource integration agents, resource management agents, resource control agents, and communication agents.

[0029] Among them, the resource perception agent is used to collect the operating environment and operating process information corresponding to the resource information; the resource description agent is used to provide the characteristic attribute information corresponding to the resource information to enhance the encapsulation capability of the resource information; the resource integration agent is used to manage the resource information library corresponding to the resource information; the resource management agent is used to manage the access and exit of the cloud service corresponding to the cloud platform and maintain the service status of the cloud service; the resource control agent is used to schedule the task execution process of the resource information; the communication agent is used to realize the communication connection within the cloud adapter and between different cloud adapters.

[0030] For example, see Figure 2 As shown, it is a schematic diagram of the structure of a cloud adapter provided by an embodiment of the present disclosure. The perception agent is used to collect resource information of each cloud platform, such as computing resources (CPU, GPU, memory), storage resources (disk, object storage), network resources (bandwidth, latency), task execution status, etc. The description agent is responsible for parsing the resource description methods of different cloud platforms and converting them into a unified standardized resource description for subsequent processing. The integration agent is responsible for managing and maintaining the resource information library of heterogeneous resources, including resource topology, resource status, historical usage records, etc. The management agent is responsible for managing the service access and exit of the cloud platform, and monitoring the service status of resources to ensure the availability and consistency of resources during the life cycle. The control agent is used to schedule resources and ensure the efficiency and reliability of task execution, and can optimize resource utilization efficiency based on scheduling strategies. The communication agent is used to realize resource calls and task scheduling instruction transmission between different cloud platforms, as well as data communication between agents inside the cloud adapter.

[0031] At the same time, the local Lightweight Directory Access Protocol (LDAP) directory is used to establish a local directory for manufacturing resources accessed through the cloud adapter to facilitate internal management of the adapter. The cloud knowledge base stores knowledge related to cloud manufacturing tasks completed by the cloud adapter.

[0032] It should be noted that cloud adapters display, collaborate or associate with other adapters or cloud platforms in the form of services. The information displayed to the outside can be abstractly represented as a four-element set: CMService=(ω, ψ, δ, φ), where ω is the relevant information of the cloud service, provided by the perception agent, description agent and integration agent, and can be represented by the set ω=(μ, ε, η); ψ is the function that the cloud service can provide, ψ=(θ, σ, λ), θ is the prerequisite input condition of the cloud service, σ is the operation set of the cloud service, and λ is the cloud service effect; δ is the status information of the cloud service; φ is the task list information of the cloud service.

[0033] Furthermore, in order to ensure that the cloud adapter can fully perceive the resource status of each cloud platform, the resource perception agent uses a variety of technical means to collect resource information. This process may include: obtaining the status information of virtual machines, storage, networks and other resources by accessing the APIs provided by each cloud platform (such as AWS EC2 API, Azure Resource Manager API, Google Cloud API). Extract resource usage from the cloud platform's log system, such as CPU load, disk I / O, network throughput, etc. Deploy a lightweight monitoring agent on the computing node of the target cloud platform, regularly collect resource usage, and send it to the cloud adapter. Analyze resource status changes by monitoring the cloud platform's data traffic, task execution logs, etc.

[0034] In the specific implementation, during the operation of the cloud adapter, multiple resource agents work together to ensure the uniformity of resource management across cloud platforms. The resource perception agent collects the resource status of each cloud platform in real time and transmits the data to the resource description agent; the resource description agent parses the resource formats of different cloud platforms and converts them into a unified resource data model; the resource integration agent stores and manages all collected resource data and maintains historical usage records; the resource management agent manages the life cycle of resources according to the cloud adapter's strategy; the resource control agent performs resource scheduling operations based on task requirements and resource allocation strategies; the communication agent ensures data transmission across cloud platforms and the issuance of task execution instructions.

[0035] In this way, by building multiple resource agents with different resource processing functions, the cloud adapter can realize the resource information collection of multiple cloud platforms and provide data support for subsequent standardized conversion, resource scheduling and task optimization.

[0036] S102. Through the cloud adapter, the resource information belonging to different cloud platforms is standardized and converted according to a preset resource description standard to form a unified resource data model.

[0037] In the specific implementation, in the cloud computing environment, due to the large differences in resource management methods, interface protocols, data formats, etc. of different cloud platforms, in order to achieve unified management and scheduling across cloud platforms, it is necessary to standardize the resource information to make it conform to the unified resource data model.

[0038] Here, different cloud platforms (such as AWS, Azure, Google Cloud, and OpenStack) provide different resource types, attribute names, units, API structures, etc. Resource naming and parameter formats of different cloud platforms are inconsistent, so a standardized conversion process is required to make them conform to a unified data format for subsequent scheduling and management.

[0039] First, the cloud adapter calls the API of each cloud platform or reads the resource information collected by the monitoring agent to parse the raw data returned by different cloud platforms. In order to achieve resource standardization, a resource description standard needs to be predefined to map the resource information of different cloud platforms into a unified format.

[0040] It should be noted that the standardized resource data is stored in a unified resource data model and managed by the resource integration agent in the cloud adapter. This model uses a relational database or a distributed NoSQL database for storage to ensure that the data can be quickly queried and expanded.

[0041] In this way, resources of different cloud platforms can be managed through a unified model to improve compatibility. After standardization, resource scheduling and allocation can be optimized based on unified indicators. At the same time, the differences between cloud platforms are shielded, simplifying development and operation and maintenance work. When a new cloud platform is added, only the mapping rules of the adapter need to be expanded without modifying the core scheduling logic.

[0042] As a possible implementation method, when a new cloud adapter is connected to the cloud platform, the cloud adapter controls the resource management agent to send information acquisition requests to the resource perception agent, resource description agent and resource integration agent respectively; controls the resource perception agent, resource description agent and resource integration agent to receive the information acquisition request and execute the corresponding resource processing function, and returns the execution result to the resource management agent; controls the resource management agent to listen to the control instructions issued by the communication agent, and executes the corresponding instruction operation according to the control instruction, and feeds back the operation behavior corresponding to the instruction operation to the communication agent.

[0043] In the specific implementation, whenever a new cloud adapter is connected to the cloud platform, the platform will send an initialization instruction to the cloud adapter. The cloud adapter starts the communication agent, management agent, description agent, integration agent, perception agent and control agent through the instruction. The communication agent serves as the message transmission medium inside the cloud adapter; the management agent sends information acquisition requests to the description agent, integration agent and perception agent respectively; the description agent, integration agent and perception agent each obtain the part of service and resource information they are responsible for, and transmit it back to the management agent. In the near-zero encapsulation mode, only the description information entered in advance by the owner can be obtained, and the integration information and perception information are basically not obtained; the control agent operates according to the instructions issued by the communication agent and conveys the feedback of the control behavior to the communication agent. Only in the highly encapsulated mode can the control agent directly control the manufacturing resources, otherwise it can only make certain requirements on the operator indirectly.

[0044] Here, the cloud adapter uses the Agent Communication Language (ACL) specified by the international agent interoperability standard defined by the Foundation for Intrigant Physical Agents (FIPA) to define the message format used in the communication process between agents. The ACL message format mainly includes the message sender, message receiver, and message content, as well as a series of contents such as the syntax and ontology used in the message content. For example, based on the use of the agent's receive method, a partial code of an ACL message is obtained from the agent message queue.

[0045] S103: Parse the API interface protocol of each of the cloud platforms, and map the user's standardized resource request to a native API call corresponding to the cloud platform.

[0046] In the specific implementation, in a multi-cloud environment, the API interface protocols of different cloud platforms are different, including request format, authentication method, resource identification, etc. Therefore, in order to achieve cross-cloud resource scheduling, the cloud adapter needs to parse the API interface protocols of each cloud platform and map the user's standardized resource request to the corresponding native API call.

[0047] Specifically, parse the resource configuration information input by the user and extract the standardized resource request of the corresponding cloud platform; match the corresponding resource parameters according to the API interface protocol, and convert the resource parameters into a standardized resource description format; parse the API interface protocol based on the standardized resource description format, and generate executable API request instructions according to the API call rules corresponding to the API interface protocol.

[0048] Here, the cloud adapter needs to parse the API protocols of different cloud platforms, extract the request method (Method), request path (Endpoint), parameter format (QueryString / JSON), authentication method (OAuth / Signature), and return data structure (JSON / XML). The API parsing component of the cloud adapter pre-stores the API protocols of each cloud platform and dynamically loads the adaptation rules.

[0049] During parsing, the cloud adapter will automatically match the corresponding API rules and perform parameter filling. Users submit standardized resource requests through the cloud management platform, and the cloud adapter needs to convert the request into the native API call format of the cloud platform. The request mapping component will map the standardized request to the native API format of the cloud platform based on the API protocol parsing results. Different cloud platforms have different authentication methods, and the authentication adapter component of the cloud adapter will select the appropriate authentication method based on the target cloud platform.

[0050] In the specific implementation, the role of the adapter interface layer is to manage multiple heterogeneous cloud computing platforms, so that various types of resource pools can be managed globally and centrally in one software system. Different cloud computing platform interfaces are encapsulated and transformed by the adapter interface layer to output unified interface methods for the upper resource management layer and system service layer to call. The final state of this layer is a set of Java class libraries that can work between different IaaS solutions, encapsulating the API interface open to the heterogeneous cloud computing environment, and providing the execution method of the adapter unified interface to the upper layer.

[0051] For example, the OpenStack cloud operating system provides a large number of APIs for YOUR APPLICATIONS, Dashboard, Monitoring & Tools. The adapter interface layer can manipulate all resources managed by OpenStack by calling these APIs. There are many cloud computing platforms based on the OpenStack open source framework. This key technology is based on a multi-version heterogeneous Openstack platform and provides IaaS cloud services at the physical resource layer. However, its open API interfaces are different. Some are based on the open source library OpenStack4j, while the heterogeneous Openstack platform uses the OpenStack native RESTful API.

[0052] Among them, the container cloud platform uses a unified interface call based on Java API. Java API is a set of Java class libraries designed to simplify the VISDK Web service interface. The adapter interface layer uses object-oriented programming methods and connects to vCenter Server through API to configure parameters and manage the entire life cycle of the virtual machine.

[0053] As a possible implementation, the primary task of the adapter interface layer is to make the Adapter and the incompatible Target work together in the same project, so interface conversion is required. Among them, the Adapter corresponds to the adapter role in the adapter interface layer, which is used to produce the specific implementation of the Adaptee object; the Target corresponds to the target object role in the adapter interface layer, which means that each module Adapter must implement the interface required by the CMP.

[0054] Specifically, the three existing cloud computing platforms open interfaces are encapsulated one by one and adapted to the target interface required by the management layer. The interface transformation process does not modify the existing classes and interfaces, but adds a new object, namely the Adapter class, from the outside to complete the conversion from the old interface to the new interface. The Adapter receives the parameters required by the cloud platform interface as input information, outputs the cloud platform interface response information, and encapsulates the input and output into JavaBeans.

[0055] Furthermore, the resource unified management layer or system service layer establishes a connection with the heterogeneous cloud computing platform through the HTTP protocol or directly calls the socket interface. The adapter interface layer using this method has good scalability, and different types and versions of cloud computing platforms can be added at any time, and can switch calls between heterogeneous cloud computing platforms at will.

[0056] In the specific implementation, the workflow of the adapter is as follows: the private cloud management system executes the method setRequestParameters() to set the request parameters, adds the parameter items required for specific functions such as viewing the cloud computing platform resource pool type and creating a cloud host instance, and completes the parameter configuration of the request method; sends a request by obtaining the adapter method getAdapter(), and calls the unified external interface in the adapter JAR package to create the corresponding adapter object xxxAdapter; after receiving the request, the adapter interface layer calls the specific implementation class, creates the adapter execution object xxxAdaptee internally, and obtains the adapter singleton; the adapter constructs the API specified by the corresponding cloud computing platform according to the specific type of the incoming request. Request URL, API request parameters mainly include host address, user name, password, tenant information, etc., and serialize the parameters and URL; the adapter interface layer executes the check method checkConnection() to detect whether the server connection is successful; the adapter interface layer executes the run method executeMethod() to run API commands for functions such as obtaining tokens, querying the cloud host list, operating the entire life cycle of virtual machines, mounting or unmounting disks, creating or canceling images, etc.; finally, the adapter interface layer will return a response message in JSON format, encapsulate it as a response class and send it back to the caller of the private cloud management system. The client program receives the response message and displays the result in a graphical interface to feed back to the user.

[0057] S104. Match the target resources that meet the standardized resource request from the resource data model, build a resource allocation plan according to the network overhead corresponding to the cloud platform, and send a resource scheduling instruction to the corresponding target cloud platform through the resource agent according to the resource allocation plan to retrieve the target resources.

[0058] In a specific implementation, target resources that meet the standardized resource request are matched from the resource data model, a resource allocation plan is constructed according to the network overhead corresponding to the cloud platform, and a resource scheduling instruction is sent to the corresponding target cloud platform through the resource agent according to the resource allocation plan to retrieve the target resources.

[0059] Here, since different cloud platforms have different resource types, specifications, pricing, and performance parameters, in order to achieve cross-cloud resource scheduling, the cloud adapter will standardize the resource information of all cloud platforms to form a unified resource data model. This model contains information such as resource unique identifier, provider type, specification, region, price, network latency, etc. When a user submits a standardized resource request, the cloud adapter will search for matching resources in the standardized resource data model.

[0060] When multiple suitable resources are matched, it is necessary to build the best resource allocation plan based on the network overhead of the cloud platform. When selecting target resources, the cloud adapter will consider giving priority to resources with lower network overhead, reducing cross-cloud platform transmission costs, and optimizing distributed scheduling of tasks.

[0061] As a possible implementation method, the host load and the host load mean corresponding to the cloud platform are determined, and the load stability characterization corresponding to the cloud platform is determined based on the host load and the host load mean; the host overload threshold is determined based on the load stability characterization and a preset safety factor; when the quadratic exponential smoothing prediction value corresponding to the host load is greater than the host overload threshold, it is determined that the current host is in an overloaded state, and the task processing virtual machine is migrated to a target host that is not overloaded.

[0062] In the specific implementation, in the data center, resources are deployed in different clusters and different platforms. Since the resource configuration of the cluster and the platform is not consistent, and the resource load will change with the scheduling of tasks, the cluster load is also constantly changing. When the cluster load is unbalanced, it will not only cause a large loss of energy utilization, but also lead to a decline in service quality. At this time, resource migration is required to balance the overall load. At the same time, network resources are also an important factor in ensuring service quality, so it is necessary to minimize the network overhead of the data center.

[0063] Here, the embodiment of the present application comprehensively considers the usage of CPU, memory, disk, and bandwidth to measure the overall load of the platform. In order to measure the load of the platform, the load of the platform can be defined as the following formula:

[0064] Among them, H load Represents the load condition of the platform; U cpu Represents the CPU usage of the platform; U mem Represents the memory usage of the platform; U disk Represents the disk usage of the platform; U bw Represents the bandwidth utilization rate of the platform; α, β, and γ represent their respective weights. The sum of the weights of the four resource utilization rates is 1, satisfying the following constraint: 0<α+β+γ<1.

[0065] It should be noted that the weight can be adjusted according to demand, that is, if the CPU resource is more important than other resources in the platform, the weight can be increased. Generally speaking, CPU resources are key resources that affect service quality and energy consumption, so the CPU weight is relatively large, and the memory, disk, and bandwidth weights are set according to specific circumstances.

[0066] Furthermore, since the host load is not always stable, the static threshold method will generate many unnecessary migrations, so the dynamic threshold method is used to determine the upper limit of the host load. For each platform, the load stability of the host can be calculated to determine whether the host load is stable. Record the load values ​​of n cycles and define the host load stability as follows:

[0067] Among them, S load Indicates the load stability of the host, H i represents the load of host i, H avg It can be seen that S load The smaller the value, the more stable the host load is, the smaller the load fluctuation is, and the lower the probability of host overload is. load It can also represent the probability of host overload.

[0068] Furthermore, the host overload threshold can be expressed as:

[0069] Where T represents the host's overload threshold; S load Indicates the load stability of the host; s represents the safety factor, which controls the impact of load balance on the overload threshold. The larger s is, the greater the impact of load balance on the overload threshold.

[0070] Here, when S load When S is larger, the overload threshold will be lowered, that is, the greater the load fluctuation of the host, the more likely the host will be overloaded. Therefore, it is necessary to lower the overload threshold and migrate the virtual machine when it exceeds the threshold, thereby reducing the risk of host overload. load The smaller the value, the larger the overload threshold will be. That is, when the load fluctuation of the host is small, the load of the host is relatively stable. The overload threshold can be increased to increase the utilization of host resources.

[0071] Furthermore, after determining the overload upper limit through dynamic threshold calculation, the prediction algorithm is used to predict the load of the host. If the prediction result exceeds the upper limit, it proves that the host is overloaded and the virtual machine needs to be migrated, that is, the quadratic exponential smoothing prediction value S is used. t+n As the predicted load value of the host, when S t+n When it is greater than the host overload threshold T, the virtual machine migration operation needs to be performed.

[0072] As a possible implementation method, the target host is determined based on the following steps: determine the resource memory and memory dirty page rate corresponding to the target resource; determine the migration overhead corresponding to the target resource based on the resource memory and memory dirty page rate; determine the overall network overhead corresponding to the cloud platform based on the communication traffic between the task processing virtual machines and the communication delay between the hosts; determine the total migration task overhead based on the virtual machine network overhead corresponding to the task processing virtual machine and the overall network overhead; select the host with the smallest total migration task overhead among the non-overloaded hosts as the target host.

[0073] In actual applications, the hosts of a data center are placed in different computer rooms and racks. Since the network topologies of different computer rooms and racks are different, the communication between the two platforms will be affected. When processing tasks, virtual machines often need to communicate with other virtual machines, so placing virtual machines on different platforms will also cause different communication overheads. Since the quality of network communication will greatly affect the quality of service, the target host selection algorithm based on network perception in the embodiment of the present application ensures that the overall network overhead of the data center is as small as possible, thereby reducing network traffic and ensuring the quality of service.

[0074] Specifically, the overall network overhead of the entire data center can be expressed as:

[0075] Among them, Cost com (DC) represents the overall network cost; C i，j Represents virtual machine V i and virtual machine V j Communication flow between kl Show host S K and S L The communication delay can be indirectly represented by the number of router hops.

[0076] At the same time, the target host needs to meet the following constraints:

[0077] Among them, V cpuneed , V memneed , V diskneed , V bwneed They represent the CPU resources, memory resources, disk resources, and bandwidth resources required by the virtual machine. cpuneed , S memneed , S diskneed , S bwneed Respectively indicate the remaining CPU resources, memory resources, disk resources, and bandwidth resources of the target host.

[0078] Furthermore, for dynamic migration of virtual machines, compared with static migration, the overhead of memory migration needs to be considered. i From the host S k Dynamically migrate to host S p , Resource V i The memory size is , the memory dirty page rate is , the number of iterations of the migration process is m. According to the research on the dynamic resource migration mechanism, since the migration process also needs to migrate the dirty pages of memory, the total amount of memory migration during the resource migration process is defined as:

[0079]

[0080] Among them, Mig (V i ) represents resource V i The total amount of memory migrated during the migration process; dirty page rate It obeys uniform distribution and its value range is 0.09~0.11; bw kp Represents the available bandwidth between hosts.

[0081] Furthermore, resource V i From the host S k Dynamically migrate to host S p The migration cost is:

[0082] Among them, Cost live represents migration cost; Data (Vi) represents resource data; d kp Show host S K and S P Communication delay of bw kp Represents the available bandwidth between hosts; Mig(V i ) represents resource V i The total amount of memory migrated during the migration process.

[0083] It should be noted that the above formula corresponds to a non-shared storage solution. If it is a shared storage solution, then Data(V i ) value is 0, which means that the migration process does not need to copy the base image and instance files.

[0084] Furthermore, by combining the above two models, the migration of virtual machines can be measured based on the overall network overhead and migration overhead. The overall network overhead is calculated when the virtual machine is deployed on the target host Sp. In order to eliminate the magnitude difference, the overall network overhead, migration overhead and virtual machine network overhead of the entire data center are normalized according to the maximum delay between hosts, the maximum communication traffic between virtual machines, the maximum static migration data of virtual machines and the minimum available bandwidth between hosts. For static migration, the total migration task overhead is:

[0085] in, Represents the total cost of the migration task; Represents the normalized overall network overhead; Represents the normalized virtual machine network overhead.

[0086] Similarly, for dynamic migration, the total migration task overhead is:

[0087] in, Represents the total cost of the migration task; Represents the normalized overall network overhead; Represents the normalized virtual machine network overhead.

[0088] It should be noted that μ and α are correlation coefficients used to determine the weights of network overhead and migration overhead, μ + α = 1. After the total migration overhead is introduced, when selecting the target host, the host with the smallest total migration overhead is selected for migration, so that the network communication overhead of multiple clusters can be minimized as much as possible, and the migration speed can also be optimized.

[0089] As a possible implementation method, during the operation of the cloud adapter, the operation status attribute information corresponding to each service operation process is recorded; using the operation status attribute information as the basic network element and the service operation corresponding to the cloud adapter as the connecting edge, a semantic directed graph corresponding to the storage cloud adapter corresponding knowledge representation and accumulation is constructed.

[0090] In the specific implementation, a cloud adapter is given an initial information representation and operation state, represented by (ω0, δ0). During the operation of the cloud adapter, the operation set σ of the cloud manufacturing service is recorded. a , σ betc., and service effects (ω1, δ1), (ω2, δ2), etc., to form multiple cloud adapter knowledge representation basic network elements based on semantic directed graphs. The information representation and operation status attribute set perceived, integrated and described by the cloud adapter in the basic network element are represented by service effects (ω1, δ1), (ω2, δ2), etc., that is, the cloud adapter after the cloud service is completed. With the operation set σ a , σ b Etc. (i.e., the set of operations brought by the cloud service) are used as connecting edges, and the basic network elements of the cloud adapter knowledge representation are combined to obtain a semantic directed graph of cloud adapter knowledge representation and accumulation.

[0091] In this way, the knowledge of the cloud resource adapter during operation can be saved, and the knowledge reuse of the cloud adapter can be realized conveniently and quickly, which helps to solve the same problems encountered in cloud service encapsulation.

[0092] The disclosed embodiment provides a cloud platform resource scheduling method, which constructs a cloud adapter composed of multiple resource agents and collects resource information of multiple cloud platforms, wherein each resource agent has at least one resource processing function; through the cloud adapter, according to the preset resource description standard, the resource information belonging to different cloud platforms is standardized and converted to form a unified resource data model; the API interface protocol of each cloud platform is parsed, and the user's standardized resource request is mapped to the native API call corresponding to the cloud platform; the target resource that meets the standardized resource request is matched from the resource data model, and a resource allocation plan is constructed according to the network overhead corresponding to the cloud platform, and the resource agent sends a resource scheduling instruction to the corresponding target cloud platform according to the resource allocation plan to retrieve the target resource. The standardized description, intelligent matching, optimized scheduling and cross-cloud collaborative execution of heterogeneous cloud resources can be realized, and efficient management and intelligent scheduling of cloud resources can be realized.

[0093] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0094] Based on the same inventive concept, the embodiment of the present disclosure also provides a cloud platform resource scheduling device corresponding to the cloud platform resource scheduling method. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned cloud platform resource scheduling method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0095] See also Figure 3 , Figure 3 A schematic diagram of a cloud platform resource scheduling device provided by an embodiment of the present disclosure. Figure 3As shown in , the cloud platform resource scheduling device 300 provided in the embodiment of the present disclosure includes: A cloud adapter construction module 310 is used to construct a cloud adapter composed of multiple resource agents and collect resource information of multiple cloud platforms, wherein each resource agent has at least one resource processing function; The data standardization module 320 is used to standardize and convert the resource information belonging to different cloud platforms according to a preset resource description standard through the cloud adapter to form a unified resource data model; The interface calling module 330 is used to parse the API interface protocol of each cloud platform and map the user's standardized resource request to a native API call corresponding to the cloud platform; The resource retrieval module 340 is used to match the target resources that meet the standardized resource request from the resource data model, build a resource allocation plan according to the network overhead corresponding to the cloud platform, and send resource scheduling instructions to the corresponding target cloud platform through the resource agent according to the resource allocation plan to retrieve the target resources.

[0096] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0097] The cloud platform resource scheduling device provided by the embodiment of the present disclosure constructs a cloud adapter composed of multiple resource agents, and collects resource information of multiple cloud platforms, wherein each resource agent has at least one resource processing function; through the cloud adapter, according to the preset resource description standard, the resource information belonging to different cloud platforms is standardized and converted to form a unified resource data model; the API interface protocol of each cloud platform is parsed, and the user's standardized resource request is mapped to the native API call corresponding to the cloud platform; the target resource that meets the standardized resource request is matched from the resource data model, and a resource allocation scheme is constructed according to the network overhead corresponding to the cloud platform, and the resource agent sends a resource scheduling instruction to the corresponding target cloud platform according to the resource allocation scheme to retrieve the target resource. The standardized description, intelligent matching, optimized scheduling and cross-cloud collaborative execution of heterogeneous cloud resources can be realized, and efficient management and intelligent scheduling of cloud resources can be realized.

[0098] Corresponds to Figure 1 The cloud platform resource scheduling method in the present disclosure also provides an electronic device 400, such as Figure 4 FIG. 4 is a schematic diagram of the structure of an electronic device 400 provided in an embodiment of the present disclosure, including: Processor 41, memory 42, and bus 43; memory 42 is used to store execution instructions, including memory 421 and external memory 422; memory 421 here is also called internal memory, which is used to temporarily store operation data in processor 41 and data exchanged with external memory 422 such as hard disk. Processor 41 exchanges data with external memory 422 through memory 421. When the electronic device 400 is running, the processor 41 communicates with the memory 42 through bus 43, so that the processor 41 executes Figure 1 Steps of the cloud platform resource scheduling method.

[0099] The embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the cloud platform resource scheduling method described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0100] The embodiments of the present disclosure also provide a computer program product, which includes computer instructions. When the computer instructions are executed by a processor, the steps of the cloud platform resource scheduling method described in the above method embodiment can be executed. For details, please refer to the above method embodiment, which will not be repeated here.

[0101] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0102] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0103] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0104] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0105] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0106] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A cloud platform resource scheduling method, characterized in that: include: Constructing a cloud adapter consisting of multiple resource agents and collecting resource information of multiple cloud platforms, wherein each resource agent has at least one resource processing function; By means of the cloud adapter, the resource information belonging to different cloud platforms is standardized and converted according to a preset resource description standard to form a unified resource data model; Parse the API interface protocol of each cloud platform and map the user's standardized resource request to a native API call corresponding to the cloud platform; Match target resources that meet the standardized resource request from the resource data model, build a resource allocation plan based on the network overhead corresponding to the cloud platform, and send resource scheduling instructions to the corresponding target cloud platform through the resource agent according to the resource allocation plan to retrieve the target resources.

2. The method according to claim 1, characterized in that The resource agent at least includes a resource perception agent, a resource description agent, a resource integration agent, a resource management agent, a resource control agent and a communication agent; The resource perception agent is used to collect the operating environment and operating process information corresponding to the resource information; the resource description agent is used to provide the characteristic attribute information corresponding to the resource information to enhance the encapsulation capability of the resource information; the resource integration agent is used to manage the resource information library corresponding to the resource information; the resource management agent is used to manage the access and exit of the cloud service corresponding to the cloud platform and maintain the service status of the cloud service; the resource control agent is used to schedule the task execution process of the resource information; The communication agent is used to implement the communication connection within the cloud adapter and between different cloud adapters.

3. The method according to claim 1, characterized in that Parse the API interface protocol of each cloud platform and map the user's standardized resource request to the native API call of the corresponding cloud platform, specifically including: Parsing the resource configuration information input by the user, and extracting the standardized resource request corresponding to the cloud platform; Matching corresponding resource parameters according to the API interface protocol and converting the resource parameters into a standardized resource description format; The API interface protocol is parsed based on the standardized resource description format, and an executable API request instruction is generated according to the API calling rule corresponding to the API interface protocol.

4. The method according to claim 1, characterized in that: Constructing a resource allocation scheme according to the network overhead corresponding to the cloud platform, the method specifically includes: Determine the host load and the host load mean corresponding to the cloud platform, and determine the load stability representation corresponding to the cloud platform according to the host load and the host load mean; Determine the host overload threshold value according to the load stability characterization and the preset safety factor; When the quadratic exponential smoothing prediction value corresponding to the host load is greater than the host overload threshold, it is determined that the current host is in an overloaded state, and the task processing virtual machine is migrated to a target host that is not overloaded.

5. The method according to claim 4, characterized in that The target host is determined based on the following steps: Determine the resource memory and memory dirty page rate corresponding to the target resource; Determining a migration overhead corresponding to the target resource according to the resource memory and the memory dirty page rate; Determine the overall network overhead corresponding to the cloud platform according to the communication traffic between the task processing virtual machines and the communication delay between the hosts; Determine the total migration task cost according to the virtual machine network cost corresponding to the task processing virtual machine and the overall network cost; A host with the smallest total overhead of the migration task among the non-overloaded hosts is selected as the target host.

6. The method according to claim 1, characterized in that After the resource agent sends a resource scheduling instruction to the corresponding target cloud platform according to the resource allocation scheme to retrieve the target resource, the method further includes: During the operation of the cloud adapter, recording the operation status attribute information corresponding to each service operation process; The running state attribute information is used as a basic network element and the service operation corresponding to the cloud adapter is used as a connecting edge to construct a semantic directed graph for storing the knowledge representation and accumulation corresponding to the cloud adapter.

7. The method according to claim 2, characterized in that The method further comprises: When the new cloud adapter is connected to the cloud platform, the resource management agent is controlled by the cloud adapter to send information acquisition requests to the resource perception agent, the resource description agent and the resource integration agent respectively; Control the resource awareness agent, the resource description agent, and the resource integration agent to receive the information acquisition request and execute the corresponding resource processing function, and return the execution result to the resource management agent; The resource management agent is controlled to listen to the control instruction issued by the communication agent, and executes the corresponding instruction operation according to the control instruction, and feeds back the operation behavior corresponding to the instruction operation to the communication agent.

8. A cloud platform resource scheduling device, characterized in that: include: A cloud adapter construction module is used to construct a cloud adapter composed of multiple resource agents and collect resource information of multiple cloud platforms, wherein each of the resource agents has at least one resource processing function; A data standardization module, used to perform standardized conversion on the resource information belonging to different cloud platforms according to a preset resource description standard through the cloud adapter to form a unified resource data model; An interface calling module, used to parse the API interface protocol of each cloud platform and map the user's standardized resource request to a native API call corresponding to the cloud platform; The resource retrieval module is used to match the target resources that meet the standardized resource request from the resource data model, build a resource allocation plan according to the network overhead corresponding to the cloud platform, and send resource scheduling instructions to the corresponding target cloud platform through the resource agent according to the resource allocation plan to retrieve the target resources.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the cloud platform resource scheduling method as described in any one of claims 1 to 7 are performed.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the cloud platform resource scheduling method as described in any one of claims 1 to 7.

Images