Multi-cloud deployment scheduling method, device, equipment, medium and program product

Abstract

The invention discloses a multi-cloud deployment scheduling method and device, equipment, a medium and a program product. The method can be applied to the field of cloud computing, and comprises the following steps: responding to a scheduling request of a to-be-processed task, and extracting state features of multiple dimensions from environment state information of a current multi-cloud environment; and constructing a first scheduling strategy based on an accumulated reward obtained by evaluating the system performance index and the user preference index and the state characteristics of the multiple dimensions, and performing deployment and resource allocation on the to-be-processed task based on the first scheduling strategy. According to the scheme, the first scheduling strategy is constructed in combination with deep learning and reinforcement learning, so that the scheduling system can realize self-learning and adapt to a complex and changeable environment, and the intelligence of multi-cloud deployment scheduling is improved; in addition, the system performance index and the user preference matching degree serve as the optimization target of reinforcement learning, and the overall system efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of cloud computing technology, and in particular to a multi-cloud deployment scheduling method, device, electronic device, computer storage medium, and computer program product. Background Art

[0002] Multi-cloud deployment is an information technology strategy that allows the use of cloud resources and services from multiple cloud service providers (CSPs) in a business architecture to support its business operations, rather than relying on a single cloud platform, enabling enterprises to select and utilize different cloud services based on their specific business needs, cost-effectiveness, security, and compliance requirements.

[0003] Although multi-cloud deployment provides enterprises with a flexible, efficient and scalable cloud computing solution, it also brings challenges in managing and coordinating different cloud service providers. The formulation and implementation of scheduling strategies are crucial to ensuring the availability, reliability and cost-effectiveness of services. Summary of the invention

[0004] The embodiments of the present application provide a multi-cloud deployment scheduling method, device, electronic device, computer-readable storage medium, and computer program product, which can improve the intelligence of multi-cloud deployment scheduling.

[0005] A multi-cloud deployment scheduling method, comprising:

[0006] In response to a scheduling request for a pending task, obtaining environmental status information in a current multi-cloud environment;

[0007] Extracting features from the environmental status information to obtain status features in multiple dimensions;

[0008] Based on the cumulative reward obtained by evaluating the system performance index and the user preference index, and the state characteristics of the multiple dimensions, constructing a first scheduling strategy corresponding to the task to be processed, wherein the cumulative reward is the sum of multiple rewards, and each reward is determined based on a parameter value on at least one system performance index and a user preference index;

[0009] The tasks to be processed are deployed and resources are allocated based on the first scheduling strategy.

[0010] Accordingly, the embodiment of the present application also provides a multi-cloud deployment scheduling device, including:

[0011] A first acquisition unit, configured to acquire environmental status information in a current multi-cloud environment in response to a scheduling request for a task to be processed;

[0012] A feature extraction unit, used to extract features from the environmental state information to obtain state features of multiple dimensions;

[0013] A first construction unit is configured to construct a first scheduling strategy corresponding to the task to be processed based on a cumulative reward obtained by evaluating the system performance index and the user preference index, and the state characteristics of the multiple dimensions, wherein the cumulative reward is the sum of multiple rewards, and each reward is determined based on a parameter value of at least one system performance index and a user preference index;

[0014] A scheduling unit is used to deploy and allocate resources for the tasks to be processed based on the first scheduling strategy.

[0015] In addition, an embodiment of the present application also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the processor is used to run the computer program in the memory to implement the steps in the multi-cloud deployment scheduling method provided in an embodiment of the present application.

[0016] In addition, an embodiment of the present application also provides a computer-readable storage medium, which stores multiple instructions, and the instructions are suitable for a processor to load to execute the steps in the multi-cloud deployment scheduling method provided in the embodiment of the present application.

[0017] In addition, an embodiment of the present application also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the steps in the multi-cloud deployment scheduling method provided in an embodiment of the present application.

[0018] In the embodiment of the present application, by responding to the scheduling request of the task to be processed, the state characteristics of multiple dimensions are extracted from the environmental state information of the current multi-cloud environment, and then a first scheduling strategy is constructed based on the cumulative rewards obtained by evaluating the system performance indicators and user preference indicators, as well as the state characteristics of multiple dimensions, and the task to be processed is deployed and resources are allocated based on the first scheduling strategy. This solution combines deep learning and reinforcement learning to construct a scheduling strategy, so that the scheduling system can self-learn and adapt to complex and changing environments, and improves the intelligence of multi-cloud deployment scheduling; in addition, the system performance indicators and user preference matching are used as the optimization goals of reinforcement learning, which improves the overall system performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 It is a scenario diagram of the multi-cloud deployment scheduling method provided in an embodiment of the present application;

[0021] Figure 2 It is a flowchart of a multi-cloud deployment scheduling method provided in an embodiment of the present application;

[0022] Figure 3 It is a logical diagram of reinforcement learning provided by an embodiment of the present application;

[0023] Figure 4 It is a flow chart of a multi-cloud deployment scheduling method based on deep reinforcement learning with multi-objective optimization provided in an embodiment of the present application;

[0024] Figure 5 This is a schematic diagram of an application scenario of a data lake construction provided by an embodiment of the present application;

[0025] Figure 6 It is a structural diagram of a multi-cloud deployment scheduling device provided in an embodiment of the present application;

[0026] Figure 7 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0028] Existing cloud deployment scheduling often relies on static rules, such as polling priority and resource allocation strategy, but cannot adapt to dynamic changes, which easily leads to uneven resource allocation and resource waste, affecting deployment efficiency. Based on this, this application uses a multi-objective optimized deep reinforcement learning algorithm to build a task scheduling model, which enables it to self-learn and adapt to complex and changing environments, solves the problem of uneven resource allocation, and improves the intelligence of multi-cloud deployment scheduling; in addition, multiple system performance indicators and user preference matching are used as optimization goals to improve the overall system performance.

[0029] The embodiments of the present application provide a multi-cloud deployment scheduling method, device, electronic device, computer-readable storage medium, and computer program product, which can improve the intelligence of multi-cloud deployment scheduling. Among them, the multi-cloud deployment scheduling device can be integrated in an electronic device, which can be a server, a terminal, or other devices.

[0030] Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services, and basic cloud computing services such as big data and artificial intelligence platforms. The terminal can be a mobile phone, a computer, an intelligent voice interaction device, a smart home appliance, a vehicle terminal, an aircraft, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.

[0031] In order to better understand the multi-cloud deployment scheduling method, device, electronic device, computer-readable storage medium, and computer program product provided in the embodiments of the present application, the application environment applicable to the embodiments of the present application is first described below.

[0032] See also Figure 1 , Figure 1 A schematic diagram of an application environment of the multi-cloud deployment scheduling method provided in an embodiment of the present application is shown. As an implementation method, the environment state acquisition method, state feature extraction method, scheduling strategy construction method and task scheduling method provided in the embodiment of the present application can be applied to the same electronic device. The electronic device can be as follows: Figure 1 The server 10 shown in the figure can be connected to the terminal device 20 through a network. The network is used to provide a medium for a communication link between the server 10 and the terminal device 20. The network can include various connection types, such as a wired communication link, a wireless communication link, etc., which is not limited in the embodiment of the present application. Optionally, in other embodiments, the electronic device can also be a smart phone, a laptop computer, etc.

[0033] It should be understood that Figure 1 The server 10, network and terminal device 20 are merely illustrative. Any number of servers, networks and terminal devices may be provided as required. For example, the server 10 may be a physical server or a server cluster composed of multiple servers, and the terminal device 20 may be a mobile phone, a tablet, a desktop computer, a laptop computer, and the like. It is understood that the embodiments of the present application may also allow multiple terminal devices 20 to access the server 10 at the same time.

[0034] In some embodiments, the terminal device 20 can receive user operations to trigger pending tasks. Further, the terminal device 20 sends a scheduling request for pending tasks to the server 10 through the network. After the server 10 receives the scheduling request for the pending tasks, it obtains the environmental state information in the current multi-cloud environment and extracts features of the environmental state information. Then, based on the extracted state features of multiple dimensions and the accumulated rewards obtained by evaluating the system performance indicators and the user preference indicators, a first scheduling strategy is constructed, and the pending tasks are deployed and resources are allocated based on the first scheduling strategy.

[0035] As another implementation mode, the environmental state acquisition, state feature extraction, scheduling strategy construction, task scheduling and other methods provided in the embodiments of the present application can be applied to different electronic devices. For example, the environmental state acquisition method and the task scheduling method are applied to electronic device A, the state feature extraction method is applied to electronic device B, the scheduling strategy construction method is applied to electronic device C, and so on. In addition, any of the environmental state acquisition, state feature extraction, scheduling strategy construction and task scheduling methods provided in the embodiments of the present application can be applied to the same electronic device. For example, the environmental state acquisition method and the task scheduling method are applied to electronic device A, and the state feature extraction method and the scheduling strategy construction method are applied to electronic device B. The embodiments of the present application do not limit the electronic devices to which the above methods are applied.

[0036] It should be noted that Figure 1 The scenario diagram of the multi-cloud deployment scheduling method application shown is only an example. The multi-cloud deployment scheduling method application and scenario described in the embodiment of the present application are for more clearly illustrating the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided by the embodiment of the present application. Ordinary technicians in this field can know that with the evolution of the application of the multi-cloud deployment scheduling method and the emergence of new business scenarios, the technical solution provided by the embodiment of the present application is also applicable to similar technical problems.

[0037] It can be understood that in the specific implementation of this application, related data such as attribute data, attribute sets and attribute subsets are involved. When the following embodiments of this application are applied to specific products or technologies, permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0038] It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments.

[0039] This embodiment will be described from the perspective of a multi-cloud deployment scheduling device. The multi-cloud deployment scheduling device can be specifically integrated into an electronic device, which can be a server or a terminal or other device; wherein the terminal can be a mobile phone, a computer, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, etc., but is not limited thereto.

[0040] The embodiment of the present application provides a multi-cloud deployment scheduling method, including: in response to a scheduling request for a pending task, obtaining environmental status information in the current multi-cloud environment; extracting features from the environmental status information to obtain status features in multiple dimensions; constructing a first scheduling strategy corresponding to the pending task based on the cumulative rewards obtained by evaluating system performance indicators and user preference indicators, and status features in multiple dimensions, wherein the cumulative rewards are the sum of multiple rewards, and each reward is determined based on a parameter value on at least one system performance indicator and user preference indicator; and deploying and allocating resources for the pending task based on the first scheduling strategy. Figure 2 As shown, the specific process of the multi-cloud deployment scheduling method is as follows:

[0041] 101. In response to a scheduling request for a task to be processed, obtain environmental status information in a current multi-cloud environment.

[0042] The scheduling request for the pending task may be triggered by a user through an interface or API, and may be a request to schedule one or a batch of tasks. The scheduling request may include a detailed description of the task, performance requirements, and deadlines. Specifically, after receiving the scheduling request, the system will start to collect environmental status information in the current multi-cloud environment to prepare for subsequent task scheduling and resource scheduling.

[0043] In one embodiment, the environment status information may include indicators of multiple different dimensions, such as cloud resource dimension, user dimension, and task dimension, etc. For example, the environment status information may include: resource usage status of each cloud node, user priority of each user using cloud computing services, and task attribute information of each task to be processed.

[0044] The resource usage status may include usage of various resource types, such as CPU utilization, memory occupancy, disk space utilization, and network bandwidth utilization, etc. In actual applications, a percentage value may be set for each resource type to represent the resource usage.

[0045] User priority can reflect the user's contribution to the system or the requirements of its service level agreement (SLA). In actual applications, multiple levels of user priority can be set and comprehensively evaluated based on factors such as the user's historical consumption and contract level.

[0046] In this embodiment, the task attribute information at least includes a position indicator in the associated call link, such as the position of the task or the application to which the task belongs in its associated call link. In practical applications, the number of dependent applications can be used to represent the size of the position indicator. For example, in a certain call link, task A depends on task B, task B depends on task C, and task C depends on task D. Then, in this call link, task A needs to depend on 3 tasks, task B needs to depend on 2 tasks, task C needs to depend on 1 task, and task D does not need to depend on any tasks. Then, the size of the position indicator of task A is 3, the size of the position indicator of task B is 2, the size of the position indicator of task C is 1, and the size of the position indicator of task D is 0. The larger the value of the position indicator, the worse the stability.

[0047] In one embodiment, the task attribute information may also include the life cycle environment, that is, the working environment corresponding to different stages in the software development and deployment process, such as the development environment, the test environment, the pre-production environment, etc. Among them, different life cycle environments affect the risk tolerance of scheduling decisions. For example, the risk tolerance of scheduling decisions corresponding to the production environment is lower, and the risk tolerance of scheduling decisions corresponding to the development environment is higher. In specific implementation, some form of identification can be used to mark whether the task is running in the production environment. For example, a binary variable can be used, where 0 represents a non-production environment (such as a development environment, a test environment, a pre-production environment, etc.), and 1 represents a production environment.

[0048] In one embodiment, the task attribute information may also include the task urgency. Specifically, the task urgency may be measured based on the task deadline and the current time difference (i.e., the remaining time). In practical applications, the remaining time and the preset importance of the task may be considered comprehensively to more accurately determine the task urgency.

[0049] It should be noted that the resource usage status, user priority, and task attribute information obtained above are for all cloud nodes in the current multi-cloud environment, all pending tasks in the task queue, and all users using cloud services.

[0050] 102. Perform feature extraction on the environmental state information to obtain state features in multiple dimensions.

[0051] Specifically, by analyzing and processing the acquired environmental status information, state features of multiple dimensions are extracted therefrom. Among them, the state features of multiple dimensions may include state features of cloud resource dimension, user dimension and task dimension. Furthermore, a finer-grained dimensional division can be performed according to actual needs. For example, the state features of multiple dimensions may include resource usage status (such as utilization rate of different types of resources), user importance (such as user priority), task stability (such as the position of the task in the call link, the number of applications associated with the task, etc.), task importance (such as the life cycle environment of the task), task urgency (such as the deadline of the task) and other state features in multiple dimensions.

[0052] In the embodiment of the present application, there are many ways to extract features from the environmental state information. For example, deep learning can be used to extract features from the environmental state information, or statistical methods, cluster analysis, etc. can be used to extract features from the environmental state information.

[0053] For example, taking the statistical method as an example, target-related features (such as resource usage status, user priority, task position in the call link, task life cycle environment, task urgency, etc.) can be extracted from the original data of environmental information, and then data preprocessing operations (such as data cleaning, normalization and standardization processing, etc.) can be performed to obtain state features of multiple dimensions, each of which corresponds to an indicator. Furthermore, the state features of multiple dimensions obtained can be quantified separately, and the quantified data of multiple dimensions can be combined into a state vector to represent the environmental state in the current multi-cloud environment.

[0054] For another example, using deep learning as an example, the acquired environmental status information can be preprocessed, and then the convolutional layer in the constructed deep neural network can be used to extract features from the preprocessed environmental status information to obtain state features in multiple dimensions. Furthermore, a vector representation can be constructed for the extracted state features in multiple dimensions to obtain a multi-dimensional state vector, which is used to represent the environmental status in the current multi-cloud environment.

[0055] In one implementation, corresponding weights may be assigned to various state features according to the importance of the indicators, and a multi-dimensional state vector may be constructed according to the state features after the weights are assigned.

[0056] 103. Based on the cumulative rewards obtained by evaluating the system performance indicators and the user preference indicators, and the state characteristics of multiple dimensions, a first scheduling strategy corresponding to the task to be processed is constructed. The cumulative reward is the sum of multiple rewards, and each reward is determined based on the parameter value of at least one system performance indicator and the user preference indicator.

[0057] Among them, the first scheduling strategy is a set of rules or guidelines for determining how resources are allocated to tasks, determining task execution time, deployment location, etc. The cumulative reward is the sum of all rewards from the beginning to the end of the learning task during the reinforcement learning process. In an embodiment of the present application, a reinforcement learning algorithm based on multi-objective optimization can be combined with a deep learning algorithm to learn the relationship between the environmental state in a multi-cloud environment, the first scheduling strategy, and the reward for multi-cloud environment feedback, so that the scheduling strategy constructed under different environmental states and applied to the multi-cloud deployment scheduling system can maximize the cumulative reward.

[0058] Due to the complexity of multi-cloud deployment scheduling scenarios, in order to adapt to complex multi-cloud deployment scheduling scenarios, this embodiment considers optimization goals at different levels, namely, system performance level and business goal level, during the reinforcement learning process of multi-cloud deployment scheduling, so that the constructed scheduling strategy is more adaptable to complex scenarios and more in line with user needs. Deep reinforcement learning provides a powerful tool for multi-cloud deployment scheduling, which can handle complex decision-making problems and achieve optimization in dynamic, multi-objective and uncertain environments.

[0059] It should be noted that in the reinforcement learning process of multi-cloud deployment scheduling in the embodiment of the present application, the optimization goal at the business level at least includes a user preference index. For example, the user preference index can be the matching degree between the first scheduling strategy and the scheduling strategy preferred by the user, and the higher the matching degree, the better. Specifically, the user's past choices and operation records can be used to deeply analyze historical patterns and trends, and a model can be established to predict their possible behavioral preferences, thereby outputting accurate user preference information as the scheduling strategy preferred by the user.

[0060] In specific implementation, user behavior logs can be collected. The behavior logs may include the user's interaction history with the multi-cloud deployment scheduling system, click behavior (such as choosing cloud computing resources from a specific cloud vendor for certain tasks), browsing history, purchase records (such as purchase records of cloud computing resources from different cloud vendors), etc. Then, the collected data is preprocessed (such as cleaning data, processing missing values, outliers, and noise), useful features are selected and extracted, and appropriate algorithms are selected to model user preferences, such as deep learning models. If there is labeled data, supervised learning methods, such as classification or regression, can be used to predict user preferences; if there is no labeled data, unsupervised learning methods, such as cluster analysis, can be used to discover the potential structure of user preferences.

[0061] In addition, in the reinforcement learning process of multi-cloud deployment scheduling in the embodiments of the present application, the optimization goals at the system performance level may include deployment speed (the faster the deployment speed, the better), business interruption cost (the lower the interruption cost, the better), user waiting time (the shorter the waiting time, the better), etc.

[0062] In specific implementation, a machine learning model can be used to process the extracted state features, thereby obtaining high-dimensional features for constructing the first scheduling strategy. That is, in one embodiment, based on the cumulative rewards obtained by evaluating the system performance indicators and the user preference indicators, and the state features of multiple dimensions, constructing the first scheduling strategy corresponding to the task to be processed can include the following process:

[0063] Fusing state features of multiple dimensions to obtain fused state features;

[0064] The task scheduling features are extracted from the fused state features using the trained task scheduling model, which is trained by maximizing the cumulative reward.

[0065] Based on the task scheduling characteristics, a first scheduling strategy corresponding to the task to be processed is generated.

[0066] Specifically, the fused state feature can be represented by a vector. For example, vector representations can be constructed for state features of multiple dimensions respectively, and the vector representations of multiple dimensions can be merged to form a multi-dimensional state vector to obtain the fused state feature. Then, the trained task scheduling model is used to extract features from the multi-dimensional state vector, and based on the extracted task scheduling features, a first scheduling strategy corresponding to the task to be processed is generated.

[0067] It should be noted that the task scheduling model in the embodiment of the present application is obtained by maximizing the cumulative reward training. The task scheduling model may include one or more convolution layers, and the task scheduling feature is obtained by reorganizing the fusion state feature through the one or more convolution layers.

[0068] In practical applications, a multi-objective optimized deep reinforcement learning algorithm can be used to pre-build a task scheduling model. That is, in one embodiment, before responding to a scheduling request for a task to be processed, the following process can also be included:

[0069] Build deep neural network models;

[0070] Deploy the deep neural network model in a simulation environment, and obtain experience data generated when the deep neural network interacts with the simulation environment, wherein the simulation environment is used to simulate a multi-cloud environment, and the experience data at least includes a reward for each interaction result, and each reward is determined based on a parameter value on at least one system performance indicator and a user preference indicator;

[0071] The deep neural network model is trained through reinforcement learning based on empirical data to obtain a task scheduling model.

[0072] Specifically, a deep Q-network (DQN) can be used in combination with a multi-objective optimization algorithm (such as a genetic algorithm, a particle swarm algorithm, a simulated annealing algorithm, an ant colony algorithm, etc.) to build a task scheduling model. The simulation environment can simulate the dynamic allocation of cloud resources, task scheduling, user interaction, and application deployment in a multi-cloud environment.

[0073] In the specific implementation, it is necessary to create a deep neural network model as a Q network to approximate the mapping from the state (i.e. the state of the simulated environment in the simulation environment) to the action value (i.e. the reward for each interaction result). Then, the state information of the simulated environment in the current simulation environment is used as the current state, and a scheduling strategy is selected from the current state to execute, and the next state and reward obtained in response to the action (i.e. the scheduling strategy) are obtained from the simulation environment. At the same time, an experience playback buffer is set to store the state, action, reward, new state (i.e. the next state) and other experience data during the interaction between the intelligent agent (i.e. the multi-cloud deployment scheduling system) and the simulation environment.

[0074] The reward is a feedback signal for the scheduling effect of the scheduling strategy, and the reward is determined based on the parameter value of at least one system performance indicator and a user preference indicator. For example, a compound reward can be set in combination with multiple optimization goals such as deployment speed, service interruption cost, user waiting time, and user preference matching. Furthermore, different weights can be set for different optimization goals to meet business needs in different scenarios.

[0075] refer to Figure 3 , the state information of the simulation environment in the current simulation environment is taken as the current state s(t), and is input to the agent. The agent outputs a scheduling strategy based on the current state s(t), as an action a(t) in response to the current state s(t), and outputs the action a(t) to the simulation environment; the simulation environment feeds back a reward r(t) to the agent based on the received action a(t) and generates the next state s(t+1). In this way, the obtained state s(t), action a(t), reward r(t) and state s(t+1) are stored in the experience playback buffer to construct a training set.

[0076] Then, a batch of samples are randomly selected from the training set for network training, and the Q network is updated using the Bellman equation. The above steps are repeated until a certain termination condition (such as reaching the maximum number of iterations or Q value convergence) is reached to stop training, so that the predicted Q value is close to the true Q value. At the same time, the trained network weights are regularly copied to the target network to stabilize the learning process.

[0077] In order to deal with multi-objective optimization, the Pareto-frontier Evolution Strategy can be introduced on the basis of the DQN algorithm. Specifically, first, by initializing the strategy group, a set of initial strategy parameters is generated, and each set of parameters represents a possible decision-making strategy. Then, each set of strategy parameters is applied to DQN, and the performance of the strategy, that is, its score in the multi-objective space, is obtained through simulation or actual operation. Then, non-dominated strategies are retained according to the Pareto dominance relationship to form a new strategy group, and new strategy parameters are generated through crossover and mutation operations to explore better strategies. Repeat the evaluation and evolution operations until a certain number of generations is reached or the strategy convergence is found, then the evolution is stopped, thereby obtaining the optimal strategy that balances multiple optimization objectives.

[0078] 104. Deploy and allocate resources for the tasks to be processed based on the first scheduling strategy.

[0079] Specifically, the first scheduling strategy output by the task scheduling model based on the current multi-cloud environment can be used to schedule the tasks to be processed, determine the deployment location of the tasks to be processed (such as which cloud node to deploy to), the execution order (such as where to arrange them in the task queue), the deployment time (that is, determine the start time of the task), resource allocation (such as which type of computing resources to allocate to execute the task), etc.

[0080] From the above, we can see that by responding to the scheduling request of the task to be processed, the state features of multiple dimensions are extracted from the environmental state information of the current multi-cloud environment, and then the first scheduling strategy is constructed based on the cumulative rewards obtained by evaluating the system performance indicators and user preference indicators, as well as the state features of multiple dimensions, and the task to be processed is deployed and resources are allocated based on the first scheduling strategy. This solution combines deep learning and reinforcement learning to construct the first scheduling strategy, so that the scheduling system can achieve self-learning and adapt to complex and changing environments, and improve the intelligence of multi-cloud deployment scheduling; in addition, the system performance indicators and user preference matching are used as the optimization goals of reinforcement learning, which improves the overall system performance.

[0081] In practical applications, in order to enhance the adaptability of the task scheduling model to the multi-cloud deployment system and improve system performance, the scheduling effect of the task scheduling model in the multi-cloud environment can be evaluated as a basis for model adjustment. That is, in one embodiment, after the tasks to be processed are deployed and resources are allocated based on the first scheduling strategy, the following process can also be included:

[0082] Execute pending tasks according to the deployment results and resource allocation results;

[0083] During the execution of the tasks to be processed, all tasks and resources in the multi-cloud environment are monitored to obtain monitoring data;

[0084] Performing statistical analysis on the monitoring data based on multiple scheduling effect evaluation indicators to obtain statistical analysis results corresponding to each scheduling effect evaluation indicator, wherein the scheduling effect evaluation indicator at least includes a system performance indicator;

[0085] A first feedback signal is determined according to the statistical analysis result, and a model parameter of the task scheduling model is updated based on the first feedback signal.

[0086] Specifically, in the process of applying the task scheduling model to perform scheduling in a multi-cloud deployment system, the monitoring data can be statistically analyzed regularly based on the scheduling effect evaluation indicators of multiple dimensions to obtain the statistical analysis results corresponding to each scheduling effect evaluation indicator. The first feedback signal is determined according to the statistical analysis results, and the model parameters of the task scheduling model are updated based on the first feedback signal.

[0087] It should be noted that the scheduling effect evaluation indicators in the embodiment of the present application include at least system performance indicators, such as user waiting time, resource utilization and business continuity, and the model parameters of the task scheduling model are adjusted to achieve a balance among multiple evaluation indicators such as user waiting time, resource utilization and business continuity.

[0088] In addition, during the execution of pending tasks, all tasks and resources in the multi-cloud environment can be monitored, and dynamic resource scheduling can be actively triggered to further improve resource scheduling efficiency and system response speed. For example, when it is detected that the execution speed of pending tasks is slow and there are other idle computing nodes, the computing resources of the idle nodes can be called and the pending tasks can be processed in parallel with the allocated computing resources.

[0089] In practical applications, in order to avoid frequent evaluation of model scheduling effects and unnecessary waste of resources, relevant restrictions can be placed on the timing of evaluating model scheduling effects. That is, in one embodiment, before determining the first feedback signal based on the statistical analysis results, key events in the multi-cloud environment can also be detected, and when a key event trigger is detected, an operation of determining the first feedback signal based on the statistical analysis results is performed.

[0090] Among them, key events are events that may have a significant impact on the operation, performance or business objectives of cloud services, and may include system failures (such as hardware failures, software failures or network failures), high load conditions, security incidents (such as security vulnerabilities, data leaks), capacity changes, business demand changes, cost anomalies, technology updates and other different types of events. In practical applications, the corresponding urgency for different types of events is also different. Therefore, in order to better solve the problems caused by these key events, the convergence speed during model training can be selectively adjusted. That is, in one embodiment, when updating the model parameters of the task scheduling model based on the first feedback signal, the following process can be included:

[0091] Identify event types for critical incidents;

[0092] Get the preset learning rate range corresponding to the event type;

[0093] Adjust the learning rate of the task scheduling model to within the preset learning rate range;

[0094] Based on the adjusted learning rate and the first feedback signal, the weight parameters of each layer of the network in the task scheduling model are updated.

[0095] Specifically, the model convergence speed can be adjusted by adjusting the learning rate of the model. Before that, the learning rate adjustment strategy corresponding to different event types can be pre-set. For example, for event types involving security, systemic failures, etc., which have a higher degree of urgency, model training can be performed at a higher learning rate to speed up the model convergence speed; for event types involving capacity, load, etc., which have a lower degree of urgency, model training can be performed at a lower learning rate to make the trained model more stable.

[0096] During specific implementation, the preset learning rate interval required for different event types can be pre-set. Then, by analyzing the event content of the key event itself, it can be classified into the corresponding event type, and the learning rate of the task scheduling model can be adjusted to the preset learning rate interval corresponding to the event type. In this embodiment, the learning rate can be set between 0.0001 and 0.001, and a more fine-grained preset learning rate interval can be set for different event types. For example, for event types involving security, systemic failures, etc., the preset learning rate interval can be set between 0.0005 and 0.001; for event types involving capacity, load, etc., the preset learning rate interval can be set between 0.0001 and 0.0005. Finally, based on the adjusted learning rate and the first feedback signal, the weight parameters of each layer of the network in the task scheduling model are updated, thereby realizing the control of the convergence speed during model training under different types of events.

[0097] In practical applications, in order to further improve the performance of the task scheduling model in the system, the currently focused optimization target can be dynamically determined according to the environmental variables. That is, in one embodiment, after scheduling the pending tasks based on the first scheduling strategy, the following process can also be included:

[0098] Detect environmental variables in multi-cloud environments;

[0099] Determine the importance level of each system performance indicator and user preference indicator based on environmental variables;

[0100] Adjust the weight of each system performance indicator and user preference indicator in the reward according to the importance level;

[0101] Retrain the task scheduling model based on the rewards with adjusted proportions.

[0102] Specifically, an environmental perception module can be integrated into the system to monitor external environmental factors such as cloud resource status, user behavior patterns (such as user operating habits, behavioral preferences, etc.), and system load changes in real time. By introducing environmental variables, the optimization goals can be dynamically adjusted based on the current environment, and the model can be retrained based on the shift in the focus of the optimization goals, so that the model can adapt to complex and changing environments. For example, when resources are tight, the goal to be optimized is resource scheduling efficiency, and its importance level is increased; during user active periods, the goal to be optimized is response speed, and its importance level is increased. Based on the importance level of each optimization goal (that is, each system performance indicator and user preference indicator), the proportion of each optimization goal in the reward structure is adjusted, and the task scheduling model is retrained based on the rewards after the adjustment, so that the constructed scheduling strategy is more in line with actual needs.

[0103] In specific implementation, when determining the importance level of each optimization target corresponding to the task scheduling model based on environmental variables, the environmental variables can be matched with each system performance indicator and user preference indicator to obtain matching results, and the importance level of each system performance indicator and user preference indicator can be determined based on the matching results and the change value of the environmental variables.

[0104] Different importance levels correspond to different weights. The higher the importance level, the greater the corresponding weight, and vice versa. By dynamically adjusting the importance level of the optimization target, it is possible to effectively adapt to changes in short-term business needs or long-term strategic planning.

[0105] In practical applications, in order to further meet user needs, business goals can also be introduced as a reference for adjusting model parameters. That is, in one embodiment, before adjusting the model parameters of the task scheduling model according to the evaluation results, user feedback information can also be obtained and content analysis can be performed to obtain content analysis results. Then, a second feedback signal is determined based on the content analysis results, and the model parameters of the task scheduling model are updated according to the second feedback signal.

[0106] In specific implementation, user feedback information can be obtained from system log information, which may include information such as delay perception and satisfaction survey. When collecting user feedback information, content analysis can be performed using corresponding processing methods based on the form of the feedback information. For example, if the feedback information is in text form, natural language processing technology can be used to perform content analysis; if the feedback information is in chart form, statistical analysis, cluster analysis, and other methods can be used for processing.

[0107] According to the method described in the above embodiment, the following examples will be given for further detailed description. In another embodiment of the present application, a multi-cloud deployment scheduling method based on deep reinforcement learning for multi-objective optimization is provided. Figure 4 and Figure 5 , the multi-cloud deployment scheduling provided by this embodiment is described in detail. The specific process is as follows:

[0108] (1) Environmental modeling and data integration

[0109] First, a simulated multi-cloud environment is built to integrate resource status, task queues, user attributes, historical preferences, and application link information in real time to provide basic data input for reinforcement learning.

[0110] (1.1) Environmental modeling stage

[0111] Specifically, it is necessary to build models of multiple dimensions: abstract model of multi-cloud environment, resource status model, task queue model, user attribute model, user preference model and application link information model. The specific description is as follows:

[0112] Abstract model of multi-cloud environment: It should accurately reflect the resource types (such as computing instances, storage services, database services, etc.), resource status (usage rate, availability), cost model and network topology of different cloud platforms;

[0113] Resource status model: defines status variables for each resource, such as CPU usage, memory usage, network bandwidth usage, etc.

[0114] Task queue model: Design a dynamic task queue system that can manage pending tasks based on their priority, resource requirements, and expected execution time;

[0115] User attribute model: includes user identity authentication information, permission level, service quality requirements, etc.

[0116] User preference model: Use the user's past choices and operation records to build a model to predict their possible behavior preferences;

[0117] Application link information model: maps the relationship between applications and their dependent services, including data flows, API calls, etc.

[0118] In the specific implementation, it is necessary to develop a multi-cloud environment simulator that can simulate scenarios such as dynamic allocation of cloud resources, task scheduling, user interaction, and application deployment based on the above model. The simulator needs to support parameterized configuration to adapt to different experimental settings and research needs.

[0119] (1.2) Data integration stage

[0120] refer to Figure 4 , a resource monitoring tool can be integrated or developed to capture the resource usage of each cloud platform in a multi-cloud environment in real time, and obtain task queue status, user activity logs and application link information through the API interface provided by the cloud service provider. In addition, it is necessary to design an algorithm to analyze user behavior logs and extract and update user preference models.

[0121] After that, the data collected from different sources will be standardized to ensure that data from different cloud platforms and different formats can be unified and integrated. In the cleaning process, invalid, duplicate or erroneous data will be removed to ensure data quality. Figure 4 , using technologies such as Apache Kafka and Flink to build a data stream processing platform to achieve real-time data reception, processing and fusion, and build a data lake (such as Hadoop HDFS, Amazon S3, Google BigQuery) to store integrated data and provide a unified access interface for subsequent analysis and learning.

[0122] (2) Construction of multi-objective state space

[0123] In this embodiment, the state space design includes resource utilization, task urgency, user importance, production environment identification, link location, etc., providing rich dimensions for multi-objective optimization. The details are as follows:

[0124] Resource usage: The current resource usage of each cloud node (such as servers and storage devices), including CPU utilization, memory usage, disk space usage, and network bandwidth usage. In actual applications, a percentage value can be set for each resource type, such as [0,100], where 0% means completely idle and 100% means completely saturated.

[0125] Task urgency: The urgency of a task is measured based on the difference between the task deadline and the current time. In practical applications, it can be expressed by combining the remaining time with the preset importance of the task, such as [0,1], where 0 indicates a lower urgency and 1 indicates a higher urgency.

[0126] User importance: reflects the user's contribution to the system or the requirements of its service level agreement. In practical applications, it can be comprehensively evaluated based on factors such as the user's historical consumption and contract level. The same range of [0,1] applies. The larger the value, the more important the user.

[0127] Production environment flag: This flag indicates whether the task is running in a production environment, which affects the risk tolerance of scheduling decisions. In practical applications, a binary variable can be used, where 0 represents a non-production environment (such as a development environment, a test environment, or a pre-production environment), and 1 represents a production environment.

[0128] Link position: The position of the application in the call link. In actual applications, the number of dependent applications can be used to represent the size of the position indicator.

[0129] In specific implementation, the above multiple dimensions are combined into a state vector to represent the environmental state at each decision. Assuming that the system has N cloud nodes and M tasks to be scheduled, a state vector S can be expressed as:

[0130] S={U1,U2,...,UN,E1,E2,...,EM,I1,I2,...,IK,P1,P2,...,PM,L1,L2,...,LM}

[0131] Among them, Ui represents the resource utilization vector (CPU, memory, disk, network) of the i-th cloud node; Ej represents the urgency of the j-th task; Ik represents the importance of the k-th user; Pj represents the binary identifier of whether the j-th task is in the production environment; Lj represents the link location indicator related to the j-th task.

[0132] (3) Action space definition

[0133] In this embodiment, the action design may include deployment location, batch size, and priority, providing executable options for multi-objective strategies. The details are as follows:

[0134] Candidate queue position adjustment: allows the position of tasks to be moved in the task candidate queue to change the order in which they are scheduled. In specific implementation, it can be defined as a relative change in the position within the queue, for example, [-2, -1, 0, +1, +2], which respectively means moving the task forward two positions, forward one position, unchanged, backward one position, and backward two positions in the queue.

[0135] Deployment location selection: Decide which cloud node to deploy the task on, considering the cost, performance, and service quality of different cloud platforms. In specific implementation, a discrete set of integers can be used to represent all possible deployment locations, such as [1,2,...,N], where N is the total number of cloud nodes.

[0136] Batch size for each deployment: Controls the number of tasks deployed simultaneously in each scheduling operation to balance response speed and scheduling efficiency. In specific implementation, an integer set can be defined, such as [1,2,...,MaxBatch], where MaxBatch represents the maximum batch size allowed by the system.

[0137] Priority setting: Assign execution priorities to tasks to reflect their urgency or contribution to system goals. In practice, priorities can be expressed in integer or floating point intervals, such as [1,2,3] for low, medium, and high priority, or [0.0,1.0], where values ​​closer to 1.0 indicate higher priority.

[0138] (4) Multi-objective reward mechanism

[0139] Specifically, compound rewards are designed to achieve a balance of multiple objectives by combining deployment speed, business interruption cost, reduction of user waiting time, preference matching, etc. Specifically, rewards can be divided into basic rewards and additional rewards. Basic rewards may include deployment speed, business interruption cost, and user waiting time; additional rewards may include preference matching. The details are as follows:

[0140] Deployment speed: Encourage quick completion of task deployment and reduce deployment cycle time. Specifically, the deployment speed can be characterized based on the negative value of the difference between the actual deployment time of the task and the preset ideal deployment time. The calculation formula is as follows:

[0141] r_{speed}=-|T_{actual}-T_{ideal}|

[0142] Among them, T_{actual} represents the actual deployment time, and T_{ideal} represents the expected minimum deployment time.

[0143] Business interruption costs: Reduce the cost of service interruptions caused by deployment or migration, especially the impact on the production environment. Specifically, if an interruption occurs, a negative reward will be given based on the interruption time, the number of affected users, and the importance of the business. The calculation formula is as follows:

[0144] r_{disruption}=-w\times D\times U\times I

[0145] Where w represents the weight factor, D represents the interruption duration, U represents the number of affected users, and I represents the service importance coefficient.

[0146] User waiting time: Encourage to reduce the waiting time of tasks in the queue and improve response speed. Specifically, the user waiting event can be characterized based on the reduction of the actual waiting time of the task and the average waiting time. The calculation formula is as follows:

[0147] r_{wait}=-(W_{actual}-W_{avg})

[0148] Among them, W_{actual}Wactual represents the actual waiting time of the task, and W_{avg}Wavg represents the average waiting time of the task.

[0149] Preference matching: Resource allocation based on user or business preferences improves user satisfaction. Specifically, positive rewards can be given based on the degree of matching between task deployment and user preferences. For example, cosine similarity is used to measure the similarity between the preference vector and the actual deployment configuration. The calculation formula is as follows:

[0150] r_{preference}=cos(θ)

[0151] Among them, θ represents the angle between two vectors, and its value range is between [0,1]. The larger the value, the higher the matching degree.

[0152] (5) Reinforcement Learning Model

[0153] In this embodiment, a deep Q network is used in combination with a multi-objective optimization algorithm to perform reinforcement learning training of the model. The specific process is as follows:

[0154] First, create a deep neural network as the Q network to approximate the mapping from state to action value, and set up an experience replay buffer to store experiences such as state, action, reward, and new state. Starting from the current state of the environment, select an action to execute according to the ε-greedy strategy, and observe the next state and the reward obtained. Then, store the interaction data between the agent and the environment (current state, selected action, reward obtained, new state) in the experience replay buffer. Next, randomly select a batch of samples from the experience replay for network training. Finally, use the Bellman equation to update the Q network, with the goal of making the predicted Q value close to the true Q value. The formula is as follows:

[0155] Q(s,a)=r+γ*max Q(s',a')

[0156] Among them, r is the immediate reward obtained when taking action a; γ is the discount factor, which is used to adjust the current value of future rewards, usually between 0 and 1; max Q(s′,a′) means that among all the actions that may be taken in the next state s′, the Q value of the action that maximizes the Q value is selected, which represents the expected value of future rewards.

[0157] The weights of the trained network are periodically copied to the target network to stabilize the learning process.

[0158] In order to handle multi-objective optimization, the Pareto-frontier exploration algorithm can be introduced based on the DQN algorithm. The specific optimization steps are as follows:

[0159] a. Initialize the strategy population: Generate an initial set of strategy parameters, each set of parameters represents a possible decision strategy.

[0160] b. Evaluation strategy: Apply each set of policy parameters to DQN and obtain the performance of the policy through simulation or actual operation, that is, its score in the multi-objective space.

[0161] c. Selection and reproduction: According to the Pareto dominance relationship, retain the non-dominated strategies to form a new strategy group. Generate new strategy parameters through crossover and mutation operations to explore a better strategy space.

[0162] d. Repeated evaluation and evolution: Repeat steps b and c until a predetermined stopping condition is reached, such as reaching a certain number of generations or finding that the strategy has converged.

[0163] (6) Batch strategy execution and feedback

[0164] In this embodiment, scheduling is performed based on the learning strategy, and feedback data is collected in real time, including user waiting, resource usage, business continuity, etc.

[0165] In the previous step, a multi-objective optimized task scheduling model has been obtained through DQN combined with Pareto-frontier exploration algorithm training. At this point, the trained model can be deployed to the scheduling system of the production environment as the core of scheduling decision-making. When a task request is received, the deployment location, time, resource allocation, etc. of each task are determined according to the task scheduling model. By implementing batch scheduling, AI, big data and other tasks can be efficiently scheduled.

[0166] During system operation, a comprehensive monitoring system needs to be deployed, including but not limited to resource monitoring (CPU, memory, network bandwidth, etc.), task execution status monitoring, system stability indicators, etc. At the same time, log information is collected, including task start time, completion time, intermediate state changes, error information, etc., as well as user feedback (such as delay perception, satisfaction survey, etc.). In addition, business continuity indicators such as fault switching time and service recovery speed need to be monitored to ensure that the user experience is not affected during resource scheduling.

[0167] After that, the collected data is preprocessed, such as cleaning, removing outliers, and standardizing the data format. Then, the preprocessed data is analyzed from multiple dimensions, as follows:

[0168] User waiting time analysis to identify whether scheduling strategies cause unnecessary delays;

[0169] Resource usage analysis to evaluate whether the scheduling strategy achieves efficient resource utilization and avoids resource waste or over-allocation;

[0170] Business continuity assessment, based on indicators such as failure frequency and recovery speed, determines the impact of strategies on business stability.

[0171] The analysis results are converted into quantitative indicators and input into the task scheduling model as feedback signals. According to the actual operation results, the model parameters are adjusted, such as adjusting the reward function weight in DQN to reflect the importance of multiple goals such as business continuity and resource efficiency.

[0172] Retrain or fine-tune the task scheduling model based on new feedback data, and continuously iterate and optimize the scheduling strategy. Implement A / B testing to compare the effects of new and old strategies to ensure that the improvements have indeed improved overall performance. Deploy the optimized strategy back to the production environment and continue to monitor its performance. Establish a feedback loop to ensure that the strategy can continue to learn and adapt to environmental changes, such as newly added cloud resources, changing user needs, etc.

[0173] (7) Dynamic model optimization

[0174] In this embodiment, the learning model parameters can be dynamically adjusted based on feedback to optimize the strategy, adapt to environmental changes, and improve scheduling efficiency and user experience. The details are as follows:

[0175] Specifically, the performance of the current scheduling strategy can be evaluated regularly or when key events are triggered, including resource utilization, task response time, business continuity indicators, etc. Analyze user feedback, especially feedback on scheduling delays and task execution success rates. According to the evaluation results and business goals, adjust the hyperparameters of the reinforcement learning model, such as the learning rate, the weight of the reward function, etc. Use online learning or small-scale offline retraining to retrain the model based on the latest data and adjusted parameters to ensure continuous evolution of the strategy.

[0176] In addition, the environment perception module can be integrated to monitor external environmental factors such as cloud resource status, user behavior patterns, and system load changes in real time. By introducing environmental variables into the model input, the strategy can be dynamically adjusted based on the current environment, such as focusing on resource efficiency when resources are tight and emphasizing response speed during user active periods. In the multi-objective optimization framework, combined with the Pareto Frontier exploration algorithm, find the best balance between different objectives (such as resource efficiency, user experience, and cost control). Dynamically adjust the target weights to adapt to changes in short-term business needs or long-term strategic planning.

[0177] From the above, it can be seen that in the implementation of this application, multi-objective parallel optimization can be performed, and multi-objective reinforcement learning can be innovatively adopted. By combining deep learning with reinforcement learning, the system can self-learn and adapt to complex and changing environments, balance deployment efficiency, user experience and resource allocation, enhance the adaptability and intelligence of the system, improve the overall system performance, and avoid the limitations of single-objective scheduling. In addition, this application also takes into account user and business needs. Through priority business refinement, users are subdivided according to production environment and business urgency to ensure that key users and tasks are prioritized and scheduled, which is more in line with actual business needs. Compared with general allocation, the targeted scheduling strategy of this application is more accurate and can more specifically meet the needs of different users, reduce resource waste, and improve user satisfaction.

[0178] In addition, this solution can dynamically adjust the batch size and order according to real-time data to minimize user waiting time. Compared with fixed static rule strategies, it can more flexibly respond to external changes, improve resource utilization efficiency and response speed, and reduce waiting time. In this application, historical deployment habits are also taken into account, and user preferences are introduced when generating scheduling strategies, so that scheduling is closer to user expectations and increases user stickiness. In addition, the call link can also be analyzed in this application, taking into account the position of the application in the business link, and avoiding congestion by optimizing the deployment order, thereby improving the smoothness and response speed of the system operation.

[0179] In order to better implement the above method, an embodiment of the present application also provides a multi-cloud deployment scheduling device, which can be integrated in an electronic device.

[0180] For example, Figure 6 As shown, the multi-cloud deployment scheduling device may include: a first acquisition unit 301, a feature extraction unit 302, a first construction unit 303 and a scheduling unit 304, as follows:

[0181] A first acquisition unit 301 is used to acquire environmental status information in a current multi-cloud environment in response to a scheduling request for a task to be processed;

[0182] A feature extraction unit 302 is used to extract features from the environmental state information to obtain state features of multiple dimensions;

[0183] A first construction unit 303 is used to construct a first scheduling strategy corresponding to the task to be processed based on the cumulative reward obtained by evaluating the system performance index and the user preference index, and the state characteristics of the multiple dimensions, where the cumulative reward is the sum of multiple rewards, and each reward is determined based on a parameter value on at least one system performance index and a user preference index;

[0184] The scheduling unit 304 is used to deploy and allocate resources for the task to be processed based on the first scheduling strategy.

[0185] Optionally, in some implementations, when constructing the first scheduling strategy corresponding to the task to be processed based on the cumulative reward obtained by evaluating the system performance index and the user preference index, and the state characteristics of multiple dimensions, the first construction unit 303 is specifically used to:

[0186] The state features of the multiple dimensions are fused to obtain a fused state feature; a task scheduling feature is extracted from the fused state feature using a trained task scheduling model, where the task scheduling model is obtained by training to maximize the cumulative reward; based on the task scheduling feature, a first scheduling strategy corresponding to the task to be processed is generated.

[0187] Optionally, in some embodiments, the device may further include:

[0188] A processing unit, configured to, after deploying and allocating resources to the task to be processed based on the first scheduling strategy, execute the task to be processed according to the deployment result and the resource allocation result;

[0189] A monitoring unit, used to monitor all tasks and resources in the multi-cloud environment during the execution of the task to be processed, and obtain monitoring data;

[0190] A first analysis unit is used to perform statistical analysis on the monitoring data based on a plurality of scheduling effect evaluation indicators to obtain a statistical analysis result corresponding to each scheduling effect evaluation indicator, wherein the scheduling effect evaluation indicator at least includes a system performance indicator;

[0191] The second updating unit is used to determine a first feedback signal according to the statistical analysis result, and update the model parameters of the task scheduling model based on the first feedback signal.

[0192] Optionally, in some embodiments, the device may further include:

[0193] A first detection unit, configured to detect a key event in the multi-cloud environment before determining a first feedback signal according to the statistical analysis result;

[0194] The updating unit is used to determine a first feedback signal according to the statistical analysis result when a key event trigger is detected.

[0195] Optionally, in some implementations, when updating the model parameters of the task scheduling model based on the first feedback signal, the updating unit is specifically configured to:

[0196] Determine the event type of the key event; obtain a preset learning rate range corresponding to the event type; adjust the learning rate of the task scheduling model to within the preset learning rate range; and update the weight parameters of each layer of the network in the task scheduling model based on the adjusted learning rate and the first feedback signal.

[0197] Optionally, in some embodiments, the device may further include:

[0198] A second detection unit is used to detect environmental variables in the multi-cloud environment after scheduling the task to be processed based on the first scheduling strategy;

[0199] A determination unit, configured to determine the importance level of each system performance indicator and the user preference indicator based on the environmental variable;

[0200] An adjustment unit, used for adjusting the proportion of each system performance indicator and the user preference indicator in the reward according to the importance level;

[0201] The first training unit is used to retrain the task scheduling model based on the reward after adjusting the proportion.

[0202] Optionally, in some implementations, when determining the importance level of each system performance indicator and the user preference indicator based on the environmental variable, the determining unit is specifically configured to:

[0203] The environment variable is matched with each system performance indicator and the user preference indicator to obtain a matching result; and the importance level of each system performance indicator and the user preference indicator is determined according to the matching result and the change value of the environment variable.

[0204] Optionally, in some embodiments, the device may further include:

[0205] The second analysis unit is used to obtain user feedback information and perform content analysis to obtain content analysis results;

[0206] The second updating unit is used to determine a second feedback signal according to the content analysis result, and update the model parameters of the task scheduling model according to the second feedback signal.

[0207] Optionally, in some embodiments, the device may further include:

[0208] A second construction unit is used to construct a deep neural network model before responding to a processing request for a task to be processed;

[0209] A second acquisition unit is used to deploy the deep neural network model in a simulation environment, and acquire experience data generated when the deep neural network interacts with the simulation environment, wherein the simulation environment is used to simulate a multi-cloud environment, and the experience data at least includes a reward for each interaction result, and each reward is determined based on a parameter value on at least one system performance indicator and a user preference indicator;

[0210] The second training unit is used to perform reinforcement learning training on the deep neural network model according to the empirical data to obtain the task scheduling model.

[0211] Optionally, in some embodiments, the environment status information includes: the resource usage status of each cloud node, the user priority of each user using the cloud computing service, and task attribute information of each task to be processed, and the task attribute information includes at least: a position indicator in the associated call link.

[0212] As can be seen from the above, the multi-cloud deployment scheduling device provided in the embodiment of the present application responds to the scheduling request of the task to be processed, extracts the state characteristics of multiple dimensions from the environmental state information of the current multi-cloud environment, and then constructs a first scheduling strategy based on the cumulative rewards obtained by evaluating the system performance indicators and user preference indicators, as well as the state characteristics of multiple dimensions, and deploys and allocates resources for the task to be processed based on the first scheduling strategy. This solution combines deep learning and reinforcement learning to construct the first scheduling strategy, so that the scheduling system can achieve self-learning and adapt to complex and changing environments, and improves the intelligence of multi-cloud deployment scheduling; in addition, the system performance indicators and user preference matching are used as the optimization goals of reinforcement learning, which improves the overall system performance.

[0213] The present application also provides an electronic device, such as Figure 7 As shown, it shows a schematic diagram of the structure of the electronic device involved in the embodiment of the present application, specifically:

[0214] The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will appreciate that Figure 7 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0215] The processor 401 is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 402, and calling data stored in the memory 402, the processor 401 performs various functions of the electronic device and processes data. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and computer programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 401.

[0216] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and scheduling by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, a computer program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0217] The electronic device also includes a power supply 403 for supplying power to each component. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 403 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0218] The electronic device may further include an input unit 404, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0219] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. It should be noted that the electronic device in this embodiment may specifically be a first server, that is, a server where the SSO authentication center is located. Specifically in this embodiment, the processor 401 in the electronic device will load the executable file corresponding to the process of one or more computer programs into the memory 402 according to the following instructions, and the processor 401 will run the computer program stored in the memory 402, thereby realizing various functions, as follows:

[0220] In response to a scheduling request for a pending task, environmental state information in a current multi-cloud environment is obtained; feature extraction is performed on the environmental state information to obtain state features in multiple dimensions; based on a cumulative reward obtained by evaluating system performance indicators and user preference indicators, and the state features in multiple dimensions, a first scheduling strategy corresponding to the pending task is constructed, wherein the cumulative reward is the sum of multiple rewards, and each reward is determined based on a parameter value on at least one system performance indicator and a user preference indicator; and based on the first scheduling strategy, the pending task is deployed and resources are allocated.

[0221] In one embodiment, when constructing a first scheduling strategy corresponding to the task to be processed based on the cumulative reward obtained by evaluating the system performance index and the user preference index, and the state characteristics of multiple dimensions, the processor 401 can be specifically used to:

[0222] The state features of the multiple dimensions are fused to obtain a fused state feature; a task scheduling feature is extracted from the fused state feature using a trained task scheduling model, where the task scheduling model is obtained by training to maximize the cumulative reward; based on the task scheduling feature, a first scheduling strategy corresponding to the task to be processed is generated.

[0223] In one implementation, after deploying and allocating resources to the task to be processed based on the first scheduling strategy, the processor 401 may also be configured to:

[0224] The task to be processed is executed according to the deployment result and the resource allocation result; during the execution of the task to be processed, all tasks and resources in the multi-cloud environment are monitored to obtain monitoring data; based on multiple scheduling effect evaluation indicators, the monitoring data is statistically analyzed to obtain the statistical analysis results corresponding to each scheduling effect evaluation indicator, wherein the scheduling effect evaluation indicator at least includes the system performance indicator; a first feedback signal is determined according to the statistical analysis result, and the model parameters of the task scheduling model are updated based on the first feedback signal.

[0225] In one implementation, before determining the first feedback signal according to the statistical analysis result, the processor 401 may also be configured to:

[0226] Detect key events in a multi-cloud environment; when a key event is detected to be triggered, determine a first feedback signal based on the statistical analysis result.

[0227] In one implementation, when updating the model parameters of the task scheduling model based on the first feedback signal, the processor 401 may be specifically configured to:

[0228] Determine the event type of the key event; obtain a preset learning rate range corresponding to the event type; adjust the learning rate of the task scheduling model to within the preset learning rate range; and update the weight parameters of each layer of the network in the task scheduling model based on the adjusted learning rate and the first feedback signal.

[0229] In one implementation, after scheduling the to-be-processed task based on the first scheduling strategy, the processor 401 may be specifically configured to:

[0230] Detect environmental variables in a multi-cloud environment; determine the importance level of each system performance indicator and the user preference indicator based on the environmental variables; adjust the proportion of each system performance indicator and the user preference indicator in the reward according to the importance level; and retrain the task scheduling model based on the reward after adjusting the proportion.

[0231] In one implementation, when determining the importance level of each system performance indicator and the user preference indicator based on the environmental variable, the processor 401 may be specifically configured to:

[0232] The environment variable is matched with each system performance indicator and the user preference indicator to obtain a matching result; and the importance level of each system performance indicator and the user preference indicator is determined according to the matching result and the change value of the environment variable.

[0233] In one embodiment, before responding to a processing request for a task to be processed, the processor 401 may further be configured to:

[0234] Construct a deep neural network model; deploy the deep neural network model in a simulation environment, and obtain experience data generated when the deep neural network interacts with the simulation environment, wherein the simulation environment is used to simulate a multi-cloud environment, and the experience data at least includes a reward for each interaction result, and each reward is determined based on a parameter value on at least one system performance indicator and a user preference indicator; perform reinforcement learning training on the deep neural network model according to the experience data to obtain the task scheduling model.

[0235] In one embodiment, the environment status information includes: the resource usage status of each cloud node, the user priority of each user using the cloud computing service, and the task attribute information of each task to be processed, and the task attribute information includes at least: the position indicator in the associated call link.

[0236] The specific implementation of the above operations can be found in the previous embodiments and will not be described in detail here.

[0237] From the above, it can be seen that in the embodiment of the present application, the electronic device extracts state features of multiple dimensions from the environmental state information of the current multi-cloud environment by responding to the scheduling request of the task to be processed, and then constructs a first scheduling strategy based on the cumulative rewards obtained by evaluating the system performance indicators and user preference indicators, as well as the state features of multiple dimensions, and deploys and allocates resources for the task to be processed based on the first scheduling strategy. This solution combines deep learning and reinforcement learning to construct the first scheduling strategy, so that the scheduling system can achieve self-learning and adapt to complex and changing environments, and improves the intelligence of multi-cloud deployment scheduling; in addition, the system performance indicators and user preference matching are used as the optimization goals of reinforcement learning, which improves the overall system performance.

[0238] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0239] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any of the multi-cloud deployment scheduling methods provided in the embodiments of the present application. For example, the instructions can execute the following steps:

[0240] In response to a scheduling request for a pending task, environmental status information in a current multi-cloud environment is obtained; feature extraction is performed on the environmental status information to obtain status features in multiple dimensions; based on the cumulative reward obtained by evaluating system performance indicators and user preference indicators, and the status features in multiple dimensions, a first scheduling strategy corresponding to the pending task is constructed, wherein the cumulative reward is the sum of multiple rewards, and each reward is determined based on a parameter value on at least one system performance indicator and a user preference indicator; and the pending task is deployed and resources are allocated based on the first scheduling strategy.

[0241] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0242] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0243] Since the instructions stored in the computer-readable storage medium can execute the steps in any multi-cloud deployment scheduling method provided in the embodiments of the present application, the beneficial effects that can be achieved by any multi-cloud deployment scheduling method provided in the embodiments of the present application can be achieved. For details, please refer to the previous embodiments and will not be repeated here.

[0244] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the methods provided in various optional implementations of the above-mentioned scheduling aspects.

[0245] The above is a detailed introduction to a multi-cloud deployment scheduling method, device, electronic device, computer-readable storage medium and computer program product provided in an embodiment of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A multi-cloud deployment scheduling method, characterized in that: include: In response to a scheduling request for a pending task, obtaining environmental status information in a current multi-cloud environment; Extracting features from the environmental status information to obtain status features in multiple dimensions; Based on the cumulative reward obtained by evaluating the system performance index and the user preference index, and the state characteristics of the multiple dimensions, constructing a first scheduling strategy corresponding to the task to be processed, wherein the cumulative reward is the sum of multiple rewards, and each reward is determined based on a parameter value of at least one system performance index and a user preference index; The tasks to be processed are deployed and resources are allocated based on the first scheduling strategy.

2. The multi-cloud deployment scheduling method according to claim 1, characterized in that: The first scheduling strategy corresponding to the task to be processed is constructed based on the cumulative rewards obtained by evaluating the system performance index and the user preference index, and the state characteristics of multiple dimensions, including: Fusing the state features of the multiple dimensions to obtain a fused state feature; Using a trained task scheduling model to extract task scheduling features from the fusion state features, the task scheduling model is obtained by maximizing the cumulative reward training; Based on the task scheduling characteristics, a first scheduling strategy corresponding to the task to be processed is generated.

3. The multi-cloud deployment scheduling method according to claim 2, characterized in that: After the to-be-processed tasks are deployed and resources are allocated based on the first scheduling strategy, the method further includes: Execute the pending tasks according to the deployment results and resource allocation results; During the execution of the task to be processed, all tasks and resources in the multi-cloud environment are monitored to obtain monitoring data; Performing statistical analysis on the monitoring data based on a plurality of scheduling effect evaluation indicators to obtain statistical analysis results corresponding to each scheduling effect evaluation indicator, wherein the scheduling effect evaluation indicator at least includes the system performance indicator; A first feedback signal is determined according to the statistical analysis result, and a model parameter of the task scheduling model is updated based on the first feedback signal.

4. The multi-cloud deployment scheduling method according to claim 3, characterized in that: Before determining the first feedback signal according to the statistical analysis result, the method further includes: Detect critical events in multi-cloud environments; When a key event trigger is detected, a first feedback signal is determined according to the statistical analysis result.

5. The multi-cloud deployment scheduling method according to claim 4, characterized in that: The updating of the model parameters of the task scheduling model based on the first feedback signal includes: determining an event type of the critical event; Obtaining a preset learning rate interval corresponding to the event type; Adjusting the learning rate of the task scheduling model to within the preset learning rate range; Based on the adjusted learning rate and the first feedback signal, the weight parameters of each layer of the network in the task scheduling model are updated.

6. The multi-cloud deployment scheduling method according to claim 2, characterized in that: After scheduling the to-be-processed tasks based on the first scheduling strategy, the method further includes: Detect environmental variables in multi-cloud environments; Based on the environmental variables, determining the importance level of each of the system performance indicators and the user preference indicators; According to the importance level, adjusting the proportion of each of the system performance indicators and the user preference indicators in the reward; Based on the weighted rewards, the task scheduling model is retrained.

7. The multi-cloud deployment scheduling method according to claim 6, characterized in that: The determining, based on the environmental variables, the importance level of each of the system performance indicators and the user preference indicators comprises: Matching the environmental variable with each of the system performance indicators and the user preference indicators to obtain a matching result; The importance level of each of the system performance indicators and the user preference indicators is determined according to the matching result and the change value of the environmental variable.

8. The multi-cloud deployment scheduling method according to claim 2, characterized in that: After the tasks to be processed are deployed and resources are allocated based on the first scheduling strategy, the method further includes: Obtain user feedback information and conduct content analysis to obtain content analysis results; A second feedback signal is determined according to the content analysis result, and a model parameter of the task scheduling model is updated according to the second feedback signal.

9. The multi-cloud deployment scheduling method according to claim 2, characterized in that: Before responding to the processing request for the pending task, it also includes: Build deep neural network models; Deploy the deep neural network model in a simulation environment, and obtain experience data generated when the deep neural network interacts with the simulation environment, wherein the simulation environment is used to simulate a multi-cloud environment, and the experience data at least includes a reward for each interaction result, and each reward is determined based on a parameter value on at least one system performance indicator and a user preference indicator; Reinforcement learning training is performed on the deep neural network model according to the empirical data to obtain the task scheduling model.

10. The multi-cloud deployment scheduling method according to any one of claims 1 to 9, characterized in that: The environment status information includes: resource usage status of each cloud node, user priority of each user using cloud computing services, and task attribute information of each task to be processed, and the task attribute information includes at least: a position indicator in an associated call link.

11. A multi-cloud deployment scheduling device, characterized in that: include: A first acquisition unit, configured to acquire environmental status information in a current multi-cloud environment in response to a scheduling request for a task to be processed; A feature extraction unit, used to extract features from the environmental state information to obtain state features of multiple dimensions; A first construction unit is configured to construct a first scheduling strategy corresponding to the task to be processed based on a cumulative reward obtained by evaluating the system performance index and the user preference index, and the state characteristics of the multiple dimensions, wherein the cumulative reward is the sum of multiple rewards, and each reward is determined based on a parameter value of at least one system performance index and a user preference index; A scheduling unit is used to deploy and allocate resources for the tasks to be processed based on the first scheduling strategy.

12. An electronic device, characterized in that: It includes a processor and a memory, the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the steps in the multi-cloud deployment scheduling method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the steps in the multi-cloud deployment scheduling method according to any one of claims 1 to 10.

14. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps in the multi-cloud deployment scheduling method according to any one of claims 1 to 10 are implemented.

Images