A Resource Intelligent Scheduling Method Based on Multi-Objective Reinforcement Learning

Abstract

This invention discloses a resource intelligent scheduling method based on multi-objective reinforcement learning. The method comprises the following steps: Step S1: Obtaining a resource scheduling scenario and extracting several objectives to be optimized based on the different scenarios; Step S2: Constructing several optimization spheres based on the different objectives, evaluating the priority of each objective, and dividing the optimization spheres into fixed spheres and moving spheres according to their priority; Step S3: Evaluating the correlation between the objectives and mapping the fixed and moving spheres to the same multi-dimensional space based on the correlation; Step S4: Adjusting the parameters of the moving spheres using reinforcement learning and causing the moving and fixed spheres to intersect. The intersection of the fixed and moving spheres is output as a resource scheduling strategy. By modeling the objectives to be optimized to generate spheres and introducing reinforcement learning to drive the intersection of the spheres, the intersection region can be obtained quickly and accurately and used as the optimal resource solution set to guide the scheduling of real-world resources.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a resource intelligent scheduling method based on multi-objective reinforcement learning. Background Technology

[0002] Multi-Objective Reinforcement Learning (MORL), as an important branch of reinforcement learning, aims to solve the problem of collaborative optimization of multiple conflicting objectives in complex systems. For example, in resource scheduling scenarios, it can simultaneously achieve efficiency improvement, cost reduction, and fairness assurance. Traditional single-objective reinforcement learning optimizes a single indicator through a scalar reward function, while MORL, by processing vector rewards or constructing Pareto optimal solution sets, can more closely reflect the decision-making needs of multi-objective trade-offs in the real world. However, existing MORL methods have significant shortcomings in handling dynamic relationships between objectives and geometric modeling: on the one hand, scalar-based methods rely on manually preset objective weights, making it difficult to adapt to environmental changes and dynamic adjustments in objective priorities; on the other hand, Pareto front-based methods have high computational complexity and lack intuitive geometric space mapping, making it difficult to explicitly characterize the conflicting relationships between objectives during policy optimization, thus limiting their application in high-dimensional complex resource scheduling scenarios.

[0003] In existing technologies, the combination of multi-objective optimization and resource scheduling mainly faces the problems of abstract target modeling and lack of dynamic interaction mechanisms. For example, when dealing with scenarios such as computing resource scheduling and logistics vehicle scheduling, the optimization space of each objective is usually regarded as an independent mathematical function or discrete constraint condition, lacking systematic modeling of the geometric relationship between objectives. When the number of objectives increases or environmental parameters change dynamically, traditional methods are unable to efficiently capture the nonlinear dependence and conflict boundary between objectives, causing the generated scheduling strategy to either fall into local optima or experience strategy oscillation in multi-objective trade-offs. Summary of the Invention

[0004] In view of this, the present invention proposes a resource intelligent scheduling method based on multi-objective reinforcement learning. By constructing multiple optimization spheres and combining them with multi-objective reinforcement learning, the optimal resource scheduling strategy can be obtained.

[0005] The technical solution of this invention is implemented as follows:

[0006] A resource intelligent scheduling method based on multi-objective reinforcement learning includes the following steps:

[0007] Step S1: Obtain resource scheduling scenarios and extract several optimization objectives based on different resource scheduling scenarios;

[0008] Step S2: Construct several optimization spheres based on different optimization objectives, evaluate the priority of the optimization objectives, and divide the optimization spheres into fixed spheres and moving spheres according to their priorities.

[0009] Step S3: Evaluate the correlation between the objectives to be optimized, and map the fixed sphere and the moving sphere into the same multidimensional space based on the correlation;

[0010] Step S4: Use reinforcement learning to adjust the parameters of the moving sphere and make the moving sphere intersect with the fixed sphere. Output the part where the fixed sphere and the moving sphere intersect as the resource scheduling strategy.

[0011] Preferably, the specific steps of step S1 are as follows:

[0012] Step S11: Collect commonly used user interaction scenarios and determine resource scheduling scenarios from these interaction scenarios;

[0013] Step S12: Input the resource scheduling scenario into the trained neural network for processing;

[0014] Step S13: The neural network processes the data to obtain several targets to be optimized.

[0015] Preferably, the training process of the neural network is as follows: collect several historical resource scheduling scenarios and corresponding optimization targets, divide the historical resource scheduling scenarios and corresponding optimization targets into training sets and test sets, use the training set to train the neural network, use the test set to test the neural network after training to a certain stage, and stop training when the test accuracy is greater than a preset threshold.

[0016] Preferably, the specific steps of step S2, which involves constructing several optimization spheres based on different optimization objectives, are as follows:

[0017] Step S21: Obtain the ideal optimal value and tolerance value for each target to be optimized, and normalize the ideal optimal value and tolerance value.

[0018] Step S22: Using the ideal optimal value as the center of the sphere and the difference between the tolerance value and the ideal optimal value as the radius, construct an optimization sphere for each objective to be optimized.

[0019] Preferably, the specific steps of step S2, which evaluates the priority of the target to be optimized and divides the spheres into fixed spheres and moving spheres according to their priorities, are as follows:

[0020] Step S23: Use expert scoring to score the priority of the optimization targets and obtain the priority ranking;

[0021] Step S24: Output the highest priority optimized sphere as a fixed sphere, and output the other optimized spheres as moving spheres.

[0022] Preferably, the specific steps of step S3 are as follows:

[0023] Step S31: Calculate the correlation coefficient between each pair of objectives to be optimized using the Pearson correlation coefficient method;

[0024] Step S32: Determine whether the fixed sphere and each moving sphere are positively or negatively correlated based on the correlation coefficient;

[0025] Step S33: Map the fixed sphere into the multidimensional space and fix its position; then map the moving sphere to the vicinity of the fixed sphere.

[0026] Step S34: Adjust the distance between the moving sphere and the fixed sphere based on the positive and negative correlation, and keep the moving sphere and the fixed sphere from intersecting.

[0027] Preferably, the adjustment principle of step S34 is as follows: map the moving sphere that is positively correlated with the fixed sphere to a position adjacent to the fixed sphere, and map the moving sphere that is negatively correlated with the fixed sphere to a position far away from the fixed sphere.

[0028] Preferably, step S34 is adjusted by the Euclidean distance between the moving sphere and the fixed sphere, wherein the stronger the positive correlation, the closer the Euclidean distance between the moving sphere and the fixed sphere, and the stronger the negative correlation, the farther the Euclidean distance between the moving sphere and the fixed sphere.

[0029] Preferably, the specific steps of step S4 are as follows:

[0030] Step S41: Initialize the reinforcement learning environment, define the state space as a combination of parameters of the moving sphere and the fixed sphere, and the action space as the center coordinate offset of the moving sphere and the radius scaling factor.

[0031] Step S42: Using the minimum distance between the moving ball and the fixed ball as the core indicator, a positive reward is given when the distance is less than the sum of the radii of the moving ball and the fixed ball, and the closer the distance, the higher the reward; otherwise, a negative penalty is imposed.

[0032] Step S43: Construct an agent model with state as input and action as output, start the training loop, and gradually optimize to maximize the cumulative reward;

[0033] Step S44: Calculate the Euclidean distance between the moving sphere and the fixed sphere. When the intersection condition is met, output the parameters of the intersection part as the resource scheduling strategy.

[0034] Preferably, step S4 further includes:

[0035] Step S45: Apply the resource scheduling strategy to the resource scheduling scenario to verify the effect, and adjust the reward based on the feedback.

[0036] Compared with the prior art, the beneficial effects of the present invention are:

[0037] The present invention provides a resource intelligent scheduling method based on multi-objective reinforcement learning, which can be applied to different types of resource scheduling scenarios. Based on the differences of each resource scheduling scenario, several objectives to be optimized can be extracted. Then, the objectives to be optimized are modeled as spheres in multi-dimensional space. The optimal solutions for different objectives to be optimized are determined by the intersection regions between the spheres. These optimal solutions are then fed back to the real resource scheduling scenario as resource scheduling strategies to guide the actual use of resources and achieve the goal of global optimization.

[0038] After constructing the optimized sphere, it needs to be processed in two steps. The first step is to divide the optimized sphere into a fixed sphere and a moving sphere according to the priority of the target to be optimized. The higher priority sphere is the fixed sphere, which is stationary in the multidimensional space. Then, the distance between the moving sphere and the fixed sphere is adjusted based on the correlation, and reinforcement learning is introduced to adjust the parameters of the moving sphere to drive the generation of intersecting regions. This effectively solves the shortcomings of the current MORL method in target association modeling and dynamic optimization efficiency, and provides a more robust solution for complex resource scheduling problems. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only preferred embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart of a resource intelligent scheduling method based on multi-objective reinforcement learning according to the present invention;

[0041] Figure 2 This is a flowchart of step S1 of a resource intelligent scheduling method based on multi-objective reinforcement learning according to the present invention;

[0042] Figure 3 This is a flowchart of step S2 of a resource intelligent scheduling method based on multi-objective reinforcement learning according to the present invention;

[0043] Figure 4 This is a flowchart of step S3 of a resource intelligent scheduling method based on multi-objective reinforcement learning according to the present invention;

[0044] Figure 5This is a flowchart of step S4 of a resource intelligent scheduling method based on multi-objective reinforcement learning according to the present invention;

[0045] Figure 6 This is a schematic diagram of a fixed sphere and a moving sphere in a resource intelligent scheduling method based on multi-objective reinforcement learning according to the present invention. Detailed Implementation

[0046] To better understand the technical content of this invention, a specific embodiment is provided below, and the invention will be further described in conjunction with the accompanying drawings.

[0047] See Figures 1 to 6 This invention provides a resource intelligent scheduling method based on multi-objective reinforcement learning, comprising the following steps:

[0048] Step S1: Obtain resource scheduling scenarios and extract several optimization objectives based on different resource scheduling scenarios;

[0049] Step S2: Construct several optimization spheres based on different optimization objectives, evaluate the priority of the optimization objectives, and divide the optimization spheres into fixed spheres and moving spheres according to their priorities.

[0050] Step S3: Evaluate the correlation between the objectives to be optimized, and map the fixed sphere and the moving sphere into the same multidimensional space based on the correlation;

[0051] Step S4: Use reinforcement learning to adjust the parameters of the moving sphere and make the moving sphere intersect with the fixed sphere. Output the part where the fixed sphere and the moving sphere intersect as the resource scheduling strategy.

[0052] This invention discloses a resource scheduling method based on multi-objective reinforcement learning, used to schedule resources used in human-computer interaction processes, such as in cloud computing and data centers, intelligent transportation, and energy management. In cloud computing and data centers, it can schedule server computing and storage resources, taking into account network latency, task processing speed, energy costs, and service quality, achieving efficient resource allocation, reducing data center operating costs, and improving resource utilization efficiency. In intelligent transportation, it can be used in urban traffic signal control and vehicle scheduling, simultaneously optimizing traffic efficiency, reducing congestion duration, and lowering carbon emissions. In energy management, it can be applied to microgrids or smart grids, coordinating the generation, storage, and consumption demands of renewable energy sources such as solar and wind power, balancing goals such as generation efficiency, electricity costs, and power supply stability, and optimizing energy resource allocation. In addition to improving energy efficiency and grid stability, the resource scheduling scenarios mentioned above also include industrial manufacturing and logistics. These scenarios are essentially the resource scheduling scenarios mentioned in step S1. Different resource scheduling scenarios require different optimization objectives, which means that several different optimization objectives can be extracted. For example, in the fields of cloud computing and data centers, optimization objectives include network latency, task processing speed, energy consumption cost, and service quality. By simulating different optimization objectives, virtual optimization spheres are formed. Then, reinforcement learning is introduced to drive the adjustment of the parameters of different optimization spheres so that the optimization spheres intersect. The intersection area is the optimal solution set of several optimization objectives, which represents the optimal resource scheduling strategy. After outputting the resource scheduling strategy, it can guide actual production work and resource allocation, ensuring that the resource scheduling scenario is in an optimal state.

[0053] To accurately determine the intersection regions between different optimization spheres, the priorities of the optimization spheres were first evaluated. Based on their priority, the optimization spheres were divided into fixed spheres and moving spheres. Fixed spheres remain stationary, while moving spheres can move and change. Each optimization sphere represents the scheduling range of the optimization objective. Therefore, optimization spheres with higher priority have higher constraints and often need to satisfy the parameter range of the fixed spheres. Thus, after designating higher-priority optimization spheres as fixed spheres, they are kept stationary to ensure that the requirements of the optimization objective for the fixed spheres can be met during subsequent optimization processes. For example, in the field of intelligent transportation, traffic efficiency takes precedence over carbon emissions, requiring minimizing carbon emissions while ensuring normal vehicle traffic. After determining the fixed and moving spheres, further evaluation is needed... The correlation between various optimization objectives is analyzed, and then, based on the different correlations, fixed and moving spheres are mapped to the same multi-dimensional space. The relative positions between the fixed and moving spheres are adjusted, and finally, reinforcement learning is introduced. After adjusting the parameters of the moving sphere, it can be driven to move to intersect with the fixed sphere. After the reinforcement learning process is completed, the part where the fixed and moving spheres intersect is output as the resource scheduling strategy, realizing the simultaneous optimization of multiple objectives. By introducing the optimization sphere, the process can be visualized, allowing schedulers to view the entire resource scheduling process. On the one hand, the optimization process can be manually verified, and on the other hand, staff can have a comprehensive understanding of the entire resource scheduling process. Finally, the resource scheduling strategy can be fed back into the resource scheduling scenario for substantive resource scheduling, ensuring that the various optimization objectives in the resource scheduling scenario can meet the actual needs.

[0054] Preferably, the specific steps of step S1 are as follows:

[0055] Step S11: Collect commonly used user interaction scenarios and determine resource scheduling scenarios from these interaction scenarios;

[0056] Step S12: Input the resource scheduling scenario into the trained neural network for processing;

[0057] Step S13: The neural network processes the data to obtain several targets to be optimized.

[0058] There are various types of resource scheduling scenarios. Therefore, it is necessary to first identify the type of resource scheduling scenario. By collecting user interaction scenarios through user's daily work and travel data, for example, when the interaction scenario is vehicle navigation, the resource scheduling scenario can be identified as intelligent transportation resource scheduling scenario. For example, when the user's daily work involves optimizing front-end and back-end servers, the resource scheduling scenario can be identified as cloud computing and data center scheduling. After obtaining the corresponding resource scheduling scenario, it is necessary to extract the optimization target from the resource scheduling scenario. This invention uses a deep learning algorithm to identify the resource scheduling scenario through a pre-trained neural network, thereby identifying the corresponding optimization target. The identified optimization target is a commonly used optimization target in this field. The optimization target can be output to the user for confirmation. If the user has special requirements, the corresponding optimization target can also be added to meet the user's own needs.

[0059] Preferably, the training process of the neural network is as follows: collect several historical resource scheduling scenarios and corresponding optimization targets, divide the historical resource scheduling scenarios and corresponding optimization targets into training sets and test sets, use the training set to train the neural network, use the test set to test the neural network after training to a certain stage, and stop training when the test accuracy is greater than a preset threshold.

[0060] For neural networks, training is performed by collecting a large amount of historical data, including historical resource scheduling scenarios and corresponding optimization targets. These two types of data are combined into a set, resulting in a large dataset. This dataset is then divided into a training set and a test set in a 7:3 ratio. The training set is used to train the constructed neural network. After training reaches a certain stage, the test set can be used for testing. If the test accuracy is reached, training can be stopped. If the test accuracy is not reached, the dataset can be readjusted and training can be repeated until the test accuracy is greater than a preset threshold.

[0061] By introducing deep learning algorithms, the target to be optimized can be automatically obtained. The target to be optimized is a target that is commonly used in the corresponding resource scheduling scenario and has been used for resource scheduling, ensuring that the target to be optimized will inevitably be used in the scenario.

[0062] Preferably, the specific steps of step S2, which involves constructing several optimization spheres based on different optimization objectives, are as follows:

[0063] Step S21: Obtain the ideal optimal value and tolerance value for each target to be optimized, and normalize the ideal optimal value and tolerance value.

[0064] Step S22: Using the ideal optimal value as the center of the sphere and the difference between the tolerance value and the ideal optimal value as the radius, construct an optimization sphere for each objective to be optimized.

[0065] After obtaining the target to be optimized, an optimization sphere needs to be constructed. The main parameters of the optimization sphere are its center and radius. For a sphere, its center represents the ideal state, and the distance from all points on the sphere's surface to the center is the same. Therefore, the ideal optimal value of the target to be optimized can be used as the center of the sphere. The radius between the sphere's surface and the sphere is represented by a tolerance value, which refers to the maximum acceptable deviation from the ideal optimal value. For example, in the fields of cloud computing and data centers, network latency is one of the targets to be optimized. To ensure normal data transmission, the network latency requirement is generally ≤10ms. Therefore, 10ms can be used as the ideal optimal value. In resource scheduling and utilization, there may be situations where latency is sacrificed to reduce energy consumption or improve work efficiency. Therefore, network latency has a tolerance value, with a maximum tolerable latency of 12ms. Thus, the difference between the tolerance value and the ideal optimal value can be used as the radius to construct an optimization sphere. Each optimization target can have its own optimization sphere constructed. The center of the optimization sphere represents different ideal optimal values, and the radius of the sphere also corresponds to different tolerance values. In order to ensure the normal intersection between subsequent optimization spheres, when constructing the optimization sphere, it is also necessary to normalize the ideal optimal value and tolerance value of each optimization target to ensure the uniformity of the size between the optimization spheres.

[0066] Preferably, the specific steps of step S2, which evaluates the priority of the target to be optimized and divides the spheres into fixed spheres and moving spheres according to their priorities, are as follows:

[0067] Step S23: Use expert scoring to score the priority of the optimization targets and obtain the priority ranking;

[0068] Step S24: Output the highest priority optimized sphere as a fixed sphere, and output the other optimized spheres as moving spheres.

[0069] After constructing the optimized spheres, it is necessary to classify them. Currently, optimized spheres are mainly divided into fixed spheres and moving spheres, based on priority. Priority is evaluated using a quantitative method. This invention employs an expert scoring method, scoring the priority of each target to be optimized to obtain a priority ranking. Since the number of targets to be optimized in general resource scheduling scenarios is not excessive, this invention only outputs the highest-priority optimized sphere as a fixed sphere, and the remaining optimized spheres are directly output as moving spheres. This classification is based on commonly used resource scheduling principles in current resource scheduling scenarios, i.e., determined according to objective realities. The priority of the targets to be optimized can also be subjectively selected. After evaluating the priority of the targets to be optimized, the priority results are output to the user, who can then judge whether they meet their requirements. If the user has a high priority for a certain target to be optimized, they can change the priority ranking to determine the highest-priority target to be optimized. In this case, the optimized sphere for the corresponding target to be optimized can be output as a fixed sphere, satisfying the user's subjective needs.

[0070] Preferably, the specific steps of step S3 are as follows:

[0071] Step S31: Calculate the correlation coefficient between each pair of objectives to be optimized using the Pearson correlation coefficient method;

[0072] Step S32: Determine whether the fixed sphere and each moving sphere are positively or negatively correlated based on the correlation coefficient;

[0073] Step S33: Map the fixed sphere into the multidimensional space and fix its position; then map the moving sphere to the vicinity of the fixed sphere.

[0074] Step S34: Adjust the distance between the moving sphere and the fixed sphere based on the positive and negative correlation, and keep the moving sphere and the fixed sphere from intersecting.

[0075] After determining the fixed and moving spheres, they need to be mapped to the same multidimensional space. However, the positions of the fixed and moving spheres are not arbitrarily mapped; they must be determined based on their correlation with the fixed sphere. This invention uses the Pearson correlation coefficient method to calculate the correlation coefficient between each optimization objective. Then, based on the different correlation coefficients, it can be determined whether each moving sphere is positively or negatively correlated with the fixed sphere. When mapping to the same multidimensional space, the distance between the moving and fixed spheres is adjusted according to the different positive and negative correlations, while maintaining the non-intersection of the fixed and moving spheres. This indicates that multiple optimization objectives cannot be simultaneously satisfied initially, facilitating the introduction of reinforcement learning methods to change parameters and gradually induce intersection, thereby obtaining resource scheduling strategies that satisfy different optimization objectives.

[0076] Preferably, the adjustment principle of step S34 is as follows: map the moving sphere that is positively correlated with the fixed sphere to a position adjacent to the fixed sphere, and map the moving sphere that is negatively correlated with the fixed sphere to a position far away from the fixed sphere, and adjust it by the Euclidean distance between the moving sphere and the fixed sphere, wherein the stronger the positive correlation, the closer the Euclidean distance between the moving sphere and the fixed sphere, and the stronger the negative correlation, the farther the Euclidean distance between the moving sphere and the fixed sphere.

[0077] When mapping fixed and moving spheres into a multidimensional space, the fixed sphere is first mapped to the center of the multidimensional space, and then the moving spheres are distributed around the fixed sphere. Moving spheres that are positively correlated with the fixed sphere will move closer to the fixed sphere, while moving spheres that are negatively correlated with the fixed sphere will move further away from the fixed sphere. The correlation coefficient between each moving sphere and the fixed sphere can be calculated using the Pearson correlation coefficient method. When there is a positive correlation, the correlation coefficient is positive; when there is a negative correlation, it is negative. The larger the positive correlation value, the closer the moving sphere will be to the fixed sphere; the larger the absolute value of the negative correlation value, the further the moving sphere will be from the fixed sphere. Based on different positive and negative correlation values, the distance of the moving spheres around the fixed sphere can be adjusted after calculating the Euclidean distance between the moving spheres and the fixed sphere, thereby forming different combinations of spheres in the multidimensional space.

[0078] In addition to correlation, the present invention can also adjust the distance by referring to the priority between the targets to be optimized. For a moving sphere with a higher priority, the distance between it and the fixed sphere can be further shortened. Regardless of the priority or correlation, it is necessary to ensure that the moving sphere and the fixed sphere do not intersect.

[0079] Preferably, the specific steps of step S4 are as follows:

[0080] Step S41: Initialize the reinforcement learning environment, define the state space as a combination of parameters of the moving sphere and the fixed sphere, and the action space as the center coordinate offset of the moving sphere and the radius scaling factor.

[0081] Step S42: Using the minimum distance between the moving ball and the fixed ball as the core indicator, a positive reward is given when the distance is less than the sum of the radii of the moving ball and the fixed ball, and the closer the distance, the higher the reward; otherwise, a negative penalty is imposed.

[0082] Step S43: Construct an agent model with state as input and action as output, start the training loop, and gradually optimize to maximize the cumulative reward;

[0083] Step S44: Calculate the Euclidean distance between the moving sphere and the fixed sphere. When the intersection condition is met, output the parameters of the intersection part as the resource scheduling strategy.

[0084] Step S45: Apply the resource scheduling strategy to the resource scheduling scenario to verify the effect, and adjust the reward based on the feedback.

[0085] After obtaining a multidimensional space containing a fixed sphere and a moving sphere, this invention introduces reinforcement learning. First, after initializing the reinforcement learning environment, a state space and an action space are designed. The state space is a combination of parameters for the moving and fixed spheres, including the spheres themselves and their radii. The action space consists of the center coordinate offset of the moving sphere and its radius scaling factor. In other words, reinforcement learning drives the movement of the moving sphere by changing its position and size. After determining the state and action spaces, a reward function needs to be designed, using the minimum distance between the moving and fixed spheres as the core objective. A positive reward is assigned when the calculated distance represents the intersection of the moving and fixed spheres; the closer the distance, the higher the reward. This completes the initialization of reinforcement learning. Then, an agent model is constructed. The agent model uses a deep neural network to fit the mapping relationship from state to action. The input is a vector in the state space, and the output is continuous or discrete adjustment parameters in the action space. The process is then initiated. The training loop iterates through the process, progressively optimizing and maximizing the cumulative reward. During each training iteration, the Euclidean distance between the moving and fixed spheres is calculated to determine if the intersection condition (distance ≤ sum of radii) is met. After training, the parameters of the intersection portion are output as a resource scheduling strategy. This strategy can be fed back into a real-world resource scheduling scenario for effectiveness verification. If the resource scheduling effect is unsatisfactory, the reward function can be adjusted. Finally, the adjusted resource scheduling strategy is applied to the resource scheduling scenario to guide actual resource allocation. By modeling the target to be optimized as an optimization sphere in a multi-dimensional space and dynamically adjusting the intersection region between the moving and fixed spheres, an explicit geometric representation of the feasible solution space between targets is achieved. This effectively addresses the shortcomings of existing MORL methods in target association modeling and dynamic optimization efficiency, providing a more robust solution for complex resource scheduling problems.

[0086] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A resource intelligent scheduling method based on multi-objective reinforcement learning, characterized in that, Includes the following steps: Step S1: Obtain resource scheduling scenarios and extract several optimization objectives based on different resource scheduling scenarios; Step S2: Construct several optimization spheres based on different optimization objectives, evaluate the priority of the optimization objectives, and divide the optimization spheres into fixed spheres and moving spheres according to their priorities. Step S3: Evaluate the correlation between the objectives to be optimized, and map the fixed sphere and the moving sphere into the same multidimensional space based on the correlation; Step S4: Use reinforcement learning to adjust the parameters of the moving sphere and make the moving sphere intersect with the fixed sphere. Output the part where the fixed sphere and the moving sphere intersect as the resource scheduling strategy. The specific steps of step S1 are as follows: Step S11: Collect commonly used user interaction scenarios and determine resource scheduling scenarios from these interaction scenarios; Step S12: Input the resource scheduling scenario into the trained neural network for processing; Step S13: Neural network processing yields several objectives to be optimized; The specific steps of constructing several optimization spheres based on different optimization objectives in step S2 are as follows: Step S21: Obtain the ideal optimal value and tolerance value for each target to be optimized, and normalize the ideal optimal value and tolerance value. Step S22: Using the ideal optimal value as the center of the sphere and the difference between the tolerance value and the ideal optimal value as the radius, construct an optimization sphere for each objective to be optimized; The specific steps of step S2, which evaluates the priority of the target to be optimized and divides the optimization sphere into fixed spheres and moving spheres according to their priorities, are as follows: Step S23: Use expert scoring to score the priority of the optimization targets and obtain the priority ranking; Step S24: Output the highest priority optimized sphere as a fixed sphere, and output the other optimized spheres as moving spheres; The specific steps of step S4 are as follows: Step S41: Initialize the reinforcement learning environment, define the state space as a combination of parameters of the moving sphere and the fixed sphere, and the action space as the center coordinate offset of the moving sphere and the radius scaling factor. Step S42: Using the minimum distance between the moving ball and the fixed ball as the core indicator, a positive reward is given when the distance is less than the sum of the radii of the moving ball and the fixed ball, and the closer the distance, the higher the reward; otherwise, a negative penalty is imposed. Step S43: Construct an agent model with state as input and action as output, start the training loop, and gradually optimize to maximize the cumulative reward; Step S44: Calculate the Euclidean distance between the moving sphere and the fixed sphere. When the intersection condition is met, output the parameters of the intersection part as the resource scheduling strategy.

2. The resource intelligent scheduling method based on multi-objective reinforcement learning according to claim 1, characterized in that, The training process of the neural network is as follows: collect several historical resource scheduling scenarios and corresponding optimization targets, divide the historical resource scheduling scenarios and corresponding optimization targets into training sets and test sets, use the training set to train the neural network, use the test set to test the neural network after training to a certain stage, and stop training when the test accuracy is greater than a preset threshold.

3. The resource intelligent scheduling method based on multi-objective reinforcement learning according to claim 1, characterized in that, The specific steps of step S3 are as follows: Step S31: Calculate the correlation coefficient between each pair of objectives to be optimized using the Pearson correlation coefficient method; Step S32: Determine whether the fixed sphere and each moving sphere are positively or negatively correlated based on the correlation coefficient; Step S33: Map the fixed sphere into the multidimensional space and fix its position; then map the moving sphere to the vicinity of the fixed sphere. Step S34: Adjust the distance between the moving sphere and the fixed sphere based on the positive and negative correlation, and keep the moving sphere and the fixed sphere from intersecting.

4. The resource intelligent scheduling method based on multi-objective reinforcement learning according to claim 3, characterized in that, The adjustment principle of step S34 is as follows: map the moving sphere that is positively correlated with the fixed sphere to a position adjacent to the fixed sphere, and map the moving sphere that is negatively correlated with the fixed sphere to a position far away from the fixed sphere.

5. The resource intelligent scheduling method based on multi-objective reinforcement learning according to claim 4, characterized in that, Step S34 is adjusted by the Euclidean distance between the moving sphere and the fixed sphere. The stronger the positive correlation, the closer the Euclidean distance between the moving sphere and the fixed sphere. The stronger the negative correlation, the farther the Euclidean distance between the moving sphere and the fixed sphere.

6. The resource intelligent scheduling method based on multi-objective reinforcement learning according to claim 1, characterized in that, Step S4 further includes: Step S45: Apply the resource scheduling strategy to the resource scheduling scenario to verify the effect, and adjust the reward based on the feedback.

Images