A computing power scheduling strategy optimization system based on reinforcement learning

Abstract

The present invention provides a computing power scheduling strategy optimization system based on reinforcement learning, which relates to the field of electrical digital data processing, including an environment perception module, a reinforcement decision module, a computing power scheduling module and a feedback reward module. The environment perception module is used to collect perception state information, the reinforcement decision module outputs a scheduling strategy based on reinforcement learning, the computing power scheduling module is used to execute the scheduling strategy to allocate computing power resources, and the feedback reward module feeds back reward information based on the scheduling effect; the system continuously screens and refines the processing scheme through a reinforcement learning model, which can make the output scheduling strategy more and more reasonable and improve the utilization efficiency of computing power resources.

Description

Technical Field

[0001] The present invention relates to the field of electronic digital data processing, and in particular to a computing power scheduling strategy optimization system based on reinforcement learning. Background Art

[0002] With the rapid development of computer technology, distributed computing and high-performance computing have been widely used in large-scale data processing, complex scientific computing and real-time application scenarios. However, with the increasing complexity and diversification of computing tasks, how to efficiently schedule and allocate computing resources in a dynamic environment has become a key issue. Traditional computing scheduling strategies are usually based on static rules or predefined optimization goals, which are difficult to adapt to real-time changes in task and resource status, and easily lead to problems such as low resource utilization and increased task execution delays. At the same time, these traditional methods are difficult to handle complex dependencies or high-dimensional resource management requirements in large-scale systems, and lack intelligent decision-making capabilities.

[0003] Now many computing power scheduling systems have been developed. After a lot of searching and reference, it is found that the existing computing power scheduling systems are such as the system disclosed in the publication number CN118626276B. These system methods generally include: building a target computing power network, receiving target computing tasks, evaluating the required task execution computing power, and comparing it with the single-center computing power; if it is less than or equal to the single-center computing power, selecting an independent computing power proxy center, and processing the target computing task through the single-center independent execution mode; if it is greater than the single-center computing power, selecting an idle computing power proxy center according to the number of computing power proxy centers, forming a temporary collaborative computing unit, and processing the target computing task through the multi-center collaborative execution mode. However, this system always uses a fixed method to schedule computing power, lacks adaptability and cannot evolve algorithms, and the utilization efficiency of computing power resources cannot be further improved. Summary of the invention

[0004] The purpose of the present invention is to propose a computing power scheduling strategy optimization system based on reinforcement learning to address the existing deficiencies.

[0005] The present invention adopts the following technical solution:

[0006] A computing power scheduling strategy optimization system based on reinforcement learning, including an environment perception module, a reinforcement decision module, a computing power scheduling module and a feedback reward module;

[0007] The environmental perception module is used to collect perception state information, the reinforcement decision module outputs a scheduling strategy based on reinforcement learning, the computing power scheduling module is used to execute the scheduling strategy to allocate computing power resources, and the feedback reward module feeds back reward information based on the scheduling effect;

[0008] The environment perception module includes a resource monitoring unit, a task analysis unit and a network communication unit. The resource monitoring unit is used to collect the usage status and load status of computing resources in real time. The task analysis unit is used to analyze the computing power requirements, priority and time limit requirements of each task. The network communication unit is used to monitor the communication delay and bandwidth status between resource nodes.

[0009] The reinforcement decision module includes a state representation unit, an action generation unit and a strategy optimization unit, wherein the state representation unit is used to convert the perception data into the input state of reinforcement learning, the action generation unit generates a scheduling strategy using the reinforcement learning model, and the strategy optimization unit optimizes the model based on the reward information;

[0010] The computing power scheduling module includes a task management unit, a task allocation unit and a task monitoring unit. The task management unit is used to manage task information. The task allocation unit allocates tasks to corresponding nodes based on the scheduling strategy. The task monitoring unit is used to monitor the running information of the tasks.

[0011] The feedback reward module includes an effect tracking unit, a reward calculation unit and a feedback management unit. The effect tracking unit is used to track the effect information of the task when it is completed, the reward calculation unit is used to calculate the reward information of the decision-making plan, and the feedback management unit is used to summarize the reward information and feed it back to the reinforcement decision module.

[0012] Furthermore, the strategy optimization unit includes a probability optimization processor, a solution optimization processor and an optimization control processor. The probability optimization processor adjusts the probability of the processing solution based on reward information, the solution optimization processor is used to develop and refine a processing solution into multiple processing solutions, and the optimization control processor is used to control the optimization process.

[0013] Furthermore, the optimization process of the strategy optimization unit includes the following steps:

[0014] S1, the probability optimization processor receives reward information;

[0015] S2, the optimization control processor optimizes the distribution of the reward information, and when the conditions are met, enters step S3;

[0016] S3, the probability optimization processor optimizes and adjusts the probability of each processing solution, and synchronizes the probability information to the probability selection processor;

[0017] S4, the optimization control processor determines whether there is a processing solution for cutting, and if so, proceeds to step S5;

[0018] S5. Delete the treatment plans that need to be trimmed, select the treatment plan with the highest probability as the target plan, and add the probabilities of the trimmed treatment plans to the probability of the target plan;

[0019] S6. The solution optimization processor subdivides the target solution into n+1 sub-processing solutions, and divides the probability of the target solution into n+1 parts as the probability of each sub-processing solution, where n is the number of the cut processing solutions;

[0020] S7. Synchronize the sub-processing scheme and probability information to the action generation unit.

[0021] Furthermore, the process of optimizing and adjusting the probability by the probability optimization processor includes the following steps:

[0022] S21, calculating the average reward value of each processing solution, denoted as Vr(i), which represents the average reward value of the i-th processing solution;

[0023] S22. Calculate the new probability of each treatment plan according to the following formula:

[0024] ;

[0025] Where P(i) represents the original probability of the i-th treatment plan, P'(i) represents the new probability of the i-th treatment plan, and N is the number of treatment plans. is the reward weight coefficient.

[0026] Further, the reward calculation unit includes a historical information register, an effect comparison processor and a reward calculation processor, wherein the historical information register is used to store the effect information of the task, the effect comparison processor is used to extract the standard from the historical effect information and compare it with the current task effect, and the reward calculation processor calculates the reward value of the processing solution corresponding to the task based on the comparison result;

[0027] The reward calculation processor calculates the reward value Vr according to the following formula:

[0028] ;

[0029] Among them, m is the effect item, k i is the weight value of the i-th effect item, Represents the comparison value of the i-th effect item.

[0030] The beneficial effects achieved by the present invention are:

[0031] This system obtains the system status information, processes the status information using the processing plan, and then outputs the scheduling strategy. Based on the execution effect of the scheduling strategy, the processing plan is screened and refined, so that the processing plan can continuously evolve during the use of the system, thereby improving the utilization efficiency of computing resources in the scheduling strategy.

[0032] To further understand the features and technical contents of the present invention, please refer to the following detailed description and drawings of the present invention. However, the drawings provided are only for reference and description and are not intended to limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a schematic diagram of the overall structural framework of the present invention;

[0034] Figure 2 This is a schematic diagram of the enhanced decision-making module of the present invention;

[0035] Figure 3 This is a schematic diagram of the computing power scheduling module of the present invention;

[0036] Figure 4 This is a schematic diagram of the feedback reward module of the present invention;

[0037] Figure 5 This is a schematic diagram of the strategy optimization unit of the present invention;

[0038] Figure 6 This is a graph showing the variation of the sample test effect of the present invention. DETAILED DESCRIPTION

[0039] The following is an explanation of the embodiments of the present invention through specific embodiments. Those skilled in the art can understand the advantages and effects of the present invention from the contents disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and the details in this specification can also be modified and changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. In addition, the drawings of the present invention are only simple schematic illustrations and are not depicted according to actual sizes. It is stated in advance. The following embodiments will further explain the relevant technical contents of the present invention in detail, but the disclosed contents are not intended to limit the scope of protection of the present invention.

[0040] Embodiment 1: This embodiment provides a computing power scheduling strategy optimization system based on reinforcement learning, combined with Figure 1 , including environment perception module, reinforcement decision module, computing power scheduling module and feedback reward module;

[0041] The environmental perception module is used to collect perception state information, the reinforcement decision module outputs a scheduling strategy based on reinforcement learning, the computing power scheduling module is used to execute the scheduling strategy to allocate computing power resources, and the feedback reward module feeds back reward information based on the scheduling effect;

[0042] The environment perception module includes a resource monitoring unit, a task analysis unit and a network communication unit. The resource monitoring unit is used to collect the usage status and load status of computing resources in real time. The task analysis unit is used to analyze the computing power requirements, priority and time limit requirements of each task. The network communication unit is used to monitor the communication delay and bandwidth status between resource nodes.

[0043] The reinforcement decision module includes a state representation unit, an action generation unit and a strategy optimization unit, wherein the state representation unit is used to convert the perception data into the input state of reinforcement learning, the action generation unit generates a scheduling strategy using the reinforcement learning model, and the strategy optimization unit optimizes the model based on the reward information;

[0044] The computing power scheduling module includes a task management unit, a task allocation unit and a task monitoring unit. The task management unit is used to manage task information. The task allocation unit allocates tasks to corresponding nodes based on the scheduling strategy. The task monitoring unit is used to monitor the running information of the tasks.

[0045] The feedback reward module includes an effect tracking unit, a reward calculation unit and a feedback management unit. The effect tracking unit is used to track the effect information of the task when it is completed, the reward calculation unit is used to calculate the reward information of the decision-making plan, and the feedback management unit is used to summarize the reward information and feed it back to the reinforcement decision module.

[0046] The strategy optimization unit includes a probability optimization processor, a solution optimization processor and an optimization control processor. The probability optimization processor adjusts the probability of the processing solution based on reward information, the solution optimization processor is used to develop and refine a processing solution into multiple processing solutions, and the optimization control processor is used to control the optimization process.

[0047] The optimization process of the strategy optimization unit includes the following steps:

[0048] S1, the probability optimization processor receives reward information;

[0049] S2, the optimization control processor optimizes the distribution of the reward information, and when the conditions are met, enters step S3;

[0050] S3, the probability optimization processor optimizes and adjusts the probability of each processing solution, and synchronizes the probability information to the probability selection processor;

[0051] S4, the optimization control processor determines whether there is a processing solution for cutting, and if so, proceeds to step S5;

[0052] S5. Delete the treatment plans that need to be trimmed, select the treatment plan with the highest probability as the target plan, and add the probabilities of the trimmed treatment plans to the probability of the target plan;

[0053] S6. The solution optimization processor subdivides the target solution into n+1 sub-processing solutions, and divides the probability of the target solution into n+1 parts as the probability of each sub-processing solution, where n is the number of the cut processing solutions;

[0054] S7. Synchronize the sub-processing scheme and probability information to the action generation unit.

[0055] The process of optimizing and adjusting the probability by the probability optimization processor comprises the following steps:

[0056] S21, calculating the average reward value of each processing solution, denoted as Vr(i), which represents the average reward value of the i-th processing solution;

[0057] S22. Calculate the new probability of each treatment plan according to the following formula:

[0058] ;

[0059] Where P(i) represents the original probability of the i-th treatment plan, P'(i) represents the new probability of the i-th treatment plan, and N is the number of treatment plans. is the reward weight coefficient.

[0060] The reward calculation unit includes a historical information register, an effect comparison processor and a reward calculation processor, wherein the historical information register is used to store the effect information of the task, the effect comparison processor is used to extract the standard from the historical effect information and compare it with the current task effect, and the reward calculation processor calculates the reward value of the processing solution corresponding to the task based on the comparison result;

[0061] The reward calculation processor calculates the reward value Vr according to the following formula:

[0062] ;

[0063] Among them, m is the effect item, k i is the weight value of the i-th effect item, Represents the comparison value of the i-th effect item.

[0064] Embodiment 2: This embodiment includes all the contents of Embodiment 1, and provides a computing power scheduling strategy optimization system based on reinforcement learning, including an environment perception module, a reinforcement decision module, a computing power scheduling module and a feedback reward module;

[0065] The environmental perception module is used to collect perception state information, the reinforcement decision module outputs a scheduling strategy based on reinforcement learning, the computing power scheduling module is used to execute the scheduling strategy to allocate computing power resources, and the feedback reward module feeds back reward information based on the scheduling effect;

[0066] The environment perception module includes a resource monitoring unit, a task analysis unit and a network communication unit. The resource monitoring unit is used to collect the usage status and load status of computing resources in real time. The task analysis unit is used to analyze the computing power requirements, priority and time limit requirements of each task. The network communication unit is used to monitor the communication delay and bandwidth status between resource nodes.

[0067] Combination Figure 2 The reinforcement decision module includes a state representation unit, an action generation unit and a strategy optimization unit. The state representation unit is used to convert the perception data into the input state of reinforcement learning. The action generation unit generates a scheduling strategy using the reinforcement learning model. The strategy optimization unit optimizes the model based on the reward information.

[0068] Combination Figure 3 The computing power scheduling module includes a task management unit, a task allocation unit and a task monitoring unit. The task management unit is used to manage task information. The task allocation unit allocates tasks to corresponding nodes based on the scheduling strategy. The task monitoring unit is used to monitor the running information of the tasks.

[0069] Combination Figure 4 The feedback reward module includes an effect tracking unit, a reward calculation unit and a feedback management unit. The effect tracking unit is used to track the effect information of the task when it is completed, the reward calculation unit is used to calculate the reward information of the decision-making plan, and the feedback management unit is used to summarize the reward information and feed it back to the reinforcement decision module;

[0070] The resource monitoring unit includes a computing resource monitoring processor, a storage resource monitoring processor and an energy consumption monitoring processor. The computing resource monitoring processor is used to monitor the usage rate of computing resources and the health status of corresponding hardware. The storage resource monitoring processor is used to monitor the usage status of storage resources. The energy consumption monitoring processor is used to monitor the energy consumption information of nodes when using resources.

[0071] The task analysis unit includes a task classification processor, a demand analysis processor and a priority evaluation processor, wherein the task classification processor is used to classify the resource intensity of the task, the demand analysis processor is used to analyze and obtain the demand for the corresponding resource, and the priority evaluation processor is used to extract the time limit information and evaluate the priority of the task;

[0072] The network communication unit includes a delay monitoring and evaluation processor, a bandwidth monitoring and allocation processor, and a data transmission optimization processor. The delay monitoring and evaluation processor is used to monitor the transmission delay data between two nodes and evaluate the delay. The bandwidth monitoring and allocation processor is used to monitor the bandwidth utilization rate of the nodes and dynamically allocate bandwidth resources. The data transmission optimization processor is used to compress the transmission data to improve the communication efficiency.

[0073] The state representation unit includes a time series encoding processor, a feature extraction processor and a state conversion processor, wherein the time series encoding processor is used to encode and pair the perceived environmental data in time sequence, the feature extraction processor is used to extract feature data from the paired environmental data, and the state conversion processor obtains a state vector based on the feature data conversion;

[0074] The action generation unit includes a processing scheme register, a probability selection processor and an action output processor, wherein the processing scheme register is used to store processing scheme information for the state vector, the probability selection processor is used to store probability information of each processing scheme and select a processing scheme based on the probability information, and the action output processor is used to execute the selected processing scheme to process the state vector and output a scheduling action;

[0075] Combination Figure 5 The strategy optimization unit includes a probability optimization processor, a solution optimization processor and an optimization control processor. The probability optimization processor adjusts the probability of the processing solution based on the reward information. The solution optimization processor is used to develop and refine a processing solution into multiple processing solutions. The optimization control processor is used to control the optimization process.

[0076] The optimization process of the strategy optimization unit includes the following steps:

[0077] S1, the probability optimization processor receives reward information;

[0078] S2, the optimization control processor optimizes the distribution of the reward information, and when the conditions are met, enters step S3;

[0079] S3, the probability optimization processor optimizes and adjusts the probability of each processing solution, and synchronizes the probability information to the probability selection processor;

[0080] S4, the optimization control processor determines whether there is a processing solution for cutting, and if so, proceeds to step S5;

[0081] S5. Delete the treatment plans that need to be trimmed, select the treatment plan with the highest probability as the target plan, and add the probabilities of the trimmed treatment plans to the probability of the target plan;

[0082] S6. The solution optimization processor subdivides the target solution into n+1 sub-processing solutions, and divides the probability of the target solution into n+1 parts as the probability of each sub-processing solution, where n is the number of the cut processing solutions;

[0083] S7, synchronizing the sub-processing scheme and probability information to the action generation unit;

[0084] The process of optimizing and adjusting the probability by the probability optimization processor comprises the following steps:

[0085] S21, calculating the average reward value of each processing solution, denoted as Vr(i), which represents the average reward value of the i-th processing solution;

[0086] S22. Calculate the new probability of each treatment plan according to the following formula:

[0087] ;

[0088] Where P(i) represents the original probability of the i-th treatment plan, P'(i) represents the new probability of the i-th treatment plan, and N is the number of treatment plans. is the reward weight coefficient;

[0089] When the number of received reward information reaches the optimization threshold, and the distribution ratio of the reward information corresponding to the processing scheme is consistent with the probability of the processing scheme, the optimization control processor determines that the condition is met in step S2;

[0090] When the probability of the processing scheme is lower than the cutting threshold, the optimization control processor determines in step S4 that the processing scheme needs to be cut;

[0091] The task management unit includes a task registration processor, a task queue manager and a task status tracker, wherein the task registration processor is used to receive new tasks and register information, the task queue manager is used to sort the tasks and process the tasks according to the queue order, and the task status tracker is used to track and record the status of the tasks;

[0092] The task allocation unit includes an action parsing processor, a task matching processor and a policy execution processor, wherein the action parsing processor is used to parse the scheduling action to obtain the scheduling policy, the task matching processor is used to match the scheduling policy with the corresponding task information, and the policy execution processor is used to execute the scheduling policy and send the matched task information to the corresponding node;

[0093] The task monitoring unit includes an operation monitoring processor and a monitoring feedback processor, wherein the operation monitoring processor is used to monitor the operation progress of each task, and the monitoring feedback processor is used to summarize and record the monitored progress information feedback;

[0094] The effect tracking unit includes a completion activation processor, a data retrieval processor and an effect calculation processor, wherein the completion activation processor is used to activate the effect tracking of the task when the task is completed, the data retrieval processor is used to retrieve the real-time environment information of the task at each progress point, and the effect calculation processor is used to calculate the comprehensive effect of the task;

[0095] The reward calculation unit includes a historical information register, an effect comparison processor and a reward calculation processor, wherein the historical information register is used to store the effect information of the task, the effect comparison processor is used to extract the standard from the historical effect information and compare it with the current task effect, and the reward calculation processor calculates the reward value of the processing solution corresponding to the task based on the comparison result;

[0096] The reward calculation processor calculates the reward value Vr according to the following formula:

[0097] ;

[0098] Among them, m is the effect item, k i is the weight value of the i-th effect item, represents the comparison value of the i-th effect item;

[0099] The feedback management unit includes a feedback statistics processor, a data packaging processor and a feedback transmission processor, wherein the feedback statistics processor is used to count the reward value of the processing scheme, the data packaging processor is used to package the statistical information, and the feedback transmission processor is used to feed back the data packet to the reinforcement decision module;

[0100] The i and j appearing in the above text are ordinal numbers used to indicate serial numbers and have no actual meaning.

[0101] Part of the code of this system is shown below:

[0102] class ResourceMonitor:

[0103] """Resource Monitoring Unit"""

[0104] def __init__(self):

[0105] pass

[0106] def monitor_resources(self):

[0107] # Collect computing resource usage status and load conditions

[0108] return {"cpu_usage": 75, "gpu_usage": 60, "memory_usage": 80}

[0109] class TaskAnalyzer:

[0110] """Task Analysis Unit"""

[0111] def __init__(self):

[0112] pass

[0113] def analyze_task(self, task):

[0114] # Analyze task requirements, priorities, and deadlines

[0115] return {"task_id": task["id"], "priority": task["priority"], "deadline": task["deadline"]}

[0116] class NetworkMonitor:

[0117] """Network Communication Unit"""

[0118] def __init__(self):

[0119] pass

[0120] def monitor_network(self):

[0121] # Monitor communication delay and bandwidth status

[0122] return {"latency": 20, "bandwidth": 100}

[0123] class StateRepresentation:

[0124] """Status display unit"""

[0125] def __init__(self):

[0126] pass

[0127] def represent_state(self, resource_state, task_state, network_state):

[0128] # Convert perception data into reinforcement learning input state

[0129] return {

[0130] "resource": resource_state,

[0131] "task": task_state,

[0132] "network": network_state

[0133] }

[0134] class ActionGenerator:

[0135] """Action Generation Unit"""

[0136] def __init__(self):

[0137] pass

[0138] def generate_action(self, state):

[0139] # Generate scheduling strategy based on reinforcement learning

[0140] return {"task_id": state["task"]["task_id"], "node": "Node_1"}

[0141] class PolicyOptimizer:

[0142] """Strategy Optimization Unit"""

[0143] def __init__(self):

[0144] pass

[0145] def optimize_policy(self, reward):

[0146] # Optimize the model based on reward information

[0147] print(f"Optimizing policy with reward: {reward}")

[0148] class TaskAllocator:

[0149] """Task Allocation Unit"""

[0150] def __init__(self):

[0151] pass

[0152] def allocate_task(self, task, node):

[0153] # Assign tasks to corresponding nodes

[0154] print(f"Allocating task {task['id']} to {node}")

[0155] def calculate_reward(self, task_status):

[0156] # Calculate reward information based on task status

[0157] return task_status["status"] == "completed"

[0158] def feedback_reward(self, reward):

[0159] # Feedback reward information to the reinforcement decision module

[0160] print(f"Feedback reward: {reward}").

[0161] Now we use a test sample task to process it at different learning stages of this system, and test the sample processing completion time, such as Figure 6 shown.

[0162] The contents disclosed above are only preferred feasible embodiments of the present invention, and do not limit the protection scope of the present invention. Therefore, all equivalent technical changes made using the contents of the present invention specification and drawings are included in the protection scope of the present invention. In addition, the elements therein can be updated as technology develops.

Claims

1. A computing power scheduling strategy optimization system based on reinforcement learning, characterized in that: It includes environment perception module, enhanced decision-making module, computing power scheduling module and feedback reward module; The environmental perception module is used to collect perception state information, the reinforcement decision module outputs a scheduling strategy based on reinforcement learning, the computing power scheduling module is used to execute the scheduling strategy to allocate computing power resources, and the feedback reward module feeds back reward information based on the scheduling effect; The environment perception module includes a resource monitoring unit, a task analysis unit and a network communication unit. The resource monitoring unit is used to collect the usage status and load status of computing resources in real time. The task analysis unit is used to analyze the computing power requirements, priority and time limit requirements of each task. The network communication unit is used to monitor the communication delay and bandwidth status between resource nodes. The reinforcement decision module includes a state representation unit, an action generation unit and a strategy optimization unit, wherein the state representation unit is used to convert the perception data into the input state of reinforcement learning, the action generation unit generates a scheduling strategy using the reinforcement learning model, and the strategy optimization unit optimizes the model based on the reward information; The computing power scheduling module includes a task management unit, a task allocation unit and a task monitoring unit. The task management unit is used to manage task information. The task allocation unit allocates tasks to corresponding nodes based on the scheduling strategy. The task monitoring unit is used to monitor the running information of the tasks. The feedback reward module includes an effect tracking unit, a reward calculation unit and a feedback management unit. The effect tracking unit is used to track the effect information of the task when it is completed, the reward calculation unit is used to calculate the reward information of the decision-making plan, and the feedback management unit is used to summarize the reward information and feed it back to the reinforcement decision module; The strategy optimization unit includes a probability optimization processor, a solution optimization processor and an optimization control processor, wherein the probability optimization processor adjusts the probability of the processing solution based on the reward information, the solution optimization processor is used to develop and refine a processing solution into multiple processing solutions, and the optimization control processor is used to control the optimization process; The optimization process of the strategy optimization unit includes the following steps: S1, the probability optimization processor receives reward information; S2, the optimization control processor optimizes the distribution of the reward information, and when the conditions are met, enters step S3; S3, the probability optimization processor optimizes and adjusts the probability of each processing solution, and synchronizes the probability information to the probability selection processor; S4, the optimization control processor determines whether there is a processing solution for cutting, and if so, proceeds to step S5; S5. Delete the treatment plans that need to be trimmed, select the treatment plan with the highest probability as the target plan, and add the probabilities of the trimmed treatment plans to the probability of the target plan; S6. The solution optimization processor subdivides the target solution into n+1 sub-processing solutions, and divides the probability of the target solution into n+1 parts as the probability of each sub-processing solution, where n is the number of the cut processing solutions; S7, synchronizing the sub-processing scheme and probability information to the action generation unit; The process of optimizing and adjusting the probability by the probability optimization processor comprises the following steps: S21, calculating the average reward value of each processing solution, denoted as Vr(i), which represents the average reward value of the i-th processing solution; S22. Calculate the new probability of each treatment plan according to the following formula: ； Where P(i) represents the original probability of the i-th treatment plan, P'(i) represents the new probability of the i-th treatment plan, and N is the number of treatment plans. is the reward weight coefficient; The reward calculation unit includes a historical information register, an effect comparison processor and a reward calculation processor, wherein the historical information register is used to store the effect information of the task, the effect comparison processor is used to extract the standard from the historical effect information and compare it with the current task effect, and the reward calculation processor calculates the reward value of the processing solution corresponding to the task based on the comparison result; The reward calculation processor calculates the reward value Vr according to the following formula: ； Among them, m is the effect item, k i is the weight value of the i-th effect item, Represents the comparison value of the i-th effect item.

Images