Intelligent radiotherapy plan cloud system and implementation method

Abstract

The invention discloses an intelligent radiotherapy plan cloud system and implementation method, and the method comprises the steps: receiving a radiotherapy plan generation task submitted by a user; performing task decomposition on the radiotherapy plan generation task to obtain a series of subtasks, and collecting the current state of each server in the radiotherapy plan cloud; and inputting the sub-tasks decomposed by the radiotherapy plan generation task and the current state of each server in the radiotherapy plan cloud into a deep Q network reinforcement learning model, outputting server resources allocated to each sub-task, and performing multi-task allocation scheduling of the radiotherapy plan generation task. The invention provides an intelligent radiotherapy plan cloud system and an implementation method, solves the problem that server computing resources in a server network architecture in a traditional method cannot be effectively utilized, and establishes an intelligent task scheduling and resource allocation scheme of radiotherapy plan cloud.

Description

Technical Field

[0001] The present invention relates to the technical field of radiotherapy planning, and more particularly to an intelligent radiotherapy planning cloud system and an implementation method thereof. Background Art

[0002] In radiotherapy planning systems, irradiation plans executed on accelerators are designed using high-performance computers. Typically, a user controls a single computer in stand-alone mode to perform planning, or the server is assigned multiple tasks based on task priority. This solution requires the configuration of multiple computers or high-performance servers, which is not conducive to the efficient use of computing resources.

[0003] Therefore, to improve the efficient utilization of server computing resources, a server network architecture for radiotherapy plan generation has been applied. However, existing server network architectures generally allocate corresponding server resources according to the execution of a complete radiotherapy plan generation task. A complete radiotherapy plan generation task generally includes computational subtasks such as input, optimized rapid dose calculation, gradient derivative calculation, step size calculation, optimized machine parameter constraint calculation, objective function value calculation, Monte Carlo high-precision dose calculation, and output. Different computational subtasks require different server computing resources. The server allocation method of the existing server network architecture cannot meet the server computing resource requirements of each computational subtask in the radiotherapy plan generation task, cannot fully utilize the server resources in the server network architecture, and cannot fully utilize the advantages of the server network architecture.

[0004] In view of this, the present invention patent is proposed. Summary of the Invention

[0005] In order to overcome the shortcomings of the existing technical solutions, the purpose of the present invention is to provide an intelligent radiotherapy planning cloud system and implementation method to solve the problem that server computing resources cannot be effectively utilized in the traditional server network architecture, and establish an intelligent task scheduling and resource allocation solution for the radiotherapy planning cloud to facilitate and quickly complete the radiotherapy plan generation task.

[0006] Specifically, the following technical solutions are adopted:

[0007] A method for implementing a smart radiotherapy planning cloud, comprising:

[0008] Receive radiation therapy plan generation tasks submitted by users;

[0009] Decompose the radiotherapy plan generation task into a series of subtasks and collect the current status of each server in the radiotherapy plan cloud;

[0010] The subtasks of the radiotherapy plan generation task and the current status of each server in the radiotherapy plan cloud are input into the deep Q network reinforcement learning model, and the server resources allocated to each subtask are output to perform multi-task allocation and scheduling of the radiotherapy plan generation task.

[0011] As an optional embodiment of the present invention, in a smart radiotherapy planning cloud implementation method of the present invention, the deep Q network reinforcement learning model includes:

[0012] Input and output: A deep Q-network reinforcement learning model is created for each radiotherapy plan generation task. The input is the state vector, and the output is the probability of entering the next subtask of each radiotherapy plan generation task, thereby selecting the action with the highest probability for execution.

[0013] Hidden layer: The hidden layer of the Q network reinforcement learning model is composed of a combination of multiple linear layers and activation functions.

[0014] As an optional embodiment of the present invention, in a smart radiotherapy planning cloud implementation method of the present invention, the training process of the deep Q network reinforcement learning model includes:

[0015] Start multi-task calculation based on the number of simultaneous requirements for the radiotherapy plan generation task and the total server resources in the radiotherapy plan cloud;

[0016] Get the initial state vector s0 = [t1, t2, ... t n , c1, c2, …c n , g1, g2, … g n ] and global time T, where ta represents the progress of the radiotherapy plan generation task a, represented by the current subtask, ca and ga represent the CPU and GPU occupancy rates of the radiotherapy plan generation task a, respectively. T counts the time from the start of the first radiotherapy plan generation task to the end of the last radiotherapy plan generation task, which is used for rewards and penalties.

[0017] Randomly initialize the Q network reinforcement learning model and use the ε-greedy strategy to select actions:

[0018]

[0019] Among them, the initial ε is set to a higher probability, which becomes smaller as the task progresses:

[0020]

[0021] Among them, C and B are fixed parameters, k represents the task, and Ik represents the subtask stage that task k reaches:

[0022] Execute action a, calculate the current CPU and GPU resource occupancy rates, and obtain the next state vector s1;

[0023] Set the scoring function based on T and the number of completed tasks

[0024]

[0025] This gives the reward function

[0026] Among them, α and β represent weights, r(S n ) represents the current state score function, represents the number of remaining subtasks of the remaining radiotherapy plan generation task k, t ki Denotes the execution time required for the i-th subtask of the radiotherapy plan generation task k, and puts (s, a, r, s1) into the experience replay pool;

[0027] Sampling and updating the Q network reinforcement learning model from the experience replay pool, the objective function is

[0028]

[0029] Where γ is the discount factor, θ - represents the model parameters of the target Q network, which remain unchanged until a fixed number of iterations is reached, and then replaced by the parameters θ of the main Q network. The mean square error loss function is used to train the Q network model parameters L(θ) = MSE(yQ(s, a; θ));

[0030] Finally, the training of the Q network reinforcement learning model parameters is completed.

[0031] As an optional embodiment of the present invention, in a smart radiotherapy planning cloud implementation method of the present invention, the radiotherapy plan generation task is decomposed to obtain a series of subtasks including:

[0032] Decompose a complete radiotherapy plan generation task into a series of subtasks based on CPU or GPU task types;

[0033] Subtasks using CPU computing resources include data input and output, step size calculation of the planning optimization algorithm, calculation of the optimization objective function value, and calculation of optimization machine parameter constraint limits;

[0034] Subtasks using GPU computing resources include optimizing the calculation of gradient derivatives, optimizing the fast dose calculation algorithm, and the Monte Carlo high-precision dose calculation algorithm;

[0035] A complete radiotherapy plan generation task includes: input, optimized fast dose calculation, gradient derivative calculation, step size calculation, optimized machine parameter constraint limit calculation, objective function value calculation, Monte Carlo high-precision dose calculation, and output. The subtasks from optimized fast dose calculation to objective function value calculation need to be repeated multiple times.

[0036] As an optional embodiment of the present invention, in a smart radiotherapy planning cloud implementation method of the present invention, the receiving of a radiotherapy plan generation task submitted by a user includes:

[0037] User-defined calculation parameters, including CT data, delineation data, radiotherapy prescription, accelerator machine parameters, plan type, radiation field, and organ constraints;

[0038] Computational tasks submitted by users include plan optimization tasks and dose calculation tasks.

[0039] As an optional embodiment of the present invention, a method for implementing a smart radiotherapy planning cloud of the present invention includes:

[0040] Each server in the radiotherapy plan cloud receives the multi-task allocation scheduling of the radiotherapy plan generation task, performs corresponding subtask calculations, and generates a radiotherapy plan;

[0041] The database server in the radiotherapy planning cloud provides calculation data and saves the calculated results.

[0042] The present invention also provides a smart radiotherapy planning cloud system, including:

[0043] The user-side module receives the radiotherapy plan generation task submitted by the user;

[0044] The task scheduling module decomposes the radiotherapy plan generation task into a series of subtasks and collects the current status of each server in the planning cloud module;

[0045] A reinforcement learning module, including a deep Q-network reinforcement learning model, wherein the task scheduling module inputs the subtasks of the radiotherapy plan generation task decomposition and the current status of each server in the plan cloud module into the deep Q-network reinforcement learning model, and outputs the server resources allocated to each subtask;

[0046] The task scheduling module performs multi-task allocation scheduling of the radiotherapy plan generation task based on the output of the reinforcement learning module.

[0047] As an optional embodiment of the present invention, a smart radiotherapy planning cloud system of the present invention includes a planning cloud module including:

[0048] A computing server receives the multi-task allocation scheduling of the radiotherapy plan generation task, performs corresponding subtask calculations, and generates a radiotherapy plan;

[0049] The database server provides calculation data and saves the calculation results.

[0050] The present invention also provides a computer-readable medium storing a computer-executable program, which, when executed, implements the intelligent radiotherapy planning cloud implementation method.

[0051] The present invention also provides an electronic device comprising a processor and a memory, wherein the memory is used to store a computer executable program. When the computer program is executed by the processor, the processor executes the intelligent radiotherapy planning cloud implementation method.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] A smart radiotherapy planning cloud implementation method of the present invention can receive radiotherapy planning generation tasks submitted by multiple users, decompose the radiotherapy planning generation tasks into specific computing subtasks, and collect the current status of each server in the radiotherapy planning cloud (including the utilization of CPU and GPU). In this way, the deep Q network reinforcement learning model outputs the server resources required for each subtask based on the subtasks and the current status of each server, which can realize the dynamic allocation and scheduling of each server in the radiotherapy planning cloud, and perform multiple computing tasks at the same time, solving the problem that server computing resources cannot be effectively utilized in the traditional network architecture. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A flow chart of a method for implementing a smart radiotherapy planning cloud according to an embodiment of the present invention;

[0055] Figure 2 A modular diagram of a smart radiotherapy planning cloud system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] To make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them.

[0057] Therefore, the following detailed description of the embodiments of the present invention is not intended to limit the scope of the claimed invention, but merely represents some embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention.

[0058] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features and technical solutions therein may be combined with each other.

[0059] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0060] In the description of the present invention, it should be noted that the terms "upper" and "lower" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, or the orientations or positional relationships in which the inventive product is typically placed when in use, or the orientations or positional relationships commonly understood by those skilled in the art. Such terms are intended solely to facilitate the description of the present invention and simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limitations on the present invention. Furthermore, the terms "first" and "second" and the like are used solely for distinction and should not be construed as indicating or implying relative importance.

[0061] See also Figure 1 As shown, a smart radiotherapy planning cloud implementation method of this embodiment includes:

[0062] Receive radiation therapy plan generation tasks submitted by users;

[0063] Decompose the radiotherapy plan generation task into a series of subtasks and collect the current status of each server in the radiotherapy plan cloud;

[0064] The subtasks of the radiotherapy plan generation task and the current status of each server in the radiotherapy plan cloud are input into the deep Q network reinforcement learning model, and the server resources allocated to each subtask are output to perform multi-task allocation and scheduling of the radiotherapy plan generation task.

[0065] A smart radiotherapy planning cloud implementation method in this embodiment can receive radiotherapy plan generation tasks submitted by multiple users, decompose the radiotherapy plan generation tasks into specific computing subtasks, and simultaneously collect the current status of each server in the radiotherapy planning cloud (including CPU and GPU utilization). In this way, the deep Q network reinforcement learning model outputs the server resources required for each subtask based on the subtasks and the current status of each server. This can achieve dynamic allocation and scheduling of each server in the radiotherapy planning cloud, perform multiple computing tasks simultaneously, and solve the problem of ineffective utilization of server computing resources in traditional network architectures.

[0066] Therefore, the intelligent radiotherapy planning cloud implementation method of this embodiment solves the problem that server computing resources cannot be effectively utilized in the traditional server network architecture, and establishes an intelligent task scheduling and resource allocation solution for the radiotherapy planning cloud, which facilitates and quickly completes the radiotherapy plan generation task.

[0067] As an optional embodiment of the present invention, in a smart radiotherapy planning cloud implementation method of this embodiment, the deep Q network reinforcement learning model includes:

[0068] Input and output: A deep Q-network reinforcement learning model is created for each radiotherapy plan generation task. The input is the state vector, and the output is the probability of entering the next subtask of each radiotherapy plan generation task, thereby selecting the action with the highest probability for execution.

[0069] Hidden layer: The hidden layer of the Q network reinforcement learning model is composed of a combination of multiple linear layers and activation functions.

[0070] Furthermore, the training process of the deep Q network reinforcement learning model described in this embodiment includes:

[0071] Start multi-task calculation based on the number of simultaneous requirements for the radiotherapy plan generation task and the total server resources in the radiotherapy plan cloud;

[0072] Get the initial state vector s0 = [t1, t2, ... t n , c1, c2, …c n , g1, g2, … g n ] and global time T, where ta represents the progress of the radiotherapy plan generation task a, represented by the current subtask (the initial state is all in the input task), ca and ga represent the CPU and GPU occupancy rates of the radiotherapy plan generation task a respectively (the initial state ca>0%, gpu=0%, and the CPU and GPU occupancy rates of any task at any stage are mutually exclusive), and T counts the time required from the start of the first radiotherapy plan generation task to the end of the last radiotherapy plan generation task, which is used for rewards and penalties;

[0073] Randomly initialize the Q network reinforcement learning model and use the ε-greedy strategy to select actions:

[0074]

[0075] Among them, the initial ε is set to a higher probability, which becomes smaller as the task progresses:

[0076]

[0077] Among them, C and B are fixed parameters, k represents the task, and Ik represents the subtask stage that task k reaches:

[0078] Execute action a, calculate the current CPU and GPU resource occupancy rates, and obtain the next state vector s1;

[0079] Set the scoring function based on T and the number of completed tasks

[0080]

[0081] This gives the reward function

[0082] Among them, α and β represent weights, r(S n ) represents the current state score function, represents the number of remaining subtasks of the remaining radiotherapy plan generation task k, t ki Denotes the execution time required for the i-th subtask of the radiotherapy plan generation task k, and puts (s, a, r, s1) into the experience replay pool;

[0083] Sampling and updating the Q network reinforcement learning model from the experience replay pool, the objective function is

[0084]

[0085] Where γ is the discount factor, θ - represents the model parameters of the target Q network, which remain unchanged until a fixed number of iterations is reached, and then replaced by the parameters θ of the main Q network. The mean square error loss function is used to train the Q network model parameters L(θ) = MSE(yQ(s, a; θ));

[0086] Finally, the training of the Q network reinforcement learning model parameters is completed.

[0087] As an optional embodiment of the present invention, in a smart radiotherapy planning cloud implementation method of this embodiment, the radiotherapy plan generation task is decomposed to obtain a series of subtasks including:

[0088] Decompose a complete radiotherapy plan generation task into a series of subtasks based on CPU or GPU task types;

[0089] Subtasks using CPU computing resources include data input and output, step size calculation of the planning optimization algorithm, calculation of the optimization objective function value, and calculation of optimization machine parameter constraint limits;

[0090] Subtasks using GPU computing resources include optimizing the calculation of gradient derivatives, optimizing the fast dose calculation algorithm, and the Monte Carlo high-precision dose calculation algorithm;

[0091] A complete radiotherapy plan generation task includes: input, optimized fast dose calculation, gradient derivative calculation, step size calculation, optimized machine parameter constraint limit calculation, objective function value calculation, Monte Carlo high-precision dose calculation, and output. The subtasks from optimized fast dose calculation to objective function value calculation need to be repeated multiple times.

[0092] As an optional implementation of the present invention, in a smart radiotherapy planning cloud implementation method of this embodiment, receiving a radiotherapy plan generation task submitted by a user includes:

[0093] User-defined calculation parameters, including CT data, delineation data, radiotherapy prescription, accelerator machine parameters, plan type, radiation field, and organ constraints;

[0094] Computational tasks submitted by users include plan optimization tasks and dose calculation tasks.

[0095] As an optional implementation of the present invention, a smart radiotherapy planning cloud implementation method of this embodiment includes:

[0096] Each server in the radiotherapy plan cloud receives the multi-task allocation scheduling of the radiotherapy plan generation task, performs corresponding subtask calculations, and generates a radiotherapy plan;

[0097] The database server in the radiotherapy planning cloud provides calculation data and saves the calculated results.

[0098] See also Figure 2 As shown, this embodiment also provides a smart radiotherapy planning cloud system, including:

[0099] The user-side module receives the radiotherapy plan generation task submitted by the user;

[0100] The task scheduling module decomposes the radiotherapy plan generation task into a series of subtasks and collects the current status of each server in the planning cloud module;

[0101] A reinforcement learning module, including a deep Q-network reinforcement learning model, wherein the task scheduling module inputs the subtasks of the radiotherapy plan generation task decomposition and the current status of each server in the plan cloud module into the deep Q-network reinforcement learning model, and outputs the server resources allocated to each subtask;

[0102] The task scheduling module performs multi-task allocation scheduling of the radiotherapy plan generation task based on the output of the reinforcement learning module.

[0103] Furthermore, a smart radiotherapy planning cloud system of this embodiment includes a planning cloud module including:

[0104] A computing server receives the multi-task allocation scheduling of the radiotherapy plan generation task, performs corresponding subtask calculations, and generates a radiotherapy plan;

[0105] The database server provides calculation data and saves the calculation results.

[0106] Specific examples of the present invention are described below:

[0107] Example 1

[0108] A method for implementing a smart radiotherapy planning cloud, comprising:

[0109] Step 1: The user sets CT data, outline data, radiotherapy prescription, accelerator machine parameters, plan type, radiation field, organ constraints and other parameters on the user computer;

[0110] Step 2. Select plan optimization or dose calculation and submit the calculation task;

[0111] Step 3: The task scheduling module accepts the computing task, analyzes the task and collects the existing status of the server;

[0112] Step 4: Input the state of tasks and resources into the deep Q network reinforcement learning model;

[0113] Step 5: The deep Q network reinforcement learning model outputs the corresponding resources for each subtask and submits them to the planning cloud module;

[0114] Step 6: The computing server of the planning cloud module accepts the assigned computing task, performs the calculation, reads the necessary computing data from the database server, and continues the planning optimization or dose calculation;

[0115] Step 7: Complete the calculation and save the result to the database server.

[0116] Example 2

[0117] Training of the Deep Q-Network reinforcement learning model involves:

[0118] Step 1. Determine the number of tasks that are started at the same time. For example, if the number of tasks is 4, initialize the Q network, initialize the state vector s0 = [t1, t2, ... t4, c1, c2, ... c4, g1, g2, ... g4] and the global time T, which represent the state of tasks and resources. The input layer size is 12, the hidden layer is a three-layer fully connected layer, the number of neurons in the first layer is 64, the number of neurons in the second layer is 128, and the number of neurons in the third layer is 64. The output layer size is 4, and the Q value of each action a is used to represent the probability of each task entering the next subtask.

[0119] Step 2: Use the ε-greedy strategy to randomly select action a0 with ε probability, where ε is initialized to 0.9 (C is 9, B is 6, and all four tasks enter the input subtask). Use the initialized Q network to predict action a0 with a probability of 1-ε.

[0120] Step 3: Set the α and β of the score function to 1, calculate the reward function r0, complete a resource allocation and execute the task, obtain the next state s1, put the state, action, reward, and next state (s0, a0, r0, s1) into the experience replay pool, and perform an action selection or prediction every time the state changes;

[0121] Step 4: Update ε. As tasks are continuously executed, the value of ε will gradually decrease, forcing more actions to be generated by the Q network.

[0122] Step 5: Continue to use the ε greedy strategy to select action ai, get the next state si+1, and get reward ri at the same time. Store (Si, ai, ri, Si+1) into the experience replay pool. Synchronously, the Q network selects samples from the experience replay pool to train the model.

[0123] Step 6: Repeat the training until convergence and obtain the optimal solution for the task time (if convergence has not been achieved but all tasks have been completed, start a new round of task execution and continue training).

[0124] This embodiment also provides a computer-readable storage medium storing a computer-executable program. When the computer-executable program is executed, the intelligent radiotherapy planning cloud implementation method is implemented.

[0125] The computer-readable storage medium described in this embodiment may include a data signal propagated in baseband or as part of a carrier wave, which carries a readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or component. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0126] This embodiment also provides an electronic device, including a processor and a memory, wherein the memory is used to store a computer executable program. When the computer program is executed by the processor, the processor executes the intelligent radiotherapy planning cloud implementation method.

[0127] The electronic device is implemented as a general-purpose computing device. The processor may be one or multiple processors operating in concert. The present invention also does not exclude distributed processing, meaning the processors may be dispersed across different physical devices. The electronic device of the present invention is not limited to a single entity but may also be the sum of multiple physical devices.

[0128] The memory stores a computer executable program, typically a machine-readable code, which can be executed by the processor to enable the electronic device to perform the method of the present invention, or at least some of the steps in the method.

[0129] The memory includes a volatile memory, such as a random access memory unit (RAM) and / or a cache memory unit, and may also be a non-volatile memory, such as a read-only memory unit (ROM).

[0130] It should be understood that the electronic devices of the present invention may also include elements or components not shown in the above examples. For example, some electronic devices also include display units such as screens, and some electronic devices also include human-computer interaction elements such as buttons and keyboards. As long as the electronic device can execute a computer-readable program stored in its memory to implement the method of the present invention or at least some of the steps of the method, it can be considered an electronic device covered by the present invention.

[0131] Through the above description of the implementation mode, it is easy for those skilled in the art to understand that the present invention can be implemented by hardware capable of executing a specific computer program, such as the system of the present invention, and the electronic processing unit, server, client, mobile phone, control unit, processor, etc. contained in the system. The present invention can also be implemented by computer software that executes the method of the present invention, such as control software executed by a microprocessor, an electronic control unit, a client, a server, etc. However, it should be noted that the computer software that executes the method of the present invention is not limited to being executed by one or a specific hardware entity, and it can also be implemented in a distributed manner by unspecified specific hardware. For computer software, the software product can be stored in a computer-readable storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), or it can be distributed and stored on a network, as long as it enables an electronic device to execute the method according to the present invention.

[0132] The above embodiments are only used to illustrate the present invention and are not intended to limit the technical solutions described in the present invention. Although this specification has described the present invention in detail with reference to the above embodiments, the present invention is not limited to the above specific implementation methods. Therefore, any modification or equivalent replacement of the present invention; and all technical solutions and improvements thereof that do not depart from the spirit and scope of the invention are included in the scope of the claims of the present invention.

Claims

1. A method for implementing a smart radiotherapy planning cloud, characterized in that: include: Receive a radiation therapy plan generation task submitted by a user; Decompose the radiotherapy plan generation task into a series of subtasks and collect the current status of each server in the radiotherapy plan cloud; The subtasks of the radiotherapy plan generation task and the current status of each server in the radiotherapy plan cloud are input into the deep Q network reinforcement learning model, and the server resources allocated to each subtask are output to perform multi-task allocation and scheduling of the radiotherapy plan generation task.

2. The method for implementing a smart radiotherapy planning cloud according to claim 1, characterized in that: The deep Q network reinforcement learning model includes: Input and output: A deep Q network reinforcement learning model is created for each radiotherapy plan generation task. The input is the state vector, and the output is the probability of each radiotherapy plan generation task entering the next subtask, so as to select the action with the highest probability to execute. Hidden layer: The hidden layer of the Q network reinforcement learning model is composed of a combination of multiple linear layers and activation functions.

3. The method for implementing a smart radiotherapy planning cloud according to claim 1, characterized in that: The training process of the deep Q network reinforcement learning model includes: Start multi-task calculation based on the number of simultaneous requirements of the radiotherapy plan generation task and the total server resources in the radiotherapy plan cloud; Get the initial state vector s0=[t1, t2, ... t n , c1, c2, … c n， g1, g2, …g n ] and global time T, where ta represents the progress of the radiotherapy plan generation task a, represented by the current subtask, ca and ga represent the CPU and GPU occupancy rates of the radiotherapy plan generation task a, respectively, and T counts the time required from the start of the first radiotherapy plan generation task to the end of the last radiotherapy plan generation task, which is used for rewards and penalties; Randomly initialize the Q network reinforcement learning model and use the ε-greedy strategy to select actions: Among them, the initial ε is set to a higher probability, which becomes smaller as the task progresses: Among them, C and B are fixed parameters, k represents the task, and Ik represents the subtask stage of task k: Execute action a, calculate the current CPU and GPU resource occupancy rate, and obtain the next state vector s1; Set the scoring function based on T and the number of completed tasks This gives the reward function Among them, α and β represent weights, r(S n ) represents the current state score function, represents the number of remaining subtasks of the remaining radiation therapy plan generation task k, t ki represents the execution time required for the ith subtask of the radiotherapy plan generation task k, and puts (s, a, r, s1) into the experience replay pool; Sample and update the Q network reinforcement learning model from the experience replay pool. The objective function is Among them, γ is the discount factor, θ - represents the model parameters of the target Q network, which remain unchanged before reaching a fixed number of iterations, and then are replaced by the parameters θ of the main Q network. The mean square error loss function is used to train the Q network model parameters L(θ) = MSE(yQ(s, a; θ)); Finally, the training of the Q network reinforcement learning model parameters is completed.

4. The method for implementing a smart radiotherapy planning cloud according to claim 1, characterized in that: The radiotherapy plan generation task is decomposed to obtain a series of subtasks including: A complete radiotherapy plan generation task is decomposed into a series of subtasks according to CPU or GPU task types; Subtasks using CPU computing resources include data input and output, step length calculation of the planning optimization algorithm, optimization objective function value calculation, and optimization machine parameter constraint limit calculation; Subtasks using GPU computing resources include optimizing the calculation of gradient derivatives, optimizing the fast dose calculation algorithm, and the Monte Carlo high-precision dose calculation algorithm; A complete radiotherapy plan generation task includes: input, optimized fast dose calculation, gradient derivative calculation, step size calculation, optimized machine parameter constraint limit calculation, objective function value calculation, Monte Carlo high-precision dose calculation, and output. The subtasks from optimized fast dose calculation to objective function value calculation need to be repeated multiple times.

5. The method for implementing a smart radiotherapy planning cloud according to claim 1, characterized in that: The task of receiving a radiation therapy plan submitted by a user and generating the plan comprises: User-defined calculation parameters, including CT data, delineation data, radiotherapy prescription, accelerator machine parameters, plan type, radiation field, and organ constraints; Computational tasks submitted by users include planning optimization tasks and dose calculation tasks.

6. The method for implementing a smart radiotherapy planning cloud according to claim 1, characterized in that: include: Each server in the radiotherapy plan cloud receives the multi-task allocation scheduling of the radiotherapy plan generation task, performs corresponding subtask calculations, and generates a radiotherapy plan; The database server in the radiotherapy planning cloud provides calculation data and saves the calculated results.

7. A smart radiotherapy planning cloud system, characterized in that: include: The user-side module receives the radiotherapy plan generation task submitted by the user; The task scheduling module decomposes the radiotherapy plan generation task into a series of subtasks and collects the current status of each server in the plan cloud module; A reinforcement learning module, including a deep Q network reinforcement learning model, wherein the task scheduling module inputs the subtasks decomposed from the radiotherapy plan generation task and the current status of each server in the plan cloud module into the deep Q network reinforcement learning model, and outputs the server resources allocated to each subtask; The task scheduling module performs multi-task allocation scheduling of radiotherapy plan generation tasks based on the output of the reinforcement learning module.

8. The intelligent radiotherapy planning cloud system according to claim 7, characterized in that: Included Plan Cloud modules include: A computing server receives the multi-task allocation scheduling of the radiotherapy plan generation task, performs corresponding subtask calculations, and generates a radiotherapy plan; The database server provides calculation data and saves the calculation results.

9. A computer-readable medium storing a computer-executable program, characterized in that: When the computer executable program is executed, a smart radiotherapy planning cloud implementation method as described in any one of claims 1-6 is implemented.

10. An electronic device, comprising a processor and a memory, wherein the memory is used to store a computer executable program, wherein: When the computer program is executed by the processor, the processor executes a smart radiotherapy planning cloud implementation method as described in any one of claims 1-6.

Images